Commit Graph

1092 Commits

Author SHA1 Message Date
Damien George f2ad152e7e extmod/modbluetooth: Run BLE IRQ callback in protected NLR context.
The call to invoke_irq_handler_run() always needs to run in a protected NLR
context, to catch exceptions from the Python handler, and the m_new's (and
also mp_local_alloc when PYSTACK is enabled).  With
MICROPY_PY_BLUETOOTH_USE_SYNC_EVENTS_WITH_INTERLOCK enabled there was
already an explicit nlr_push, and that is now used in all cases.

Without this change, on stm32 (for example), the callbacks from the BLE
stack to invoke_irq_handler() were made via static scheduled nodes which do
not have any NLR protection, and hence would lead to a hard fault (uncaught
NLR) if an exception was raised in the Python BLE IRQ handler.  This was a
regression introduced by 8045ac07f5, which is
fixed by this commit.

Signed-off-by: Damien George <damien@micropython.org>
2022-09-23 15:21:54 +10:00
Damien George db668742a5 extmod/modbluetooth: Do GATTC reassembly in protected uPy context.
The calls to m_new and m_del require an exclusive uPy (really a GC)
context.  In particular these functions cannot be called directly from a
FreeRTOS task on esp32.

Fixes issue #9369.

Signed-off-by: Damien George <damien@micropython.org>
2022-09-22 11:49:58 +10:00
Damien George ed41d51746 extmod/modbluetooth: Change data_len type from size_t to uint16_t.
For consistency, and to remove the need for additional conversion of types.

Signed-off-by: Damien George <damien@micropython.org>
2022-09-22 11:47:03 +10:00
stijn 9ae8d38204 extmod/vfs_posix_file: Implement finaliser for files.
Prevent handle leaks when file objects aren't closed explicitly and
fix some MICROPY_CPYTHON_COMPAT issues: this wasn't properly adhered
to because #ifdef was used so it was always on, and closing files
multiple times should be avoided unconditionally.
2022-09-19 23:44:50 +10:00
Jim Mussared 94beeabd2e py/obj: Convert make_new into a mp_obj_type_t slot.
Instead of being an explicit field, it's now a slot like all the other
methods.

This is a marginal code size improvement because most types have a make_new
(100/138 on PYBV11), however it improves consistency in how types are
declared, removing the special case for make_new.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:15 +10:00
Jim Mussared 6da41b5900 py/obj: Merge getiter and iternext mp_obj_type_t slots.
The goal here is to remove a slot (making way to turn make_new into a slot)
as well as reduce code size by the ~40 references to mp_identity_getiter
and mp_stream_unbuffered_iter.

This introduces two new type flags:
- MP_TYPE_FLAG_ITER_IS_ITERNEXT: This means that the "iter" slot in the
  type is "iternext", and should use the identity getiter.
- MP_TYPE_FLAG_ITER_IS_CUSTOM: This means that the "iter" slot is a pointer
  to a mp_getiter_iternext_custom_t instance, which then defines both
  getiter and iternext.

And a third flag that is the OR of both, MP_TYPE_FLAG_ITER_IS_STREAM: This
means that the type should use the identity getiter, and
mp_stream_unbuffered_iter as iternext.

Finally, MP_TYPE_FLAG_ITER_IS_GETITER is defined as a no-op flag to give
the default case where "iter" is "getiter".

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:13 +10:00
Jim Mussared a52cd5b07d py/obj: Add accessors for type slots and use everywhere.
This is a no-op, but sets the stage for changing the mp_obj_type_t
representation.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:07 +10:00
Jim Mussared e8355eb163 py/obj: Add "full" and "empty" non-variable-length mp_obj_type_t.
This will always have the maximum/minimum size of a mp_obj_type_t
representation and can be used as a member in other structs.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:04 +10:00
Jim Mussared 9dce82776d all: Remove unnecessary locals_dict cast.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:01 +10:00
Jim Mussared b7d6ee9b75 all: Fix #if inside MP_DEFINE_CONST_OBJ_TYPE for msvc.
Changes:

    MP_DEFINE_CONST_OBJ_TYPE(
       ...
       #if FOO
       ...
       #endif
       ...
    );

to:

    MP_DEFINE_CONST_OBJ_TYPE(
       ...
       FOO_TYPE_ATTR
       ...
    );

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:01 +10:00
Jim Mussared 662b9761b3 all: Make all mp_obj_type_t defs use MP_DEFINE_CONST_OBJ_TYPE.
In preparation for upcoming rework of mp_obj_type_t layout.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:01 +10:00
Jim Mussared 42587c7870 all: Standardise mp_obj_type_t initialisation.
Remove setting unused slots.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 18:41:29 +10:00
Jim Mussared fb2a57800a all: Simplify buffer protocol to just a "get buffer" callback.
The buffer protocol type only has a single member, and this existing layout
creates problems for the upcoming split/slot-index mp_obj_type_t layout
optimisations.

If we need to make the buffer protocol more sophisticated in the future
either we can rely on the mp_obj_type_t optimisations to just add
additional slots to mp_obj_type_t or re-visit the buffer protocol then.

This change is a no-op in terms of generated code.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 18:40:39 +10:00
Andrew Leech 4e0964b59f extmod/vfs: Add finaliser to ilistdir to close directory handle.
When iterating over filesystem/folders with os.iterdir(), an open file
(directory) handle is used internally.  Currently this file handle is only
closed once the iterator is completely drained, eg. once all entries have
been looped over / converted into list etc.

If a program opens an iterdir but does not loop over it, or starts to loop
over the iterator but breaks out of the loop, then the handle never gets
closed.  In this state, when the iter object is cleaned up by the garbage
collector this open handle can cause corruption of the filesystem.

Fixes issues #6568 and #8506.
2022-09-13 13:00:42 +10:00
Jim Mussared cacc96d98c extmod/modbluetooth: Replace def_handle with end_handle in char IRQ.
This is technically a breaking change, but:
a) We need the end handle to do descriptor discovery properly.
b) We have no possible use for the existing definition handle in the
characteristic result IRQ. None of the methods can use it, and therefore
no existing code should be using it in a way that changing it to a
different integer value should break.

Unfortunately NimBLE doesn't make it easy to get the end handle, so also
implement a mechanism to use the following characteristic to calculate
the previous characteristic's end handle.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-09 11:24:24 +10:00
Jim Mussared 82fc16f298 extmod/modbluetooth: Fix descriptor registration with empty tuple.
Incorrect use of "continue" when the tuple was length zero meant it
broke the rest of the argument handling.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-09 11:24:21 +10:00
Jim Mussared 24678fe452 drivers: Remove drivers that are now in micropython-lib.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-08 11:27:05 +10:00
Jeff Epler e90b85cc98 extmod/modure: Convert byte offsets to unicode indices when necessary.
And add a test.

Fixes issue #9202.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2022-09-06 17:08:18 +10:00
Jim Mussared 203dae41fb all: Update all manifest.py files to use new features.
Changes in this commit:
- Manifest include's now use the directory path where possible (no longer
  necessary to include the manifest.py file explicitly).
- Add manifest.py for all drivers and components that are referenced by
  port/board manifests.
- Replace all uses of freeze() with package()/module(), except for port and
  board modules.
- Use opt=3 everywhere, for consistency and to reduce code size.
- Use require() instead of include() for all micropython-lib references.
- Remove support for optional board-level manifest.py in mimxrt port, to
  make it behave the same as other ports (the board must set
  FROZEN_MANIFEST to a custom manifest.py, which can optionally include the
  default, port-level manifest).
- Also reinstates modules that were accidentally removed from the esp8266
  512k build in fbe9417b90.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
2022-09-05 18:43:18 +10:00
robert-hh 0b26efe73d extmod/machine_i2c: Call MICROPY_PY_EVENT_HOOK during i2c.scan().
Avoiding a watchdog reset during i2c.scan() if the hardware is not properly
set up (eg on esp8266), and also allowing to stop the scan with a
KeyboardInterrupt.

Fixes issue #8876.
2022-08-31 12:06:11 +10:00
Jim Mussared 3a910b1565 py/objstr: Optimise mp_obj_new_str_from_vstr for known-safe strings.
The new `mp_obj_new_str_from_utf8_vstr` can be used when you know you
already have a unicode-safe string.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-26 16:44:35 +10:00
Jim Mussared 8a0ee5a5c0 py/objstr: Split mp_obj_str_from_vstr into bytes/str versions.
Previously the desired output type was specified.  Now make the type part
of the function name.  Because this function is used in a few places this
saves code size due to smaller call-site.

This makes `mp_obj_new_str_type_from_vstr` a private function of objstr.c
(which is almost the only place where the output type isn't a compile-time
constant).

This saves ~140 bytes on PYBV11.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-26 16:43:55 +10:00
robert-hh 8308f9c977 extmod/network_wiznet5k: Use the configured DNS address if available.
Instead of the default 8.8.8.8.  The change was suggested by @omogenot.
2022-08-23 15:00:00 +10:00
Jim Mussared c616721b1a extmod/modframebuf: Improve poly-fill boundary pixels.
Rather than drawing the entire boundary to catch missing pixels, just
detect the cases where boundary pixels are skipped during node calculation
and pre-emptively draw them then.

This adds 72 bytes on PYBV11, but makes filled poly() 20% faster.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-19 23:31:28 +10:00
Mat Booth 04a655c744 extmod/modframebuf: Add polygon drawing methods.
Add method for drawing polygons.

For non-filled polygons, uses the existing line-drawing code to render
arbitrary polygons using the given coords list, at the given x,y position,
in the given colour.

For filled polygons, arbitrary closed polygons are rendered using a fast
point-in-polygon algorithm to determine where the edges of the polygon lie
on each pixel row.

Tests and documentation updates are also included.

Signed-off-by: Mat Booth <mat.booth@gmail.com>
2022-08-19 23:31:28 +10:00
Peter Hinch 42ec9703a0 extmod/modframebuf: Add ellipse drawing method. 2022-08-19 23:31:28 +10:00
Jim Mussared 127b340438 extmod/modframebuf: Add fill argument to rect().
We plan to add `ellipse` and `poly` methods, but rather than having to
implement a `fill_xyz` version of each, we can make them take an optional
fill argument. This commit add this to `rect` as a starting point.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-19 23:31:28 +10:00
Jim Mussared 470a44bd3a extmod/modframebuf: Optimise argument handling.
Several methods extract mp_int_t from adjacent arguments. This reduces
code size for the repeated calls to mp_obj_get_int.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-19 15:21:31 +10:00
Damien George 8f4c108025 all: Remove MICROPY_PY_IO_FILEIO config option.
Since commit e65d1e69e8 there is no longer an
io.FileIO class, so this option is no longer needed.

This option also controlled whether or not files supported being opened in
binary mode (eg 'rb'), and could, if disabled, lead to confusion as to why
opening a file in binary mode silently did the wrong thing (it would just
open in text mode if MICROPY_PY_IO_FILEIO was disabled).

The various VFS implementations (POSIX, FAT, LFS) were the only places
where enabling this option made a difference, and in almost all cases where
one of these filesystems were enabled, MICROPY_PY_IO_FILEIO was also
enabled.  So it makes sense to just unconditionally enable this feature
(ability to open a file in binary mode) in all cases, and so just remove
this config option altogether.  That makes configuration simpler and means
binary file support always exists (and opening a file in binary mode is
arguably more fundamental than opening in text mode, so if anything should
be configurable then it should be the ability to open in text mode).

Signed-off-by: Damien George <damien@micropython.org>
2022-08-18 11:54:17 +10:00
Damien George 237a393bec extmod/vfs_posix_file: Remove unused MICROPY_VFS_POSIX_FILE.
This was made obsolete by 2b409ef8a4

Signed-off-by: Damien George <damien@micropython.org>
2022-08-18 11:48:45 +10:00
MrJake222 69719927f1 extmod/modlwip: Add support for leaving multicast groups. 2022-08-12 23:09:13 +10:00
Damien George 01514e80c9 extmod/uasyncio: Rename internal _flag to state, to save a qstr.
Saves about 16 bytes of flash when uasyncio is frozen in.

Signed-off-by: Damien George <damien@micropython.org>
2022-08-12 22:33:55 +10:00
Ned Konz 5543b2a9cc extmod/uasyncio: Add clear method to ThreadSafeFlag.
This is useful in situations where the ThreadSafeFlag is reused and needs
to be cleared of any previous, unwanted event.

For example, clear the flag at the start of an operation, trigger the
operation (eg an I2C write), then (a)wait for an external event to set the
flag (eg a pin IRQ).  Further events may trigger the flag again but these
are unwanted and should be cleared before the next cycle starts.
2022-08-12 17:06:28 +10:00
Jim Mussared 28aaab9590 py/objstr: Add hex/fromhex to bytes/memoryview/bytearray.
These were added in Python 3.5.

Enabled via MICROPY_PY_BUILTINS_BYTES_HEX, and enabled by default for all
ports that currently have ubinascii.

Rework ubinascii to use the implementation of these methods.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-12 12:44:30 +10:00
Damien George af6d2845fa extmod/network_wiznet5k: Extract SPI transfer function dynamically.
Instead of using the fixed machine_spi_type entity to get the SPI transfer
function, this transfer function is now extracted dynamically from the type
of the SPI object.

This allows the SPI object used to communicate with the WIZNET5K hardware
to be SoftSPI or hardware SPI, or anything that has the SPI protocol (at
the C level).

Signed-off-by: Damien George <damien@micropython.org>
2022-08-10 14:01:58 +10:00
robert-hh f000ac9e82 extmod/network_wiznet5k: Rearrange the function wiznet5k_poll().
To have just one exit and a more compact flag test.  This is just a style
change without impact to the functionality.
2022-08-09 16:42:03 +10:00
robert-hh 999b66d531 extmod/network_wiznet5k: Schedule clearing of interrupt flags.
Avoiding conflicts between the IRQ and an active transfers.  Before this
change the device could lock up in heavy traffic situations.

Fix found and code supplied by @omogenot.
2022-08-09 16:42:03 +10:00
robert-hh 73699a846c extmod/network_wiznet5k: Deinit the NIC before (re-)initialisation.
If nic.active(True) is called several times in a row, the device may lock
up.  Even if that is bad coding practice, calling wiznet5k_deinit() in
wiznet5k_init() prevents the lock.
2022-08-09 16:35:57 +10:00
robert-hh be2beab71a extmod/network_wiznet5k: Drop obsolete argument count check.
Drop an obsolete and wrong argument check, which prevented specifying a pin
for the interrupt signal.  The proper checks are now done further down in
the code.
2022-08-09 16:34:26 +10:00
robert-hh 736b427220 extmod/network_wiznet5k: Register NIC when the lwIP stack is used.
That was missing, and network.route() returned an empty list.
2022-08-09 16:33:55 +10:00
iabdalkader 9dfabcd6d3 extmod/network_cyw43: Add hostname config option. 2022-08-06 00:30:57 +10:00
Ian Davies 1bf2fd0592 extmod/modussl_mbedtls: Set a more sensible default debug log level. 2022-08-06 00:11:46 +10:00
Ian Davies fbe9417b90 extmod/ntptime: Factor out ntptime module from esp8266 port.
The ntptime module was previously only included in the ESP8266 port.  This
commit factors that module out into the extmod directory, makes it support
different epochs, and includes it in the rp2 port.
2022-08-06 00:08:32 +10:00
Jim Mussared b22abcdbbe extmod/uasyncio: Handle gather with no awaitables.
This previously resulted in gather() yielding but with no way to be
resumed.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-07-26 18:16:19 +10:00
Jim Mussared e65d1e69e8 py/modio: Remove FileIO and TextIOWrapper from io module.
On ports with more than one filesystem, the type will be wrong, for example
if using LFS but FAT enabled, then the type will be FAT.  So it's not
possible to use these classes to identify a file object type.

Furthermore, constructing an io.FileIO currently crashes on FAT, and
make_new isn't supported on LFS.

And the io.TextIOWrapper class does not match CPython at all.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-07-26 17:58:01 +10:00
Jim Mussared 924e55aca1 extmod/webrepl: Allow the page to run from the device (over HTTP).
The device will respond to a non-WS request with a simple page that loads
websocket_content.js from a static host (http or https). However, even
if the resources are https, the page is still http and therefore allows
requesting to a WS (not WSS) websocket on the device.

Removed unused client_handshake from websocket_helper, and then merges the
remainder of this file (server_handshake) into webrepl.py (to reduce
firmware size). Also added the respond-as-HTTP handling to
server_handshake.

The default HTTP response is a simple page that sets the base URL and then
loads webrepl_content.js which document.write's the actual HTML.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-07-23 12:43:08 +10:00
Damien George e05d0a6335 extmod/modbluetooth: Add support for running sync irq on system thread.
If the Bluetooth stack runs on another OS thread then synchronous BLE irq
callbacks, which block the Bluetooth stack until the callback to Python is
complete, must coordinate with the main thread and configure the
MicroPython thread-local-state.

This commit adds MICROPY_PY_BLUETOOTH_USE_SYNC_EVENTS_WITH_INTERLOCK which
can be enabled if the system has these requirements.

Signed-off-by: Damien George <damien@micropython.org>
2022-07-22 17:38:15 +10:00
Jim Mussared a053827084 extmod/network_ninaw10: Move ninaw10 root pointer registrations here.
Originally in drivers/ninaw10/nina_wifi_bsp.c but that isn't a QSTR source.

Also remove outdated commment about root pointers in mpconfigport.h.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-07-21 16:21:50 +10:00
Damien George bdad63eda2 extmod/btstack: Fix descriptor discovery handle range and events.
This fixes two problems with the BTstack implementation of descriptor
discovery:

- The call to gatt_client_discover_characteristic_descriptors needs to have
  value_handle set to the starting handle (actually characteristic handle)
  to start the search from.

- The BTstack event for a descriptor query result is
  GATT_EVENT_ALL_CHARACTERISTIC_DESCRIPTORS_QUERY_RESULT.

With this change the test tests/multi_bluetooth/ble_subscribe.py now passes
when BTstack is instance1 (for BTstack to pass as instance0 requires
gatts_write to support sending an update on BTstack).

Signed-off-by: Damien George <damien@micropython.org>
2022-07-20 17:01:37 +10:00
Carlosgg b41cfea02a extmod/modussl_mbedtls: Implement cert_reqs and cadata arguments.
Add cert_reqs and cadata keyword-args to ssl.wrap_socket() and
ssl.CERT_NONE, ssl.CERT_OPTIONAL, ssl.CERT_REQUIRED constants to allow
certificate validation.

CPython doesn't accept cadata in ssl.wrap_socket(), but it does in
SSLContext.load_verify_locations(), so we use this name to at least match
the same name in load_verify_locations().

Add docs for these new arguments, as well as docs for the existing
server_hostname argument which is important for certificate validation.

Tests are added as well.

Signed-off-by: Carlos Gil <carlosgilglez@gmail.com>
2022-07-20 16:46:04 +10:00