Commit Graph

1182 Commits

Author SHA1 Message Date
Jim Mussared 9573d31071 all: Remove query-variants make target.
This is difficult to implement on cmake-based ports, and having the list
of variants in mpconfigboard.{cmake,mk} duplicates information that's
already in board.json.

This removes the existing query-variants make target from stm32 & rp2
and the definition of BOARD_VARIANTS from the various board files.

Also renames the cmake variable to MICROPY_BOARD_VARIANT to match other
variables such as MICROPY_BOARD. The make variable stays as
BOARD_VARIANT.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-15 17:37:44 +10:00
Jim Mussared ad123ed013 esp32/Makefile: Implement `make submodules` to match other ports.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-15 17:37:44 +10:00
Damien George f6af48416d extmod/modselect: Properly track number of poll objects that are fd's.
Signed-off-by: Damien George <damien@micropython.org>
2023-08-14 18:48:47 +10:00
Damien George d529c20674 extmod/modssl_mbedtls: Fix ioctl of a socket in closed/error state.
Signed-off-by: Damien George <damien@micropython.org>
2023-08-09 13:22:57 +10:00
Damien George 20d3a6b196 extmod/modssl_mbedtls: Reject ioctls that are not supported.
An SSL stream can only handle CLOSE and POLL ioctls.  Other ones do not
make sense, or at least it doesn't make sense to pass the ioctl request
directly down to the underlying stream.

In particular MP_STREAM_GET_FILENO should not be passed to the underlying
stream because the SSL stream is not directly related to a file descriptor,
and the SSL stream must handle the polling itself.

Signed-off-by: Damien George <damien@micropython.org>
2023-08-09 13:22:32 +10:00
Damien George 3f417e8943 extmod/modselect: Remove undocumented support for flags arg to poll.
The signature of this method was poller.poll(timeout=-1, flags=0, /) but
the flags argument was not documented and is not CPython compatible.  So
it's removed in this commit.

(The optional flags remains for the ipoll() method, which is documented.)

Signed-off-by: Damien George <damien@micropython.org>
2023-08-07 12:37:47 +10:00
Damien George ef71028f77 extmod/modselect: Add optimisation to use system poll when possible.
A previous commit removed the unix-specific select module implementation
and made unix use the common one.

This commit adds an optimisation so that the system poll function is used
when polling objects that have a file descriptor.  With this optimisation
enabled, if code registers both file-descriptor-based objects, and non-
file-descriptor-based objects with select.poll() then the following occurs:

- the system poll is called for all file-descriptor-based objects with a
  timeout of 1ms

- then the bare-metal polling implementation is used for remaining objects,
  which calls into their ioctl method (which can be in C or Python)

In the case where all objects have file descriptors, the system poll is
called with the full timeout requested by the caller.  That makes it as
efficient as possible in the case everything has a file descriptor.

Benefits of this approach:

- all ports use the same select module implementation

- the unix port now supports polling of all objects and matches bare metal
  implementations

- it's still efficient for existing cases where only files and sockets are
  polled (on unix)

- the bare metal implementation does not change

- polling of SSL objects will now work on unix by calling in to the ioctl
  method on SSL objects (this is required for asyncio ssl support)

Note that extmod/vfs_posix_file.c has poll disable when the optimisation is
enabled, because the code is not reachable when the optimisation is used.

Signed-off-by: Damien George <damien@micropython.org>
2023-08-07 12:11:40 +10:00
Damien George ebc6556346 extmod/modselect: Factor low-level polling code into common function.
Signed-off-by: Damien George <damien@micropython.org>
2023-08-07 12:11:40 +10:00
Damien George 7f2efb4144 extmod/modselect: Abstract out a poll_set_t struct and functions.
To make it easier to extend and modify this polling implementation.

Signed-off-by: Damien George <damien@micropython.org>
2023-08-07 12:11:40 +10:00
Damien George 22106bf2fa extmod/vfs_posix_file: Add poll support for missing ERR,HUP,NVAL values.
Signed-off-by: Damien George <damien@micropython.org>
2023-08-06 11:54:06 +10:00
Jim Mussared 70c564324c extmod/modssl_mbedtls: Reference SSLContext from SSLSocket.
Prevent the GC cleaning up (and finalising) the SSLContext while the
socket is still live.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-01 15:12:33 +10:00
Jim Mussared 3533924c36 extmod/moddeflate: Add deflate module providing the DeflateIO class.
This provides similar functionality to the former zlib.DecompIO and
especially CPython's gzip.GzipFile for both compression and decompression.

This class can be used directly, and also can be used from Python to
implement (via io.BytesIO) zlib.decompress and zlib.compress, as well as
gzip.GzipFile.

Enable/disable this on all ports/boards that zlib was previously configured
for.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 19:32:40 +10:00
Jim Mussared 198311c780 py/stream: Add mp_stream___exit___obj that calls mp_stream_close.
There are enough places that implement __exit__ by forwarding directly to
mp_stream_close that this saves code size.

For the cases where __exit__ is a no-op, additionally make their
MP_STREAM_CLOSE ioctl handled as a no-op.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 18:49:03 +10:00
Jim Mussared add1200343 all: Remove the zlib module.
This will be replaced with a new deflate module providing the same
functionality, with an optional frozen Python wrapper providing a
replacement zlib module.

binascii.crc32 is temporarily disabled.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 18:48:29 +10:00
Damien George e8a4c1dd53 extmod/modssl: Add SSLContext class.
This commit adds the SSLContext class to the ssl module, and retains the
existing ssl.wrap_socket() function to maintain backwards compatibility.

CPython deprecated the ssl.wrap_socket() function since CPython 3.7 and
instead one should use ssl.SSLContext().wrap_socket().  This commit makes
that possible.

For the axtls implementation:
- ssl.SSLContext is added, although it doesn't hold much state because
  axtls requires calling ssl_ctx_new() for each new socket
- ssl.SSLContext.wrap_socket() is added
- ssl.PROTOCOL_TLS_CLIENT and ssl.PROTOCOL_TLS_SERVER are added

For the mbedtls implementation:
- ssl.SSLContext is added, and holds most of the mbedtls state
- ssl.verify_mode is added (getter and setter)
- ssl.SSLContext.wrap_socket() is added
- ssl.PROTOCOL_TLS_CLIENT and ssl.PROTOCOL_TLS_SERVER are added

The signatures match CPython:
- SSLContext(protocol)
- SSLContext.wrap_socket(sock, *, server_side=False,
    do_handshake_on_connect=True, server_hostname=None)

The existing ssl.wrap_socket() functions retain their existing signature.

Signed-off-by: Damien George <damien@micropython.org>
2023-06-26 16:34:41 +10:00
Damien George 18caf49a7f extmod/modbtree: Undefine queue macros before including berkeley-db.
To prevent warnings when building with ESP IDF v5.

Signed-off-by: Damien George <damien@micropython.org>
2023-06-23 15:34:22 +10:00
Damien George 402fdc40fb extmod/modplatform: Set MICROPY_PLATFORM_ARCH on riscv platforms.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-23 15:34:22 +10:00
Damien George e7ae3ad92d extmod: Update to support mbedtls 3.x.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-23 15:22:27 +10:00
Duncan Lowther ae77836370
extmod/modbinascii: Fix buffer length error.
The mod_binascii_a2b_base64() function allocates a buffer which may be
too small. It needs to be no less than three-quarters of the input
length, but is calculated as (<length> / 4) * 3 + 1, which may be less
due to integer division. Changed to (<length> * 3) / 4 + 1.

Signed-off-by: Duncan Lowther <Duncan.Lowther@glasgow.ac.uk>
2023-06-21 09:48:11 +01:00
Jim Mussared ca79b49619 extmod/asyncio/uasyncio.py: Add backwards-compatible uasyncio alias.
This allows existing code that does `import uasyncio` or
`import uasyncio as asyncio` to continue working.

It uses the same lazy-loading as asyncio to prevent loading of unused
features.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-19 18:37:34 +10:00
Jim Mussared 2fbc08c462 extmod/asyncio: Rename uasyncio to asyncio.
The asyncio module now has much better CPython compatibility and
deserves to be just called "asyncio".

This will avoid people having to write `from uasyncio import asyncio`.

Renames all files, and updates port manifests to use the new path. Also
renames the built-in _uasyncio to _asyncio.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-19 17:33:03 +10:00
Peter Harper ba83f6d50c extmod/btstack: Fix marking of static addresses in set_random_address.
Marking address as static was not applied to all code paths.

Signed-off-by: Damien George <damien@micropython.org>
2023-06-14 22:20:20 +10:00
Peter Harper 35182a8688 extmod/btstack: Add cmake support for BTstack.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-14 22:20:20 +10:00
Jim Mussared a1fbb1980c extmod/modtimeq: Remove timeq module.
This is a MicroPython-specific module that existed to support the old
version of uasyncio.  It's undocumented and not enabled on all ports and
takes up code size unnecessarily.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:28 +10:00
Jim Mussared 5fd042e7d1 all: Replace all uses of umodule in Python code.
Applies to drivers/examples/extmod/port-modules/tools.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared 2eba98f1e0 all: Use MP_REGISTER_EXTENSIBLE_MODULE for overrideable built-ins.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:21 +10:00
Jim Mussared 45ac651d1a all: Rename *umodule*.c to remove the "u" prefix.
Updates any includes, and references from Makefiles/CMake.

This essentially reverts what was done long ago in commit
136b5cbd76

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:17 +10:00
Jim Mussared 0ceccd4cf8 all: Rename *umodule*.h to remove the "u" prefix.
This work was funded through GitHub Sponsors.

Also updates #includes.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:14 +10:00
Jim Mussared f5f9edf645 all: Rename UMODULE to MODULE in preprocessor/Makefile vars.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:11 +10:00
Jim Mussared 7f5d5c7271 all: Rename mod_umodule*, ^umodule* to remove the "u" prefix.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:07 +10:00
Jim Mussared 1bf2dcb15e all: Rename mp_umodule*, mp_module_umodule* to remove the "u" prefix.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:04 +10:00
Jim Mussared 30628d1bb7 all: Rename MP_QSTR_umodule to MP_QSTR_module everywhere.
This renames the builtin-modules, such that help('modules') and printing
the module object will show "module" rather than "umodule".

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:53:57 +10:00
Damien George 48ffd6596e py: Change MP_UNARY_OP_INT to MP_UNARY_OP_INT_MAYBE.
To be consistent with MP_UNARY_OP_INT_FLOAT and MP_UNARY_OP_INT_COMPLEX,
and allow int() to first check if a type supports __int__ before trying
other things (as per CPython).

Signed-off-by: Damien George <damien@micropython.org>
2023-06-01 13:01:07 +10:00
robert-hh 5a991b4290 extmod/moduos: Move os.sync() into extmod/moduos.c.
Avoiding code duplication.  To enable it, set MICROPY_PY_UOS_SYNC in the
port's mpconfigport.h.  It is operational only for FAT file system.  For
other filesystems it's a no-op.

Signed-off-by: robert-hh <robert@hammelrath.com>
2023-06-01 11:38:58 +10:00
Chris Wilson 1ecc54888f extmod/extmod.mk: Suppress deprecated-non-prototype warning.
Signed-off-by: Chris Wilson <chris@cgnd.dev>
2023-05-19 22:03:17 +10:00
Mingjie Shen a9fc0343f0 extmod/vfs_lfsx: Fix offset used before range check.
Use of offset 'from' should follow the range check.

Signed-off-by: Mingjie Shen <shen497@purdue.edu>
2023-05-19 22:01:10 +10:00
Damien George a31e3de400 extmod/network_cyw43: Add power management constants.
And allow querying the current power management mode.

Signed-off-by: Damien George <damien@micropython.org>
2023-05-08 11:56:31 +10:00
Damien George 38243cd8e0 extmod/machine_pwm: Remove PWM_INIT and PWM_DUTY_U16_NS config options.
All ports that enable MICROPY_PY_MACHINE_PWM now enable these two
sub-options, so remove these sub-options altogether to force consistency in
new ports that implement machine.PWM.

Signed-off-by: Damien George <damien@micropython.org>
2023-05-04 13:19:19 +10:00
Christian Clauss 79e57473b2 all: Fix various Python coding inconsistencies found by ruff.
This fixes:
- type-comparison (E721): do not compare types, use isinstance().
- string-dot-format-missing-arguments (F524): .format call is missing
  argument(s) for placeholder(s): {message}.
- f-string-missing-placeholders (F541).
- is-literal (F632): Use != to compare constant literals.

The last one is fixed by just comparing for truthfulness of `state`.
2023-05-02 16:14:45 +10:00
Carlosgg 7e0a38f7f7 lib/mbedtls: Update to mbedtls v2.28.1.
Signed-off-by: Carlos Gil <carlosgilglez@gmail.com>
2023-04-30 18:07:57 +10:00
Damien George b1229efbd1 all: Fix spelling mistakes based on codespell check.
Signed-off-by: Damien George <damien@micropython.org>
2023-04-27 18:03:06 +10:00
Damien George 9955553001 extmod/modutime: Provide a generic time module.
Based on extmod/utime_mphal.c, with:
- a globals dict added
- time.localtime wrapper added
- time.time wrapper added
- time.time_ns function added

New configuration options are added for this module:
- MICROPY_PY_UTIME (enabled at basic features level)
- MICROPY_PY_UTIME_GMTIME_LOCALTIME_MKTIME
- MICROPY_PY_UTIME_TIME_TIME_NS

Signed-off-by: Damien George <damien@micropython.org>
2023-04-27 15:09:56 +10:00
Damien George 26cc647fce extmod/utime_mphal: Provide a general mktime function.
Signed-off-by: Damien George <damien@micropython.org>
2023-04-27 14:55:03 +10:00
Jim Mussared 67fb0beddb extmod/btstack/modbluetooth_btstack: Add default services.
Even if the user doesn't call ble.gatts_register_services, always
provide the default services.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-04-26 11:37:20 +10:00
Jim Mussared a42a8022d8 extmod/modbluetooth: Make all HCI transports trace in the same format.
- Use HCI_TRACE macro consistently.
 - Use the same colour formatting.
 - Add a tool to convert to .pcap for Wireshark.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-04-26 11:37:20 +10:00
Jim Mussared e6de6656f4 extmod/nimble: Fix flags for descriptor registration.
The att_flags for descriptors does not use the same bitfield as for
characteristics. This was leading to NimBLE descriptors getting the wrong
flags set.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-04-26 11:37:20 +10:00
Jim Mussared bb4ee06d98 extmod/btstack: Add support for CCCD to allow client subscriptions.
This allows gatts_write(..., send_update=True) to work, which will send
notifications/indications to subscribed clients.

btstack already created the CCCD but writes to it were ignored.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-04-26 11:37:20 +10:00
Jim Mussared a2df439dd3 extmod/btstack: Fix MTU handling.
There was no event handler for central-initiated MTU exchange.

Fix truncation of notify/indicate to match NimBLE.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-04-26 11:37:20 +10:00
Jim Mussared a6aa7397d8 extmod/btstack: Include value handle in client read/write events.
This replaces the previous pending operation queue (that used to also be
shared with pending server notify/indicate ops) with a single pending
operation per connection. This allows the value handle to be correctly
passed to the Python-level events.

Also re-structure GATT client event handling to simplify the packet handler
functions.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-04-26 11:37:20 +10:00
Jim Mussared 256f47e2f8 extmod/btstack: Fix indicate/notify queuing.
This adds a mechanism to track a pending notify/indicate operation that
is deferred due to the send buffer being full. This uses a tracked alloc
that is passed as the content arg to the callback.

This replaces the previous mechanism that did this via the global pending
op queue, shared with client read/write ops.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-04-26 11:37:20 +10:00