An SSL stream can only handle CLOSE and POLL ioctls. Other ones do not
make sense, or at least it doesn't make sense to pass the ioctl request
directly down to the underlying stream.
In particular MP_STREAM_GET_FILENO should not be passed to the underlying
stream because the SSL stream is not directly related to a file descriptor,
and the SSL stream must handle the polling itself.
Signed-off-by: Damien George <damien@micropython.org>
The signature of this method was poller.poll(timeout=-1, flags=0, /) but
the flags argument was not documented and is not CPython compatible. So
it's removed in this commit.
(The optional flags remains for the ipoll() method, which is documented.)
Signed-off-by: Damien George <damien@micropython.org>
A previous commit removed the unix-specific select module implementation
and made unix use the common one.
This commit adds an optimisation so that the system poll function is used
when polling objects that have a file descriptor. With this optimisation
enabled, if code registers both file-descriptor-based objects, and non-
file-descriptor-based objects with select.poll() then the following occurs:
- the system poll is called for all file-descriptor-based objects with a
timeout of 1ms
- then the bare-metal polling implementation is used for remaining objects,
which calls into their ioctl method (which can be in C or Python)
In the case where all objects have file descriptors, the system poll is
called with the full timeout requested by the caller. That makes it as
efficient as possible in the case everything has a file descriptor.
Benefits of this approach:
- all ports use the same select module implementation
- the unix port now supports polling of all objects and matches bare metal
implementations
- it's still efficient for existing cases where only files and sockets are
polled (on unix)
- the bare metal implementation does not change
- polling of SSL objects will now work on unix by calling in to the ioctl
method on SSL objects (this is required for asyncio ssl support)
Note that extmod/vfs_posix_file.c has poll disable when the optimisation is
enabled, because the code is not reachable when the optimisation is used.
Signed-off-by: Damien George <damien@micropython.org>
This provides similar functionality to the former zlib.DecompIO and
especially CPython's gzip.GzipFile for both compression and decompression.
This class can be used directly, and also can be used from Python to
implement (via io.BytesIO) zlib.decompress and zlib.compress, as well as
gzip.GzipFile.
Enable/disable this on all ports/boards that zlib was previously configured
for.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
There are enough places that implement __exit__ by forwarding directly to
mp_stream_close that this saves code size.
For the cases where __exit__ is a no-op, additionally make their
MP_STREAM_CLOSE ioctl handled as a no-op.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This will be replaced with a new deflate module providing the same
functionality, with an optional frozen Python wrapper providing a
replacement zlib module.
binascii.crc32 is temporarily disabled.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This commit adds the SSLContext class to the ssl module, and retains the
existing ssl.wrap_socket() function to maintain backwards compatibility.
CPython deprecated the ssl.wrap_socket() function since CPython 3.7 and
instead one should use ssl.SSLContext().wrap_socket(). This commit makes
that possible.
For the axtls implementation:
- ssl.SSLContext is added, although it doesn't hold much state because
axtls requires calling ssl_ctx_new() for each new socket
- ssl.SSLContext.wrap_socket() is added
- ssl.PROTOCOL_TLS_CLIENT and ssl.PROTOCOL_TLS_SERVER are added
For the mbedtls implementation:
- ssl.SSLContext is added, and holds most of the mbedtls state
- ssl.verify_mode is added (getter and setter)
- ssl.SSLContext.wrap_socket() is added
- ssl.PROTOCOL_TLS_CLIENT and ssl.PROTOCOL_TLS_SERVER are added
The signatures match CPython:
- SSLContext(protocol)
- SSLContext.wrap_socket(sock, *, server_side=False,
do_handshake_on_connect=True, server_hostname=None)
The existing ssl.wrap_socket() functions retain their existing signature.
Signed-off-by: Damien George <damien@micropython.org>
The mod_binascii_a2b_base64() function allocates a buffer which may be
too small. It needs to be no less than three-quarters of the input
length, but is calculated as (<length> / 4) * 3 + 1, which may be less
due to integer division. Changed to (<length> * 3) / 4 + 1.
Signed-off-by: Duncan Lowther <Duncan.Lowther@glasgow.ac.uk>
This allows existing code that does `import uasyncio` or
`import uasyncio as asyncio` to continue working.
It uses the same lazy-loading as asyncio to prevent loading of unused
features.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The asyncio module now has much better CPython compatibility and
deserves to be just called "asyncio".
This will avoid people having to write `from uasyncio import asyncio`.
Renames all files, and updates port manifests to use the new path. Also
renames the built-in _uasyncio to _asyncio.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This is a MicroPython-specific module that existed to support the old
version of uasyncio. It's undocumented and not enabled on all ports and
takes up code size unnecessarily.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Applies to drivers/examples/extmod/port-modules/tools.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Updates any includes, and references from Makefiles/CMake.
This essentially reverts what was done long ago in commit
136b5cbd76
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This renames the builtin-modules, such that help('modules') and printing
the module object will show "module" rather than "umodule".
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
To be consistent with MP_UNARY_OP_INT_FLOAT and MP_UNARY_OP_INT_COMPLEX,
and allow int() to first check if a type supports __int__ before trying
other things (as per CPython).
Signed-off-by: Damien George <damien@micropython.org>
Avoiding code duplication. To enable it, set MICROPY_PY_UOS_SYNC in the
port's mpconfigport.h. It is operational only for FAT file system. For
other filesystems it's a no-op.
Signed-off-by: robert-hh <robert@hammelrath.com>
Tell the compiler we know what we are doing, and that the bytes are
correctly aligned, to avoid compiler warning:
error: cast increases required alignment of target type
All ports that enable MICROPY_PY_MACHINE_PWM now enable these two
sub-options, so remove these sub-options altogether to force consistency in
new ports that implement machine.PWM.
Signed-off-by: Damien George <damien@micropython.org>
This fixes:
- type-comparison (E721): do not compare types, use isinstance().
- string-dot-format-missing-arguments (F524): .format call is missing
argument(s) for placeholder(s): {message}.
- f-string-missing-placeholders (F541).
- is-literal (F632): Use != to compare constant literals.
The last one is fixed by just comparing for truthfulness of `state`.
Based on extmod/utime_mphal.c, with:
- a globals dict added
- time.localtime wrapper added
- time.time wrapper added
- time.time_ns function added
New configuration options are added for this module:
- MICROPY_PY_UTIME (enabled at basic features level)
- MICROPY_PY_UTIME_GMTIME_LOCALTIME_MKTIME
- MICROPY_PY_UTIME_TIME_TIME_NS
Signed-off-by: Damien George <damien@micropython.org>
- Use HCI_TRACE macro consistently.
- Use the same colour formatting.
- Add a tool to convert to .pcap for Wireshark.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The att_flags for descriptors does not use the same bitfield as for
characteristics. This was leading to NimBLE descriptors getting the wrong
flags set.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This allows gatts_write(..., send_update=True) to work, which will send
notifications/indications to subscribed clients.
btstack already created the CCCD but writes to it were ignored.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
There was no event handler for central-initiated MTU exchange.
Fix truncation of notify/indicate to match NimBLE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This replaces the previous pending operation queue (that used to also be
shared with pending server notify/indicate ops) with a single pending
operation per connection. This allows the value handle to be correctly
passed to the Python-level events.
Also re-structure GATT client event handling to simplify the packet handler
functions.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This adds a mechanism to track a pending notify/indicate operation that
is deferred due to the send buffer being full. This uses a tracked alloc
that is passed as the content arg to the callback.
This replaces the previous mechanism that did this via the global pending
op queue, shared with client read/write ops.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Makes gatts_notify and gatts_indicate work in the same way: by default they
send the DB value, but you can manually override the payload.
In other words, makes gatts_indicate work the same as gatts_notify.
Note: This removes support for queuing notifications and indications on
btstack when the ACL buffer is full. This functionality will be
reimplemented in a future commit.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This enables the use of WLAN(0).status('rssi') to get current RSSI of the
AP that the STA is connected to.
Signed-off-by: Damien George <damien@micropython.org>
The NINA socket types have the same values as modnetwork, but that may
change in the future. So check the socket types passed to socket() and
convert them (if needed) to their respective Nina socket types.
Also remove the unnecessary socket type check code from bind(), as pointed
out by @robert-hh.
When iterating over os.ilistdir(), the special directories '.' and '..'
are filtered from the results. But the code inadvertently also filtered
any file/directory which happened to match '..*'. This change fixes the
filter.
Fixes issue #11032.
Signed-off-by: Jeremy Rand <jeremy@rand-family.com>
This removes the previous WiFi driver from drivers/cyw43 (but leaves behind
the BT driver), and makes the stm32 port (i.e. PYBD and Portenta) use the
new "lib/cyw43-driver" open-source driver already in use by the rp2 port.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This allows for a port (e.g. esp8266/esp32) to use extmod/modnetwork.c
and provide the globals dict, rather than just a list of interfaces.
When this is used, the default implementation of `network.route` and the
NIC list is not enabled.
Also splits out the LWIP-specific helpers from modnetwork.c into
network_lwip.c.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This provides a standard interface to setting the global networking config
for all interfaces and interface types.
For ports that already use either a static hostname (mimxrt, rp2) they will
now use the configured value. The default is configured by the port
(or optionally the board).
For interfaces that previously supported .config(hostname), this is still
supported but now implemented using the global network.hostname.
Similarly, pyb.country and rp2.country are now deprecated, but the methods
still exist (and forward to network.hostname).
Because ESP32/ESP8266 do not use extmod/modnetwork.c they are not affected
by this commit.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Since commit d6d8722558, modbtree.c is
included unconditionally in the build (if SRC_EXTMOD_C is used). So guard
the includes of system headers files in case a target doesn't have them.
Signed-off-by: Damien George <damien@micropython.org>
MicroPython overrides the axTLS port configuration file, but fails to
include <arpa/inet.h> (needed for htonl) and <sys/time.h> (needed for
gettimeofday). This results in build failures with compilers which do not
support implicit function declarations (which were removed from C in 1999).
This commit adds back the needed headers that were removed in this commit:
bd08017309
Signed-off-by: Damien George <damien@micropython.org>
Prior to this commit, the default security=-1 would be passed directly
through to the cyw43 driver to auto-detect the security type, but that
driver did not correctly handle the case of open security.
The cyw43 driver has now been changed to no longer support auto-detection,
rather it is up to the caller to always select the security type. The
defaults are now implemented in the Python bindings and are:
- if no key is given then it selects open security
- if a key is given then it selects WPA2_MIXED_PSK
Calling `wlan.connect(<ssid>)` will now connect to an open network, on
both rp2 and stm32 ports. The form `wlan.connect(<ssid>, <key>)` will
connect to a WPA2 network.
Fixes issue #9016.
Signed-off-by: Damien George <damien@micropython.org>
During the initial handshake or subsequent renegotiation, the protocol
might need to read in order to write (or conversely to write in order
to read). It might be blocked from doing so by the state of the
underlying socket (i.e. there is no data to read, or there is no space
to write).
The library indicates this condition by returning one of the errors
`MBEDTLS_ERR_SSL_WANT_READ` or `MBEDTLS_ERR_SSL_WANT_WRITE`. When that
happens, we need to enforce that the next poll operation only considers
the direction that the library indicated.
In addition, mbedtls does its own read buffering that we need to take
into account while polling, and we need to save the last error between
read()/write() and ioctl().
This was previously implemented by adding additional members to the
mp_obj_type_t defined for each NIC, which is difficult to do cleanly with
the new object type slots mechanism. The way this works is also not
supported on GCC 8.x and below.
Instead replace it with the type protocol, which is a much simpler way of
achieving the same thing.
This affects the WizNet (in non-LWIP mode) and Nina NIC drivers.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The compiler is not picky right now, but these are actually all syntax
errors:
- await is only valid in an async function
- async functions that use yield are actually async generators (a construct
not supported by the compiler right now)
Previously the only other way of determining whether the Vfs has been mounted
read-write or read-only appears to be to attempt a write operation and detect a
possible OSError.
It wasn't possible for the user code to keep track of the state of the state
since the boot VM has to decide whether to (re)mount read-write or read-only,
but can't (easily) pass this information on to the runtime VM.
`_WIN64` is defined for all 64-bit targets, including Arm, so it doesn't
work for detecting `x86_64`. We can use `_M_X64` instead.
Signed-off-by: David Lechner <david@pybricks.com>
Since _M_IX86 is already being checked in the x86 case, it will never
be true in the xtensa case and can be removed.
Signed-off-by: David Lechner <david@pybricks.com>
Instead of defining `MICROPY_PY_BTREE` in `mpconfigport.h` we can define
it via CMake similar to how other ports that use Makefiles define it in
`mpconfigport.mk`.
Signed-off-by: David Lechner <david@pybricks.com>
Before this patch, WiFi connection was blocking, and could raise exceptions
if the connection failed for any reason (including timeouts). This doesn't
match the behavior of other WiFi modules, which connect asynchronously, and
requires handling of exceptions on connect. This change makes `connect()`
work asynchronously by scheduling code to poll connection status, and
handle reconnects (if needed), and return immediately without blocking.
If a board defines MICROPY_BLUETOOTH_BTSTACK_CONFIG_FILE as the path to a
header file, then that file will be used for the BTstack config.
Signed-off-by: Damien George <damien@micropython.org>
The actual underlying error number raised from the lwIP subsystem when
attempting to listen on a socket is swallowed and replaced with an
out-of-memory error which is confusing.
This commit passes the underlying error message from the lwIP subsystem to
the appropriate OSError exception.
supervisor_ticks_ms is ALREADY a small int, so passing it to
MP_OBJ_SMALL_INT again messes things up. I don't know why this passed
muster with the C type system, but oh well.
They are much slower than NIST (SECP) curves and shouldn't be needed.
Reduces rp2 PICO_W firmware by 1328 bytes.
Thanks to @Carglglz for the information.
Signed-off-by: Damien George <damien@micropython.org>
Curve25519 arithmetic is supported in mbedtls, but it's not used for TLS.
So there's no need to have this option enabled.
Reduces rp2 PICO_W firmware by 2440 bytes.
Thanks to @Carglglz for the information.
Signed-off-by: Damien George <damien@micropython.org>
This is necessary to access sites that only support these protocols.
The rp2 port already has ECDH enabled, so this just adds ECDSA there. The
other ports now gain both ECDH and ECDSA. The code size increase is:
- rp2 (PICO_W): +2916 bytes flash, +24 bytes BSS
- stm32 (PYBD_SF6): +20480 bytes flash, +32 bytes data, +48 bytes BSS
- mimxrt (TEENSY41): +20708 bytes flash, +32 bytes data, +48 bytes BSS
- unix (standard x86-64): +39344 executable, +1744 bytes data, +96 BSS
This is obviously a large increase in code size. But there doesn't seem to
be any other option because without elliptic curve cryptography devices are
partially cut off from the internet. For use cases that require small
firmware size, they'll need to build custom firmware with a custom mbedtls
config.
Signed-off-by: Damien George <damien@micropython.org>
Dan Halbert
Damien George
Jeff Epler
Jim Mussared
Duncan Lowther
Peter Harper
robert-hh
Graeme Winter
Chris Wilson
Mingjie Shen
Christian Clauss
Carlosgg
iabdalkader
Oliver Joos
Jeremy Rand
Florian Weimer
Damien Tournoud
TPReal
s-ol
David Lechner
Jared Hancock