Commit Graph

1119 Commits

Author SHA1 Message Date
Jim Mussared 8b27482692 top: Update Python formatting to black "2023 stable style".
See https://black.readthedocs.io/en/stable/the_black_code_style/index.html

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-02-02 12:51:03 +11:00
Florian Weimer f24cfd1a69 extmod/axtls-include: Add back needed header files for building axTLS.
MicroPython overrides the axTLS port configuration file, but fails to
include <arpa/inet.h> (needed for htonl) and <sys/time.h> (needed for
gettimeofday).  This results in build failures with compilers which do not
support implicit function declarations (which were removed from C in 1999).

This commit adds back the needed headers that were removed in this commit:
bd08017309

Signed-off-by: Damien George <damien@micropython.org>
2023-01-12 16:04:57 +11:00
Damien George 699477d12d extmod/network_cyw43: Fix handling of networks with open security.
Prior to this commit, the default security=-1 would be passed directly
through to the cyw43 driver to auto-detect the security type, but that
driver did not correctly handle the case of open security.

The cyw43 driver has now been changed to no longer support auto-detection,
rather it is up to the caller to always select the security type.  The
defaults are now implemented in the Python bindings and are:
- if no key is given then it selects open security
- if a key is given then it selects WPA2_MIXED_PSK

Calling `wlan.connect(<ssid>)` will now connect to an open network, on
both rp2 and stm32 ports.  The form `wlan.connect(<ssid>, <key>)` will
connect to a WPA2 network.

Fixes issue #9016.

Signed-off-by: Damien George <damien@micropython.org>
2022-12-20 16:06:40 +11:00
Damien Tournoud ed58d6e4ce extmod/modussl_mbedtls: Fix support for ioctl(MP_STREAM_POLL).
During the initial handshake or subsequent renegotiation, the protocol
might need to read in order to write (or conversely to write in order
to read). It might be blocked from doing so by the state of the
underlying socket (i.e. there is no data to read, or there is no space
to write).

The library indicates this condition by returning one of the errors
`MBEDTLS_ERR_SSL_WANT_READ` or `MBEDTLS_ERR_SSL_WANT_WRITE`. When that
happens, we need to enforce that the next poll operation only considers
the direction that the library indicated.

In addition, mbedtls does its own read buffering that we need to take
into account while polling, and we need to save the last error between
read()/write() and ioctl().
2022-12-15 12:06:22 -08:00
Jim Mussared 5f8f32f917 extmod/modnetwork: Use a type protocol to implement NIC functions.
This was previously implemented by adding additional members to the
mp_obj_type_t defined for each NIC, which is difficult to do cleanly with
the new object type slots mechanism. The way this works is also not
supported on GCC 8.x and below.

Instead replace it with the type protocol, which is a much simpler way of
achieving the same thing.

This affects the WizNet (in non-LWIP mode) and Nina NIC drivers.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-12-15 17:40:06 +11:00
Damien Tournoud 0eba00a92c extmod/uasyncio: Fix syntax of generator functions.
The compiler is not picky right now, but these are actually all syntax
errors:
- await is only valid in an async function
- async functions that use yield are actually async generators (a construct
  not supported by the compiler right now)
2022-12-14 13:25:24 +11:00
TPReal bf49a087b2 extmod/modframebuf: Fix crash in FrameBuffer scrolling beyond extents.
Fixed the crash occurring when scrolling by at least the size of the
framebuffer.
2022-12-09 16:47:23 +11:00
David Lechner 5b08e0361e extmod/moduplatform: Fix MSVC x86_64 check.
`_WIN64` is defined for all 64-bit targets, including Arm, so it doesn't
work for detecting `x86_64`. We can use `_M_X64` instead.

Signed-off-by: David Lechner <david@pybricks.com>
2022-11-18 15:13:00 +11:00
David Lechner 67e919db32 extmod/moduplatform: Remove _M_IX86 test for xtensa.
Since _M_IX86 is already being checked in the x86 case, it will never
be true in the xtensa case and can be removed.

Signed-off-by: David Lechner <david@pybricks.com>
2022-11-18 15:13:00 +11:00
David Lechner 4dcfd19bbf extmod/extmod.cmake: Add MICROPY_PY_BTREE compiler definition.
Instead of defining `MICROPY_PY_BTREE` in `mpconfigport.h` we can define
it via CMake similar to how other ports that use Makefiles define it in
`mpconfigport.mk`.

Signed-off-by: David Lechner <david@pybricks.com>
2022-11-17 23:20:58 +11:00
iabdalkader b9c1e4c205 drivers/ninaw10: Connect to WiFi asynchronously.
Before this patch, WiFi connection was blocking, and could raise exceptions
if the connection failed for any reason (including timeouts).  This doesn't
match the behavior of other WiFi modules, which connect asynchronously, and
requires handling of exceptions on connect.  This change makes `connect()`
work asynchronously by scheduling code to poll connection status, and
handle reconnects (if needed), and return immediately without blocking.
2022-11-16 09:46:43 +11:00
Damien George a513558e3a extmod: Add and reorganise compilation guards and includes.
To reduce dependencies on header files when extmod components are disabled.

Signed-off-by: Damien George <damien.p.george@gmail.com>
2022-11-11 16:24:32 +11:00
Damien George 9c9f06ad9d extmod/btstack: Allow the BTstack config to be overridden by a board.
If a board defines MICROPY_BLUETOOTH_BTSTACK_CONFIG_FILE as the path to a
header file, then that file will be used for the BTstack config.

Signed-off-by: Damien George <damien@micropython.org>
2022-11-11 15:14:14 +11:00
iabdalkader 71881116e6 extmod/extmod.mk: Set default mbedtls config file in extmod Makefile. 2022-11-09 18:43:13 +01:00
Jared Hancock a2fd382c34 extmod/modlwip: Use actual errno in exception for error in listen.
The actual underlying error number raised from the lwIP subsystem when
attempting to listen on a socket is swallowed and replaced with an
out-of-memory error which is confusing.

This commit passes the underlying error message from the lwIP subsystem to
the appropriate OSError exception.
2022-11-09 10:48:53 +11:00
iabdalkader ecd4d54391 extmod/extmod.cmake: Allow overriding the default MBEDTLS_CONFIG_FILE. 2022-11-08 23:50:17 +11:00
Damien George da36b84d45 extmod/vfs_posix: Include errno.h and unistd.h headers.
errno.h is needed for the errno variable, and unistd.h is needed for chdir.

Signed-off-by: Damien George <damien@micropython.org>
2022-11-01 13:10:45 +11:00
Damien George cd35b8a2a3 extmod/machine_timer: Move stm32's implementation of machine.Timer here.
So it can be reused by other ports.

Signed-off-by: Damien George <damien@micropython.org>
2022-10-27 14:32:43 +11:00
Damien George 68f166dae9 extmod/mbedtls: Remove brainpool curves from config.
They are much slower than NIST (SECP) curves and shouldn't be needed.

Reduces rp2 PICO_W firmware by 1328 bytes.

Thanks to @Carglglz for the information.

Signed-off-by: Damien George <damien@micropython.org>
2022-10-22 19:12:46 +11:00
Damien George e24159dec9 extmod/mbedtls: Remove MBEDTLS_ECP_DP_CURVE25519_ENABLED config.
Curve25519 arithmetic is supported in mbedtls, but it's not used for TLS.
So there's no need to have this option enabled.

Reduces rp2 PICO_W firmware by 2440 bytes.

Thanks to @Carglglz for the information.

Signed-off-by: Damien George <damien@micropython.org>
2022-10-22 19:12:46 +11:00
Damien George 8874a09119 extmod/mbedtls: Enable elliptic curve DH and DSA cryptography.
This is necessary to access sites that only support these protocols.

The rp2 port already has ECDH enabled, so this just adds ECDSA there.  The
other ports now gain both ECDH and ECDSA.  The code size increase is:

- rp2 (PICO_W): +2916 bytes flash, +24 bytes BSS
- stm32 (PYBD_SF6): +20480 bytes flash, +32 bytes data, +48 bytes BSS
- mimxrt (TEENSY41): +20708 bytes flash, +32 bytes data, +48 bytes BSS
- unix (standard x86-64): +39344 executable, +1744 bytes data, +96 BSS

This is obviously a large increase in code size.  But there doesn't seem to
be any other option because without elliptic curve cryptography devices are
partially cut off from the internet.  For use cases that require small
firmware size, they'll need to build custom firmware with a custom mbedtls
config.

Signed-off-by: Damien George <damien@micropython.org>
2022-10-22 19:08:21 +11:00
Damien George 9347545f9e extmod/mbedtls: Enable MBEDTLS_TLS_DEFAULT_ALLOW_SHA1_IN_KEY_EXCHANGE.
This was already enabled on all ports except mimxrt.  Now it's enabled on
all of them.

Signed-off-by: Damien George <damien@micropython.org>
2022-10-22 19:06:21 +11:00
Damien George b337678964 extmod/mbedtls: Add common configuration file, and use it in all ports.
This is a no-op change.

Signed-off-by: Damien George <damien@micropython.org>
2022-10-22 19:06:21 +11:00
Damien George 67f98ba10c extmod/btstack: Update BTstack bindings to work with latest BTstack.
The following multi-tests pass (eg with PYBD_SF6+LEGO_HUB_NO6):

    ble_gap_advertise.py
    ble_gap_connect.py
    ble_gap_device_name.py
    ble_gattc_discover_services.py
    ble_gatt_data_transfer.py
    perf_gatt_char_write.py
    perf_gatt_notify.py
    stress_log_filesystem.py

These are the same tests that passed prior to this BTstack update.

Also tested on the unix port using H4 transport.

Signed-off-by: Damien George <damien@micropython.org>
2022-10-22 14:28:25 +11:00
Damien George 815920c87f extmod/utime_mphal: Make ticks_add check for overflow of delta.
Work done in collaboration with @jimmo.

Signed-off-by: Damien George <damien@micropython.org>
2022-10-14 15:54:53 +11:00
Jim Mussared d6d8722558 extmod: Make extmod.mk self-contained.
This makes it so that all a port needs to do is set the relevant variables
and "include extmod.mk" and doesn't need to worry about adding anything to
OBJ, CFLAGS, SRC_QSTR, etc.

Make all extmod variables (src, flags, etc) private to extmod.mk.

Also move common/shared, extmod-related fragments (e.g. wiznet, cyw43,
bluetooth) into extmod.mk.

Now that SRC_MOD, CFLAGS_MOD, CXXFLAGS_MOD are unused by both extmod.mk
(and user-C-modules in a previous commit), remove all uses of them from
port makefiles.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-11 23:31:49 +11:00
Jim Mussared 87011f6353 extmod/extmod.mk: Make extmod.mk handle GIT_SUBMODULES.
This applies to nimble, btstack, axtls, mbedtls, lwip.

Rather than having the ports individually manage GIT_SUBMODULES for these
components, make extmod.mk append them when the relevant feature is
enabled.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-11 23:29:09 +11:00
Damien George f2ad152e7e extmod/modbluetooth: Run BLE IRQ callback in protected NLR context.
The call to invoke_irq_handler_run() always needs to run in a protected NLR
context, to catch exceptions from the Python handler, and the m_new's (and
also mp_local_alloc when PYSTACK is enabled).  With
MICROPY_PY_BLUETOOTH_USE_SYNC_EVENTS_WITH_INTERLOCK enabled there was
already an explicit nlr_push, and that is now used in all cases.

Without this change, on stm32 (for example), the callbacks from the BLE
stack to invoke_irq_handler() were made via static scheduled nodes which do
not have any NLR protection, and hence would lead to a hard fault (uncaught
NLR) if an exception was raised in the Python BLE IRQ handler.  This was a
regression introduced by 8045ac07f5, which is
fixed by this commit.

Signed-off-by: Damien George <damien@micropython.org>
2022-09-23 15:21:54 +10:00
Damien George db668742a5 extmod/modbluetooth: Do GATTC reassembly in protected uPy context.
The calls to m_new and m_del require an exclusive uPy (really a GC)
context.  In particular these functions cannot be called directly from a
FreeRTOS task on esp32.

Fixes issue #9369.

Signed-off-by: Damien George <damien@micropython.org>
2022-09-22 11:49:58 +10:00
Damien George ed41d51746 extmod/modbluetooth: Change data_len type from size_t to uint16_t.
For consistency, and to remove the need for additional conversion of types.

Signed-off-by: Damien George <damien@micropython.org>
2022-09-22 11:47:03 +10:00
stijn 9ae8d38204 extmod/vfs_posix_file: Implement finaliser for files.
Prevent handle leaks when file objects aren't closed explicitly and
fix some MICROPY_CPYTHON_COMPAT issues: this wasn't properly adhered
to because #ifdef was used so it was always on, and closing files
multiple times should be avoided unconditionally.
2022-09-19 23:44:50 +10:00
Jim Mussared 94beeabd2e py/obj: Convert make_new into a mp_obj_type_t slot.
Instead of being an explicit field, it's now a slot like all the other
methods.

This is a marginal code size improvement because most types have a make_new
(100/138 on PYBV11), however it improves consistency in how types are
declared, removing the special case for make_new.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:15 +10:00
Jim Mussared 6da41b5900 py/obj: Merge getiter and iternext mp_obj_type_t slots.
The goal here is to remove a slot (making way to turn make_new into a slot)
as well as reduce code size by the ~40 references to mp_identity_getiter
and mp_stream_unbuffered_iter.

This introduces two new type flags:
- MP_TYPE_FLAG_ITER_IS_ITERNEXT: This means that the "iter" slot in the
  type is "iternext", and should use the identity getiter.
- MP_TYPE_FLAG_ITER_IS_CUSTOM: This means that the "iter" slot is a pointer
  to a mp_getiter_iternext_custom_t instance, which then defines both
  getiter and iternext.

And a third flag that is the OR of both, MP_TYPE_FLAG_ITER_IS_STREAM: This
means that the type should use the identity getiter, and
mp_stream_unbuffered_iter as iternext.

Finally, MP_TYPE_FLAG_ITER_IS_GETITER is defined as a no-op flag to give
the default case where "iter" is "getiter".

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:13 +10:00
Jim Mussared a52cd5b07d py/obj: Add accessors for type slots and use everywhere.
This is a no-op, but sets the stage for changing the mp_obj_type_t
representation.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:07 +10:00
Jim Mussared e8355eb163 py/obj: Add "full" and "empty" non-variable-length mp_obj_type_t.
This will always have the maximum/minimum size of a mp_obj_type_t
representation and can be used as a member in other structs.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:04 +10:00
Jim Mussared 9dce82776d all: Remove unnecessary locals_dict cast.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:01 +10:00
Jim Mussared b7d6ee9b75 all: Fix #if inside MP_DEFINE_CONST_OBJ_TYPE for msvc.
Changes:

    MP_DEFINE_CONST_OBJ_TYPE(
       ...
       #if FOO
       ...
       #endif
       ...
    );

to:

    MP_DEFINE_CONST_OBJ_TYPE(
       ...
       FOO_TYPE_ATTR
       ...
    );

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:01 +10:00
Jim Mussared 662b9761b3 all: Make all mp_obj_type_t defs use MP_DEFINE_CONST_OBJ_TYPE.
In preparation for upcoming rework of mp_obj_type_t layout.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:01 +10:00
Jim Mussared 42587c7870 all: Standardise mp_obj_type_t initialisation.
Remove setting unused slots.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 18:41:29 +10:00
Jim Mussared fb2a57800a all: Simplify buffer protocol to just a "get buffer" callback.
The buffer protocol type only has a single member, and this existing layout
creates problems for the upcoming split/slot-index mp_obj_type_t layout
optimisations.

If we need to make the buffer protocol more sophisticated in the future
either we can rely on the mp_obj_type_t optimisations to just add
additional slots to mp_obj_type_t or re-visit the buffer protocol then.

This change is a no-op in terms of generated code.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 18:40:39 +10:00
Andrew Leech 4e0964b59f extmod/vfs: Add finaliser to ilistdir to close directory handle.
When iterating over filesystem/folders with os.iterdir(), an open file
(directory) handle is used internally.  Currently this file handle is only
closed once the iterator is completely drained, eg. once all entries have
been looped over / converted into list etc.

If a program opens an iterdir but does not loop over it, or starts to loop
over the iterator but breaks out of the loop, then the handle never gets
closed.  In this state, when the iter object is cleaned up by the garbage
collector this open handle can cause corruption of the filesystem.

Fixes issues #6568 and #8506.
2022-09-13 13:00:42 +10:00
Jim Mussared cacc96d98c extmod/modbluetooth: Replace def_handle with end_handle in char IRQ.
This is technically a breaking change, but:
a) We need the end handle to do descriptor discovery properly.
b) We have no possible use for the existing definition handle in the
characteristic result IRQ. None of the methods can use it, and therefore
no existing code should be using it in a way that changing it to a
different integer value should break.

Unfortunately NimBLE doesn't make it easy to get the end handle, so also
implement a mechanism to use the following characteristic to calculate
the previous characteristic's end handle.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-09 11:24:24 +10:00
Jim Mussared 82fc16f298 extmod/modbluetooth: Fix descriptor registration with empty tuple.
Incorrect use of "continue" when the tuple was length zero meant it
broke the rest of the argument handling.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-09 11:24:21 +10:00
Jim Mussared 24678fe452 drivers: Remove drivers that are now in micropython-lib.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-08 11:27:05 +10:00
Jeff Epler e90b85cc98 extmod/modure: Convert byte offsets to unicode indices when necessary.
And add a test.

Fixes issue #9202.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2022-09-06 17:08:18 +10:00
Jim Mussared 203dae41fb all: Update all manifest.py files to use new features.
Changes in this commit:
- Manifest include's now use the directory path where possible (no longer
  necessary to include the manifest.py file explicitly).
- Add manifest.py for all drivers and components that are referenced by
  port/board manifests.
- Replace all uses of freeze() with package()/module(), except for port and
  board modules.
- Use opt=3 everywhere, for consistency and to reduce code size.
- Use require() instead of include() for all micropython-lib references.
- Remove support for optional board-level manifest.py in mimxrt port, to
  make it behave the same as other ports (the board must set
  FROZEN_MANIFEST to a custom manifest.py, which can optionally include the
  default, port-level manifest).
- Also reinstates modules that were accidentally removed from the esp8266
  512k build in fbe9417b90.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
2022-09-05 18:43:18 +10:00
robert-hh 0b26efe73d extmod/machine_i2c: Call MICROPY_PY_EVENT_HOOK during i2c.scan().
Avoiding a watchdog reset during i2c.scan() if the hardware is not properly
set up (eg on esp8266), and also allowing to stop the scan with a
KeyboardInterrupt.

Fixes issue #8876.
2022-08-31 12:06:11 +10:00
Jim Mussared 3a910b1565 py/objstr: Optimise mp_obj_new_str_from_vstr for known-safe strings.
The new `mp_obj_new_str_from_utf8_vstr` can be used when you know you
already have a unicode-safe string.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-26 16:44:35 +10:00
Jim Mussared 8a0ee5a5c0 py/objstr: Split mp_obj_str_from_vstr into bytes/str versions.
Previously the desired output type was specified.  Now make the type part
of the function name.  Because this function is used in a few places this
saves code size due to smaller call-site.

This makes `mp_obj_new_str_type_from_vstr` a private function of objstr.c
(which is almost the only place where the output type isn't a compile-time
constant).

This saves ~140 bytes on PYBV11.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-26 16:43:55 +10:00
robert-hh 8308f9c977 extmod/network_wiznet5k: Use the configured DNS address if available.
Instead of the default 8.8.8.8.  The change was suggested by @omogenot.
2022-08-23 15:00:00 +10:00