This commit fixes a problem with a race between cancellation of task A and
completion of task B, when A waits on B. If task B completes just before
task A is cancelled then the cancellation of A does not work. Instead,
the CancelledError meant to cancel A gets passed through to B (that's
expected behaviour) but B handles it as a "Task exception wasn't retrieved"
scenario, printing out such a message (this is because finished tasks point
their "coro" attribute to themselves to indicate they are done, and
implement the throw() method, but that method inadvertently catches the
CancelledError). The correct behaviour is for B to bounce that
CancelledError back out.
This bug is mainly seen when wait_for() is used, and in that context the
symptoms are:
- occurs when using wait_for(T, S), if the task T being waited on finishes
at exactly the same time as the wait-for timeout S expires
- task T will have run to completion
- the "Task exception wasn't retrieved message" is printed with
"<class 'CancelledError'>" as the error (ie no traceback)
- the wait_for(T, S) call never returns (it's never put back on the
uasyncio run queue) and all tasks waiting on this are blocked forever
from running
- uasyncio otherwise continues to function and other tasks continue to be
scheduled as normal
The fix here reworks the "waiting" attribute of Task to be called "state"
and uses it to indicate whether a task is: running and not awaited on,
running and awaited on, finished and not awaited on, or finished and
awaited on. This means the task does not need to point "coro" to itself to
indicate finished, and also allows removal of the throw() method.
A benefit of this is that "Task exception wasn't retrieved" messages can go
back to being able to print the name of the coroutine function.
Fixes issue #7386.
Signed-off-by: Damien George <damien@micropython.org>
With docs and a multi-test using TCP server/client.
This method is a MicroPython extension, although there is discussion of
adding it to CPython: https://bugs.python.org/issue41305
Signed-off-by: Mike Teachman <mike.teachman@gmail.com>
This fix prevents server.wait_closed() from raising an AttributeError when
trying to access server.task. This can happen if it is called immediately
after start_server().
The random module's getrandbits() method didn't give a proper error message
when calling it with a value that was outside of the range of 1-32, which
can lead to confusion using this function (which under CPython can accept
numbers larger than 32). Now instead of simply giving a ValueError it
gives an error message that states that the number of bits is constrained.
Also, since the random module's functions getrandbits() and randint()
differ from CPython, tests have been added to describe these differences.
For getrandbits the relevant documentation is shown and added to the docs.
The same is given for randint method so that the information is more easily
found.
Finally, since the int object lacks the bit_length() method there is a test
for that method also to include within the docs, showing the difference to
CPython.
If digest is called then the hash object is put in a "final" state and
calling update() or digest() again will raise a ValueError (instead of
silently producing the wrong result).
See issue #4119.
Signed-off-by: Damien George <damien@micropython.org>
* modframebuf: _mp_framebuf_p_t is not "really" a protocol, but the
QSTR assignment caused problems when building as a dynamic module
* modure: str_index_to_ptr is not in the natmod API, disable URE match
spans when dynamic. mp_obj_len() is a bugfix, we should throw here
if the object is not string-like
* moduzlib: Correct paths to uzlib headers & sources. this relative
path (from moduzlib.c to the referenced file) works in all cases,
the other only worked from ports/PORTNAME.
* dynruntime: Handle 2-arg m_malloc, assert_native_inited, add a
micropythonish mp_arg_check_num_mp, fix mp_raise_msg to use dumb
strings, add mp_raise_arg1
* nativeglue: ad assert_native_inited
* translate: MP_ERROR_TEXT evaluates to its argument for DYNRUNTIME
* mpy-tool: A straggling magic number change
* mpy_ld: Have to renumber manually after dynruntime change
* import_mpy_native_gc.py: Update copy of features0 baked into this test
uctypes.FLOAT32 has a special value representation and
uctypes_struct_scalar_size() should be used instead of GET_SCALAR_SIZE().
Signed-off-by: Damien George <damien@micropython.org>
The generated regex code is limited in the range of jumps and counts, and
this commit checks all cases which can overflow given the right kind of
input regex, and returns an error in such a case.
This change assumes that the results that overflow an int8_t do not
overflow a platform int.
Closes: #7078
Signed-off-by: Jeff Epler <jepler@gmail.com>
# Conflicts:
# extmod/re1.5/compilecode.c
The generated regex code is limited in the range of jumps and counts, and
this commit checks all cases which can overflow given the right kind of
input regex, and returns an error in such a case.
This change assumes that the results that overflow an int8_t do not
overflow a platform int.
Closes: #7078
Signed-off-by: Jeff Epler <jepler@gmail.com>
This helps to reduce memory fragmentation, by freeing the heap data as soon
as it is not needed. It also helps the compiler keeps a reference to the
beginning of both arrays, which need to be traceable by the GC (otherwise
some compilers may optimise this reference to something else).
Signed-off-by: Damien George <damien@micropython.org>
Previously, the MICROPY_PY_BLUETOOTH_ENABLE_CENTRAL_MODE macro
controlled enabling both the central mode and the GATT client
functionality (because usually the two go together).
This commits adds a new MICROPY_PY_BLUETOOTH_ENABLE_GATT_CLIENT
macro that separately enables the GATT client functionality.
This defaults to MICROPY_PY_BLUETOOTH_ENABLE_CENTRAL_MODE.
This also fixes a bug in the NimBLE bindings where a notification
or indication would not be received by a peripheral (acting as client)
as gap_event_cb wasn't handling it. Now both central_gap_event_cb
and peripheral_gap_event_cb share the same common handler for these
events.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Since the datasheet cast some doubt on the strength of the "rosc_hw->randombit",
I use the SHA256 hash function to create a high quality random seed
from random values of uncertain entropy, as well as to generate a sequence
of random values from that seed using SHA256 as a cryptographically-secure
random number generator.
In practice, it produces over 100kB/s of random data which does not
have any gross problems according to _PractRand_.
Zephyr controllers can be queried for a static address (computed from the
device ID). BlueKitchen already supports this, but make them both use the
same macro to enable the feature.
This is a MicroPython-extension that allows for code running in IRQ
(hard or soft) or scheduler context to sequence asyncio code.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
On error, the handle is only available on err->att_handle rather than
in attr->handle used in the non-error case.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
If the _IRQ_L2CAP_RECV handler does the actual consumption of the incoming
data (i.e. via l2cap_recvinto), rather than setting a flag for
non-scheduler-context to handle it later, then two things can happen:
- It can starve the VM (i.e. the scheduled task never terminates). This is
because calling l2cap_recvinto will empty the rx buffer, which will grant
more credits to the channel (an HCI command), meaning more data can
arrive. This means that the loop in hal_uart.c that keeps reading HCI
data from the uart and executing NimBLE events as they are created will
not terminate, preventing other VM code from running.
- There's no flow control (i.e. data will arrive too quickly). The channel
shouldn't be given credits until after we return from scheduler context.
It's preferable that no work is done in scheduler/IRQ context. But to
prevent this being a problem this commit changes l2cap_recvinto so that if
it is called in IRQ context, and the Python handler empties the rx buffer,
then don't grant credits until the Python handler is complete.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
These args are already bounds checked and clipped, and using unsigned ints
can be more efficient. It also eliminates possible issues and compiler
warnings with shifting of signed integers.
Signed-off-by: Damien George <damien@micropython.org>
The superblock for littlefs is in block 0 and 1, but block 0 may be erased
or partially written, so block 1 must be checked if block 0 does not have a
valid littlefs superblock in it.
Prior to this commit, the mount of a block device which auto-detected the
filysystem type would fail for littlefs if block 0 did not contain a valid
superblock. That is now fixed.
Signed-off-by: Damien George <damien@micropython.org>
This allows sending arbitrary HCI commands and getting the response. The
return value of the function is the status of the command.
This is intended for debugging and not to be a part of the public API, and
must be enabled via mpconfigboard.h. It's currently only implemented for
NimBLE bindings.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This commit prevents uos.mount() from raising an AttributeError.
vfs_autodetect() is supposed to return an object that has a "mount" method,
so if no filesystem is found it should raise an OSError(ENODEV) and not
return the bdev itself which has no "mount" method.
Since CPython 3.8 the optional "sep" argument to hexlify is officially
supported, so update comments in the code and the docs to reflect this.
Signed-off-by: Damien George <damien@micropython.org>
Rather than dealing with the different int types, just pass them all as a
single array of mp_int_t with n_unsigned (before addr) and n_signed (after
addr).
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This adds `_IRQ_GET_SECRET` and `_IRQ_SET_SECRET` events to allow the BT
stack to request the Python code retrive/store/delete secret key data. The
actual keys and values are opaque to Python and stack-specific.
Only NimBLE is implemented (pending moving btstack to sync events). The
secret store is designed to be compatible with BlueKitchen's TLV store API.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This allows the application to be notified if any of encrypted,
authenticated and bonded state change, as well as the encryption key size.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Enable it on STM32/Unix NimBLE only (pairing/bonding requires synchronous
events and full bindings).
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Instead of returning None/bool from the IRQ, return None/int (where a zero
value means success). This mirrors how the L2CAP_ACCEPT return value
works.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This widens the characteristic/descriptor flags to 16-bit, to allow setting
encryption/authentication requirements.
Sets the required flags for NimBLE and btstack implementations.
The BLE.FLAG_* constants will eventually be deprecated in favour of copy
and paste Python constants (like the IRQs).
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This allows the application to be notified of changes to the connection
interval, connection latency and supervision timeout.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This commit switches the roles of the helper task from a cancellation task
to a runner task, to get the correct semantics for cancellation of
wait_for.
Some uasyncio tests are now disabled for the native emitter due to issues
with native code generation of generators and yield-from.
Fixes#5797.
Signed-off-by: Damien George <damien@micropython.org>
This is added because task.coro==None is no longer the way to detect if a
task is finished. Providing a (CPython compatible) function for this
allows the implementation to be abstracted away.
Signed-off-by: Damien George <damien@micropython.org>
When a tasks raises an exception which is uncaught, and no other task
await's on that task, then an error message is printed (or a user function
called) via a call to Loop.call_exception_handler. In CPython this call is
made when the Task object is freed (eg via reference counting) because it's
at that point that it is known that the exception that was raised will
never be handled.
MicroPython does not have reference counting and the current behaviour is
to deal with uncaught exceptions as early as possible, ie as soon as they
terminate the task. But this can be undesirable because in certain cases
a task can start and raise an exception immediately (before any await is
executed in that task's coro) and before any other task gets a chance to
await on it to catch the exception.
This commit changes the behaviour so that tasks which end due to an
uncaught exception are scheduled one more time for execution, and if they
are not await'ed on by the next scheduling loop, then the exception handler
is called (eg the exception is printed out).
Signed-off-by: Damien George <damien@micropython.org>
Disable certain classes of diagnostic when building ulab. We should
submit patches upstream to (A) fix these errors and (B) upgrade their
CI so that the problems are caught before we want to integrate with
CircuitPython, but not right now.
Also known as L2CAP "connection oriented channels". This provides a
socket-like data transfer mechanism for BLE.
Currently only implemented for NimBLE on STM32 / Unix.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Hardware I2C implementations must provide a .init() protocol method if they
want to support reconfiguration. Otherwise the default is that i2c.init()
raises an OSError (currently the case for all ports).
mp_machine_soft_i2c_locals_dict is renamed to mp_machine_i2c_locals_dict to
match the generic SPI bindings.
Fixes issue #6623 (where calling .init() on a HW I2C would crash).
Signed-off-by: Damien George <damien@micropython.org>
This changes stm32 from using PENDSV to run NimBLE to use the MicroPython
scheduler instead. This allows Python BLE callbacks to be invoked directly
(and therefore synchronously) rather than via the ringbuffer.
The NimBLE UART HCI and event processing now happens in a scheduled task
every 128ms. When RX IRQ idle events arrive, it will also schedule this
task to improve latency.
There is a similar change for the unix port where the background thread now
queues the scheduled task.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This requires that the event handlers are called from non-interrupt context
(i.e. the MicroPython scheduler).
This will allow the BLE stack (e.g. NimBLE) to run from the scheduler
rather than an IRQ like PENDSV, and therefore be able to invoke Python
callbacks directly/synchronously. This allows writing Python BLE handlers
for events that require immediate response such as _IRQ_READ_REQUEST (which
was previous a hard IRQ) and future events relating to pairing/bonding.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Using a semaphore (the previous approach) will only run the UART, whereas
for startup we need to also run the event queue.
This change makes it run the full scheduler hook.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Instead of having the stack indicate a "start", "data"..., "end", pass
through the data in one callback as an array of chunks of data.
This is because the upcoming non-ringbuffer modbluetooth implementation
cannot buffer the data in the ringbuffer and requires instead a single
callback with all the data, to pass to the Python callback.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Prior to this change machine.mem32['foo'] (or using any other non-integer
subscript) could result in a fault due to 'foo' being interpreted as an
integer. And when writing code it's hard to tell if the fault is due to a
bad subscript type, or an integer subscript that specifies an invalid
memory address.
The type of the object used in the subscript is now tested to be an
integer by using mp_obj_get_int_truncated instead of
mp_obj_int_get_truncated. The performance hit of this change is minimal,
and machine.memX objects are more for convenience than performance (there
are many other ways to read/write memory in a faster way),
Fixes issue #6588.
Damien George
Mike Teachman
Jeff Epler
Miguel Grinberg
Macarthur Inbody
Scott Shawcroft
Artyom Skrobov
Scott Shawcroft
microDev
Jim Mussared
Thorsten von Eicken
Jeff Epler
Oliver Joos
Andrew Leech
Jeff Epler
Arrowana