Commit Graph

900 Commits

Author SHA1 Message Date
Jeff Epler
413f34cd8f all: Fix signed shifts and NULL access errors from -fsanitize=undefined.
Fixes the following (the line numbers match commit 0e87459e2b):

../../extmod/crypto-algorithms/sha256.c:49:19: runtime error: left shif...
../../extmod/moduasyncio.c:106:35: runtime error: member access within ...
../../py/binary.c:210:13: runtime error: left shift of negative value -...
../../py/mpz.c:744:16: runtime error: negation of -9223372036854775808 ...
../../py/objint.c:109:22: runtime error: left shift of 1 by 31 places c...
../../py/objint_mpz.c:374:9: runtime error: left shift of 4611686018427...
../../py/objint_mpz.c:374:9: runtime error: left shift of negative valu...
../../py/parsenum.c:106:14: runtime error: left shift of 46116860184273...
../../py/runtime.c:395:33: runtime error: left shift of negative value ...
../../py/showbc.c:177:28: runtime error: left shift of negative value -...
../../py/vm.c:321:36: runtime error: left shift of negative value -1```

Testing was done on an amd64 Debian Buster system using gcc-8.3 and these
settings:

    CFLAGS += -g3 -Og -fsanitize=undefined
    LDFLAGS += -fsanitize=undefined

The introduced TASK_PAIRHEAP macro's conditional (x ? &x->i : NULL)
assembles (under amd64 gcc 8.3 -Os) to the same as &x->i, since i is the
initial field of the struct.  However, for the purposes of undefined
behavior analysis the conditional is needed.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2021-06-24 23:01:04 +10:00
Damien George
6ed5b843cf extmod/btstack: Check that BLE is active before performing operations.
Otherwise it can easily lead to a hard crash.

Signed-off-by: Damien George <damien@micropython.org>
2021-06-23 14:58:39 +10:00
Damien George
0fc0f7536b extmod/btstack: Add missing call to mp_bluetooth_hci_uart_deinit.
Signed-off-by: Damien George <damien@micropython.org>
2021-06-23 13:14:26 +10:00
Damien George
38bc5a9f67 stm32: Provide a custom BTstack runloop that integrates with soft timer.
It reschedules the BT HCI poll soft timer so that it is called exactly when
the next timer expires.

Signed-off-by: Damien George <damien@micropython.org>
2021-06-23 13:14:26 +10:00
Damien George
8107c9b75b extmod/nimble: Remove TODO comment about notify_custom freeing om.
The comments in NimBLE for ble_gattc_notify_custom() state that "This
function consumes the supplied mbuf regardless of the outcome.".  And
inspection of NimBLE code shows that this is the case.  So the comment can
be removed.

Signed-off-by: Damien George <damien@micropython.org>
2021-06-17 14:54:04 +10:00
Damien George
514bf1a191 extmod/uasyncio: Fix race with cancelled task waiting on finished task.
This commit fixes a problem with a race between cancellation of task A and
completion of task B, when A waits on B.  If task B completes just before
task A is cancelled then the cancellation of A does not work.  Instead,
the CancelledError meant to cancel A gets passed through to B (that's
expected behaviour) but B handles it as a "Task exception wasn't retrieved"
scenario, printing out such a message (this is because finished tasks point
their "coro" attribute to themselves to indicate they are done, and
implement the throw() method, but that method inadvertently catches the
CancelledError).  The correct behaviour is for B to bounce that
CancelledError back out.

This bug is mainly seen when wait_for() is used, and in that context the
symptoms are:
- occurs when using wait_for(T, S), if the task T being waited on finishes
  at exactly the same time as the wait-for timeout S expires
- task T will have run to completion
- the "Task exception wasn't retrieved message" is printed with
  "<class 'CancelledError'>" as the error (ie no traceback)
- the wait_for(T, S) call never returns (it's never put back on the
  uasyncio run queue) and all tasks waiting on this are blocked forever
  from running
- uasyncio otherwise continues to function and other tasks continue to be
  scheduled as normal

The fix here reworks the "waiting" attribute of Task to be called "state"
and uses it to indicate whether a task is: running and not awaited on,
running and awaited on, finished and not awaited on, or finished and
awaited on.  This means the task does not need to point "coro" to itself to
indicate finished, and also allows removal of the throw() method.

A benefit of this is that "Task exception wasn't retrieved" messages can go
back to being able to print the name of the coroutine function.

Fixes issue #7386.

Signed-off-by: Damien George <damien@micropython.org>
2021-06-16 13:02:37 +10:00
Mike Teachman
b0b8ebc4f6 extmod/uasyncio: Add readinto() method to Stream class.
With docs and a multi-test using TCP server/client.

This method is a MicroPython extension, although there is discussion of
adding it to CPython: https://bugs.python.org/issue41305

Signed-off-by: Mike Teachman <mike.teachman@gmail.com>
2021-06-15 13:13:35 +10:00
Miguel Grinberg
de2e081260 extmod/uasyncio: Fix start_server and wait_closed race condition.
This fix prevents server.wait_closed() from raising an AttributeError when
trying to access server.task.  This can happen if it is called immediately
after start_server().
2021-06-08 15:10:50 +10:00
Damien George
c3199f5649 extmod/modurandom: Support an argument of bits=0 to getrandbits.
This was changed in CPython 3.9; see https://bugs.python.org/issue40282.

Signed-off-by: Damien George <damien@micropython.org>
2021-05-30 17:05:56 +10:00
Macarthur Inbody
34d4dab683 extmod/modurandom: Add error message when getrandbits has bad value.
The random module's getrandbits() method didn't give a proper error message
when calling it with a value that was outside of the range of 1-32, which
can lead to confusion using this function (which under CPython can accept
numbers larger than 32).  Now instead of simply giving a ValueError it
gives an error message that states that the number of bits is constrained.

Also, since the random module's functions getrandbits() and randint()
differ from CPython, tests have been added to describe these differences.
For getrandbits the relevant documentation is shown and added to the docs.
The same is given for randint method so that the information is more easily
found.

Finally, since the int object lacks the bit_length() method there is a test
for that method also to include within the docs, showing the difference to
CPython.
2021-05-30 16:41:30 +10:00
Damien George
6a127810c0 extmod/moduhashlib: Put hash obj in final state after digest is called.
If digest is called then the hash object is put in a "final" state and
calling update() or digest() again will raise a ValueError (instead of
silently producing the wrong result).

See issue #4119.

Signed-off-by: Damien George <damien@micropython.org>
2021-05-26 21:44:46 +10:00
Damien George
47583d8cbd extmod/moductypes: Fix size and offset calculation for ARRAY of FLOAT32.
uctypes.FLOAT32 has a special value representation and
uctypes_struct_scalar_size() should be used instead of GET_SCALAR_SIZE().

Signed-off-by: Damien George <damien@micropython.org>
2021-05-06 13:11:33 +10:00
Damien George
350a66a863 extmod/moductypes: Replace numbers with macro constants.
Signed-off-by: Damien George <damien@micropython.org>
2021-05-06 12:40:53 +10:00
Damien George
02dc1644b6 extmod/moductypes: Remove double blank lines and debugging printf's.
Signed-off-by: Damien George <damien@micropython.org>
2021-05-06 12:32:09 +10:00
Damien George
4791d290c6 extmod: Remove old comments used for auto-doc generation.
They are no longer used, and the text in the docs is more up to date.

Signed-off-by: Damien George <damien@micropython.org>
2021-05-06 12:27:31 +10:00
Damien George
e9e9c76ddf all: Rename mp_keyboard_interrupt to mp_sched_keyboard_interrupt.
To match mp_sched_exception() and mp_sched_schedule().

Signed-off-by: Damien George <damien@micropython.org>
2021-04-30 15:13:43 +10:00
Damien George
342d55529d extmod/uasyncio: Use .errno instead of .args[0] for OSError exceptions.
Signed-off-by: Damien George <damien@micropython.org>
2021-04-23 22:03:46 +10:00
Damien George
212fe7f33e extmod/extmod.cmake: Add support to build btree module with CMake.
Signed-off-by: Damien George <damien@micropython.org>
2021-04-09 13:33:26 +10:00
Jeff Epler
172fb5230a extmod/re1.5: Check and report byte overflow errors in _compilecode.
The generated regex code is limited in the range of jumps and counts, and
this commit checks all cases which can overflow given the right kind of
input regex, and returns an error in such a case.

This change assumes that the results that overflow an int8_t do not
overflow a platform int.

Closes: #7078

Signed-off-by: Jeff Epler <jepler@gmail.com>
2021-04-06 13:36:42 +10:00
Damien George
2b888aa2f3 extmod/modbluetooth: Free temp arrays in gatts register services.
This helps to reduce memory fragmentation, by freeing the heap data as soon
as it is not needed.  It also helps the compiler keeps a reference to the
beginning of both arrays, which need to be traceable by the GC (otherwise
some compilers may optimise this reference to something else).

Signed-off-by: Damien George <damien@micropython.org>
2021-03-16 13:55:45 +11:00
Damien George
eccd73a403 extmod/extmod.cmake: Add modonewire.c to MICROPY_SOURCE_EXTMOD list.
Signed-off-by: Damien George <damien@micropython.org>
2021-03-14 15:51:28 +11:00
Jim Mussared
a76604afba extmod/modbluetooth: Separate enabling of "client" from "central".
Previously, the MICROPY_PY_BLUETOOTH_ENABLE_CENTRAL_MODE macro
controlled enabling both the central mode and the GATT client
functionality (because usually the two go together).

This commits adds a new MICROPY_PY_BLUETOOTH_ENABLE_GATT_CLIENT
macro that separately enables the GATT client functionality.
This defaults to MICROPY_PY_BLUETOOTH_ENABLE_CENTRAL_MODE.

This also fixes a bug in the NimBLE bindings where a notification
or indication would not be received by a peripheral (acting as client)
as gap_event_cb wasn't handling it. Now both central_gap_event_cb
and peripheral_gap_event_cb share the same common handler for these
events.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-02-19 17:53:43 +11:00
Damien George
cf6a015880 extmod/btstack: Use MICROPY_HW_BLE_UART_BAUDRATE for first UART init.
Otherwise the UART may be left in a state at baudrate=0.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-17 16:06:56 +11:00
Thorsten von Eicken
2c1299b007 extmod/modussl: Fix ussl read/recv/send/write errors when non-blocking.
Also fix related problems with socket on esp32, improve docs for
wrap_socket, and add more tests.
2021-02-17 11:50:54 +11:00
Jim Mussared
4005138882 extmod/modbluetooth: Allow NimBLE to use Zephyr static address.
Zephyr controllers can be queried for a static address (computed from the
device ID).  BlueKitchen already supports this, but make them both use the
same macro to enable the feature.
2021-02-17 11:25:02 +11:00
Jim Mussared
236274f08f extmod/nimble/hal/hal_uart: Fix HCI_TRACE format specifiers.
Makes this work consistently on unix and stm32 ports.
2021-02-17 11:24:48 +11:00
Jim Mussared
5e96e89999 extmod/uasyncio: Add ThreadSafeFlag.
This is a MicroPython-extension that allows for code running in IRQ
(hard or soft) or scheduler context to sequence asyncio code.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-02-16 16:35:37 +11:00
Jim Mussared
fce0bd1a2a extmod/moduselect: Fix unsigned/signed comparison for timeout!=-1.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-02-16 14:59:19 +11:00
Jim Mussared
a1a2815799 extmod/nimble: Ensure handle is set on read error.
On error, the handle is only available on err->att_handle rather than
in attr->handle used in the non-error case.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-02-16 14:57:10 +11:00
Damien George
66098c0985 py,extmod: Add core cmake rule files.
These allow a port to use cmake natively instead of make.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-15 12:48:18 +11:00
Jim Mussared
7ed99544e4 extmod/uasyncio: Add asyncio.current_task().
Matches CPython behavior.

Fixes #6686
2021-02-13 15:11:17 +11:00
Damien George
50615fef89 extmod/btstack: Enable SYNC_EVENTS, PAIRING_BONDING by default.
Synchronous events work on stm32 and unix ports.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-12 12:08:09 +11:00
Damien George
24a8a408a9 extmod/btstack: Add stub functions for passkey, l2cap bindings.
Signed-off-by: Damien George <damien@micropython.org>
2021-02-12 12:07:51 +11:00
Damien George
7535f67dfb extmod/btstack: Add HCI trace debugging option in btstack_hci_uart.
Signed-off-by: Damien George <damien@micropython.org>
2021-02-12 12:07:29 +11:00
Damien George
26b4ef4c46 extmod/vfs_posix_file: Allow closing an already closed file.
Signed-off-by: Damien George <damien@micropython.org>
2021-02-11 22:54:41 +11:00
Jim Mussared
47d02b3104 extmod/nimble: Improve the flow control for l2cap recv path.
If the _IRQ_L2CAP_RECV handler does the actual consumption of the incoming
data (i.e. via l2cap_recvinto), rather than setting a flag for
non-scheduler-context to handle it later, then two things can happen:

- It can starve the VM (i.e. the scheduled task never terminates).  This is
  because calling l2cap_recvinto will empty the rx buffer, which will grant
  more credits to the channel (an HCI command), meaning more data can
  arrive.  This means that the loop in hal_uart.c that keeps reading HCI
  data from the uart and executing NimBLE events as they are created will
  not terminate, preventing other VM code from running.

- There's no flow control (i.e. data will arrive too quickly).  The channel
  shouldn't be given credits until after we return from scheduler context.

It's preferable that no work is done in scheduler/IRQ context.  But to
prevent this being a problem this commit changes l2cap_recvinto so that if
it is called in IRQ context, and the Python handler empties the rx buffer,
then don't grant credits until the Python handler is complete.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-01-30 13:22:40 +11:00
Damien George
ec0503bd0c extmod/modonewire: Use pin_od_high/pin_od_low instead of pin_write.
The pin is configured in open-drain mode so these od_high/od_low methods
should be used.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 23:57:10 +11:00
Damien George
15ac5a3df9 extmod/modframebuf: Change int to unsigned int in format methods args.
These args are already bounds checked and clipped, and using unsigned ints
can be more efficient.  It also eliminates possible issues and compiler
warnings with shifting of signed integers.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 23:57:10 +11:00
Damien George
71ea438561 extmod/vfs: Check block 0 and 1 when auto-detecting littlefs.
The superblock for littlefs is in block 0 and 1, but block 0 may be erased
or partially written, so block 1 must be checked if block 0 does not have a
valid littlefs superblock in it.

Prior to this commit, the mount of a block device which auto-detected the
filysystem type would fail for littlefs if block 0 did not contain a valid
superblock.  That is now fixed.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 15:02:55 +11:00
Jim Mussared
aa136b4d78 extmod/modbluetooth: Add ble.hci_cmd(ogf, ocf, req, resp) function.
This allows sending arbitrary HCI commands and getting the response.  The
return value of the function is the status of the command.

This is intended for debugging and not to be a part of the public API, and
must be enabled via mpconfigboard.h.  It's currently only implemented for
NimBLE bindings.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-01-22 18:15:12 +11:00
Jim Mussared
f7aafc0628 extmod/nimble: Don't assert on save-IRK failure. 2020-12-23 10:08:00 +11:00
Jim Mussared
f42a190247 extmod/nimble: Reset NimBLE BSS in mp_bluetooth_init.
Without this fix, each time service registration happened it would do an
increasingly large malloc() for service state.

See https://github.com/apache/mynewt-nimble/issues/896.
2020-12-23 10:07:49 +11:00
Oliver Joos
a13d1b50c9 extmod/vfs: Raise OSError(ENODEV) if mounting bdev without a filesystem.
This commit prevents uos.mount() from raising an AttributeError.
vfs_autodetect() is supposed to return an object that has a "mount" method,
so if no filesystem is found it should raise an OSError(ENODEV) and not
return the bdev itself which has no "mount" method.
2020-12-17 22:44:03 +11:00
Damien George
1719459c28 extmod/modubinascii: Update code, docs for hexlify now CPython has sep.
Since CPython 3.8 the optional "sep" argument to hexlify is officially
supported, so update comments in the code and the docs to reflect this.

Signed-off-by: Damien George <damien@micropython.org>
2020-12-14 14:35:29 +11:00
Jim Mussared
d79b9c6c7c extmod/nimble: Generate and persist a unique IRK.
This provides a workaround for
https://github.com/apache/mynewt-nimble/issues/887.

Without this, all devices would share a fixed default IRK.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:44:39 +11:00
Jim Mussared
e4f27cbee7 extmod/modbluetooth: Add support for passkey authentication.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:43:32 +11:00
Jim Mussared
4bcbbfdb6c extmod/modbluetooth: Simplify synchronous invoke_irq_handler signature.
Rather than dealing with the different int types, just pass them all as a
single array of mp_int_t with n_unsigned (before addr) and n_signed (after
addr).

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:43:01 +11:00
Jim Mussared
c4d08aa4e3 extmod/modbluetooth: Add support for bonding (key persistence).
This adds `_IRQ_GET_SECRET` and `_IRQ_SET_SECRET` events to allow the BT
stack to request the Python code retrive/store/delete secret key data.  The
actual keys and values are opaque to Python and stack-specific.

Only NimBLE is implemented (pending moving btstack to sync events).  The
secret store is designed to be compatible with BlueKitchen's TLV store API.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:41:41 +11:00
Jim Mussared
801e8ffacf extmod/modbluetooth: Add gap_pair(conn_handle) func to intiate pairing.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:41:26 +11:00
Jim Mussared
a1fcf30121 extmod/modbluetooth: Allow configuration of pairing/bonding parameters.
This allows setting the security and MITM-protection requirements.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:40:49 +11:00