Commit Graph

1027 Commits

Author SHA1 Message Date
Jeff Epler 0593e464bf ulab: bump to version 1.6.1 2021-01-11 13:52:46 -06:00
Jeff Epler 09bf58407f ulab: update to 1.6.0
This fixes several problems I encountered.
2021-01-04 12:50:08 -06:00
Jim Mussared f7aafc0628 extmod/nimble: Don't assert on save-IRK failure. 2020-12-23 10:08:00 +11:00
Jim Mussared f42a190247 extmod/nimble: Reset NimBLE BSS in mp_bluetooth_init.
Without this fix, each time service registration happened it would do an
increasingly large malloc() for service state.

See https://github.com/apache/mynewt-nimble/issues/896.
2020-12-23 10:07:49 +11:00
Oliver Joos a13d1b50c9 extmod/vfs: Raise OSError(ENODEV) if mounting bdev without a filesystem.
This commit prevents uos.mount() from raising an AttributeError.
vfs_autodetect() is supposed to return an object that has a "mount" method,
so if no filesystem is found it should raise an OSError(ENODEV) and not
return the bdev itself which has no "mount" method.
2020-12-17 22:44:03 +11:00
Damien George 1719459c28 extmod/modubinascii: Update code, docs for hexlify now CPython has sep.
Since CPython 3.8 the optional "sep" argument to hexlify is officially
supported, so update comments in the code and the docs to reflect this.

Signed-off-by: Damien George <damien@micropython.org>
2020-12-14 14:35:29 +11:00
Jim Mussared d79b9c6c7c extmod/nimble: Generate and persist a unique IRK.
This provides a workaround for
https://github.com/apache/mynewt-nimble/issues/887.

Without this, all devices would share a fixed default IRK.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:44:39 +11:00
Jim Mussared e4f27cbee7 extmod/modbluetooth: Add support for passkey authentication.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:43:32 +11:00
Jim Mussared 4bcbbfdb6c extmod/modbluetooth: Simplify synchronous invoke_irq_handler signature.
Rather than dealing with the different int types, just pass them all as a
single array of mp_int_t with n_unsigned (before addr) and n_signed (after
addr).

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:43:01 +11:00
Jim Mussared c4d08aa4e3 extmod/modbluetooth: Add support for bonding (key persistence).
This adds `_IRQ_GET_SECRET` and `_IRQ_SET_SECRET` events to allow the BT
stack to request the Python code retrive/store/delete secret key data.  The
actual keys and values are opaque to Python and stack-specific.

Only NimBLE is implemented (pending moving btstack to sync events).  The
secret store is designed to be compatible with BlueKitchen's TLV store API.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:41:41 +11:00
Jim Mussared 801e8ffacf extmod/modbluetooth: Add gap_pair(conn_handle) func to intiate pairing.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:41:26 +11:00
Jim Mussared a1fcf30121 extmod/modbluetooth: Allow configuration of pairing/bonding parameters.
This allows setting the security and MITM-protection requirements.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:40:49 +11:00
Andrew Leech 05fef8c6a4 extmod/modbluetooth: Add _IRQ_ENCRYPTION_UPDATE event.
This allows the application to be notified if any of encrypted,
authenticated and bonded state change, as well as the encryption key size.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:40:15 +11:00
Jim Mussared ac89267fef extmod/modbluetooth: Add compile-config flag to enable pairing/bonding.
Enable it on STM32/Unix NimBLE only (pairing/bonding requires synchronous
events and full bindings).

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:39:41 +11:00
Jim Mussared 60830bcba4 extmod/modbluetooth: Allow user-specified reason in read request IRQ.
Instead of returning None/bool from the IRQ, return None/int (where a zero
value means success).  This mirrors how the L2CAP_ACCEPT return value
works.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:37:55 +11:00
Andrew Leech 1697ff335d extmod/modbluetooth: Allow setting char/desc enc/auth options.
This widens the characteristic/descriptor flags to 16-bit, to allow setting
encryption/authentication requirements.

Sets the required flags for NimBLE and btstack implementations.

The BLE.FLAG_* constants will eventually be deprecated in favour of copy
and paste Python constants (like the IRQs).

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:36:50 +11:00
Andrew Leech c70665fb0b extmod/modbluetooth: Add _IRQ_CONNECTION_UPDATE event.
This allows the application to be notified of changes to the connection
interval, connection latency and supervision timeout.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:35:39 +11:00
Jim Mussared f2a9a0ac41 extmod/nimble: Fail read if the characteristic is too big.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-12-02 14:35:17 +11:00
Damien George b505971069 extmod/uasyncio: Fix cancellation handling of wait_for.
This commit switches the roles of the helper task from a cancellation task
to a runner task, to get the correct semantics for cancellation of
wait_for.

Some uasyncio tests are now disabled for the native emitter due to issues
with native code generation of generators and yield-from.

Fixes #5797.

Signed-off-by: Damien George <damien@micropython.org>
2020-12-02 12:31:37 +11:00
Damien George 309dfe39e0 extmod/uasyncio: Add Task.done() method.
This is added because task.coro==None is no longer the way to detect if a
task is finished.  Providing a (CPython compatible) function for this
allows the implementation to be abstracted away.

Signed-off-by: Damien George <damien@micropython.org>
2020-12-02 12:07:06 +11:00
Damien George ca40eb0fda extmod/uasyncio: Delay calling Loop.call_exception_handler by 1 loop.
When a tasks raises an exception which is uncaught, and no other task
await's on that task, then an error message is printed (or a user function
called) via a call to Loop.call_exception_handler.  In CPython this call is
made when the Task object is freed (eg via reference counting) because it's
at that point that it is known that the exception that was raised will
never be handled.

MicroPython does not have reference counting and the current behaviour is
to deal with uncaught exceptions as early as possible, ie as soon as they
terminate the task.  But this can be undesirable because in certain cases
a task can start and raise an exception immediately (before any await is
executed in that task's coro) and before any other task gets a chance to
await on it to catch the exception.

This commit changes the behaviour so that tasks which end due to an
uncaught exception are scheduled one more time for execution, and if they
are not await'ed on by the next scheduling loop, then the exception handler
is called (eg the exception is printed out).

Signed-off-by: Damien George <damien@micropython.org>
2020-12-02 12:07:06 +11:00
Jeff Epler 22af5a1fe9
Merge pull request #3728 from tannewt/speedup_json
Speed up JSON parsing with readinto
2020-11-24 09:42:18 -06:00
Jim Mussared 5a7027915c extmod/nimble/modbluetooth_nimble: Fix build when l2cap unavailable.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-24 11:57:29 +11:00
Scott Shawcroft f8dcb25170
Merge pull request #3694 from jepler/update-ulab2
ulab: Update to release tag 1.1.0
2020-11-23 15:17:46 -08:00
Scott Shawcroft 368fa88312
Ignore size parameter 2020-11-23 12:14:58 -08:00
Jeff Epler 9d8be648ee ulab: Update to release tag 1.1.0
Disable certain classes of diagnostic when building ulab.  We should
submit patches upstream to (A) fix these errors and (B) upgrade their
CI so that the problems are caught before we want to integrate with
CircuitPython, but not right now.
2020-11-23 10:23:50 -06:00
Jim Mussared 0e8af2b370 extmod/modbluetooth: Add API for L2CAP channels.
Also known as L2CAP "connection oriented channels". This provides a
socket-like data transfer mechanism for BLE.

Currently only implemented for NimBLE on STM32 / Unix.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-24 01:07:17 +11:00
Damien George 64180f0742 extmod/machine_i2c: Add init protocol method for generic I2C bindings.
Hardware I2C implementations must provide a .init() protocol method if they
want to support reconfiguration.  Otherwise the default is that i2c.init()
raises an OSError (currently the case for all ports).

mp_machine_soft_i2c_locals_dict is renamed to mp_machine_i2c_locals_dict to
match the generic SPI bindings.

Fixes issue #6623 (where calling .init() on a HW I2C would crash).

Signed-off-by: Damien George <damien@micropython.org>
2020-11-23 19:45:04 +11:00
Scott Shawcroft 123210a989
Speed up JSON parsing with readinto
Get a chunk of data from readinto instead of a single byte. This
speeds up the parsing by reducing the number of function calls.

Fixes #3703
2020-11-19 17:13:24 -08:00
Jeff Epler c06fc8e02d Introduce, use mp_raise_arg1
This raises an exception with a given object value.  Saves a bit of
code size.
2020-11-19 16:15:06 -06:00
Jeff Epler d5f6748d1b Use mp_raise instead of nlr_raise(new_exception) where possible
This saves a bit of code space
2020-11-19 16:13:01 -06:00
Jim Mussared 61d1e4b01b extmod/nimble: Make stm32 and unix NimBLE ports use synchronous events.
This changes stm32 from using PENDSV to run NimBLE to use the MicroPython
scheduler instead.  This allows Python BLE callbacks to be invoked directly
(and therefore synchronously) rather than via the ringbuffer.

The NimBLE UART HCI and event processing now happens in a scheduled task
every 128ms.  When RX IRQ idle events arrive, it will also schedule this
task to improve latency.

There is a similar change for the unix port where the background thread now
queues the scheduled task.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-13 17:19:05 +11:00
Jim Mussared 81e92d3d6e extmod/modbluetooth: Re-instate optional no-ringbuf modbluetooth.
This requires that the event handlers are called from non-interrupt context
(i.e. the MicroPython scheduler).

This will allow the BLE stack (e.g. NimBLE) to run from the scheduler
rather than an IRQ like PENDSV, and therefore be able to invoke Python
callbacks directly/synchronously.  This allows writing Python BLE handlers
for events that require immediate response such as _IRQ_READ_REQUEST (which
was previous a hard IRQ) and future events relating to pairing/bonding.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-13 17:19:05 +11:00
Jim Mussared 6d9fdff8d0 extmod/nimble: Poll startup directly rather than using NimBLE sem.
Using a semaphore (the previous approach) will only run the UART, whereas
for startup we need to also run the event queue.

This change makes it run the full scheduler hook.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-13 17:19:05 +11:00
Jim Mussared c398e46b29 extmod/modbluetooth: Combine gattc-data-available callbacks into one.
Instead of having the stack indicate a "start", "data"..., "end", pass
through the data in one callback as an array of chunks of data.

This is because the upcoming non-ringbuffer modbluetooth implementation
cannot buffer the data in the ringbuffer and requires instead a single
callback with all the data, to pass to the Python callback.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-13 17:19:05 +11:00
Jim Mussared 3d890e7ab4 extmod/modbluetooth: Make UUID type accessible outside modbluetooth.c.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-13 17:19:05 +11:00
Arrowana 922f81dfd1 extmod/machine_mem: Only allow integers in machine.memX subscript.
Prior to this change machine.mem32['foo'] (or using any other non-integer
subscript) could result in a fault due to 'foo' being interpreted as an
integer.  And when writing code it's hard to tell if the fault is due to a
bad subscript type, or an integer subscript that specifies an invalid
memory address.

The type of the object used in the subscript is now tested to be an
integer by using mp_obj_get_int_truncated instead of
mp_obj_int_get_truncated.  The performance hit of this change is minimal,
and machine.memX objects are more for convenience than performance (there
are many other ways to read/write memory in a faster way),

Fixes issue #6588.
2020-11-13 11:13:37 +11:00
Damien George bdfb584b29 extmod/moductypes: Fix storing to (U)INT64 arrays on 32-bit archs.
Fixes issue #6583.

Signed-off-by: Damien George <damien@micropython.org>
2020-11-11 22:18:24 +11:00
Jim Mussared b7883ce74c extmod/nimble/nimble.mk: Add -Wno-old-style-declaration.
This is needed since -Wextra was added to the build in
bef412789e

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-05 10:14:19 +11:00
Jim Mussared 2ae3c890bd extmod/btstack/btstack.mk: Add -Wimplicit-fallthrough=0.
This is needed since -Wextra was added to the build in
bef412789e

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-05 10:13:12 +11:00
Damien George 6f34800884 extmod/modurandom: Support urandom.seed() without an argument.
If a port provides MICROPY_PY_URANDOM_SEED_INIT_FUNC as a source of
randomness then this will be used when urandom.seed() is called without
an argument (or with None as the argument) to seed the pRNG.

Other related changes in this commit:
- mod_urandom___init__ is changed to call seed() without arguments, instead
  of explicitly passing in the result of MICROPY_PY_URANDOM_SEED_INIT_FUNC.
- mod_urandom___init__ will only ever seed the pRNG once (before it could
  seed it again if imported by, eg, random and then urandom).
- The Yasmarang state is moved to the BSS for builds where the state is
  guaranteed to be initialised on import of the (u)random module.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-29 14:15:16 +11:00
Damien George 03a1f94ea1 extmod/vfs_lfs: Support mounting LFS filesystems in read-only mode.
Signed-off-by: Damien George <damien@micropython.org>
2020-10-29 11:43:52 +11:00
Jeff Epler 543316a8fb ulab: Update again
.. pull in various doc build fixes that prevented the previous commit from building.  This is still "0.54.5", the tag was updated in micropython-ulab (since no functional difference was introduced, only doc and CI differences, I imagine)
2020-10-25 13:26:55 -05:00
Jeff Epler 4cad5478e6 ulab: Incorporate bugfixes (update to tag 0.54.5)
In particular, this closes #3954.
2020-10-24 13:16:10 -05:00
Emil Renner Berthing ccd92335a1 py, extmod: Introduce and use MP_FALLTHROUGH macro.
Newer GCC versions are able to warn about switch cases that fall
through.  This is usually a sign of a forgotten break statement, but in
the few cases where a fall through is intended we annotate it with this
macro to avoid the warning.
2020-10-22 11:53:16 +02:00
Emil Renner Berthing dde3db21fc extmod: Disable -Wmissing-field-initializers for lfs2. 2020-10-22 11:47:36 +02:00
Emil Renner Berthing 9aa58cf8ba py, extmod: Add explicit initializers for default values.
When compiling with -Wextra which includes -Wmissing-field-initializers
GCC will warn that the defval field of mp_arg_val_t is not initialized.
This is just a warning as it is defined to be zero initialized, but since
it is a union it makes sense to be explicit about which member we're
going to use, so add the explicit initializers and get rid of the
warning.
2020-10-22 11:47:36 +02:00
Scott Shawcroft 09bc415751
Unify iMX flash config and add Metro M7 1011
This unifies the flash config to the settings used by the Boot ROM.
This makes the config unique per board which allows for changing
quad enable and status bit differences per flash device. It also
allows for timing differences due to the board layout.

This change also tweaks linker layout to leave more ram space for
the CircuitPython heap.
2020-10-07 15:23:47 -07:00
Damien George d4b61b0017 extmod/utime_mphal: Add generic utime.time_ns() function.
It requires mp_hal_time_ns() to be provided by a port.  This function
allows very accurate absolute timestamps.

Enabled on unix, windows, stm32, esp8266 and esp32.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-01 14:20:42 +10:00
Damien George 71f3ade770 ports: Support legacy soft I2C/SPI construction via id=-1 arg.
With a warning that this way of constructing software I2C/SPI is
deprecated.  The check and warning will be removed in a future release.

This should help existing code to migrate to the new SoftI2C/SoftSPI types.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-01 12:57:10 +10:00
Damien George 9e0533b9e1 extmod/machine_spi: Remove "id" arg in SoftSPI constructor.
The SoftSPI constructor is now used soley to create SoftSPI instances, it
can no longer delegate to create a hardware-based SPI instance.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-01 12:57:10 +10:00
Damien George aaed33896b extmod/machine_i2c: Remove "id" arg in SoftI2C constructor.
The SoftI2C constructor is now used soley to create SoftI2C instances, it
can no longer delegate to create a hardware-based I2C instance.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-01 12:57:10 +10:00
Damien George c35deb2625 extmod/machine_i2c: Rename type to SoftI2C and add custom print method.
Also rename machine_i2c_type to mp_machine_soft_i2c_type.  These changes
make it clear that it's a soft-I2C implementation, and match SoftSPI.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-01 12:57:10 +10:00
Andrew Leech 319437d4bd extmod/modure: Allow \\ in re.sub replacements. 2020-09-30 23:18:34 +10:00
Damien George 81f2162ca0 extmod/modbluetooth: Change module-owned bytes objects to memoryview.
A read-only memoryview object is a better representation of the data, which
is owned by the ubluetooth module and may change between calls to the
user's irq callback function.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-25 12:23:11 +10:00
Damien George 71adf506ce extmod/vfs: Fix lookup of entry in root dir so it fails correctly.
Prior to this commit, uos.chdir('/') followed by uos.stat('noexist') would
succeed that stat even though the entry did not exist (some other functions
like listdir would have similar issues).  This is because, if the current
directory was the root and the path was relative, mp_vfs_lookup_path would
return success for bad paths.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-23 16:23:35 +10:00
Damien George 8f20cdc353 all: Rename absolute time-based functions to include "epoch".
For time-based functions that work with absolute time there is the need for
an Epoch, to set the zero-point at which the absolute time starts counting.
Such functions include time.time() and filesystem stat return values.  And
different ports may use a different Epoch.

To make it clearer what functions use the Epoch (whatever it may be), and
make the ports more consistent with their use of the Epoch, this commit
renames all Epoch related functions to include the word "epoch" in their
name (and remove references to "2000").

Along with this rename, the following things have changed:

- mp_hal_time_ns() is now specified to return the number of nanoseconds
  since the Epoch, rather than since 1970 (but since this is an internal
  function it doesn't change anything for the user).

- littlefs timestamps on the esp8266 have been fixed (they were previously
  off by 30 years in nanoseconds).

Otherwise, there is no functional change made by this commit.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-18 17:20:34 +10:00
Jim Mussared 857e2c8fd5 extmod/modbluetooth: Implement MTU. 2020-09-18 12:51:21 +10:00
Jim Mussared 5be3bfcb7e extmod/modbluetooth: Print UUIDs correctly.
In particular, the printed string can now be re-evaluated to construct the
same UUID object.
2020-09-15 15:29:13 +10:00
Jim Mussared 6a6a5f9e15 extmod/modbluetooth: Make BLE.irq() method positional only.
Simplifcation now that the trigger arg has been removed.
2020-09-15 15:28:45 +10:00
Jim Mussared 504522bd02 extmod/modbluetooth: Fix handling of optional data/uuid args.
For the following 3 functions, previously the code relied on whether the
arg was passed at all, to make it optional.  Now it allows them to be
explicitly `None` to indicate they are not used:

- gatts_notify(..., [data])
- gattc_discover_services(..., [uuid])
- gattc_discover_characteristics(..., [uuid])

Also ensure that the uuid arguments are actually instances of the uuid
type, and fix handling of the 5th arg in gattc_discover_characteristics().
2020-09-15 15:23:51 +10:00
Jeff Epler 12d826d941 Add FALLTHROUGH comments as needed
I investigated these cases and confirmed that the fallthrough behavior
was intentional.
2020-09-12 15:11:29 -05:00
Damien George 373b400632 extmod/modussl_axtls: Reduce size of code that makes exception.
Change in code size (for ports that use axtls) is:

   unix x64:  -152 -0.030% [incl -160(data)]
unix nanbox:  -112 -0.025% [incl -96(data)]
    esp8266:   -64 -0.009% GENERIC

Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 10:22:19 +10:00
Jim Mussared 126f972c34 extmod/nimble: Add timeout for HCI sync on startup.
This allows `ble.active(1)` to fail correctly if the HCI controller is
unavailable.

It also avoids an infine loop in the NimBLE event handler where NimBLE
doesn't correctly detect that the HCI controller is unavailable and keeps
trying to reset.

Furthermore, it fixes an issue where GATT service registrations were left
allocated, which led to a bad realloc if the stack was activated multiple
times.
2020-09-08 12:53:24 +10:00
Jim Mussared 99a29ec705 extmod/btstack: Detect HCI UART init failure. 2020-09-08 12:53:24 +10:00
Andrew Leech 8b00aeab8f extmod/btstack: Add btstack support for _IRQ_GATTS_READ_REQUEST. 2020-09-08 12:53:24 +10:00
Jim Mussared 52a2ce45de extmod/modbluetooth: Allow using mp_hal_get_mac as a static address.
Generally a controller should either have its own public address hardcoded,
or loaded by the driver (e.g. cywbt43).

However, for a controller that has no public address where you still want a
long-term stable address, this allows you to use a static address generated
by the port.  Typically on STM32 this will be an LAA, but a board might
override this.
2020-09-08 12:53:24 +10:00
Jim Mussared c4af714d58 extmod/modbluetooth: Implement configuration of address modes.
Changes `BLE.config('mac')` to return a tuple (addr_mode, addr).

Adds `BLE.config(addr_mode=...)` to set the addressing mode.
2020-09-08 12:53:24 +10:00
Jim Mussared 1b1b22905e unix: Implement BLE H4 HCI UART for btstack/nimble.
This commit adds support for using Bluetooth on the unix port via a H4
serial interface (distinct from a USB dongle), with both BTstack and NimBLE
Bluetooth stacks.

Note that MICROPY_PY_BLUETOOTH is now disabled for the coverage variant.
Prior to this commit Bluetooth was anyway not being built on Travis because
libusb was not detected.  But now that bluetooth works in H4 mode it will
be built, and will lead to a large decrease in coverage because Bluetooth
tests cannot be run on Travis.
2020-09-08 12:53:24 +10:00
Jim Mussared aa18ab7db2 extmod/nimble: Implement NimBLE mutex. 2020-09-08 11:41:31 +10:00
Jim Mussared f3f31ac959 extmod/nimble: Make nimble_malloc work with allocated size. 2020-09-08 11:41:31 +10:00
Jim Mussared 5b08676d6a extmod/nimble: Set struct alignment correctly on 64-bit arch. 2020-09-08 11:41:31 +10:00
Jim Mussared ed14435a8e extmod/modbluetooth: Refactor stack/hci/driver/port bindings.
Previously the interaction between the different layers of the Bluetooth
stack was different on each port and each stack.  This commit defines
common interfaces between them and implements them for cyw43, btstack,
nimble, stm32, unix.
2020-09-08 11:41:31 +10:00
Jim Mussared e46aac24ba extmod/modbluetooth: Rename logging macro to be just DEBUG_printf.
And prefix the debug message with "btstack:" or "nimble:", depending on the
context.  Also use correct format specifier for %zu.
2020-09-08 10:48:23 +10:00
stijn 40ad8f1666 all: Rename "sys" module to "usys".
This is consistent with the other 'micro' modules and allows implementing
additional features in Python via e.g. micropython-lib's sys.

Note this is a breaking change (not backwards compatible) for ports which
do not enable weak links, as "import sys" must now be replaced with
"import usys".
2020-09-04 00:10:24 +10:00
Damien George 2a72e90ab8 extmod/vfs: Add option to use 1970 as Epoch.
By setting MICROPY_EPOCH_IS_1970 a port can opt to use 1970/1/1 as the
Epoch for timestamps returned by stat().  And this setting is enabled on
the unix and windows ports because that's what they use.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-01 12:36:28 +10:00
Damien George c70e599659 extmod/vfs: Support larger integer range in VFS stat time fields.
On ports like unix where the Epoch is 1970/1/1 and atime/mtime/ctime are in
seconds since the Epoch, this value will overflow a small-int on 32-bit
systems.  So far this is only an issue on 32-bit unix builds that use the
VFS layer (eg dev and coverage unix variants) but the fix (using
mp_obj_new_int_from_uint instead of MP_OBJ_NEW_SMALL_INT) is there for all
ports so as to not complicate the code, and because they will need the
range one day.

Also apply a similar fix to other fields in VfsPosix.stat because they may
also be large.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-01 12:36:28 +10:00
Damien George d1995e50eb extmod/modlwip: Fix error return for TCP recv when not connected.
This commit fixes the cases when a TCP socket is in STATE_NEW,
STATE_LISTENING or STATE_CONNECTING and recv() is called on it.  It now
raises ENOTCONN instead of a random error code due to it previously
indexing beyond the start of error_lookup_table[].

Signed-off-by: Damien George <damien@micropython.org>
2020-08-30 13:20:51 +10:00
Damien George 06659077a8 all: Update Python code to conform to latest black formatting.
Updating to Black v20.8b1 there are two changes that affect the code in
this repository:

- If there is a trailing comma in a list (eg [], () or function call) then
  that list is now written out with one line per element.  So remove such
  trailing commas where the list should stay on one line.

- Spaces at the start of """ doc strings are removed.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-29 15:18:01 +10:00
Michael Buesch cef678b2db extmod/machine_i2c: Fix buffer overrun if 'addrsize' is bigger than 32.
The memory operation functions read_mem() and write_mem() create a
temporary buffer on the local C stack for the address bytes with the size
of 4 bytes.  This buffer is filled in a loop from the user supplied address
and address length.  If the user supplied 'addrsize' is bigger than 32, the
local buffer is overrun.

Fix this by raising an exception for invalid 'addrsize' values.

Signed-off-by: Michael Buesch <m@bues.ch>
2020-08-27 12:39:11 +10:00
Andrew Leech a80a146858 extmod/bluetooth: Support active scanning in BLE.gap_scan().
This adds an additional optional parameter to gap_scan() to select active
scanning, where scan responses are returned as well as normal scan results.
This parameter is False by default which retains the existing behaviour.
2020-08-26 15:00:11 +10:00
Jim Mussared 0bc2c1c105 extmod/modbluetooth: Fix race between READ_REQUEST and other IRQs.
The READ_REQUEST callback is handled as a hard interrupt (because the BLE
stack needs an immediate response from it so it can continue) and so calls
to Python require extra protection:

- the caller-owned tuple passed into the callback must be separate from the
  tuple used by other callback events (which are soft interrupts);

- the GC and scheduler must be locked during callback execution.
2020-08-26 14:57:36 +10:00
Jim Mussared 3d9a7ed02f extmod/btstack: Implement GAP scan duration_ms parameter.
This commit makes scanning work when duration_ms is set to zero.  Prior to
this it would not work with duration_ms set to zero.
2020-08-26 14:55:52 +10:00
Damien George 2acc087880 extmod/vfs_lfs: Add mtime support to littlefs files.
This commit adds support for modification time of files on littlefs v2
filesystems, using file attributes.  For some background see issue #6114.

Features/properties of this implementation:
- Only supported on littlefs2 (not littlefs1).
- Uses littlefs2's general file attributes to store the timestamp.
- The timestamp is 64-bits and stores nanoseconds since 1970/1/1 (if the
  range to the year 2554 is not enough then additional bits can be added to
  this timestamp by adding another file attribute).
- mtime is enabled by default but can be disabled in the constructor, eg:
  uos.mount(uos.VfsLfs2(bdev, mtime=False), '/flash')
- It's fully backwards compatible, existing littlefs2 filesystems will work
  without reformatting and timestamps will be added transparently to
  existing files (once they are opened for writing).
- Files without timestamps will open correctly, and stat will just return 0
  for their timestamp.
- mtime can be disabled or enabled each mount time and timestamps will only
  be updated if mtime is enabled (otherwise they will be untouched).

Signed-off-by: Damien George <damien@micropython.org>
2020-08-25 17:35:19 +10:00
Damien George 55c76eaac1 extmod/uasyncio: Truncate negative sleeps to 0.
Otherwise a task that continuously awaits on a large negative sleep can
monopolise the scheduler (because its wake time is always less than
everything else in the pairing heap).

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 12:17:06 +10:00
Damien George 60f5b941e0 extmod/vfs_reader: Fix mp_reader_new_file to open file in "rb" mode.
mp_reader_new_file() is used to read in files for importing, either .py or
.mpy files, for the lexer and persistent code loader respectively.  In both
cases the file should be opened in raw bytes mode: the lexer handles
unicode characters itself, and .mpy files contain 8-bit bytes by nature.

Before this commit importing was working correctly because, although the
file was opened in text mode, all native filesystem implementations (POSIX,
FAT, LFS) would access the file in raw bytes mode via mp_stream_rw()
calling mp_stream_p_t.read().  So it was only an issue for non-native
filesystems, such as those implemented in Python.  For Python-based
filesystem implementations, a call to mp_stream_rw() would go via IOBase
and then to readinto() at the Python level, and readinto() is only defined
on files opened in raw bytes mode.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-12 23:40:50 +10:00
Jeff Epler c849b781c0 Combine 'index out of range' messages 2020-08-04 14:45:45 -05:00
Jeff Epler 89797fd3f9 various: Use mp_obj_get_type_qstr more widely
This removes runtime allocations of the cstring version of the qstring.

It is not a size improvement
2020-08-04 14:45:45 -05:00
Jeff Epler 9b8df7f635 Upgrade ulab
This version
 * moves source files to reflect module structure
 * adds inline documentation suitable for extract_pyi
 * incompatibly moves spectrogram to fft
 * incompatibly removes "extras"

There are some remaining markup errors in the specific revision of
extmod/ulab but they do not prevent the doc building process from
completing.
2020-07-28 16:57:48 -05:00
Damien George 441460d81f extmod/uasyncio: Add StreamReader.readexactly(n) method.
It raises on EOFError instead of an IncompleteReadError (which is what
CPython does).  But the latter is derived from EOFError so code compatible
with MicroPython and CPython can be written by catching EOFError (eg see
included test).

Fixes issue #6156.

Signed-off-by: Damien George <damien@micropython.org>
2020-07-25 23:10:05 +10:00
Thorsten von Eicken 5264478007 extmod/modussl_mbedtls: Integrate shorter error strings.
The stm32 and esp32 ports now use shorter error strings for mbedtls errors.
Also, MBEDTLS_ERROR_C is enabled on stm32 by default to get these strings.
2020-07-21 00:31:05 +10:00
Thorsten von Eicken 9aa214077e extmod/modussl: Improve exception error messages.
This commit adds human readable error messages when mbedtls or axtls raise
an exception.  Currently often just an EIO error is raised so the user is
lost and can't tell whether it's a cert error, buffer overrun, connecting
to a non-ssl port, etc.  The axtls and mbedtls error raising in the ussl
module is modified to raise:

    OSError(-err_num, "error string")

For axtls a small error table of strings is added and used for the second
argument of the OSErrer.  For mbedtls the code uses mbedtls' built-in
strerror function, and if there is an out of memory condition it just
produces OSError(-err_num).  Producing the error string for mbedtls is
conditional on them being included in the mbedtls build, via
MBEDTLS_ERROR_C.
2020-07-20 23:41:45 +10:00
Jim Mussared 9d823a5d9a extmod/modbluetooth: Add event for "indicate acknowledgement".
This commit adds the IRQ_GATTS_INDICATE_DONE BLE event which will be raised
with the status of gatts_indicate (unlike notify, indications require
acknowledgement).

An example of its use is added to ble_temperature.py, and to the multitests
in ble_characteristic.py.

Implemented for btstack and nimble bindings, tested in both directions
between unix/btstack and pybd/nimble.
2020-07-20 23:26:41 +10:00
Jim Mussared 3c7ca2004c extmod/modbluetooth: Fix so it builds in peripheral-only mode. 2020-07-20 23:25:56 +10:00
Jim Mussared 43ceadac55 extmod/modbluetooth: Ignore unused self_in in ble_gatts_indicate. 2020-07-20 23:25:22 +10:00
Jim Mussared b7698841b2 docs/library: Add gatts_indicate() doc to ubluetooth.rst.
Also clarify behavior of `gatts_notify` and add some TODOs about adding an
event for indication acknowledgement.
2020-07-18 14:34:44 +10:00
Jim Mussared e152d0c197 extmod/btstack: Schedule notify/indicate/write ops for bg completion.
The goal of this commit is to allow using ble.gatts_notify() at any time,
even if the stack is not ready to send the notification right now.  It also
addresses the same issue for ble.gatts_indicate() and ble.gattc_write()
(without response).  In addition this commit fixes the case where the
buffer passed to write-with-response wasn't copied, meaning it could be
modified by the caller, affecting the in-progress write.

The changes are:

- gatts_notify/indicate will now run in the background if the ACL buffer is
  currently full, meaning that notify/indicate can be called at any time.

- gattc_write(mode=0) (no response) will now allow for one outstanding
  write.

- gattc_write(mode=1) (with response) will now copy the buffer so that it
  can't be modified by the caller while the write is in progress.

All four paths also now track the buffer while the operation is in
progress, which prevents the GC free'ing the buffer while it's still
needed.
2020-07-18 14:23:47 +10:00
Dan Halbert 88d8956378
Merge pull request #3143 from tannewt/improve_json
Add support to json.load for any object with readinto
2020-07-16 14:33:33 -04:00
Scott Shawcroft 372bcf8a95
Fix stream version and add basic readinto test 2020-07-10 17:33:17 -07:00
Scott Shawcroft 734661e79c
Add support to json.load for any object with readinto
This way we don't need to load the whole string version of the
json into memory.
2020-07-10 16:42:32 -07:00