Commit Graph

1080 Commits

Author SHA1 Message Date
Scott Shawcroft
09bc415751
Unify iMX flash config and add Metro M7 1011
This unifies the flash config to the settings used by the Boot ROM.
This makes the config unique per board which allows for changing
quad enable and status bit differences per flash device. It also
allows for timing differences due to the board layout.

This change also tweaks linker layout to leave more ram space for
the CircuitPython heap.
2020-10-07 15:23:47 -07:00
Damien George
d4b61b0017 extmod/utime_mphal: Add generic utime.time_ns() function.
It requires mp_hal_time_ns() to be provided by a port.  This function
allows very accurate absolute timestamps.

Enabled on unix, windows, stm32, esp8266 and esp32.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-01 14:20:42 +10:00
Damien George
71f3ade770 ports: Support legacy soft I2C/SPI construction via id=-1 arg.
With a warning that this way of constructing software I2C/SPI is
deprecated.  The check and warning will be removed in a future release.

This should help existing code to migrate to the new SoftI2C/SoftSPI types.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-01 12:57:10 +10:00
Damien George
9e0533b9e1 extmod/machine_spi: Remove "id" arg in SoftSPI constructor.
The SoftSPI constructor is now used soley to create SoftSPI instances, it
can no longer delegate to create a hardware-based SPI instance.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-01 12:57:10 +10:00
Damien George
aaed33896b extmod/machine_i2c: Remove "id" arg in SoftI2C constructor.
The SoftI2C constructor is now used soley to create SoftI2C instances, it
can no longer delegate to create a hardware-based I2C instance.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-01 12:57:10 +10:00
Damien George
c35deb2625 extmod/machine_i2c: Rename type to SoftI2C and add custom print method.
Also rename machine_i2c_type to mp_machine_soft_i2c_type.  These changes
make it clear that it's a soft-I2C implementation, and match SoftSPI.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-01 12:57:10 +10:00
Andrew Leech
319437d4bd extmod/modure: Allow \\ in re.sub replacements. 2020-09-30 23:18:34 +10:00
Damien George
81f2162ca0 extmod/modbluetooth: Change module-owned bytes objects to memoryview.
A read-only memoryview object is a better representation of the data, which
is owned by the ubluetooth module and may change between calls to the
user's irq callback function.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-25 12:23:11 +10:00
Damien George
71adf506ce extmod/vfs: Fix lookup of entry in root dir so it fails correctly.
Prior to this commit, uos.chdir('/') followed by uos.stat('noexist') would
succeed that stat even though the entry did not exist (some other functions
like listdir would have similar issues).  This is because, if the current
directory was the root and the path was relative, mp_vfs_lookup_path would
return success for bad paths.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-23 16:23:35 +10:00
Damien George
8f20cdc353 all: Rename absolute time-based functions to include "epoch".
For time-based functions that work with absolute time there is the need for
an Epoch, to set the zero-point at which the absolute time starts counting.
Such functions include time.time() and filesystem stat return values.  And
different ports may use a different Epoch.

To make it clearer what functions use the Epoch (whatever it may be), and
make the ports more consistent with their use of the Epoch, this commit
renames all Epoch related functions to include the word "epoch" in their
name (and remove references to "2000").

Along with this rename, the following things have changed:

- mp_hal_time_ns() is now specified to return the number of nanoseconds
  since the Epoch, rather than since 1970 (but since this is an internal
  function it doesn't change anything for the user).

- littlefs timestamps on the esp8266 have been fixed (they were previously
  off by 30 years in nanoseconds).

Otherwise, there is no functional change made by this commit.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-18 17:20:34 +10:00
Jim Mussared
857e2c8fd5 extmod/modbluetooth: Implement MTU. 2020-09-18 12:51:21 +10:00
Jim Mussared
5be3bfcb7e extmod/modbluetooth: Print UUIDs correctly.
In particular, the printed string can now be re-evaluated to construct the
same UUID object.
2020-09-15 15:29:13 +10:00
Jim Mussared
6a6a5f9e15 extmod/modbluetooth: Make BLE.irq() method positional only.
Simplifcation now that the trigger arg has been removed.
2020-09-15 15:28:45 +10:00
Jim Mussared
504522bd02 extmod/modbluetooth: Fix handling of optional data/uuid args.
For the following 3 functions, previously the code relied on whether the
arg was passed at all, to make it optional.  Now it allows them to be
explicitly `None` to indicate they are not used:

- gatts_notify(..., [data])
- gattc_discover_services(..., [uuid])
- gattc_discover_characteristics(..., [uuid])

Also ensure that the uuid arguments are actually instances of the uuid
type, and fix handling of the 5th arg in gattc_discover_characteristics().
2020-09-15 15:23:51 +10:00
Jeff Epler
12d826d941 Add FALLTHROUGH comments as needed
I investigated these cases and confirmed that the fallthrough behavior
was intentional.
2020-09-12 15:11:29 -05:00
Damien George
373b400632 extmod/modussl_axtls: Reduce size of code that makes exception.
Change in code size (for ports that use axtls) is:

   unix x64:  -152 -0.030% [incl -160(data)]
unix nanbox:  -112 -0.025% [incl -96(data)]
    esp8266:   -64 -0.009% GENERIC

Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 10:22:19 +10:00
Jim Mussared
126f972c34 extmod/nimble: Add timeout for HCI sync on startup.
This allows `ble.active(1)` to fail correctly if the HCI controller is
unavailable.

It also avoids an infine loop in the NimBLE event handler where NimBLE
doesn't correctly detect that the HCI controller is unavailable and keeps
trying to reset.

Furthermore, it fixes an issue where GATT service registrations were left
allocated, which led to a bad realloc if the stack was activated multiple
times.
2020-09-08 12:53:24 +10:00
Jim Mussared
99a29ec705 extmod/btstack: Detect HCI UART init failure. 2020-09-08 12:53:24 +10:00
Andrew Leech
8b00aeab8f extmod/btstack: Add btstack support for _IRQ_GATTS_READ_REQUEST. 2020-09-08 12:53:24 +10:00
Jim Mussared
52a2ce45de extmod/modbluetooth: Allow using mp_hal_get_mac as a static address.
Generally a controller should either have its own public address hardcoded,
or loaded by the driver (e.g. cywbt43).

However, for a controller that has no public address where you still want a
long-term stable address, this allows you to use a static address generated
by the port.  Typically on STM32 this will be an LAA, but a board might
override this.
2020-09-08 12:53:24 +10:00
Jim Mussared
c4af714d58 extmod/modbluetooth: Implement configuration of address modes.
Changes `BLE.config('mac')` to return a tuple (addr_mode, addr).

Adds `BLE.config(addr_mode=...)` to set the addressing mode.
2020-09-08 12:53:24 +10:00
Jim Mussared
1b1b22905e unix: Implement BLE H4 HCI UART for btstack/nimble.
This commit adds support for using Bluetooth on the unix port via a H4
serial interface (distinct from a USB dongle), with both BTstack and NimBLE
Bluetooth stacks.

Note that MICROPY_PY_BLUETOOTH is now disabled for the coverage variant.
Prior to this commit Bluetooth was anyway not being built on Travis because
libusb was not detected.  But now that bluetooth works in H4 mode it will
be built, and will lead to a large decrease in coverage because Bluetooth
tests cannot be run on Travis.
2020-09-08 12:53:24 +10:00
Jim Mussared
aa18ab7db2 extmod/nimble: Implement NimBLE mutex. 2020-09-08 11:41:31 +10:00
Jim Mussared
f3f31ac959 extmod/nimble: Make nimble_malloc work with allocated size. 2020-09-08 11:41:31 +10:00
Jim Mussared
5b08676d6a extmod/nimble: Set struct alignment correctly on 64-bit arch. 2020-09-08 11:41:31 +10:00
Jim Mussared
ed14435a8e extmod/modbluetooth: Refactor stack/hci/driver/port bindings.
Previously the interaction between the different layers of the Bluetooth
stack was different on each port and each stack.  This commit defines
common interfaces between them and implements them for cyw43, btstack,
nimble, stm32, unix.
2020-09-08 11:41:31 +10:00
Jim Mussared
e46aac24ba extmod/modbluetooth: Rename logging macro to be just DEBUG_printf.
And prefix the debug message with "btstack:" or "nimble:", depending on the
context.  Also use correct format specifier for %zu.
2020-09-08 10:48:23 +10:00
stijn
40ad8f1666 all: Rename "sys" module to "usys".
This is consistent with the other 'micro' modules and allows implementing
additional features in Python via e.g. micropython-lib's sys.

Note this is a breaking change (not backwards compatible) for ports which
do not enable weak links, as "import sys" must now be replaced with
"import usys".
2020-09-04 00:10:24 +10:00
Damien George
2a72e90ab8 extmod/vfs: Add option to use 1970 as Epoch.
By setting MICROPY_EPOCH_IS_1970 a port can opt to use 1970/1/1 as the
Epoch for timestamps returned by stat().  And this setting is enabled on
the unix and windows ports because that's what they use.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-01 12:36:28 +10:00
Damien George
c70e599659 extmod/vfs: Support larger integer range in VFS stat time fields.
On ports like unix where the Epoch is 1970/1/1 and atime/mtime/ctime are in
seconds since the Epoch, this value will overflow a small-int on 32-bit
systems.  So far this is only an issue on 32-bit unix builds that use the
VFS layer (eg dev and coverage unix variants) but the fix (using
mp_obj_new_int_from_uint instead of MP_OBJ_NEW_SMALL_INT) is there for all
ports so as to not complicate the code, and because they will need the
range one day.

Also apply a similar fix to other fields in VfsPosix.stat because they may
also be large.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-01 12:36:28 +10:00
Damien George
d1995e50eb extmod/modlwip: Fix error return for TCP recv when not connected.
This commit fixes the cases when a TCP socket is in STATE_NEW,
STATE_LISTENING or STATE_CONNECTING and recv() is called on it.  It now
raises ENOTCONN instead of a random error code due to it previously
indexing beyond the start of error_lookup_table[].

Signed-off-by: Damien George <damien@micropython.org>
2020-08-30 13:20:51 +10:00
Damien George
06659077a8 all: Update Python code to conform to latest black formatting.
Updating to Black v20.8b1 there are two changes that affect the code in
this repository:

- If there is a trailing comma in a list (eg [], () or function call) then
  that list is now written out with one line per element.  So remove such
  trailing commas where the list should stay on one line.

- Spaces at the start of """ doc strings are removed.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-29 15:18:01 +10:00
Michael Buesch
cef678b2db extmod/machine_i2c: Fix buffer overrun if 'addrsize' is bigger than 32.
The memory operation functions read_mem() and write_mem() create a
temporary buffer on the local C stack for the address bytes with the size
of 4 bytes.  This buffer is filled in a loop from the user supplied address
and address length.  If the user supplied 'addrsize' is bigger than 32, the
local buffer is overrun.

Fix this by raising an exception for invalid 'addrsize' values.

Signed-off-by: Michael Buesch <m@bues.ch>
2020-08-27 12:39:11 +10:00
Andrew Leech
a80a146858 extmod/bluetooth: Support active scanning in BLE.gap_scan().
This adds an additional optional parameter to gap_scan() to select active
scanning, where scan responses are returned as well as normal scan results.
This parameter is False by default which retains the existing behaviour.
2020-08-26 15:00:11 +10:00
Jim Mussared
0bc2c1c105 extmod/modbluetooth: Fix race between READ_REQUEST and other IRQs.
The READ_REQUEST callback is handled as a hard interrupt (because the BLE
stack needs an immediate response from it so it can continue) and so calls
to Python require extra protection:

- the caller-owned tuple passed into the callback must be separate from the
  tuple used by other callback events (which are soft interrupts);

- the GC and scheduler must be locked during callback execution.
2020-08-26 14:57:36 +10:00
Jim Mussared
3d9a7ed02f extmod/btstack: Implement GAP scan duration_ms parameter.
This commit makes scanning work when duration_ms is set to zero.  Prior to
this it would not work with duration_ms set to zero.
2020-08-26 14:55:52 +10:00
Damien George
2acc087880 extmod/vfs_lfs: Add mtime support to littlefs files.
This commit adds support for modification time of files on littlefs v2
filesystems, using file attributes.  For some background see issue #6114.

Features/properties of this implementation:
- Only supported on littlefs2 (not littlefs1).
- Uses littlefs2's general file attributes to store the timestamp.
- The timestamp is 64-bits and stores nanoseconds since 1970/1/1 (if the
  range to the year 2554 is not enough then additional bits can be added to
  this timestamp by adding another file attribute).
- mtime is enabled by default but can be disabled in the constructor, eg:
  uos.mount(uos.VfsLfs2(bdev, mtime=False), '/flash')
- It's fully backwards compatible, existing littlefs2 filesystems will work
  without reformatting and timestamps will be added transparently to
  existing files (once they are opened for writing).
- Files without timestamps will open correctly, and stat will just return 0
  for their timestamp.
- mtime can be disabled or enabled each mount time and timestamps will only
  be updated if mtime is enabled (otherwise they will be untouched).

Signed-off-by: Damien George <damien@micropython.org>
2020-08-25 17:35:19 +10:00
Damien George
55c76eaac1 extmod/uasyncio: Truncate negative sleeps to 0.
Otherwise a task that continuously awaits on a large negative sleep can
monopolise the scheduler (because its wake time is always less than
everything else in the pairing heap).

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 12:17:06 +10:00
Damien George
60f5b941e0 extmod/vfs_reader: Fix mp_reader_new_file to open file in "rb" mode.
mp_reader_new_file() is used to read in files for importing, either .py or
.mpy files, for the lexer and persistent code loader respectively.  In both
cases the file should be opened in raw bytes mode: the lexer handles
unicode characters itself, and .mpy files contain 8-bit bytes by nature.

Before this commit importing was working correctly because, although the
file was opened in text mode, all native filesystem implementations (POSIX,
FAT, LFS) would access the file in raw bytes mode via mp_stream_rw()
calling mp_stream_p_t.read().  So it was only an issue for non-native
filesystems, such as those implemented in Python.  For Python-based
filesystem implementations, a call to mp_stream_rw() would go via IOBase
and then to readinto() at the Python level, and readinto() is only defined
on files opened in raw bytes mode.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-12 23:40:50 +10:00
Jeff Epler
c849b781c0 Combine 'index out of range' messages 2020-08-04 14:45:45 -05:00
Jeff Epler
89797fd3f9 various: Use mp_obj_get_type_qstr more widely
This removes runtime allocations of the cstring version of the qstring.

It is not a size improvement
2020-08-04 14:45:45 -05:00
Jeff Epler
9b8df7f635 Upgrade ulab
This version
 * moves source files to reflect module structure
 * adds inline documentation suitable for extract_pyi
 * incompatibly moves spectrogram to fft
 * incompatibly removes "extras"

There are some remaining markup errors in the specific revision of
extmod/ulab but they do not prevent the doc building process from
completing.
2020-07-28 16:57:48 -05:00
Damien George
441460d81f extmod/uasyncio: Add StreamReader.readexactly(n) method.
It raises on EOFError instead of an IncompleteReadError (which is what
CPython does).  But the latter is derived from EOFError so code compatible
with MicroPython and CPython can be written by catching EOFError (eg see
included test).

Fixes issue #6156.

Signed-off-by: Damien George <damien@micropython.org>
2020-07-25 23:10:05 +10:00
Thorsten von Eicken
5264478007 extmod/modussl_mbedtls: Integrate shorter error strings.
The stm32 and esp32 ports now use shorter error strings for mbedtls errors.
Also, MBEDTLS_ERROR_C is enabled on stm32 by default to get these strings.
2020-07-21 00:31:05 +10:00
Thorsten von Eicken
9aa214077e extmod/modussl: Improve exception error messages.
This commit adds human readable error messages when mbedtls or axtls raise
an exception.  Currently often just an EIO error is raised so the user is
lost and can't tell whether it's a cert error, buffer overrun, connecting
to a non-ssl port, etc.  The axtls and mbedtls error raising in the ussl
module is modified to raise:

    OSError(-err_num, "error string")

For axtls a small error table of strings is added and used for the second
argument of the OSErrer.  For mbedtls the code uses mbedtls' built-in
strerror function, and if there is an out of memory condition it just
produces OSError(-err_num).  Producing the error string for mbedtls is
conditional on them being included in the mbedtls build, via
MBEDTLS_ERROR_C.
2020-07-20 23:41:45 +10:00
Jim Mussared
9d823a5d9a extmod/modbluetooth: Add event for "indicate acknowledgement".
This commit adds the IRQ_GATTS_INDICATE_DONE BLE event which will be raised
with the status of gatts_indicate (unlike notify, indications require
acknowledgement).

An example of its use is added to ble_temperature.py, and to the multitests
in ble_characteristic.py.

Implemented for btstack and nimble bindings, tested in both directions
between unix/btstack and pybd/nimble.
2020-07-20 23:26:41 +10:00
Jim Mussared
3c7ca2004c extmod/modbluetooth: Fix so it builds in peripheral-only mode. 2020-07-20 23:25:56 +10:00
Jim Mussared
43ceadac55 extmod/modbluetooth: Ignore unused self_in in ble_gatts_indicate. 2020-07-20 23:25:22 +10:00
Jim Mussared
b7698841b2 docs/library: Add gatts_indicate() doc to ubluetooth.rst.
Also clarify behavior of `gatts_notify` and add some TODOs about adding an
event for indication acknowledgement.
2020-07-18 14:34:44 +10:00
Jim Mussared
e152d0c197 extmod/btstack: Schedule notify/indicate/write ops for bg completion.
The goal of this commit is to allow using ble.gatts_notify() at any time,
even if the stack is not ready to send the notification right now.  It also
addresses the same issue for ble.gatts_indicate() and ble.gattc_write()
(without response).  In addition this commit fixes the case where the
buffer passed to write-with-response wasn't copied, meaning it could be
modified by the caller, affecting the in-progress write.

The changes are:

- gatts_notify/indicate will now run in the background if the ACL buffer is
  currently full, meaning that notify/indicate can be called at any time.

- gattc_write(mode=0) (no response) will now allow for one outstanding
  write.

- gattc_write(mode=1) (with response) will now copy the buffer so that it
  can't be modified by the caller while the write is in progress.

All four paths also now track the buffer while the operation is in
progress, which prevents the GC free'ing the buffer while it's still
needed.
2020-07-18 14:23:47 +10:00
Dan Halbert
88d8956378
Merge pull request #3143 from tannewt/improve_json
Add support to json.load for any object with readinto
2020-07-16 14:33:33 -04:00
Scott Shawcroft
372bcf8a95
Fix stream version and add basic readinto test 2020-07-10 17:33:17 -07:00
Scott Shawcroft
734661e79c
Add support to json.load for any object with readinto
This way we don't need to load the whole string version of the
json into memory.
2020-07-10 16:42:32 -07:00
Thomas Friebel
b6146ca1a1 extmod/nimble: Fix attr NULL ptr dereference in ble_gatt_attr_read_cb.
In case of error, NimBLE calls the read callback with attr = NULL.
2020-07-09 22:41:10 +10:00
Diego Elio Pettenò
34b4993d63 Add license to some obvious files. 2020-07-06 19:16:25 +01:00
Jeff Epler
783846c605 ulab: update to 0.51.1 2020-07-01 10:00:28 -05:00
Jeff Epler
103ad9966a remove machine_i2c, machine_spi
This micropython code is not used in circuitpython
2020-06-25 11:44:21 -05:00
Damien George
0c77668d11 extmod/vfs_lfs: Fix littlefs bindings to build in nan-box mode.
Signed-off-by: Damien George <damien@micropython.org>
2020-06-25 16:32:48 +10:00
Damien George
7dd480ad55 extmod/moductypes: Use mp_obj_is_dict_or_ordereddict to simplify code.
Signed-off-by: Damien George <damien@micropython.org>
2020-06-24 12:05:00 +10:00
David Lechner
77ed6f69ac tools/uncrustify: Enable more opts to remove space between func and '('.
With only `sp_func_proto_paren = remove` set there are some cases where
uncrustify misses removing a space between the function name and the
opening '('.  This sets all of the related options to `force` as well.
2020-06-19 22:07:32 +10:00
Damien George
026fda605e tools/codeformat.py: Include extmod/{btstack,nimble} in code formatting.
Signed-off-by: Damien George <damien@micropython.org>
2020-06-18 22:20:20 +10:00
jp-96
3705bc418c extmod/modbluetooth: Register default GATT service and fix esp32 init.
This is for the NimBLE bindings, to make sure the default GATT service
appears and that the esp32 initialises NimBLE correctly (it now matches
stm32).
2020-06-10 22:33:29 +10:00
Damien George
a4c96fb3b0 extmod/uasyncio: Add asyncio.wait_for_ms function.
Fixes issue #6107.
2020-06-10 22:29:44 +10:00
stijn
51fd6c9777 extmod/ure: Use single function for match/search/sub.
Saves about 500 bytes on unix x64 and enables CPython-conform
usage of passing a re object to these functions.
2020-06-08 09:16:09 +02:00
Jim Mussared
8b7ae4e099 extmod/modbluetooth: Support bigger characteristic values.
The ring buffer previously used a single unsigned byte field to save the
length, meaning that it would overflow for large characteristic value
responses.

With this commit it now use a 16-bit length instead and has code to
explicitly truncate at UINT16_MAX (although this should be impossible to
achieve in practice).
2020-06-05 14:11:46 +10:00
Jim Mussared
1cad63c0bc extmod/modbluetooth: Ensure status=0 always on success.
This commit makes sure that all discovery complete and read/write status
events set the status to zero on success.

The status value will be implementation-dependent on non-success cases.
2020-06-05 14:11:04 +10:00
Jim Mussared
c07ea3e4c2 extmod/modbluetooth: Implement read done event.
On btstack there's no status associated with the read result, it comes
through as a separate event.  This allows you to detect read failures or
timeouts.
2020-06-05 14:08:15 +10:00
Jim Mussared
919d640aec extmod/modbluetooth: Allow discovery of svc/char by uuid.
In most situations this is a more efficient way of going straight to the
service and characteristic you need.
2020-06-05 14:08:07 +10:00
Jim Mussared
6a3c89d584 extmod/modbluetooth: Add discover complete events for svc/char/desc.
Without this it's difficult to implement a state machine correctly if the
desired services are not found.
2020-06-05 14:07:52 +10:00
Jim Mussared
e6881f0829 extmod/modbluetooth: Make modbluetooth event not a bitfield.
There doesn't appear to be any use for only triggering on specific events,
so it's just easier to number them sequentially.  This makes them smaller
values so they take up only 1 byte in the ringbuf, only 1 byte for the
opcode in the bytecode, and makes room for more events.

Also add a couple of new event types that need to be implemented (to avoid
re-numbering later).

And rename _COMPLETE and _STATUS to _DONE for consistency.

In the future the "trigger" keyword argument can be reinstated by requiring
the user to compute the bitmask, eg:

    ble.irq(handler, 1 << _IRQ_SCAN_RESULT | 1 << _IRQ_SCAN_DONE)
2020-06-05 14:04:20 +10:00
Diego Elio Pettenò
dd5d7c86d2 Fix up end of file and trailing whitespace.
This can be enforced by pre-commit, but correct it separately to make it easier to review.
2020-06-03 10:56:35 +01:00
Jeff Epler
7f3fd20ea9 ulab: update submodule again 2020-06-01 11:03:38 -05:00
Jeff Epler
18c659780e ulab: update
.. add new modules and functions to our shared-bindings stubs
2020-06-01 09:02:29 -05:00
Damien George
22806ed5df extmod/vfs: Retain previous working directory if chdir fails.
Fixes issue #6069.
2020-05-29 23:05:01 +10:00
Damien George
7dffbfd22a extmod/vfs_lfsx: Fix import_stat so it takes into account current dir.
CPython semantics require searching the current directory if the import is
not absolute (when "" is in sys.path).

Fixes issue #6037.
2020-05-15 11:31:32 +10:00
Thomas Friebel
18fb5b4432 extmod/nimble: Make error code mapping default to MP_EIO.
Before this change, any NimBLE error that does not appear in the
ble_hs_err_to_errno_table maps to return code 0, meaning success.  If we
miss adding an error code to the table we end up returning success in case
of failure.

Instead, handle the zero case explicitly and default to MP_EIO.  This
allows removing the now-redundant MP_EIO entries from the mapping.
2020-05-11 22:31:30 +10:00
Damien George
3b6c9119eb extmod/modbluetooth: Add support for changing the GAP device name.
This commit allows the user to set/get the GAP device name used by service
0x1800, characteristic 0x2a00.  The usage is:

    BLE.config(gap_name="myname")
    print(BLE.config("gap_name"))

As part of this change the compile-time setting
MICROPY_PY_BLUETOOTH_DEFAULT_NAME is renamed to
MICROPY_PY_BLUETOOTH_DEFAULT_GAP_NAME to emphasise its link to GAP and this
new "gap_name" config value.  And the default value of this for the NimBLE
bindings is changed from "PYBD" to "MPY NIMBLE" to be more generic.
2020-05-11 21:30:41 +10:00
warriorofwire
d8491f3176 ujson: back out overeager loads() change; only change load() 2020-05-10 21:56:01 -07:00
warriorofwire
eb3d5fa453 ujson: do not eat trailing whitespace
Ujson should only worry about whitespace before JSON.  This becomes apparent when you are using MP stream protocol to read directly from input buffers.

When you attempt to read(1) on a UART (and possibly other protocols) you have to wait for either the byte or the timeout.

Fixes:
- Waiting for a timeout after you have completed reading a correct and complete JSON off the input.
- Raising an OSError after reading a correct and complete JSON off the input.
- Eating more data than semantically owned off the input buffer.
- Blocking to start parsing JSON until the entire JSON body has been loaded into a potentially large, contiguous Python object.

Code you would write before:
```
line = board_busio_uart_port.read_line()
json_dict = json.loads(line)
```
or reaching for fixed buffers and swapping them around in Python.

Code that did not work before that does now:
```
json_dict = json.load(board_busio_uart_port)
```

- This removes the need for intermediate copies of data when reading JSON from micropython stream protocol inputs.
- It also increases total application speed by parsing JSON concurrently with receiving on boards that read from UART via DMA.
- It simplifies code that users write while improving their apps.
2020-05-10 20:45:42 -07:00
robert
0f83ef395c extmod/vfs_lfsx: Fix rename to respect cur dir for new path.
If the new name start with '/', cur_dir is not prepened any more, so that
the current working directory is respected.  And extend the test cases for
rename to cover this functionality.
2020-05-08 21:54:04 +10:00
robert
d3ea28d04a extmod/vfs_lfsx: Normalize path name in chdir.
This change scans for '.', '..' and multiple '/' and normalizes the new
path name.  If the resulting path does not exist, an error is raised.
Non-existing interim path elements are ignored if they are removed during
normalization.
2020-05-08 21:52:15 +10:00
robert
a5ea4b9f3f extmod/vfs_lfsx: Fix path handling in uos.stat() to consider cur dir.
This fixes the bug, that stat(filename) would not consider the current
working directory.  So if e.g. the cwd is "lib", then stat("main.py") would
return the info for "/main.py" instead of "/lib/main.py".
2020-05-08 21:37:51 +10:00
Damien George
73c58150f5 extmod/modbtree: Retain reference to underlying stream so it's not GC'd.
For ports that have a system malloc which is not garbage collected (eg
unix, esp32), the stream object for the DB must be retained separately to
prevent it from being reclaimed by the MicroPython GC (because the
berkeley-db library uses malloc to allocate the DB structure which stores
the only reference to the stream).

Although in some cases the user code will explicitly retain a reference to
the underlying stream because it needs to call close() on it, this is not
always the case, eg in cases where the DB is intended to live forever.

Fixes issue #5940.
2020-05-02 16:08:04 +10:00
Jim Mussared
2e3c42775a unix: Add btstack to the unix submodules list.
But only when bluetooth is enabled, i.e. if building the dev or coverage
variants, and we have libusb available.

Update travis to match, i.e. specify the variant when doing
`make submodules`.
2020-04-29 16:54:12 +10:00
Jim Mussared
ebfd9ff2e6 extmod/modbluetooth: Fix sign compare and unused variable warnings. 2020-04-29 16:54:12 +10:00
Jim Mussared
7563d58210 unix: Add support for modbluetooth and BLE using btstack.
This commit adds full support to the unix port for Bluetooth using the
common extmod/modbluetooth Python bindings.  This uses the libusb HCI
transport, which supports many common USB BT adaptors.
2020-04-29 16:45:46 +10:00
Jim Mussared
c987adb9e9 extmod/btstack: Implement more robust init/deinit sequencing. 2020-04-29 16:45:46 +10:00
Jim Mussared
8119ec0765 extmod/modbluetooth: Don't hold atomic section during mp_sched_schedule.
Because, for example, on unix the atomic section isn't re-entrant, and
mp_sched_schedule() will try to re-acquire the atomic section.
2020-04-29 16:45:40 +10:00
Jim Mussared
0da47ecc93 stm32/Makefile: Rename SRC_LIB to LIB_SRC_C to match other ports. 2020-04-29 16:38:18 +10:00
Damien George
e08ca78f40 py/stream: Remove mp_stream_errno and use system errno instead.
This change is made for two reasons:

1. A 3rd-party library (eg berkeley-db-1.xx, axtls) may use the system
   provided errno for certain errors, and yet MicroPython stream objects
   that it calls will be using the internal mp_stream_errno.  So if the
   library returns an error it is not known whether the corresponding errno
   code is stored in the system errno or mp_stream_errno.  Using the system
   errno in all cases (eg in the mp_stream_posix_XXX wrappers) fixes this
   ambiguity.

2. For systems that have threading the system-provided errno should always
   be used because the errno value is thread-local.

For systems that do not have an errno, the new lib/embed/__errno.c file is
provided.
2020-04-27 23:58:46 +10:00
Jim Mussared
347c8917dc extmod/nimble: Update to work with NimBLE 1.3. 2020-04-27 22:51:10 +10:00
Jeff Epler
1a6df71cec ulab: update 2020-04-27 07:46:48 -05:00
Jeff Epler
c016ea6f0a ulab: actually update the submodule
PR#2802 missed the submodule update itself.
2020-04-26 10:12:56 -05:00
stijn
84fa3312cf all: Format code to add space after C++-style comment start.
Note: the uncrustify configuration is explicitly set to 'add' instead of
'force' in order not to alter the comments which use extra spaces after //
as a means of indenting text for clarity.
2020-04-23 11:24:25 +10:00
stijn
70affd9ba2 all: Fix implicit floating point to integer conversions.
These are found when building with -Wfloat-conversion.
2020-04-18 22:42:24 +10:00
stijn
bcf01d1686 all: Fix implicit conversion from double to float.
These are found when building with -Wfloat-conversion.
2020-04-18 22:42:24 +10:00
stijn
0ba68f8a1d all: Fix implicit floating point promotion.
Initially some of these were found building the unix coverage variant on
MacOS because that build uses clang and has -Wdouble-promotion enabled, and
clang performs more vigorous promotion checks than gcc.  Additionally the
codebase has been compiled with clang and msvc (the latter with warning
level 3), and with MICROPY_FLOAT_IMPL_FLOAT to find the rest of the
conversions.

Fixes are implemented either as explicit casts, or by using the correct
type, or by using one of the utility functions to handle floating point
casting; these have been moved from nativeglue.c to the public API.
2020-04-18 22:36:14 +10:00
stijn
b909e8b2dd Revert "all: Fix implicit casts of float/double, and signed comparison."
This reverts commit a2110bd3fc.  There's
nothing inherently wrong with it, but upcoming commits will apply similar
fixes in a slightly different way.
2020-04-18 22:36:06 +10:00
Jeff Epler
cc21bed0e4 ulab: Get updates from upstream 2020-04-14 10:09:00 -05:00
Damien George
5f0661b4fe extmod/uasyncio: Change cannot to can't in error message, and test exp.
Follow up to 8e048d2548 which missed these.
2020-04-14 21:51:25 +10:00
Jeff Epler
ca97964701 ulab: Get updates from upstream 2020-04-13 19:00:16 -05:00
Damien George
8e048d2548 all: Clean up error strings to use lowercase and change cannot to can't.
Now that error string compression is supported it's more important to have
consistent error string formatting (eg all lowercase English words,
consistent contractions).  This commit cleans up some of the strings to
make them more consistent.
2020-04-13 22:19:37 +10:00
Damien George
db137e70dc extmod/uasyncio: Add Loop.new_event_loop method.
This commit adds Loop.new_event_loop() which is used to reset the singleton
event loop.  This functionality is put here instead of in Loop.close() to
make it possible to write code that is compatible with CPython.
2020-04-13 22:16:52 +10:00
Damien George
c5a21a94f8 extmod/modbluetooth: Provide FLAG_WRITE_NO_RESPONSE for characteristics.
This flag is supported and needs to be set if characteristics are write-
without-response.
2020-04-07 13:46:56 +10:00
Damien George
899e89d4c6 extmod/btstack: Pass through SCAN_RSP events.
The latest version of BTstack has a bug fixed so that it correctly
configures scan parameters if they are set right after activating the
stack.  This means that BLE.gap_scan() will correctly set the scanning to
passive and so SCAN_RSP events are not passed through, so we don't need to
explicitly filter them in our bindings.
2020-04-07 13:46:56 +10:00
Jim Mussared
def76fe4d9 all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
Jim Mussared
1921224272 extmod/modubinascii: Make code private and module self-contained.
This commit makes all functions and function wrappers in modubinascii.c
STATIC and conditional on the MICROPY_PY_UBINASCII setting, which will
exclude the file from qstr/ compressed-string searching when ubinascii is
not enabled.  The now-unused modubinascii.h header file is also removed.

The cc3200 port is updated accordingly to use this module in its entirety
instead of providing its own top-level definition of ubinascii.

This was originally like this because the cc3200 port has its own ubinascii
module which referenced these methods.  The plan appeared to be that the
API might diverge (e.g. hardware crc), but this should be done similar to
I2C/SPI via a port-specific handler, rather than the port having its own
definition of the module.  Having a centralised module definition also
enforces consistency of the API among ports.
2020-04-05 14:13:53 +10:00
Kevin Köck
15f41c2dbf extmod/uasyncio: Add global exception handling methods.
This commit adds support for global exception handling in uasyncio
according to the CPython error handling:
https://docs.python.org/3/library/asyncio-eventloop.html#error-handling-api

This allows a program to receive exceptions from detached tasks and log
them to an appropriate location, instead of them being printed to the REPL.

The implementation preallocates a context dictionary so in case of an
exception there shouldn't be any RAM allocation.

The approach here is compatible with CPython except that in CPython the
exception handler is called once the task that threw an uncaught exception
is freed, whereas in MicroPython the exception handler is called
immediately when the exception is thrown.
2020-04-04 10:37:00 +11:00
Damien George
f97b5395ed extmod/uasyncio: Add StreamReader/StreamWriter as aliases of Stream cls.
To be compatible with CPython.  Fixes issue #5847.
2020-04-02 00:51:00 +11:00
Kevin Köck
aca19c25d2 extmod/uasyncio: Add error message to Lock.release's RuntimeError.
Otherwise it can be hard to understand what the error is if a blank
RuntimeError is raised.
2020-04-02 00:40:23 +11:00
Damien George
b389bc0afa extmod/uasyncio: Implement Loop.stop() to stop the event loop. 2020-04-02 00:14:18 +11:00
Damien George
711dd392d3 extmod/uasyncio: Don't create a Loop instance in get_event_loop().
The event loop is (for now) just a singleton so make it so that Loop
instances are not needed.
2020-04-01 23:56:31 +11:00
David Lechner
a2110bd3fc all: Fix implicit casts of float/double, and signed comparison.
These were found by buiding the unix coverage variant on macOS (so clang
compiler).  Mostly, these are fixing implicit cast of float/double to
mp_float_t which is one of those two and one mp_int_t to size_t fix for
good measure.
2020-03-30 12:04:21 +11:00
Damien George
1a3e386c67 all: Remove spaces inside and around parenthesis.
Using new options enabled in the uncrustify configuration.
2020-03-28 23:36:44 +11:00
David Lechner
9418611c8a unix: Implement PEP 475 to retry syscalls failing with EINTR.
https://www.python.org/dev/peps/pep-0475/

This implements something similar to PEP 475 on the unix port, and for the
VfsPosix class.

There are a few differences from the CPython implementation:
- Since we call mp_handle_pending() between any ENITR's, additional
  functions could be called if MICROPY_ENABLE_SCHEDULER is enabled, not
  just signal handlers.
- CPython only handles signal on the main thread, so other threads will
  raise InterruptedError instead of retrying.  On MicroPython,
  mp_handle_pending() will currently raise exceptions on any thread.

A new macro MP_HAL_RETRY_SYSCALL is introduced to reduce duplicated code
and ensure that all instances behave the same.  This will also allow other
ports that use POSIX-like system calls (and use, eg, VfsPosix) to provide
their own implementation if needed.
2020-03-27 14:40:46 +11:00
Damien George
3b68f36175 extmod/uasyncio: Add manifest.py for freezing uasyncio Py files. 2020-03-26 01:25:45 +11:00
Damien George
bc009fdd62 extmod/uasyncio: Add optional implementation of core uasyncio in C.
Implements Task and TaskQueue classes in C, using a pairing-heap data
structure.  Using this reduces RAM use of each Task, and improves overall
performance of the uasyncio scheduler.
2020-03-26 01:25:45 +11:00
Damien George
63b9944382 extmod/uasyncio: Add new implementation of uasyncio module.
This commit adds a completely new implementation of the uasyncio module.
The aim of this version (compared to the original one in micropython-lib)
is to be more compatible with CPython's asyncio module, so that one can
more easily write code that runs under both MicroPython and CPython (and
reuse CPython asyncio libraries, follow CPython asyncio tutorials, etc).
Async code is not easy to write and any knowledge users already have from
CPython asyncio should transfer to uasyncio without effort, and vice versa.

The implementation here attempts to provide good compatibility with
CPython's asyncio while still being "micro" enough to run where MicroPython
runs. This follows the general philosophy of MicroPython itself, to make it
feel like Python.

The main change is to use a Task object for each coroutine.  This allows
more flexibility to queue tasks in various places, eg the main run loop,
tasks waiting on events, locks or other tasks.  It no longer requires
pre-allocating a fixed queue size for the main run loop.

A pairing heap is used to queue Tasks.

It's currently implemented in pure Python, separated into components with
lazy importing for optional components.  In the future parts of this
implementation can be moved to C to improve speed and reduce memory usage.
But the aim is to maintain a pure-Python version as a reference version.
2020-03-26 01:25:45 +11:00
David Lechner
58d9a4815d extmod/vfs_posix_file: Include unistd.h to get STD{IN,OUT,ERR}_FILENO. 2020-03-25 00:59:39 +11:00
Damien George
2cdf1d25f5 unix: Remove custom file implementation to use extmod's VFS POSIX one.
The implementation in extmod/vfs_posix_file.c is now equivalent to that in
ports/unix/file.c, so remove the latter and use the former instead.
2020-03-18 21:01:07 +11:00
Damien George
68b1bc2042 extmod/vfs_posix_file: Lock GIL when writing and allow stdio flush.
Also support MP_STREAM_GET_FILENO ioctl.  The stdio flush change was done
previously for the unix port in 3e0b46b9af.
These changes make this POSIX file implementation equivalent to the unix
file implementation.
2020-03-18 21:01:07 +11:00
Damien George
ad9a0ec8ab all: Convert exceptions to use mp_raise_XXX helpers in remaining places. 2020-03-18 17:26:19 +11:00
Damien George
8f0778b209 extmod/modlwip: Properly handle non-blocking and timeout on UDP recv.
Fixes UDP non-blocking recv so it returns EAGAIN instead of ETIMEDOUT.
Timeout waiting for incoming data is also improved by replacing 100ms delay
with poll_sockets(), as is done in other parts of this module.

Fixes issue #5759.
2020-03-18 10:51:32 +11:00
Damien George
00267aae0b extmod/modlwip: Fix polling of UDP socket so it doesn't return HUP.
STATE_NEW will return HUP when polled so put active UDP sockets into a new
state which is different to STATE_NEW.

Fixes issue #5758.
2020-03-18 10:49:27 +11:00
Jeff Epler
772ac705d3 update ulab 2020-03-17 08:32:52 -05:00
Damien George
ed848553b4 extmod/vfs: Factor out vfs mount-and-chdir helper from stm32. 2020-03-11 14:24:26 +11:00
Damien George
7bf62562ce extmod/nimble: When getting BLE MAC try public address if random fails.
This is needed for BLE.config('mac') to work on esp32.
2020-03-11 14:02:13 +11:00
Damien George
dd0bc26e65 extmod/modbluetooth: Change scan result's "connectable" to "adv_type".
This commit changes the BLE _IRQ_SCAN_RESULT data from:

    addr_type, addr, connectable, rssi, adv_data

to:

    addr_type, addr, adv_type, rssi, adv_data

This allows _IRQ_SCAN_RESULT to handle all scan result types (not just
connectable and non-connectable passive scans), and to distinguish between
them using adv_type which is an integer taking values 0x00-0x04 per the BT
specification.

This is a breaking change to the API, albeit a very minor one: the existing
connectable value was a boolean and True now becomes 0x00, False becomes
0x02.

Documentation is updated and a test added.

Fixes #5738.
2020-03-11 14:00:44 +11:00
Thomas Friebel
41a9f1dec8 extmod/modbluetooth: Unify error handling in remaining places.
Most error handling is done via `bluetooth_handle_errno()` already.
Replace the remaining few manual checks with calls to that function.
2020-03-11 13:01:35 +11:00
Jim Mussared
1937fb22ab extmod/nimble: Clarify active state and check for active in all methods.
This commit ensures that the BLE stack is active before allowing operations
that may otherwise crash if it's not active.  It also clarifies the state
better (adding the "stopping" state) and renames mp_bluetooth_is_enabled to
the more self-explanatory mp_bluetooth_is_active.
2020-03-11 13:01:35 +11:00
Jeff Epler
eab9b670d8 ulab: Use https://github.com/v923z/micropython-ulab/
The only delta we were carrying was in the README.
2020-03-09 14:48:52 -05:00
Damien George
e965363b6b stm32: Refactor Bluetooth HCI RX to be independent of transport layer.
Now all HCI specific code (eg UART vs WB55 internal messaging) is confined
to modbluetooth_hci.c.
2020-03-10 01:53:42 +11:00
Damien George
d7259f6b1c extmod/btstack: Implement notifications/indications for GATT clients.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George
018ce122ca extmod/btstack: Implement scan and gatt client, connect and disconnect.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George
372e5a280e extmod/btstack: Implement gatts_db for btstack.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George
86c26db3ff extmod/btstack: Implement service registration.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George
0674917bc5 extmod/btstack: Implement advertising.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George
fbefcef1df extmod/btstack: Add empty modbluetooth implementation.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George
0ac06a510a extmod/modbluetooth: Extract out gatts_db functionality from nimble.
For use by other stacks, if they need it.

Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George
894c550c86 stm32: Refactor bluetooth stack/hci/driver bindings.
This makes a cleaner separation between the: driver, HCI UART and BT stack.
Also updated the naming to be more consistent (mp_bluetooth_hci_*).

Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George
c44d52f33e extmod/modbluetooth_nimble: Move nimble specific code, factor nimble.mk.
Move extmod/modbluetooth_nimble.* to extmod/nimble.  And move common
Makefile lines to extmod/nimble/nimble.mk (which was previously only used
by stm32).  This allows (upcoming) btstack to follow a similar structure.

Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-06 12:35:20 +11:00
Damien George
69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
Jeff Epler
39cfe32c34 Update ulab from upstream again 2020-02-27 14:14:05 -06:00
Jeff Epler
fa3b9eba92 ulab: Incorporate it 2020-02-27 11:03:03 -06:00
Jim Mussared
f8449dd092 extmod/modframebuf: Allow blit source to be a subclass of FrameBuffer. 2020-02-21 13:32:48 +11:00
Jim Mussared
cddb90e8b0 extmod/modbluetooth_nimble: Fix wrong offset used for descriptor flags. 2020-02-18 16:37:38 +11:00
Jim Mussared
66ac2e1fc0 extmod/modbluetooth.h: Fix typo in comment about registering services. 2020-02-18 16:36:37 +11:00
Thomas Friebel
f4726735cf extmod/modbluetooth: Implement config getter for BLE rxbuf size.
Knowing the buffer size can be important, to ensure that valid data will be
received.
2020-02-18 13:37:50 +11:00
Damien George
ce39c958ef py: Factor out definition of mp_float_union_t to one location. 2020-02-18 13:04:36 +11:00
Dan Halbert
e0753c4ad2 avoid os.stat() int overflow on smallint-only builds 2020-02-14 18:33:37 -05:00
Damien George
ad7213d3c3 py: Add mp_raise_msg_varg helper and use it where appropriate.
This commit adds mp_raise_msg_varg(type, fmt, ...) as a helper for
nlr_raise(mp_obj_new_exception_msg_varg(type, fmt, ...)).  It makes the
C-level API for raising exceptions more consistent, and reduces code size
on most ports:

   bare-arm:   +28 +0.042%
minimal x86:  +100 +0.067%
   unix x64:   -56 -0.011%
unix nanbox:  -300 -0.068%
      stm32:  -204 -0.054% PYBV10
     cc3200:    +0 +0.000%
    esp8266:   -64 -0.010% GENERIC
      esp32:  -104 -0.007% GENERIC
        nrf:  -136 -0.094% pca10040
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2020-02-13 11:52:40 +11:00
Damien George
97eca38c4f py: Add mp_raise_type helper macro and use it where appropriate.
This provides a more consistent C-level API to raise exceptions, ie moving
away from nlr_raise towards mp_raise_XXX.  It also reduces code size by a
small amount on some ports.
2020-02-13 11:03:37 +11:00
Dan Halbert
5164719e67 preserve errno in OSError 2020-02-04 16:19:40 -05:00
Dan Halbert
57b566e736 Include filename for 'No such file/directory', etc. 2020-02-04 13:32:39 -05:00
Scott Shawcroft
f6a635b102
Fix subclassing of objects that are tested. Others may still be broken. 2020-01-27 14:52:42 -08:00
David Lechner
35f66d38b8 extmod/vfs_posix: Release GIL during system calls.
This releases the GIL during syscalls that could block.
2020-01-26 23:26:14 +11:00
Damien George
bfbd94401d py: Make mp_obj_get_type() return a const ptr to mp_obj_type_t.
Most types are in rodata/ROM, and mp_obj_base_t.type is a constant pointer,
so enforce this const-ness throughout the code base.  If a type ever needs
to be modified (eg a user type) then a simple cast can be used.
2020-01-09 11:25:26 +11:00
Roy Hooper
767ce1cdf8 remove unnecessary GCC pragmas 2020-01-02 18:03:18 -05:00
Roy Hooper
ccf158b030 raise mp_raise_NotImplementedError 2020-01-02 18:00:36 -05:00
Yonatan Goldschmidt
aca8873bb8 extmod/modbluetooth: Fix func prototype, empty args should be (void).
This fixes a -Wstrict-prototypes error.
2019-12-27 23:57:46 +11:00
Damien George
6f872f81d6 extmod: Fix modbluetooth and modwebrepl to build in nanbox mode. 2019-12-27 22:54:53 +11:00
Damien George
d97b40bdaa py: Introduce MP_ROM_FALSE/MP_ROM_TRUE for ROM to refer to bool objects.
This helps to prevent mistakes, and allows easily changing the ROM value of
False/True if needed.
2019-12-27 22:54:20 +11:00
Damien George
09376f0e47 py: Introduce MP_ROM_NONE macro for ROM to refer to None object.
This helps to prevent mistakes, and allows easily changing the ROM value of
None if needed.
2019-12-27 22:51:17 +11:00
Damien George
ed2be79b49 extmod/uzlib: Explicitly cast ptr-diff-expr to unsigned.
The struct member "dest" should never be less than "destStart", so their
difference is never negative.  Cast as such to make the comparison
explicitly unsigned, ensuring the compiler produces the correct comparison
instruction, and avoiding any compiler warnings.
2019-12-23 00:07:03 +11:00
Jim Mussared
7ce1e0b1dc extmod/webrepl: Move webrepl scripts to common place and use manifest.
Move webrepl support code from ports/esp8266/modules into extmod/webrepl
(to be alongside extmod/modwebrepl.c), and use frozen manifests to include
it in the build on esp8266 and esp32.

A small modification is made to webrepl.py to make it work on non-ESP
ports, i.e. don't call dupterm_notify if not available.
2019-12-20 12:59:13 +11:00
Roy Hooper
dc8dd6df20 Revert subscr signature change 2019-12-13 14:29:15 -05:00
Roy Hooper
15072c69c9 push changes 2019-12-13 14:07:53 -05:00
Damien George
ba84453f77 examples/natmod: Add urandom native module example. 2019-12-13 13:33:40 +11:00
Damien George
bbeaafd9aa extmod: Add dynamic-runtime guards to btree/framebuf/uheapq/ure/uzlib.
So they can be built as dynamic native modules, as well as existing static
native modules.
2019-12-12 20:15:28 +11:00
Damien George
e5acd06ad5 extmod/modbtree: Use mp_printf instead of printf. 2019-12-12 20:15:28 +11:00
Roy Hooper
0326c98fd5 Merge branch 'master' into new-pixelbuf-api 2019-12-10 20:44:43 -05:00
Damien George
84958a8fe1 extmod/modbluetooth: Allow setting ringbuf size via BLE.config(rxbuf=).
The size of the event ringbuf was previously fixed to compile-time config
value, but it's necessary to sometimes increase this for applications that
have large characteristic buffers to read, or many events at once.

With this commit the size can be set via BLE.config(rxbuf=512), for
example.  This also resizes the internal event data buffer which sets the
maximum size of incoming data passed to the event handler.
2019-12-05 11:30:35 +11:00
Jeff Epler
238e121236 protocols: Allow them to be (optionally) type-safe
Protocols are nice, but there is no way for C code to verify whether
a type's "protocol" structure actually implements some particular
protocol.  As a result, you can pass an object that implements the
"vfs" protocol to one that expects the "stream" protocol, and the
opposite of awesomeness ensues.

This patch adds an OPTIONAL (but enabled by default) protocol identifier
as the first member of any protocol structure.  This identifier is
simply a unique QSTR chosen by the protocol designer and used by each
protocol implementer.  When checking for protocol support, instead of
just checking whether the object's type has a non-NULL protocol field,
use `mp_proto_get` which implements the protocol check when possible.

The existing protocols are now named:
    protocol_framebuf
    protocol_i2c
    protocol_pin
    protocol_stream
    protocol_spi
    protocol_vfs
(most of these are unused in CP and are just inherited from MP; vfs and
stream are definitely used though)

I did not find any crashing examples, but here's one to give a flavor of what
is improved, using `micropython_coverage`.  Before the change,
the vfs "ioctl" protocol is invoked, and the result is not intelligible
as json (but it could have resulted in a hard fault, potentially):

    >>> import uos, ujson
    >>> u = uos.VfsPosix('/tmp')
    >>> ujson.load(u)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: syntax error in JSON

After the change, the vfs object is correctly detected as not supporting
the stream protocol:
    >>> ujson.load(p)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    OSError: stream operation not supported
2019-12-04 09:29:57 -06:00
Damien George
7aeafe2ae9 extmod/modbluetooth: Add optional 4th arg to gattc_write for write mode.
This allows the user to explicitly select the behaviour of the write to the
remote peripheral.  This is needed for peripherals that have
characteristics with WRITE_NO_RESPONSE set (instead of normal WRITE).  The
function's signature is now:

    BLE.gattc_write(conn_handle, value_handle, data, mode=0)

mode=0 means write without response, while mode=1 means write with
response.  The latter was the original behaviour so this commit is a change
in behaviour of this method, and one should specify 1 as the 4th argument
to get back the old behaviour.

In the future there could be more modes supported, such as long writes.
2019-12-04 23:23:07 +11:00
Damien George
40cc7ec677 stm32/mpconfigport.h: Use IRQ_PRI_PENDSV to protect bluetooth ringbuf.
The default protection for the BLE ringbuf is to use
MICROPY_BEGIN_ATOMIC_SECTION, which disables all interrupts. On stm32 it
only needs to disable the lowest priority IRQ, pendsv, because that's the
IRQ level at which the BLE stack is driven.
2019-12-04 13:39:22 +11:00
Damien George
8ce69288e9 extmod/modbluetooth: Remove limit on data coming from gattc data input.
This removes the limit on data coming in from a BLE.gattc_read() request,
or a notify with payload (coming in to a central).  In both cases the data
coming in to the BLE callback is now limited only by the available data in
the ringbuf, whereas before it was capped at (default hard coded) 20 bytes.
2019-12-02 23:27:25 +11:00
Damien George
d6e051082a extmod/modbluetooth: Simplify how BLE IRQ callback is scheduled.
Instead of enqueue_irq() inspecting the ringbuf to decide whether to
schedule the IRQ callback (if ringbuf is empty), maintain a flag that knows
if the callback is on the schedule queue or not.  This saves about 150
bytes of code (for stm32 builds), and simplifies all uses of enqueue_irq()
and schedule_ringbuf().
2019-12-02 23:25:36 +11:00
Roy Hooper
56720eae0a remove unnecessary intermediate mp_obj_subscr wrapper 2019-11-26 18:39:08 -05:00
Damien George
6b3404f25e extmod/vfs_lfs: Fix bug when passing no args to constructor and mkfs. 2019-11-26 00:08:57 +11:00
Damien George
5634a31a98 extmod/vfs_lfs: Pass flag along to ioctl when init'ing bdev for lfs.
To hint to the block device that the extended block protocol will be used.
2019-11-26 00:07:42 +11:00
Jim Mussared
e873d352ad extmod/modbluetooth: Simplify management of pre-allocated event data.
The address, adv payload and uuid fields of the event are pre-allocated by
modbluetooth, and reused in the IRQ handler.  Simplify this and move all
storage into the `mp_obj_bluetooth_ble_t` instance.

This now allows users to hold on to a reference to these instances without
crashes, although they may be overwritten by future events.  If they want
to hold onto the values longer term they need to copy them.
2019-11-25 17:32:10 +11:00
Jim Mussared
438c0dc2a4 extmod/modbluetooh_nimble: Fix UUID conversion for 16 and 32 bit values. 2019-11-25 17:31:59 +11:00
Jim Mussared
2ae755d9e1 extmod/modbluetooth_nimble: Make gap_scan_stop no-op if no scan ongoing.
No need for this to throw an exception if the intent (don't be scanning) is
clear, and avoids a race with the scan duration timeout.
2019-11-25 17:27:40 +11:00
Jim Mussared
d19c6d0519 extmod/modbluetooth: Create UUID from bytes and allow comparison ops.
This allows construction of UUID objects from advertising data payloads and
matching against known UUIDs.
2019-11-25 17:20:51 +11:00
Roy Hooper
a9baa441c9 disable -Wunused-parameter on subscr functions 2019-11-23 13:52:14 -05:00
Roy Hooper
6108fa3766 Merge branch 'master' into new-pixelbuf-api 2019-11-23 12:22:04 -05:00
Roy Hooper
c770ccd939 make ->subscr take an instance to pass when instance_subscr is called from subscr. 2019-11-23 12:09:26 -05:00
Jim Mussared
334ba01c90 extmod/modbluetooth: Prioritise non-scan-result events.
Remove existing scan result events from the ringbuf if the ringbuf is full
and we're trying to enqueue any other event.  This is needed so that events
such as SCAN_COMPLETE are always put on the ringbuf.
2019-11-21 12:04:57 +11:00
Damien George
799b6d1e0c extmod: Consolidate FAT FS config to MICROPY_VFS_FAT across all ports.
This commit removes the Makefile-level MICROPY_FATFS config and moves the
MICROPY_VFS_FAT config to the Makefile level to replace it.  It also moves
the include of the oofatfs source files in the build from each port to a
central place in extmod/extmod.mk.

For a port to enabled VFS FAT support it should now set MICROPY_VFS_FAT=1
at the level of the Makefile.  This will include the relevant oofatfs files
in the build and set MICROPY_VFS_FAT=1 at the C (preprocessor) level.
2019-11-11 11:37:38 +11:00
Damien George
9b27069e2f extmod/vfs: Add autodetect of littlefs filesystem when mounting. 2019-11-06 12:15:34 +11:00
Damien George
c13f9f209d all: Convert nlr_raise(mp_obj_new_exception_msg(x)) to mp_raise_msg(x).
This helper function was added a while ago and these are the remaining
cases to convert, to save a bit of code size.
2019-11-05 11:35:45 +11:00
Damien George
2ee9e1a4ed extmod/modbtree: Make FILEVTABLE const to put it in ROM. 2019-11-01 17:25:40 +11:00
Damien George
40ea1915fc extmod/nimble: Factor out stm32-specific HCI UART RX/TX code. 2019-11-01 12:41:37 +11:00
Damien George
78145b98ef extmod/nimble: Remove unneeded nimble_sprintf wrapper function. 2019-11-01 12:15:07 +11:00
Damien George
9dd9f9ff06 extmod/modussl_mbedtls: Check for invalid key/cert data. 2019-10-31 16:22:42 +11:00
Damien George
07ea81fbc5 extmod/modussl_mbedtls: Fix getpeercert to return None if no cert avail. 2019-10-31 13:42:24 +11:00
Damien George
26d8fd2c0a extmod/modlwip: Unconditionally return POLLHUP/POLLERR when polling.
POSIX poll should always return POLLERR and POLLHUP in revents, regardless
of whether they were requested in the input events flags.

See issues #4290 and #5172.
2019-10-31 13:37:51 +11:00
Damien George
feaa251674 extmod/modlwip: Make socket poll return POLLNVAL in case of bad file. 2019-10-31 12:54:37 +11:00
Damien George
71401d5065 extmod/modlwip: Unconditionally return POLLHUP when polling new socket.
POSIX poll should always return POLLERR and POLLHUP in revents, regardless
of whether they were requested in the input events flags.

See issues #4290 and #5172.
2019-10-31 12:54:37 +11:00
Damien George
660a61a388 extmod/vfs_lfs: Allow compiling in VfsLfs1 and VfsLfs2 separately.
These classes are enabled via the config options MICROPY_VFS_LFS1 and
MICROPY_VFS_LFS2, which are disabled by default.
2019-10-30 12:08:58 +11:00