Commit Graph

1051 Commits

Author SHA1 Message Date
Damien George c70e599659 extmod/vfs: Support larger integer range in VFS stat time fields.
On ports like unix where the Epoch is 1970/1/1 and atime/mtime/ctime are in
seconds since the Epoch, this value will overflow a small-int on 32-bit
systems.  So far this is only an issue on 32-bit unix builds that use the
VFS layer (eg dev and coverage unix variants) but the fix (using
mp_obj_new_int_from_uint instead of MP_OBJ_NEW_SMALL_INT) is there for all
ports so as to not complicate the code, and because they will need the
range one day.

Also apply a similar fix to other fields in VfsPosix.stat because they may
also be large.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-01 12:36:28 +10:00
Damien George d1995e50eb extmod/modlwip: Fix error return for TCP recv when not connected.
This commit fixes the cases when a TCP socket is in STATE_NEW,
STATE_LISTENING or STATE_CONNECTING and recv() is called on it.  It now
raises ENOTCONN instead of a random error code due to it previously
indexing beyond the start of error_lookup_table[].

Signed-off-by: Damien George <damien@micropython.org>
2020-08-30 13:20:51 +10:00
Damien George 06659077a8 all: Update Python code to conform to latest black formatting.
Updating to Black v20.8b1 there are two changes that affect the code in
this repository:

- If there is a trailing comma in a list (eg [], () or function call) then
  that list is now written out with one line per element.  So remove such
  trailing commas where the list should stay on one line.

- Spaces at the start of """ doc strings are removed.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-29 15:18:01 +10:00
Michael Buesch cef678b2db extmod/machine_i2c: Fix buffer overrun if 'addrsize' is bigger than 32.
The memory operation functions read_mem() and write_mem() create a
temporary buffer on the local C stack for the address bytes with the size
of 4 bytes.  This buffer is filled in a loop from the user supplied address
and address length.  If the user supplied 'addrsize' is bigger than 32, the
local buffer is overrun.

Fix this by raising an exception for invalid 'addrsize' values.

Signed-off-by: Michael Buesch <m@bues.ch>
2020-08-27 12:39:11 +10:00
Andrew Leech a80a146858 extmod/bluetooth: Support active scanning in BLE.gap_scan().
This adds an additional optional parameter to gap_scan() to select active
scanning, where scan responses are returned as well as normal scan results.
This parameter is False by default which retains the existing behaviour.
2020-08-26 15:00:11 +10:00
Jim Mussared 0bc2c1c105 extmod/modbluetooth: Fix race between READ_REQUEST and other IRQs.
The READ_REQUEST callback is handled as a hard interrupt (because the BLE
stack needs an immediate response from it so it can continue) and so calls
to Python require extra protection:

- the caller-owned tuple passed into the callback must be separate from the
  tuple used by other callback events (which are soft interrupts);

- the GC and scheduler must be locked during callback execution.
2020-08-26 14:57:36 +10:00
Jim Mussared 3d9a7ed02f extmod/btstack: Implement GAP scan duration_ms parameter.
This commit makes scanning work when duration_ms is set to zero.  Prior to
this it would not work with duration_ms set to zero.
2020-08-26 14:55:52 +10:00
Damien George 2acc087880 extmod/vfs_lfs: Add mtime support to littlefs files.
This commit adds support for modification time of files on littlefs v2
filesystems, using file attributes.  For some background see issue #6114.

Features/properties of this implementation:
- Only supported on littlefs2 (not littlefs1).
- Uses littlefs2's general file attributes to store the timestamp.
- The timestamp is 64-bits and stores nanoseconds since 1970/1/1 (if the
  range to the year 2554 is not enough then additional bits can be added to
  this timestamp by adding another file attribute).
- mtime is enabled by default but can be disabled in the constructor, eg:
  uos.mount(uos.VfsLfs2(bdev, mtime=False), '/flash')
- It's fully backwards compatible, existing littlefs2 filesystems will work
  without reformatting and timestamps will be added transparently to
  existing files (once they are opened for writing).
- Files without timestamps will open correctly, and stat will just return 0
  for their timestamp.
- mtime can be disabled or enabled each mount time and timestamps will only
  be updated if mtime is enabled (otherwise they will be untouched).

Signed-off-by: Damien George <damien@micropython.org>
2020-08-25 17:35:19 +10:00
Damien George 55c76eaac1 extmod/uasyncio: Truncate negative sleeps to 0.
Otherwise a task that continuously awaits on a large negative sleep can
monopolise the scheduler (because its wake time is always less than
everything else in the pairing heap).

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 12:17:06 +10:00
Damien George 60f5b941e0 extmod/vfs_reader: Fix mp_reader_new_file to open file in "rb" mode.
mp_reader_new_file() is used to read in files for importing, either .py or
.mpy files, for the lexer and persistent code loader respectively.  In both
cases the file should be opened in raw bytes mode: the lexer handles
unicode characters itself, and .mpy files contain 8-bit bytes by nature.

Before this commit importing was working correctly because, although the
file was opened in text mode, all native filesystem implementations (POSIX,
FAT, LFS) would access the file in raw bytes mode via mp_stream_rw()
calling mp_stream_p_t.read().  So it was only an issue for non-native
filesystems, such as those implemented in Python.  For Python-based
filesystem implementations, a call to mp_stream_rw() would go via IOBase
and then to readinto() at the Python level, and readinto() is only defined
on files opened in raw bytes mode.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-12 23:40:50 +10:00
Jeff Epler c849b781c0 Combine 'index out of range' messages 2020-08-04 14:45:45 -05:00
Jeff Epler 89797fd3f9 various: Use mp_obj_get_type_qstr more widely
This removes runtime allocations of the cstring version of the qstring.

It is not a size improvement
2020-08-04 14:45:45 -05:00
Jeff Epler 9b8df7f635 Upgrade ulab
This version
 * moves source files to reflect module structure
 * adds inline documentation suitable for extract_pyi
 * incompatibly moves spectrogram to fft
 * incompatibly removes "extras"

There are some remaining markup errors in the specific revision of
extmod/ulab but they do not prevent the doc building process from
completing.
2020-07-28 16:57:48 -05:00
Damien George 441460d81f extmod/uasyncio: Add StreamReader.readexactly(n) method.
It raises on EOFError instead of an IncompleteReadError (which is what
CPython does).  But the latter is derived from EOFError so code compatible
with MicroPython and CPython can be written by catching EOFError (eg see
included test).

Fixes issue #6156.

Signed-off-by: Damien George <damien@micropython.org>
2020-07-25 23:10:05 +10:00
Thorsten von Eicken 5264478007 extmod/modussl_mbedtls: Integrate shorter error strings.
The stm32 and esp32 ports now use shorter error strings for mbedtls errors.
Also, MBEDTLS_ERROR_C is enabled on stm32 by default to get these strings.
2020-07-21 00:31:05 +10:00
Thorsten von Eicken 9aa214077e extmod/modussl: Improve exception error messages.
This commit adds human readable error messages when mbedtls or axtls raise
an exception.  Currently often just an EIO error is raised so the user is
lost and can't tell whether it's a cert error, buffer overrun, connecting
to a non-ssl port, etc.  The axtls and mbedtls error raising in the ussl
module is modified to raise:

    OSError(-err_num, "error string")

For axtls a small error table of strings is added and used for the second
argument of the OSErrer.  For mbedtls the code uses mbedtls' built-in
strerror function, and if there is an out of memory condition it just
produces OSError(-err_num).  Producing the error string for mbedtls is
conditional on them being included in the mbedtls build, via
MBEDTLS_ERROR_C.
2020-07-20 23:41:45 +10:00
Jim Mussared 9d823a5d9a extmod/modbluetooth: Add event for "indicate acknowledgement".
This commit adds the IRQ_GATTS_INDICATE_DONE BLE event which will be raised
with the status of gatts_indicate (unlike notify, indications require
acknowledgement).

An example of its use is added to ble_temperature.py, and to the multitests
in ble_characteristic.py.

Implemented for btstack and nimble bindings, tested in both directions
between unix/btstack and pybd/nimble.
2020-07-20 23:26:41 +10:00
Jim Mussared 3c7ca2004c extmod/modbluetooth: Fix so it builds in peripheral-only mode. 2020-07-20 23:25:56 +10:00
Jim Mussared 43ceadac55 extmod/modbluetooth: Ignore unused self_in in ble_gatts_indicate. 2020-07-20 23:25:22 +10:00
Jim Mussared b7698841b2 docs/library: Add gatts_indicate() doc to ubluetooth.rst.
Also clarify behavior of `gatts_notify` and add some TODOs about adding an
event for indication acknowledgement.
2020-07-18 14:34:44 +10:00
Jim Mussared e152d0c197 extmod/btstack: Schedule notify/indicate/write ops for bg completion.
The goal of this commit is to allow using ble.gatts_notify() at any time,
even if the stack is not ready to send the notification right now.  It also
addresses the same issue for ble.gatts_indicate() and ble.gattc_write()
(without response).  In addition this commit fixes the case where the
buffer passed to write-with-response wasn't copied, meaning it could be
modified by the caller, affecting the in-progress write.

The changes are:

- gatts_notify/indicate will now run in the background if the ACL buffer is
  currently full, meaning that notify/indicate can be called at any time.

- gattc_write(mode=0) (no response) will now allow for one outstanding
  write.

- gattc_write(mode=1) (with response) will now copy the buffer so that it
  can't be modified by the caller while the write is in progress.

All four paths also now track the buffer while the operation is in
progress, which prevents the GC free'ing the buffer while it's still
needed.
2020-07-18 14:23:47 +10:00
Dan Halbert 88d8956378
Merge pull request #3143 from tannewt/improve_json
Add support to json.load for any object with readinto
2020-07-16 14:33:33 -04:00
Scott Shawcroft 372bcf8a95
Fix stream version and add basic readinto test 2020-07-10 17:33:17 -07:00
Scott Shawcroft 734661e79c
Add support to json.load for any object with readinto
This way we don't need to load the whole string version of the
json into memory.
2020-07-10 16:42:32 -07:00
Thomas Friebel b6146ca1a1 extmod/nimble: Fix attr NULL ptr dereference in ble_gatt_attr_read_cb.
In case of error, NimBLE calls the read callback with attr = NULL.
2020-07-09 22:41:10 +10:00
Diego Elio Pettenò 34b4993d63 Add license to some obvious files. 2020-07-06 19:16:25 +01:00
Jeff Epler 783846c605 ulab: update to 0.51.1 2020-07-01 10:00:28 -05:00
Jeff Epler 103ad9966a remove machine_i2c, machine_spi
This micropython code is not used in circuitpython
2020-06-25 11:44:21 -05:00
Damien George 0c77668d11 extmod/vfs_lfs: Fix littlefs bindings to build in nan-box mode.
Signed-off-by: Damien George <damien@micropython.org>
2020-06-25 16:32:48 +10:00
Damien George 7dd480ad55 extmod/moductypes: Use mp_obj_is_dict_or_ordereddict to simplify code.
Signed-off-by: Damien George <damien@micropython.org>
2020-06-24 12:05:00 +10:00
David Lechner 77ed6f69ac tools/uncrustify: Enable more opts to remove space between func and '('.
With only `sp_func_proto_paren = remove` set there are some cases where
uncrustify misses removing a space between the function name and the
opening '('.  This sets all of the related options to `force` as well.
2020-06-19 22:07:32 +10:00
Damien George 026fda605e tools/codeformat.py: Include extmod/{btstack,nimble} in code formatting.
Signed-off-by: Damien George <damien@micropython.org>
2020-06-18 22:20:20 +10:00
jp-96 3705bc418c extmod/modbluetooth: Register default GATT service and fix esp32 init.
This is for the NimBLE bindings, to make sure the default GATT service
appears and that the esp32 initialises NimBLE correctly (it now matches
stm32).
2020-06-10 22:33:29 +10:00
Damien George a4c96fb3b0 extmod/uasyncio: Add asyncio.wait_for_ms function.
Fixes issue #6107.
2020-06-10 22:29:44 +10:00
stijn 51fd6c9777 extmod/ure: Use single function for match/search/sub.
Saves about 500 bytes on unix x64 and enables CPython-conform
usage of passing a re object to these functions.
2020-06-08 09:16:09 +02:00
Jim Mussared 8b7ae4e099 extmod/modbluetooth: Support bigger characteristic values.
The ring buffer previously used a single unsigned byte field to save the
length, meaning that it would overflow for large characteristic value
responses.

With this commit it now use a 16-bit length instead and has code to
explicitly truncate at UINT16_MAX (although this should be impossible to
achieve in practice).
2020-06-05 14:11:46 +10:00
Jim Mussared 1cad63c0bc extmod/modbluetooth: Ensure status=0 always on success.
This commit makes sure that all discovery complete and read/write status
events set the status to zero on success.

The status value will be implementation-dependent on non-success cases.
2020-06-05 14:11:04 +10:00
Jim Mussared c07ea3e4c2 extmod/modbluetooth: Implement read done event.
On btstack there's no status associated with the read result, it comes
through as a separate event.  This allows you to detect read failures or
timeouts.
2020-06-05 14:08:15 +10:00
Jim Mussared 919d640aec extmod/modbluetooth: Allow discovery of svc/char by uuid.
In most situations this is a more efficient way of going straight to the
service and characteristic you need.
2020-06-05 14:08:07 +10:00
Jim Mussared 6a3c89d584 extmod/modbluetooth: Add discover complete events for svc/char/desc.
Without this it's difficult to implement a state machine correctly if the
desired services are not found.
2020-06-05 14:07:52 +10:00
Jim Mussared e6881f0829 extmod/modbluetooth: Make modbluetooth event not a bitfield.
There doesn't appear to be any use for only triggering on specific events,
so it's just easier to number them sequentially.  This makes them smaller
values so they take up only 1 byte in the ringbuf, only 1 byte for the
opcode in the bytecode, and makes room for more events.

Also add a couple of new event types that need to be implemented (to avoid
re-numbering later).

And rename _COMPLETE and _STATUS to _DONE for consistency.

In the future the "trigger" keyword argument can be reinstated by requiring
the user to compute the bitmask, eg:

    ble.irq(handler, 1 << _IRQ_SCAN_RESULT | 1 << _IRQ_SCAN_DONE)
2020-06-05 14:04:20 +10:00
Diego Elio Pettenò dd5d7c86d2 Fix up end of file and trailing whitespace.
This can be enforced by pre-commit, but correct it separately to make it easier to review.
2020-06-03 10:56:35 +01:00
Jeff Epler 7f3fd20ea9 ulab: update submodule again 2020-06-01 11:03:38 -05:00
Jeff Epler 18c659780e ulab: update
.. add new modules and functions to our shared-bindings stubs
2020-06-01 09:02:29 -05:00
Damien George 22806ed5df extmod/vfs: Retain previous working directory if chdir fails.
Fixes issue #6069.
2020-05-29 23:05:01 +10:00
Damien George 7dffbfd22a extmod/vfs_lfsx: Fix import_stat so it takes into account current dir.
CPython semantics require searching the current directory if the import is
not absolute (when "" is in sys.path).

Fixes issue #6037.
2020-05-15 11:31:32 +10:00
Thomas Friebel 18fb5b4432 extmod/nimble: Make error code mapping default to MP_EIO.
Before this change, any NimBLE error that does not appear in the
ble_hs_err_to_errno_table maps to return code 0, meaning success.  If we
miss adding an error code to the table we end up returning success in case
of failure.

Instead, handle the zero case explicitly and default to MP_EIO.  This
allows removing the now-redundant MP_EIO entries from the mapping.
2020-05-11 22:31:30 +10:00
Damien George 3b6c9119eb extmod/modbluetooth: Add support for changing the GAP device name.
This commit allows the user to set/get the GAP device name used by service
0x1800, characteristic 0x2a00.  The usage is:

    BLE.config(gap_name="myname")
    print(BLE.config("gap_name"))

As part of this change the compile-time setting
MICROPY_PY_BLUETOOTH_DEFAULT_NAME is renamed to
MICROPY_PY_BLUETOOTH_DEFAULT_GAP_NAME to emphasise its link to GAP and this
new "gap_name" config value.  And the default value of this for the NimBLE
bindings is changed from "PYBD" to "MPY NIMBLE" to be more generic.
2020-05-11 21:30:41 +10:00
warriorofwire d8491f3176 ujson: back out overeager loads() change; only change load() 2020-05-10 21:56:01 -07:00
warriorofwire eb3d5fa453 ujson: do not eat trailing whitespace
Ujson should only worry about whitespace before JSON.  This becomes apparent when you are using MP stream protocol to read directly from input buffers.

When you attempt to read(1) on a UART (and possibly other protocols) you have to wait for either the byte or the timeout.

Fixes:
- Waiting for a timeout after you have completed reading a correct and complete JSON off the input.
- Raising an OSError after reading a correct and complete JSON off the input.
- Eating more data than semantically owned off the input buffer.
- Blocking to start parsing JSON until the entire JSON body has been loaded into a potentially large, contiguous Python object.

Code you would write before:
```
line = board_busio_uart_port.read_line()
json_dict = json.loads(line)
```
or reaching for fixed buffers and swapping them around in Python.

Code that did not work before that does now:
```
json_dict = json.load(board_busio_uart_port)
```

- This removes the need for intermediate copies of data when reading JSON from micropython stream protocol inputs.
- It also increases total application speed by parsing JSON concurrently with receiving on boards that read from UART via DMA.
- It simplifies code that users write while improving their apps.
2020-05-10 20:45:42 -07:00
robert 0f83ef395c extmod/vfs_lfsx: Fix rename to respect cur dir for new path.
If the new name start with '/', cur_dir is not prepened any more, so that
the current working directory is respected.  And extend the test cases for
rename to cover this functionality.
2020-05-08 21:54:04 +10:00
robert d3ea28d04a extmod/vfs_lfsx: Normalize path name in chdir.
This change scans for '.', '..' and multiple '/' and normalizes the new
path name.  If the resulting path does not exist, an error is raised.
Non-existing interim path elements are ignored if they are removed during
normalization.
2020-05-08 21:52:15 +10:00
robert a5ea4b9f3f extmod/vfs_lfsx: Fix path handling in uos.stat() to consider cur dir.
This fixes the bug, that stat(filename) would not consider the current
working directory.  So if e.g. the cwd is "lib", then stat("main.py") would
return the info for "/main.py" instead of "/lib/main.py".
2020-05-08 21:37:51 +10:00
Damien George 73c58150f5 extmod/modbtree: Retain reference to underlying stream so it's not GC'd.
For ports that have a system malloc which is not garbage collected (eg
unix, esp32), the stream object for the DB must be retained separately to
prevent it from being reclaimed by the MicroPython GC (because the
berkeley-db library uses malloc to allocate the DB structure which stores
the only reference to the stream).

Although in some cases the user code will explicitly retain a reference to
the underlying stream because it needs to call close() on it, this is not
always the case, eg in cases where the DB is intended to live forever.

Fixes issue #5940.
2020-05-02 16:08:04 +10:00
Jim Mussared 2e3c42775a unix: Add btstack to the unix submodules list.
But only when bluetooth is enabled, i.e. if building the dev or coverage
variants, and we have libusb available.

Update travis to match, i.e. specify the variant when doing
`make submodules`.
2020-04-29 16:54:12 +10:00
Jim Mussared ebfd9ff2e6 extmod/modbluetooth: Fix sign compare and unused variable warnings. 2020-04-29 16:54:12 +10:00
Jim Mussared 7563d58210 unix: Add support for modbluetooth and BLE using btstack.
This commit adds full support to the unix port for Bluetooth using the
common extmod/modbluetooth Python bindings.  This uses the libusb HCI
transport, which supports many common USB BT adaptors.
2020-04-29 16:45:46 +10:00
Jim Mussared c987adb9e9 extmod/btstack: Implement more robust init/deinit sequencing. 2020-04-29 16:45:46 +10:00
Jim Mussared 8119ec0765 extmod/modbluetooth: Don't hold atomic section during mp_sched_schedule.
Because, for example, on unix the atomic section isn't re-entrant, and
mp_sched_schedule() will try to re-acquire the atomic section.
2020-04-29 16:45:40 +10:00
Jim Mussared 0da47ecc93 stm32/Makefile: Rename SRC_LIB to LIB_SRC_C to match other ports. 2020-04-29 16:38:18 +10:00
Damien George e08ca78f40 py/stream: Remove mp_stream_errno and use system errno instead.
This change is made for two reasons:

1. A 3rd-party library (eg berkeley-db-1.xx, axtls) may use the system
   provided errno for certain errors, and yet MicroPython stream objects
   that it calls will be using the internal mp_stream_errno.  So if the
   library returns an error it is not known whether the corresponding errno
   code is stored in the system errno or mp_stream_errno.  Using the system
   errno in all cases (eg in the mp_stream_posix_XXX wrappers) fixes this
   ambiguity.

2. For systems that have threading the system-provided errno should always
   be used because the errno value is thread-local.

For systems that do not have an errno, the new lib/embed/__errno.c file is
provided.
2020-04-27 23:58:46 +10:00
Jim Mussared 347c8917dc extmod/nimble: Update to work with NimBLE 1.3. 2020-04-27 22:51:10 +10:00
Jeff Epler 1a6df71cec ulab: update 2020-04-27 07:46:48 -05:00
Jeff Epler c016ea6f0a ulab: actually update the submodule
PR#2802 missed the submodule update itself.
2020-04-26 10:12:56 -05:00
stijn 84fa3312cf all: Format code to add space after C++-style comment start.
Note: the uncrustify configuration is explicitly set to 'add' instead of
'force' in order not to alter the comments which use extra spaces after //
as a means of indenting text for clarity.
2020-04-23 11:24:25 +10:00
stijn 70affd9ba2 all: Fix implicit floating point to integer conversions.
These are found when building with -Wfloat-conversion.
2020-04-18 22:42:24 +10:00
stijn bcf01d1686 all: Fix implicit conversion from double to float.
These are found when building with -Wfloat-conversion.
2020-04-18 22:42:24 +10:00
stijn 0ba68f8a1d all: Fix implicit floating point promotion.
Initially some of these were found building the unix coverage variant on
MacOS because that build uses clang and has -Wdouble-promotion enabled, and
clang performs more vigorous promotion checks than gcc.  Additionally the
codebase has been compiled with clang and msvc (the latter with warning
level 3), and with MICROPY_FLOAT_IMPL_FLOAT to find the rest of the
conversions.

Fixes are implemented either as explicit casts, or by using the correct
type, or by using one of the utility functions to handle floating point
casting; these have been moved from nativeglue.c to the public API.
2020-04-18 22:36:14 +10:00
stijn b909e8b2dd Revert "all: Fix implicit casts of float/double, and signed comparison."
This reverts commit a2110bd3fc.  There's
nothing inherently wrong with it, but upcoming commits will apply similar
fixes in a slightly different way.
2020-04-18 22:36:06 +10:00
Jeff Epler cc21bed0e4 ulab: Get updates from upstream 2020-04-14 10:09:00 -05:00
Damien George 5f0661b4fe extmod/uasyncio: Change cannot to can't in error message, and test exp.
Follow up to 8e048d2548 which missed these.
2020-04-14 21:51:25 +10:00
Jeff Epler ca97964701 ulab: Get updates from upstream 2020-04-13 19:00:16 -05:00
Damien George 8e048d2548 all: Clean up error strings to use lowercase and change cannot to can't.
Now that error string compression is supported it's more important to have
consistent error string formatting (eg all lowercase English words,
consistent contractions).  This commit cleans up some of the strings to
make them more consistent.
2020-04-13 22:19:37 +10:00
Damien George db137e70dc extmod/uasyncio: Add Loop.new_event_loop method.
This commit adds Loop.new_event_loop() which is used to reset the singleton
event loop.  This functionality is put here instead of in Loop.close() to
make it possible to write code that is compatible with CPython.
2020-04-13 22:16:52 +10:00
Damien George c5a21a94f8 extmod/modbluetooth: Provide FLAG_WRITE_NO_RESPONSE for characteristics.
This flag is supported and needs to be set if characteristics are write-
without-response.
2020-04-07 13:46:56 +10:00
Damien George 899e89d4c6 extmod/btstack: Pass through SCAN_RSP events.
The latest version of BTstack has a bug fixed so that it correctly
configures scan parameters if they are set right after activating the
stack.  This means that BLE.gap_scan() will correctly set the scanning to
passive and so SCAN_RSP events are not passed through, so we don't need to
explicitly filter them in our bindings.
2020-04-07 13:46:56 +10:00
Jim Mussared def76fe4d9 all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
Jim Mussared 1921224272 extmod/modubinascii: Make code private and module self-contained.
This commit makes all functions and function wrappers in modubinascii.c
STATIC and conditional on the MICROPY_PY_UBINASCII setting, which will
exclude the file from qstr/ compressed-string searching when ubinascii is
not enabled.  The now-unused modubinascii.h header file is also removed.

The cc3200 port is updated accordingly to use this module in its entirety
instead of providing its own top-level definition of ubinascii.

This was originally like this because the cc3200 port has its own ubinascii
module which referenced these methods.  The plan appeared to be that the
API might diverge (e.g. hardware crc), but this should be done similar to
I2C/SPI via a port-specific handler, rather than the port having its own
definition of the module.  Having a centralised module definition also
enforces consistency of the API among ports.
2020-04-05 14:13:53 +10:00
Kevin Köck 15f41c2dbf extmod/uasyncio: Add global exception handling methods.
This commit adds support for global exception handling in uasyncio
according to the CPython error handling:
https://docs.python.org/3/library/asyncio-eventloop.html#error-handling-api

This allows a program to receive exceptions from detached tasks and log
them to an appropriate location, instead of them being printed to the REPL.

The implementation preallocates a context dictionary so in case of an
exception there shouldn't be any RAM allocation.

The approach here is compatible with CPython except that in CPython the
exception handler is called once the task that threw an uncaught exception
is freed, whereas in MicroPython the exception handler is called
immediately when the exception is thrown.
2020-04-04 10:37:00 +11:00
Damien George f97b5395ed extmod/uasyncio: Add StreamReader/StreamWriter as aliases of Stream cls.
To be compatible with CPython.  Fixes issue #5847.
2020-04-02 00:51:00 +11:00
Kevin Köck aca19c25d2 extmod/uasyncio: Add error message to Lock.release's RuntimeError.
Otherwise it can be hard to understand what the error is if a blank
RuntimeError is raised.
2020-04-02 00:40:23 +11:00
Damien George b389bc0afa extmod/uasyncio: Implement Loop.stop() to stop the event loop. 2020-04-02 00:14:18 +11:00
Damien George 711dd392d3 extmod/uasyncio: Don't create a Loop instance in get_event_loop().
The event loop is (for now) just a singleton so make it so that Loop
instances are not needed.
2020-04-01 23:56:31 +11:00
David Lechner a2110bd3fc all: Fix implicit casts of float/double, and signed comparison.
These were found by buiding the unix coverage variant on macOS (so clang
compiler).  Mostly, these are fixing implicit cast of float/double to
mp_float_t which is one of those two and one mp_int_t to size_t fix for
good measure.
2020-03-30 12:04:21 +11:00
Damien George 1a3e386c67 all: Remove spaces inside and around parenthesis.
Using new options enabled in the uncrustify configuration.
2020-03-28 23:36:44 +11:00
David Lechner 9418611c8a unix: Implement PEP 475 to retry syscalls failing with EINTR.
https://www.python.org/dev/peps/pep-0475/

This implements something similar to PEP 475 on the unix port, and for the
VfsPosix class.

There are a few differences from the CPython implementation:
- Since we call mp_handle_pending() between any ENITR's, additional
  functions could be called if MICROPY_ENABLE_SCHEDULER is enabled, not
  just signal handlers.
- CPython only handles signal on the main thread, so other threads will
  raise InterruptedError instead of retrying.  On MicroPython,
  mp_handle_pending() will currently raise exceptions on any thread.

A new macro MP_HAL_RETRY_SYSCALL is introduced to reduce duplicated code
and ensure that all instances behave the same.  This will also allow other
ports that use POSIX-like system calls (and use, eg, VfsPosix) to provide
their own implementation if needed.
2020-03-27 14:40:46 +11:00
Damien George 3b68f36175 extmod/uasyncio: Add manifest.py for freezing uasyncio Py files. 2020-03-26 01:25:45 +11:00
Damien George bc009fdd62 extmod/uasyncio: Add optional implementation of core uasyncio in C.
Implements Task and TaskQueue classes in C, using a pairing-heap data
structure.  Using this reduces RAM use of each Task, and improves overall
performance of the uasyncio scheduler.
2020-03-26 01:25:45 +11:00
Damien George 63b9944382 extmod/uasyncio: Add new implementation of uasyncio module.
This commit adds a completely new implementation of the uasyncio module.
The aim of this version (compared to the original one in micropython-lib)
is to be more compatible with CPython's asyncio module, so that one can
more easily write code that runs under both MicroPython and CPython (and
reuse CPython asyncio libraries, follow CPython asyncio tutorials, etc).
Async code is not easy to write and any knowledge users already have from
CPython asyncio should transfer to uasyncio without effort, and vice versa.

The implementation here attempts to provide good compatibility with
CPython's asyncio while still being "micro" enough to run where MicroPython
runs. This follows the general philosophy of MicroPython itself, to make it
feel like Python.

The main change is to use a Task object for each coroutine.  This allows
more flexibility to queue tasks in various places, eg the main run loop,
tasks waiting on events, locks or other tasks.  It no longer requires
pre-allocating a fixed queue size for the main run loop.

A pairing heap is used to queue Tasks.

It's currently implemented in pure Python, separated into components with
lazy importing for optional components.  In the future parts of this
implementation can be moved to C to improve speed and reduce memory usage.
But the aim is to maintain a pure-Python version as a reference version.
2020-03-26 01:25:45 +11:00
David Lechner 58d9a4815d extmod/vfs_posix_file: Include unistd.h to get STD{IN,OUT,ERR}_FILENO. 2020-03-25 00:59:39 +11:00
Damien George 2cdf1d25f5 unix: Remove custom file implementation to use extmod's VFS POSIX one.
The implementation in extmod/vfs_posix_file.c is now equivalent to that in
ports/unix/file.c, so remove the latter and use the former instead.
2020-03-18 21:01:07 +11:00
Damien George 68b1bc2042 extmod/vfs_posix_file: Lock GIL when writing and allow stdio flush.
Also support MP_STREAM_GET_FILENO ioctl.  The stdio flush change was done
previously for the unix port in 3e0b46b9af.
These changes make this POSIX file implementation equivalent to the unix
file implementation.
2020-03-18 21:01:07 +11:00
Damien George ad9a0ec8ab all: Convert exceptions to use mp_raise_XXX helpers in remaining places. 2020-03-18 17:26:19 +11:00
Damien George 8f0778b209 extmod/modlwip: Properly handle non-blocking and timeout on UDP recv.
Fixes UDP non-blocking recv so it returns EAGAIN instead of ETIMEDOUT.
Timeout waiting for incoming data is also improved by replacing 100ms delay
with poll_sockets(), as is done in other parts of this module.

Fixes issue #5759.
2020-03-18 10:51:32 +11:00
Damien George 00267aae0b extmod/modlwip: Fix polling of UDP socket so it doesn't return HUP.
STATE_NEW will return HUP when polled so put active UDP sockets into a new
state which is different to STATE_NEW.

Fixes issue #5758.
2020-03-18 10:49:27 +11:00
Jeff Epler 772ac705d3 update ulab 2020-03-17 08:32:52 -05:00
Damien George ed848553b4 extmod/vfs: Factor out vfs mount-and-chdir helper from stm32. 2020-03-11 14:24:26 +11:00
Damien George 7bf62562ce extmod/nimble: When getting BLE MAC try public address if random fails.
This is needed for BLE.config('mac') to work on esp32.
2020-03-11 14:02:13 +11:00
Damien George dd0bc26e65 extmod/modbluetooth: Change scan result's "connectable" to "adv_type".
This commit changes the BLE _IRQ_SCAN_RESULT data from:

    addr_type, addr, connectable, rssi, adv_data

to:

    addr_type, addr, adv_type, rssi, adv_data

This allows _IRQ_SCAN_RESULT to handle all scan result types (not just
connectable and non-connectable passive scans), and to distinguish between
them using adv_type which is an integer taking values 0x00-0x04 per the BT
specification.

This is a breaking change to the API, albeit a very minor one: the existing
connectable value was a boolean and True now becomes 0x00, False becomes
0x02.

Documentation is updated and a test added.

Fixes #5738.
2020-03-11 14:00:44 +11:00
Thomas Friebel 41a9f1dec8 extmod/modbluetooth: Unify error handling in remaining places.
Most error handling is done via `bluetooth_handle_errno()` already.
Replace the remaining few manual checks with calls to that function.
2020-03-11 13:01:35 +11:00
Jim Mussared 1937fb22ab extmod/nimble: Clarify active state and check for active in all methods.
This commit ensures that the BLE stack is active before allowing operations
that may otherwise crash if it's not active.  It also clarifies the state
better (adding the "stopping" state) and renames mp_bluetooth_is_enabled to
the more self-explanatory mp_bluetooth_is_active.
2020-03-11 13:01:35 +11:00
Jeff Epler eab9b670d8 ulab: Use https://github.com/v923z/micropython-ulab/
The only delta we were carrying was in the README.
2020-03-09 14:48:52 -05:00
Damien George e965363b6b stm32: Refactor Bluetooth HCI RX to be independent of transport layer.
Now all HCI specific code (eg UART vs WB55 internal messaging) is confined
to modbluetooth_hci.c.
2020-03-10 01:53:42 +11:00
Damien George d7259f6b1c extmod/btstack: Implement notifications/indications for GATT clients.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George 018ce122ca extmod/btstack: Implement scan and gatt client, connect and disconnect.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George 372e5a280e extmod/btstack: Implement gatts_db for btstack.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George 86c26db3ff extmod/btstack: Implement service registration.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George 0674917bc5 extmod/btstack: Implement advertising.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George fbefcef1df extmod/btstack: Add empty modbluetooth implementation.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George 0ac06a510a extmod/modbluetooth: Extract out gatts_db functionality from nimble.
For use by other stacks, if they need it.

Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George 894c550c86 stm32: Refactor bluetooth stack/hci/driver bindings.
This makes a cleaner separation between the: driver, HCI UART and BT stack.
Also updated the naming to be more consistent (mp_bluetooth_hci_*).

Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George c44d52f33e extmod/modbluetooth_nimble: Move nimble specific code, factor nimble.mk.
Move extmod/modbluetooth_nimble.* to extmod/nimble.  And move common
Makefile lines to extmod/nimble/nimble.mk (which was previously only used
by stm32).  This allows (upcoming) btstack to follow a similar structure.

Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-06 12:35:20 +11:00
Damien George 69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
Jeff Epler 39cfe32c34 Update ulab from upstream again 2020-02-27 14:14:05 -06:00
Jeff Epler fa3b9eba92 ulab: Incorporate it 2020-02-27 11:03:03 -06:00
Jim Mussared f8449dd092 extmod/modframebuf: Allow blit source to be a subclass of FrameBuffer. 2020-02-21 13:32:48 +11:00
Jim Mussared cddb90e8b0 extmod/modbluetooth_nimble: Fix wrong offset used for descriptor flags. 2020-02-18 16:37:38 +11:00
Jim Mussared 66ac2e1fc0 extmod/modbluetooth.h: Fix typo in comment about registering services. 2020-02-18 16:36:37 +11:00
Thomas Friebel f4726735cf extmod/modbluetooth: Implement config getter for BLE rxbuf size.
Knowing the buffer size can be important, to ensure that valid data will be
received.
2020-02-18 13:37:50 +11:00
Damien George ce39c958ef py: Factor out definition of mp_float_union_t to one location. 2020-02-18 13:04:36 +11:00
Dan Halbert e0753c4ad2 avoid os.stat() int overflow on smallint-only builds 2020-02-14 18:33:37 -05:00
Damien George ad7213d3c3 py: Add mp_raise_msg_varg helper and use it where appropriate.
This commit adds mp_raise_msg_varg(type, fmt, ...) as a helper for
nlr_raise(mp_obj_new_exception_msg_varg(type, fmt, ...)).  It makes the
C-level API for raising exceptions more consistent, and reduces code size
on most ports:

   bare-arm:   +28 +0.042%
minimal x86:  +100 +0.067%
   unix x64:   -56 -0.011%
unix nanbox:  -300 -0.068%
      stm32:  -204 -0.054% PYBV10
     cc3200:    +0 +0.000%
    esp8266:   -64 -0.010% GENERIC
      esp32:  -104 -0.007% GENERIC
        nrf:  -136 -0.094% pca10040
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2020-02-13 11:52:40 +11:00
Damien George 97eca38c4f py: Add mp_raise_type helper macro and use it where appropriate.
This provides a more consistent C-level API to raise exceptions, ie moving
away from nlr_raise towards mp_raise_XXX.  It also reduces code size by a
small amount on some ports.
2020-02-13 11:03:37 +11:00
Dan Halbert 5164719e67 preserve errno in OSError 2020-02-04 16:19:40 -05:00
Dan Halbert 57b566e736 Include filename for 'No such file/directory', etc. 2020-02-04 13:32:39 -05:00
Scott Shawcroft f6a635b102
Fix subclassing of objects that are tested. Others may still be broken. 2020-01-27 14:52:42 -08:00
David Lechner 35f66d38b8 extmod/vfs_posix: Release GIL during system calls.
This releases the GIL during syscalls that could block.
2020-01-26 23:26:14 +11:00
Damien George bfbd94401d py: Make mp_obj_get_type() return a const ptr to mp_obj_type_t.
Most types are in rodata/ROM, and mp_obj_base_t.type is a constant pointer,
so enforce this const-ness throughout the code base.  If a type ever needs
to be modified (eg a user type) then a simple cast can be used.
2020-01-09 11:25:26 +11:00
Roy Hooper 767ce1cdf8 remove unnecessary GCC pragmas 2020-01-02 18:03:18 -05:00
Roy Hooper ccf158b030 raise mp_raise_NotImplementedError 2020-01-02 18:00:36 -05:00
Yonatan Goldschmidt aca8873bb8 extmod/modbluetooth: Fix func prototype, empty args should be (void).
This fixes a -Wstrict-prototypes error.
2019-12-27 23:57:46 +11:00
Damien George 6f872f81d6 extmod: Fix modbluetooth and modwebrepl to build in nanbox mode. 2019-12-27 22:54:53 +11:00
Damien George d97b40bdaa py: Introduce MP_ROM_FALSE/MP_ROM_TRUE for ROM to refer to bool objects.
This helps to prevent mistakes, and allows easily changing the ROM value of
False/True if needed.
2019-12-27 22:54:20 +11:00
Damien George 09376f0e47 py: Introduce MP_ROM_NONE macro for ROM to refer to None object.
This helps to prevent mistakes, and allows easily changing the ROM value of
None if needed.
2019-12-27 22:51:17 +11:00
Damien George ed2be79b49 extmod/uzlib: Explicitly cast ptr-diff-expr to unsigned.
The struct member "dest" should never be less than "destStart", so their
difference is never negative.  Cast as such to make the comparison
explicitly unsigned, ensuring the compiler produces the correct comparison
instruction, and avoiding any compiler warnings.
2019-12-23 00:07:03 +11:00
Jim Mussared 7ce1e0b1dc extmod/webrepl: Move webrepl scripts to common place and use manifest.
Move webrepl support code from ports/esp8266/modules into extmod/webrepl
(to be alongside extmod/modwebrepl.c), and use frozen manifests to include
it in the build on esp8266 and esp32.

A small modification is made to webrepl.py to make it work on non-ESP
ports, i.e. don't call dupterm_notify if not available.
2019-12-20 12:59:13 +11:00
Roy Hooper dc8dd6df20 Revert subscr signature change 2019-12-13 14:29:15 -05:00
Roy Hooper 15072c69c9 push changes 2019-12-13 14:07:53 -05:00
Damien George ba84453f77 examples/natmod: Add urandom native module example. 2019-12-13 13:33:40 +11:00
Damien George bbeaafd9aa extmod: Add dynamic-runtime guards to btree/framebuf/uheapq/ure/uzlib.
So they can be built as dynamic native modules, as well as existing static
native modules.
2019-12-12 20:15:28 +11:00
Damien George e5acd06ad5 extmod/modbtree: Use mp_printf instead of printf. 2019-12-12 20:15:28 +11:00
Roy Hooper 0326c98fd5 Merge branch 'master' into new-pixelbuf-api 2019-12-10 20:44:43 -05:00
Damien George 84958a8fe1 extmod/modbluetooth: Allow setting ringbuf size via BLE.config(rxbuf=).
The size of the event ringbuf was previously fixed to compile-time config
value, but it's necessary to sometimes increase this for applications that
have large characteristic buffers to read, or many events at once.

With this commit the size can be set via BLE.config(rxbuf=512), for
example.  This also resizes the internal event data buffer which sets the
maximum size of incoming data passed to the event handler.
2019-12-05 11:30:35 +11:00
Jeff Epler 238e121236 protocols: Allow them to be (optionally) type-safe
Protocols are nice, but there is no way for C code to verify whether
a type's "protocol" structure actually implements some particular
protocol.  As a result, you can pass an object that implements the
"vfs" protocol to one that expects the "stream" protocol, and the
opposite of awesomeness ensues.

This patch adds an OPTIONAL (but enabled by default) protocol identifier
as the first member of any protocol structure.  This identifier is
simply a unique QSTR chosen by the protocol designer and used by each
protocol implementer.  When checking for protocol support, instead of
just checking whether the object's type has a non-NULL protocol field,
use `mp_proto_get` which implements the protocol check when possible.

The existing protocols are now named:
    protocol_framebuf
    protocol_i2c
    protocol_pin
    protocol_stream
    protocol_spi
    protocol_vfs
(most of these are unused in CP and are just inherited from MP; vfs and
stream are definitely used though)

I did not find any crashing examples, but here's one to give a flavor of what
is improved, using `micropython_coverage`.  Before the change,
the vfs "ioctl" protocol is invoked, and the result is not intelligible
as json (but it could have resulted in a hard fault, potentially):

    >>> import uos, ujson
    >>> u = uos.VfsPosix('/tmp')
    >>> ujson.load(u)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: syntax error in JSON

After the change, the vfs object is correctly detected as not supporting
the stream protocol:
    >>> ujson.load(p)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    OSError: stream operation not supported
2019-12-04 09:29:57 -06:00
Damien George 7aeafe2ae9 extmod/modbluetooth: Add optional 4th arg to gattc_write for write mode.
This allows the user to explicitly select the behaviour of the write to the
remote peripheral.  This is needed for peripherals that have
characteristics with WRITE_NO_RESPONSE set (instead of normal WRITE).  The
function's signature is now:

    BLE.gattc_write(conn_handle, value_handle, data, mode=0)

mode=0 means write without response, while mode=1 means write with
response.  The latter was the original behaviour so this commit is a change
in behaviour of this method, and one should specify 1 as the 4th argument
to get back the old behaviour.

In the future there could be more modes supported, such as long writes.
2019-12-04 23:23:07 +11:00
Damien George 40cc7ec677 stm32/mpconfigport.h: Use IRQ_PRI_PENDSV to protect bluetooth ringbuf.
The default protection for the BLE ringbuf is to use
MICROPY_BEGIN_ATOMIC_SECTION, which disables all interrupts. On stm32 it
only needs to disable the lowest priority IRQ, pendsv, because that's the
IRQ level at which the BLE stack is driven.
2019-12-04 13:39:22 +11:00
Damien George 8ce69288e9 extmod/modbluetooth: Remove limit on data coming from gattc data input.
This removes the limit on data coming in from a BLE.gattc_read() request,
or a notify with payload (coming in to a central).  In both cases the data
coming in to the BLE callback is now limited only by the available data in
the ringbuf, whereas before it was capped at (default hard coded) 20 bytes.
2019-12-02 23:27:25 +11:00
Damien George d6e051082a extmod/modbluetooth: Simplify how BLE IRQ callback is scheduled.
Instead of enqueue_irq() inspecting the ringbuf to decide whether to
schedule the IRQ callback (if ringbuf is empty), maintain a flag that knows
if the callback is on the schedule queue or not.  This saves about 150
bytes of code (for stm32 builds), and simplifies all uses of enqueue_irq()
and schedule_ringbuf().
2019-12-02 23:25:36 +11:00
Roy Hooper 56720eae0a remove unnecessary intermediate mp_obj_subscr wrapper 2019-11-26 18:39:08 -05:00
Damien George 6b3404f25e extmod/vfs_lfs: Fix bug when passing no args to constructor and mkfs. 2019-11-26 00:08:57 +11:00
Damien George 5634a31a98 extmod/vfs_lfs: Pass flag along to ioctl when init'ing bdev for lfs.
To hint to the block device that the extended block protocol will be used.
2019-11-26 00:07:42 +11:00
Jim Mussared e873d352ad extmod/modbluetooth: Simplify management of pre-allocated event data.
The address, adv payload and uuid fields of the event are pre-allocated by
modbluetooth, and reused in the IRQ handler.  Simplify this and move all
storage into the `mp_obj_bluetooth_ble_t` instance.

This now allows users to hold on to a reference to these instances without
crashes, although they may be overwritten by future events.  If they want
to hold onto the values longer term they need to copy them.
2019-11-25 17:32:10 +11:00
Jim Mussared 438c0dc2a4 extmod/modbluetooh_nimble: Fix UUID conversion for 16 and 32 bit values. 2019-11-25 17:31:59 +11:00
Jim Mussared 2ae755d9e1 extmod/modbluetooth_nimble: Make gap_scan_stop no-op if no scan ongoing.
No need for this to throw an exception if the intent (don't be scanning) is
clear, and avoids a race with the scan duration timeout.
2019-11-25 17:27:40 +11:00
Jim Mussared d19c6d0519 extmod/modbluetooth: Create UUID from bytes and allow comparison ops.
This allows construction of UUID objects from advertising data payloads and
matching against known UUIDs.
2019-11-25 17:20:51 +11:00
Roy Hooper a9baa441c9 disable -Wunused-parameter on subscr functions 2019-11-23 13:52:14 -05:00
Roy Hooper 6108fa3766 Merge branch 'master' into new-pixelbuf-api 2019-11-23 12:22:04 -05:00
Roy Hooper c770ccd939 make ->subscr take an instance to pass when instance_subscr is called from subscr. 2019-11-23 12:09:26 -05:00
Jim Mussared 334ba01c90 extmod/modbluetooth: Prioritise non-scan-result events.
Remove existing scan result events from the ringbuf if the ringbuf is full
and we're trying to enqueue any other event.  This is needed so that events
such as SCAN_COMPLETE are always put on the ringbuf.
2019-11-21 12:04:57 +11:00
Damien George 799b6d1e0c extmod: Consolidate FAT FS config to MICROPY_VFS_FAT across all ports.
This commit removes the Makefile-level MICROPY_FATFS config and moves the
MICROPY_VFS_FAT config to the Makefile level to replace it.  It also moves
the include of the oofatfs source files in the build from each port to a
central place in extmod/extmod.mk.

For a port to enabled VFS FAT support it should now set MICROPY_VFS_FAT=1
at the level of the Makefile.  This will include the relevant oofatfs files
in the build and set MICROPY_VFS_FAT=1 at the C (preprocessor) level.
2019-11-11 11:37:38 +11:00
Damien George 9b27069e2f extmod/vfs: Add autodetect of littlefs filesystem when mounting. 2019-11-06 12:15:34 +11:00
Damien George c13f9f209d all: Convert nlr_raise(mp_obj_new_exception_msg(x)) to mp_raise_msg(x).
This helper function was added a while ago and these are the remaining
cases to convert, to save a bit of code size.
2019-11-05 11:35:45 +11:00
Damien George 2ee9e1a4ed extmod/modbtree: Make FILEVTABLE const to put it in ROM. 2019-11-01 17:25:40 +11:00
Damien George 40ea1915fc extmod/nimble: Factor out stm32-specific HCI UART RX/TX code. 2019-11-01 12:41:37 +11:00
Damien George 78145b98ef extmod/nimble: Remove unneeded nimble_sprintf wrapper function. 2019-11-01 12:15:07 +11:00
Damien George 9dd9f9ff06 extmod/modussl_mbedtls: Check for invalid key/cert data. 2019-10-31 16:22:42 +11:00
Damien George 07ea81fbc5 extmod/modussl_mbedtls: Fix getpeercert to return None if no cert avail. 2019-10-31 13:42:24 +11:00
Damien George 26d8fd2c0a extmod/modlwip: Unconditionally return POLLHUP/POLLERR when polling.
POSIX poll should always return POLLERR and POLLHUP in revents, regardless
of whether they were requested in the input events flags.

See issues #4290 and #5172.
2019-10-31 13:37:51 +11:00
Damien George feaa251674 extmod/modlwip: Make socket poll return POLLNVAL in case of bad file. 2019-10-31 12:54:37 +11:00
Damien George 71401d5065 extmod/modlwip: Unconditionally return POLLHUP when polling new socket.
POSIX poll should always return POLLERR and POLLHUP in revents, regardless
of whether they were requested in the input events flags.

See issues #4290 and #5172.
2019-10-31 12:54:37 +11:00
Damien George 660a61a388 extmod/vfs_lfs: Allow compiling in VfsLfs1 and VfsLfs2 separately.
These classes are enabled via the config options MICROPY_VFS_LFS1 and
MICROPY_VFS_LFS2, which are disabled by default.
2019-10-30 12:08:58 +11:00
Jim Mussared d16a27da51 extmod/modbluetooth: Add gatts_set_buffer.
- Adds an explicit way to set the size of a value's internal buffer,
  replacing `ble.gatts_write(handle, bytes(size))` (although that
  still works).
- Add an "append" mode for values, which means that remote writes
  will append to the buffer.
2019-10-29 23:09:02 +11:00
Damien George 4cf054a130 extmod/vfs: Add MP_BLOCKDEV_IOCTL_BLOCK_ERASE constant. 2019-10-29 14:17:29 +11:00
Damien George cfe1c5abf8 extmod/vfs: Rename BP_IOCTL_xxx constants to MP_BLOCKDEV_IOCTL_xxx.
Also rename SEC_COUNT to BLOCK_COUNT and SEC_SIZE to BLOCK_SIZE.
2019-10-29 14:17:29 +11:00
Damien George a099505420 extmod: Add VFS littlefs bindings.
Both LFS1 and LFS2 are supported at the same time.
2019-10-29 14:17:29 +11:00
Damien George 98beea9ced extmod/vfs_blockdev: Add extended read/write methods.
This commit adds helper functions to call readblocks/writeblocks with a
fourth argument, the byte offset within a block.

Although the mp_vfs_blockdev_t struct has grown here by 2 machine words, in
all current uses of this struct within this repository it still fits within
the same number of GC blocks.
2019-10-29 12:55:34 +11:00
Damien George e1c7b1cb43 extmod/vfs_blockdev: Factor out block device interface code. 2019-10-29 12:55:17 +11:00
Damien George 9aabb6c01b extmod: Factor out block-device struct to make independent of fatfs. 2019-10-29 12:12:37 +11:00
Jim Mussared b02d7e612d extmod/modbluetooth: Rename module to "ubluetooth".
For consistency with "umachine". Now that weak links are enabled
by default for built-in modules, this should be a no-op, but allows
extension of the bluetooth module by user code.

Also move registration of ubluetooth to objmodule rather than
port-specific.
2019-10-22 21:58:05 +11:00
Jim Mussared 2c1f269918 extmod/modbluetooth_nimble: Use `data_alloc` length to truncate writes.
This allows the maximum size of a characteristic/descriptor to be
increased by locally writing to it first.
2019-10-22 14:30:23 +11:00
Jim Mussared f34e16dbc6 extmod/modbluetooth: Persist reference to NimBLE service instances.
NimBLE doesn't actually copy this data, it requires it to stay live.
Only dereference when we register a new set of services.

Fixes #5226

This will allow incrementally adding services in the future, so
rename `reset` to `append` to make it clearer.
2019-10-22 14:30:23 +11:00
Jim Mussared 56fc3edf98 extmod/modbluetooth: Make UUID support the buffer protocol.
Internally change the representation of UUIDs to LE uint8* to simplify this.

This allows UUIDs to be easily used in BLE payloads (such as advertising).

Ref: #5186
2019-10-22 13:54:05 +11:00
Jim Mussared ebf8332104 extmod/re1.5: Support escaping within RE classes.
Fixes issues #3178 and #5220.

Tests are added, including all the cases mentioned in both bugs.
2019-10-18 12:20:32 +11:00
Damien George 8f7f671236 extmod/modbluetooth: In gap_advertise only accept None to stop adv.
To match the docs, and interval=0 may be used in the future to indicate
something else.
2019-10-15 17:29:27 +11:00
Jim Mussared 36502bdfdc extmod/modbluetooth: Make gap_disconnect not raise when disconnected.
Previously it raised OSError(MP_ENOTCONN) if the conn_handle was already
disconnected.  Now it returns True/False.
2019-10-15 17:22:53 +11:00
Jim Mussared 423e67d0a0 extmod/modbluetooth: Improve ringbuf handling.
No need to share the irq_data buffer with addresses.  Split them into two
separate buffers and manage their max length independently.
2019-10-15 17:22:07 +11:00
Jim Mussared cb73103f57 extmod/modbluetooth: Fix order of params to IRQ_GATTS_WRITE event. 2019-10-15 17:22:01 +11:00
Jim Mussared ba16a22991 extmod/modbluetooth: Clear gap_advertise payload when data is empty.
Also fix default adv interval to 500ms.
2019-10-15 17:21:17 +11:00
Jim Mussared 418f12c5f5 extmod/modbluetooth: Increase maximum connections from 1 to 4.
This avoids a confusing ENOMEM raised from gap_advertise if there is
currently an active connection.  This refers to the static connection
buffer pre-allocated by Nimble (nothing to do with MicroPython heap
memory).
2019-10-15 17:20:25 +11:00
Dan Halbert e89bcce711 Move font dependency to extmod from stm32 2019-10-13 19:33:03 -04:00
Jim Mussared 76f474129e extmod/modbluetooth: Use us instead of ms for advertising interval.
This is to more accurately match the BLE spec, where intervals are
configured in units of channel hop time (625us). When it was
specified in ms, not all "valid" intervals were able to be
specified.

Now that we're also allowing configuration of scan interval, this
commit updates advertising to match.
2019-10-11 13:51:07 +11:00
Jim Mussared b65cc387cd extmod/modbluetooth: Allow config of scan interval/window.
This adds two additional optional kwargs to `gap_scan()`:
  - `interval_us`: How long between scans.
  - `window_us`: How long to scan for during a scan.

The default with NimBLE is a 11.25ms window with a 1.28s interval.

Changing these parameters is important for detecting low-frequency
advertisements (e.g. beacons).

Note: these params are in microseconds, not milliseconds in order
to allow the 625us granularity offered by the spec.
2019-10-11 13:51:07 +11:00
Jim Mussared 902bb4ceae stm32: Extract port-specific Nimble implementation.
On other ports (e.g. ESP32) they provide a complete Nimble implementation
(i.e. we don't need to use the code in extmod/nimble). This change
extracts out the bits that we don't need to use in other ports:
 - malloc/free/realloc for Nimble memory.
 - pendsv poll handler
 - depowering the cywbt

Also cleans up the root pointer management.
2019-10-08 14:40:35 +11:00
Andrew Leech eb1b6858a2 extmod/modbluetooth: Allow MP_BLUETOOTH_MAX_ATTR_SIZE in board config. 2019-10-01 09:51:02 +10:00
Andrew Leech 5dc592d117 extmod/modbluetooth_nimble: Use random addr if public isn't available. 2019-10-01 09:51:02 +10:00
Jim Mussared 497dae45e7 extmod/modbluetooth_nimble: Implement modbluetooth API with Nimble. 2019-10-01 09:51:02 +10:00
Jim Mussared 16f8ceeaaa extmod/modbluetooth: Add low-level Python BLE API. 2019-10-01 09:51:02 +10:00
Damien George 07f6644a38 extmod/nimble: Add nimble bindings. 2019-10-01 09:51:02 +10:00
Bob Fanger bff2771da1 extmod/vfs_posix: Include stdio.h for declaration of function 'rename'. 2019-09-23 15:14:42 +10:00
Damien George 24c3e9b283 py/modstruct: Fix struct.pack_into with unaligned offset of native type.
Following the same fix for unpack.
2019-09-02 13:14:16 +10:00