Commit Graph

1092 Commits

Author SHA1 Message Date
Jim Mussared 0bc2c1c105 extmod/modbluetooth: Fix race between READ_REQUEST and other IRQs.
The READ_REQUEST callback is handled as a hard interrupt (because the BLE
stack needs an immediate response from it so it can continue) and so calls
to Python require extra protection:

- the caller-owned tuple passed into the callback must be separate from the
  tuple used by other callback events (which are soft interrupts);

- the GC and scheduler must be locked during callback execution.
2020-08-26 14:57:36 +10:00
Jim Mussared 3d9a7ed02f extmod/btstack: Implement GAP scan duration_ms parameter.
This commit makes scanning work when duration_ms is set to zero.  Prior to
this it would not work with duration_ms set to zero.
2020-08-26 14:55:52 +10:00
Damien George 2acc087880 extmod/vfs_lfs: Add mtime support to littlefs files.
This commit adds support for modification time of files on littlefs v2
filesystems, using file attributes.  For some background see issue #6114.

Features/properties of this implementation:
- Only supported on littlefs2 (not littlefs1).
- Uses littlefs2's general file attributes to store the timestamp.
- The timestamp is 64-bits and stores nanoseconds since 1970/1/1 (if the
  range to the year 2554 is not enough then additional bits can be added to
  this timestamp by adding another file attribute).
- mtime is enabled by default but can be disabled in the constructor, eg:
  uos.mount(uos.VfsLfs2(bdev, mtime=False), '/flash')
- It's fully backwards compatible, existing littlefs2 filesystems will work
  without reformatting and timestamps will be added transparently to
  existing files (once they are opened for writing).
- Files without timestamps will open correctly, and stat will just return 0
  for their timestamp.
- mtime can be disabled or enabled each mount time and timestamps will only
  be updated if mtime is enabled (otherwise they will be untouched).

Signed-off-by: Damien George <damien@micropython.org>
2020-08-25 17:35:19 +10:00
Damien George 55c76eaac1 extmod/uasyncio: Truncate negative sleeps to 0.
Otherwise a task that continuously awaits on a large negative sleep can
monopolise the scheduler (because its wake time is always less than
everything else in the pairing heap).

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 12:17:06 +10:00
Damien George 60f5b941e0 extmod/vfs_reader: Fix mp_reader_new_file to open file in "rb" mode.
mp_reader_new_file() is used to read in files for importing, either .py or
.mpy files, for the lexer and persistent code loader respectively.  In both
cases the file should be opened in raw bytes mode: the lexer handles
unicode characters itself, and .mpy files contain 8-bit bytes by nature.

Before this commit importing was working correctly because, although the
file was opened in text mode, all native filesystem implementations (POSIX,
FAT, LFS) would access the file in raw bytes mode via mp_stream_rw()
calling mp_stream_p_t.read().  So it was only an issue for non-native
filesystems, such as those implemented in Python.  For Python-based
filesystem implementations, a call to mp_stream_rw() would go via IOBase
and then to readinto() at the Python level, and readinto() is only defined
on files opened in raw bytes mode.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-12 23:40:50 +10:00
Damien George 441460d81f extmod/uasyncio: Add StreamReader.readexactly(n) method.
It raises on EOFError instead of an IncompleteReadError (which is what
CPython does).  But the latter is derived from EOFError so code compatible
with MicroPython and CPython can be written by catching EOFError (eg see
included test).

Fixes issue #6156.

Signed-off-by: Damien George <damien@micropython.org>
2020-07-25 23:10:05 +10:00
Thorsten von Eicken 5264478007 extmod/modussl_mbedtls: Integrate shorter error strings.
The stm32 and esp32 ports now use shorter error strings for mbedtls errors.
Also, MBEDTLS_ERROR_C is enabled on stm32 by default to get these strings.
2020-07-21 00:31:05 +10:00
Thorsten von Eicken 9aa214077e extmod/modussl: Improve exception error messages.
This commit adds human readable error messages when mbedtls or axtls raise
an exception.  Currently often just an EIO error is raised so the user is
lost and can't tell whether it's a cert error, buffer overrun, connecting
to a non-ssl port, etc.  The axtls and mbedtls error raising in the ussl
module is modified to raise:

    OSError(-err_num, "error string")

For axtls a small error table of strings is added and used for the second
argument of the OSErrer.  For mbedtls the code uses mbedtls' built-in
strerror function, and if there is an out of memory condition it just
produces OSError(-err_num).  Producing the error string for mbedtls is
conditional on them being included in the mbedtls build, via
MBEDTLS_ERROR_C.
2020-07-20 23:41:45 +10:00
Jim Mussared 9d823a5d9a extmod/modbluetooth: Add event for "indicate acknowledgement".
This commit adds the IRQ_GATTS_INDICATE_DONE BLE event which will be raised
with the status of gatts_indicate (unlike notify, indications require
acknowledgement).

An example of its use is added to ble_temperature.py, and to the multitests
in ble_characteristic.py.

Implemented for btstack and nimble bindings, tested in both directions
between unix/btstack and pybd/nimble.
2020-07-20 23:26:41 +10:00
Jim Mussared 3c7ca2004c extmod/modbluetooth: Fix so it builds in peripheral-only mode. 2020-07-20 23:25:56 +10:00
Jim Mussared 43ceadac55 extmod/modbluetooth: Ignore unused self_in in ble_gatts_indicate. 2020-07-20 23:25:22 +10:00
Jim Mussared b7698841b2 docs/library: Add gatts_indicate() doc to ubluetooth.rst.
Also clarify behavior of `gatts_notify` and add some TODOs about adding an
event for indication acknowledgement.
2020-07-18 14:34:44 +10:00
Jim Mussared e152d0c197 extmod/btstack: Schedule notify/indicate/write ops for bg completion.
The goal of this commit is to allow using ble.gatts_notify() at any time,
even if the stack is not ready to send the notification right now.  It also
addresses the same issue for ble.gatts_indicate() and ble.gattc_write()
(without response).  In addition this commit fixes the case where the
buffer passed to write-with-response wasn't copied, meaning it could be
modified by the caller, affecting the in-progress write.

The changes are:

- gatts_notify/indicate will now run in the background if the ACL buffer is
  currently full, meaning that notify/indicate can be called at any time.

- gattc_write(mode=0) (no response) will now allow for one outstanding
  write.

- gattc_write(mode=1) (with response) will now copy the buffer so that it
  can't be modified by the caller while the write is in progress.

All four paths also now track the buffer while the operation is in
progress, which prevents the GC free'ing the buffer while it's still
needed.
2020-07-18 14:23:47 +10:00
Thomas Friebel b6146ca1a1 extmod/nimble: Fix attr NULL ptr dereference in ble_gatt_attr_read_cb.
In case of error, NimBLE calls the read callback with attr = NULL.
2020-07-09 22:41:10 +10:00
Damien George 0c77668d11 extmod/vfs_lfs: Fix littlefs bindings to build in nan-box mode.
Signed-off-by: Damien George <damien@micropython.org>
2020-06-25 16:32:48 +10:00
Damien George 7dd480ad55 extmod/moductypes: Use mp_obj_is_dict_or_ordereddict to simplify code.
Signed-off-by: Damien George <damien@micropython.org>
2020-06-24 12:05:00 +10:00
David Lechner 77ed6f69ac tools/uncrustify: Enable more opts to remove space between func and '('.
With only `sp_func_proto_paren = remove` set there are some cases where
uncrustify misses removing a space between the function name and the
opening '('.  This sets all of the related options to `force` as well.
2020-06-19 22:07:32 +10:00
Damien George 026fda605e tools/codeformat.py: Include extmod/{btstack,nimble} in code formatting.
Signed-off-by: Damien George <damien@micropython.org>
2020-06-18 22:20:20 +10:00
jp-96 3705bc418c extmod/modbluetooth: Register default GATT service and fix esp32 init.
This is for the NimBLE bindings, to make sure the default GATT service
appears and that the esp32 initialises NimBLE correctly (it now matches
stm32).
2020-06-10 22:33:29 +10:00
Damien George a4c96fb3b0 extmod/uasyncio: Add asyncio.wait_for_ms function.
Fixes issue #6107.
2020-06-10 22:29:44 +10:00
stijn 51fd6c9777 extmod/ure: Use single function for match/search/sub.
Saves about 500 bytes on unix x64 and enables CPython-conform
usage of passing a re object to these functions.
2020-06-08 09:16:09 +02:00
Jim Mussared 8b7ae4e099 extmod/modbluetooth: Support bigger characteristic values.
The ring buffer previously used a single unsigned byte field to save the
length, meaning that it would overflow for large characteristic value
responses.

With this commit it now use a 16-bit length instead and has code to
explicitly truncate at UINT16_MAX (although this should be impossible to
achieve in practice).
2020-06-05 14:11:46 +10:00
Jim Mussared 1cad63c0bc extmod/modbluetooth: Ensure status=0 always on success.
This commit makes sure that all discovery complete and read/write status
events set the status to zero on success.

The status value will be implementation-dependent on non-success cases.
2020-06-05 14:11:04 +10:00
Jim Mussared c07ea3e4c2 extmod/modbluetooth: Implement read done event.
On btstack there's no status associated with the read result, it comes
through as a separate event.  This allows you to detect read failures or
timeouts.
2020-06-05 14:08:15 +10:00
Jim Mussared 919d640aec extmod/modbluetooth: Allow discovery of svc/char by uuid.
In most situations this is a more efficient way of going straight to the
service and characteristic you need.
2020-06-05 14:08:07 +10:00
Jim Mussared 6a3c89d584 extmod/modbluetooth: Add discover complete events for svc/char/desc.
Without this it's difficult to implement a state machine correctly if the
desired services are not found.
2020-06-05 14:07:52 +10:00
Jim Mussared e6881f0829 extmod/modbluetooth: Make modbluetooth event not a bitfield.
There doesn't appear to be any use for only triggering on specific events,
so it's just easier to number them sequentially.  This makes them smaller
values so they take up only 1 byte in the ringbuf, only 1 byte for the
opcode in the bytecode, and makes room for more events.

Also add a couple of new event types that need to be implemented (to avoid
re-numbering later).

And rename _COMPLETE and _STATUS to _DONE for consistency.

In the future the "trigger" keyword argument can be reinstated by requiring
the user to compute the bitmask, eg:

    ble.irq(handler, 1 << _IRQ_SCAN_RESULT | 1 << _IRQ_SCAN_DONE)
2020-06-05 14:04:20 +10:00
Damien George 22806ed5df extmod/vfs: Retain previous working directory if chdir fails.
Fixes issue #6069.
2020-05-29 23:05:01 +10:00
Damien George 7dffbfd22a extmod/vfs_lfsx: Fix import_stat so it takes into account current dir.
CPython semantics require searching the current directory if the import is
not absolute (when "" is in sys.path).

Fixes issue #6037.
2020-05-15 11:31:32 +10:00
Thomas Friebel 18fb5b4432 extmod/nimble: Make error code mapping default to MP_EIO.
Before this change, any NimBLE error that does not appear in the
ble_hs_err_to_errno_table maps to return code 0, meaning success.  If we
miss adding an error code to the table we end up returning success in case
of failure.

Instead, handle the zero case explicitly and default to MP_EIO.  This
allows removing the now-redundant MP_EIO entries from the mapping.
2020-05-11 22:31:30 +10:00
Damien George 3b6c9119eb extmod/modbluetooth: Add support for changing the GAP device name.
This commit allows the user to set/get the GAP device name used by service
0x1800, characteristic 0x2a00.  The usage is:

    BLE.config(gap_name="myname")
    print(BLE.config("gap_name"))

As part of this change the compile-time setting
MICROPY_PY_BLUETOOTH_DEFAULT_NAME is renamed to
MICROPY_PY_BLUETOOTH_DEFAULT_GAP_NAME to emphasise its link to GAP and this
new "gap_name" config value.  And the default value of this for the NimBLE
bindings is changed from "PYBD" to "MPY NIMBLE" to be more generic.
2020-05-11 21:30:41 +10:00
robert 0f83ef395c extmod/vfs_lfsx: Fix rename to respect cur dir for new path.
If the new name start with '/', cur_dir is not prepened any more, so that
the current working directory is respected.  And extend the test cases for
rename to cover this functionality.
2020-05-08 21:54:04 +10:00
robert d3ea28d04a extmod/vfs_lfsx: Normalize path name in chdir.
This change scans for '.', '..' and multiple '/' and normalizes the new
path name.  If the resulting path does not exist, an error is raised.
Non-existing interim path elements are ignored if they are removed during
normalization.
2020-05-08 21:52:15 +10:00
robert a5ea4b9f3f extmod/vfs_lfsx: Fix path handling in uos.stat() to consider cur dir.
This fixes the bug, that stat(filename) would not consider the current
working directory.  So if e.g. the cwd is "lib", then stat("main.py") would
return the info for "/main.py" instead of "/lib/main.py".
2020-05-08 21:37:51 +10:00
Damien George 73c58150f5 extmod/modbtree: Retain reference to underlying stream so it's not GC'd.
For ports that have a system malloc which is not garbage collected (eg
unix, esp32), the stream object for the DB must be retained separately to
prevent it from being reclaimed by the MicroPython GC (because the
berkeley-db library uses malloc to allocate the DB structure which stores
the only reference to the stream).

Although in some cases the user code will explicitly retain a reference to
the underlying stream because it needs to call close() on it, this is not
always the case, eg in cases where the DB is intended to live forever.

Fixes issue #5940.
2020-05-02 16:08:04 +10:00
Jim Mussared 2e3c42775a unix: Add btstack to the unix submodules list.
But only when bluetooth is enabled, i.e. if building the dev or coverage
variants, and we have libusb available.

Update travis to match, i.e. specify the variant when doing
`make submodules`.
2020-04-29 16:54:12 +10:00
Jim Mussared ebfd9ff2e6 extmod/modbluetooth: Fix sign compare and unused variable warnings. 2020-04-29 16:54:12 +10:00
Jim Mussared 7563d58210 unix: Add support for modbluetooth and BLE using btstack.
This commit adds full support to the unix port for Bluetooth using the
common extmod/modbluetooth Python bindings.  This uses the libusb HCI
transport, which supports many common USB BT adaptors.
2020-04-29 16:45:46 +10:00
Jim Mussared c987adb9e9 extmod/btstack: Implement more robust init/deinit sequencing. 2020-04-29 16:45:46 +10:00
Jim Mussared 8119ec0765 extmod/modbluetooth: Don't hold atomic section during mp_sched_schedule.
Because, for example, on unix the atomic section isn't re-entrant, and
mp_sched_schedule() will try to re-acquire the atomic section.
2020-04-29 16:45:40 +10:00
Jim Mussared 0da47ecc93 stm32/Makefile: Rename SRC_LIB to LIB_SRC_C to match other ports. 2020-04-29 16:38:18 +10:00
Damien George e08ca78f40 py/stream: Remove mp_stream_errno and use system errno instead.
This change is made for two reasons:

1. A 3rd-party library (eg berkeley-db-1.xx, axtls) may use the system
   provided errno for certain errors, and yet MicroPython stream objects
   that it calls will be using the internal mp_stream_errno.  So if the
   library returns an error it is not known whether the corresponding errno
   code is stored in the system errno or mp_stream_errno.  Using the system
   errno in all cases (eg in the mp_stream_posix_XXX wrappers) fixes this
   ambiguity.

2. For systems that have threading the system-provided errno should always
   be used because the errno value is thread-local.

For systems that do not have an errno, the new lib/embed/__errno.c file is
provided.
2020-04-27 23:58:46 +10:00
Jim Mussared 347c8917dc extmod/nimble: Update to work with NimBLE 1.3. 2020-04-27 22:51:10 +10:00
stijn 84fa3312cf all: Format code to add space after C++-style comment start.
Note: the uncrustify configuration is explicitly set to 'add' instead of
'force' in order not to alter the comments which use extra spaces after //
as a means of indenting text for clarity.
2020-04-23 11:24:25 +10:00
stijn 70affd9ba2 all: Fix implicit floating point to integer conversions.
These are found when building with -Wfloat-conversion.
2020-04-18 22:42:24 +10:00
stijn bcf01d1686 all: Fix implicit conversion from double to float.
These are found when building with -Wfloat-conversion.
2020-04-18 22:42:24 +10:00
stijn 0ba68f8a1d all: Fix implicit floating point promotion.
Initially some of these were found building the unix coverage variant on
MacOS because that build uses clang and has -Wdouble-promotion enabled, and
clang performs more vigorous promotion checks than gcc.  Additionally the
codebase has been compiled with clang and msvc (the latter with warning
level 3), and with MICROPY_FLOAT_IMPL_FLOAT to find the rest of the
conversions.

Fixes are implemented either as explicit casts, or by using the correct
type, or by using one of the utility functions to handle floating point
casting; these have been moved from nativeglue.c to the public API.
2020-04-18 22:36:14 +10:00
stijn b909e8b2dd Revert "all: Fix implicit casts of float/double, and signed comparison."
This reverts commit a2110bd3fc.  There's
nothing inherently wrong with it, but upcoming commits will apply similar
fixes in a slightly different way.
2020-04-18 22:36:06 +10:00
Damien George 5f0661b4fe extmod/uasyncio: Change cannot to can't in error message, and test exp.
Follow up to 8e048d2548 which missed these.
2020-04-14 21:51:25 +10:00
Damien George 8e048d2548 all: Clean up error strings to use lowercase and change cannot to can't.
Now that error string compression is supported it's more important to have
consistent error string formatting (eg all lowercase English words,
consistent contractions).  This commit cleans up some of the strings to
make them more consistent.
2020-04-13 22:19:37 +10:00
Damien George db137e70dc extmod/uasyncio: Add Loop.new_event_loop method.
This commit adds Loop.new_event_loop() which is used to reset the singleton
event loop.  This functionality is put here instead of in Loop.close() to
make it possible to write code that is compatible with CPython.
2020-04-13 22:16:52 +10:00
Damien George c5a21a94f8 extmod/modbluetooth: Provide FLAG_WRITE_NO_RESPONSE for characteristics.
This flag is supported and needs to be set if characteristics are write-
without-response.
2020-04-07 13:46:56 +10:00
Damien George 899e89d4c6 extmod/btstack: Pass through SCAN_RSP events.
The latest version of BTstack has a bug fixed so that it correctly
configures scan parameters if they are set right after activating the
stack.  This means that BLE.gap_scan() will correctly set the scanning to
passive and so SCAN_RSP events are not passed through, so we don't need to
explicitly filter them in our bindings.
2020-04-07 13:46:56 +10:00
Jim Mussared def76fe4d9 all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
Jim Mussared 1921224272 extmod/modubinascii: Make code private and module self-contained.
This commit makes all functions and function wrappers in modubinascii.c
STATIC and conditional on the MICROPY_PY_UBINASCII setting, which will
exclude the file from qstr/ compressed-string searching when ubinascii is
not enabled.  The now-unused modubinascii.h header file is also removed.

The cc3200 port is updated accordingly to use this module in its entirety
instead of providing its own top-level definition of ubinascii.

This was originally like this because the cc3200 port has its own ubinascii
module which referenced these methods.  The plan appeared to be that the
API might diverge (e.g. hardware crc), but this should be done similar to
I2C/SPI via a port-specific handler, rather than the port having its own
definition of the module.  Having a centralised module definition also
enforces consistency of the API among ports.
2020-04-05 14:13:53 +10:00
Kevin Köck 15f41c2dbf extmod/uasyncio: Add global exception handling methods.
This commit adds support for global exception handling in uasyncio
according to the CPython error handling:
https://docs.python.org/3/library/asyncio-eventloop.html#error-handling-api

This allows a program to receive exceptions from detached tasks and log
them to an appropriate location, instead of them being printed to the REPL.

The implementation preallocates a context dictionary so in case of an
exception there shouldn't be any RAM allocation.

The approach here is compatible with CPython except that in CPython the
exception handler is called once the task that threw an uncaught exception
is freed, whereas in MicroPython the exception handler is called
immediately when the exception is thrown.
2020-04-04 10:37:00 +11:00
Damien George f97b5395ed extmod/uasyncio: Add StreamReader/StreamWriter as aliases of Stream cls.
To be compatible with CPython.  Fixes issue #5847.
2020-04-02 00:51:00 +11:00
Kevin Köck aca19c25d2 extmod/uasyncio: Add error message to Lock.release's RuntimeError.
Otherwise it can be hard to understand what the error is if a blank
RuntimeError is raised.
2020-04-02 00:40:23 +11:00
Damien George b389bc0afa extmod/uasyncio: Implement Loop.stop() to stop the event loop. 2020-04-02 00:14:18 +11:00
Damien George 711dd392d3 extmod/uasyncio: Don't create a Loop instance in get_event_loop().
The event loop is (for now) just a singleton so make it so that Loop
instances are not needed.
2020-04-01 23:56:31 +11:00
David Lechner a2110bd3fc all: Fix implicit casts of float/double, and signed comparison.
These were found by buiding the unix coverage variant on macOS (so clang
compiler).  Mostly, these are fixing implicit cast of float/double to
mp_float_t which is one of those two and one mp_int_t to size_t fix for
good measure.
2020-03-30 12:04:21 +11:00
Damien George 1a3e386c67 all: Remove spaces inside and around parenthesis.
Using new options enabled in the uncrustify configuration.
2020-03-28 23:36:44 +11:00
David Lechner 9418611c8a unix: Implement PEP 475 to retry syscalls failing with EINTR.
https://www.python.org/dev/peps/pep-0475/

This implements something similar to PEP 475 on the unix port, and for the
VfsPosix class.

There are a few differences from the CPython implementation:
- Since we call mp_handle_pending() between any ENITR's, additional
  functions could be called if MICROPY_ENABLE_SCHEDULER is enabled, not
  just signal handlers.
- CPython only handles signal on the main thread, so other threads will
  raise InterruptedError instead of retrying.  On MicroPython,
  mp_handle_pending() will currently raise exceptions on any thread.

A new macro MP_HAL_RETRY_SYSCALL is introduced to reduce duplicated code
and ensure that all instances behave the same.  This will also allow other
ports that use POSIX-like system calls (and use, eg, VfsPosix) to provide
their own implementation if needed.
2020-03-27 14:40:46 +11:00
Damien George 3b68f36175 extmod/uasyncio: Add manifest.py for freezing uasyncio Py files. 2020-03-26 01:25:45 +11:00
Damien George bc009fdd62 extmod/uasyncio: Add optional implementation of core uasyncio in C.
Implements Task and TaskQueue classes in C, using a pairing-heap data
structure.  Using this reduces RAM use of each Task, and improves overall
performance of the uasyncio scheduler.
2020-03-26 01:25:45 +11:00
Damien George 63b9944382 extmod/uasyncio: Add new implementation of uasyncio module.
This commit adds a completely new implementation of the uasyncio module.
The aim of this version (compared to the original one in micropython-lib)
is to be more compatible with CPython's asyncio module, so that one can
more easily write code that runs under both MicroPython and CPython (and
reuse CPython asyncio libraries, follow CPython asyncio tutorials, etc).
Async code is not easy to write and any knowledge users already have from
CPython asyncio should transfer to uasyncio without effort, and vice versa.

The implementation here attempts to provide good compatibility with
CPython's asyncio while still being "micro" enough to run where MicroPython
runs. This follows the general philosophy of MicroPython itself, to make it
feel like Python.

The main change is to use a Task object for each coroutine.  This allows
more flexibility to queue tasks in various places, eg the main run loop,
tasks waiting on events, locks or other tasks.  It no longer requires
pre-allocating a fixed queue size for the main run loop.

A pairing heap is used to queue Tasks.

It's currently implemented in pure Python, separated into components with
lazy importing for optional components.  In the future parts of this
implementation can be moved to C to improve speed and reduce memory usage.
But the aim is to maintain a pure-Python version as a reference version.
2020-03-26 01:25:45 +11:00
David Lechner 58d9a4815d extmod/vfs_posix_file: Include unistd.h to get STD{IN,OUT,ERR}_FILENO. 2020-03-25 00:59:39 +11:00
Damien George 2cdf1d25f5 unix: Remove custom file implementation to use extmod's VFS POSIX one.
The implementation in extmod/vfs_posix_file.c is now equivalent to that in
ports/unix/file.c, so remove the latter and use the former instead.
2020-03-18 21:01:07 +11:00
Damien George 68b1bc2042 extmod/vfs_posix_file: Lock GIL when writing and allow stdio flush.
Also support MP_STREAM_GET_FILENO ioctl.  The stdio flush change was done
previously for the unix port in 3e0b46b9af.
These changes make this POSIX file implementation equivalent to the unix
file implementation.
2020-03-18 21:01:07 +11:00
Damien George ad9a0ec8ab all: Convert exceptions to use mp_raise_XXX helpers in remaining places. 2020-03-18 17:26:19 +11:00
Damien George 8f0778b209 extmod/modlwip: Properly handle non-blocking and timeout on UDP recv.
Fixes UDP non-blocking recv so it returns EAGAIN instead of ETIMEDOUT.
Timeout waiting for incoming data is also improved by replacing 100ms delay
with poll_sockets(), as is done in other parts of this module.

Fixes issue #5759.
2020-03-18 10:51:32 +11:00
Damien George 00267aae0b extmod/modlwip: Fix polling of UDP socket so it doesn't return HUP.
STATE_NEW will return HUP when polled so put active UDP sockets into a new
state which is different to STATE_NEW.

Fixes issue #5758.
2020-03-18 10:49:27 +11:00
Damien George ed848553b4 extmod/vfs: Factor out vfs mount-and-chdir helper from stm32. 2020-03-11 14:24:26 +11:00
Damien George 7bf62562ce extmod/nimble: When getting BLE MAC try public address if random fails.
This is needed for BLE.config('mac') to work on esp32.
2020-03-11 14:02:13 +11:00
Damien George dd0bc26e65 extmod/modbluetooth: Change scan result's "connectable" to "adv_type".
This commit changes the BLE _IRQ_SCAN_RESULT data from:

    addr_type, addr, connectable, rssi, adv_data

to:

    addr_type, addr, adv_type, rssi, adv_data

This allows _IRQ_SCAN_RESULT to handle all scan result types (not just
connectable and non-connectable passive scans), and to distinguish between
them using adv_type which is an integer taking values 0x00-0x04 per the BT
specification.

This is a breaking change to the API, albeit a very minor one: the existing
connectable value was a boolean and True now becomes 0x00, False becomes
0x02.

Documentation is updated and a test added.

Fixes #5738.
2020-03-11 14:00:44 +11:00
Thomas Friebel 41a9f1dec8 extmod/modbluetooth: Unify error handling in remaining places.
Most error handling is done via `bluetooth_handle_errno()` already.
Replace the remaining few manual checks with calls to that function.
2020-03-11 13:01:35 +11:00
Jim Mussared 1937fb22ab extmod/nimble: Clarify active state and check for active in all methods.
This commit ensures that the BLE stack is active before allowing operations
that may otherwise crash if it's not active.  It also clarifies the state
better (adding the "stopping" state) and renames mp_bluetooth_is_enabled to
the more self-explanatory mp_bluetooth_is_active.
2020-03-11 13:01:35 +11:00
Damien George e965363b6b stm32: Refactor Bluetooth HCI RX to be independent of transport layer.
Now all HCI specific code (eg UART vs WB55 internal messaging) is confined
to modbluetooth_hci.c.
2020-03-10 01:53:42 +11:00
Damien George d7259f6b1c extmod/btstack: Implement notifications/indications for GATT clients.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George 018ce122ca extmod/btstack: Implement scan and gatt client, connect and disconnect.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George 372e5a280e extmod/btstack: Implement gatts_db for btstack.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George 86c26db3ff extmod/btstack: Implement service registration.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George 0674917bc5 extmod/btstack: Implement advertising.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George fbefcef1df extmod/btstack: Add empty modbluetooth implementation.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George 0ac06a510a extmod/modbluetooth: Extract out gatts_db functionality from nimble.
For use by other stacks, if they need it.

Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George 894c550c86 stm32: Refactor bluetooth stack/hci/driver bindings.
This makes a cleaner separation between the: driver, HCI UART and BT stack.
Also updated the naming to be more consistent (mp_bluetooth_hci_*).

Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George c44d52f33e extmod/modbluetooth_nimble: Move nimble specific code, factor nimble.mk.
Move extmod/modbluetooth_nimble.* to extmod/nimble.  And move common
Makefile lines to extmod/nimble/nimble.mk (which was previously only used
by stm32).  This allows (upcoming) btstack to follow a similar structure.

Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-06 12:35:20 +11:00
Damien George 69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
Jim Mussared f8449dd092 extmod/modframebuf: Allow blit source to be a subclass of FrameBuffer. 2020-02-21 13:32:48 +11:00
Jim Mussared cddb90e8b0 extmod/modbluetooth_nimble: Fix wrong offset used for descriptor flags. 2020-02-18 16:37:38 +11:00
Jim Mussared 66ac2e1fc0 extmod/modbluetooth.h: Fix typo in comment about registering services. 2020-02-18 16:36:37 +11:00
Thomas Friebel f4726735cf extmod/modbluetooth: Implement config getter for BLE rxbuf size.
Knowing the buffer size can be important, to ensure that valid data will be
received.
2020-02-18 13:37:50 +11:00
Damien George ce39c958ef py: Factor out definition of mp_float_union_t to one location. 2020-02-18 13:04:36 +11:00
Damien George ad7213d3c3 py: Add mp_raise_msg_varg helper and use it where appropriate.
This commit adds mp_raise_msg_varg(type, fmt, ...) as a helper for
nlr_raise(mp_obj_new_exception_msg_varg(type, fmt, ...)).  It makes the
C-level API for raising exceptions more consistent, and reduces code size
on most ports:

   bare-arm:   +28 +0.042%
minimal x86:  +100 +0.067%
   unix x64:   -56 -0.011%
unix nanbox:  -300 -0.068%
      stm32:  -204 -0.054% PYBV10
     cc3200:    +0 +0.000%
    esp8266:   -64 -0.010% GENERIC
      esp32:  -104 -0.007% GENERIC
        nrf:  -136 -0.094% pca10040
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2020-02-13 11:52:40 +11:00
Damien George 97eca38c4f py: Add mp_raise_type helper macro and use it where appropriate.
This provides a more consistent C-level API to raise exceptions, ie moving
away from nlr_raise towards mp_raise_XXX.  It also reduces code size by a
small amount on some ports.
2020-02-13 11:03:37 +11:00
David Lechner 35f66d38b8 extmod/vfs_posix: Release GIL during system calls.
This releases the GIL during syscalls that could block.
2020-01-26 23:26:14 +11:00
Damien George bfbd94401d py: Make mp_obj_get_type() return a const ptr to mp_obj_type_t.
Most types are in rodata/ROM, and mp_obj_base_t.type is a constant pointer,
so enforce this const-ness throughout the code base.  If a type ever needs
to be modified (eg a user type) then a simple cast can be used.
2020-01-09 11:25:26 +11:00
Yonatan Goldschmidt aca8873bb8 extmod/modbluetooth: Fix func prototype, empty args should be (void).
This fixes a -Wstrict-prototypes error.
2019-12-27 23:57:46 +11:00
Damien George 6f872f81d6 extmod: Fix modbluetooth and modwebrepl to build in nanbox mode. 2019-12-27 22:54:53 +11:00
Damien George d97b40bdaa py: Introduce MP_ROM_FALSE/MP_ROM_TRUE for ROM to refer to bool objects.
This helps to prevent mistakes, and allows easily changing the ROM value of
False/True if needed.
2019-12-27 22:54:20 +11:00
Damien George 09376f0e47 py: Introduce MP_ROM_NONE macro for ROM to refer to None object.
This helps to prevent mistakes, and allows easily changing the ROM value of
None if needed.
2019-12-27 22:51:17 +11:00
Damien George ed2be79b49 extmod/uzlib: Explicitly cast ptr-diff-expr to unsigned.
The struct member "dest" should never be less than "destStart", so their
difference is never negative.  Cast as such to make the comparison
explicitly unsigned, ensuring the compiler produces the correct comparison
instruction, and avoiding any compiler warnings.
2019-12-23 00:07:03 +11:00
Jim Mussared 7ce1e0b1dc extmod/webrepl: Move webrepl scripts to common place and use manifest.
Move webrepl support code from ports/esp8266/modules into extmod/webrepl
(to be alongside extmod/modwebrepl.c), and use frozen manifests to include
it in the build on esp8266 and esp32.

A small modification is made to webrepl.py to make it work on non-ESP
ports, i.e. don't call dupterm_notify if not available.
2019-12-20 12:59:13 +11:00
Damien George ba84453f77 examples/natmod: Add urandom native module example. 2019-12-13 13:33:40 +11:00
Damien George bbeaafd9aa extmod: Add dynamic-runtime guards to btree/framebuf/uheapq/ure/uzlib.
So they can be built as dynamic native modules, as well as existing static
native modules.
2019-12-12 20:15:28 +11:00
Damien George e5acd06ad5 extmod/modbtree: Use mp_printf instead of printf. 2019-12-12 20:15:28 +11:00
Damien George 84958a8fe1 extmod/modbluetooth: Allow setting ringbuf size via BLE.config(rxbuf=).
The size of the event ringbuf was previously fixed to compile-time config
value, but it's necessary to sometimes increase this for applications that
have large characteristic buffers to read, or many events at once.

With this commit the size can be set via BLE.config(rxbuf=512), for
example.  This also resizes the internal event data buffer which sets the
maximum size of incoming data passed to the event handler.
2019-12-05 11:30:35 +11:00
Damien George 7aeafe2ae9 extmod/modbluetooth: Add optional 4th arg to gattc_write for write mode.
This allows the user to explicitly select the behaviour of the write to the
remote peripheral.  This is needed for peripherals that have
characteristics with WRITE_NO_RESPONSE set (instead of normal WRITE).  The
function's signature is now:

    BLE.gattc_write(conn_handle, value_handle, data, mode=0)

mode=0 means write without response, while mode=1 means write with
response.  The latter was the original behaviour so this commit is a change
in behaviour of this method, and one should specify 1 as the 4th argument
to get back the old behaviour.

In the future there could be more modes supported, such as long writes.
2019-12-04 23:23:07 +11:00
Damien George 40cc7ec677 stm32/mpconfigport.h: Use IRQ_PRI_PENDSV to protect bluetooth ringbuf.
The default protection for the BLE ringbuf is to use
MICROPY_BEGIN_ATOMIC_SECTION, which disables all interrupts. On stm32 it
only needs to disable the lowest priority IRQ, pendsv, because that's the
IRQ level at which the BLE stack is driven.
2019-12-04 13:39:22 +11:00
Damien George 8ce69288e9 extmod/modbluetooth: Remove limit on data coming from gattc data input.
This removes the limit on data coming in from a BLE.gattc_read() request,
or a notify with payload (coming in to a central).  In both cases the data
coming in to the BLE callback is now limited only by the available data in
the ringbuf, whereas before it was capped at (default hard coded) 20 bytes.
2019-12-02 23:27:25 +11:00
Damien George d6e051082a extmod/modbluetooth: Simplify how BLE IRQ callback is scheduled.
Instead of enqueue_irq() inspecting the ringbuf to decide whether to
schedule the IRQ callback (if ringbuf is empty), maintain a flag that knows
if the callback is on the schedule queue or not.  This saves about 150
bytes of code (for stm32 builds), and simplifies all uses of enqueue_irq()
and schedule_ringbuf().
2019-12-02 23:25:36 +11:00
Damien George 6b3404f25e extmod/vfs_lfs: Fix bug when passing no args to constructor and mkfs. 2019-11-26 00:08:57 +11:00
Damien George 5634a31a98 extmod/vfs_lfs: Pass flag along to ioctl when init'ing bdev for lfs.
To hint to the block device that the extended block protocol will be used.
2019-11-26 00:07:42 +11:00
Jim Mussared e873d352ad extmod/modbluetooth: Simplify management of pre-allocated event data.
The address, adv payload and uuid fields of the event are pre-allocated by
modbluetooth, and reused in the IRQ handler.  Simplify this and move all
storage into the `mp_obj_bluetooth_ble_t` instance.

This now allows users to hold on to a reference to these instances without
crashes, although they may be overwritten by future events.  If they want
to hold onto the values longer term they need to copy them.
2019-11-25 17:32:10 +11:00
Jim Mussared 438c0dc2a4 extmod/modbluetooh_nimble: Fix UUID conversion for 16 and 32 bit values. 2019-11-25 17:31:59 +11:00
Jim Mussared 2ae755d9e1 extmod/modbluetooth_nimble: Make gap_scan_stop no-op if no scan ongoing.
No need for this to throw an exception if the intent (don't be scanning) is
clear, and avoids a race with the scan duration timeout.
2019-11-25 17:27:40 +11:00
Jim Mussared d19c6d0519 extmod/modbluetooth: Create UUID from bytes and allow comparison ops.
This allows construction of UUID objects from advertising data payloads and
matching against known UUIDs.
2019-11-25 17:20:51 +11:00
Jim Mussared 334ba01c90 extmod/modbluetooth: Prioritise non-scan-result events.
Remove existing scan result events from the ringbuf if the ringbuf is full
and we're trying to enqueue any other event.  This is needed so that events
such as SCAN_COMPLETE are always put on the ringbuf.
2019-11-21 12:04:57 +11:00
Damien George 799b6d1e0c extmod: Consolidate FAT FS config to MICROPY_VFS_FAT across all ports.
This commit removes the Makefile-level MICROPY_FATFS config and moves the
MICROPY_VFS_FAT config to the Makefile level to replace it.  It also moves
the include of the oofatfs source files in the build from each port to a
central place in extmod/extmod.mk.

For a port to enabled VFS FAT support it should now set MICROPY_VFS_FAT=1
at the level of the Makefile.  This will include the relevant oofatfs files
in the build and set MICROPY_VFS_FAT=1 at the C (preprocessor) level.
2019-11-11 11:37:38 +11:00
Damien George 9b27069e2f extmod/vfs: Add autodetect of littlefs filesystem when mounting. 2019-11-06 12:15:34 +11:00
Damien George c13f9f209d all: Convert nlr_raise(mp_obj_new_exception_msg(x)) to mp_raise_msg(x).
This helper function was added a while ago and these are the remaining
cases to convert, to save a bit of code size.
2019-11-05 11:35:45 +11:00
Damien George 2ee9e1a4ed extmod/modbtree: Make FILEVTABLE const to put it in ROM. 2019-11-01 17:25:40 +11:00
Damien George 40ea1915fc extmod/nimble: Factor out stm32-specific HCI UART RX/TX code. 2019-11-01 12:41:37 +11:00
Damien George 78145b98ef extmod/nimble: Remove unneeded nimble_sprintf wrapper function. 2019-11-01 12:15:07 +11:00
Damien George 9dd9f9ff06 extmod/modussl_mbedtls: Check for invalid key/cert data. 2019-10-31 16:22:42 +11:00
Damien George 07ea81fbc5 extmod/modussl_mbedtls: Fix getpeercert to return None if no cert avail. 2019-10-31 13:42:24 +11:00
Damien George 26d8fd2c0a extmod/modlwip: Unconditionally return POLLHUP/POLLERR when polling.
POSIX poll should always return POLLERR and POLLHUP in revents, regardless
of whether they were requested in the input events flags.

See issues #4290 and #5172.
2019-10-31 13:37:51 +11:00
Damien George feaa251674 extmod/modlwip: Make socket poll return POLLNVAL in case of bad file. 2019-10-31 12:54:37 +11:00
Damien George 71401d5065 extmod/modlwip: Unconditionally return POLLHUP when polling new socket.
POSIX poll should always return POLLERR and POLLHUP in revents, regardless
of whether they were requested in the input events flags.

See issues #4290 and #5172.
2019-10-31 12:54:37 +11:00
Damien George 660a61a388 extmod/vfs_lfs: Allow compiling in VfsLfs1 and VfsLfs2 separately.
These classes are enabled via the config options MICROPY_VFS_LFS1 and
MICROPY_VFS_LFS2, which are disabled by default.
2019-10-30 12:08:58 +11:00
Jim Mussared d16a27da51 extmod/modbluetooth: Add gatts_set_buffer.
- Adds an explicit way to set the size of a value's internal buffer,
  replacing `ble.gatts_write(handle, bytes(size))` (although that
  still works).
- Add an "append" mode for values, which means that remote writes
  will append to the buffer.
2019-10-29 23:09:02 +11:00
Damien George 4cf054a130 extmod/vfs: Add MP_BLOCKDEV_IOCTL_BLOCK_ERASE constant. 2019-10-29 14:17:29 +11:00
Damien George cfe1c5abf8 extmod/vfs: Rename BP_IOCTL_xxx constants to MP_BLOCKDEV_IOCTL_xxx.
Also rename SEC_COUNT to BLOCK_COUNT and SEC_SIZE to BLOCK_SIZE.
2019-10-29 14:17:29 +11:00
Damien George a099505420 extmod: Add VFS littlefs bindings.
Both LFS1 and LFS2 are supported at the same time.
2019-10-29 14:17:29 +11:00
Damien George 98beea9ced extmod/vfs_blockdev: Add extended read/write methods.
This commit adds helper functions to call readblocks/writeblocks with a
fourth argument, the byte offset within a block.

Although the mp_vfs_blockdev_t struct has grown here by 2 machine words, in
all current uses of this struct within this repository it still fits within
the same number of GC blocks.
2019-10-29 12:55:34 +11:00
Damien George e1c7b1cb43 extmod/vfs_blockdev: Factor out block device interface code. 2019-10-29 12:55:17 +11:00
Damien George 9aabb6c01b extmod: Factor out block-device struct to make independent of fatfs. 2019-10-29 12:12:37 +11:00
Jim Mussared b02d7e612d extmod/modbluetooth: Rename module to "ubluetooth".
For consistency with "umachine". Now that weak links are enabled
by default for built-in modules, this should be a no-op, but allows
extension of the bluetooth module by user code.

Also move registration of ubluetooth to objmodule rather than
port-specific.
2019-10-22 21:58:05 +11:00
Jim Mussared 2c1f269918 extmod/modbluetooth_nimble: Use `data_alloc` length to truncate writes.
This allows the maximum size of a characteristic/descriptor to be
increased by locally writing to it first.
2019-10-22 14:30:23 +11:00
Jim Mussared f34e16dbc6 extmod/modbluetooth: Persist reference to NimBLE service instances.
NimBLE doesn't actually copy this data, it requires it to stay live.
Only dereference when we register a new set of services.

Fixes #5226

This will allow incrementally adding services in the future, so
rename `reset` to `append` to make it clearer.
2019-10-22 14:30:23 +11:00
Jim Mussared 56fc3edf98 extmod/modbluetooth: Make UUID support the buffer protocol.
Internally change the representation of UUIDs to LE uint8* to simplify this.

This allows UUIDs to be easily used in BLE payloads (such as advertising).

Ref: #5186
2019-10-22 13:54:05 +11:00
Jim Mussared ebf8332104 extmod/re1.5: Support escaping within RE classes.
Fixes issues #3178 and #5220.

Tests are added, including all the cases mentioned in both bugs.
2019-10-18 12:20:32 +11:00
Damien George 8f7f671236 extmod/modbluetooth: In gap_advertise only accept None to stop adv.
To match the docs, and interval=0 may be used in the future to indicate
something else.
2019-10-15 17:29:27 +11:00
Jim Mussared 36502bdfdc extmod/modbluetooth: Make gap_disconnect not raise when disconnected.
Previously it raised OSError(MP_ENOTCONN) if the conn_handle was already
disconnected.  Now it returns True/False.
2019-10-15 17:22:53 +11:00
Jim Mussared 423e67d0a0 extmod/modbluetooth: Improve ringbuf handling.
No need to share the irq_data buffer with addresses.  Split them into two
separate buffers and manage their max length independently.
2019-10-15 17:22:07 +11:00
Jim Mussared cb73103f57 extmod/modbluetooth: Fix order of params to IRQ_GATTS_WRITE event. 2019-10-15 17:22:01 +11:00
Jim Mussared ba16a22991 extmod/modbluetooth: Clear gap_advertise payload when data is empty.
Also fix default adv interval to 500ms.
2019-10-15 17:21:17 +11:00
Jim Mussared 418f12c5f5 extmod/modbluetooth: Increase maximum connections from 1 to 4.
This avoids a confusing ENOMEM raised from gap_advertise if there is
currently an active connection.  This refers to the static connection
buffer pre-allocated by Nimble (nothing to do with MicroPython heap
memory).
2019-10-15 17:20:25 +11:00
Jim Mussared 76f474129e extmod/modbluetooth: Use us instead of ms for advertising interval.
This is to more accurately match the BLE spec, where intervals are
configured in units of channel hop time (625us). When it was
specified in ms, not all "valid" intervals were able to be
specified.

Now that we're also allowing configuration of scan interval, this
commit updates advertising to match.
2019-10-11 13:51:07 +11:00
Jim Mussared b65cc387cd extmod/modbluetooth: Allow config of scan interval/window.
This adds two additional optional kwargs to `gap_scan()`:
  - `interval_us`: How long between scans.
  - `window_us`: How long to scan for during a scan.

The default with NimBLE is a 11.25ms window with a 1.28s interval.

Changing these parameters is important for detecting low-frequency
advertisements (e.g. beacons).

Note: these params are in microseconds, not milliseconds in order
to allow the 625us granularity offered by the spec.
2019-10-11 13:51:07 +11:00
Jim Mussared 902bb4ceae stm32: Extract port-specific Nimble implementation.
On other ports (e.g. ESP32) they provide a complete Nimble implementation
(i.e. we don't need to use the code in extmod/nimble). This change
extracts out the bits that we don't need to use in other ports:
 - malloc/free/realloc for Nimble memory.
 - pendsv poll handler
 - depowering the cywbt

Also cleans up the root pointer management.
2019-10-08 14:40:35 +11:00
Andrew Leech eb1b6858a2 extmod/modbluetooth: Allow MP_BLUETOOTH_MAX_ATTR_SIZE in board config. 2019-10-01 09:51:02 +10:00
Andrew Leech 5dc592d117 extmod/modbluetooth_nimble: Use random addr if public isn't available. 2019-10-01 09:51:02 +10:00
Jim Mussared 497dae45e7 extmod/modbluetooth_nimble: Implement modbluetooth API with Nimble. 2019-10-01 09:51:02 +10:00
Jim Mussared 16f8ceeaaa extmod/modbluetooth: Add low-level Python BLE API. 2019-10-01 09:51:02 +10:00
Damien George 07f6644a38 extmod/nimble: Add nimble bindings. 2019-10-01 09:51:02 +10:00
Bob Fanger bff2771da1 extmod/vfs_posix: Include stdio.h for declaration of function 'rename'. 2019-09-23 15:14:42 +10:00
Damien George 24c3e9b283 py/modstruct: Fix struct.pack_into with unaligned offset of native type.
Following the same fix for unpack.
2019-09-02 13:14:16 +10:00
Tom McDermott 1022f9cc35 py/modstruct: Fix struct.unpack with unaligned offset of native type.
With this patch alignment is done relative to the start of the buffer that
is being unpacked, not the raw pointer value, as per CPython.

Fixes issue #3314.
2019-09-02 13:10:55 +10:00
Damien George 519746cae4 extmod/crypto-algorithms: Add source to header and populate copyright.
As per the README.md of the upstream source at
https://github.com/B-Con/crypto-algorithms, this source code was released
into the public domain, so make that explicit in the copyright line in the
header.
2019-08-22 17:20:16 +10:00
Damien George 2dfa69efbb extmod/modujson: Support passing bytes/bytearray to json.loads.
CPython allows this, and it can be useful to reduce the number of memory
allocations.

Fixes issue #5031.
2019-08-22 15:32:26 +10:00
Damien George 0cc8910bc5 extmod: Give vars/funcs unique names so STATIC can be set to nothing.
Fixes issue #5018.
2019-08-20 15:21:09 +10:00
Damien George 3327dfc16e extmod/moducryptolib: Use "static" not "STATIC" for inline functions. 2019-08-20 15:21:09 +10:00
Damien George 7d851a27f1 extmod/modure: Make regex dump-code debugging feature optional.
Enabled via MICROPY_PY_URE_DEBUG, disabled by default (but enabled on unix
coverage build).  This is a rarely used feature that costs a lot of code
(500-800 bytes flash).  Debugging of regular expressions can be done
offline with other tools.
2019-08-19 16:43:00 +10:00
Damien George 80f5cef8d4 extmod/modlwip: Implement raw sockets for lwIP.
Configurable via MICROPY_PY_LWIP_SOCK_RAW.
2019-08-06 15:56:05 +10:00
Damien George c60caf1995 extmod/modlwip: Use mp_sched_schedule to schedule socket callbacks.
The helper function exec_user_callback executes within the context of an
lwIP C callback, and the user (Python) callback to be scheduled may want to
perform further TCP/IP actions, so the latter should be scheduled to run
outside the lwIP context (otherwise it's effectively a "hard IRQ" and such
callbacks have lots of restrictions).
2019-07-03 16:22:48 +10:00
Damien George 1d6cb6357a extmod/modlwip: For TCP send keep trying tcp_write if it returns ERR_MEM
If tcp_write returns ERR_MEM then it's not a fatal error but instead means
the caller should retry the write later on (and this is what lwIP's netconn
API does).

This fixes problems where a TCP send would raise OSError(ENOMEM) in
situations where the TCP/IP stack is under heavy load.  See eg issues #1897
and #1971.
2019-07-03 15:50:13 +10:00
Damien George ef00048fed extmod/modwebrepl: Add config option to put filebuf[512] on stack/bss.
Since the esp8266 has a small stack this buffer is kept in the BSS.
2019-07-03 12:55:57 +10:00
Damien George fa2c7ece8f extmod/modwebrepl: Make prompt/ver static arrays const to not use RAM.
The esp8266 lwip_open library is compiled with -mforce-l32 so these arrays
do not need to be in RAM.
2019-07-03 12:50:13 +10:00
Damien George fad3d08d2d extmod/moduwebsocket: Make close_resp static array const to not use RAM.
The esp8266 lwip_open library is compiled with -mforce-l32 so this array
does not need to be in RAM.
2019-07-03 12:47:13 +10:00
Damien George 89a23a05b3 esp8266: Provide custom machine_time_pulse_us that feeds soft WDT.
So that the timeout for machine.time_pulse_us() can be large.

Fixes issue #2775.
2019-07-01 22:53:00 +10:00
Damien George 964ae328cd extmod/uos_dupterm: Add mp_uos_dupterm_poll to poll all dupterms. 2019-07-01 17:10:12 +10:00
Damien George 49388e339e extmod/extmod.mk: Include mdns app source in lwIP build. 2019-06-05 16:13:34 +10:00
Damien George ef7357d4ab extmod/modussl_mbedtls: Allow to build with object representation D. 2019-06-05 15:33:15 +10:00
Damien George 678ec182cd extmod/extmod.mk: Integrate mbedTLS so it is built from source.
Setting MICROPY_PY_USSL and MICROPY_SSL_MBEDTLS at the Makefile-level will
now build mbedTLS from source and include it in the build, with the ussl
module using this TLS library.  Extra settings like MBEDTLS_CONFIG_FILE may
need to be provided by a given port.

If a port wants to use its own mbedTLS library then it should not set
MICROPY_SSL_MBEDTLS at the Makefile-level but rather set it at the C level,
and provide the library as part of the build in its own way (see eg esp32
port).
2019-06-05 15:22:21 +10:00
Damien George cd6b115815 extmod: Factor out makefile rules from py.mk to new extmod.mk file.
To logically separate extmod related rules out, and prevent py.mk from
growing too large.
2019-06-05 14:23:12 +10:00
Damien George cf1c131f73 extmod: Add network-level class binding to cyw43 driver. 2019-06-03 16:47:35 +10:00
Damien George 019dd84af1 extmod/modlwip: Register TCP close-timeout callback before closing PCB.
In d5f0c87bb9 this call to tcp_poll() was
added to put a timeout on closing TCP sockets.  But after calling
tcp_close() the PCB may be freed and therefore invalid, so tcp_poll() can
not be used at that point.  As a fix this commit calls tcp_poll() before
closing the TCP PCB.  If the PCB is subsequently closed and freed by
tcp_close() or tcp_abort() then the PCB will not be on any active list and
the callback will not be executed, which is the desired behaviour (the
_lwip_tcp_close_poll() callback only needs to be called if the PCB remains
active for longer than the timeout).
2019-05-29 01:29:48 +10:00
Damien George 734ada3e29 extmod/modlwip: Free any incoming bufs/connections before closing PCB.
Commit 2848a613ac introduced a bug where
lwip_socket_free_incoming() accessed pcb.tcp->state after the PCB was
closed.  The state may have changed due to that close call, or the PCB may
be freed and therefore invalid.  This commit fixes that by calling
lwip_socket_free_incoming() before the PCB is closed.
2019-05-29 01:24:43 +10:00
Damien George b10d0664be extmod/machine_i2c: Add i2c.writevto() that can write a vector of bufs.
For example: i2c.writevto(addr, (buf1, buf2)).  This allows to efficiently
(wrt memory) write data composed of separate buffers, such as a command
followed by a large amount of data.
2019-05-20 15:04:29 +10:00
Damien George 8bcb552d97 extmod/machine_i2c: Remove need for temporary memory in writemem() call. 2019-05-20 15:04:29 +10:00
Damien George 606ea2b10f extmod/machine_i2c: Change C-level API to allow split I2C transactions.
API is:

    int transfer(
        mp_obj_base_t *obj,
        uint16_t addr,
        size_t n,
        mp_machine_i2c_buf_t *bufs,
        unsigned int flags
    )
2019-05-20 15:04:29 +10:00
Paul Sokolovsky 016d9a40fe various: Add and update my copyright line based on git history.
For modules I initially created or made substantial contributions to.
2019-05-17 18:04:15 +10:00
Damien George 7c5cf59f8b extmod/modujson: Handle parsing of floats with + in the exponent.
Fixes issue #4780.
2019-05-14 14:45:54 +10:00
Yonatan Goldschmidt 32ba679924 extmod/moducryptolib: Add AES-CTR support for axTLS builds. 2019-05-06 18:20:56 +10:00
Yonatan Goldschmidt ef9843653b extmod/moducryptolib: Add AES-CTR support.
Selectable at compile time via MICROPY_PY_UCRYPTOLIB_CTR.  Disabled by
default.
2019-05-06 18:09:48 +10:00
Paul Sokolovsky c76445315f extmod/modussl_axtls: Add non-blocking mode support.
It consists of:

1. "do_handhake" param (default True) to wrap_socket(). If it's False,
handshake won't be performed by wrap_socket(), as it would be done in
blocking way normally. Instead, SSL socket can be set to non-blocking mode,
and handshake would be performed before the first read/write request (by
just returning EAGAIN to these requests, while instead reading/writing/
processing handshake over the connection). Unfortunately, axTLS doesn't
really support non-blocking handshake correctly. So, while framework for
this is implemented on MicroPython's module side, in case of axTLS, it
won't work reliably.

2. Implementation of .setblocking() method. It must be called on SSL socket
for blocking vs non-blocking operation to be handled correctly (for
example, it's not enough to wrap non-blocking socket with wrap_socket()
call - resulting SSL socket won't be itself non-blocking).  Note that
.setblocking() propagates call to the underlying socket object, as
expected.
2019-04-30 17:26:37 +10:00
Paul Sokolovsky 9c7c082396 extmod/modussl_mbedtls: Support non-blocking handshake.
For this, add wrap_socket(do_handshake=False) param. CPython doesn't have
such a param at a module's global function, and at SSLContext.wrap_socket()
it has do_handshake_on_connect param, but that uselessly long.

Beyond that, make write() handle not just MBEDTLS_ERR_SSL_WANT_WRITE, but
also MBEDTLS_ERR_SSL_WANT_READ, as during handshake, write call may be
actually preempted by need to read next handshake message from peer.
Likewise, for read(). And even after the initial negotiation, situations
like that may happen e.g. with renegotiation. Both
MBEDTLS_ERR_SSL_WANT_READ and MBEDTLS_ERR_SSL_WANT_WRITE are however mapped
to the same None return code. The idea is that if the same read()/write()
method is called repeatedly, the progress will be made step by step anyway.
The caveat is if user wants to add the underlying socket to uselect.poll().
To be reliable, in this case, the socket should be polled for both POLL_IN
and POLL_OUT, as we don't know the actual expected direction. But that's
actually problematic. Consider for example that write() ends with
MBEDTLS_ERR_SSL_WANT_READ, but gets converted to None. We put the
underlying socket on pull using POLL_IN|POLL_OUT but that probably returns
immediately with POLL_OUT, as underlyings socket is writable. We call the
same ussl write() again, which again results in MBEDTLS_ERR_SSL_WANT_READ,
etc. We thus go into busy-loop.

So, the handling in this patch is temporary and needs fixing. But exact way
to fix it is not clear. One way is to provide explicit function for
handshake (CPython has do_handshake()), and let *that* return distinct
codes like WANT_READ/WANT_WRITE. But as mentioned above, past the initial
handshake, such situation may happen again with at least renegotiation. So
apparently, the only robust solution is to return "out of bound" special
sentinels like WANT_READ/WANT_WRITE from read()/write() directly. CPython
throws exceptions for these, but those are expensive to adopt that way for
efficiency-conscious implementation like MicroPython.
2019-04-30 17:24:46 +10:00
Damien George 775ffdcc3b extmod/machine_signal: Fix fault when no args are passed to Signal(). 2019-04-26 14:47:31 +10:00
Léa Saviot a6e5846ba7 extmod/modurandom: Add init method to seed the Yasmarang generator.
In CPython the random module is seeded differently on each import, and so
this new macro option MICROPY_PY_URANDOM_SEED_INIT_FUNC allows to implement
such a behaviour.
2019-04-16 14:54:36 +10:00
Damien George d5f0c87bb9 extmod/modlwip: Abort TCP conns that didn't close cleanly in a while. 2019-04-11 11:18:10 +10:00
Damien George 3dda964785 extmod/modlwip: Use correct listening socket object in accept callback.
Since commit da938a83b5 the tcp_arg() that is
set for the new connection is the new connection itself, and the parent
listening socket is found in the pcb->connected entry.
2019-04-03 16:43:44 +11:00
Damien George 2848a613ac extmod/modlwip: Free any stored incoming bufs/connections on TCP error. 2019-04-01 13:36:44 +11:00
Damien George 490e0f39d1 extmod/modlwip: Protect socket.accept with lwIP concurrency lock.
This is needed now that the accept queue can have pending connections
removed asynchronously.
2019-04-01 13:36:43 +11:00
Damien George 2ec7838967 extmod/modlwip: Handle case of accept callback called with null PCB. 2019-04-01 13:36:43 +11:00
Damien George da938a83b5 extmod/modlwip: Handle case of connection closing while on accept queue.
In such a case the connection is aborted by lwIP and so must be removed
from the pending accept queue.
2019-04-01 13:36:43 +11:00
Andrew Leech 9d6f70f715 stm32: Make default USB_VCP stream go through uos.dupterm for main REPL.
Use uos.dupterm for REPL configuration of the main USB_VCP(0) stream on
dupterm slot 1, if USB is enabled.  This means dupterm can also be used to
disable the boot REPL port if desired, via uos.dupterm(None, 1).

For efficiency this adds a simple hook to the global uos.dupterm code to
work with streams that are known to be native streams.
2019-04-01 13:04:05 +11:00
Andrew Leech 74d07469f2 extmod/vfs_fat: Fallback to FAT32 if standard FAT16/SFD format fails.
This allows formatting SD cards, larger flash etc which do not support the
default FAT16/SFD format mode.
2019-03-26 17:15:23 +11:00
Wolf Vollprecht 921b999225 extmod/moduselect: Adjust select_select and poll_register to use size_t. 2019-03-13 23:18:59 +11:00
Damien George 68a5d6fe77 extmod/modlwip: Fix case where concurrency lock isn't released on error. 2019-03-12 22:35:52 +11:00
Damien George e959f21986 extmod/vfs_fat: Update for new oofatfs version. 2019-03-05 15:56:39 +11:00
Damien George 9b2a97a903 extmod/modwebrepl: Fix logic to handle a put of file of size 0.
Fixes issue #4499.
2019-02-28 15:30:48 +11:00
Damien George ed1a88e263 extmod/modlwip: Don't require a port to define concurrency macros. 2019-02-27 10:27:56 +11:00
Damien George 39ea132e1d extmod/modlwip: Add concurrency protection macros.
Some users of this module may require the LwIP stack to run at an elevated
priority, to protect against concurrency issues with processing done by the
underlying network interface.  Since LwIP doesn't provide such protection
it must be done here (the other option is to run LwIP in a separate thread,
and use thread protection mechanisms, but that is a more heavyweight
solution).
2019-02-26 23:32:19 +11:00
Damien George 42c0e440b9 extmod/modlwip: Fix bug when polling listening socket with backlog=1.
The bug polling for readability was: if alloc==0 and tcp.item==NULL then
the code would incorrectly check tcp.array[iget] which is an invalid
dereference when alloc==0.  This patch refactors the code to use a helper
function lwip_socket_incoming_array() to return the correct pointer for the
incomming connection array.

Fixes issue #4511.
2019-02-18 14:23:35 +11:00
Damien George 7ef9482b8a extmod/modlwip: Change #ifdef to #if for check of MICROPY_PY_LWIP.
Otherwise this code will be included if MICROPY_PY_LWIP is defined to 0.
2019-02-15 16:07:24 +11:00
Yonatan Goldschmidt bc4f8b438b extmod/moduwebsocket: Refactor `websocket` to `uwebsocket`.
As mentioned in #4450, `websocket` was experimental with a single intended
user, `webrepl`. Therefore, we'll make this change without a weak
link `websocket` -> `uwebsocket`.
2019-02-14 00:35:45 +11:00
Damien George f03601779e extmod: Convert legacy uppercase macro names to lowercase. 2019-02-12 14:54:51 +11:00
Yonatan Goldschmidt f024b2610f extmod/moduhashlib: Include implementation of sha256 only when required.
Previously crypto-algorithms impl was included even if MICROPY_SSL_MBEDTLS
was in effect, thus we relied on the compiler/linker to cut out the unused
functions.
2019-02-07 23:27:58 +11:00
Paul Sokolovsky 2f5d113fad py/warning: Support categories for warnings.
Python defines warnings as belonging to categories, where category is a
warning type (descending from exception type). This is useful, as e.g.
allows to disable warnings selectively and provide user-defined warning
types.  So, implement this in MicroPython, except that categories are
represented just with strings.  However, enough hooks are left to implement
categories differently per-port (e.g. as types), without need to patch each
and every usage.
2019-01-31 16:48:30 +11:00
Damien George 2a7a307baa extmod/modlwip: Add support for polling UDP sockets for writability. 2019-01-31 11:22:03 +11:00
Paul Sokolovsky c7ed17bc4b extmod/modussl_mbedtls: Remove deprecated mbedtls/net.h header include.
This header is deprecated as of mbedtls 2.8.0, as shipped with Ubuntu
18.04.  Leads to #warning which is promoted to error with uPy compile
options.

Note that the current version of mbedtls is 2.14 at the time of writing.
2019-01-27 12:26:09 +11:00
Paul Sokolovsky 35687a87ec extmod/moduzlib: Update for uzlib 2.9.2. 2019-01-27 10:59:49 +11:00
Paul Sokolovsky b1cca8fbe0 extmod/uzlib: Update uzlib to v2.9.2.
Major changes include robust parsing of erroneous compressed streams and
updated API.
2019-01-27 10:59:30 +11:00
Damien George 7cd59c5bc3 py/mpconfig: Move MICROPY_VERSION macros to static ones in mpconfig.h.
It's more robust to have the version defined statically in a header file,
rather than dynamically generating it via git using a git tag.  In case
git doesn't exist, or a different source control tool is used, it's
important to still have the uPy version number available.
2018-12-22 01:40:38 +11:00
Paul Sokolovsky 38151f35c1 extmod/moductypes: Add aliases for native C types.
SHORT, INT, LONG, LONGLONG, and unsigned (U*) variants are being defined.
This is done at compile using GCC-style predefined macros like
__SIZEOF_INT__.  If the compiler doesn't have such defines, no such types
will be defined.
2018-12-10 14:40:43 +11:00
Paul Sokolovsky 9d864bde04 extmod/moductypes: Implement __int__ for PTR.
Allows to get address a pointer contains, as an integer.
2018-12-10 14:25:05 +11:00
Damien George 29da9f0670 extmod/modlwip: Fix read-polling of listening socket with a backlog.
The recent implementation of the listen backlog meant that the logic to
test for readability of such a socket changed, and this commit updates the
logic to work again.
2018-12-03 18:02:10 +11:00
Damien George 4737ff8054 extmod/modlwip: Implement TCP listen/accept backlog.
Array to hold waiting connections is in-place if backlog=1, else is a
dynamically allocated array.  Incoming connections are processed FIFO
style to maintain fairness.
2018-12-01 17:23:44 +11:00
Paul Sokolovsky 2411f42ccb extmod/moductypes: Make sizeof() accept "layout" parameter.
sizeof() can work in two ways: a) calculate size of already instantiated
structure ("sizeof variable") - in this case we already no layout; b) size
of structure decsription ("sizeof type"). In the latter case, LAYOUT_NATIVE
was assumed, but there should possibility to calculate size for other
layouts too. So, with this patch, there're now 2 forms:

uctypes.sizeof(struct)
uctypes.sizeof(struct_desc, layout)
2018-10-23 11:32:02 +11:00
Damien George 4904663748 extmod/modonewire: Fix reset timings to match 1-wire specs.
Fixes issue #4116.
2018-10-17 15:52:07 +11:00
Paul Sokolovsky 9fbd12f2fa extmod/moductypes: Accept OrderedDict as a structure description.
Using OrderedDict (i.e. stable order of fields) would for example allow to
automatically calculate field offsets in structures.
2018-10-13 16:08:12 +11:00
Paul Sokolovsky 18f45d2e23 extmod/moductypes: Remove BITFIELD from aggregate types enum.
This value is unused. It was an artifact of early draft design, but
bitfields were optimized to use scalar one-word encoding, to allow
compact encoding of typical multiple bitfields in MCU control
registers.
2018-10-05 17:02:15 +10:00
Damien George 05959c6465 extmod/moduhashlib: Add md5 implementation using mbedtls. 2018-09-12 16:08:53 +10:00
Damien George 87d45f4d49 extmod/moduhashlib: Use newer message digest API for mbedtls >=2.7.0.
Since mbedtls 2.7.0 new digest functions were introduced with a "_ret"
suffix to allow the functions to return an error message (eg, if the
underlying hardware acceleration failed).  These new functions must be used
instead of the old ones to prevent deprecation warnings, or link errors for
missing functions, depending on the mbedtls configuration.
2018-09-12 16:04:18 +10:00
Paul Sokolovsky 5fe3730a30 extmod/moduhashlib: Add md5 implementation, using axTLS.
MD5 is still widely used, and may be important in some cases for networking
interoperability, e.g. HTTP Digest authentication.
2018-09-11 14:51:52 +10:00
Damien George 0be2ea50e9 py/py.mk: Build axtls library directly from its source files.
This removes the need for a separate axtls build stage, and builds all
axtls object files along with other code.  This simplifies and cleans up
the build process, automatically builds axtls when needed, and puts the
axtls object files in the correct $(BUILD) location.

The MicroPython axtls configuration file is provided in
extmod/axtls-include/config.h
2018-09-08 00:07:23 +10:00
Damien George 206c65f22c extmod/modussl_axtls: Use MP_ROM_PTR for objects in allowed args array. 2018-08-14 21:47:07 +10:00
Damien George b8b2525576 extmod/modbtree: Update to work with new mp_stream_posix_XXX signatures. 2018-08-14 17:41:23 +10:00
Damien George 7a67f057d7 extmod/modussl: Support polling in ussl objects by passing through ioctl
The underlying socket can handling polling, and any other transparent ioctl
requests.  Note that CPython handles the case of polling an ssl object by
polling the file descriptor of the underlying socket file, and that
behaviour is emulated here.
2018-07-20 13:05:04 +10:00
Damien George 4a2051eec7 extmod/modlwip: Deregister all lwIP callbacks when closing a socket.
Otherwise they may be called on a socket that no longer exists.

For example, if the GC calls the finaliser on the socket and then reuses
its heap memory, the "callback" entry of the old socket may contain invalid
data.  If lwIP then calls the TCP callback the code may try to call the
user callback object which is now invalid.  The lwIP callbacks must be
deregistered during the closing of the socket, before all the pcb pointers
are set to NULL.
2018-07-20 12:59:24 +10:00
Damien George d974ee1c2f extmod/vfs_posix: Use DTTOIF if available to convert type in ilistdir. 2018-07-11 16:07:44 +10:00
Damien George c700ff52a0 extmod/vfs_posix: Support ilistdir with no (or empty) argument. 2018-07-10 12:51:09 +10:00
Damien George aa735dc6a4 extmod: Fix to support compiling with object representation D. 2018-07-08 23:15:44 +10:00
Damien George a3c3dbd955 extmod/vfs: Support opening a file descriptor (int) with VfsPosix.
Fixes issue #3865.
2018-07-03 13:04:29 +10:00
Damien George e30a5fc7bc extmod/modure: Add ure.sub() function and method, and tests.
This feature is controlled at compile time by MICROPY_PY_URE_SUB, disabled
by default.

Thanks to @dmazzella for the original patch for this feature; see #3770.
2018-07-02 14:55:02 +10:00
Damien George 1e9b871d29 extmod/modure: Add match.span(), start() and end() methods, and tests.
This feature is controlled at compile time by
MICROPY_PY_URE_MATCH_SPAN_START_END, disabled by default.

Thanks to @dmazzella for the original patch for this feature; see #3770.
2018-07-02 14:54:56 +10:00
Damien George 1f86460910 extmod/modure: Add match.groups() method, and tests.
This feature is controlled at compile time by MICROPY_PY_URE_MATCH_GROUPS,
disabled by default.

Thanks to @dmazzella for the original patch for this feature; see #3770.
2018-07-02 14:53:30 +10:00
Damien George 8769a3e38c extmod/moducryptolib: Don't include arpa/inet.h, it's not needed.
And some ports (eg esp8266) don't have it.
2018-06-27 16:44:59 +10:00
Yonatan Goldschmidt 31f2f1e967 extmod/moducryptolib: Shorten exception messages to reduce code size. 2018-06-27 16:44:59 +10:00
Yonatan Goldschmidt d0507c084c extmod/moducryptolib: Prefix all Python methods/objects with ucryptolib.
Follows what was done in b045ebd35 for uhashlib.
2018-06-27 16:44:59 +10:00
Yonatan Goldschmidt eacb233b8f extmod/moducryptolib: Add an mbedTLS implementation for this module. 2018-06-27 16:44:59 +10:00
Yonatan Goldschmidt e328b4593c extmod/moducryptolib: Refactor functions for clean interface with axTLS.
This will allow implementations other than axTLS.

This commit includes additions of checks and clarifications of exceptions
related to user input.

To make the interface cleaner, I've disallowed switching from encrypt to
decrypt in the same object, as this is not always possible with other
crypto libraries (not all libraries have AES_convert_key like axTLS).
2018-06-27 16:44:59 +10:00
Yonatan Goldschmidt 473fe45da2 extmod/moducryptolib: Optionally export MODE_* constants to Python.
Allow including crypto consts based on compilation settings.  Disabled by
default to reduce code size; if one wants extra code readability, can
enable them.
2018-06-27 16:29:26 +10:00
Paul Sokolovsky 567bc2d6ce extmod/moducryptolib: Add ucryptolib module with crypto functions.
The API follows guidelines of https://www.python.org/dev/peps/pep-0272/,
but is optimized for code size, with the idea that full PEP 0272
compatibility can be added with a simple Python wrapper mode.

The naming of the module follows (u)hashlib pattern.

At the bare minimum, this module is expected to provide:

* AES128, ECB (i.e. "null") mode, encrypt only

Implementation in this commit is based on axTLS routines, and implements
following:

* AES 128 and 256
* ECB and CBC modes
* encrypt and decrypt
2018-06-27 14:54:40 +10:00
Damien George e8398a5856 extmod: Update to use new mp_get_stream helper.
With this patch objects are only checked that they have the stream protocol
at the start of their use as a stream, and afterwards the efficient
mp_get_stream() helper is used to extract the stream protocol C methods.
2018-06-18 12:35:56 +10:00
Damien George 564abb01a5 extmod/vfs_fat_diskio: Factor disk ioctl code to reduce code size.
Functionality is unchanged.
2018-06-16 18:21:42 +10:00
Damien George d11fb09333 extmod/modussl_axtls: Fix __del__ to point to mp_stream_close_obj. 2018-06-13 13:16:21 +10:00
Damien George 035906419d extmod/uos_dupterm: Use native C stream methods on dupterm object.
This patch changes dupterm to call the native C stream methods on the
connected stream objects, instead of calling the Python readinto/write
methods.  This is much more efficient for native stream objects like UART
and webrepl and doesn't require allocating a special dupterm array.

This change is a minor breaking change from the user's perspective because
dupterm no longer accepts pure user stream objects to duplicate on.  But
with the recent addition of uio.IOBase it is possible to still create such
classes just by inheriting from uio.IOBase, for example:

    import uio, uos

    class MyStream(uio.IOBase):
        def write(self, buf):
            # existing write implementation
        def readinto(self, buf):
            # existing readinto implementation

    uos.dupterm(MyStream())
2018-06-12 15:06:11 +10:00
Yonatan Goldschmidt c2fb725e72 extmod/moduhashlib: Make function objects STATIC.
These are not exported to anyone anyway.
2018-06-12 13:50:11 +10:00
Yonatan Goldschmidt 6963ee9075 extmod/moduhashlib: Allow using the sha256 implementation of mbedTLS. 2018-06-12 13:50:11 +10:00
Yonatan Goldschmidt 6630354ffe extmod/moduhashlib: Allow to disable the sha256 class.
Via the config value MICROPY_PY_UHASHLIB_SHA256.  Default to enabled to
keep backwards compatibility.

Also add default value for the sha1 class, to at least document its
existence.
2018-06-12 13:50:11 +10:00
Yonatan Goldschmidt 38682d4629 extmod/moduhashlib: Reorder funcs so that they are grouped by hash type.
Makes the code much more readable by reducing the number of #ifdefs and
keeping related functions close.
2018-06-12 13:47:20 +10:00
Yonatan Goldschmidt b045ebd354 extmod/moduhashlib: Prefix all Python methods and objects with uhashlib.
For consistency with other modules, and to help avoid clashes with the
actual underlying functions that do the hashing (eg
crypto-algorithms/sha256.c:sha256_update).
2018-06-12 13:44:09 +10:00
Damien George c117effddd extmod/vfs: Introduce a C-level VFS protocol, with fast import_stat.
Following other C-level protocols, this VFS protocol is added to help
abstract away implementation details of the underlying VFS in an efficient
way.  As a starting point, the import_stat function is put into this
protocol so that the VFS sub-system does not need to know about every VFS
implementation in order to do an efficient stat for importing files.

In the future it might be worth adding other functions to this protocol.
2018-06-06 14:33:42 +10:00
Damien George d4ce57e4e3 extmod/vfs: Add fast path for stating VfsPosix filesystem. 2018-06-06 14:28:23 +10:00
Damien George 8d82b0edbd extmod: Add VfsPosix filesystem component.
This VFS component allows to mount a host POSIX filesystem within the uPy
VFS sub-system.  All traditional POSIX file access then goes through the
VFS, allowing to sandbox a uPy process to a certain sub-dir of the host
system, as well as mount other filesystem types alongside the host
filesystem.
2018-06-06 14:28:23 +10:00
Damien George f35aae366c extmod/vfs_fat: Rename FileIO/TextIO types to mp_type_vfs_fat_XXX.
So they don't clash with other VFS implementations.
2018-06-06 14:28:23 +10:00
Damien George 172c23fe5d extmod/vfs: Use u_rom_obj properly in argument structures. 2018-06-06 14:28:23 +10:00
Damien George ea22406f76 extmod/modussl_mbedtls: Use mbedtls_entropy_func for CTR-DRBG entropy.
If mbedtls_ctr_drbg_seed() is available in the mbedtls bulid then so should
be mbedtls_entropy_func().  Then it's up to the port to configure a valid
entropy source, eg via MBEDTLS_ENTROPY_HARDWARE_ALT.
2018-05-31 21:52:29 +10:00
Damien George 98b9f0fc9d extmod/modussl_mbedtls: Populate sock member right away in wrap_socket.
Otherwise the "sock" member may have an undefined value if wrap_socket
fails with an exception and exits early, and then if the finaliser runs it
will try to close an invalid stream object.

Fixes issue #3828.
2018-05-31 21:47:26 +10:00
Damien George 41766ba7e6 extmod/modlwip: Allow to compile with MICROPY_PY_LWIP disabled. 2018-05-21 16:46:30 +10:00
Damien George 1e2a6a84a2 extmod/modlwip: Set POLLHUP flag for sockets that are new.
This matches CPython behaviour on Linux: a socket that is new and not
listening or connected is considered "hung up".

Thanks to @rkojedzinszky for the initial patch, PR #3457.
2018-05-17 23:17:36 +10:00
Damien George 9c2044717c extmod/modlwip: Update to work with lwIP v2.0.
lwIP v2.0.3 has been tested with this lwip module and it works very well.
2018-05-17 12:58:34 +10:00
Damien George 318f874cda extmod/modlwip: In ioctl handle case when socket is in an error state.
Using MP_STREAM_POLL_HUP for ERR_RST state follows how *nix handles this
case.
2018-05-04 15:15:04 +10:00
Damien George 5936168150 extmod/uzlib: Fix C-language sequencing error with uzlib_get_byte calls.
The order of function calls in an arithmetic expression is undefined and so
they must be written out as sequential statements.

Thanks to @dv-extrarius for reporting this issue, see issue #3690.
2018-05-02 23:16:22 +10:00
Damien George 89b1c4a60c extmod/vfs: Delegate import_stat to vfs.stat to allow generic FS import. 2018-05-02 17:08:48 +10:00
Damien George bdff68db9c extmod/modlwip: Check if getaddrinfo() constraints are supported or not.
In particular don't issue a warning if the passed-in constraints are
actually supported because they are the default values.
2018-04-23 16:38:20 +10:00
armink 6a693db71d extmod/re1.5: Fix compilecode.c compile problem on IAR tool chain.
The 2nd and 3rd args of the ternary operator are treated like they are in
the same expression and must have similar types.  void is not compatible
with int so that's why the compiler is complaining.
2018-04-10 13:54:22 +10:00
Damien George cf31d384f1 py/stream: Switch stream close operation from method to ioctl.
This patch moves the implementation of stream closure from a dedicated
method to the ioctl of the stream protocol, for each type that implements
closing.  The benefits of this are:

1. Rounds out the stream ioctl function, which already includes flush,
   seek and poll (among other things).

2. Makes calling mp_stream_close() on an object slightly more efficient
   because it now no longer needs to lookup the close method and call it,
   rather it just delegates straight to the ioctl function (if it exists).

3. Reduces code size and allows future types that implement the stream
   protocol to be smaller because they don't need a dedicated close method.

Code size reduction is around 200 bytes smaller for x86 archs and around
30 bytes smaller for the bare-metal archs.
2018-04-10 13:41:32 +10:00
Tom Collins 4d3a92c67c extmod/vfs_fat: Add file size as 4th element of uos.ilistdir tuple. 2018-03-12 12:26:36 +11:00
Damien George 58ebeca6a9 drivers/bus: Pull out software SPI implementation to dedicated driver.
This patch takes the software SPI implementation from extmod/machine_spi.c
and moves it to a dedicated file in drivers/bus/softspi.c.  This allows the
SPI driver to be used independently of the uPy runtime, making it a more
general component.
2018-03-10 00:59:43 +11:00
Damien George 21d5527edf extmod/machine_spi: Make SPI protocol structure public.
So it can be referenced directly without the need for the uPy object.
2018-03-02 23:55:08 +11:00
Damien George 09be031e04 extmod/vfs_fat_diskio: Use a C-stack-allocated bytearray for block buf.
This patch eliminates heap allocation in the VFS FAT disk IO layer, when
calling the underlying readblocks/writeblocks methods.  The bytearray
object that is passed to these methods is now allocated on the C stack
rather than the heap (it's only 4 words big).

This means that these methods should not retain a pointer to the buffer
object that is passed in, but this was already a restriction because the
original heap-allocated bytearray had its buffer passed by reference.
2018-02-28 15:11:20 +11:00
Damien George 2ad555bc76 extmod/vfs_fat: Remove declaration of mp_builtin_open_obj.
It's declared already in py/builtin.h.
2018-02-23 17:41:47 +11:00
Damien George eb570f47a2 extmod/vfs_fat: Make fat_vfs_open_obj wrapper public, not its function.
This patch just moves the definition of the wrapper object fat_vfs_open_obj
to the location of the definition of its function, which matches how it's
done in most other places in the code base.
2018-02-23 17:33:26 +11:00
Damien George 638b860066 extmod/vfs_fat: Merge remaining vfs_fat_misc.c code into vfs_fat.c.
The only function left in vfs_fat_misc.c is fat_vfs_import_stat() which
can logically go into vfs_fat.c, allowing to remove vfs_fat_misc.c.
2018-02-23 17:24:57 +11:00
Damien George ae4a07730a extmod/vfs_fat: Move ilistdir implementation from misc to main file.
The fat_vfs_ilistdir2() function was only used by fat_vfs_ilistdir_func()
so moving the former into the same file as the latter allows it to be
placed directly into the latter function, thus saving code size.
2018-02-23 17:17:32 +11:00
Damien George d9bca1f7bd extmod/modujson: Implement ujson.dump() function. 2018-02-15 11:35:42 +11:00
Ayke van Laethem 7642785881 extmod/vfs_fat_file: Implement SEEK_CUR for non-zero offset.
CPython doesn't allow SEEK_CUR with non-zero offset for files in text mode,
and uPy inherited this behaviour for both text and binary files.  It makes
sense to provide full support for SEEK_CUR of binary-mode files in uPy, and
to do this in a minimal way means also allowing to use SEEK_CUR with
non-zero offsets on text-mode files.  That seems to be a fair compromise.
2018-01-31 17:33:07 +11:00
Damien George 46b35356e1 extmod/modframebuf: Add 8-bit greyscale format (GS8). 2017-12-14 17:36:13 +11:00
Petr Viktorin 34247465c3 extmod/modframebuf: Add 2-bit color format (GS2_HMSB).
This format is used in 2-color LED matrices and in e-ink displays like
SSD1606.
2017-12-14 17:13:02 +11:00
Damien George 1b223a42bf extmod/modure: Add cast to workaround bug in MSVC. 2017-12-13 22:22:57 +11:00
Damien George 7df2ebbfea extmod/modussl_mbedtls: Clean up mbedtls state when error during setup.
Without this patch, if the SSL handshake fails (eg the connection was lost)
then the mbedtls state (memory) will never be freed.
2017-12-13 14:48:53 +11:00
Damien George ab750ee2fb extmod/modure: Convert alloca() to use new scoped allocation API. 2017-12-11 13:49:09 +11:00
Damien George 6df7b2f2fe extmod/machine_signal: Change VLA to use new scoped allocation API. 2017-12-11 13:49:09 +11:00
Damien George 48f6990fbc extmod/modlwip: Commit TCP out data to lower layers if buffer gets full.
Dramatically improves TCP sending throughput because without an explicit
call to tcp_output() the data is only sent to the lower layers via the
lwIP slow timer which (by default) ticks every 500ms.
2017-11-24 15:52:32 +11:00
Damien George e511f24ddd extmod/modussl_axtls: Implement key and cert kw args to wrap_socket.
The key and cert must both be a str/bytes object in DER format.
2017-11-24 15:50:40 +11:00
Damien George 5e34a113ea py/runtime: Add MP_BINARY_OP_CONTAINS as reverse of MP_BINARY_OP_IN.
Before this patch MP_BINARY_OP_IN had two meanings: coming from bytecode it
meant that the args needed to be swapped, but coming from within the
runtime meant that the args were already in the correct order.  This lead
to some confusion in the code and comments stating how args were reversed.
It also lead to 2 bugs: 1) containment for a subclass of a native type
didn't work; 2) the expression "{True} in True" would illegally succeed and
return True.  In both of these cases it was because the args to
MP_BINARY_OP_IN ended up being reversed twice.

To fix these things this patch introduces MP_BINARY_OP_CONTAINS which
corresponds exactly to the __contains__ special method, and this is the
operator that built-in types should implement.  MP_BINARY_OP_IN is now only
emitted by the compiler and is converted to MP_BINARY_OP_CONTAINS by
swapping the arguments.
2017-11-24 14:48:23 +11:00
Damien George 12ad64bc55 extmod/vfs_fat: Mount FatFS on creation so VFS methods can be used.
It's possible to use the methods (eg ilistdir) of a VFS FatFS object
without it being mounted in the VFS itself.  This previously worked but
only because FatFS was "mounting" the filesystem automatically when any
function (eg f_opendir) was called.  But it didn't work for ports that used
synchronisation objects (_FS_REENTRANT) because they are only initialised
via a call to f_mount.  So, call f_mount explicitly when creating a new
FatFS object so that everything is set up correctly.  Then also provide a
finaliser to do the f_umount call, but only if synchronisation objects are
enabled (since otherwise the f_umount call does nothing).
2017-11-20 11:46:40 +11:00
Damien George 4601759bf5 py/objstr: Remove "make_qstr_if_not_already" arg from mp_obj_new_str.
This patch simplifies the str creation API to favour the common case of
creating a str object that is not forced to be interned.  To force
interning of a new str the new mp_obj_new_str_via_qstr function is added,
and should only be used if warranted.

Apart from simplifying the mp_obj_new_str function (and making it have the
same signature as mp_obj_new_bytes), this patch also reduces code size by a
bit (-16 bytes for bare-arm and roughly -40 bytes on the bare-metal archs).
2017-11-16 13:17:51 +11:00
Damien George 6bc55b657b extmod/vfs: Use existing qstr for forward-slash string object. 2017-11-16 13:13:24 +11:00
Christopher Cooper 7413b3ce3e extmod/moduhashlib: Enable SHA1 hashing when using "mbedtls" library.
The SHA1 hashing functionality is provided via the "axtls" library's
implementation, and hence is unavailable when the "axtls" library is not being
used.  This change provides the same SHA1 hashing functionality when using the
"mbedtls" library by using its implementation instead.
2017-11-12 21:46:23 +02:00
Paul Sokolovsky 1cf6d488b3 extmod/modussl_axtls: Typo fix in comment. 2017-11-02 00:16:03 +02:00
Paul Sokolovsky 0719c936fb extmod/modussl_axtls: socket_read: Handle EAGAIN.
If SSL_EAGAIN is returned (which is a feature of MicroPython's axTLS fork),
return EAGAIN.

Original axTLS returns SSL_OK both when there's no data to return to user
yet and when the underlying stream returns EAGAIN. That's not distinctive
enough, for example, original module code works well for blocking stream,
but will infinite-loop for non-blocking socket with EAGAIN. But if we fix
non-blocking case, blocking calls to .read() will return few None's initially
(while axTLS progresses thru handshake).

Using SSL_EAGAIN allows to fix non-blocking case without regressing the
blocking one.

Note that this only handles case of non-blocking reads of application data.
Initial handshake and writes still don't support non-blocking mode and must
be done in the blocking way.
2017-11-02 00:14:11 +02:00
Damien George 10b76a9620 extmod/modussl_mbedtls: Allow to compile with unix coverage build.
Fixes a few C warnings.  No functional changes.
2017-10-30 15:41:37 +11:00
Eric Poulsen 74ec52d857 extmod/modussl: Add finaliser support for ussl objects.
Per the comment found here
https://github.com/micropython/micropython-esp32/issues/209#issuecomment-339855157,
this patch adds finaliser code to prevent memory leaks from ussl objects,
which is especially useful when memory for a ussl context is allocated
outside the uPy heap.  This patch is in-line with the finaliser code found
in many modsocket implementations for various ports.

This feature is configured via MICROPY_PY_USSL_FINALISER and is disabled by
default because there may be issues using it when the ussl state *is*
allocated on the uPy heap, rather than externally.
2017-10-30 15:25:32 +11:00
Damien George c64eb4f8ce extmod/vfs: Replace VLA in proxy func with small, static sized array.
VLAs can be expensive on stack usage due to stack alignment requirements,
and also the fact that extra local variables are needed to track the
dynamic size of the stack.  So using fixed-size arrays when possible can
help to reduce code size and stack usage.

In this particular case, the maximum value of n_args in the VLA is 2 and so
it's more efficient to just allocate this array with a fixed size.  This
reduces code size by around 30 bytes on Thumb2 and Xtensa archs.  It also
reduces total stack usage of the function: on Thumb2 the usage with VLA is
between 40 and 48 bytes, which is reduced to 32; on Xtensa, VLA usage is
between 64 and 80 bytes, reduced to 32; on x86-64 it's at least 88 bytes
reduced to 80.
2017-10-27 18:01:25 +11:00
Damien George f4059dcc0c all: Use NULL instead of "" when calling mp_raise exception helpers.
This is the established way of doing it and reduces code size by a little
bit.
2017-10-24 22:39:36 +11:00
Damien George 9725a654bd extmod/uos_dupterm: Swallow any errors from dupterm closing the stream.
Without this the board will crash when deactivating a stream that doesn't
have a close() method (eg UART) or that raises an exception within the
method (eg user-defined function).
2017-10-19 14:10:17 +11:00
Damien George 37282f8fc1 extmod/uos_dupterm: Update uos.dupterm() and helper funcs to have index.
The uos.dupterm() signature and behaviour is updated to reflect the latest
enhancements in the docs.  It has minor backwards incompatibility in that
it no longer accepts zero arguments.

The dupterm_rx helper function is moved from esp8266 to extmod and
generalised to support multiple dupterm slots.

A port can specify multiple slots by defining the MICROPY_PY_OS_DUPTERM
config macro to an integer, being the number of slots it wants to have;
0 means to disable the dupterm feature altogether.

The unix and esp8266 ports are updated to work with the new interface and
are otherwise unchanged with respect to functionality.
2017-10-13 20:01:57 +11:00
Damien George a3dc1b1957 all: Remove inclusion of internal py header files.
Header files that are considered internal to the py core and should not
normally be included directly are:
    py/nlr.h - internal nlr configuration and declarations
    py/bc0.h - contains bytecode macro definitions
    py/runtime0.h - contains basic runtime enums

Instead, the top-level header files to include are one of:
    py/obj.h - includes runtime0.h and defines everything to use the
        mp_obj_t type
    py/runtime.h - includes mpstate.h and hence nlr.h, obj.h, runtime0.h,
        and defines everything to use the general runtime support functions

Additional, specific headers (eg py/objlist.h) can be included if needed.
2017-10-04 12:37:50 +11:00
Paul Sokolovsky aba1f9167a extmod/modure: Add stack overflow checking when executing a regex. 2017-10-03 00:20:10 +03:00
Paul Sokolovsky c9a0b2a818 extmod/re1.5: Upgrade to v0.8.2, adds hook for stack overflow checking. 2017-10-02 21:20:47 +03:00
Paul Sokolovsky 4a314a6f63 extmod/re1.5: Update to 0.8.1.
Allow literal minus in char classes to be in trailing position, e.g. [a-c-].
(Previously, minus was allowed only at the start.)

This increases ARM Thumb2 code size by 8 bytes.
2017-09-24 10:19:24 +03:00
Damien George b02be234e1 extmod/machine_pinbase: Put PinBase singleton in ROM.
This patch also removes the empty type "pinbase_type" (which crashes if
accessed) and uses "machine_pinbase_type" instead as the type of the
PinBase singleton.
2017-09-12 16:00:21 +10:00
Damien George beeb7483d8 extmod/modussl_mbedtls: Allow to compile with MBEDTLS_DEBUG_C disabled.
With MBEDTLS_DEBUG_C disabled the function mbedtls_debug_set_threshold()
doesn't exist.  There's also no need to call mbedtls_ssl_conf_dbg() so a
few bytes can be saved on disabling that and not needing the mbedtls_debug
callback.
2017-09-06 17:34:45 +10:00
Damien George 4a93801c12 all: Update Makefiles and others to build with new ports/ dir layout.
Also renames "stmhal" to "stm32" in documentation and everywhere else.
2017-09-06 14:09:13 +10:00
Damien George 09547f0f51 extmod/modubinascii: Only include uzlib/tinf.h when it's really needed. 2017-08-31 14:10:49 +10:00
Damien George 71c9cfb028 all: Convert remaining "mp_uint_t n_args" to "size_t n_args".
This is to have consistency across the whole repository.
2017-08-30 10:59:58 +10:00
Damien George 58321dd985 all: Convert mp_uint_t to mp_unary_op_t/mp_binary_op_t where appropriate
The unary-op/binary-op enums are already defined, and there are no
arithmetic tricks used with these types, so it makes sense to use the
correct enum type for arguments that take these values.  It also reduces
code size quite a bit for nan-boxing builds.
2017-08-29 13:16:30 +10:00
Damien George 1c6b442d32 extmod/modubinascii: Don't post-increment variable that won't be used. 2017-08-21 22:05:39 +10:00
Damien George 4c736ea8fc extmod,unix: For uos.stat interpret st_size member as an unsigned int.
This prevents large files (eg larger than 2gb on a 32-bit arch) from
showing up as having a negative size.  Fixes issue #3227.
2017-08-21 20:47:22 +10:00
Alex Robbins c89254fd0f extmod/modubinascii: Rewrite mod_binascii_a2b_base64.
This implementation ignores invalid characters in the input. This allows
it to decode the output of b2a_base64, and also mimics the behavior of
CPython.
2017-08-17 09:25:51 +03:00
Eric Poulsen d5191edf7f extmod/modussl_mbedtls.c: Add ussl.getpeercert() method.
Behaviour is as per CPython but only the binary form is implemented here.
A test is included.
2017-08-16 15:01:00 +10:00
Javier Candeira 35a1fea90b all: Raise exceptions via mp_raise_XXX
- Changed: ValueError, TypeError, NotImplementedError
  - OSError invocations unchanged, because the corresponding utility
    function takes ints, not strings like the long form invocation.
  - OverflowError, IndexError and RuntimeError etc. not changed for now
    until we decide whether to add new utility functions.
2017-08-13 22:52:33 +10:00
Damien George 0f12082f5b py,extmod,stmhal: Use "static inline" for funcs that should be inline.
"STATIC inline" can expand to "inline" if STATIC is defined to nothing, and
this case can lead to link errors.
2017-08-02 13:42:34 +10:00
Paul Sokolovsky ee04525097 extmod/modlwip: Implement setsockopt(IP_ADD_MEMBERSHIP).
Allows to join multicast groups.
2017-07-31 22:38:37 +03:00
Alexander Steffen 55f33240f3 all: Use the name MicroPython consistently in comments
There were several different spellings of MicroPython present in comments,
when there should be only one.
2017-07-31 18:35:40 +10:00
Damien George bbced3b4bb extmod: Use MP_ROM_INT for int values in an mp_rom_map_elem_t. 2017-07-31 13:00:34 +10:00
Paul Sokolovsky f2140f9446 extmod/mod{lwip,onewire,webrepl}: Convert to mp_rom_map_elem_t. 2017-07-29 18:24:16 +03:00
Paul Sokolovsky 036b58228c extmod/modframebuf: Use correct initialization for .locals_dict. 2017-07-29 10:26:41 +03:00
Damien George 653a0c2d71 extmod/machine_signal: Fix parsing of invert arg when Pin is first arg. 2017-07-26 12:51:46 +10:00
Eric Poulsen 6b4d4a25ce extmod/modussl_mbedtls: Implement non-blocking SSL sockets. 2017-07-26 11:34:33 +10:00
Damien George f3687109d5 extmod/modframebuf: Consistently use "col" as name for colour variables.
Thanks to @kamikaze, aka Oleg Korsak, for the original idea and patch.
2017-07-25 14:06:44 +10:00
Damien George 0893b273b9 extmod/modussl_mbedtls: Make socket.close() free all TLS resources.
Also, use mp_stream_close() helper to close the underlying socket.
2017-07-25 14:00:45 +10:00
Radomir Dopieralski 363087aa11 extmod/modframebuf: Fix invalid stride for odd widths in GS4_HMSB fmt.
Since the stride is specified in pixels, in a 4-bit horizontal format it
has to always be even, otherwise the computation is wrong and we can
write outside of the buffer sometimes.
2017-07-25 12:29:02 +10:00
Damien George a10467b58a extmod/modussl_mbedtls: When reading and peer wants to close, return 0.
If this particular code is returned then there's no more data, it's not
really an error.
2017-07-25 11:53:26 +10:00
Damien George aa7be82a4d all: Don't include system errno.h when it's not needed. 2017-07-24 18:43:14 +10:00
Damien George 513dfcf4fe extmod/modussl_mbedtls: Support server_side mode.
To use server_side mode one must pass valid values in the "key" and "cert"
parameters.
2017-07-24 15:08:59 +10:00
Paul Sokolovsky 4368ae3142 extmod/modussl_axtls: Allow to close ssl stream multiple times.
Make sure that 2nd close has no effect and operations on closed streams
are handled properly.
2017-07-20 00:20:53 +03:00
Damien George 761e4c7ff6 all: Remove trailing spaces, per coding conventions. 2017-07-19 13:12:10 +10:00
Alexander Steffen 299bc62586 all: Unify header guard usage.
The code conventions suggest using header guards, but do not define how
those should look like and instead point to existing files. However, not
all existing files follow the same scheme, sometimes omitting header guards
altogether, sometimes using non-standard names, making it easy to
accidentally pick a "wrong" example.

This commit ensures that all header files of the MicroPython project (that
were not simply copied from somewhere else) follow the same pattern, that
was already present in the majority of files, especially in the py folder.

The rules are as follows.

Naming convention:
* start with the words MICROPY_INCLUDED
* contain the full path to the file
* replace special characters with _

In addition, there are no empty lines before #ifndef, between #ifndef and
one empty line before #endif. #endif is followed by a comment containing
the name of the guard macro.

py/grammar.h cannot use header guards by design, since it has to be
included multiple times in a single C file. Several other files also do not
need header guards as they are only used internally and guaranteed to be
included only once:
* MICROPY_MPHALPORT_H
* mpconfigboard.h
* mpconfigport.h
* mpthreadport.h
* pin_defs_*.h
* qstrdefs*.h
2017-07-18 11:57:39 +10:00
Tom Collins 145796f037 py,extmod: Some casts and minor refactors to quiet compiler warnings. 2017-07-07 11:32:22 +10:00
Damien George f110dbd795 extmod/modujson: Properly initialise temporary StringIO object. 2017-07-05 10:38:20 +10:00
Damien George b86c65d31c extmod/modubinascii: Add check for empty buffer passed to hexlify.
Previous to this patch hexlify(b'', b':') would lead to a bad crash due to
the computed length of the result being -1=0xffffffff.
2017-07-03 14:52:00 +10:00
Paul Sokolovsky 58b7b01cb5 extmod/modure: If input string is bytes, return bytes results too.
This applies to match.group() and split().

For ARM Thumb2, this increased code size by 12 bytes.
2017-07-01 01:25:45 +03:00
Paul Sokolovsky 91e93a9684 extmod/moduzlib: decompress: Remove stale "(void)n_args".
n_args is now actually used in this function.
2017-06-24 16:36:05 +03:00
Damien George eeaab1897b extmmod/modonewire: Rename public module to mp_module_onewire.
This follows naming scheme of other modules in extmod.
2017-06-22 16:17:46 +10:00
Damien George 6cc4da4cb8 extmod: Move modonewire.c from esp8266 to extmod directory.
It's now generic enough to be used by any port.
2017-06-22 16:06:00 +10:00
Damien George 48d867b4a6 all: Make more use of mp_raise_{msg,TypeError,ValueError} helpers. 2017-06-15 11:54:41 +10:00
Paul Sokolovsky 82b9915b34 extmod/modussl_axtls: Implement server_hostname arg to wrap_socket().
As enabled by SNI support in axTLS v2+.
2017-06-14 01:01:12 +03:00
Paul Sokolovsky 75c3f2a7ab extmod/modussl_axtls: Update for axTLS 2.1.3.
ssl_client_new() accepts new SSL_EXTENSIONS* argument.
2017-06-13 17:41:46 +03:00
Paul Sokolovsky 0a7735f1a6 extmod/modframebuf: Fix signed/unsigned comparison pendantic warning.
Happened with 32-bit gcc 4.8.4.
2017-06-10 20:34:38 +03:00
Damien George 7ecfbb8267 extmod/vfs: Allow "buffering" and "encoding" args to VFS's open().
These args are currently ignored but are parsed to make it easier to
write portable scripts between CPython and MicroPython.
2017-06-07 15:29:53 +10:00
Damien George f6ef8e3f17 extmod/vfs: Allow to statvfs the root directory. 2017-06-07 15:17:45 +10:00
Paul Sokolovsky 50de6d2fab extmod/modlwip: accept: Fix error code for non-blocking mode.
In non-blocking mode, if no pending connection available, should return
EAGAIN, not ETIMEDOUT.
2017-06-04 13:45:37 +03:00
Paul Sokolovsky 5da8de2b66 extmod/modlwip: Fix error codes for duplicate calls to connect().
If socket is already connected, POSIX requires returning EISCONN. If
connection was requested, but not yet complete (for non-blocking
socket), error code is EALREADY.

http://pubs.opengroup.org/onlinepubs/7908799/xns/connect.html
2017-06-04 12:30:41 +03:00
Paul Sokolovsky a0dbbbebb8 extmod/modlwip: connect: For non-blocking mode, return EINPROGRESS.
Instead of ETIMEDOUT. This is consistent with POSIX:
http://pubs.opengroup.org/onlinepubs/7908799/xns/connect.html
2017-06-03 22:48:31 +03:00
Ville Skyttä ca16c38210 various: Spelling fixes 2017-05-29 11:36:05 +03:00