Commit Graph

912 Commits

Author SHA1 Message Date
Damien George
372e5a280e extmod/btstack: Implement gatts_db for btstack.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George
86c26db3ff extmod/btstack: Implement service registration.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George
0674917bc5 extmod/btstack: Implement advertising.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George
fbefcef1df extmod/btstack: Add empty modbluetooth implementation.
Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George
0ac06a510a extmod/modbluetooth: Extract out gatts_db functionality from nimble.
For use by other stacks, if they need it.

Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George
894c550c86 stm32: Refactor bluetooth stack/hci/driver bindings.
This makes a cleaner separation between the: driver, HCI UART and BT stack.
Also updated the naming to be more consistent (mp_bluetooth_hci_*).

Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-10 01:53:42 +11:00
Damien George
c44d52f33e extmod/modbluetooth_nimble: Move nimble specific code, factor nimble.mk.
Move extmod/modbluetooth_nimble.* to extmod/nimble.  And move common
Makefile lines to extmod/nimble/nimble.mk (which was previously only used
by stm32).  This allows (upcoming) btstack to follow a similar structure.

Work done in collaboration with Jim Mussared aka @jimmo.
2020-03-06 12:35:20 +11:00
Damien George
69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
Jim Mussared
f8449dd092 extmod/modframebuf: Allow blit source to be a subclass of FrameBuffer. 2020-02-21 13:32:48 +11:00
Jim Mussared
cddb90e8b0 extmod/modbluetooth_nimble: Fix wrong offset used for descriptor flags. 2020-02-18 16:37:38 +11:00
Jim Mussared
66ac2e1fc0 extmod/modbluetooth.h: Fix typo in comment about registering services. 2020-02-18 16:36:37 +11:00
Thomas Friebel
f4726735cf extmod/modbluetooth: Implement config getter for BLE rxbuf size.
Knowing the buffer size can be important, to ensure that valid data will be
received.
2020-02-18 13:37:50 +11:00
Damien George
ce39c958ef py: Factor out definition of mp_float_union_t to one location. 2020-02-18 13:04:36 +11:00
Damien George
ad7213d3c3 py: Add mp_raise_msg_varg helper and use it where appropriate.
This commit adds mp_raise_msg_varg(type, fmt, ...) as a helper for
nlr_raise(mp_obj_new_exception_msg_varg(type, fmt, ...)).  It makes the
C-level API for raising exceptions more consistent, and reduces code size
on most ports:

   bare-arm:   +28 +0.042%
minimal x86:  +100 +0.067%
   unix x64:   -56 -0.011%
unix nanbox:  -300 -0.068%
      stm32:  -204 -0.054% PYBV10
     cc3200:    +0 +0.000%
    esp8266:   -64 -0.010% GENERIC
      esp32:  -104 -0.007% GENERIC
        nrf:  -136 -0.094% pca10040
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2020-02-13 11:52:40 +11:00
Damien George
97eca38c4f py: Add mp_raise_type helper macro and use it where appropriate.
This provides a more consistent C-level API to raise exceptions, ie moving
away from nlr_raise towards mp_raise_XXX.  It also reduces code size by a
small amount on some ports.
2020-02-13 11:03:37 +11:00
David Lechner
35f66d38b8 extmod/vfs_posix: Release GIL during system calls.
This releases the GIL during syscalls that could block.
2020-01-26 23:26:14 +11:00
Damien George
bfbd94401d py: Make mp_obj_get_type() return a const ptr to mp_obj_type_t.
Most types are in rodata/ROM, and mp_obj_base_t.type is a constant pointer,
so enforce this const-ness throughout the code base.  If a type ever needs
to be modified (eg a user type) then a simple cast can be used.
2020-01-09 11:25:26 +11:00
Yonatan Goldschmidt
aca8873bb8 extmod/modbluetooth: Fix func prototype, empty args should be (void).
This fixes a -Wstrict-prototypes error.
2019-12-27 23:57:46 +11:00
Damien George
6f872f81d6 extmod: Fix modbluetooth and modwebrepl to build in nanbox mode. 2019-12-27 22:54:53 +11:00
Damien George
d97b40bdaa py: Introduce MP_ROM_FALSE/MP_ROM_TRUE for ROM to refer to bool objects.
This helps to prevent mistakes, and allows easily changing the ROM value of
False/True if needed.
2019-12-27 22:54:20 +11:00
Damien George
09376f0e47 py: Introduce MP_ROM_NONE macro for ROM to refer to None object.
This helps to prevent mistakes, and allows easily changing the ROM value of
None if needed.
2019-12-27 22:51:17 +11:00
Damien George
ed2be79b49 extmod/uzlib: Explicitly cast ptr-diff-expr to unsigned.
The struct member "dest" should never be less than "destStart", so their
difference is never negative.  Cast as such to make the comparison
explicitly unsigned, ensuring the compiler produces the correct comparison
instruction, and avoiding any compiler warnings.
2019-12-23 00:07:03 +11:00
Jim Mussared
7ce1e0b1dc extmod/webrepl: Move webrepl scripts to common place and use manifest.
Move webrepl support code from ports/esp8266/modules into extmod/webrepl
(to be alongside extmod/modwebrepl.c), and use frozen manifests to include
it in the build on esp8266 and esp32.

A small modification is made to webrepl.py to make it work on non-ESP
ports, i.e. don't call dupterm_notify if not available.
2019-12-20 12:59:13 +11:00
Damien George
ba84453f77 examples/natmod: Add urandom native module example. 2019-12-13 13:33:40 +11:00
Damien George
bbeaafd9aa extmod: Add dynamic-runtime guards to btree/framebuf/uheapq/ure/uzlib.
So they can be built as dynamic native modules, as well as existing static
native modules.
2019-12-12 20:15:28 +11:00
Damien George
e5acd06ad5 extmod/modbtree: Use mp_printf instead of printf. 2019-12-12 20:15:28 +11:00
Damien George
84958a8fe1 extmod/modbluetooth: Allow setting ringbuf size via BLE.config(rxbuf=).
The size of the event ringbuf was previously fixed to compile-time config
value, but it's necessary to sometimes increase this for applications that
have large characteristic buffers to read, or many events at once.

With this commit the size can be set via BLE.config(rxbuf=512), for
example.  This also resizes the internal event data buffer which sets the
maximum size of incoming data passed to the event handler.
2019-12-05 11:30:35 +11:00
Damien George
7aeafe2ae9 extmod/modbluetooth: Add optional 4th arg to gattc_write for write mode.
This allows the user to explicitly select the behaviour of the write to the
remote peripheral.  This is needed for peripherals that have
characteristics with WRITE_NO_RESPONSE set (instead of normal WRITE).  The
function's signature is now:

    BLE.gattc_write(conn_handle, value_handle, data, mode=0)

mode=0 means write without response, while mode=1 means write with
response.  The latter was the original behaviour so this commit is a change
in behaviour of this method, and one should specify 1 as the 4th argument
to get back the old behaviour.

In the future there could be more modes supported, such as long writes.
2019-12-04 23:23:07 +11:00
Damien George
40cc7ec677 stm32/mpconfigport.h: Use IRQ_PRI_PENDSV to protect bluetooth ringbuf.
The default protection for the BLE ringbuf is to use
MICROPY_BEGIN_ATOMIC_SECTION, which disables all interrupts. On stm32 it
only needs to disable the lowest priority IRQ, pendsv, because that's the
IRQ level at which the BLE stack is driven.
2019-12-04 13:39:22 +11:00
Damien George
8ce69288e9 extmod/modbluetooth: Remove limit on data coming from gattc data input.
This removes the limit on data coming in from a BLE.gattc_read() request,
or a notify with payload (coming in to a central).  In both cases the data
coming in to the BLE callback is now limited only by the available data in
the ringbuf, whereas before it was capped at (default hard coded) 20 bytes.
2019-12-02 23:27:25 +11:00
Damien George
d6e051082a extmod/modbluetooth: Simplify how BLE IRQ callback is scheduled.
Instead of enqueue_irq() inspecting the ringbuf to decide whether to
schedule the IRQ callback (if ringbuf is empty), maintain a flag that knows
if the callback is on the schedule queue or not.  This saves about 150
bytes of code (for stm32 builds), and simplifies all uses of enqueue_irq()
and schedule_ringbuf().
2019-12-02 23:25:36 +11:00
Damien George
6b3404f25e extmod/vfs_lfs: Fix bug when passing no args to constructor and mkfs. 2019-11-26 00:08:57 +11:00
Damien George
5634a31a98 extmod/vfs_lfs: Pass flag along to ioctl when init'ing bdev for lfs.
To hint to the block device that the extended block protocol will be used.
2019-11-26 00:07:42 +11:00
Jim Mussared
e873d352ad extmod/modbluetooth: Simplify management of pre-allocated event data.
The address, adv payload and uuid fields of the event are pre-allocated by
modbluetooth, and reused in the IRQ handler.  Simplify this and move all
storage into the `mp_obj_bluetooth_ble_t` instance.

This now allows users to hold on to a reference to these instances without
crashes, although they may be overwritten by future events.  If they want
to hold onto the values longer term they need to copy them.
2019-11-25 17:32:10 +11:00
Jim Mussared
438c0dc2a4 extmod/modbluetooh_nimble: Fix UUID conversion for 16 and 32 bit values. 2019-11-25 17:31:59 +11:00
Jim Mussared
2ae755d9e1 extmod/modbluetooth_nimble: Make gap_scan_stop no-op if no scan ongoing.
No need for this to throw an exception if the intent (don't be scanning) is
clear, and avoids a race with the scan duration timeout.
2019-11-25 17:27:40 +11:00
Jim Mussared
d19c6d0519 extmod/modbluetooth: Create UUID from bytes and allow comparison ops.
This allows construction of UUID objects from advertising data payloads and
matching against known UUIDs.
2019-11-25 17:20:51 +11:00
Jim Mussared
334ba01c90 extmod/modbluetooth: Prioritise non-scan-result events.
Remove existing scan result events from the ringbuf if the ringbuf is full
and we're trying to enqueue any other event.  This is needed so that events
such as SCAN_COMPLETE are always put on the ringbuf.
2019-11-21 12:04:57 +11:00
Damien George
799b6d1e0c extmod: Consolidate FAT FS config to MICROPY_VFS_FAT across all ports.
This commit removes the Makefile-level MICROPY_FATFS config and moves the
MICROPY_VFS_FAT config to the Makefile level to replace it.  It also moves
the include of the oofatfs source files in the build from each port to a
central place in extmod/extmod.mk.

For a port to enabled VFS FAT support it should now set MICROPY_VFS_FAT=1
at the level of the Makefile.  This will include the relevant oofatfs files
in the build and set MICROPY_VFS_FAT=1 at the C (preprocessor) level.
2019-11-11 11:37:38 +11:00
Damien George
9b27069e2f extmod/vfs: Add autodetect of littlefs filesystem when mounting. 2019-11-06 12:15:34 +11:00
Damien George
c13f9f209d all: Convert nlr_raise(mp_obj_new_exception_msg(x)) to mp_raise_msg(x).
This helper function was added a while ago and these are the remaining
cases to convert, to save a bit of code size.
2019-11-05 11:35:45 +11:00
Damien George
2ee9e1a4ed extmod/modbtree: Make FILEVTABLE const to put it in ROM. 2019-11-01 17:25:40 +11:00
Damien George
40ea1915fc extmod/nimble: Factor out stm32-specific HCI UART RX/TX code. 2019-11-01 12:41:37 +11:00
Damien George
78145b98ef extmod/nimble: Remove unneeded nimble_sprintf wrapper function. 2019-11-01 12:15:07 +11:00
Damien George
9dd9f9ff06 extmod/modussl_mbedtls: Check for invalid key/cert data. 2019-10-31 16:22:42 +11:00
Damien George
07ea81fbc5 extmod/modussl_mbedtls: Fix getpeercert to return None if no cert avail. 2019-10-31 13:42:24 +11:00
Damien George
26d8fd2c0a extmod/modlwip: Unconditionally return POLLHUP/POLLERR when polling.
POSIX poll should always return POLLERR and POLLHUP in revents, regardless
of whether they were requested in the input events flags.

See issues #4290 and #5172.
2019-10-31 13:37:51 +11:00
Damien George
feaa251674 extmod/modlwip: Make socket poll return POLLNVAL in case of bad file. 2019-10-31 12:54:37 +11:00
Damien George
71401d5065 extmod/modlwip: Unconditionally return POLLHUP when polling new socket.
POSIX poll should always return POLLERR and POLLHUP in revents, regardless
of whether they were requested in the input events flags.

See issues #4290 and #5172.
2019-10-31 12:54:37 +11:00
Damien George
660a61a388 extmod/vfs_lfs: Allow compiling in VfsLfs1 and VfsLfs2 separately.
These classes are enabled via the config options MICROPY_VFS_LFS1 and
MICROPY_VFS_LFS2, which are disabled by default.
2019-10-30 12:08:58 +11:00
Jim Mussared
d16a27da51 extmod/modbluetooth: Add gatts_set_buffer.
- Adds an explicit way to set the size of a value's internal buffer,
  replacing `ble.gatts_write(handle, bytes(size))` (although that
  still works).
- Add an "append" mode for values, which means that remote writes
  will append to the buffer.
2019-10-29 23:09:02 +11:00
Damien George
4cf054a130 extmod/vfs: Add MP_BLOCKDEV_IOCTL_BLOCK_ERASE constant. 2019-10-29 14:17:29 +11:00
Damien George
cfe1c5abf8 extmod/vfs: Rename BP_IOCTL_xxx constants to MP_BLOCKDEV_IOCTL_xxx.
Also rename SEC_COUNT to BLOCK_COUNT and SEC_SIZE to BLOCK_SIZE.
2019-10-29 14:17:29 +11:00
Damien George
a099505420 extmod: Add VFS littlefs bindings.
Both LFS1 and LFS2 are supported at the same time.
2019-10-29 14:17:29 +11:00
Damien George
98beea9ced extmod/vfs_blockdev: Add extended read/write methods.
This commit adds helper functions to call readblocks/writeblocks with a
fourth argument, the byte offset within a block.

Although the mp_vfs_blockdev_t struct has grown here by 2 machine words, in
all current uses of this struct within this repository it still fits within
the same number of GC blocks.
2019-10-29 12:55:34 +11:00
Damien George
e1c7b1cb43 extmod/vfs_blockdev: Factor out block device interface code. 2019-10-29 12:55:17 +11:00
Damien George
9aabb6c01b extmod: Factor out block-device struct to make independent of fatfs. 2019-10-29 12:12:37 +11:00
Jim Mussared
b02d7e612d extmod/modbluetooth: Rename module to "ubluetooth".
For consistency with "umachine". Now that weak links are enabled
by default for built-in modules, this should be a no-op, but allows
extension of the bluetooth module by user code.

Also move registration of ubluetooth to objmodule rather than
port-specific.
2019-10-22 21:58:05 +11:00
Jim Mussared
2c1f269918 extmod/modbluetooth_nimble: Use data_alloc length to truncate writes.
This allows the maximum size of a characteristic/descriptor to be
increased by locally writing to it first.
2019-10-22 14:30:23 +11:00
Jim Mussared
f34e16dbc6 extmod/modbluetooth: Persist reference to NimBLE service instances.
NimBLE doesn't actually copy this data, it requires it to stay live.
Only dereference when we register a new set of services.

Fixes #5226

This will allow incrementally adding services in the future, so
rename `reset` to `append` to make it clearer.
2019-10-22 14:30:23 +11:00
Jim Mussared
56fc3edf98 extmod/modbluetooth: Make UUID support the buffer protocol.
Internally change the representation of UUIDs to LE uint8* to simplify this.

This allows UUIDs to be easily used in BLE payloads (such as advertising).

Ref: #5186
2019-10-22 13:54:05 +11:00
Jim Mussared
ebf8332104 extmod/re1.5: Support escaping within RE classes.
Fixes issues #3178 and #5220.

Tests are added, including all the cases mentioned in both bugs.
2019-10-18 12:20:32 +11:00
Damien George
8f7f671236 extmod/modbluetooth: In gap_advertise only accept None to stop adv.
To match the docs, and interval=0 may be used in the future to indicate
something else.
2019-10-15 17:29:27 +11:00
Jim Mussared
36502bdfdc extmod/modbluetooth: Make gap_disconnect not raise when disconnected.
Previously it raised OSError(MP_ENOTCONN) if the conn_handle was already
disconnected.  Now it returns True/False.
2019-10-15 17:22:53 +11:00
Jim Mussared
423e67d0a0 extmod/modbluetooth: Improve ringbuf handling.
No need to share the irq_data buffer with addresses.  Split them into two
separate buffers and manage their max length independently.
2019-10-15 17:22:07 +11:00
Jim Mussared
cb73103f57 extmod/modbluetooth: Fix order of params to IRQ_GATTS_WRITE event. 2019-10-15 17:22:01 +11:00
Jim Mussared
ba16a22991 extmod/modbluetooth: Clear gap_advertise payload when data is empty.
Also fix default adv interval to 500ms.
2019-10-15 17:21:17 +11:00
Jim Mussared
418f12c5f5 extmod/modbluetooth: Increase maximum connections from 1 to 4.
This avoids a confusing ENOMEM raised from gap_advertise if there is
currently an active connection.  This refers to the static connection
buffer pre-allocated by Nimble (nothing to do with MicroPython heap
memory).
2019-10-15 17:20:25 +11:00
Jim Mussared
76f474129e extmod/modbluetooth: Use us instead of ms for advertising interval.
This is to more accurately match the BLE spec, where intervals are
configured in units of channel hop time (625us). When it was
specified in ms, not all "valid" intervals were able to be
specified.

Now that we're also allowing configuration of scan interval, this
commit updates advertising to match.
2019-10-11 13:51:07 +11:00
Jim Mussared
b65cc387cd extmod/modbluetooth: Allow config of scan interval/window.
This adds two additional optional kwargs to `gap_scan()`:
  - `interval_us`: How long between scans.
  - `window_us`: How long to scan for during a scan.

The default with NimBLE is a 11.25ms window with a 1.28s interval.

Changing these parameters is important for detecting low-frequency
advertisements (e.g. beacons).

Note: these params are in microseconds, not milliseconds in order
to allow the 625us granularity offered by the spec.
2019-10-11 13:51:07 +11:00
Jim Mussared
902bb4ceae stm32: Extract port-specific Nimble implementation.
On other ports (e.g. ESP32) they provide a complete Nimble implementation
(i.e. we don't need to use the code in extmod/nimble). This change
extracts out the bits that we don't need to use in other ports:
 - malloc/free/realloc for Nimble memory.
 - pendsv poll handler
 - depowering the cywbt

Also cleans up the root pointer management.
2019-10-08 14:40:35 +11:00
Andrew Leech
eb1b6858a2 extmod/modbluetooth: Allow MP_BLUETOOTH_MAX_ATTR_SIZE in board config. 2019-10-01 09:51:02 +10:00
Andrew Leech
5dc592d117 extmod/modbluetooth_nimble: Use random addr if public isn't available. 2019-10-01 09:51:02 +10:00
Jim Mussared
497dae45e7 extmod/modbluetooth_nimble: Implement modbluetooth API with Nimble. 2019-10-01 09:51:02 +10:00
Jim Mussared
16f8ceeaaa extmod/modbluetooth: Add low-level Python BLE API. 2019-10-01 09:51:02 +10:00
Damien George
07f6644a38 extmod/nimble: Add nimble bindings. 2019-10-01 09:51:02 +10:00
Bob Fanger
bff2771da1 extmod/vfs_posix: Include stdio.h for declaration of function 'rename'. 2019-09-23 15:14:42 +10:00
Damien George
24c3e9b283 py/modstruct: Fix struct.pack_into with unaligned offset of native type.
Following the same fix for unpack.
2019-09-02 13:14:16 +10:00
Tom McDermott
1022f9cc35 py/modstruct: Fix struct.unpack with unaligned offset of native type.
With this patch alignment is done relative to the start of the buffer that
is being unpacked, not the raw pointer value, as per CPython.

Fixes issue #3314.
2019-09-02 13:10:55 +10:00
Damien George
519746cae4 extmod/crypto-algorithms: Add source to header and populate copyright.
As per the README.md of the upstream source at
https://github.com/B-Con/crypto-algorithms, this source code was released
into the public domain, so make that explicit in the copyright line in the
header.
2019-08-22 17:20:16 +10:00
Damien George
2dfa69efbb extmod/modujson: Support passing bytes/bytearray to json.loads.
CPython allows this, and it can be useful to reduce the number of memory
allocations.

Fixes issue #5031.
2019-08-22 15:32:26 +10:00
Damien George
0cc8910bc5 extmod: Give vars/funcs unique names so STATIC can be set to nothing.
Fixes issue #5018.
2019-08-20 15:21:09 +10:00
Damien George
3327dfc16e extmod/moducryptolib: Use "static" not "STATIC" for inline functions. 2019-08-20 15:21:09 +10:00
Damien George
7d851a27f1 extmod/modure: Make regex dump-code debugging feature optional.
Enabled via MICROPY_PY_URE_DEBUG, disabled by default (but enabled on unix
coverage build).  This is a rarely used feature that costs a lot of code
(500-800 bytes flash).  Debugging of regular expressions can be done
offline with other tools.
2019-08-19 16:43:00 +10:00
Damien George
80f5cef8d4 extmod/modlwip: Implement raw sockets for lwIP.
Configurable via MICROPY_PY_LWIP_SOCK_RAW.
2019-08-06 15:56:05 +10:00
Damien George
c60caf1995 extmod/modlwip: Use mp_sched_schedule to schedule socket callbacks.
The helper function exec_user_callback executes within the context of an
lwIP C callback, and the user (Python) callback to be scheduled may want to
perform further TCP/IP actions, so the latter should be scheduled to run
outside the lwIP context (otherwise it's effectively a "hard IRQ" and such
callbacks have lots of restrictions).
2019-07-03 16:22:48 +10:00
Damien George
1d6cb6357a extmod/modlwip: For TCP send keep trying tcp_write if it returns ERR_MEM
If tcp_write returns ERR_MEM then it's not a fatal error but instead means
the caller should retry the write later on (and this is what lwIP's netconn
API does).

This fixes problems where a TCP send would raise OSError(ENOMEM) in
situations where the TCP/IP stack is under heavy load.  See eg issues #1897
and #1971.
2019-07-03 15:50:13 +10:00
Damien George
ef00048fed extmod/modwebrepl: Add config option to put filebuf[512] on stack/bss.
Since the esp8266 has a small stack this buffer is kept in the BSS.
2019-07-03 12:55:57 +10:00
Damien George
fa2c7ece8f extmod/modwebrepl: Make prompt/ver static arrays const to not use RAM.
The esp8266 lwip_open library is compiled with -mforce-l32 so these arrays
do not need to be in RAM.
2019-07-03 12:50:13 +10:00
Damien George
fad3d08d2d extmod/moduwebsocket: Make close_resp static array const to not use RAM.
The esp8266 lwip_open library is compiled with -mforce-l32 so this array
does not need to be in RAM.
2019-07-03 12:47:13 +10:00
Damien George
89a23a05b3 esp8266: Provide custom machine_time_pulse_us that feeds soft WDT.
So that the timeout for machine.time_pulse_us() can be large.

Fixes issue #2775.
2019-07-01 22:53:00 +10:00
Damien George
964ae328cd extmod/uos_dupterm: Add mp_uos_dupterm_poll to poll all dupterms. 2019-07-01 17:10:12 +10:00
Damien George
49388e339e extmod/extmod.mk: Include mdns app source in lwIP build. 2019-06-05 16:13:34 +10:00
Damien George
ef7357d4ab extmod/modussl_mbedtls: Allow to build with object representation D. 2019-06-05 15:33:15 +10:00
Damien George
678ec182cd extmod/extmod.mk: Integrate mbedTLS so it is built from source.
Setting MICROPY_PY_USSL and MICROPY_SSL_MBEDTLS at the Makefile-level will
now build mbedTLS from source and include it in the build, with the ussl
module using this TLS library.  Extra settings like MBEDTLS_CONFIG_FILE may
need to be provided by a given port.

If a port wants to use its own mbedTLS library then it should not set
MICROPY_SSL_MBEDTLS at the Makefile-level but rather set it at the C level,
and provide the library as part of the build in its own way (see eg esp32
port).
2019-06-05 15:22:21 +10:00
Damien George
cd6b115815 extmod: Factor out makefile rules from py.mk to new extmod.mk file.
To logically separate extmod related rules out, and prevent py.mk from
growing too large.
2019-06-05 14:23:12 +10:00
Damien George
cf1c131f73 extmod: Add network-level class binding to cyw43 driver. 2019-06-03 16:47:35 +10:00
Damien George
019dd84af1 extmod/modlwip: Register TCP close-timeout callback before closing PCB.
In d5f0c87bb9 this call to tcp_poll() was
added to put a timeout on closing TCP sockets.  But after calling
tcp_close() the PCB may be freed and therefore invalid, so tcp_poll() can
not be used at that point.  As a fix this commit calls tcp_poll() before
closing the TCP PCB.  If the PCB is subsequently closed and freed by
tcp_close() or tcp_abort() then the PCB will not be on any active list and
the callback will not be executed, which is the desired behaviour (the
_lwip_tcp_close_poll() callback only needs to be called if the PCB remains
active for longer than the timeout).
2019-05-29 01:29:48 +10:00
Damien George
734ada3e29 extmod/modlwip: Free any incoming bufs/connections before closing PCB.
Commit 2848a613ac introduced a bug where
lwip_socket_free_incoming() accessed pcb.tcp->state after the PCB was
closed.  The state may have changed due to that close call, or the PCB may
be freed and therefore invalid.  This commit fixes that by calling
lwip_socket_free_incoming() before the PCB is closed.
2019-05-29 01:24:43 +10:00
Damien George
b10d0664be extmod/machine_i2c: Add i2c.writevto() that can write a vector of bufs.
For example: i2c.writevto(addr, (buf1, buf2)).  This allows to efficiently
(wrt memory) write data composed of separate buffers, such as a command
followed by a large amount of data.
2019-05-20 15:04:29 +10:00
Damien George
8bcb552d97 extmod/machine_i2c: Remove need for temporary memory in writemem() call. 2019-05-20 15:04:29 +10:00
Damien George
606ea2b10f extmod/machine_i2c: Change C-level API to allow split I2C transactions.
API is:

    int transfer(
        mp_obj_base_t *obj,
        uint16_t addr,
        size_t n,
        mp_machine_i2c_buf_t *bufs,
        unsigned int flags
    )
2019-05-20 15:04:29 +10:00
Paul Sokolovsky
016d9a40fe various: Add and update my copyright line based on git history.
For modules I initially created or made substantial contributions to.
2019-05-17 18:04:15 +10:00
Damien George
7c5cf59f8b extmod/modujson: Handle parsing of floats with + in the exponent.
Fixes issue #4780.
2019-05-14 14:45:54 +10:00
Yonatan Goldschmidt
32ba679924 extmod/moducryptolib: Add AES-CTR support for axTLS builds. 2019-05-06 18:20:56 +10:00
Yonatan Goldschmidt
ef9843653b extmod/moducryptolib: Add AES-CTR support.
Selectable at compile time via MICROPY_PY_UCRYPTOLIB_CTR.  Disabled by
default.
2019-05-06 18:09:48 +10:00
Paul Sokolovsky
c76445315f extmod/modussl_axtls: Add non-blocking mode support.
It consists of:

1. "do_handhake" param (default True) to wrap_socket(). If it's False,
handshake won't be performed by wrap_socket(), as it would be done in
blocking way normally. Instead, SSL socket can be set to non-blocking mode,
and handshake would be performed before the first read/write request (by
just returning EAGAIN to these requests, while instead reading/writing/
processing handshake over the connection). Unfortunately, axTLS doesn't
really support non-blocking handshake correctly. So, while framework for
this is implemented on MicroPython's module side, in case of axTLS, it
won't work reliably.

2. Implementation of .setblocking() method. It must be called on SSL socket
for blocking vs non-blocking operation to be handled correctly (for
example, it's not enough to wrap non-blocking socket with wrap_socket()
call - resulting SSL socket won't be itself non-blocking).  Note that
.setblocking() propagates call to the underlying socket object, as
expected.
2019-04-30 17:26:37 +10:00
Paul Sokolovsky
9c7c082396 extmod/modussl_mbedtls: Support non-blocking handshake.
For this, add wrap_socket(do_handshake=False) param. CPython doesn't have
such a param at a module's global function, and at SSLContext.wrap_socket()
it has do_handshake_on_connect param, but that uselessly long.

Beyond that, make write() handle not just MBEDTLS_ERR_SSL_WANT_WRITE, but
also MBEDTLS_ERR_SSL_WANT_READ, as during handshake, write call may be
actually preempted by need to read next handshake message from peer.
Likewise, for read(). And even after the initial negotiation, situations
like that may happen e.g. with renegotiation. Both
MBEDTLS_ERR_SSL_WANT_READ and MBEDTLS_ERR_SSL_WANT_WRITE are however mapped
to the same None return code. The idea is that if the same read()/write()
method is called repeatedly, the progress will be made step by step anyway.
The caveat is if user wants to add the underlying socket to uselect.poll().
To be reliable, in this case, the socket should be polled for both POLL_IN
and POLL_OUT, as we don't know the actual expected direction. But that's
actually problematic. Consider for example that write() ends with
MBEDTLS_ERR_SSL_WANT_READ, but gets converted to None. We put the
underlying socket on pull using POLL_IN|POLL_OUT but that probably returns
immediately with POLL_OUT, as underlyings socket is writable. We call the
same ussl write() again, which again results in MBEDTLS_ERR_SSL_WANT_READ,
etc. We thus go into busy-loop.

So, the handling in this patch is temporary and needs fixing. But exact way
to fix it is not clear. One way is to provide explicit function for
handshake (CPython has do_handshake()), and let *that* return distinct
codes like WANT_READ/WANT_WRITE. But as mentioned above, past the initial
handshake, such situation may happen again with at least renegotiation. So
apparently, the only robust solution is to return "out of bound" special
sentinels like WANT_READ/WANT_WRITE from read()/write() directly. CPython
throws exceptions for these, but those are expensive to adopt that way for
efficiency-conscious implementation like MicroPython.
2019-04-30 17:24:46 +10:00
Damien George
775ffdcc3b extmod/machine_signal: Fix fault when no args are passed to Signal(). 2019-04-26 14:47:31 +10:00
Léa Saviot
a6e5846ba7 extmod/modurandom: Add init method to seed the Yasmarang generator.
In CPython the random module is seeded differently on each import, and so
this new macro option MICROPY_PY_URANDOM_SEED_INIT_FUNC allows to implement
such a behaviour.
2019-04-16 14:54:36 +10:00
Damien George
d5f0c87bb9 extmod/modlwip: Abort TCP conns that didn't close cleanly in a while. 2019-04-11 11:18:10 +10:00
Damien George
3dda964785 extmod/modlwip: Use correct listening socket object in accept callback.
Since commit da938a83b5 the tcp_arg() that is
set for the new connection is the new connection itself, and the parent
listening socket is found in the pcb->connected entry.
2019-04-03 16:43:44 +11:00
Damien George
2848a613ac extmod/modlwip: Free any stored incoming bufs/connections on TCP error. 2019-04-01 13:36:44 +11:00
Damien George
490e0f39d1 extmod/modlwip: Protect socket.accept with lwIP concurrency lock.
This is needed now that the accept queue can have pending connections
removed asynchronously.
2019-04-01 13:36:43 +11:00
Damien George
2ec7838967 extmod/modlwip: Handle case of accept callback called with null PCB. 2019-04-01 13:36:43 +11:00
Damien George
da938a83b5 extmod/modlwip: Handle case of connection closing while on accept queue.
In such a case the connection is aborted by lwIP and so must be removed
from the pending accept queue.
2019-04-01 13:36:43 +11:00
Andrew Leech
9d6f70f715 stm32: Make default USB_VCP stream go through uos.dupterm for main REPL.
Use uos.dupterm for REPL configuration of the main USB_VCP(0) stream on
dupterm slot 1, if USB is enabled.  This means dupterm can also be used to
disable the boot REPL port if desired, via uos.dupterm(None, 1).

For efficiency this adds a simple hook to the global uos.dupterm code to
work with streams that are known to be native streams.
2019-04-01 13:04:05 +11:00
Andrew Leech
74d07469f2 extmod/vfs_fat: Fallback to FAT32 if standard FAT16/SFD format fails.
This allows formatting SD cards, larger flash etc which do not support the
default FAT16/SFD format mode.
2019-03-26 17:15:23 +11:00
Wolf Vollprecht
921b999225 extmod/moduselect: Adjust select_select and poll_register to use size_t. 2019-03-13 23:18:59 +11:00
Damien George
68a5d6fe77 extmod/modlwip: Fix case where concurrency lock isn't released on error. 2019-03-12 22:35:52 +11:00
Damien George
e959f21986 extmod/vfs_fat: Update for new oofatfs version. 2019-03-05 15:56:39 +11:00
Damien George
9b2a97a903 extmod/modwebrepl: Fix logic to handle a put of file of size 0.
Fixes issue #4499.
2019-02-28 15:30:48 +11:00
Damien George
ed1a88e263 extmod/modlwip: Don't require a port to define concurrency macros. 2019-02-27 10:27:56 +11:00
Damien George
39ea132e1d extmod/modlwip: Add concurrency protection macros.
Some users of this module may require the LwIP stack to run at an elevated
priority, to protect against concurrency issues with processing done by the
underlying network interface.  Since LwIP doesn't provide such protection
it must be done here (the other option is to run LwIP in a separate thread,
and use thread protection mechanisms, but that is a more heavyweight
solution).
2019-02-26 23:32:19 +11:00
Damien George
42c0e440b9 extmod/modlwip: Fix bug when polling listening socket with backlog=1.
The bug polling for readability was: if alloc==0 and tcp.item==NULL then
the code would incorrectly check tcp.array[iget] which is an invalid
dereference when alloc==0.  This patch refactors the code to use a helper
function lwip_socket_incoming_array() to return the correct pointer for the
incomming connection array.

Fixes issue #4511.
2019-02-18 14:23:35 +11:00
Damien George
7ef9482b8a extmod/modlwip: Change #ifdef to #if for check of MICROPY_PY_LWIP.
Otherwise this code will be included if MICROPY_PY_LWIP is defined to 0.
2019-02-15 16:07:24 +11:00
Yonatan Goldschmidt
bc4f8b438b extmod/moduwebsocket: Refactor websocket to uwebsocket.
As mentioned in #4450, `websocket` was experimental with a single intended
user, `webrepl`. Therefore, we'll make this change without a weak
link `websocket` -> `uwebsocket`.
2019-02-14 00:35:45 +11:00
Damien George
f03601779e extmod: Convert legacy uppercase macro names to lowercase. 2019-02-12 14:54:51 +11:00
Yonatan Goldschmidt
f024b2610f extmod/moduhashlib: Include implementation of sha256 only when required.
Previously crypto-algorithms impl was included even if MICROPY_SSL_MBEDTLS
was in effect, thus we relied on the compiler/linker to cut out the unused
functions.
2019-02-07 23:27:58 +11:00
Paul Sokolovsky
2f5d113fad py/warning: Support categories for warnings.
Python defines warnings as belonging to categories, where category is a
warning type (descending from exception type). This is useful, as e.g.
allows to disable warnings selectively and provide user-defined warning
types.  So, implement this in MicroPython, except that categories are
represented just with strings.  However, enough hooks are left to implement
categories differently per-port (e.g. as types), without need to patch each
and every usage.
2019-01-31 16:48:30 +11:00
Damien George
2a7a307baa extmod/modlwip: Add support for polling UDP sockets for writability. 2019-01-31 11:22:03 +11:00
Paul Sokolovsky
c7ed17bc4b extmod/modussl_mbedtls: Remove deprecated mbedtls/net.h header include.
This header is deprecated as of mbedtls 2.8.0, as shipped with Ubuntu
18.04.  Leads to #warning which is promoted to error with uPy compile
options.

Note that the current version of mbedtls is 2.14 at the time of writing.
2019-01-27 12:26:09 +11:00
Paul Sokolovsky
35687a87ec extmod/moduzlib: Update for uzlib 2.9.2. 2019-01-27 10:59:49 +11:00
Paul Sokolovsky
b1cca8fbe0 extmod/uzlib: Update uzlib to v2.9.2.
Major changes include robust parsing of erroneous compressed streams and
updated API.
2019-01-27 10:59:30 +11:00
Damien George
7cd59c5bc3 py/mpconfig: Move MICROPY_VERSION macros to static ones in mpconfig.h.
It's more robust to have the version defined statically in a header file,
rather than dynamically generating it via git using a git tag.  In case
git doesn't exist, or a different source control tool is used, it's
important to still have the uPy version number available.
2018-12-22 01:40:38 +11:00
Paul Sokolovsky
38151f35c1 extmod/moductypes: Add aliases for native C types.
SHORT, INT, LONG, LONGLONG, and unsigned (U*) variants are being defined.
This is done at compile using GCC-style predefined macros like
__SIZEOF_INT__.  If the compiler doesn't have such defines, no such types
will be defined.
2018-12-10 14:40:43 +11:00
Paul Sokolovsky
9d864bde04 extmod/moductypes: Implement __int__ for PTR.
Allows to get address a pointer contains, as an integer.
2018-12-10 14:25:05 +11:00
Damien George
29da9f0670 extmod/modlwip: Fix read-polling of listening socket with a backlog.
The recent implementation of the listen backlog meant that the logic to
test for readability of such a socket changed, and this commit updates the
logic to work again.
2018-12-03 18:02:10 +11:00
Damien George
4737ff8054 extmod/modlwip: Implement TCP listen/accept backlog.
Array to hold waiting connections is in-place if backlog=1, else is a
dynamically allocated array.  Incoming connections are processed FIFO
style to maintain fairness.
2018-12-01 17:23:44 +11:00
Paul Sokolovsky
2411f42ccb extmod/moductypes: Make sizeof() accept "layout" parameter.
sizeof() can work in two ways: a) calculate size of already instantiated
structure ("sizeof variable") - in this case we already no layout; b) size
of structure decsription ("sizeof type"). In the latter case, LAYOUT_NATIVE
was assumed, but there should possibility to calculate size for other
layouts too. So, with this patch, there're now 2 forms:

uctypes.sizeof(struct)
uctypes.sizeof(struct_desc, layout)
2018-10-23 11:32:02 +11:00
Damien George
4904663748 extmod/modonewire: Fix reset timings to match 1-wire specs.
Fixes issue #4116.
2018-10-17 15:52:07 +11:00
Paul Sokolovsky
9fbd12f2fa extmod/moductypes: Accept OrderedDict as a structure description.
Using OrderedDict (i.e. stable order of fields) would for example allow to
automatically calculate field offsets in structures.
2018-10-13 16:08:12 +11:00
Paul Sokolovsky
18f45d2e23 extmod/moductypes: Remove BITFIELD from aggregate types enum.
This value is unused. It was an artifact of early draft design, but
bitfields were optimized to use scalar one-word encoding, to allow
compact encoding of typical multiple bitfields in MCU control
registers.
2018-10-05 17:02:15 +10:00
Damien George
05959c6465 extmod/moduhashlib: Add md5 implementation using mbedtls. 2018-09-12 16:08:53 +10:00
Damien George
87d45f4d49 extmod/moduhashlib: Use newer message digest API for mbedtls >=2.7.0.
Since mbedtls 2.7.0 new digest functions were introduced with a "_ret"
suffix to allow the functions to return an error message (eg, if the
underlying hardware acceleration failed).  These new functions must be used
instead of the old ones to prevent deprecation warnings, or link errors for
missing functions, depending on the mbedtls configuration.
2018-09-12 16:04:18 +10:00
Paul Sokolovsky
5fe3730a30 extmod/moduhashlib: Add md5 implementation, using axTLS.
MD5 is still widely used, and may be important in some cases for networking
interoperability, e.g. HTTP Digest authentication.
2018-09-11 14:51:52 +10:00
Damien George
0be2ea50e9 py/py.mk: Build axtls library directly from its source files.
This removes the need for a separate axtls build stage, and builds all
axtls object files along with other code.  This simplifies and cleans up
the build process, automatically builds axtls when needed, and puts the
axtls object files in the correct $(BUILD) location.

The MicroPython axtls configuration file is provided in
extmod/axtls-include/config.h
2018-09-08 00:07:23 +10:00
Damien George
206c65f22c extmod/modussl_axtls: Use MP_ROM_PTR for objects in allowed args array. 2018-08-14 21:47:07 +10:00
Damien George
b8b2525576 extmod/modbtree: Update to work with new mp_stream_posix_XXX signatures. 2018-08-14 17:41:23 +10:00
Damien George
7a67f057d7 extmod/modussl: Support polling in ussl objects by passing through ioctl
The underlying socket can handling polling, and any other transparent ioctl
requests.  Note that CPython handles the case of polling an ssl object by
polling the file descriptor of the underlying socket file, and that
behaviour is emulated here.
2018-07-20 13:05:04 +10:00
Damien George
4a2051eec7 extmod/modlwip: Deregister all lwIP callbacks when closing a socket.
Otherwise they may be called on a socket that no longer exists.

For example, if the GC calls the finaliser on the socket and then reuses
its heap memory, the "callback" entry of the old socket may contain invalid
data.  If lwIP then calls the TCP callback the code may try to call the
user callback object which is now invalid.  The lwIP callbacks must be
deregistered during the closing of the socket, before all the pcb pointers
are set to NULL.
2018-07-20 12:59:24 +10:00
Damien George
d974ee1c2f extmod/vfs_posix: Use DTTOIF if available to convert type in ilistdir. 2018-07-11 16:07:44 +10:00
Damien George
c700ff52a0 extmod/vfs_posix: Support ilistdir with no (or empty) argument. 2018-07-10 12:51:09 +10:00
Damien George
aa735dc6a4 extmod: Fix to support compiling with object representation D. 2018-07-08 23:15:44 +10:00
Damien George
a3c3dbd955 extmod/vfs: Support opening a file descriptor (int) with VfsPosix.
Fixes issue #3865.
2018-07-03 13:04:29 +10:00
Damien George
e30a5fc7bc extmod/modure: Add ure.sub() function and method, and tests.
This feature is controlled at compile time by MICROPY_PY_URE_SUB, disabled
by default.

Thanks to @dmazzella for the original patch for this feature; see #3770.
2018-07-02 14:55:02 +10:00
Damien George
1e9b871d29 extmod/modure: Add match.span(), start() and end() methods, and tests.
This feature is controlled at compile time by
MICROPY_PY_URE_MATCH_SPAN_START_END, disabled by default.

Thanks to @dmazzella for the original patch for this feature; see #3770.
2018-07-02 14:54:56 +10:00
Damien George
1f86460910 extmod/modure: Add match.groups() method, and tests.
This feature is controlled at compile time by MICROPY_PY_URE_MATCH_GROUPS,
disabled by default.

Thanks to @dmazzella for the original patch for this feature; see #3770.
2018-07-02 14:53:30 +10:00
Damien George
8769a3e38c extmod/moducryptolib: Don't include arpa/inet.h, it's not needed.
And some ports (eg esp8266) don't have it.
2018-06-27 16:44:59 +10:00
Yonatan Goldschmidt
31f2f1e967 extmod/moducryptolib: Shorten exception messages to reduce code size. 2018-06-27 16:44:59 +10:00
Yonatan Goldschmidt
d0507c084c extmod/moducryptolib: Prefix all Python methods/objects with ucryptolib.
Follows what was done in b045ebd35 for uhashlib.
2018-06-27 16:44:59 +10:00
Yonatan Goldschmidt
eacb233b8f extmod/moducryptolib: Add an mbedTLS implementation for this module. 2018-06-27 16:44:59 +10:00
Yonatan Goldschmidt
e328b4593c extmod/moducryptolib: Refactor functions for clean interface with axTLS.
This will allow implementations other than axTLS.

This commit includes additions of checks and clarifications of exceptions
related to user input.

To make the interface cleaner, I've disallowed switching from encrypt to
decrypt in the same object, as this is not always possible with other
crypto libraries (not all libraries have AES_convert_key like axTLS).
2018-06-27 16:44:59 +10:00
Yonatan Goldschmidt
473fe45da2 extmod/moducryptolib: Optionally export MODE_* constants to Python.
Allow including crypto consts based on compilation settings.  Disabled by
default to reduce code size; if one wants extra code readability, can
enable them.
2018-06-27 16:29:26 +10:00
Paul Sokolovsky
567bc2d6ce extmod/moducryptolib: Add ucryptolib module with crypto functions.
The API follows guidelines of https://www.python.org/dev/peps/pep-0272/,
but is optimized for code size, with the idea that full PEP 0272
compatibility can be added with a simple Python wrapper mode.

The naming of the module follows (u)hashlib pattern.

At the bare minimum, this module is expected to provide:

* AES128, ECB (i.e. "null") mode, encrypt only

Implementation in this commit is based on axTLS routines, and implements
following:

* AES 128 and 256
* ECB and CBC modes
* encrypt and decrypt
2018-06-27 14:54:40 +10:00
Damien George
e8398a5856 extmod: Update to use new mp_get_stream helper.
With this patch objects are only checked that they have the stream protocol
at the start of their use as a stream, and afterwards the efficient
mp_get_stream() helper is used to extract the stream protocol C methods.
2018-06-18 12:35:56 +10:00
Damien George
564abb01a5 extmod/vfs_fat_diskio: Factor disk ioctl code to reduce code size.
Functionality is unchanged.
2018-06-16 18:21:42 +10:00
Damien George
d11fb09333 extmod/modussl_axtls: Fix __del__ to point to mp_stream_close_obj. 2018-06-13 13:16:21 +10:00
Damien George
035906419d extmod/uos_dupterm: Use native C stream methods on dupterm object.
This patch changes dupterm to call the native C stream methods on the
connected stream objects, instead of calling the Python readinto/write
methods.  This is much more efficient for native stream objects like UART
and webrepl and doesn't require allocating a special dupterm array.

This change is a minor breaking change from the user's perspective because
dupterm no longer accepts pure user stream objects to duplicate on.  But
with the recent addition of uio.IOBase it is possible to still create such
classes just by inheriting from uio.IOBase, for example:

    import uio, uos

    class MyStream(uio.IOBase):
        def write(self, buf):
            # existing write implementation
        def readinto(self, buf):
            # existing readinto implementation

    uos.dupterm(MyStream())
2018-06-12 15:06:11 +10:00
Yonatan Goldschmidt
c2fb725e72 extmod/moduhashlib: Make function objects STATIC.
These are not exported to anyone anyway.
2018-06-12 13:50:11 +10:00
Yonatan Goldschmidt
6963ee9075 extmod/moduhashlib: Allow using the sha256 implementation of mbedTLS. 2018-06-12 13:50:11 +10:00
Yonatan Goldschmidt
6630354ffe extmod/moduhashlib: Allow to disable the sha256 class.
Via the config value MICROPY_PY_UHASHLIB_SHA256.  Default to enabled to
keep backwards compatibility.

Also add default value for the sha1 class, to at least document its
existence.
2018-06-12 13:50:11 +10:00
Yonatan Goldschmidt
38682d4629 extmod/moduhashlib: Reorder funcs so that they are grouped by hash type.
Makes the code much more readable by reducing the number of #ifdefs and
keeping related functions close.
2018-06-12 13:47:20 +10:00
Yonatan Goldschmidt
b045ebd354 extmod/moduhashlib: Prefix all Python methods and objects with uhashlib.
For consistency with other modules, and to help avoid clashes with the
actual underlying functions that do the hashing (eg
crypto-algorithms/sha256.c:sha256_update).
2018-06-12 13:44:09 +10:00
Damien George
c117effddd extmod/vfs: Introduce a C-level VFS protocol, with fast import_stat.
Following other C-level protocols, this VFS protocol is added to help
abstract away implementation details of the underlying VFS in an efficient
way.  As a starting point, the import_stat function is put into this
protocol so that the VFS sub-system does not need to know about every VFS
implementation in order to do an efficient stat for importing files.

In the future it might be worth adding other functions to this protocol.
2018-06-06 14:33:42 +10:00
Damien George
d4ce57e4e3 extmod/vfs: Add fast path for stating VfsPosix filesystem. 2018-06-06 14:28:23 +10:00
Damien George
8d82b0edbd extmod: Add VfsPosix filesystem component.
This VFS component allows to mount a host POSIX filesystem within the uPy
VFS sub-system.  All traditional POSIX file access then goes through the
VFS, allowing to sandbox a uPy process to a certain sub-dir of the host
system, as well as mount other filesystem types alongside the host
filesystem.
2018-06-06 14:28:23 +10:00
Damien George
f35aae366c extmod/vfs_fat: Rename FileIO/TextIO types to mp_type_vfs_fat_XXX.
So they don't clash with other VFS implementations.
2018-06-06 14:28:23 +10:00
Damien George
172c23fe5d extmod/vfs: Use u_rom_obj properly in argument structures. 2018-06-06 14:28:23 +10:00
Damien George
ea22406f76 extmod/modussl_mbedtls: Use mbedtls_entropy_func for CTR-DRBG entropy.
If mbedtls_ctr_drbg_seed() is available in the mbedtls bulid then so should
be mbedtls_entropy_func().  Then it's up to the port to configure a valid
entropy source, eg via MBEDTLS_ENTROPY_HARDWARE_ALT.
2018-05-31 21:52:29 +10:00
Damien George
98b9f0fc9d extmod/modussl_mbedtls: Populate sock member right away in wrap_socket.
Otherwise the "sock" member may have an undefined value if wrap_socket
fails with an exception and exits early, and then if the finaliser runs it
will try to close an invalid stream object.

Fixes issue #3828.
2018-05-31 21:47:26 +10:00
Damien George
41766ba7e6 extmod/modlwip: Allow to compile with MICROPY_PY_LWIP disabled. 2018-05-21 16:46:30 +10:00
Damien George
1e2a6a84a2 extmod/modlwip: Set POLLHUP flag for sockets that are new.
This matches CPython behaviour on Linux: a socket that is new and not
listening or connected is considered "hung up".

Thanks to @rkojedzinszky for the initial patch, PR #3457.
2018-05-17 23:17:36 +10:00
Damien George
9c2044717c extmod/modlwip: Update to work with lwIP v2.0.
lwIP v2.0.3 has been tested with this lwip module and it works very well.
2018-05-17 12:58:34 +10:00
Damien George
318f874cda extmod/modlwip: In ioctl handle case when socket is in an error state.
Using MP_STREAM_POLL_HUP for ERR_RST state follows how *nix handles this
case.
2018-05-04 15:15:04 +10:00
Damien George
5936168150 extmod/uzlib: Fix C-language sequencing error with uzlib_get_byte calls.
The order of function calls in an arithmetic expression is undefined and so
they must be written out as sequential statements.

Thanks to @dv-extrarius for reporting this issue, see issue #3690.
2018-05-02 23:16:22 +10:00
Damien George
89b1c4a60c extmod/vfs: Delegate import_stat to vfs.stat to allow generic FS import. 2018-05-02 17:08:48 +10:00
Damien George
bdff68db9c extmod/modlwip: Check if getaddrinfo() constraints are supported or not.
In particular don't issue a warning if the passed-in constraints are
actually supported because they are the default values.
2018-04-23 16:38:20 +10:00
armink
6a693db71d extmod/re1.5: Fix compilecode.c compile problem on IAR tool chain.
The 2nd and 3rd args of the ternary operator are treated like they are in
the same expression and must have similar types.  void is not compatible
with int so that's why the compiler is complaining.
2018-04-10 13:54:22 +10:00
Damien George
cf31d384f1 py/stream: Switch stream close operation from method to ioctl.
This patch moves the implementation of stream closure from a dedicated
method to the ioctl of the stream protocol, for each type that implements
closing.  The benefits of this are:

1. Rounds out the stream ioctl function, which already includes flush,
   seek and poll (among other things).

2. Makes calling mp_stream_close() on an object slightly more efficient
   because it now no longer needs to lookup the close method and call it,
   rather it just delegates straight to the ioctl function (if it exists).

3. Reduces code size and allows future types that implement the stream
   protocol to be smaller because they don't need a dedicated close method.

Code size reduction is around 200 bytes smaller for x86 archs and around
30 bytes smaller for the bare-metal archs.
2018-04-10 13:41:32 +10:00
Tom Collins
4d3a92c67c extmod/vfs_fat: Add file size as 4th element of uos.ilistdir tuple. 2018-03-12 12:26:36 +11:00
Damien George
58ebeca6a9 drivers/bus: Pull out software SPI implementation to dedicated driver.
This patch takes the software SPI implementation from extmod/machine_spi.c
and moves it to a dedicated file in drivers/bus/softspi.c.  This allows the
SPI driver to be used independently of the uPy runtime, making it a more
general component.
2018-03-10 00:59:43 +11:00
Damien George
21d5527edf extmod/machine_spi: Make SPI protocol structure public.
So it can be referenced directly without the need for the uPy object.
2018-03-02 23:55:08 +11:00
Damien George
09be031e04 extmod/vfs_fat_diskio: Use a C-stack-allocated bytearray for block buf.
This patch eliminates heap allocation in the VFS FAT disk IO layer, when
calling the underlying readblocks/writeblocks methods.  The bytearray
object that is passed to these methods is now allocated on the C stack
rather than the heap (it's only 4 words big).

This means that these methods should not retain a pointer to the buffer
object that is passed in, but this was already a restriction because the
original heap-allocated bytearray had its buffer passed by reference.
2018-02-28 15:11:20 +11:00
Damien George
2ad555bc76 extmod/vfs_fat: Remove declaration of mp_builtin_open_obj.
It's declared already in py/builtin.h.
2018-02-23 17:41:47 +11:00
Damien George
eb570f47a2 extmod/vfs_fat: Make fat_vfs_open_obj wrapper public, not its function.
This patch just moves the definition of the wrapper object fat_vfs_open_obj
to the location of the definition of its function, which matches how it's
done in most other places in the code base.
2018-02-23 17:33:26 +11:00
Damien George
638b860066 extmod/vfs_fat: Merge remaining vfs_fat_misc.c code into vfs_fat.c.
The only function left in vfs_fat_misc.c is fat_vfs_import_stat() which
can logically go into vfs_fat.c, allowing to remove vfs_fat_misc.c.
2018-02-23 17:24:57 +11:00
Damien George
ae4a07730a extmod/vfs_fat: Move ilistdir implementation from misc to main file.
The fat_vfs_ilistdir2() function was only used by fat_vfs_ilistdir_func()
so moving the former into the same file as the latter allows it to be
placed directly into the latter function, thus saving code size.
2018-02-23 17:17:32 +11:00
Damien George
d9bca1f7bd extmod/modujson: Implement ujson.dump() function. 2018-02-15 11:35:42 +11:00
Ayke van Laethem
7642785881 extmod/vfs_fat_file: Implement SEEK_CUR for non-zero offset.
CPython doesn't allow SEEK_CUR with non-zero offset for files in text mode,
and uPy inherited this behaviour for both text and binary files.  It makes
sense to provide full support for SEEK_CUR of binary-mode files in uPy, and
to do this in a minimal way means also allowing to use SEEK_CUR with
non-zero offsets on text-mode files.  That seems to be a fair compromise.
2018-01-31 17:33:07 +11:00
Damien George
46b35356e1 extmod/modframebuf: Add 8-bit greyscale format (GS8). 2017-12-14 17:36:13 +11:00
Petr Viktorin
34247465c3 extmod/modframebuf: Add 2-bit color format (GS2_HMSB).
This format is used in 2-color LED matrices and in e-ink displays like
SSD1606.
2017-12-14 17:13:02 +11:00
Damien George
1b223a42bf extmod/modure: Add cast to workaround bug in MSVC. 2017-12-13 22:22:57 +11:00
Damien George
7df2ebbfea extmod/modussl_mbedtls: Clean up mbedtls state when error during setup.
Without this patch, if the SSL handshake fails (eg the connection was lost)
then the mbedtls state (memory) will never be freed.
2017-12-13 14:48:53 +11:00
Damien George
ab750ee2fb extmod/modure: Convert alloca() to use new scoped allocation API. 2017-12-11 13:49:09 +11:00
Damien George
6df7b2f2fe extmod/machine_signal: Change VLA to use new scoped allocation API. 2017-12-11 13:49:09 +11:00
Damien George
48f6990fbc extmod/modlwip: Commit TCP out data to lower layers if buffer gets full.
Dramatically improves TCP sending throughput because without an explicit
call to tcp_output() the data is only sent to the lower layers via the
lwIP slow timer which (by default) ticks every 500ms.
2017-11-24 15:52:32 +11:00
Damien George
e511f24ddd extmod/modussl_axtls: Implement key and cert kw args to wrap_socket.
The key and cert must both be a str/bytes object in DER format.
2017-11-24 15:50:40 +11:00
Damien George
5e34a113ea py/runtime: Add MP_BINARY_OP_CONTAINS as reverse of MP_BINARY_OP_IN.
Before this patch MP_BINARY_OP_IN had two meanings: coming from bytecode it
meant that the args needed to be swapped, but coming from within the
runtime meant that the args were already in the correct order.  This lead
to some confusion in the code and comments stating how args were reversed.
It also lead to 2 bugs: 1) containment for a subclass of a native type
didn't work; 2) the expression "{True} in True" would illegally succeed and
return True.  In both of these cases it was because the args to
MP_BINARY_OP_IN ended up being reversed twice.

To fix these things this patch introduces MP_BINARY_OP_CONTAINS which
corresponds exactly to the __contains__ special method, and this is the
operator that built-in types should implement.  MP_BINARY_OP_IN is now only
emitted by the compiler and is converted to MP_BINARY_OP_CONTAINS by
swapping the arguments.
2017-11-24 14:48:23 +11:00
Damien George
12ad64bc55 extmod/vfs_fat: Mount FatFS on creation so VFS methods can be used.
It's possible to use the methods (eg ilistdir) of a VFS FatFS object
without it being mounted in the VFS itself.  This previously worked but
only because FatFS was "mounting" the filesystem automatically when any
function (eg f_opendir) was called.  But it didn't work for ports that used
synchronisation objects (_FS_REENTRANT) because they are only initialised
via a call to f_mount.  So, call f_mount explicitly when creating a new
FatFS object so that everything is set up correctly.  Then also provide a
finaliser to do the f_umount call, but only if synchronisation objects are
enabled (since otherwise the f_umount call does nothing).
2017-11-20 11:46:40 +11:00
Damien George
4601759bf5 py/objstr: Remove "make_qstr_if_not_already" arg from mp_obj_new_str.
This patch simplifies the str creation API to favour the common case of
creating a str object that is not forced to be interned.  To force
interning of a new str the new mp_obj_new_str_via_qstr function is added,
and should only be used if warranted.

Apart from simplifying the mp_obj_new_str function (and making it have the
same signature as mp_obj_new_bytes), this patch also reduces code size by a
bit (-16 bytes for bare-arm and roughly -40 bytes on the bare-metal archs).
2017-11-16 13:17:51 +11:00
Damien George
6bc55b657b extmod/vfs: Use existing qstr for forward-slash string object. 2017-11-16 13:13:24 +11:00
Christopher Cooper
7413b3ce3e extmod/moduhashlib: Enable SHA1 hashing when using "mbedtls" library.
The SHA1 hashing functionality is provided via the "axtls" library's
implementation, and hence is unavailable when the "axtls" library is not being
used.  This change provides the same SHA1 hashing functionality when using the
"mbedtls" library by using its implementation instead.
2017-11-12 21:46:23 +02:00
Paul Sokolovsky
1cf6d488b3 extmod/modussl_axtls: Typo fix in comment. 2017-11-02 00:16:03 +02:00
Paul Sokolovsky
0719c936fb extmod/modussl_axtls: socket_read: Handle EAGAIN.
If SSL_EAGAIN is returned (which is a feature of MicroPython's axTLS fork),
return EAGAIN.

Original axTLS returns SSL_OK both when there's no data to return to user
yet and when the underlying stream returns EAGAIN. That's not distinctive
enough, for example, original module code works well for blocking stream,
but will infinite-loop for non-blocking socket with EAGAIN. But if we fix
non-blocking case, blocking calls to .read() will return few None's initially
(while axTLS progresses thru handshake).

Using SSL_EAGAIN allows to fix non-blocking case without regressing the
blocking one.

Note that this only handles case of non-blocking reads of application data.
Initial handshake and writes still don't support non-blocking mode and must
be done in the blocking way.
2017-11-02 00:14:11 +02:00
Damien George
10b76a9620 extmod/modussl_mbedtls: Allow to compile with unix coverage build.
Fixes a few C warnings.  No functional changes.
2017-10-30 15:41:37 +11:00
Eric Poulsen
74ec52d857 extmod/modussl: Add finaliser support for ussl objects.
Per the comment found here
https://github.com/micropython/micropython-esp32/issues/209#issuecomment-339855157,
this patch adds finaliser code to prevent memory leaks from ussl objects,
which is especially useful when memory for a ussl context is allocated
outside the uPy heap.  This patch is in-line with the finaliser code found
in many modsocket implementations for various ports.

This feature is configured via MICROPY_PY_USSL_FINALISER and is disabled by
default because there may be issues using it when the ussl state *is*
allocated on the uPy heap, rather than externally.
2017-10-30 15:25:32 +11:00
Damien George
c64eb4f8ce extmod/vfs: Replace VLA in proxy func with small, static sized array.
VLAs can be expensive on stack usage due to stack alignment requirements,
and also the fact that extra local variables are needed to track the
dynamic size of the stack.  So using fixed-size arrays when possible can
help to reduce code size and stack usage.

In this particular case, the maximum value of n_args in the VLA is 2 and so
it's more efficient to just allocate this array with a fixed size.  This
reduces code size by around 30 bytes on Thumb2 and Xtensa archs.  It also
reduces total stack usage of the function: on Thumb2 the usage with VLA is
between 40 and 48 bytes, which is reduced to 32; on Xtensa, VLA usage is
between 64 and 80 bytes, reduced to 32; on x86-64 it's at least 88 bytes
reduced to 80.
2017-10-27 18:01:25 +11:00
Damien George
f4059dcc0c all: Use NULL instead of "" when calling mp_raise exception helpers.
This is the established way of doing it and reduces code size by a little
bit.
2017-10-24 22:39:36 +11:00
Damien George
9725a654bd extmod/uos_dupterm: Swallow any errors from dupterm closing the stream.
Without this the board will crash when deactivating a stream that doesn't
have a close() method (eg UART) or that raises an exception within the
method (eg user-defined function).
2017-10-19 14:10:17 +11:00
Damien George
37282f8fc1 extmod/uos_dupterm: Update uos.dupterm() and helper funcs to have index.
The uos.dupterm() signature and behaviour is updated to reflect the latest
enhancements in the docs.  It has minor backwards incompatibility in that
it no longer accepts zero arguments.

The dupterm_rx helper function is moved from esp8266 to extmod and
generalised to support multiple dupterm slots.

A port can specify multiple slots by defining the MICROPY_PY_OS_DUPTERM
config macro to an integer, being the number of slots it wants to have;
0 means to disable the dupterm feature altogether.

The unix and esp8266 ports are updated to work with the new interface and
are otherwise unchanged with respect to functionality.
2017-10-13 20:01:57 +11:00
Damien George
a3dc1b1957 all: Remove inclusion of internal py header files.
Header files that are considered internal to the py core and should not
normally be included directly are:
    py/nlr.h - internal nlr configuration and declarations
    py/bc0.h - contains bytecode macro definitions
    py/runtime0.h - contains basic runtime enums

Instead, the top-level header files to include are one of:
    py/obj.h - includes runtime0.h and defines everything to use the
        mp_obj_t type
    py/runtime.h - includes mpstate.h and hence nlr.h, obj.h, runtime0.h,
        and defines everything to use the general runtime support functions

Additional, specific headers (eg py/objlist.h) can be included if needed.
2017-10-04 12:37:50 +11:00
Paul Sokolovsky
aba1f9167a extmod/modure: Add stack overflow checking when executing a regex. 2017-10-03 00:20:10 +03:00
Paul Sokolovsky
c9a0b2a818 extmod/re1.5: Upgrade to v0.8.2, adds hook for stack overflow checking. 2017-10-02 21:20:47 +03:00
Paul Sokolovsky
4a314a6f63 extmod/re1.5: Update to 0.8.1.
Allow literal minus in char classes to be in trailing position, e.g. [a-c-].
(Previously, minus was allowed only at the start.)

This increases ARM Thumb2 code size by 8 bytes.
2017-09-24 10:19:24 +03:00
Damien George
b02be234e1 extmod/machine_pinbase: Put PinBase singleton in ROM.
This patch also removes the empty type "pinbase_type" (which crashes if
accessed) and uses "machine_pinbase_type" instead as the type of the
PinBase singleton.
2017-09-12 16:00:21 +10:00
Damien George
beeb7483d8 extmod/modussl_mbedtls: Allow to compile with MBEDTLS_DEBUG_C disabled.
With MBEDTLS_DEBUG_C disabled the function mbedtls_debug_set_threshold()
doesn't exist.  There's also no need to call mbedtls_ssl_conf_dbg() so a
few bytes can be saved on disabling that and not needing the mbedtls_debug
callback.
2017-09-06 17:34:45 +10:00
Damien George
4a93801c12 all: Update Makefiles and others to build with new ports/ dir layout.
Also renames "stmhal" to "stm32" in documentation and everywhere else.
2017-09-06 14:09:13 +10:00
Damien George
09547f0f51 extmod/modubinascii: Only include uzlib/tinf.h when it's really needed. 2017-08-31 14:10:49 +10:00
Damien George
71c9cfb028 all: Convert remaining "mp_uint_t n_args" to "size_t n_args".
This is to have consistency across the whole repository.
2017-08-30 10:59:58 +10:00
Damien George
58321dd985 all: Convert mp_uint_t to mp_unary_op_t/mp_binary_op_t where appropriate
The unary-op/binary-op enums are already defined, and there are no
arithmetic tricks used with these types, so it makes sense to use the
correct enum type for arguments that take these values.  It also reduces
code size quite a bit for nan-boxing builds.
2017-08-29 13:16:30 +10:00
Damien George
1c6b442d32 extmod/modubinascii: Don't post-increment variable that won't be used. 2017-08-21 22:05:39 +10:00
Damien George
4c736ea8fc extmod,unix: For uos.stat interpret st_size member as an unsigned int.
This prevents large files (eg larger than 2gb on a 32-bit arch) from
showing up as having a negative size.  Fixes issue #3227.
2017-08-21 20:47:22 +10:00
Alex Robbins
c89254fd0f extmod/modubinascii: Rewrite mod_binascii_a2b_base64.
This implementation ignores invalid characters in the input. This allows
it to decode the output of b2a_base64, and also mimics the behavior of
CPython.
2017-08-17 09:25:51 +03:00
Eric Poulsen
d5191edf7f extmod/modussl_mbedtls.c: Add ussl.getpeercert() method.
Behaviour is as per CPython but only the binary form is implemented here.
A test is included.
2017-08-16 15:01:00 +10:00
Javier Candeira
35a1fea90b all: Raise exceptions via mp_raise_XXX
- Changed: ValueError, TypeError, NotImplementedError
  - OSError invocations unchanged, because the corresponding utility
    function takes ints, not strings like the long form invocation.
  - OverflowError, IndexError and RuntimeError etc. not changed for now
    until we decide whether to add new utility functions.
2017-08-13 22:52:33 +10:00
Damien George
0f12082f5b py,extmod,stmhal: Use "static inline" for funcs that should be inline.
"STATIC inline" can expand to "inline" if STATIC is defined to nothing, and
this case can lead to link errors.
2017-08-02 13:42:34 +10:00
Paul Sokolovsky
ee04525097 extmod/modlwip: Implement setsockopt(IP_ADD_MEMBERSHIP).
Allows to join multicast groups.
2017-07-31 22:38:37 +03:00
Alexander Steffen
55f33240f3 all: Use the name MicroPython consistently in comments
There were several different spellings of MicroPython present in comments,
when there should be only one.
2017-07-31 18:35:40 +10:00
Damien George
bbced3b4bb extmod: Use MP_ROM_INT for int values in an mp_rom_map_elem_t. 2017-07-31 13:00:34 +10:00
Paul Sokolovsky
f2140f9446 extmod/mod{lwip,onewire,webrepl}: Convert to mp_rom_map_elem_t. 2017-07-29 18:24:16 +03:00
Paul Sokolovsky
036b58228c extmod/modframebuf: Use correct initialization for .locals_dict. 2017-07-29 10:26:41 +03:00
Damien George
653a0c2d71 extmod/machine_signal: Fix parsing of invert arg when Pin is first arg. 2017-07-26 12:51:46 +10:00
Eric Poulsen
6b4d4a25ce extmod/modussl_mbedtls: Implement non-blocking SSL sockets. 2017-07-26 11:34:33 +10:00
Damien George
f3687109d5 extmod/modframebuf: Consistently use "col" as name for colour variables.
Thanks to @kamikaze, aka Oleg Korsak, for the original idea and patch.
2017-07-25 14:06:44 +10:00
Damien George
0893b273b9 extmod/modussl_mbedtls: Make socket.close() free all TLS resources.
Also, use mp_stream_close() helper to close the underlying socket.
2017-07-25 14:00:45 +10:00
Radomir Dopieralski
363087aa11 extmod/modframebuf: Fix invalid stride for odd widths in GS4_HMSB fmt.
Since the stride is specified in pixels, in a 4-bit horizontal format it
has to always be even, otherwise the computation is wrong and we can
write outside of the buffer sometimes.
2017-07-25 12:29:02 +10:00
Damien George
a10467b58a extmod/modussl_mbedtls: When reading and peer wants to close, return 0.
If this particular code is returned then there's no more data, it's not
really an error.
2017-07-25 11:53:26 +10:00
Damien George
aa7be82a4d all: Don't include system errno.h when it's not needed. 2017-07-24 18:43:14 +10:00
Damien George
513dfcf4fe extmod/modussl_mbedtls: Support server_side mode.
To use server_side mode one must pass valid values in the "key" and "cert"
parameters.
2017-07-24 15:08:59 +10:00