Commit Graph

620 Commits

Author SHA1 Message Date
Damien George 49388e339e extmod/extmod.mk: Include mdns app source in lwIP build. 2019-06-05 16:13:34 +10:00
Damien George ef7357d4ab extmod/modussl_mbedtls: Allow to build with object representation D. 2019-06-05 15:33:15 +10:00
Damien George 678ec182cd extmod/extmod.mk: Integrate mbedTLS so it is built from source.
Setting MICROPY_PY_USSL and MICROPY_SSL_MBEDTLS at the Makefile-level will
now build mbedTLS from source and include it in the build, with the ussl
module using this TLS library.  Extra settings like MBEDTLS_CONFIG_FILE may
need to be provided by a given port.

If a port wants to use its own mbedTLS library then it should not set
MICROPY_SSL_MBEDTLS at the Makefile-level but rather set it at the C level,
and provide the library as part of the build in its own way (see eg esp32
port).
2019-06-05 15:22:21 +10:00
Damien George cd6b115815 extmod: Factor out makefile rules from py.mk to new extmod.mk file.
To logically separate extmod related rules out, and prevent py.mk from
growing too large.
2019-06-05 14:23:12 +10:00
Damien George cf1c131f73 extmod: Add network-level class binding to cyw43 driver. 2019-06-03 16:47:35 +10:00
Damien George 019dd84af1 extmod/modlwip: Register TCP close-timeout callback before closing PCB.
In d5f0c87bb9 this call to tcp_poll() was
added to put a timeout on closing TCP sockets.  But after calling
tcp_close() the PCB may be freed and therefore invalid, so tcp_poll() can
not be used at that point.  As a fix this commit calls tcp_poll() before
closing the TCP PCB.  If the PCB is subsequently closed and freed by
tcp_close() or tcp_abort() then the PCB will not be on any active list and
the callback will not be executed, which is the desired behaviour (the
_lwip_tcp_close_poll() callback only needs to be called if the PCB remains
active for longer than the timeout).
2019-05-29 01:29:48 +10:00
Damien George 734ada3e29 extmod/modlwip: Free any incoming bufs/connections before closing PCB.
Commit 2848a613ac introduced a bug where
lwip_socket_free_incoming() accessed pcb.tcp->state after the PCB was
closed.  The state may have changed due to that close call, or the PCB may
be freed and therefore invalid.  This commit fixes that by calling
lwip_socket_free_incoming() before the PCB is closed.
2019-05-29 01:24:43 +10:00
Damien George b10d0664be extmod/machine_i2c: Add i2c.writevto() that can write a vector of bufs.
For example: i2c.writevto(addr, (buf1, buf2)).  This allows to efficiently
(wrt memory) write data composed of separate buffers, such as a command
followed by a large amount of data.
2019-05-20 15:04:29 +10:00
Damien George 8bcb552d97 extmod/machine_i2c: Remove need for temporary memory in writemem() call. 2019-05-20 15:04:29 +10:00
Damien George 606ea2b10f extmod/machine_i2c: Change C-level API to allow split I2C transactions.
API is:

    int transfer(
        mp_obj_base_t *obj,
        uint16_t addr,
        size_t n,
        mp_machine_i2c_buf_t *bufs,
        unsigned int flags
    )
2019-05-20 15:04:29 +10:00
Paul Sokolovsky 016d9a40fe various: Add and update my copyright line based on git history.
For modules I initially created or made substantial contributions to.
2019-05-17 18:04:15 +10:00
Damien George 7c5cf59f8b extmod/modujson: Handle parsing of floats with + in the exponent.
Fixes issue #4780.
2019-05-14 14:45:54 +10:00
Yonatan Goldschmidt 32ba679924 extmod/moducryptolib: Add AES-CTR support for axTLS builds. 2019-05-06 18:20:56 +10:00
Yonatan Goldschmidt ef9843653b extmod/moducryptolib: Add AES-CTR support.
Selectable at compile time via MICROPY_PY_UCRYPTOLIB_CTR.  Disabled by
default.
2019-05-06 18:09:48 +10:00
Paul Sokolovsky c76445315f extmod/modussl_axtls: Add non-blocking mode support.
It consists of:

1. "do_handhake" param (default True) to wrap_socket(). If it's False,
handshake won't be performed by wrap_socket(), as it would be done in
blocking way normally. Instead, SSL socket can be set to non-blocking mode,
and handshake would be performed before the first read/write request (by
just returning EAGAIN to these requests, while instead reading/writing/
processing handshake over the connection). Unfortunately, axTLS doesn't
really support non-blocking handshake correctly. So, while framework for
this is implemented on MicroPython's module side, in case of axTLS, it
won't work reliably.

2. Implementation of .setblocking() method. It must be called on SSL socket
for blocking vs non-blocking operation to be handled correctly (for
example, it's not enough to wrap non-blocking socket with wrap_socket()
call - resulting SSL socket won't be itself non-blocking).  Note that
.setblocking() propagates call to the underlying socket object, as
expected.
2019-04-30 17:26:37 +10:00
Paul Sokolovsky 9c7c082396 extmod/modussl_mbedtls: Support non-blocking handshake.
For this, add wrap_socket(do_handshake=False) param. CPython doesn't have
such a param at a module's global function, and at SSLContext.wrap_socket()
it has do_handshake_on_connect param, but that uselessly long.

Beyond that, make write() handle not just MBEDTLS_ERR_SSL_WANT_WRITE, but
also MBEDTLS_ERR_SSL_WANT_READ, as during handshake, write call may be
actually preempted by need to read next handshake message from peer.
Likewise, for read(). And even after the initial negotiation, situations
like that may happen e.g. with renegotiation. Both
MBEDTLS_ERR_SSL_WANT_READ and MBEDTLS_ERR_SSL_WANT_WRITE are however mapped
to the same None return code. The idea is that if the same read()/write()
method is called repeatedly, the progress will be made step by step anyway.
The caveat is if user wants to add the underlying socket to uselect.poll().
To be reliable, in this case, the socket should be polled for both POLL_IN
and POLL_OUT, as we don't know the actual expected direction. But that's
actually problematic. Consider for example that write() ends with
MBEDTLS_ERR_SSL_WANT_READ, but gets converted to None. We put the
underlying socket on pull using POLL_IN|POLL_OUT but that probably returns
immediately with POLL_OUT, as underlyings socket is writable. We call the
same ussl write() again, which again results in MBEDTLS_ERR_SSL_WANT_READ,
etc. We thus go into busy-loop.

So, the handling in this patch is temporary and needs fixing. But exact way
to fix it is not clear. One way is to provide explicit function for
handshake (CPython has do_handshake()), and let *that* return distinct
codes like WANT_READ/WANT_WRITE. But as mentioned above, past the initial
handshake, such situation may happen again with at least renegotiation. So
apparently, the only robust solution is to return "out of bound" special
sentinels like WANT_READ/WANT_WRITE from read()/write() directly. CPython
throws exceptions for these, but those are expensive to adopt that way for
efficiency-conscious implementation like MicroPython.
2019-04-30 17:24:46 +10:00
Damien George 775ffdcc3b extmod/machine_signal: Fix fault when no args are passed to Signal(). 2019-04-26 14:47:31 +10:00
Léa Saviot a6e5846ba7 extmod/modurandom: Add init method to seed the Yasmarang generator.
In CPython the random module is seeded differently on each import, and so
this new macro option MICROPY_PY_URANDOM_SEED_INIT_FUNC allows to implement
such a behaviour.
2019-04-16 14:54:36 +10:00
Damien George d5f0c87bb9 extmod/modlwip: Abort TCP conns that didn't close cleanly in a while. 2019-04-11 11:18:10 +10:00
Damien George 3dda964785 extmod/modlwip: Use correct listening socket object in accept callback.
Since commit da938a83b5 the tcp_arg() that is
set for the new connection is the new connection itself, and the parent
listening socket is found in the pcb->connected entry.
2019-04-03 16:43:44 +11:00
Damien George 2848a613ac extmod/modlwip: Free any stored incoming bufs/connections on TCP error. 2019-04-01 13:36:44 +11:00
Damien George 490e0f39d1 extmod/modlwip: Protect socket.accept with lwIP concurrency lock.
This is needed now that the accept queue can have pending connections
removed asynchronously.
2019-04-01 13:36:43 +11:00
Damien George 2ec7838967 extmod/modlwip: Handle case of accept callback called with null PCB. 2019-04-01 13:36:43 +11:00
Damien George da938a83b5 extmod/modlwip: Handle case of connection closing while on accept queue.
In such a case the connection is aborted by lwIP and so must be removed
from the pending accept queue.
2019-04-01 13:36:43 +11:00
Andrew Leech 9d6f70f715 stm32: Make default USB_VCP stream go through uos.dupterm for main REPL.
Use uos.dupterm for REPL configuration of the main USB_VCP(0) stream on
dupterm slot 1, if USB is enabled.  This means dupterm can also be used to
disable the boot REPL port if desired, via uos.dupterm(None, 1).

For efficiency this adds a simple hook to the global uos.dupterm code to
work with streams that are known to be native streams.
2019-04-01 13:04:05 +11:00
Andrew Leech 74d07469f2 extmod/vfs_fat: Fallback to FAT32 if standard FAT16/SFD format fails.
This allows formatting SD cards, larger flash etc which do not support the
default FAT16/SFD format mode.
2019-03-26 17:15:23 +11:00
Wolf Vollprecht 921b999225 extmod/moduselect: Adjust select_select and poll_register to use size_t. 2019-03-13 23:18:59 +11:00
Damien George 68a5d6fe77 extmod/modlwip: Fix case where concurrency lock isn't released on error. 2019-03-12 22:35:52 +11:00
Damien George e959f21986 extmod/vfs_fat: Update for new oofatfs version. 2019-03-05 15:56:39 +11:00
Damien George 9b2a97a903 extmod/modwebrepl: Fix logic to handle a put of file of size 0.
Fixes issue #4499.
2019-02-28 15:30:48 +11:00
Damien George ed1a88e263 extmod/modlwip: Don't require a port to define concurrency macros. 2019-02-27 10:27:56 +11:00
Damien George 39ea132e1d extmod/modlwip: Add concurrency protection macros.
Some users of this module may require the LwIP stack to run at an elevated
priority, to protect against concurrency issues with processing done by the
underlying network interface.  Since LwIP doesn't provide such protection
it must be done here (the other option is to run LwIP in a separate thread,
and use thread protection mechanisms, but that is a more heavyweight
solution).
2019-02-26 23:32:19 +11:00
Damien George 42c0e440b9 extmod/modlwip: Fix bug when polling listening socket with backlog=1.
The bug polling for readability was: if alloc==0 and tcp.item==NULL then
the code would incorrectly check tcp.array[iget] which is an invalid
dereference when alloc==0.  This patch refactors the code to use a helper
function lwip_socket_incoming_array() to return the correct pointer for the
incomming connection array.

Fixes issue #4511.
2019-02-18 14:23:35 +11:00
Damien George 7ef9482b8a extmod/modlwip: Change #ifdef to #if for check of MICROPY_PY_LWIP.
Otherwise this code will be included if MICROPY_PY_LWIP is defined to 0.
2019-02-15 16:07:24 +11:00
Yonatan Goldschmidt bc4f8b438b extmod/moduwebsocket: Refactor `websocket` to `uwebsocket`.
As mentioned in #4450, `websocket` was experimental with a single intended
user, `webrepl`. Therefore, we'll make this change without a weak
link `websocket` -> `uwebsocket`.
2019-02-14 00:35:45 +11:00
Damien George f03601779e extmod: Convert legacy uppercase macro names to lowercase. 2019-02-12 14:54:51 +11:00
Yonatan Goldschmidt f024b2610f extmod/moduhashlib: Include implementation of sha256 only when required.
Previously crypto-algorithms impl was included even if MICROPY_SSL_MBEDTLS
was in effect, thus we relied on the compiler/linker to cut out the unused
functions.
2019-02-07 23:27:58 +11:00
Paul Sokolovsky 2f5d113fad py/warning: Support categories for warnings.
Python defines warnings as belonging to categories, where category is a
warning type (descending from exception type). This is useful, as e.g.
allows to disable warnings selectively and provide user-defined warning
types.  So, implement this in MicroPython, except that categories are
represented just with strings.  However, enough hooks are left to implement
categories differently per-port (e.g. as types), without need to patch each
and every usage.
2019-01-31 16:48:30 +11:00
Damien George 2a7a307baa extmod/modlwip: Add support for polling UDP sockets for writability. 2019-01-31 11:22:03 +11:00
Paul Sokolovsky c7ed17bc4b extmod/modussl_mbedtls: Remove deprecated mbedtls/net.h header include.
This header is deprecated as of mbedtls 2.8.0, as shipped with Ubuntu
18.04.  Leads to #warning which is promoted to error with uPy compile
options.

Note that the current version of mbedtls is 2.14 at the time of writing.
2019-01-27 12:26:09 +11:00
Paul Sokolovsky 35687a87ec extmod/moduzlib: Update for uzlib 2.9.2. 2019-01-27 10:59:49 +11:00
Paul Sokolovsky b1cca8fbe0 extmod/uzlib: Update uzlib to v2.9.2.
Major changes include robust parsing of erroneous compressed streams and
updated API.
2019-01-27 10:59:30 +11:00
Damien George 7cd59c5bc3 py/mpconfig: Move MICROPY_VERSION macros to static ones in mpconfig.h.
It's more robust to have the version defined statically in a header file,
rather than dynamically generating it via git using a git tag.  In case
git doesn't exist, or a different source control tool is used, it's
important to still have the uPy version number available.
2018-12-22 01:40:38 +11:00
Paul Sokolovsky 38151f35c1 extmod/moductypes: Add aliases for native C types.
SHORT, INT, LONG, LONGLONG, and unsigned (U*) variants are being defined.
This is done at compile using GCC-style predefined macros like
__SIZEOF_INT__.  If the compiler doesn't have such defines, no such types
will be defined.
2018-12-10 14:40:43 +11:00
Paul Sokolovsky 9d864bde04 extmod/moductypes: Implement __int__ for PTR.
Allows to get address a pointer contains, as an integer.
2018-12-10 14:25:05 +11:00
Damien George 29da9f0670 extmod/modlwip: Fix read-polling of listening socket with a backlog.
The recent implementation of the listen backlog meant that the logic to
test for readability of such a socket changed, and this commit updates the
logic to work again.
2018-12-03 18:02:10 +11:00
Damien George 4737ff8054 extmod/modlwip: Implement TCP listen/accept backlog.
Array to hold waiting connections is in-place if backlog=1, else is a
dynamically allocated array.  Incoming connections are processed FIFO
style to maintain fairness.
2018-12-01 17:23:44 +11:00
Paul Sokolovsky 2411f42ccb extmod/moductypes: Make sizeof() accept "layout" parameter.
sizeof() can work in two ways: a) calculate size of already instantiated
structure ("sizeof variable") - in this case we already no layout; b) size
of structure decsription ("sizeof type"). In the latter case, LAYOUT_NATIVE
was assumed, but there should possibility to calculate size for other
layouts too. So, with this patch, there're now 2 forms:

uctypes.sizeof(struct)
uctypes.sizeof(struct_desc, layout)
2018-10-23 11:32:02 +11:00
Damien George 4904663748 extmod/modonewire: Fix reset timings to match 1-wire specs.
Fixes issue #4116.
2018-10-17 15:52:07 +11:00
Paul Sokolovsky 9fbd12f2fa extmod/moductypes: Accept OrderedDict as a structure description.
Using OrderedDict (i.e. stable order of fields) would for example allow to
automatically calculate field offsets in structures.
2018-10-13 16:08:12 +11:00