Commit Graph

235 Commits

Author SHA1 Message Date
Damien George
5fb276de33 tests/extmod: Make uasyncio_fair test more reliable by adjusting sleeps.
With sleep(0.2) a multiple of sleep(0.1), the order of task 2 and 3
execution is not well defined, and depends on the precision of the system
clock and how fast the rest of the code runs.  So change 0.2 to 0.18 to
make the test more reliable.

Also fix a typo of t3/t4, and cancel t4 at the end.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-26 17:05:52 +10:00
Damien George
2acc087880 extmod/vfs_lfs: Add mtime support to littlefs files.
This commit adds support for modification time of files on littlefs v2
filesystems, using file attributes.  For some background see issue #6114.

Features/properties of this implementation:
- Only supported on littlefs2 (not littlefs1).
- Uses littlefs2's general file attributes to store the timestamp.
- The timestamp is 64-bits and stores nanoseconds since 1970/1/1 (if the
  range to the year 2554 is not enough then additional bits can be added to
  this timestamp by adding another file attribute).
- mtime is enabled by default but can be disabled in the constructor, eg:
  uos.mount(uos.VfsLfs2(bdev, mtime=False), '/flash')
- It's fully backwards compatible, existing littlefs2 filesystems will work
  without reformatting and timestamps will be added transparently to
  existing files (once they are opened for writing).
- Files without timestamps will open correctly, and stat will just return 0
  for their timestamp.
- mtime can be disabled or enabled each mount time and timestamps will only
  be updated if mtime is enabled (otherwise they will be untouched).

Signed-off-by: Damien George <damien@micropython.org>
2020-08-25 17:35:19 +10:00
Damien George
55c76eaac1 extmod/uasyncio: Truncate negative sleeps to 0.
Otherwise a task that continuously awaits on a large negative sleep can
monopolise the scheduler (because its wake time is always less than
everything else in the pairing heap).

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 12:17:06 +10:00
Damien George
20948a3d54 tests/extmod: Add test for uasyncio.sleep of a negative time.
It should take 0 time to await on a negative sleep.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 12:17:06 +10:00
Damien George
60f5b941e0 extmod/vfs_reader: Fix mp_reader_new_file to open file in "rb" mode.
mp_reader_new_file() is used to read in files for importing, either .py or
.mpy files, for the lexer and persistent code loader respectively.  In both
cases the file should be opened in raw bytes mode: the lexer handles
unicode characters itself, and .mpy files contain 8-bit bytes by nature.

Before this commit importing was working correctly because, although the
file was opened in text mode, all native filesystem implementations (POSIX,
FAT, LFS) would access the file in raw bytes mode via mp_stream_rw()
calling mp_stream_p_t.read().  So it was only an issue for non-native
filesystems, such as those implemented in Python.  For Python-based
filesystem implementations, a call to mp_stream_rw() would go via IOBase
and then to readinto() at the Python level, and readinto() is only defined
on files opened in raw bytes mode.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-12 23:40:50 +10:00
Thorsten von Eicken
9aa214077e extmod/modussl: Improve exception error messages.
This commit adds human readable error messages when mbedtls or axtls raise
an exception.  Currently often just an EIO error is raised so the user is
lost and can't tell whether it's a cert error, buffer overrun, connecting
to a non-ssl port, etc.  The axtls and mbedtls error raising in the ussl
module is modified to raise:

    OSError(-err_num, "error string")

For axtls a small error table of strings is added and used for the second
argument of the OSErrer.  For mbedtls the code uses mbedtls' built-in
strerror function, and if there is an out of memory condition it just
produces OSError(-err_num).  Producing the error string for mbedtls is
conditional on them being included in the mbedtls build, via
MBEDTLS_ERROR_C.
2020-07-20 23:41:45 +10:00
Damien George
a4c96fb3b0 extmod/uasyncio: Add asyncio.wait_for_ms function.
Fixes issue #6107.
2020-06-10 22:29:44 +10:00
stijn
51fd6c9777 extmod/ure: Use single function for match/search/sub.
Saves about 500 bytes on unix x64 and enables CPython-conform
usage of passing a re object to these functions.
2020-06-08 09:16:09 +02:00
Damien George
22806ed5df extmod/vfs: Retain previous working directory if chdir fails.
Fixes issue #6069.
2020-05-29 23:05:01 +10:00
Damien George
7dffbfd22a extmod/vfs_lfsx: Fix import_stat so it takes into account current dir.
CPython semantics require searching the current directory if the import is
not absolute (when "" is in sys.path).

Fixes issue #6037.
2020-05-15 11:31:32 +10:00
robert
0f83ef395c extmod/vfs_lfsx: Fix rename to respect cur dir for new path.
If the new name start with '/', cur_dir is not prepened any more, so that
the current working directory is respected.  And extend the test cases for
rename to cover this functionality.
2020-05-08 21:54:04 +10:00
robert
d3ea28d04a extmod/vfs_lfsx: Normalize path name in chdir.
This change scans for '.', '..' and multiple '/' and normalizes the new
path name.  If the resulting path does not exist, an error is raised.
Non-existing interim path elements are ignored if they are removed during
normalization.
2020-05-08 21:52:15 +10:00
robert
a5ea4b9f3f extmod/vfs_lfsx: Fix path handling in uos.stat() to consider cur dir.
This fixes the bug, that stat(filename) would not consider the current
working directory.  So if e.g. the cwd is "lib", then stat("main.py") would
return the info for "/main.py" instead of "/lib/main.py".
2020-05-08 21:37:51 +10:00
Damien George
73c58150f5 extmod/modbtree: Retain reference to underlying stream so it's not GC'd.
For ports that have a system malloc which is not garbage collected (eg
unix, esp32), the stream object for the DB must be retained separately to
prevent it from being reclaimed by the MicroPython GC (because the
berkeley-db library uses malloc to allocate the DB structure which stores
the only reference to the stream).

Although in some cases the user code will explicitly retain a reference to
the underlying stream because it needs to call close() on it, this is not
always the case, eg in cases where the DB is intended to live forever.

Fixes issue #5940.
2020-05-02 16:08:04 +10:00
Damien George
4fa6d939d6 tests/extmod: Add btree test for errors raised by btree DB library.
This test now passes given the previous two commits.
2020-04-27 23:59:09 +10:00
Damien George
5f0661b4fe extmod/uasyncio: Change cannot to can't in error message, and test exp.
Follow up to 8e048d2548 which missed these.
2020-04-14 21:51:25 +10:00
Damien George
db137e70dc extmod/uasyncio: Add Loop.new_event_loop method.
This commit adds Loop.new_event_loop() which is used to reset the singleton
event loop.  This functionality is put here instead of in Loop.close() to
make it possible to write code that is compatible with CPython.
2020-04-13 22:16:52 +10:00
Damien George
a5f2ae10fe tests/extmod: Update littlefs test output to match new library version.
The amount of free space on the VfsLfs2 filesystem after creating a large
file is reduced by 2 blocks in this test.
2020-04-09 15:59:28 +10:00
Damien George
df156b18e5 docs,tests: Add docs and test for uasyncio custom exc handler methods. 2020-04-04 10:47:48 +11:00
Damien George
b389bc0afa extmod/uasyncio: Implement Loop.stop() to stop the event loop. 2020-04-02 00:14:18 +11:00
David Lechner
3dc324d3f1 tests: Format all Python code with black, except tests in basics subdir.
This adds the Python files in the tests/ directory to be formatted with
./tools/codeformat.py.  The basics/ subdirectory is excluded for now so we
aren't changing too much at once.

In a few places `# fmt: off`/`# fmt: on` was used where the code had
special formatting for readability or where the test was actually testing
the specific formatting.
2020-03-30 13:21:58 +11:00
Damien George
c4935f3049 tests/extmod: Add uasyncio tests.
All .exp files are included because they require CPython 3.8 which may not
always be available.
2020-03-26 01:25:45 +11:00
Damien George
8f0778b209 extmod/modlwip: Properly handle non-blocking and timeout on UDP recv.
Fixes UDP non-blocking recv so it returns EAGAIN instead of ETIMEDOUT.
Timeout waiting for incoming data is also improved by replacing 100ms delay
with poll_sockets(), as is done in other parts of this module.

Fixes issue #5759.
2020-03-18 10:51:32 +11:00
Damien George
00267aae0b extmod/modlwip: Fix polling of UDP socket so it doesn't return HUP.
STATE_NEW will return HUP when polled so put active UDP sockets into a new
state which is different to STATE_NEW.

Fixes issue #5758.
2020-03-18 10:49:27 +11:00
Jim Mussared
f8449dd092 extmod/modframebuf: Allow blit source to be a subclass of FrameBuffer. 2020-02-21 13:32:48 +11:00
Yonatan Goldschmidt
0852acfc74 tests/run-tests: Auto-skip extmod/ticks_diff, extmod/time_ms_us tests. 2020-02-11 10:56:49 +11:00
Damien George
cfddc6a8c7 tests/extmod: Add basic machine.Timer test. 2020-01-22 17:31:18 +11:00
Damien George
99a04b8060 tests/extmod: Split out VfsFat finaliser tests to separate test file.
It tests independent functionality and may need to be skipped for a given
port.
2019-12-27 12:30:51 +11:00
Damien George
7280bf40d9 tests/extmod/vfs_lfs_error: Use small ints in seek error test.
So accessing the seek offset (at the C level) doesn't cause an
OverflowError on 32-bit targets.
2019-12-13 15:59:08 +11:00
Andrew Leech
1e87f11d3f py/objdict: Support ujson.dump() of OrderedDict objects.
Following CPython, OrderedDict are dumped with the syntax of dict.
2019-11-13 13:51:18 +11:00
Damien George
43f53a2bbd tests/extmod: Add test for ussl when passing in key/cert params. 2019-10-31 16:38:20 +11:00
Damien George
feaa251674 extmod/modlwip: Make socket poll return POLLNVAL in case of bad file. 2019-10-31 12:54:37 +11:00
Damien George
cfe1c5abf8 extmod/vfs: Rename BP_IOCTL_xxx constants to MP_BLOCKDEV_IOCTL_xxx.
Also rename SEC_COUNT to BLOCK_COUNT and SEC_SIZE to BLOCK_SIZE.
2019-10-29 14:17:29 +11:00
Damien George
7c8fb27f38 tests/extmod: Add test for blockdev with standard and extended protocol. 2019-10-29 14:17:29 +11:00
Damien George
73fddb84e5 tests/extmod: Add littlefs tests. 2019-10-29 14:17:29 +11:00
Jim Mussared
ebf8332104 extmod/re1.5: Support escaping within RE classes.
Fixes issues #3178 and #5220.

Tests are added, including all the cases mentioned in both bugs.
2019-10-18 12:20:32 +11:00
Damien George
2eb88f5df7 tests/extmod: Split json.loads of bytes/bytearray into separate test.
Because this functionality was introduced in Python 3.6.
2019-08-22 15:45:13 +10:00
Damien George
2dfa69efbb extmod/modujson: Support passing bytes/bytearray to json.loads.
CPython allows this, and it can be useful to reduce the number of memory
allocations.

Fixes issue #5031.
2019-08-22 15:32:26 +10:00
Damien George
7d851a27f1 extmod/modure: Make regex dump-code debugging feature optional.
Enabled via MICROPY_PY_URE_DEBUG, disabled by default (but enabled on unix
coverage build).  This is a rarely used feature that costs a lot of code
(500-800 bytes flash).  Debugging of regular expressions can be done
offline with other tools.
2019-08-19 16:43:00 +10:00
Eric Poulsen
01054f2092 py/objdict: Quote non-string types when used as keys in JSON output.
JSON requires that keys of objects be strings.  CPython will therefore
automatically quote simple types (NoneType, bool, int, float) when they are
used directly as keys in JSON output.  To prevent subtle bugs and emit
compliant JSON, MicroPython should at least test for such keys so they
aren't silently let through.  Then doing the actual quoting is a similar
cost to raising an exception, so that's what is implemented by this patch.

Fixes issue #4790.
2019-07-30 16:34:27 +10:00
Damien George
7c5cf59f8b extmod/modujson: Handle parsing of floats with + in the exponent.
Fixes issue #4780.
2019-05-14 14:45:54 +10:00
Yonatan Goldschmidt
ef9843653b extmod/moducryptolib: Add AES-CTR support.
Selectable at compile time via MICROPY_PY_UCRYPTOLIB_CTR.  Disabled by
default.
2019-05-06 18:09:48 +10:00
Paul Sokolovsky
7b5400134b tests/ussl_basic: Disable setblocking() calls.
Now that setblocking() is implemented in modussl_axtls, it calls into the
underlying stream object, and io.BytesIO doesn't have setblocking().
2019-04-30 17:27:28 +10:00
Paul Sokolovsky
c76445315f extmod/modussl_axtls: Add non-blocking mode support.
It consists of:

1. "do_handhake" param (default True) to wrap_socket(). If it's False,
handshake won't be performed by wrap_socket(), as it would be done in
blocking way normally. Instead, SSL socket can be set to non-blocking mode,
and handshake would be performed before the first read/write request (by
just returning EAGAIN to these requests, while instead reading/writing/
processing handshake over the connection). Unfortunately, axTLS doesn't
really support non-blocking handshake correctly. So, while framework for
this is implemented on MicroPython's module side, in case of axTLS, it
won't work reliably.

2. Implementation of .setblocking() method. It must be called on SSL socket
for blocking vs non-blocking operation to be handled correctly (for
example, it's not enough to wrap non-blocking socket with wrap_socket()
call - resulting SSL socket won't be itself non-blocking).  Note that
.setblocking() propagates call to the underlying socket object, as
expected.
2019-04-30 17:26:37 +10:00
Damien George
ca39ea7cef tests: Skip tests needing machine module if (u)machine doesn't exist. 2019-04-28 22:12:17 +10:00
Damien George
968b688055 tests/extmod: Add test for FAT filesystem on a very large block device. 2019-03-27 10:22:38 +11:00
Yonatan Goldschmidt
bc4f8b438b extmod/moduwebsocket: Refactor websocket to uwebsocket.
As mentioned in #4450, `websocket` was experimental with a single intended
user, `webrepl`. Therefore, we'll make this change without a weak
link `websocket` -> `uwebsocket`.
2019-02-14 00:35:45 +11:00
Damien George
074597f172 tests/extmod/uctypes_error: Add test for unsupported unary op. 2018-12-10 14:29:41 +11:00
Paul Sokolovsky
0de6815ec1 tests/extmod/uctypes_ptr_le: Test int() operation on a pointer field. 2018-12-10 14:25:06 +11:00
stijn
06643a0df4 tests/extmod: Skip uselect test when CPython doesn't have poll().
CPython does not have an implementation of select.poll() on some
operating systems (Windows, OSX depending on version) so skip the
test in those cases instead of failing it.
2018-10-30 14:49:23 +11:00