Commit Graph

1965 Commits

Author SHA1 Message Date
Damien George 1d9528210b tests/multi_bluetooth: Add performance test for gatt char writes.
Signed-off-by: Damien George <damien@micropython.org>
2021-04-30 16:14:48 +10:00
Damien George 76dab3bf31 tests/run-multitests.py: Provide some convenient serial device shorcuts.
It's now possible to specify a device serial port using shorcuts like:

    $ ./run-multitests.py -i pyb:a0 -i pyb:u1 multi_bluetooth/*.py

Signed-off-by: Damien George <damien@micropython.org>
2021-04-30 15:47:22 +10:00
Damien George 3123f6918b tests: Use .errno instead of .args[0] for OSError exceptions.
Signed-off-by: Damien George <damien@micropython.org>
2021-04-23 22:03:46 +10:00
Damien George 3c4bfd1dec py/objexcept: Support errno attribute on OSError exceptions.
This commit adds the errno attribute to exceptions, so code can retrieve
errno codes from an OSError using exc.errno.

The implementation here simply lets `errno` (and the existing `value`)
attributes work on any exception instance (they both alias args[0]).  This
is for efficiency and to keep code size down.  The pros and cons of this
are:

Pros:
- more compatible with CPython, less difference to document and learn
- OSError().errno will correctly return None, whereas the current way of
  doing it via OSError().args[0] will raise an IndexError
- it reduces code size on most bare-metal ports (because they already have
  the errno qstr)
- for Python code that uses exc.errno the generated bytecode is 2 bytes
  smaller and more efficient to execute (compared with exc.args[0]); so
  bytecode loaded to RAM saves 2 bytes RAM for each use of this attribute,
  and bytecode that is frozen saves 2 bytes flash/ROM for each use
- it's easier/shorter to type, and saves 2 bytes of space in .py files that
  use it (for each use)

Cons:
- increases code size by 4-8 bytes on minimal ports that don't already have
  the `errno` qstr
- all exceptions now have .errno and .value attributes (a cpydiff test is
  added to address this)

See also #2407.

Signed-off-by: Damien George <damien@micropython.org>
2021-04-23 22:03:46 +10:00
Damien George 7d911d2069 tests/net_inet: Add 'Strict-Transport-Security' to exp file.
Because micropython.org now adds this to the headers.

Signed-off-by: Damien George <damien@micropython.org>
2021-04-18 23:20:26 +10:00
Damien George 8459f538eb tests/feature_check: Check for lack of pass result rather than failure.
Commit cb68a5741a broke automatic Python
feature detection when running tests, because some detection relied on a
crash of a feature script returning exactly b"CRASH".

This commit fixes this and improves the situation by testing for the lack
of a known pass result, rather than an exact failure result.

Signed-off-by: Damien George <damien@micropython.org>
2021-04-15 00:52:56 +10:00
stijn a66286f3a0 unix: Improve command line argument processing.
Per CPython everything which comes after the command, module or file
argument is not an option for the interpreter itself.  Hence the processing
of options should stop when encountering those, and the remainder be passed
as sys.argv.  Note the latter was already the case for a module or file but
not for a command.

This fixes issues like 'micropython myfile.py -h' showing the help and
exiting instead of passing '-h' as sys.argv[1], likewise for
'-X <something>' being treated as a special option no matter where it
occurs on the command line.
2021-04-07 12:41:25 +10:00
Jeff Epler 172fb5230a extmod/re1.5: Check and report byte overflow errors in _compilecode.
The generated regex code is limited in the range of jumps and counts, and
this commit checks all cases which can overflow given the right kind of
input regex, and returns an error in such a case.

This change assumes that the results that overflow an int8_t do not
overflow a platform int.

Closes: #7078

Signed-off-by: Jeff Epler <jepler@gmail.com>
2021-04-06 13:36:42 +10:00
Damien George cb68a5741a tests/run-tests.py: Provide more info if script run via pyboard crashes.
Signed-off-by: Damien George <damien@micropython.org>
2021-03-16 14:49:57 +11:00
Damien George a79d97cb76 tests/extmod/vfs_fat_fileio2.py: Close test file at end of test.
Otherwise it can lead to inconsistent results running subsequent tests.

Signed-off-by: Damien George <damien@micropython.org>
2021-03-16 14:49:57 +11:00
Damien George e98ff3f08e tests/multi_bluetooth: Skip tests when BLE features are unsupported.
Signed-off-by: Damien George <damien@micropython.org>
2021-03-12 20:08:20 +11:00
Damien George 2a38d71036 tests/run-tests.py: Reformat with Black.
Signed-off-by: Damien George <damien@micropython.org>
2021-03-12 19:56:09 +11:00
Damien George 6129b8e401 tests: Rename run-tests to run-tests.py for consistency.
Signed-off-by: Damien George <damien@micropython.org>
2021-03-12 19:56:09 +11:00
Jim Mussared 1342debb9b tests/multi_bluetooth: Add basic performance tests.
1. Exchange GATT notifications.
2. Transmit a stream of data over L2CAP.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-02-19 17:53:43 +11:00
Thorsten von Eicken c10d431819 esp32: Add basic support for Non-Volatile-Storage in esp32 module.
This commit implements basic NVS support for the esp32.  It follows the
pattern of the esp32.Partition class and exposes an NVS object per NVS
namespace.  The initial support provided is only for signed 32-bit integers
and binary blobs.  It's easy (albeit a bit tedious) to add support for
more types.

See discussions in: #4436, #4707, #6780
2021-02-19 15:05:19 +11:00
Thorsten von Eicken 2c1299b007 extmod/modussl: Fix ussl read/recv/send/write errors when non-blocking.
Also fix related problems with socket on esp32, improve docs for
wrap_socket, and add more tests.
2021-02-17 11:50:54 +11:00
Jim Mussared 83d23059ef tests/extmod: Add test for ThreadSafeFlag.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-02-16 17:08:36 +11:00
Thorsten von Eicken 902da05a18 esp32: Set MICROPY_USE_INTERNAL_ERRNO=0 to use toolchain's errno.h.
The underlying OS (the ESP-IDF) uses it's own internal errno codes and so
it's simpler and cleaner to use those rather than trying to convert
everything to the values defined in py/mperrno.h.
2021-02-15 23:47:02 +11:00
Jim Mussared 7ed99544e4 extmod/uasyncio: Add asyncio.current_task().
Matches CPython behavior.

Fixes #6686
2021-02-13 15:11:17 +11:00
Damien George df85e48813 tests/extmod/vfs_posix.py: Add more tests for VfsPosix class.
Signed-off-by: Damien George <damien@micropython.org>
2021-02-11 23:49:44 +11:00
Damien George 26b4ef4c46 extmod/vfs_posix_file: Allow closing an already closed file.
Signed-off-by: Damien George <damien@micropython.org>
2021-02-11 22:54:41 +11:00
Damien George 0a59938574 py/mpz: Fix overflow of borrow in mpn_div.
For certain operands to mpn_div, the existing code path for
`DIG_SIZE == MPZ_DBL_DIG_SIZE / 2` had a bug in it where borrow could still
overflow in the `(x >= *n || *n - x <= borrow)` branch, ie
`borrow + x - (mpz_dbl_dig_t)*n` overflows the borrow variable.  In such
cases the subsequent right-shift of borrow would not bring in the overflow
bit, leading to an error in the result.  An example division that had
overflow when MPZ_DIG_SIZE = 16 is `(2 ** 48 - 1) ** 2 // (2 ** 48 - 1)`.

This is fixed in this commit by simplifying the code and handling the low
digits of borrow first, and then the upper bits (to shift down) separately.
There is no longer a distinction between `DIG_SIZE < MPZ_DBL_DIG_SIZE / 2`
and `DIG_SIZE == MPZ_DBL_DIG_SIZE / 2`.

This commit also simplifies the second part of the calculation so that
borrow does not need to be negated (instead the code just works knowing
that borrow is negative and using + instead of - in calculations involving
borrow).

Fixes #6777.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-08 11:50:05 +11:00
stijn 0397448501 tests/run-tests: Change default Python command used on Windows.
Default to just calling python since that is most commonly available: the
official installer or zipfiles from python.org, anaconda, nupkg all result
in python being available but not python3.  In other words: the default
used so far is wrong.  Note that os.name is 'posix' when running the python
version which comes with Cygwin or MSys2 so they are not affected by this.
However of all possible ways to get Python on Windows, only Cygwin provides
no python command so update the default way for running tests in the
README.
2021-02-02 21:32:20 +11:00
Damien George 35a6f6231e tests/extmod/utime_time_ns.py: Relax bounds on time_ns measurement.
Some devices have lower precision than 1ms for time_ns() (eg PYBv1.x has
3.9ms resolution of the RTC) so make the test more lenient for them.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-01 18:44:28 +11:00
Damien George 7a97e4351b tests: Move native for test from pybnative to micropython.
And make it generic so it can be run on any target.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 23:57:10 +11:00
Damien George 925bd67cfb py/objfun: Support fun.__globals__ attribute.
This returns a reference to the globals dict associated with the function,
ie the global scope that the function was defined in.  This attribute is
read-only but the dict itself is modifiable, per CPython behaviour.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 23:57:10 +11:00
Damien George 71ea438561 extmod/vfs: Check block 0 and 1 when auto-detecting littlefs.
The superblock for littlefs is in block 0 and 1, but block 0 may be erased
or partially written, so block 1 must be checked if block 0 does not have a
valid littlefs superblock in it.

Prior to this commit, the mount of a block device which auto-detected the
filysystem type would fail for littlefs if block 0 did not contain a valid
superblock.  That is now fixed.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 15:02:55 +11:00
Oliver Joos 419134bea4 tests/extmod: Add test for the precision of utime functions.
According to documentation time() has a precision of at least 1 second.
This test runs for 2.5 seconds and calls all utime functions every 100ms.
Then it checks if they returned enough different results.  All functions
with sub-second precision will return ~25 results.  This test passes with
15 results or more.  Functions that do not exist are skipped silently.
2021-01-23 16:54:57 +11:00
stijn 069557edef tests/misc/sys_settrace_features.py: Fix running with non-dflt encoding.
Notably git-cmd which comes with git installations on Windows alters the
encoding resulting in CPython tracing encodings/cp1252.py calls.
2020-12-18 13:57:17 +11:00
stijn 108183fcc0 tests/misc/sys_settrace: Make test output independent of invoked path.
The original logic of reducing a full path to a relative one assumes
"tests/misc" is in the filename which is limited in usage: it never works
for CPython on Windows since that will use a backslash as path separator,
and also won't work when the filename is a path not relative to the tests
directory which happens for example in the common case of running
"./run-tests -d misc".

Fix all cases by printing only the bare filename, which requires them all
to start with sys_settrace_ hence the renaming.
2020-12-18 13:56:45 +11:00
Oliver Joos dc1fd4df73 tests/extmod: Add test to try and mount a block device directly.
Mounting a bdev directly tries to auto-detect the filesystem and if none is
found an OSError(19,) should be raised.

The fourth parameter of readblocks() and writeblocks() must be optional to
support ports with MICROPY_VFS_FAT=1.  Otherwise mounting a bdev may fail
because looking for a FATFS will call readblocks() with only 3 parameters.
2020-12-17 22:43:19 +11:00
Damien George e0bb7a53c3 tests/misc/sys_settrace_features.py: Ignore CPython zipimport traces.
Signed-off-by: Damien George <damien@micropython.org>
2020-12-14 13:04:50 +11:00
Joris Peeraer 5020b14d54 py/mpprint: Fix length calculation for strings with precision-modifier.
Two issues are tackled:

1. The calculation of the correct length to print is fixed to treat the
   precision as a maximum length instead as the exact length.
   This is done for both qstr (%q) and for regular str (%s).

2. Fix the incorrect use of mp_printf("%.*s") to mp_print_strn().

   Because of the fix of above issue, some testcases that would print
   an embedded null-byte (^@ in test-output) would now fail.
   The bug here is that "%s" was used to print null-bytes. Instead,
   mp_print_strn is used to make sure all bytes are outputted and the
   exact length is respected.

Test-cases are added for both %s and %q with a combination of precision
and padding specifiers.
2020-12-07 23:32:06 +11:00
Damien George c8b0557178 tests/multi_bluetooth: Add multitests for BLE pairing and bonding.
Signed-off-by: Damien George <damien@micropython.org>
2020-12-02 14:44:55 +11:00
Damien George b505971069 extmod/uasyncio: Fix cancellation handling of wait_for.
This commit switches the roles of the helper task from a cancellation task
to a runner task, to get the correct semantics for cancellation of
wait_for.

Some uasyncio tests are now disabled for the native emitter due to issues
with native code generation of generators and yield-from.

Fixes #5797.

Signed-off-by: Damien George <damien@micropython.org>
2020-12-02 12:31:37 +11:00
Damien George 309dfe39e0 extmod/uasyncio: Add Task.done() method.
This is added because task.coro==None is no longer the way to detect if a
task is finished.  Providing a (CPython compatible) function for this
allows the implementation to be abstracted away.

Signed-off-by: Damien George <damien@micropython.org>
2020-12-02 12:07:06 +11:00
Damien George ca40eb0fda extmod/uasyncio: Delay calling Loop.call_exception_handler by 1 loop.
When a tasks raises an exception which is uncaught, and no other task
await's on that task, then an error message is printed (or a user function
called) via a call to Loop.call_exception_handler.  In CPython this call is
made when the Task object is freed (eg via reference counting) because it's
at that point that it is known that the exception that was raised will
never be handled.

MicroPython does not have reference counting and the current behaviour is
to deal with uncaught exceptions as early as possible, ie as soon as they
terminate the task.  But this can be undesirable because in certain cases
a task can start and raise an exception immediately (before any await is
executed in that task's coro) and before any other task gets a chance to
await on it to catch the exception.

This commit changes the behaviour so that tasks which end due to an
uncaught exception are scheduled one more time for execution, and if they
are not await'ed on by the next scheduling loop, then the exception handler
is called (eg the exception is printed out).

Signed-off-by: Damien George <damien@micropython.org>
2020-12-02 12:07:06 +11:00
Damien George 02b44a0154 tests/run-tests: Update skipped tests on CI for GitHub Actions.
Signed-off-by: Damien George <damien@micropython.org>
2020-11-30 10:48:41 +11:00
Damien George 6a3d70db96 tests/extmod: Add vfs_posix.py test for uos.VfsPosix class.
Signed-off-by: Damien George <damien@micropython.org>
2020-11-29 17:31:24 +11:00
Jim Mussared 23fad2526d tests/multi_bluetooth: Add L2CAP channels multi-test.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-24 01:07:17 +11:00
Jim Mussared efc0800132 tests/multi_bluetooth: Add a test for WB55 concurrent flash access.
This test currently passes on Unix/PYBD, but fails on WB55 because it lacks
synchronisation of the internal flash.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-16 17:04:02 +11:00
Jim Mussared 7e75245d54 tests/multi_bluetooth: Change dict index-and-del to pop, to clear event.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-16 17:03:27 +11:00
Jim Mussared c75ce37910 tests/run-multitests.py: Add a -p flag to run permutations of instances.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-13 17:19:05 +11:00
Jim Mussared ccfd535af4 tests/multi_bluetooth: Improve reliability of event waiting.
Use the same `wait_for_event` in all tests that doesn't hold a reference to
the event data tuple and handles repeat events.

Also fix a few misc reliability issues around timeouts and sequencing.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-13 17:19:05 +11:00
Jim Mussared 309fb822e6 tests/run-multitests.py: Fix diff order, show changes relative to truth.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-13 17:18:20 +11:00
Arrowana 922f81dfd1 extmod/machine_mem: Only allow integers in machine.memX subscript.
Prior to this change machine.mem32['foo'] (or using any other non-integer
subscript) could result in a fault due to 'foo' being interpreted as an
integer.  And when writing code it's hard to tell if the fault is due to a
bad subscript type, or an integer subscript that specifies an invalid
memory address.

The type of the object used in the subscript is now tested to be an
integer by using mp_obj_get_int_truncated instead of
mp_obj_int_get_truncated.  The performance hit of this change is minimal,
and machine.memX objects are more for convenience than performance (there
are many other ways to read/write memory in a faster way),

Fixes issue #6588.
2020-11-13 11:13:37 +11:00
Damien George d7e1526593 py/binary: Fix sign extension setting wide integer on 32-bit archs.
Signed-off-by: Damien George <damien@micropython.org>
2020-11-11 22:18:24 +11:00
Damien George bdfb584b29 extmod/moductypes: Fix storing to (U)INT64 arrays on 32-bit archs.
Fixes issue #6583.

Signed-off-by: Damien George <damien@micropython.org>
2020-11-11 22:18:24 +11:00
Damien George 1fef5662ab py/mpz: Do sign extension in mpz_as_bytes for negative values.
Signed-off-by: Damien George <damien@micropython.org>
2020-11-11 22:18:24 +11:00
Damien George ed7ddd4dd4 tests/micropython/extreme_exc.py: Unlink alloc'd lists earlier in chain.
To help the GC collect this memory that's no longer needed after the test.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-29 23:35:36 +11:00
stijn 25c4563f26 examples: Add example code for user C modules, both C and C++.
Add working example code to provide a starting point for users with files
that they can just copy, and include the modules in the coverage test to
verify the complete user C module build functionality.  The cexample module
uses the code originally found in cmodules.rst, which has been updated to
reflect this and partially rewritten with more complete information.
2020-10-29 15:30:42 +11:00
stijn fad4079778 esp32,unix: Support building C++ code.
Support building .cpp files and linking them into the micropython
executable in a way similar to how it is done for .c files.  The main
incentive here is to enable user C modules to use C++ files (which are put
in SRC_MOD_CXX by py.mk) since the core itself does not utilize C++.

However, to verify build functionality a unix overage test is added.  The
esp32 port already has CXXFLAGS so just add the user modules' flags to it.
For the unix port use a copy of the CFLAGS but strip the ones which are not
usable for C++.
2020-10-29 15:29:50 +11:00
Damien George 6f34800884 extmod/modurandom: Support urandom.seed() without an argument.
If a port provides MICROPY_PY_URANDOM_SEED_INIT_FUNC as a source of
randomness then this will be used when urandom.seed() is called without
an argument (or with None as the argument) to seed the pRNG.

Other related changes in this commit:
- mod_urandom___init__ is changed to call seed() without arguments, instead
  of explicitly passing in the result of MICROPY_PY_URANDOM_SEED_INIT_FUNC.
- mod_urandom___init__ will only ever seed the pRNG once (before it could
  seed it again if imported by, eg, random and then urandom).
- The Yasmarang state is moved to the BSS for builds where the state is
  guaranteed to be initialised on import of the (u)random module.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-29 14:15:16 +11:00
Damien George 03a1f94ea1 extmod/vfs_lfs: Support mounting LFS filesystems in read-only mode.
Signed-off-by: Damien George <damien@micropython.org>
2020-10-29 11:43:52 +11:00
Damien George 368c1a0961 tests/thread/stress_schedule.py: Assign globals before running test.
When threading is enabled without the GIL then there can be races between
the threads accessing the globals dict.  Avoid this issue by making sure
all globals variables are allocated before starting the threads.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-28 00:54:30 +11:00
Jim Mussared b137d064e9 py/objtype: Handle __dict__ attribute when type has no locals. 2020-10-10 00:16:32 +11:00
Damien George 817b80a102 unix/variants: Enable MICROPY_DEBUG_PARSE_RULE_NAME on coverage build.
Signed-off-by: Damien George <damien@micropython.org>
2020-10-01 15:26:43 +10:00
Damien George d4b61b0017 extmod/utime_mphal: Add generic utime.time_ns() function.
It requires mp_hal_time_ns() to be provided by a port.  This function
allows very accurate absolute timestamps.

Enabled on unix, windows, stm32, esp8266 and esp32.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-01 14:20:42 +10:00
Andrew Leech 319437d4bd extmod/modure: Allow \\ in re.sub replacements. 2020-09-30 23:18:34 +10:00
Damien George 9123b67d64 tests/run-tests: Use -BS flags when running CPython.
The use of -S ensures that only the CPython standard library is accessible,
which makes tests run the same regardless of any site-packages that are
installed.  It also improves start-up time of CPython, reducing the overall
time spent running the test suite.

tests/basics/containment.py is updated to work around issue with old Python
versions not being able to str-format a dict-keys object, which becomes
apparent when -S is used.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-25 12:27:23 +10:00
Damien George 81f2162ca0 extmod/modbluetooth: Change module-owned bytes objects to memoryview.
A read-only memoryview object is a better representation of the data, which
is owned by the ubluetooth module and may change between calls to the
user's irq callback function.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-25 12:23:11 +10:00
Damien George 71adf506ce extmod/vfs: Fix lookup of entry in root dir so it fails correctly.
Prior to this commit, uos.chdir('/') followed by uos.stat('noexist') would
succeed that stat even though the entry did not exist (some other functions
like listdir would have similar issues).  This is because, if the current
directory was the root and the path was relative, mp_vfs_lookup_path would
return success for bad paths.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-23 16:23:35 +10:00
Damien George 42342fa3cb tests/basics: Add test for MICROPY_PY_ALL_INPLACE_SPECIAL_METHODS ops.
And enable this feature on unix, the coverage variant.  The .exp test file
is needed so the test can run on CPython versions prior to "@=" operator
support.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-18 18:40:42 +10:00
Damien George c410a86814 tests/basics: Enable == and != special-method tests now that they work.
These work since 3aab54bf43

Signed-off-by: Damien George <damien@micropython.org>
2020-09-18 18:40:42 +10:00
Jim Mussared 3086d35e16 tests/multi_bluetooth/ble_mtu.py: Add multitest for BLE MTU. 2020-09-18 12:51:21 +10:00
Jim Mussared 06dda48144 tests/run-multitests.py: Show test/truth diff. 2020-09-18 12:51:21 +10:00
Jim Mussared fe642ced43 tests/multi_bluetooth: Update UUID format in .exp files. 2020-09-18 12:51:21 +10:00
stijn 2e54d9d146 py: Fix handling of NaN in certain pow implementations.
Adds a new compile-time option MICROPY_PY_MATH_POW_FIX_NAN for use with
toolchains that don't handle pow-of-NaN correctly.
2020-09-11 10:04:57 +10:00
Damien George 8d5a40c86e py/objfloat: Fix handling of negative float to power of nan.
Prior to this commit, pow(-2, float('nan')) would return (nan+nanj), or
raise an exception on targets that don't support complex numbers.  This is
fixed to return simply nan, as CPython does.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 10:03:57 +10:00
Jim Mussared 5eda362e0a tests/multi_bluetooth: Make ble_gap_connect robust against event timing. 2020-09-08 23:53:12 +10:00
Jim Mussared 26b66804e9 tests/multi_bluetooth: Update to new config('mac') behaviour. 2020-09-08 12:53:24 +10:00
stijn 40ad8f1666 all: Rename "sys" module to "usys".
This is consistent with the other 'micro' modules and allows implementing
additional features in Python via e.g. micropython-lib's sys.

Note this is a breaking change (not backwards compatible) for ports which
do not enable weak links, as "import sys" must now be replaced with
"import usys".
2020-09-04 00:10:24 +10:00
Damien George 0e6ef40359 tests/extmod: Add tests for verifying FAT and littlefs mtime values.
Verifies mtime timestamps on files match the value returned by time.time().

Also update vfs_fat_ramdisk.py so it doesn't check FAT timestamp of the
root, because that may change across runs/ports.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-02 00:19:38 +10:00
Damien George d1995e50eb extmod/modlwip: Fix error return for TCP recv when not connected.
This commit fixes the cases when a TCP socket is in STATE_NEW,
STATE_LISTENING or STATE_CONNECTING and recv() is called on it.  It now
raises ENOTCONN instead of a random error code due to it previously
indexing beyond the start of error_lookup_table[].

Signed-off-by: Damien George <damien@micropython.org>
2020-08-30 13:20:51 +10:00
Damien George 06659077a8 all: Update Python code to conform to latest black formatting.
Updating to Black v20.8b1 there are two changes that affect the code in
this repository:

- If there is a trailing comma in a list (eg [], () or function call) then
  that list is now written out with one line per element.  So remove such
  trailing commas where the list should stay on one line.

- Spaces at the start of """ doc strings are removed.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-29 15:18:01 +10:00
Damien George 0c7354afaf tests: Split out complex reverse-op tests to separate test file.
So they can be skipped if __rOP__'s are not supported on the target.  Also
fix the typo in the complex_special_methods.py filename.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-29 14:12:20 +10:00
stijn 0c3f9d58a5 tests/run-tests: Make test output directory configurable.
A configurable result directory is advantageous because it enables
using a dedicated location, eventually outside of the source tree,
instead of forcing the output files into a fixed directory which might
also contain other files already. For that reason the default output
directory also has been changed to tests/results/.
2020-08-27 11:12:08 +10:00
stijn 405893afc6 tests/run-tests: Use absolute paths where possible.
Replace some usages of paths relative to the current working directory
with absolute paths relative to the tests directory.

Fixes and resulting changes:
- default values of MICROPYTHON and MPYCROSS are absolute paths and
  always correct
- likewise, the correct full paths for tools and extmod directories
  are appended to sys.path
- printing/cleaning failures works properly since it expects the .exp
  and .out files in the tests directory which is also where they
  are written to now, plus no more need for changing directories

This fixes #5872 and allows running custom tests which use run-tests
without having to cd to the tests directory first, and the test output
still is in the tests/ directory instead of the current working directory.

Discovery of tests and all skip test logic based on paths relative to
the current working directory remains unchanged which essentially means
that for running most of MicroPython's own tests, run-tests must still
be ran from within it's directory, so document that.
2020-08-27 11:12:08 +10:00
Damien George 5fb276de33 tests/extmod: Make uasyncio_fair test more reliable by adjusting sleeps.
With sleep(0.2) a multiple of sleep(0.1), the order of task 2 and 3
execution is not well defined, and depends on the precision of the system
clock and how fast the rest of the code runs.  So change 0.2 to 0.18 to
make the test more reliable.

Also fix a typo of t3/t4, and cancel t4 at the end.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-26 17:05:52 +10:00
Damien George 2acc087880 extmod/vfs_lfs: Add mtime support to littlefs files.
This commit adds support for modification time of files on littlefs v2
filesystems, using file attributes.  For some background see issue #6114.

Features/properties of this implementation:
- Only supported on littlefs2 (not littlefs1).
- Uses littlefs2's general file attributes to store the timestamp.
- The timestamp is 64-bits and stores nanoseconds since 1970/1/1 (if the
  range to the year 2554 is not enough then additional bits can be added to
  this timestamp by adding another file attribute).
- mtime is enabled by default but can be disabled in the constructor, eg:
  uos.mount(uos.VfsLfs2(bdev, mtime=False), '/flash')
- It's fully backwards compatible, existing littlefs2 filesystems will work
  without reformatting and timestamps will be added transparently to
  existing files (once they are opened for writing).
- Files without timestamps will open correctly, and stat will just return 0
  for their timestamp.
- mtime can be disabled or enabled each mount time and timestamps will only
  be updated if mtime is enabled (otherwise they will be untouched).

Signed-off-by: Damien George <damien@micropython.org>
2020-08-25 17:35:19 +10:00
Damien George 55c76eaac1 extmod/uasyncio: Truncate negative sleeps to 0.
Otherwise a task that continuously awaits on a large negative sleep can
monopolise the scheduler (because its wake time is always less than
everything else in the pairing heap).

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 12:17:06 +10:00
Damien George 20948a3d54 tests/extmod: Add test for uasyncio.sleep of a negative time.
It should take 0 time to await on a negative sleep.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 12:17:06 +10:00
Damien George 5f9b105244 py/runtime: Fix builtin compile() in "single" mode so it prints exprs.
As per CPython behaviour, compile(stmt, "file", "single") should create
code which prints to stdout (via __repl_print__) the results of any
expressions in stmt.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 11:38:46 +10:00
Damien George 60f5b941e0 extmod/vfs_reader: Fix mp_reader_new_file to open file in "rb" mode.
mp_reader_new_file() is used to read in files for importing, either .py or
.mpy files, for the lexer and persistent code loader respectively.  In both
cases the file should be opened in raw bytes mode: the lexer handles
unicode characters itself, and .mpy files contain 8-bit bytes by nature.

Before this commit importing was working correctly because, although the
file was opened in text mode, all native filesystem implementations (POSIX,
FAT, LFS) would access the file in raw bytes mode via mp_stream_rw()
calling mp_stream_p_t.read().  So it was only an issue for non-native
filesystems, such as those implemented in Python.  For Python-based
filesystem implementations, a call to mp_stream_rw() would go via IOBase
and then to readinto() at the Python level, and readinto() is only defined
on files opened in raw bytes mode.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-12 23:40:50 +10:00
Damien George 9883d8e818 py/persistentcode: Maintain root ptr list of imported native .mpy code.
On ports where normal heap memory can contain executable code (eg ARM-based
ports such as stm32), native code loaded from an .mpy file may be reclaimed
by the GC because there's no reference to the very start of the native
machine code block that is reachable from root pointers (only pointers to
internal parts of the machine code block are reachable, but that doesn't
help the GC find the memory).

This commit fixes this issue by maintaining an explicit list of root
pointers pointing to native code that is loaded from an .mpy file.  This
is not needed for all ports so is selectable by the new configuration
option MICROPY_PERSISTENT_CODE_TRACK_RELOC_CODE.  It's enabled by default
if a port does not specify any special functions to allocate or commit
executable memory.

A test is included to test that native code loaded from an .mpy file does
not get reclaimed by the GC.

Fixes #6045.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-02 22:34:09 +10:00
Damien George 8da40baa47 tests/micropython: Improve .mpy import tests to run on more targets.
All imports are now tested to see if the test should be skipped,
UserFile.read is removed, and UserFile.readinto is made more efficient.

Signed-off-by: Damien George <damien@micropython.org>
2020-07-26 22:44:59 +10:00
Damien George 0c0cef9870 tests: Move .mpy import tests from import/ to micropython/ dir.
These tests are specific to MicroPython so have a better home in the
micropython/ test subdir, and putting them here allows them to be run by
all targets, not just those that have access to the local filesystem (eg
the unix port).

Signed-off-by: Damien George <damien@micropython.org>
2020-07-26 22:04:31 +10:00
Damien George 441460d81f extmod/uasyncio: Add StreamReader.readexactly(n) method.
It raises on EOFError instead of an IncompleteReadError (which is what
CPython does).  But the latter is derived from EOFError so code compatible
with MicroPython and CPython can be written by catching EOFError (eg see
included test).

Fixes issue #6156.

Signed-off-by: Damien George <damien@micropython.org>
2020-07-25 23:10:05 +10:00
Jonathan Hogg 37e1b5c891 py/compile: Don't await __aiter__ special method in async-for.
MicroPython's original implementation of __aiter__ was correct for an
earlier (provisional) version of PEP492 (CPython 3.5), where __aiter__ was
an async-def function.  But that changed in the final version of PEP492 (in
CPython 3.5.2) where the function was changed to a normal one.  See
https://www.python.org/dev/peps/pep-0492/#why-aiter-does-not-return-an-awaitable
See also the note at the end of this subsection in the docs:
https://docs.python.org/3.5/reference/datamodel.html#asynchronous-iterators
And for completeness the BPO: https://bugs.python.org/issue27243

To be consistent with the Python spec as it stands today (and now that
PEP492 is final) this commit changes MicroPython's behaviour to match
CPython:  __aiter__ should return an async-iterable object, but is not
itself awaitable.

The relevant tests are updated to match.

See #6267.
2020-07-25 00:58:18 +10:00
Damien George a853fff838 py/obj.h: Fix mp_seq_replace_slice_no_grow to use memmove not memcpy.
Because the argument arrays may overlap, as show by the new tests in this
commit.

Also remove the debugging comments for these macros, add a new comment
about overlapping regions, and separate the macros by blank lines to make
them easier to read.

Fixes issue #6244.

Signed-off-by: Damien George <damien@micropython.org>
2020-07-21 01:05:59 +10:00
Damien George 895b1dbdda tests/basics: Split out memoryview slice-assign tests to separate file.
Signed-off-by: Damien George <damien@micropython.org>
2020-07-21 01:05:59 +10:00
Thorsten von Eicken 5264478007 extmod/modussl_mbedtls: Integrate shorter error strings.
The stm32 and esp32 ports now use shorter error strings for mbedtls errors.
Also, MBEDTLS_ERROR_C is enabled on stm32 by default to get these strings.
2020-07-21 00:31:05 +10:00
Thorsten von Eicken 9aa214077e extmod/modussl: Improve exception error messages.
This commit adds human readable error messages when mbedtls or axtls raise
an exception.  Currently often just an EIO error is raised so the user is
lost and can't tell whether it's a cert error, buffer overrun, connecting
to a non-ssl port, etc.  The axtls and mbedtls error raising in the ussl
module is modified to raise:

    OSError(-err_num, "error string")

For axtls a small error table of strings is added and used for the second
argument of the OSErrer.  For mbedtls the code uses mbedtls' built-in
strerror function, and if there is an out of memory condition it just
produces OSError(-err_num).  Producing the error string for mbedtls is
conditional on them being included in the mbedtls build, via
MBEDTLS_ERROR_C.
2020-07-20 23:41:45 +10:00
Jim Mussared 9d823a5d9a extmod/modbluetooth: Add event for "indicate acknowledgement".
This commit adds the IRQ_GATTS_INDICATE_DONE BLE event which will be raised
with the status of gatts_indicate (unlike notify, indications require
acknowledgement).

An example of its use is added to ble_temperature.py, and to the multitests
in ble_characteristic.py.

Implemented for btstack and nimble bindings, tested in both directions
between unix/btstack and pybd/nimble.
2020-07-20 23:26:41 +10:00
Jim Mussared e152d0c197 extmod/btstack: Schedule notify/indicate/write ops for bg completion.
The goal of this commit is to allow using ble.gatts_notify() at any time,
even if the stack is not ready to send the notification right now.  It also
addresses the same issue for ble.gatts_indicate() and ble.gattc_write()
(without response).  In addition this commit fixes the case where the
buffer passed to write-with-response wasn't copied, meaning it could be
modified by the caller, affecting the in-progress write.

The changes are:

- gatts_notify/indicate will now run in the background if the ACL buffer is
  currently full, meaning that notify/indicate can be called at any time.

- gattc_write(mode=0) (no response) will now allow for one outstanding
  write.

- gattc_write(mode=1) (with response) will now copy the buffer so that it
  can't be modified by the caller while the write is in progress.

All four paths also now track the buffer while the operation is in
progress, which prevents the GC free'ing the buffer while it's still
needed.
2020-07-18 14:23:47 +10:00
Damien George 332d83343f py: Rework mp_convert_member_lookup to properly handle built-ins.
This commit fixes lookups of class members to make it so that built-in
functions that are used as methods/functions of a class work correctly.

The mp_convert_member_lookup() function is pretty much completely changed
by this commit, but for the most part it's just reorganised and the
indenting changed.  The functional changes are:

- staticmethod and classmethod checks moved to later in the if-logic,
  because they are less common and so should be checked after the more
  common cases.

- The explicit mp_obj_is_type(member, &mp_type_type) check is removed
  because it's now subsumed by other, more general tests in this function.

- MP_TYPE_FLAG_BINDS_SELF and MP_TYPE_FLAG_BUILTIN_FUN type flags added to
  make the checks in this function much simpler (now they just test this
  bit in type->flags).

- An extra check is made for mp_obj_is_instance_type(type) to fix lookup of
  built-in functions.

Fixes #1326 and #6198.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-30 23:55:32 +10:00
Damien George 9f911d822e py/objcomplex: Add mp_obj_get_complex_maybe for use in complex bin-op.
This allows complex binary operations to fail gracefully with unsupported
operation rather than raising an exception, so that special methods work
correctly.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-27 01:03:10 +10:00
Damien George 41fa8b5482 py/emitnative: Implement binary operations for viper uint operands.
uint types in viper mode can now be used for all binary operators except
floor-divide and modulo.

Fixes issue #1847 and issue #6177.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-27 00:24:04 +10:00
Damien George 457fdf61c3 py/objtype: Support passing in an OrderedDict to type() as the locals.
An OrderedDict can now be used for the locals when creating a type
explicitly via type(name, bases, locals).

Signed-off-by: Damien George <damien@micropython.org>
2020-06-24 12:05:40 +10:00
Damien George a51eef4471 tests/basics: Add tests for variable annotations. 2020-06-16 23:18:01 +10:00