Commit Graph

2029 Commits

Author SHA1 Message Date
Jim Mussared 26b66804e9 tests/multi_bluetooth: Update to new config('mac') behaviour. 2020-09-08 12:53:24 +10:00
Jeff Epler 7d58cdb12c update expected result with new method 2020-09-07 07:11:23 -05:00
Jeff Epler 20c2dd0c08 core: add int.bit_length() when MICROPY_CYPTHON_COMPAT is enabled
This method of integer objects is needed for a port of python3's
decimal.py module.

MICROPY_CPYTHON_COMPAT is enabled by CIRCUITPY_FULL_BUILD.
2020-09-06 09:53:16 -05:00
stijn 40ad8f1666 all: Rename "sys" module to "usys".
This is consistent with the other 'micro' modules and allows implementing
additional features in Python via e.g. micropython-lib's sys.

Note this is a breaking change (not backwards compatible) for ports which
do not enable weak links, as "import sys" must now be replaced with
"import usys".
2020-09-04 00:10:24 +10:00
Damien George 0e6ef40359 tests/extmod: Add tests for verifying FAT and littlefs mtime values.
Verifies mtime timestamps on files match the value returned by time.time().

Also update vfs_fat_ramdisk.py so it doesn't check FAT timestamp of the
root, because that may change across runs/ports.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-02 00:19:38 +10:00
Damien George d1995e50eb extmod/modlwip: Fix error return for TCP recv when not connected.
This commit fixes the cases when a TCP socket is in STATE_NEW,
STATE_LISTENING or STATE_CONNECTING and recv() is called on it.  It now
raises ENOTCONN instead of a random error code due to it previously
indexing beyond the start of error_lookup_table[].

Signed-off-by: Damien George <damien@micropython.org>
2020-08-30 13:20:51 +10:00
Damien George 06659077a8 all: Update Python code to conform to latest black formatting.
Updating to Black v20.8b1 there are two changes that affect the code in
this repository:

- If there is a trailing comma in a list (eg [], () or function call) then
  that list is now written out with one line per element.  So remove such
  trailing commas where the list should stay on one line.

- Spaces at the start of """ doc strings are removed.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-29 15:18:01 +10:00
Damien George 0c7354afaf tests: Split out complex reverse-op tests to separate test file.
So they can be skipped if __rOP__'s are not supported on the target.  Also
fix the typo in the complex_special_methods.py filename.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-29 14:12:20 +10:00
stijn 0c3f9d58a5 tests/run-tests: Make test output directory configurable.
A configurable result directory is advantageous because it enables
using a dedicated location, eventually outside of the source tree,
instead of forcing the output files into a fixed directory which might
also contain other files already. For that reason the default output
directory also has been changed to tests/results/.
2020-08-27 11:12:08 +10:00
stijn 405893afc6 tests/run-tests: Use absolute paths where possible.
Replace some usages of paths relative to the current working directory
with absolute paths relative to the tests directory.

Fixes and resulting changes:
- default values of MICROPYTHON and MPYCROSS are absolute paths and
  always correct
- likewise, the correct full paths for tools and extmod directories
  are appended to sys.path
- printing/cleaning failures works properly since it expects the .exp
  and .out files in the tests directory which is also where they
  are written to now, plus no more need for changing directories

This fixes #5872 and allows running custom tests which use run-tests
without having to cd to the tests directory first, and the test output
still is in the tests/ directory instead of the current working directory.

Discovery of tests and all skip test logic based on paths relative to
the current working directory remains unchanged which essentially means
that for running most of MicroPython's own tests, run-tests must still
be ran from within it's directory, so document that.
2020-08-27 11:12:08 +10:00
Damien George 5fb276de33 tests/extmod: Make uasyncio_fair test more reliable by adjusting sleeps.
With sleep(0.2) a multiple of sleep(0.1), the order of task 2 and 3
execution is not well defined, and depends on the precision of the system
clock and how fast the rest of the code runs.  So change 0.2 to 0.18 to
make the test more reliable.

Also fix a typo of t3/t4, and cancel t4 at the end.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-26 17:05:52 +10:00
Damien George 2acc087880 extmod/vfs_lfs: Add mtime support to littlefs files.
This commit adds support for modification time of files on littlefs v2
filesystems, using file attributes.  For some background see issue #6114.

Features/properties of this implementation:
- Only supported on littlefs2 (not littlefs1).
- Uses littlefs2's general file attributes to store the timestamp.
- The timestamp is 64-bits and stores nanoseconds since 1970/1/1 (if the
  range to the year 2554 is not enough then additional bits can be added to
  this timestamp by adding another file attribute).
- mtime is enabled by default but can be disabled in the constructor, eg:
  uos.mount(uos.VfsLfs2(bdev, mtime=False), '/flash')
- It's fully backwards compatible, existing littlefs2 filesystems will work
  without reformatting and timestamps will be added transparently to
  existing files (once they are opened for writing).
- Files without timestamps will open correctly, and stat will just return 0
  for their timestamp.
- mtime can be disabled or enabled each mount time and timestamps will only
  be updated if mtime is enabled (otherwise they will be untouched).

Signed-off-by: Damien George <damien@micropython.org>
2020-08-25 17:35:19 +10:00
Damien George 55c76eaac1 extmod/uasyncio: Truncate negative sleeps to 0.
Otherwise a task that continuously awaits on a large negative sleep can
monopolise the scheduler (because its wake time is always less than
everything else in the pairing heap).

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 12:17:06 +10:00
Damien George 20948a3d54 tests/extmod: Add test for uasyncio.sleep of a negative time.
It should take 0 time to await on a negative sleep.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 12:17:06 +10:00
Damien George 5f9b105244 py/runtime: Fix builtin compile() in "single" mode so it prints exprs.
As per CPython behaviour, compile(stmt, "file", "single") should create
code which prints to stdout (via __repl_print__) the results of any
expressions in stmt.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 11:38:46 +10:00
Damien George 60f5b941e0 extmod/vfs_reader: Fix mp_reader_new_file to open file in "rb" mode.
mp_reader_new_file() is used to read in files for importing, either .py or
.mpy files, for the lexer and persistent code loader respectively.  In both
cases the file should be opened in raw bytes mode: the lexer handles
unicode characters itself, and .mpy files contain 8-bit bytes by nature.

Before this commit importing was working correctly because, although the
file was opened in text mode, all native filesystem implementations (POSIX,
FAT, LFS) would access the file in raw bytes mode via mp_stream_rw()
calling mp_stream_p_t.read().  So it was only an issue for non-native
filesystems, such as those implemented in Python.  For Python-based
filesystem implementations, a call to mp_stream_rw() would go via IOBase
and then to readinto() at the Python level, and readinto() is only defined
on files opened in raw bytes mode.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-12 23:40:50 +10:00
Damien George 9883d8e818 py/persistentcode: Maintain root ptr list of imported native .mpy code.
On ports where normal heap memory can contain executable code (eg ARM-based
ports such as stm32), native code loaded from an .mpy file may be reclaimed
by the GC because there's no reference to the very start of the native
machine code block that is reachable from root pointers (only pointers to
internal parts of the machine code block are reachable, but that doesn't
help the GC find the memory).

This commit fixes this issue by maintaining an explicit list of root
pointers pointing to native code that is loaded from an .mpy file.  This
is not needed for all ports so is selectable by the new configuration
option MICROPY_PERSISTENT_CODE_TRACK_RELOC_CODE.  It's enabled by default
if a port does not specify any special functions to allocate or commit
executable memory.

A test is included to test that native code loaded from an .mpy file does
not get reclaimed by the GC.

Fixes #6045.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-02 22:34:09 +10:00
Damien George 8da40baa47 tests/micropython: Improve .mpy import tests to run on more targets.
All imports are now tested to see if the test should be skipped,
UserFile.read is removed, and UserFile.readinto is made more efficient.

Signed-off-by: Damien George <damien@micropython.org>
2020-07-26 22:44:59 +10:00
Damien George 0c0cef9870 tests: Move .mpy import tests from import/ to micropython/ dir.
These tests are specific to MicroPython so have a better home in the
micropython/ test subdir, and putting them here allows them to be run by
all targets, not just those that have access to the local filesystem (eg
the unix port).

Signed-off-by: Damien George <damien@micropython.org>
2020-07-26 22:04:31 +10:00
Damien George 441460d81f extmod/uasyncio: Add StreamReader.readexactly(n) method.
It raises on EOFError instead of an IncompleteReadError (which is what
CPython does).  But the latter is derived from EOFError so code compatible
with MicroPython and CPython can be written by catching EOFError (eg see
included test).

Fixes issue #6156.

Signed-off-by: Damien George <damien@micropython.org>
2020-07-25 23:10:05 +10:00
Jonathan Hogg 901f3dce6e py/compile: Don't await __aiter__ special method in async-for.
MicroPython's original implementation of __aiter__ was correct for an
earlier (provisional) version of PEP492 (CPython 3.5), where __aiter__ was
an async-def function.  But that changed in the final version of PEP492 (in
CPython 3.5.2) where the function was changed to a normal one.  See
https://www.python.org/dev/peps/pep-0492/#why-aiter-does-not-return-an-awaitable
See also the note at the end of this subsection in the docs:
https://docs.python.org/3.5/reference/datamodel.html#asynchronous-iterators
And for completeness the BPO: https://bugs.python.org/issue27243

To be consistent with the Python spec as it stands today (and now that
PEP492 is final) this commit changes MicroPython's behaviour to match
CPython:  __aiter__ should return an async-iterable object, but is not
itself awaitable.

The relevant tests are updated to match.

See #6267.
2020-07-24 22:55:37 -07:00
Jonathan Hogg 37e1b5c891 py/compile: Don't await __aiter__ special method in async-for.
MicroPython's original implementation of __aiter__ was correct for an
earlier (provisional) version of PEP492 (CPython 3.5), where __aiter__ was
an async-def function.  But that changed in the final version of PEP492 (in
CPython 3.5.2) where the function was changed to a normal one.  See
https://www.python.org/dev/peps/pep-0492/#why-aiter-does-not-return-an-awaitable
See also the note at the end of this subsection in the docs:
https://docs.python.org/3.5/reference/datamodel.html#asynchronous-iterators
And for completeness the BPO: https://bugs.python.org/issue27243

To be consistent with the Python spec as it stands today (and now that
PEP492 is final) this commit changes MicroPython's behaviour to match
CPython:  __aiter__ should return an async-iterable object, but is not
itself awaitable.

The relevant tests are updated to match.

See #6267.
2020-07-25 00:58:18 +10:00
Kenny 764d49e641 also disable async_coroutine test in native emitter 2020-07-23 20:40:16 -07:00
Kenny 51a79b1af7 add coroutine behavior for generators
coroutines don't have __next__; they also call themselves coroutines.
This does not change the fact that `async def` methods are generators,
but it does make them behave more like CPython.
2020-07-23 20:40:16 -07:00
Damien George a853fff838 py/obj.h: Fix mp_seq_replace_slice_no_grow to use memmove not memcpy.
Because the argument arrays may overlap, as show by the new tests in this
commit.

Also remove the debugging comments for these macros, add a new comment
about overlapping regions, and separate the macros by blank lines to make
them easier to read.

Fixes issue #6244.

Signed-off-by: Damien George <damien@micropython.org>
2020-07-21 01:05:59 +10:00
Damien George 895b1dbdda tests/basics: Split out memoryview slice-assign tests to separate file.
Signed-off-by: Damien George <damien@micropython.org>
2020-07-21 01:05:59 +10:00
Thorsten von Eicken 5264478007 extmod/modussl_mbedtls: Integrate shorter error strings.
The stm32 and esp32 ports now use shorter error strings for mbedtls errors.
Also, MBEDTLS_ERROR_C is enabled on stm32 by default to get these strings.
2020-07-21 00:31:05 +10:00
Thorsten von Eicken 9aa214077e extmod/modussl: Improve exception error messages.
This commit adds human readable error messages when mbedtls or axtls raise
an exception.  Currently often just an EIO error is raised so the user is
lost and can't tell whether it's a cert error, buffer overrun, connecting
to a non-ssl port, etc.  The axtls and mbedtls error raising in the ussl
module is modified to raise:

    OSError(-err_num, "error string")

For axtls a small error table of strings is added and used for the second
argument of the OSErrer.  For mbedtls the code uses mbedtls' built-in
strerror function, and if there is an out of memory condition it just
produces OSError(-err_num).  Producing the error string for mbedtls is
conditional on them being included in the mbedtls build, via
MBEDTLS_ERROR_C.
2020-07-20 23:41:45 +10:00
Jim Mussared 9d823a5d9a extmod/modbluetooth: Add event for "indicate acknowledgement".
This commit adds the IRQ_GATTS_INDICATE_DONE BLE event which will be raised
with the status of gatts_indicate (unlike notify, indications require
acknowledgement).

An example of its use is added to ble_temperature.py, and to the multitests
in ble_characteristic.py.

Implemented for btstack and nimble bindings, tested in both directions
between unix/btstack and pybd/nimble.
2020-07-20 23:26:41 +10:00
Jim Mussared e152d0c197 extmod/btstack: Schedule notify/indicate/write ops for bg completion.
The goal of this commit is to allow using ble.gatts_notify() at any time,
even if the stack is not ready to send the notification right now.  It also
addresses the same issue for ble.gatts_indicate() and ble.gattc_write()
(without response).  In addition this commit fixes the case where the
buffer passed to write-with-response wasn't copied, meaning it could be
modified by the caller, affecting the in-progress write.

The changes are:

- gatts_notify/indicate will now run in the background if the ACL buffer is
  currently full, meaning that notify/indicate can be called at any time.

- gattc_write(mode=0) (no response) will now allow for one outstanding
  write.

- gattc_write(mode=1) (with response) will now copy the buffer so that it
  can't be modified by the caller while the write is in progress.

All four paths also now track the buffer while the operation is in
progress, which prevents the GC free'ing the buffer while it's still
needed.
2020-07-18 14:23:47 +10:00
Dan Halbert 88d8956378
Merge pull request #3143 from tannewt/improve_json
Add support to json.load for any object with readinto
2020-07-16 14:33:33 -04:00
Scott Shawcroft 372bcf8a95
Fix stream version and add basic readinto test 2020-07-10 17:33:17 -07:00
Diego Elio Pettenò 34b4993d63 Add license to some obvious files. 2020-07-06 19:16:25 +01:00
Damien George 332d83343f py: Rework mp_convert_member_lookup to properly handle built-ins.
This commit fixes lookups of class members to make it so that built-in
functions that are used as methods/functions of a class work correctly.

The mp_convert_member_lookup() function is pretty much completely changed
by this commit, but for the most part it's just reorganised and the
indenting changed.  The functional changes are:

- staticmethod and classmethod checks moved to later in the if-logic,
  because they are less common and so should be checked after the more
  common cases.

- The explicit mp_obj_is_type(member, &mp_type_type) check is removed
  because it's now subsumed by other, more general tests in this function.

- MP_TYPE_FLAG_BINDS_SELF and MP_TYPE_FLAG_BUILTIN_FUN type flags added to
  make the checks in this function much simpler (now they just test this
  bit in type->flags).

- An extra check is made for mp_obj_is_instance_type(type) to fix lookup of
  built-in functions.

Fixes #1326 and #6198.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-30 23:55:32 +10:00
Damien George 9f911d822e py/objcomplex: Add mp_obj_get_complex_maybe for use in complex bin-op.
This allows complex binary operations to fail gracefully with unsupported
operation rather than raising an exception, so that special methods work
correctly.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-27 01:03:10 +10:00
Damien George 41fa8b5482 py/emitnative: Implement binary operations for viper uint operands.
uint types in viper mode can now be used for all binary operators except
floor-divide and modulo.

Fixes issue #1847 and issue #6177.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-27 00:24:04 +10:00
Jeff Epler d82292fa1f README: Remove a statement that's not particularly true at the moment 2020-06-25 11:44:21 -05:00
Jeff Epler 3c11f6cc05 tests: Remove a test for micropython-specific SPI behavior 2020-06-25 11:44:21 -05:00
Jeff Epler 9737dd9c30 Scripts: Change wording for pseudoterminals 2020-06-25 11:42:23 -05:00
Damien George 457fdf61c3 py/objtype: Support passing in an OrderedDict to type() as the locals.
An OrderedDict can now be used for the locals when creating a type
explicitly via type(name, bases, locals).

Signed-off-by: Damien George <damien@micropython.org>
2020-06-24 12:05:40 +10:00
Damien George a51eef4471 tests/basics: Add tests for variable annotations. 2020-06-16 23:18:01 +10:00
Damien George a3c89cf907 tests/cpydiff: Add CPy diff test for assignment expression behaviour. 2020-06-16 22:06:47 +10:00
Damien George e0fe8ea644 tests/basics: Add tests for assignment operator :=. 2020-06-16 22:06:30 +10:00
Damien George 1783950311 py/compile: Implement PEP 572, assignment expressions with := operator.
The syntax matches CPython and the semantics are equivalent except that,
unlike CPython, MicroPython allows using := to assign to comprehension
iteration variables, because disallowing this would take a lot of code to
check for it.

The new compile-time option MICROPY_PY_ASSIGN_EXPR selects this feature and
is enabled by default, following MICROPY_PY_ASYNC_AWAIT.
2020-06-16 22:02:24 +10:00
Jim Mussared 00c3e2156a tests/run-multitests.py: Allow passing unique env vars to each instance.
For example, to run the BLE multitests entirely with the unix port:

    env MICROPY_MICROPYTHON=../ports/unix/micropython-dev ./run-multitests.py \
        -i micropython,MICROPYBTUSB=01 \
        -i micropython,MICROPYBTUSB=02:02 \
        multi_bluetooth/ble_*.py
2020-06-10 22:41:57 +10:00
Damien George a4c96fb3b0 extmod/uasyncio: Add asyncio.wait_for_ms function.
Fixes issue #6107.
2020-06-10 22:29:44 +10:00
Andrew Leech 28370c0450 py/objtype: Add __dict__ attribute for class objects.
The behavior mirrors the instance object dict attribute where a copy of the
local attributes are provided (unless the dict is read-only, then that dict
itself is returned, as an optimisation).  MicroPython does not support
modifying this dict because the changes will not be reflected in the class.

The feature is only enabled if MICROPY_CPYTHON_COMPAT is set, the same as
the instance version.
2020-06-10 21:58:13 +10:00
stijn 51fd6c9777 extmod/ure: Use single function for match/search/sub.
Saves about 500 bytes on unix x64 and enables CPython-conform
usage of passing a re object to these functions.
2020-06-08 09:16:09 +02:00
Jim Mussared 9902ce12eb tests/multi_bluetooth: Update to work with new BLE events.
Updates the tests to use non-bitmask events, event renames, as well as some
of the new completion events to improve reliability of the tests.
2020-06-05 14:08:47 +10:00
Diego Elio Pettenò dd5d7c86d2 Fix up end of file and trailing whitespace.
This can be enforced by pre-commit, but correct it separately to make it easier to review.
2020-06-03 10:56:35 +01:00
Damien George 203b10703e py/modbuiltins: Fix getattr to work with class raising AttributeError.
Fixes issue #6089.
2020-06-02 15:42:20 +10:00
Damien George 22806ed5df extmod/vfs: Retain previous working directory if chdir fails.
Fixes issue #6069.
2020-05-29 23:05:01 +10:00
stijn 81db22f693 py/modmath: Work around msvc float bugs in atan2, fmod and modf.
Older implementations deal with infinity/negative zero incorrectly.  This
commit adds generic fixes that can be enabled by any port that needs them,
along with new tests cases.
2020-05-28 09:54:54 +10:00
Damien George 7dffbfd22a extmod/vfs_lfsx: Fix import_stat so it takes into account current dir.
CPython semantics require searching the current directory if the import is
not absolute (when "" is in sys.path).

Fixes issue #6037.
2020-05-15 11:31:32 +10:00
Damien George 3b6c9119eb extmod/modbluetooth: Add support for changing the GAP device name.
This commit allows the user to set/get the GAP device name used by service
0x1800, characteristic 0x2a00.  The usage is:

    BLE.config(gap_name="myname")
    print(BLE.config("gap_name"))

As part of this change the compile-time setting
MICROPY_PY_BLUETOOTH_DEFAULT_NAME is renamed to
MICROPY_PY_BLUETOOTH_DEFAULT_GAP_NAME to emphasise its link to GAP and this
new "gap_name" config value.  And the default value of this for the NimBLE
bindings is changed from "PYBD" to "MPY NIMBLE" to be more generic.
2020-05-11 21:30:41 +10:00
Thorsten von Eicken ab4e197707 esp32/modsocket: Fix getaddrinfo to raise on error.
This commit fixes the behaviour of socket.getaddrinfo on the ESP32 so it
raises an OSError when the name resolution fails instead of returning a []
or a resolution for 0.0.0.0.

Tests are added (generic and ESP32-specific) to verify behaviour consistent
with CPython, modulo the different types of exceptions per MicroPython
documentation.
2020-05-09 16:43:48 +10:00
Damien George adb6733022 tests/run-tests: Skip REPL feature checks when running via pyboard.py. 2020-05-09 16:20:40 +10:00
Damien George 035059eaf5 tests/run-multitests.py: Add TRACE banner and move TEST output to end.
To make it easier to understand the output when both -s and -t are used.
2020-05-09 00:06:06 +10:00
robert 0f83ef395c extmod/vfs_lfsx: Fix rename to respect cur dir for new path.
If the new name start with '/', cur_dir is not prepened any more, so that
the current working directory is respected.  And extend the test cases for
rename to cover this functionality.
2020-05-08 21:54:04 +10:00
robert d3ea28d04a extmod/vfs_lfsx: Normalize path name in chdir.
This change scans for '.', '..' and multiple '/' and normalizes the new
path name.  If the resulting path does not exist, an error is raised.
Non-existing interim path elements are ignored if they are removed during
normalization.
2020-05-08 21:52:15 +10:00
robert a5ea4b9f3f extmod/vfs_lfsx: Fix path handling in uos.stat() to consider cur dir.
This fixes the bug, that stat(filename) would not consider the current
working directory.  So if e.g. the cwd is "lib", then stat("main.py") would
return the info for "/main.py" instead of "/lib/main.py".
2020-05-08 21:37:51 +10:00
Damien George 037c83b0ed tests/multi_bluetooth: Fix typo printing wrong IRQ type. 2020-05-08 13:23:45 +10:00
yangfl 138a28dc07 tests/thread/thread_stacksize1.py: Increase stack size for CPython.
On arm64 with CPython:

  >>> _thread.stack_size(32*1024)
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
  ValueError: size not valid: 32768 bytes

So increase the CPython value in the test to 512k so it runs on more
systems (on modern Linux the default stack size is usually 8MB).
2020-05-05 00:43:24 +10:00
Damien George 4ede703687 py/parse: Support constant folding of power operator for integers.
Constant expression like "2 ** 3" will now be folded, and the special form
"X = const(2 ** 3)" will now compile because the argument to the const is
now a constant.

Fixes issue #5865.
2020-05-03 16:23:19 +10:00
Thorsten von Eicken 952ff8a8ea esp32: Improve support for OTA updates.
This commit adds several small items to improve the support for OTA
updates on an esp32:

- a partition table for 4MB flash modules that has two OTA partitions ready
  to go to do updates
- a GENERIC_OTA board that uses that partition table and that enables
  automatic roll-back in the bootloader
- a new esp32.Partition.mark_app_valid_cancel_rollback() class-method to
  signal that the boot is successful and should not be rolled back at the
  next reset
- an automated test for doing an OTA update
- documentation updates
2020-05-03 15:00:45 +10:00
Jim Mussared 309c19d39b tests/cpydiff: Add cpydiff test for __all__ used in imported package. 2020-05-02 17:41:04 +10:00
Damien George 73c58150f5 extmod/modbtree: Retain reference to underlying stream so it's not GC'd.
For ports that have a system malloc which is not garbage collected (eg
unix, esp32), the stream object for the DB must be retained separately to
prevent it from being reclaimed by the MicroPython GC (because the
berkeley-db library uses malloc to allocate the DB structure which stores
the only reference to the stream).

Although in some cases the user code will explicitly retain a reference to
the underlying stream because it needs to call close() on it, this is not
always the case, eg in cases where the DB is intended to live forever.

Fixes issue #5940.
2020-05-02 16:08:04 +10:00
Jim Mussared 59a7865a7d tests/run-multitests.py: Add shortcuts for local Python instances.
One can now use `-i micropython` and `-i cpython` to add instances using
the `MICROPYTHON` and `CPYTHON3` variables (which can be overridden by env
vars).
2020-04-29 16:45:46 +10:00
Jim Mussared 9b06efb943 tests/multi_bluetooth/ble_gap_advertise: Fix bytes/str compare warning. 2020-04-29 16:45:46 +10:00
Jim Mussared f588138bbe tests/run-multitests.py: Allow filtering out lines from stdout.
And use this new feature to filter out certain lines in the Bluetooth
multitests.
2020-04-29 16:45:46 +10:00
Damien George 4fa6d939d6 tests/extmod: Add btree test for errors raised by btree DB library.
This test now passes given the previous two commits.
2020-04-27 23:59:09 +10:00
Jim Mussared 57fce3bdb2 py/objdict: Fix popitem for ordered dicts.
The popitem method wasn't implemented for ordered dicts and would result in
an invalid state.

Fixes issue #5956.
2020-04-27 23:53:17 +10:00
Thorsten von Eicken 1ae7e0e561 esp32: Consolidate check_esp_err functions and add IDF error string.
This commit consolidates a number of check_esp_err functions that check
whether an ESP-IDF return code is OK and raises an exception if not.  The
exception raised is an OSError with the error code as the first argument
(negative if it's ESP-IDF specific) and the ESP-IDF error string as the
second argument.

This commit also fixes esp32.Partition.set_boot to use check_esp_err, and
uses that function for a unit test.
2020-04-23 10:59:07 +10:00
Thorsten von Eicken a177831c46 esp32/modesp32: Add idf_heap_info(capabilities) to esp32 module.
This commit adds an idf_heap_info(capabilities) method to the esp32 module
which returns info about the ESP-IDF heaps.  It's useful to get a bit of a
picture of what's going on when code fails because ESP-IDF can't allocate
memory anymore.  Includes documentation and a test.
2020-04-23 00:02:11 +10:00
stijn f31f9a8b70 py/objint: Do not use fpclassify.
For combinations of certain versions of glibc and gcc the definition of
fpclassify always takes float as argument instead of adapting itself to
float/double/long double as required by the C99 standard.  At the time of
writing this happens for instance for glibc 2.27 with gcc 7.5.0 when
compiled with -Os and glibc 3.0.7 with gcc 9.3.0.  When calling fpclassify
with double as argument, as in objint.c, this results in an implicit
narrowing conversion which is not really correct plus results in a warning
when compiled with -Wfloat-conversion.  So fix this by spelling out the
logic manually.
2020-04-18 22:42:24 +10:00
stijn 7fb9edf436 tests/float: Fix cmath_fun_special for MICROPY_FLOAT_IMPL_FLOAT.
When the unix and windows ports use MICROPY_FLOAT_IMPL_FLOAT instead of
MICROPY_FLOAT_IMPL_DOUBLE, the test output has for example
complex(-0.15052, 0.34109) instead of the expected
complex(-0.15051, 0.34109).

Use one decimal place less for the output printing to fix this.
2020-04-18 22:36:49 +10:00
Damien George 5f0661b4fe extmod/uasyncio: Change cannot to can't in error message, and test exp.
Follow up to 8e048d2548 which missed these.
2020-04-14 21:51:25 +10:00
Jeff Epler d1a2a1a333 test requires yield, can't run native 2020-04-13 19:00:16 -05:00
Damien George db137e70dc extmod/uasyncio: Add Loop.new_event_loop method.
This commit adds Loop.new_event_loop() which is used to reset the singleton
event loop.  This functionality is put here instead of in Loop.close() to
make it possible to write code that is compatible with CPython.
2020-04-13 22:16:52 +10:00
Jim Mussared 8470cd0be9 py/scheduler: Add assert that scheduler is locked when unlocking.
And add a test that shows how this can happen when multiple threads are
accessing the scheduler, which fails if atomic sections are not used.
2020-04-13 21:55:47 +10:00
stijn f66c989516 tests/run-tests: Make diff tool user configurable. 2020-04-13 16:43:01 +10:00
Damien George 4914731e58 py/parse: Remove unnecessary check in const folding for ** operator.
In this part of the code there is no way to get the ** operator, so no need
to check for it.

This commit also adds tests for this, and other related, invalid const
operations.
2020-04-09 16:02:39 +10:00
Damien George a5f2ae10fe tests/extmod: Update littlefs test output to match new library version.
The amount of free space on the VfsLfs2 filesystem after creating a large
file is reduced by 2 blocks in this test.
2020-04-09 15:59:28 +10:00
Damien George c5a21a94f8 extmod/modbluetooth: Provide FLAG_WRITE_NO_RESPONSE for characteristics.
This flag is supported and needs to be set if characteristics are write-
without-response.
2020-04-07 13:46:56 +10:00
Jim Mussared 85858e72df py/objexcept: Allow compression of exception message text.
The decompression of error-strings is only done if the string is accessed
via printing or via er.args.  Tests are added for this feature to ensure
the decompression works.
2020-04-05 15:02:06 +10:00
Damien George df156b18e5 docs,tests: Add docs and test for uasyncio custom exc handler methods. 2020-04-04 10:47:48 +11:00
Damien George aa10e5c334 tests: Add .exp files for basics/parser and import/import_override.
Because CPython 3.8.0 now produces different output:
- basics/parser.py: CPython does not allow '\\\n' as input.
- import/import_override: CPython imports _io.
2020-04-01 16:31:33 -05:00
Damien George b389bc0afa extmod/uasyncio: Implement Loop.stop() to stop the event loop. 2020-04-02 00:14:18 +11:00
Damien George 764e65fb11 tests/basics: Provide .exp files for generator tests that fail PEP479.
PEP479 (see https://www.python.org/dev/peps/pep-0479/) prohibited raising
StopIteration from within a generator (it is turned into a RuntimeError).
This behaviour was introduced in Python 3.5 and in 3.7 was made compulsory.
Until uPy implements PEP479, this patch adds .py.exp files for the relevant
tests so they can be run under Python 3.7.
2020-03-31 17:27:10 -05:00
Damien George 8f0147cf00 tests: Modify tests that print repr of an exception with 1 arg.
In Python 3.7 the behaviour of repr() of an exception with one argument
changed: it no longer prints a trailing comma in the argument list.  See
https://bugs.python.org/issue30399

This patch modifies tests that rely on this behaviour to not rely on it.
And the python34.py test is updated to include a test for this behaviour
with a .exp file.
2020-03-31 17:27:10 -05:00
Damien George 4c4f81f8f2 tests/basics/int_big_error.py: Use bytearray to test for int overflow.
In Python 3.7 "1 >> (big int)" is now allowed, it no longer raises an
OverflowError.  So use bytearray to test big-int conversion overflow.
2020-03-31 17:27:10 -05:00
Damien George 8b5fd95897 tests/basics/set_pop.py: Sort set before printing for consistent output. 2020-03-31 17:27:10 -05:00
David Lechner 581f9135a4 tests/run-tests: Add commands to print and clean *.exp,out files.
This adds a couple of commands to the run-tests script to print the diffs
of failed tests and also to clean up the .exp and .out files after failed
tests.  (And a spelling error is fixed while we are touching nearby code.)

Travis is also updated to use these new commands, including using it for
more builds.
2020-03-30 13:25:58 +11:00
David Lechner 6110cd3078 tests/float: Add new lexer test to test parsing of float without prefix.
Since automatically formatting tests with black, we have lost one line of
code coverage.  This adds an explicit test to ensure we are testing the
case that is no longer covered implicitly.
2020-03-30 13:23:05 +11:00
David Lechner 3dc324d3f1 tests: Format all Python code with black, except tests in basics subdir.
This adds the Python files in the tests/ directory to be formatted with
./tools/codeformat.py.  The basics/ subdirectory is excluded for now so we
aren't changing too much at once.

In a few places `# fmt: off`/`# fmt: on` was used where the code had
special formatting for readability or where the test was actually testing
the specific formatting.
2020-03-30 13:21:58 +11:00
David Lechner 488613bca6 tests/micropython/heapalloc_fail_set.py: Remove extra trailing comma.
Unlike tuples, sets do not need trailing comma when there is only one item.
2020-03-30 13:10:24 +11:00
David Lechner 688323307a tests/basics/dict_pop.py: Remove extra comma in call and fix grammar. 2020-03-28 23:41:08 +11:00
Damien George 081d067662 tests/net_inet: Add uasyncio internet tests. 2020-03-26 01:25:45 +11:00
Damien George 38904b8937 tests/multi_net: Add uasyncio test for TCP server and client.
Includes a test where the (non uasyncio) client does a RST on the
connection, as a simple TCP server/client test where both sides are using
uasyncio, and a test for TCP stream close then write.
2020-03-26 01:25:45 +11:00
Damien George 18fa65e474 tests: Make default MICROPYPATH include extmod to find uasyncio. 2020-03-26 01:25:45 +11:00
Damien George 5d09a40df9 tests/run-tests: Skip uasyncio if no async, and skip one test on native. 2020-03-26 01:25:45 +11:00
Damien George c4935f3049 tests/extmod: Add uasyncio tests.
All .exp files are included because they require CPython 3.8 which may not
always be available.
2020-03-26 01:25:45 +11:00
Damien George 19ea30bdd5 tests/run-multitests.py: Print test summary and do exit(1) on failure. 2020-03-18 21:33:40 +11:00
Damien George 8f0778b209 extmod/modlwip: Properly handle non-blocking and timeout on UDP recv.
Fixes UDP non-blocking recv so it returns EAGAIN instead of ETIMEDOUT.
Timeout waiting for incoming data is also improved by replacing 100ms delay
with poll_sockets(), as is done in other parts of this module.

Fixes issue #5759.
2020-03-18 10:51:32 +11:00
Damien George 00267aae0b extmod/modlwip: Fix polling of UDP socket so it doesn't return HUP.
STATE_NEW will return HUP when polled so put active UDP sockets into a new
state which is different to STATE_NEW.

Fixes issue #5758.
2020-03-18 10:49:27 +11:00
Damien George ea1ea909d7 tests/run-tests: Consider all tests as native when emit=native is used.
So that they are skipped when running on a target that doesn't support the
native emitter, eg a nanbox build.
2020-03-11 20:20:18 +11:00
Andrew Leech 86bfabec11 py/modmicropython: Add heap_locked function to test state of heap.
This commit adds micropython.heap_locked() which returns the current
lock-depth of the heap, and can be used by Python code to check if the heap
is locked or not.  This new function is configured via
MICROPY_PY_MICROPYTHON_HEAP_LOCKED and is disabled by default.

This commit also changes the return value of micropython.heap_unlock() so
it returns the current lock-depth as well.
2020-03-11 16:54:16 +11:00
Andrew Leech ed93778e00 py/objstringio: Expose tell() on StringIO and BytesIO objects.
To match file objects.

Fixes issue #5581.
2020-03-11 14:43:03 +11:00
Damien George dd0bc26e65 extmod/modbluetooth: Change scan result's "connectable" to "adv_type".
This commit changes the BLE _IRQ_SCAN_RESULT data from:

    addr_type, addr, connectable, rssi, adv_data

to:

    addr_type, addr, adv_type, rssi, adv_data

This allows _IRQ_SCAN_RESULT to handle all scan result types (not just
connectable and non-connectable passive scans), and to distinguish between
them using adv_type which is an integer taking values 0x00-0x04 per the BT
specification.

This is a breaking change to the API, albeit a very minor one: the existing
connectable value was a boolean and True now becomes 0x00, False becomes
0x02.

Documentation is updated and a test added.

Fixes #5738.
2020-03-11 14:00:44 +11:00
Jeff Epler ef195d6e1b Update tests
* string_pep498_fstring.py: Not compatible with python3.5
 * cmd_parsetree.py: enumerated constants changed
2020-03-09 21:13:33 -05:00
Damien George 4fda7a5b44 tests/multi_bluetooth: Add initial tests for bluetooth BLE. 2020-03-10 02:22:34 +11:00
Damien George 8f44c0dd16 tests/multi_net: Add initial set of multi-instance tests for network. 2020-03-10 02:22:34 +11:00
Damien George df9a949891 tests/run-multitests.py: Add new test runner for multiple Py instances.
This commit adds a test runner and initial test scripts which run multiple
Python/MicroPython instances (eg executables, target boards) in parallel.
This is useful for testing, eg, network and Bluetooth functionality.

Each test file has a set of functions called instanceX(), where X ranges
from 0 up to the maximum number of instances that are needed, N-1.  Then
run-multitests.py will execute this script on N separate instances (eg
micropython executables, or attached boards via pyboard.py) at the same
time, synchronising their start in the right order, possibly passing IP
address (or other address like bluetooth MAC) from the "server" instance to
the "client" instances so they can connect to each other.  It then runs
them to completion, collects the output, and then tests against what
CPython gives (or what's in a provided .py.exp file).

The tests will be run using the standard unix executable for all instances
by default, eg:

    $ ./run-multitests.py multi_net/*.py

Or they can be run with a board and unix executable via:

    $ ./run-multitests.py --instance pyb:/dev/ttyACM0 --instance exec:micropython multi_net/*.py
2020-03-10 02:22:34 +11:00
Jeff Epler 32647cd9b4 lexer: catch concatenation of f'' and '' strings
This turns the "edge case" into a parse-time error.
2020-03-09 09:03:25 -05:00
Josh Klar 40bc05ee1e Address dpgeorge feedback - largely simplifications 2020-03-09 08:16:07 -05:00
Josh Klar 3a7a5ba686 py: Implement partial PEP-498 (f-string) support
This implements (most of) the PEP-498 spec for f-strings, with two
exceptions:

- raw f-strings (`fr` or `rf` prefixes) raise `NotImplementedError`
- one special corner case does not function as specified in the PEP
(more on that in a moment)

This is implemented in the core as a syntax translation, brute-forcing
all f-strings to run through `String.format`. For example, the statement
`x='world'; print(f'hello {x}')` gets translated *at a syntax level*
(injected into the lexer) to `x='world'; print('hello {}'.format(x))`.
While this may lead to weird column results in tracebacks, it seemed
like the fastest, most efficient, and *likely* most RAM-friendly option,
despite being implemented under the hood with a completely separate
`vstr_t`.

Since [string concatenation of adjacent literals is implemented in the
lexer](534b7c368d),
two side effects emerge:

- All strings with at least one f-string portion are concatenated into a
single literal which *must* be run through `String.format()` wholesale,
and:
- Concatenation of a raw string with interpolation characters with an
f-string will cause `IndexError`/`KeyError`, which is both different
from CPython *and* different from the corner case mentioned in the PEP
(which gave an example of the following:)

```python
x = 10
y = 'hi'
assert ('a' 'b' f'{x}' '{c}' f'str<{y:^4}>' 'd' 'e') == 'ab10{c}str< hi >de'
```

The above-linked commit detailed a pretty solid case for leaving string
concatenation in the lexer rather than putting it in the parser, and
undoing that decision would likely be disproportionately costly on
resources for the sake of a probably-low-impact corner case. An
alternative to become complaint with this corner case of the PEP would
be to revert to string concatenation in the parser *only when an
f-string is part of concatenation*, though I've done no investigation on
the difficulty or costs of doing this.

A decent set of tests is included. I've manually tested this on the
`unix` port on Linux and on a Feather M4 Express (`atmel-samd`) and
things seem sane.
2020-03-09 08:16:07 -05:00
Jeff Epler c0bfa11766 Use tests from ulab extmod, instead of our own 2020-03-03 12:46:59 -06:00
Damien George 1993c8cf9a py/builtinevex: Support passing in a bytearray/buffer to eval/exec.
CPython allows this and it's a simple generalisation of the existing code
which just supported str/bytes.

Fixes issue #5704.
2020-02-28 12:45:36 +11:00
Jeff Epler 39cfe32c34 Update ulab from upstream again 2020-02-27 14:14:05 -06:00
Jeff Epler fa3b9eba92 ulab: Incorporate it 2020-02-27 11:03:03 -06:00
Jim Mussared 54db464a1b tests/basics/array1.py: Add equality testing for array. 2020-02-21 14:25:03 +11:00
Jim Mussared 3ccce89b83 py/objarray: Turn on MP_TYPE_FLAG_EQ_CHECKS_OTHER_TYPE for memoryview.
And add corresponding tests.

Fixes #5674 (comparison of memoryview against bytes).
2020-02-21 14:24:07 +11:00
Jim Mussared f8449dd092 extmod/modframebuf: Allow blit source to be a subclass of FrameBuffer. 2020-02-21 13:32:48 +11:00
Damien George 819380c964 tests/basics: Add test for tuple compare with class derived from tuple.
Only the "==" operator was tested by the test suite in for such arguments.
Other comparison operators like "<" take a different path in the code so
need to be tested separately.
2020-02-20 10:48:03 +11:00
Dan Halbert c592bd612a Implement to_bytes(..., signed=True) 2020-02-14 15:12:20 -05:00
Damien George 6a3ca96fe5 tests/basics: Add test for equality between tuple and namedtuple. 2020-02-11 11:06:17 +11:00
Damien George 27465e6b24 tests/basics: Add tests for equality between bool and int/float/complex.
False/True should be implicitly converted to 0/1 when compared with numeric
types.
2020-02-11 11:06:17 +11:00
Yonatan Goldschmidt 0852acfc74 tests/run-tests: Auto-skip extmod/ticks_diff, extmod/time_ms_us tests. 2020-02-11 10:56:49 +11:00
Damien George eaf30c516a tests/unix: Add coverage tests for kbd-intr and scheduler. 2020-02-07 16:08:29 +11:00
Petr Viktorin dbed8f576d tests/basics: Move test for "return" outside function to own file.
Because its behaviour is conditional on MICROPY_CPYTHON_COMPAT.
2020-02-06 00:42:34 +11:00
Damien George 69b415f745 tests: Move CPy diff test to real test now that subclass equality works.
Testing for equality of subclassed strings now works, thanks to commit
3aab54bf43
2020-02-04 18:25:34 +11:00
David Lechner 74106757ac tests/cpydiff: Add os module environ differences. 2020-02-04 17:54:31 +11:00
David Lechner 4a97f7aaf3 tests/cmdline: Add test for MICROPYINSPECT environment variable.
When this variable is set to non-empty string it triggers the REPL after a
command/module/file finishes running.

The Python file without the file extension is because the cmdline: parser
in run-test splits on spaces, so we can't use the -c option since
`import os` can't be written without a space.
2020-02-04 17:54:31 +11:00
David Lechner 7c24f55285 tests/cmdline/repl_inspect: Add new test for -i option.
This adds a new test to verify that the inspect (-i) command line option
works.
2020-02-04 17:53:06 +11:00
Nicko van Someren c96a2f636b tests/basics: Expand test cases for equality of subclasses. 2020-01-30 14:53:07 +11:00
Nicko van Someren 3aab54bf43 py: Support non-boolean results for equality and inequality tests.
This commit implements a more complete replication of CPython's behaviour
for equality and inequality testing of objects.  This addresses the issues
discussed in #5382 and a few other inconsistencies.  Improvements over the
old code include:

- Support for returning non-boolean results from comparisons (as used by
  numpy and others).
- Support for non-reflexive equality tests.
- Preferential use of __ne__ methods and MP_BINARY_OP_NOT_EQUAL binary
  operators for inequality tests, when available.
- Fallback to op2 == op1 or op2 != op1 when op1 does not implement the
  (in)equality operators.

The scheme here makes use of a new flag, MP_TYPE_FLAG_NEEDS_FULL_EQ_TEST,
in the flags word of mp_obj_type_t to indicate if various shortcuts can or
cannot be used when performing equality and inequality tests.  Currently
four built-in classes have the flag set: float and complex are
non-reflexive (since nan != nan) while bytearray and frozenszet instances
can equal other builtin class instances (bytes and set respectively).  The
flag is also set for any new class defined by the user.

This commit also includes a more comprehensive set of tests for the
behaviour of (in)equality operators implemented in special methods.
2020-01-30 14:53:07 +11:00
Jim Mussared 1f4b607116 tests: Add tests for generator throw and yield-from with exc handlers.
This commit adds a generator test for throwing into a nested exception, and
one when using yield-from with a pending exception cleanup.  Both these
tests currently fail on the native emitter, and are simplified versions of
native test failures from uasyncio in #5332.
2020-01-27 13:16:06 +11:00
Yonatan Goldschmidt cb4472df42 tests: Add boolean-as-integer formatting tests for fixed regression.
As suggested by @dpgeorge in #5538.
2020-01-24 10:57:17 +11:00
Yonatan Goldschmidt 35e664d779 tests/unix: Add coverage tests for mp_obj_is_type() and variants. 2020-01-24 10:57:17 +11:00
Damien George 27f41e624c tests/unix: Add coverage test for mp_obj_new_exception_args.
Because it's no longer called anywhere in the code.
2020-01-23 13:37:25 +11:00
Damien George dccace6f3f tests/unix: Add coverage tests for pairheap data structure. 2020-01-22 17:31:18 +11:00
Damien George cfddc6a8c7 tests/extmod: Add basic machine.Timer test. 2020-01-22 17:31:18 +11:00
Thorsten von Eicken 5c5f93c1b8 tests: Make run-tests help and README be more descriptive of behaviour. 2020-01-22 16:35:24 +11:00
Damien George 3448e69c2d tests/unix: Add coverage test for new mp_obj_int_get_uint_checked func. 2020-01-14 23:45:56 +11:00
Yonatan Goldschmidt 853aaa06f2 lib/mp-readline: Add word-based move/delete EMACS key sequences.
This commit adds backward-word, backward-kill-word, forward-word,
forward-kill-word sequences for the REPL, with bindings to Alt+F, Alt+B,
Alt+D and Alt+Backspace respectively.  It is disabled by default and can be
enabled via MICROPY_REPL_EMACS_WORDS_MOVE.

Further enabling MICROPY_REPL_EMACS_EXTRA_WORDS_MOVE adds extra bindings
for these new sequences: Ctrl+Right, Ctrl+Left and Ctrl+W.

The features are enabled on unix micropython-coverage and micropython-dev.
2020-01-12 13:09:27 +11:00
Jim Mussared 977b532c8f unix: Rename unix binaries to micropython-variant (not _variant).
For consistency with mpy-cross, and other unix tools in general.
2020-01-12 10:37:40 +11:00
Roy Hooper 2cb8f7b2df Add test for issue #2465 - tuple subsclass subscript 2020-01-09 20:13:53 -05:00
stijn 54a2584de1 tests/unix: Make unix time test pass on more platforms.
As the mktime documentation for CPython states: "The earliest date for
which it can generate a time is platform-dependent".  In particular on
Windows this depends on the timezone so e.g. for UTC+2 the earliest is 2
hours past midnight January 1970.  So change the reference to the earliest
possible, for UTC+14.
2020-01-06 23:28:42 +11:00
Damien George 4d528bbaa8 tests/cpydiff: Add CPy diff-test for using dict.keys() as a set.
See issue #5493.
2020-01-06 23:26:00 +11:00
David Lechner 1bc9fc8082 tests/run-tests: Handle 'CRASH' return by float.py feature test.
It is possile for `run_feature_check(pyb, args, base_path, 'float.py')` to
return `b'CRASH'`.  This causes an unhandled exception in `int()`.

This commit fixes the problem by first testing for `b'CRASH'` before trying
to convert the return value to an integer.
2020-01-06 22:22:27 +11:00