Commit Graph

2029 Commits

Author SHA1 Message Date
Damien George 5d09a40df9 tests/run-tests: Skip uasyncio if no async, and skip one test on native. 2020-03-26 01:25:45 +11:00
Damien George c4935f3049 tests/extmod: Add uasyncio tests.
All .exp files are included because they require CPython 3.8 which may not
always be available.
2020-03-26 01:25:45 +11:00
Damien George 19ea30bdd5 tests/run-multitests.py: Print test summary and do exit(1) on failure. 2020-03-18 21:33:40 +11:00
Damien George 8f0778b209 extmod/modlwip: Properly handle non-blocking and timeout on UDP recv.
Fixes UDP non-blocking recv so it returns EAGAIN instead of ETIMEDOUT.
Timeout waiting for incoming data is also improved by replacing 100ms delay
with poll_sockets(), as is done in other parts of this module.

Fixes issue #5759.
2020-03-18 10:51:32 +11:00
Damien George 00267aae0b extmod/modlwip: Fix polling of UDP socket so it doesn't return HUP.
STATE_NEW will return HUP when polled so put active UDP sockets into a new
state which is different to STATE_NEW.

Fixes issue #5758.
2020-03-18 10:49:27 +11:00
Damien George ea1ea909d7 tests/run-tests: Consider all tests as native when emit=native is used.
So that they are skipped when running on a target that doesn't support the
native emitter, eg a nanbox build.
2020-03-11 20:20:18 +11:00
Andrew Leech 86bfabec11 py/modmicropython: Add heap_locked function to test state of heap.
This commit adds micropython.heap_locked() which returns the current
lock-depth of the heap, and can be used by Python code to check if the heap
is locked or not.  This new function is configured via
MICROPY_PY_MICROPYTHON_HEAP_LOCKED and is disabled by default.

This commit also changes the return value of micropython.heap_unlock() so
it returns the current lock-depth as well.
2020-03-11 16:54:16 +11:00
Andrew Leech ed93778e00 py/objstringio: Expose tell() on StringIO and BytesIO objects.
To match file objects.

Fixes issue #5581.
2020-03-11 14:43:03 +11:00
Damien George dd0bc26e65 extmod/modbluetooth: Change scan result's "connectable" to "adv_type".
This commit changes the BLE _IRQ_SCAN_RESULT data from:

    addr_type, addr, connectable, rssi, adv_data

to:

    addr_type, addr, adv_type, rssi, adv_data

This allows _IRQ_SCAN_RESULT to handle all scan result types (not just
connectable and non-connectable passive scans), and to distinguish between
them using adv_type which is an integer taking values 0x00-0x04 per the BT
specification.

This is a breaking change to the API, albeit a very minor one: the existing
connectable value was a boolean and True now becomes 0x00, False becomes
0x02.

Documentation is updated and a test added.

Fixes #5738.
2020-03-11 14:00:44 +11:00
Jeff Epler ef195d6e1b Update tests
* string_pep498_fstring.py: Not compatible with python3.5
 * cmd_parsetree.py: enumerated constants changed
2020-03-09 21:13:33 -05:00
Damien George 4fda7a5b44 tests/multi_bluetooth: Add initial tests for bluetooth BLE. 2020-03-10 02:22:34 +11:00
Damien George 8f44c0dd16 tests/multi_net: Add initial set of multi-instance tests for network. 2020-03-10 02:22:34 +11:00
Damien George df9a949891 tests/run-multitests.py: Add new test runner for multiple Py instances.
This commit adds a test runner and initial test scripts which run multiple
Python/MicroPython instances (eg executables, target boards) in parallel.
This is useful for testing, eg, network and Bluetooth functionality.

Each test file has a set of functions called instanceX(), where X ranges
from 0 up to the maximum number of instances that are needed, N-1.  Then
run-multitests.py will execute this script on N separate instances (eg
micropython executables, or attached boards via pyboard.py) at the same
time, synchronising their start in the right order, possibly passing IP
address (or other address like bluetooth MAC) from the "server" instance to
the "client" instances so they can connect to each other.  It then runs
them to completion, collects the output, and then tests against what
CPython gives (or what's in a provided .py.exp file).

The tests will be run using the standard unix executable for all instances
by default, eg:

    $ ./run-multitests.py multi_net/*.py

Or they can be run with a board and unix executable via:

    $ ./run-multitests.py --instance pyb:/dev/ttyACM0 --instance exec:micropython multi_net/*.py
2020-03-10 02:22:34 +11:00
Jeff Epler 32647cd9b4 lexer: catch concatenation of f'' and '' strings
This turns the "edge case" into a parse-time error.
2020-03-09 09:03:25 -05:00
Josh Klar 40bc05ee1e Address dpgeorge feedback - largely simplifications 2020-03-09 08:16:07 -05:00
Josh Klar 3a7a5ba686 py: Implement partial PEP-498 (f-string) support
This implements (most of) the PEP-498 spec for f-strings, with two
exceptions:

- raw f-strings (`fr` or `rf` prefixes) raise `NotImplementedError`
- one special corner case does not function as specified in the PEP
(more on that in a moment)

This is implemented in the core as a syntax translation, brute-forcing
all f-strings to run through `String.format`. For example, the statement
`x='world'; print(f'hello {x}')` gets translated *at a syntax level*
(injected into the lexer) to `x='world'; print('hello {}'.format(x))`.
While this may lead to weird column results in tracebacks, it seemed
like the fastest, most efficient, and *likely* most RAM-friendly option,
despite being implemented under the hood with a completely separate
`vstr_t`.

Since [string concatenation of adjacent literals is implemented in the
lexer](534b7c368d),
two side effects emerge:

- All strings with at least one f-string portion are concatenated into a
single literal which *must* be run through `String.format()` wholesale,
and:
- Concatenation of a raw string with interpolation characters with an
f-string will cause `IndexError`/`KeyError`, which is both different
from CPython *and* different from the corner case mentioned in the PEP
(which gave an example of the following:)

```python
x = 10
y = 'hi'
assert ('a' 'b' f'{x}' '{c}' f'str<{y:^4}>' 'd' 'e') == 'ab10{c}str< hi >de'
```

The above-linked commit detailed a pretty solid case for leaving string
concatenation in the lexer rather than putting it in the parser, and
undoing that decision would likely be disproportionately costly on
resources for the sake of a probably-low-impact corner case. An
alternative to become complaint with this corner case of the PEP would
be to revert to string concatenation in the parser *only when an
f-string is part of concatenation*, though I've done no investigation on
the difficulty or costs of doing this.

A decent set of tests is included. I've manually tested this on the
`unix` port on Linux and on a Feather M4 Express (`atmel-samd`) and
things seem sane.
2020-03-09 08:16:07 -05:00
Jeff Epler c0bfa11766 Use tests from ulab extmod, instead of our own 2020-03-03 12:46:59 -06:00
Damien George 1993c8cf9a py/builtinevex: Support passing in a bytearray/buffer to eval/exec.
CPython allows this and it's a simple generalisation of the existing code
which just supported str/bytes.

Fixes issue #5704.
2020-02-28 12:45:36 +11:00
Jeff Epler 39cfe32c34 Update ulab from upstream again 2020-02-27 14:14:05 -06:00
Jeff Epler fa3b9eba92 ulab: Incorporate it 2020-02-27 11:03:03 -06:00
Jim Mussared 54db464a1b tests/basics/array1.py: Add equality testing for array. 2020-02-21 14:25:03 +11:00
Jim Mussared 3ccce89b83 py/objarray: Turn on MP_TYPE_FLAG_EQ_CHECKS_OTHER_TYPE for memoryview.
And add corresponding tests.

Fixes #5674 (comparison of memoryview against bytes).
2020-02-21 14:24:07 +11:00
Jim Mussared f8449dd092 extmod/modframebuf: Allow blit source to be a subclass of FrameBuffer. 2020-02-21 13:32:48 +11:00
Damien George 819380c964 tests/basics: Add test for tuple compare with class derived from tuple.
Only the "==" operator was tested by the test suite in for such arguments.
Other comparison operators like "<" take a different path in the code so
need to be tested separately.
2020-02-20 10:48:03 +11:00
Dan Halbert c592bd612a Implement to_bytes(..., signed=True) 2020-02-14 15:12:20 -05:00
Damien George 6a3ca96fe5 tests/basics: Add test for equality between tuple and namedtuple. 2020-02-11 11:06:17 +11:00
Damien George 27465e6b24 tests/basics: Add tests for equality between bool and int/float/complex.
False/True should be implicitly converted to 0/1 when compared with numeric
types.
2020-02-11 11:06:17 +11:00
Yonatan Goldschmidt 0852acfc74 tests/run-tests: Auto-skip extmod/ticks_diff, extmod/time_ms_us tests. 2020-02-11 10:56:49 +11:00
Damien George eaf30c516a tests/unix: Add coverage tests for kbd-intr and scheduler. 2020-02-07 16:08:29 +11:00
Petr Viktorin dbed8f576d tests/basics: Move test for "return" outside function to own file.
Because its behaviour is conditional on MICROPY_CPYTHON_COMPAT.
2020-02-06 00:42:34 +11:00
Damien George 69b415f745 tests: Move CPy diff test to real test now that subclass equality works.
Testing for equality of subclassed strings now works, thanks to commit
3aab54bf43
2020-02-04 18:25:34 +11:00
David Lechner 74106757ac tests/cpydiff: Add os module environ differences. 2020-02-04 17:54:31 +11:00
David Lechner 4a97f7aaf3 tests/cmdline: Add test for MICROPYINSPECT environment variable.
When this variable is set to non-empty string it triggers the REPL after a
command/module/file finishes running.

The Python file without the file extension is because the cmdline: parser
in run-test splits on spaces, so we can't use the -c option since
`import os` can't be written without a space.
2020-02-04 17:54:31 +11:00
David Lechner 7c24f55285 tests/cmdline/repl_inspect: Add new test for -i option.
This adds a new test to verify that the inspect (-i) command line option
works.
2020-02-04 17:53:06 +11:00
Nicko van Someren c96a2f636b tests/basics: Expand test cases for equality of subclasses. 2020-01-30 14:53:07 +11:00
Nicko van Someren 3aab54bf43 py: Support non-boolean results for equality and inequality tests.
This commit implements a more complete replication of CPython's behaviour
for equality and inequality testing of objects.  This addresses the issues
discussed in #5382 and a few other inconsistencies.  Improvements over the
old code include:

- Support for returning non-boolean results from comparisons (as used by
  numpy and others).
- Support for non-reflexive equality tests.
- Preferential use of __ne__ methods and MP_BINARY_OP_NOT_EQUAL binary
  operators for inequality tests, when available.
- Fallback to op2 == op1 or op2 != op1 when op1 does not implement the
  (in)equality operators.

The scheme here makes use of a new flag, MP_TYPE_FLAG_NEEDS_FULL_EQ_TEST,
in the flags word of mp_obj_type_t to indicate if various shortcuts can or
cannot be used when performing equality and inequality tests.  Currently
four built-in classes have the flag set: float and complex are
non-reflexive (since nan != nan) while bytearray and frozenszet instances
can equal other builtin class instances (bytes and set respectively).  The
flag is also set for any new class defined by the user.

This commit also includes a more comprehensive set of tests for the
behaviour of (in)equality operators implemented in special methods.
2020-01-30 14:53:07 +11:00
Jim Mussared 1f4b607116 tests: Add tests for generator throw and yield-from with exc handlers.
This commit adds a generator test for throwing into a nested exception, and
one when using yield-from with a pending exception cleanup.  Both these
tests currently fail on the native emitter, and are simplified versions of
native test failures from uasyncio in #5332.
2020-01-27 13:16:06 +11:00
Yonatan Goldschmidt cb4472df42 tests: Add boolean-as-integer formatting tests for fixed regression.
As suggested by @dpgeorge in #5538.
2020-01-24 10:57:17 +11:00
Yonatan Goldschmidt 35e664d779 tests/unix: Add coverage tests for mp_obj_is_type() and variants. 2020-01-24 10:57:17 +11:00
Damien George 27f41e624c tests/unix: Add coverage test for mp_obj_new_exception_args.
Because it's no longer called anywhere in the code.
2020-01-23 13:37:25 +11:00
Damien George dccace6f3f tests/unix: Add coverage tests for pairheap data structure. 2020-01-22 17:31:18 +11:00
Damien George cfddc6a8c7 tests/extmod: Add basic machine.Timer test. 2020-01-22 17:31:18 +11:00
Thorsten von Eicken 5c5f93c1b8 tests: Make run-tests help and README be more descriptive of behaviour. 2020-01-22 16:35:24 +11:00
Damien George 3448e69c2d tests/unix: Add coverage test for new mp_obj_int_get_uint_checked func. 2020-01-14 23:45:56 +11:00
Yonatan Goldschmidt 853aaa06f2 lib/mp-readline: Add word-based move/delete EMACS key sequences.
This commit adds backward-word, backward-kill-word, forward-word,
forward-kill-word sequences for the REPL, with bindings to Alt+F, Alt+B,
Alt+D and Alt+Backspace respectively.  It is disabled by default and can be
enabled via MICROPY_REPL_EMACS_WORDS_MOVE.

Further enabling MICROPY_REPL_EMACS_EXTRA_WORDS_MOVE adds extra bindings
for these new sequences: Ctrl+Right, Ctrl+Left and Ctrl+W.

The features are enabled on unix micropython-coverage and micropython-dev.
2020-01-12 13:09:27 +11:00
Jim Mussared 977b532c8f unix: Rename unix binaries to micropython-variant (not _variant).
For consistency with mpy-cross, and other unix tools in general.
2020-01-12 10:37:40 +11:00
Roy Hooper 2cb8f7b2df Add test for issue #2465 - tuple subsclass subscript 2020-01-09 20:13:53 -05:00
stijn 54a2584de1 tests/unix: Make unix time test pass on more platforms.
As the mktime documentation for CPython states: "The earliest date for
which it can generate a time is platform-dependent".  In particular on
Windows this depends on the timezone so e.g. for UTC+2 the earliest is 2
hours past midnight January 1970.  So change the reference to the earliest
possible, for UTC+14.
2020-01-06 23:28:42 +11:00
Damien George 4d528bbaa8 tests/cpydiff: Add CPy diff-test for using dict.keys() as a set.
See issue #5493.
2020-01-06 23:26:00 +11:00
David Lechner 1bc9fc8082 tests/run-tests: Handle 'CRASH' return by float.py feature test.
It is possile for `run_feature_check(pyb, args, base_path, 'float.py')` to
return `b'CRASH'`.  This causes an unhandled exception in `int()`.

This commit fixes the problem by first testing for `b'CRASH'` before trying
to convert the return value to an integer.
2020-01-06 22:22:27 +11:00
Nicko van Someren 4c93955b7b py/objslice: Add support for indices() method on slice objects.
Instances of the slice class are passed to __getitem__() on objects when
the user indexes them with a slice.  In practice the majority of the time
(other than passing it on untouched) is to work out what the slice means in
the context of an array dimension of a particular length.  Since Python 2.3
there has been a method on the slice class, indices(), that takes a
dimension length and returns the real start, stop and step, accounting for
missing or negative values in the slice spec.  This commit implements such
a indices() method on the slice class.

It is configurable at compile-time via MICROPY_PY_BUILTINS_SLICE_INDICES,
disabled by default, enabled on unix, stm32 and esp32 ports.

This commit also adds new tests for slice indices and for slicing unicode
strings.
2019-12-28 23:55:15 +11:00
Andrew Leech 1b844e908c unix/modtime: Add utime.mktime function, to complement utime.localtime.
This also adds it to the windows port.
2019-12-28 11:11:54 +11:00
Damien George f5eec903fa py/objsingleton: Use mp_generic_unary_op for singleton objects.
So these types more closely match NoneType, eg they can be hashed, like in
CPython.
2019-12-27 12:53:36 +11:00
Damien George de8c04317b tests/micropython: Add test for yield-from while heap is locked. 2019-12-27 12:34:22 +11:00
Damien George 99a04b8060 tests/extmod: Split out VfsFat finaliser tests to separate test file.
It tests independent functionality and may need to be skipped for a given
port.
2019-12-27 12:30:51 +11:00
Damien George 865827ed8e tests/run-tests: Add "--mpy-cross-flags" arg to specify mpy-cross flags. 2019-12-27 12:30:51 +11:00
Yonatan Goldschmidt 42e45bd694 py/objobject: Add object.__delattr__ function.
Similar to object.__setattr__.
2019-12-21 00:14:22 +11:00
Yonatan Goldschmidt 07ccb5588c py/objobject: Add object.__setattr__ function.
Allows assigning attributes on class instances that implement their own
__setattr__.  Both object.__setattr__ and super(A, b).__setattr__ will work
with this commit.
2019-12-21 00:12:08 +11:00
Damien George 39bc430e44 tests/pyb: Adjust UART and Timer tests to work on PYBD_SF6. 2019-12-20 16:42:38 +11:00
Damien George 4b184d1281 tests/pyb: Refactor pyboard tests to work on PYBv1, PYBLITEv1 and PYBD. 2019-12-13 17:27:29 +11:00
Damien George 7280bf40d9 tests/extmod/vfs_lfs_error: Use small ints in seek error test.
So accessing the seek offset (at the C level) doesn't cause an
OverflowError on 32-bit targets.
2019-12-13 15:59:08 +11:00
Damien George 1098d1d630 tests/basics/memoryview_itemsize: Make portable to 32- and 64-bit archs. 2019-12-13 15:59:08 +11:00
Damien George 33b0a7e601 tests/stress/qstr_limit: Tune params to run with stm32 port.
Because MICROPY_ALLOC_PATH_MAX is only 128 for this port.
2019-12-13 15:58:28 +11:00
Damien George 624f4ca39b tests: Add .exp files for basics/parser and import/import_override.
Because CPython 3.8.0 now produces different output:
- basics/parser.py: CPython does not allow '\\\n' as input.
- import/import_override: CPython imports _io.
2019-12-13 14:20:47 +11:00
Damien George ba84453f77 examples/natmod: Add urandom native module example. 2019-12-13 13:33:40 +11:00
Damien George 4eef940edb tests: Add script to run dynamic-native-module tests. 2019-12-12 20:15:28 +11:00
Damien George 111d1ffb64 tests/import: Add test for importing viper code with additional flags. 2019-12-12 20:15:28 +11:00
Damien George 1675b98e74 tests/stress: Add test for maximum length limit of qstrs. 2019-11-26 13:15:00 +11:00
Léa Saviot a7bc4d1a14 py/builtinimport: Raise exception on empty module name.
To prevent a crash returning MP_OBJ_NULL.  A test is added for this case.
2019-11-26 00:28:32 +11:00
Andrew Leech 1e87f11d3f py/objdict: Support ujson.dump() of OrderedDict objects.
Following CPython, OrderedDict are dumped with the syntax of dict.
2019-11-13 13:51:18 +11:00
Damien George 80df377e95 py/modsys: Report .mpy version in sys.implementation.
This commit adds a sys.implementation.mpy entry when the system supports
importing .mpy files.  This entry is a 16-bit integer which encodes two
bytes of information from the header of .mpy files that are supported by
the system being run: the second and third bytes, .mpy version, and flags
and native architecture.  This allows determining the supported .mpy file
dynamically by code, and also for the user to find it out by inspecting
this value.  It's further possible to dynamically detect if the system
supports importing .mpy files by `hasattr(sys.implementation, 'mpy')`.
2019-11-04 16:00:41 +11:00
Jim Mussared 5578182ec9 py/objgenerator: Allow pend_throw to an unstarted generator.
Replace the is_running field with a tri-state variable to indicate
running/not-running/pending-exception.

Update tests to cover the various cases.

This allows cancellation in uasyncio even if the coroutine hasn't been
executed yet.  Fixes #5242
2019-11-04 15:51:16 +11:00
Damien George 43f53a2bbd tests/extmod: Add test for ussl when passing in key/cert params. 2019-10-31 16:38:20 +11:00
Damien George feaa251674 extmod/modlwip: Make socket poll return POLLNVAL in case of bad file. 2019-10-31 12:54:37 +11:00
Damien George 71401d5065 extmod/modlwip: Unconditionally return POLLHUP when polling new socket.
POSIX poll should always return POLLERR and POLLHUP in revents, regardless
of whether they were requested in the input events flags.

See issues #4290 and #5172.
2019-10-31 12:54:37 +11:00
Damien George 9ec73aedb4 stm32/timer: Fix Timer.freq() calc so mult doesn't overflow uint32_t.
Fixes issue #5280.
2019-10-31 12:49:18 +11:00
Damien George 943dd33b5f tests/basics: Split sys.exit test to separate file so it can be skipped. 2019-10-29 22:22:37 +11:00
Damien George 1d51115246 tests: Add feature check for uio module and skip corresponding tests. 2019-10-29 22:22:37 +11:00
Damien George eebffb2b5b tests/basics: Automatically skip tests that use str/bytes modulo-format. 2019-10-29 22:22:37 +11:00
Damien George 52299ed3f0 tests/run-tests: Add misc list of tests that use slice, to skip them. 2019-10-29 22:22:37 +11:00
Damien George b5186c9271 tests/basics: Split out specific slice tests to separate files.
So they can be automatically skipped if slice is not enabled.
2019-10-29 22:22:37 +11:00
Damien George ecb77e40e0 tests: Add feature check for slice and skip corresponding tests. 2019-10-29 22:22:37 +11:00
Damien George 9162a87d4d tests/basics: Use bytes not bytearray when checking user buffer proto.
Using bytes will test the same path for the buffer protocol in
py/objtype.c.
2019-10-29 22:22:37 +11:00
Damien George aeea204e98 tests/basics: Split out specific bytearray tests to separate files.
So they can be automatically skipped if bytearray is not enabled.
2019-10-29 22:22:37 +11:00
Damien George 6e9ba1cf4b tests: Add feature check for bytearray and skip corresponding tests. 2019-10-29 22:22:37 +11:00
Damien George 7a49fc387c tests/basics/builtin_dir.py: Look for "version" in dir(sys).
Because "version" will always be there, but "exit" may not.
2019-10-29 22:22:37 +11:00
Damien George 709136e844 tests/basics: Use str.format instead of % for formatting messages.
Only use % formatting when testing % itself, because only str.format is
guaranteed to be available on any port.
2019-10-29 22:22:37 +11:00
Damien George cfe1c5abf8 extmod/vfs: Rename BP_IOCTL_xxx constants to MP_BLOCKDEV_IOCTL_xxx.
Also rename SEC_COUNT to BLOCK_COUNT and SEC_SIZE to BLOCK_SIZE.
2019-10-29 14:17:29 +11:00
Damien George 7c8fb27f38 tests/extmod: Add test for blockdev with standard and extended protocol. 2019-10-29 14:17:29 +11:00
Damien George 73fddb84e5 tests/extmod: Add littlefs tests. 2019-10-29 14:17:29 +11:00
Damien George 30e25174bb tests: Rename "array" module to "uarray". 2019-10-22 19:16:54 +11:00
clach04 ffd11486d4 tests/cpydiff: Fix typo in types_bytes_keywords.py doc comments. 2019-10-21 23:25:09 +11:00
Josh Lloyd 8f9e2e325a py/objtype: Add type.__bases__ attribute.
Enabled as part of MICROPY_CPYTHON_COMPAT.
2019-10-18 15:20:56 +11:00
Jim Mussared ebf8332104 extmod/re1.5: Support escaping within RE classes.
Fixes issues #3178 and #5220.

Tests are added, including all the cases mentioned in both bugs.
2019-10-18 12:20:32 +11:00
Damien George 23f0691fdd py/persistentcode: Make .mpy more compact with qstr directly in prelude.
Instead of encoding 4 zero bytes as placeholders for the simple_name and
source_file qstrs, and storing the qstrs after the bytecode, store the
qstrs at the location of these 4 bytes.  This saves 4 bytes per bytecode
function stored in a .mpy file (for example lcd160cr.mpy drops by 232
bytes, 4x 58 functions).  And resulting code size is slightly reduced on
ports that use this feature.
2019-10-15 16:56:27 +11:00
Damien George 858e992d2e tests/run-perfbench.py: Skip complex tests if target doesn't enable it. 2019-10-15 16:46:06 +11:00
Jim Mussared f1882636c0 tests/run-perfbench.py: Show error when truth check fails. 2019-10-15 16:38:11 +11:00
Jim Mussared cfd17f4ebe tests/perf_bench: Add bm_fft test.
This is mostly a test of complex number performance.

The FFT implementation is from Project Nayuki and is MIT licensed.
2019-10-15 16:38:11 +11:00
Scott Shawcroft 9435e01f9e
Support __bytes
Fixes #1763
2019-10-14 16:05:17 -07:00
Damien George 27fe84e661 tests/basics: Add test for throw into yield-from with normal return.
This test was found by missing coverage of a branch in py/nativeglue.c.
2019-10-04 23:27:48 +10:00
Damien George 809d89c794 py/runtime: Fix PEP479 behaviour throwing StopIteration into yield from.
Commit 3f6ffe059f implemented PEP479 but did
not catch the case fixed in this commit.  Found by coverage analysis, that
the VM had uncovered code.
2019-10-04 23:27:00 +10:00
Damien George 82c494a97e py/vm: Fix handling of unwind jump out of active finally.
Prior to this commit, when unwinding through an active finally the stack
was not being correctly popped/folded, which resulting in the VM crashing
for complicated unwinding of nested finallys.

This should be fixed with this commit, and more tests for return/break/
continue within a finally have been added to exercise this.
2019-10-04 23:01:29 +10:00
Petr Viktorin 25a9bccdee py/compile: Disallow 'import *' outside module level.
This check follows CPython's behaviour, because 'import *' always populates
the globals with the imported names, not locals.

Since it's safe to do this (doesn't lead to a crash or undefined behaviour)
the check is only enabled for MICROPY_CPYTHON_COMPAT.

Fixes issue #5121.
2019-10-04 16:46:47 +10:00
Damien George 4102320e90 tests/basics: Add test for getting name of func with closed over locals.
Tests correct decoding of the prelude to get the function name.
2019-10-01 12:26:22 +10:00
Damien George c8c0fd4ca3 py: Rework and compress second part of bytecode prelude.
This patch compresses the second part of the bytecode prelude which
contains the source file name, function name, source-line-number mapping
and cell closure information.  This part of the prelude now begins with a
single varible length unsigned integer which encodes 2 numbers, being the
byte-size of the following 2 sections in the header: the "source info
section" and the "closure section".  After decoding this variable unsigned
integer it's possible to skip over one or both of these sections very
easily.

This scheme saves about 2 bytes for most functions compared to the original
format: one in the case that there are no closure cells, and one because
padding was eliminated.
2019-10-01 12:26:22 +10:00
Damien George b5ebfadbd6 py: Compress first part of bytecode prelude.
The start of the bytecode prelude contains 6 numbers telling the amount of
stack needed for the Python values and exceptions, and the signature of the
function.  Prior to this patch these numbers were all encoded one after the
other (2x variable unsigned integers, then 4x bytes), but using so many
bytes is unnecessary.

An entropy analysis of around 150,000 bytecode functions from the CPython
standard library showed that the optimal Shannon coding would need about
7.1 bits on average to encode these 6 numbers, compared to the existing 48
bits.

This patch attempts to get close to this optimal value by packing the 6
numbers into a single, varible-length unsigned integer via bit-wise
interleaving.  The interleaving scheme is chosen to minimise the average
number of bytes needed, and at the same time keep the scheme simple enough
so it can be implemented without too much overhead in code size or speed.
The scheme requires about 10.5 bits on average to store the 6 numbers.

As a result most functions which originally took 6 bytes to encode these 6
numbers now need only 1 byte (in 80% of cases).
2019-10-01 12:26:22 +10:00
Jim Mussared f67fd95f8d unix/coverage: Add coverage tests for ringbuf. 2019-10-01 09:51:02 +10:00
Damien George 095f90f04e tests/micropython: Add test for native generators. 2019-09-26 16:53:47 +10:00
Damien George 5716c5cf65 py/persistentcode: Bump .mpy version to 5.
The bytecode opcodes have changed (there are more, and they have been
reordered).
2019-09-26 16:39:37 +10:00
Josh Lloyd 7d58a197cf py: Rename MP_QSTR_NULL to MP_QSTRnull to avoid intern collisions.
Fixes #5140.
2019-09-26 16:04:56 +10:00
Damien George 02db91a7a3 py: Split RAISE_VARARGS opcode into 3 separate ones.
From the beginning of this project the RAISE_VARARGS opcode was named and
implemented following CPython, where it has an argument (to the opcode)
counting how many args the raise takes:

    raise # 0 args (re-raise previous exception)
    raise exc # 1 arg
    raise exc from exc2 # 2 args (chained raise)

In the bytecode this operation therefore takes 2 bytes, one for
RAISE_VARARGS and one for the number of args.

This patch splits this opcode into 3, where each is now a single byte.
This reduces bytecode size by 1 byte for each use of raise.  Every byte
counts!  It also has the benefit of reducing code size (on all ports except
nanbox).
2019-09-26 15:39:50 +10:00
Damien George 67fdfebe64 tests: Update tests for changes to opcode ordering. 2019-09-26 15:27:11 +10:00
Damien George fe4e1fe4b9 tests/basics: Add test for matmul operator.
This is a Python 3.5 feature so the .exp file is needed.
2019-09-26 15:15:34 +10:00
Damien George 2069c563f9 py: Add support for matmul operator @ as per PEP 465.
To make progress towards MicroPython supporting Python 3.5, adding the
matmul operator is important because it's a really "low level" part of the
language, being a new token and modifications to the grammar.

It doesn't make sense to make it configurable because 1) it would make the
grammar and lexer complicated/messy; 2) no other operators are
configurable; 3) it's not a feature that can be "dynamically plugged in"
via an import.

And matmul can be useful as a general purpose user-defined operator, it
doesn't have to be just for numpy use.

Based on work done by Jim Mussared.
2019-09-26 15:12:39 +10:00
Damien George b29fae0c56 py/bc: Fix size calculation of UNWIND_JUMP opcode in mp_opcode_format.
Prior to this patch mp_opcode_format would calculate the incorrect size of
the MP_BC_UNWIND_JUMP opcode, missing the additional byte.  But, because
opcodes below 0x10 are unused and treated as bytes in the .mpy load/save
and freezing code, this bug did not show any symptoms, since nested unwind
jumps would rarely (if ever) reach a depth of 16 (so the extra byte of this
opcode would be between 0x01 and 0x0f and be correctly loaded/saved/frozen
simply as an undefined opcode).

This patch fixes this bug by correctly accounting for the additional byte.
        .
2019-09-02 13:30:16 +10:00
Damien George 24c3e9b283 py/modstruct: Fix struct.pack_into with unaligned offset of native type.
Following the same fix for unpack.
2019-09-02 13:14:16 +10:00
Tom McDermott 1022f9cc35 py/modstruct: Fix struct.unpack with unaligned offset of native type.
With this patch alignment is done relative to the start of the buffer that
is being unpacked, not the raw pointer value, as per CPython.

Fixes issue #3314.
2019-09-02 13:10:55 +10:00
Milan Rossa 498e35219e tests: Add tests for sys.settrace feature. 2019-08-30 16:48:22 +10:00
Damien George b3152b2de7 tests: Split out test for optimisation level and line-no printing. 2019-08-28 12:47:58 +10:00
Damien George 08c1fe5569 py/vm: Don't add traceback info for exceptions that are re-raised.
With this patch exceptions that are re-raised have improved tracebacks
(less confusing, match CPython), and it makes re-raise slightly more
efficient (in time and RAM) because they no longer need to add a traceback.
Also general VM performance is not measurably affected.

Partially fixes issue #2928.
2019-08-28 12:31:53 +10:00
Damien George 16f6169c88 py/vm: Don't add traceback info for exc's propagated through a finally.
With this patch exception tracebacks that go through a finally are improved
(less confusing, match CPython), and it makes finally's slightly more
efficient (in time and RAM) because they no longer need to add a traceback.

Partially fixes issue #2928.
2019-08-28 12:31:49 +10:00
Damien George 2eb88f5df7 tests/extmod: Split json.loads of bytes/bytearray into separate test.
Because this functionality was introduced in Python 3.6.
2019-08-22 15:45:13 +10:00
Damien George 2dfa69efbb extmod/modujson: Support passing bytes/bytearray to json.loads.
CPython allows this, and it can be useful to reduce the number of memory
allocations.

Fixes issue #5031.
2019-08-22 15:32:26 +10:00
Jim Mussared 0bd1eb80ff qemu-arm: Add testing of frozen native modules.
- Split 'qemu-arm' from 'unix' for generating tests.
- Add frozen module to the qemu-arm test build.
- Add test that reproduces the requirement to half-word align native
  function data.
2019-08-20 15:14:08 +10:00
Milan Rossa ae6fe8b43c py/compile: Improve the line numbering precision for comprehensions.
The line number for comprehensions is now always reported as the correct
global location in the script, instead of just "line 1".
2019-08-19 23:50:30 +10:00
Damien George 7d851a27f1 extmod/modure: Make regex dump-code debugging feature optional.
Enabled via MICROPY_PY_URE_DEBUG, disabled by default (but enabled on unix
coverage build).  This is a rarely used feature that costs a lot of code
(500-800 bytes flash).  Debugging of regular expressions can be done
offline with other tools.
2019-08-19 16:43:00 +10:00
stijn af5c998f37 py/modmath: Implement math.isclose() for non-complex numbers.
As per PEP 485, this function appeared in for Python 3.5.  Configured via
MICROPY_PY_MATH_ISCLOSE which is disabled by default, but enabled for the
ports which already have MICROPY_PY_MATH_SPECIAL_FUNCTIONS enabled.
2019-08-17 23:23:17 +10:00
Damien George acfbb9febd py/objarray: Fix amount of free space in array when doing slice assign.
Prior to this patch the amount of free space in an array (including
bytearray) was not being maintained correctly for the case of slice
assignment which changed the size of the array.  Under certain cases (as
encoded in the new test) it was possible that the array could grow beyond
its allocated memory block and corrupt the heap.

Fixes issue #4127.
2019-08-15 23:02:04 +10:00
Damien George 64abc1f47a tests/unix: Update extra_coverage expected output with new atexit func. 2019-08-15 18:56:01 +10:00
Milan Rossa 28cb15d131 tests/misc/sys_atexit: Add test for new sys.atexit feature. 2019-08-15 17:31:04 +10:00
Damien George cd35dd9d9a py: Allow to pass in read-only buffers to viper and inline-asm funcs.
Fixes #4936.
2019-08-06 15:58:23 +10:00
Damien George 48f43b77aa tests: Add tests for overriding builtins.__import__. 2019-07-31 22:37:44 +10:00
Paul m. p. P a8e3201b37 py/builtinimport: Populate __file__ when importing frozen or mpy files.
Note that bytecode already includes the source filename as a qstr so there
is no additional memory used by the interning operation here.
2019-07-31 17:00:11 +10:00
Eric Poulsen 01054f2092 py/objdict: Quote non-string types when used as keys in JSON output.
JSON requires that keys of objects be strings.  CPython will therefore
automatically quote simple types (NoneType, bool, int, float) when they are
used directly as keys in JSON output.  To prevent subtle bugs and emit
compliant JSON, MicroPython should at least test for such keys so they
aren't silently let through.  Then doing the actual quoting is a similar
cost to raising an exception, so that's what is implemented by this patch.

Fixes issue #4790.
2019-07-30 16:34:27 +10:00
Damien George 3967dd68e8 tests/run-perfbench.py: Add --emit option to select emitter for tests. 2019-07-19 14:07:41 +10:00
Damien George 3e55830066 tests/stress/recursive_iternext.py: Increase large depth to 5000.
So it fails correctly on Linux with clang.
2019-07-17 15:52:41 +10:00
Damien George 73fccf5967 tests/perf_bench: Add some viper performance benchmarks.
To test raw viper function call overhead: function entry, exit and
conversion of arguments to/from objects.
2019-06-28 16:30:01 +10:00
Damien George 73c269414f tests/perf_bench: Add some miscellaneous performance benchmarks.
misc_aes.py and misc_mandel.py are adapted from sources in this repository.
misc_pystone.py is the standard Python pystone test.  misc_raytrace.py is
written from scratch.
2019-06-28 16:29:23 +10:00
Damien George 127714c3af tests/perf_bench: Add some benchmarks from python-performance.
From https://github.com/python/pyperformance commit
6690642ddeda46fc5ee6e97c3ef4b2f292348ab8
2019-06-28 16:29:23 +10:00
Damien George e92c9aa9c9 tests: Add performance benchmarking test-suite framework.
This benchmarking test suite is intended to be run on any MicroPython
target.  As such all tests are parameterised with N and M: N is the
approximate CPU frequency (in MHz) of the target and M is the approximate
amount of heap memory (in kbytes) available on the target.  When running
the benchmark suite these parameters must be specified and then each test
is tuned to run on that target in a reasonable time (<1 second).

The test scripts are not standalone: they require adding some extra code at
the end to run the test with the appropriate parameters.  This is done
automatically by the run-perfbench.py script, in such a way that imports
are minimised (so the tests can be run on targets without filesystem
support).

To interface with the benchmarking framework, each test provides a
bm_params dict and a bm_setup function, with the later taking a set of
parameters (chosen based on N, M) and returning a pair of functions, one to
run the test and one to get the results.

When running the test the number of microseconds taken by the test are
recorded.  Then this is converted into a benchmark score by inverting it
(so higher number is faster) and normalising it with an appropriate factor
(based roughly on the amount of work done by the test, eg number of
iterations).

Test outputs are also compared against a "truth" value, computed by running
the test with CPython.  This provides a basic way of making sure the test
actually ran correctly.

Each test is run multiple times and the results averaged and standard
deviation computed.  This is output as a summary of the test.

To make comparisons of performance across different runs the
run-perfbench.py script also includes a diff mode that reads in the output
of two previous runs and computes the difference in performance.  Reports
are given as a percentage change in performance with a combined standard
deviation to give an indication if the noise in the benchmarking is less
than the thing that is being measured.

Example invocations for PC, pyboard and esp8266 targets respectively:

    $ ./run-perfbench.py 1000 1000
    $ ./run-perfbench.py --pyboard 100 100
    $ ./run-perfbench.py --pyboard --device /dev/ttyUSB0 50 25
2019-06-28 16:29:23 +10:00
Damien George d86fb670e6 tests: Rename "bench" tests to "internal_bench" and run-internalbench.py
To emphasise these benchmark tests compare the internal performance of
features amongst themselves, rather than absolute performance testing.
2019-06-28 16:28:59 +10:00
Scott Shawcroft 6d2072f9cb
Fix tests 2019-06-26 11:02:18 -07:00
stijn fb54736bdb py/objarray: Add decode method to bytearray.
Reuse the implementation for bytes since it works the same way regardless
of the underlying type.  This method gets added for CPython compatibility
of bytearray, but to keep the code simple and small array.array now also
has a working decode method, which is non-standard but doesn't hurt.
2019-05-21 14:24:04 +10:00
Damien George a474ddf959 tests/basics: Add coverage tests for memoryview attributes. 2019-05-14 17:22:49 +10:00
stijn 90fae9172a py/objarray: Add support for memoryview.itemsize attribute.
This allows figuring out the number of bytes in the memoryview object as
len(memview) * memview.itemsize.

The feature is enabled via MICROPY_PY_BUILTINS_MEMORYVIEW_ITEMSIZE and is
disabled by default.
2019-05-14 17:15:17 +10:00
Damien George 38cb95710a tests/pyb: Update UART expected output now that default timeout is 0.
Follow up to commit 34942d0a72
2019-05-14 14:49:18 +10:00
Damien George 7c5cf59f8b extmod/modujson: Handle parsing of floats with + in the exponent.
Fixes issue #4780.
2019-05-14 14:45:54 +10:00
Dan Halbert 8664a6574b use approx of original @godlygeek code for smallints; add tests 2019-05-12 11:17:29 -04:00
Matt Wozniski e041df73bb Add tests for overflows converting ints to bytes 2019-05-09 03:22:24 -04:00
Damien George dac9d47671 py/objgenerator: Fix handling of None passed as 2nd arg to throw().
Fixes issue #4527.
2019-05-09 13:40:28 +10:00
Kathryn Lingel eb3cb4d99b Fix tests and clarify first character access 2019-05-07 09:30:55 -07:00
Kathryn Lingel 33b5cff8db Correct output expectation 2019-05-06 15:13:28 -07:00
Kathryn Lingel 77f177be9c Update tests for new autocomplete behavior 2019-05-06 14:19:07 -07:00
Yonatan Goldschmidt ef9843653b extmod/moducryptolib: Add AES-CTR support.
Selectable at compile time via MICROPY_PY_UCRYPTOLIB_CTR.  Disabled by
default.
2019-05-06 18:09:48 +10:00
Damien George 906fb89fd7 unix/coverage: Add test for printing literal % character. 2019-05-03 23:21:28 +10:00
Damien George c2bb451908 tests/basics/sys1.py: Add test for calling sys.exit() without any args. 2019-05-03 23:21:08 +10:00
Damien George 5ea38e4d74 py/native: Improve support for bool type in viper functions.
Variables with type bool now act more like an int, and there is proper
casting to/from Python objects.
2019-05-03 23:18:30 +10:00
Paul Sokolovsky 7b5400134b tests/ussl_basic: Disable setblocking() calls.
Now that setblocking() is implemented in modussl_axtls, it calls into the
underlying stream object, and io.BytesIO doesn't have setblocking().
2019-04-30 17:27:28 +10:00
Paul Sokolovsky c76445315f extmod/modussl_axtls: Add non-blocking mode support.
It consists of:

1. "do_handhake" param (default True) to wrap_socket(). If it's False,
handshake won't be performed by wrap_socket(), as it would be done in
blocking way normally. Instead, SSL socket can be set to non-blocking mode,
and handshake would be performed before the first read/write request (by
just returning EAGAIN to these requests, while instead reading/writing/
processing handshake over the connection). Unfortunately, axTLS doesn't
really support non-blocking handshake correctly. So, while framework for
this is implemented on MicroPython's module side, in case of axTLS, it
won't work reliably.

2. Implementation of .setblocking() method. It must be called on SSL socket
for blocking vs non-blocking operation to be handled correctly (for
example, it's not enough to wrap non-blocking socket with wrap_socket()
call - resulting SSL socket won't be itself non-blocking).  Note that
.setblocking() propagates call to the underlying socket object, as
expected.
2019-04-30 17:26:37 +10:00
Damien George ca39ea7cef tests: Skip tests needing machine module if (u)machine doesn't exist. 2019-04-28 22:12:17 +10:00
Damien George eb1f81b209 tests/micropython: Add some tests for failed heap allocation.
This adds tests for some locations in the code where a memory allocation
should raise an exception.
2019-04-18 14:34:12 +10:00
stijn d89ce2ed1d tests/run-tests: Ignore exception in process kill when ending repl test.
When running Linux on WSL, Popen.kill() can raise a ProcessLookupError if
the process does not exist anymore, which can happen here since the
previous statement already tries to close the process by sending Ctrl-D to
the running repl.  This doesn't seem to be a problem on other OSes, so just
swallow the exception silently since it indicates the process has been
closed already, which after all is what we want.
2019-04-04 15:24:29 +11:00
Damien George 968b688055 tests/extmod: Add test for FAT filesystem on a very large block device. 2019-03-27 10:22:38 +11:00
Andrew Leech 8977c7eb58 py/scheduler: Convert micropythyon.schedule() to a circular buffer.
This means the schedule operates on a first-in, first-executed manner
rather than the current last-in, first executed.
2019-03-26 16:35:42 +11:00
Damien George 1e23a29c8a tests/import: Add test for importing x64 native code. 2019-03-08 17:20:17 +11:00
Damien George 69955238a2 tests/run-tests: Support running native tests via mpy. 2019-03-08 16:51:09 +11:00
Damien George 5996eeb48f py/persistentcode: Add a qstr window to save mpy files more efficiently.
This is an implementation of a sliding qstr window used to reduce the
number of qstrs stored in a .mpy file.  The window size is configured to 32
entries which takes a fixed 64 bytes (16-bits each) on the C stack when
loading/saving a .mpy file.  It allows to remember the most recent 32 qstrs
so they don't need to be stored again in the .mpy file.  The qstr window
uses a simple least-recently-used mechanism to discard the least recently
used qstr when the window overflows (similar to dictionary compression).
This scheme only needs a single pass to save/load the .mpy file.

Reduces mpy file size by about 25% with a window size of 32.
2019-03-05 16:25:07 +11:00
Damien George 5a2599d962 py: Replace POP_BLOCK and POP_EXCEPT opcodes with POP_EXCEPT_JUMP.
POP_BLOCK and POP_EXCEPT are now the same, and are always followed by a
JUMP.  So this optimisation reduces code size, and RAM usage of bytecode by
two bytes for each try-except handler.
2019-03-05 16:09:58 +11:00
Damien George e1fb03f3e2 py: Fix VM crash with unwinding jump out of a finally block.
This patch fixes a bug in the VM when breaking within a try-finally.  The
bug has to do with executing a break within the finally block of a
try-finally statement.  For example:

    def f():
        for x in (1,):
            print('a', x)
            try:
                raise Exception
            finally:
                print(1)
                break
            print('b', x)
    f()

Currently in uPy the above code will print:

    a 1
    1
    1
    segmentation fault (core dumped)  micropython

Not only is there a seg fault, but the "1" in the finally block is printed
twice.  This is because when the VM executes a finally block it doesn't
really know if that block was executed due to a fall-through of the try (no
exception raised), or because an exception is active.  In particular, for
nested finallys the VM has no idea which of the nested ones have active
exceptions and which are just fall-throughs.  So when a break (or continue)
is executed it tries to unwind all of the finallys, when in fact only some
may be active.

It's questionable whether break (or return or continue) should be allowed
within a finally block, because they implicitly swallow any active
exception, but nevertheless it's allowed by CPython (although almost never
used in the standard library).  And uPy should at least not crash in such a
case.

The solution here relies on the fact that exception and finally handlers
always appear in the bytecode after the try body.

Note: there was a similar bug with a return in a finally block, but that
was previously fixed in b735208403
2019-03-05 16:05:05 +11:00
Damien George 12ce9f2689 py/compile: Fix handling of unwinding BaseException in async with.
All exceptions that unwind through the async-with must be caught and
BaseException is the top-level class, which includes Exception and others.

Fixes issue #4552.
2019-02-26 23:52:10 +11:00
Damien George be41d6d6f9 tests/basics: Add tests for try-except-else and try-except-else-finally. 2019-02-21 16:22:41 +11:00
Radomir Dopieralski 20a787adb4 extmod/ure: Handle some escape sequences.
Fix MicroPython #3176

Handle escape sequences inside regular expressions. This adds
handling for \a, \b, \f, \n, \r, \v and \\.
2019-02-14 15:42:25 +01:00
Damien George b9dc23c070 extmod/modure: Add ure.sub() function and method, and tests.
This feature is controlled at compile time by MICROPY_PY_URE_SUB, disabled
by default.

Thanks to @dmazzella for the original patch for this feature; see #3770.
2019-02-14 15:42:22 +01:00
Damien George cbeac094ef extmod/modure: Add match.span(), start() and end() methods, and tests.
This feature is controlled at compile time by
MICROPY_PY_URE_MATCH_SPAN_START_END, disabled by default.

Thanks to @dmazzella for the original patch for this feature; see #3770.
2019-02-14 15:42:21 +01:00
Damien George a24fabbb6f extmod/modure: Add match.groups() method, and tests.
This feature is controlled at compile time by MICROPY_PY_URE_MATCH_GROUPS,
disabled by default.

Thanks to @dmazzella for the original patch for this feature; see #3770.
2019-02-14 15:42:19 +01:00
Yonatan Goldschmidt bc4f8b438b extmod/moduwebsocket: Refactor `websocket` to `uwebsocket`.
As mentioned in #4450, `websocket` was experimental with a single intended
user, `webrepl`. Therefore, we'll make this change without a weak
link `websocket` -> `uwebsocket`.
2019-02-14 00:35:45 +11:00
stijn 42863830be py: Add optional support for 2-argument version of built-in next().
Configurable via MICROPY_PY_BUILTINS_NEXT2, disabled by default.
2019-01-27 13:01:28 +11:00
Paul Sokolovsky d4d4bc5827 tests/basics/special_methods2: Typo fix in comment. 2018-12-13 01:29:01 +11:00
Damien George 074597f172 tests/extmod/uctypes_error: Add test for unsupported unary op. 2018-12-10 14:29:41 +11:00
Paul Sokolovsky 0de6815ec1 tests/extmod/uctypes_ptr_le: Test int() operation on a pointer field. 2018-12-10 14:25:06 +11:00
Paul Sokolovsky d690c2e148 tests/basics/special_methods: Add testcases for __int__. 2018-12-07 17:28:04 +11:00
Damien George 113f00a9ab py/objboundmeth: Support loading generic attrs from the method.
Instead of assuming that the method is a bytecode object, and only
supporting load of __name__, make the operation generic by delegating the
load to the method object itself.  Saves a bit of code size and fixes the
case of attempting to load __name__ on a native method, see issue #4028.
2018-12-06 18:02:41 +11:00
Damien George 8007d0bd16 stm32/uart: Add rxbuf keyword arg to UART constructor and init method.
As per the machine.UART documentation, this is used to set the length of
the RX buffer.  The legacy read_buf_len argument is retained for backwards
compatibility, with rxbuf overriding it if provided.
2018-12-05 13:24:11 +11:00
Damien George 9262f54138 stm32/uart: Always show the flow setting when printing a UART object.
Also change the order of printing of flow so it is after stop (so bits,
parity, stop are one after the other), and reduce code size by using
mp_print_str instead of mp_printf where possible.

See issue #1981.
2018-12-04 19:16:16 +11:00
Paul Sokolovsky 5c34c2ff7f tests/io: Update tests to use uos.remove() instead of uos.unlink().
After Unix port switches from one to another, to be consistent with
baremetal ports.
2018-11-26 23:27:28 +11:00
Damien George 7c85c7c210 py/unicode: Fix check for valid utf8 being stricter about contn chars. 2018-11-26 16:13:08 +11:00
Paul Sokolovsky d94aa577a6 tests/import_long_dyn: Test for "import *" of a long dynamic name.
Such names aren't stored as qstr in module dict, and there was a bug in
"import *" handling which assumed any name in a module dict is a qstr.
2018-11-01 13:33:16 +11:00
stijn 06643a0df4 tests/extmod: Skip uselect test when CPython doesn't have poll().
CPython does not have an implementation of select.poll() on some
operating systems (Windows, OSX depending on version) so skip the
test in those cases instead of failing it.
2018-10-30 14:49:23 +11:00
Damien George 9201f46cc8 py/compile: Fix case of eager implicit conversion of local to nonlocal.
This ensures that implicit variables are only converted to implicit
closed-over variables (nonlocals) at the very end of the function scope.
If variables are closed-over when first used (read from, as was done prior
to this commit) then this can be incorrect because the variable may be
assigned to later on in the function which means they are just a plain
local, not closed over.

Fixes issue #4272.
2018-10-28 00:33:08 +11:00
Damien George c2074e7b66 tests/cmdline/cmd_showbc.py: Fix test to explicitly declare nonlocal.
The way it was written previously the variable x was not an implicit
nonlocal, it was just a normal local (but the compiler has a bug which
incorrectly makes it a nonlocal).
2018-10-27 23:57:14 +11:00
Damien George 27ca9ab8b2 tests/import: Add .exp file for module_getattr.py to not require Py 3.7. 2018-10-23 11:56:58 +11:00
Paul Sokolovsky c638d86660 tests/extmod/uctypes_sizeof_layout: Test for sizeof of different layout.
On almost all realistic platforms, native layout should be larger (or
equal) than packed layout.
2018-10-23 11:33:35 +11:00
Paul m. p. P 454cca6016 py/objmodule: Implement PEP 562's __getattr__ for modules.
Configurable via MICROPY_MODULE_GETATTR, disabled by default.  Among other
things __getattr__ for modules can help to build lazy loading / code
unloading at runtime.
2018-10-23 11:22:50 +11:00
Paul Sokolovsky a527313382 tests: Make bytes/str.count() tests skippable. 2018-10-22 22:50:28 +11:00
Damien George a07e56cbd8 tests/basics/class_getattr: Remove invalid test for __getattribute__.
Part of this test was trying to test some functionality of __getattribute__
but this method name was misspelt so it wasn't doing anything useful.
Fixing the typo in this name makes the test fail because MicroPython
doesn't support user defined __getattribute__ methods.  So this part of the
test is removed.  The remaining tests are modified slightly to make it
clearer what they are testing.
2018-10-18 12:28:09 +11:00
Damien George 7eb29c2000 py/objtype: Remove comment about catching exc from user __getattr__.
Any exception raised in a user __getattr__ should be propagated out.  A
test is added to verify these semantics.
2018-10-18 12:15:16 +11:00
Paul Sokolovsky 7059b4af6d tests/uctypes_sizeof_od: Test for using OrderedDict as struct descriptor
Just a copy of uctypes_sizeof.py with minimal changes.
2018-10-13 16:08:25 +11:00
Paul Sokolovsky 6ef783527d tests/uselect_poll_basic: Add basic test for uselect.poll invariants.
This test doesn't check the actual I/O behavior, just "static" invariants
like behavior on duplicate calls or calls when I/O object is not registered
with poller.
2018-10-05 16:57:40 +10:00
Paul Sokolovsky cb66b75692 tests/unix/ffi_float: Skip if strtof() is not available.
As the case for e.g. Android's Bionic Libc.
2018-10-05 16:49:32 +10:00
Damien George 5cc9517fc5 tests/run-tests: Enabled native tests that pass now that yield works. 2018-10-01 13:31:11 +10:00
Damien George dd288904db py/objtype: Support full object model for get/set/delitem special meths.
This makes these special methods have the same calling behaviour as other
methods in a class instance (mp_convert_member_lookup() is already called
by mp_obj_class_lookup()).
2018-09-28 23:22:34 +10:00
Damien George 0c9d452370 py/vm: Fix case of throwing GeneratorExit type into yield-from.
mp_make_raise_obj must be used to convert a possible exception type to an
instance object, otherwise the VM may raise a non-exception object.

An existing test is adjusted to test this case, with the original test
already moved to generator_throw.py.
2018-09-28 11:39:35 +10:00
Damien George e6078dfed2 tests/basics: Split out gen throw tests from yield-from-throw tests. 2018-09-28 11:35:31 +10:00
Damien George ac81cee3fc tests/micropython: Test loading const objs in native and viper funcs. 2018-09-27 23:39:08 +10:00
Damien George b3eadf3f3d py/objfloat: Fix abs(-0.0) so it returns 0.0.
Nan and inf (signed and unsigned) are also handled correctly by using
signbit (they were also handled correctly with "val<0", but that didn't
handle -0.0 correctly).  A test case is added for this behaviour.
2018-09-27 15:21:25 +10:00
Damien George fc1bb51af5 py/objgenerator: Remove TODO about returning gen being called again.
The code implements correct behaviour, as tested by the new test case added
in this commit.
2018-09-27 15:18:24 +10:00
Paul Sokolovsky 8181ec04a4 tests/cpydiff: Add case for difference in behaviour of bytes.format(). 2018-09-26 15:31:10 +10:00
Christopher Swenson 8c656754aa py/modmath: Add math.factorial, optimised and non-opt implementations.
This commit adds the math.factorial function in two variants:
- squared difference, which is faster than the naive version, relatively
  compact, and non-recursive;
- a mildly optimised recursive version, faster than the above one.

There are some more optimisations that could be done, but they tend to take
more code, and more storage space.  The recursive version seems like a
sensible compromise.

The new function is disabled by default, and uses the non-optimised version
by default if it is enabled.  The options are MICROPY_PY_MATH_FACTORIAL
and MICROPY_OPT_MATH_FACTORIAL.
2018-09-26 15:03:04 +10:00
Damien George 9849209ad8 tests/float/float_parse.py: Add tests for accuracy of small decimals. 2018-09-20 22:26:53 +10:00
Damien George 3f6ffe059f py/objgenerator: Implement PEP479, StopIteration convs to RuntimeError.
This commit implements PEP479 which disallows raising StopIteration inside
a generator to signal that it should be finished.  Instead, the generator
should simply return when it is complete.

See https://www.python.org/dev/peps/pep-0479/ for details.
2018-09-20 15:36:59 +10:00
Damien George b01f66c5f1 py: Shorten error messages by using contractions and some rewording. 2018-09-20 14:33:10 +10:00
Damien George 93d71c5436 py/emitnative: Make viper funcs run with their correct globals context.
Viper functions will now capture the globals at the point they were defined
and use these globals when executing.
2018-09-15 22:39:27 +10:00
Damien George a676b5acf6 py/emitnative: Support arbitrary number of arguments to viper functions. 2018-09-15 22:39:27 +10:00
Damien George 9f2067288a py/compile: Factor code that compiles viper type annotations. 2018-09-15 13:44:39 +10:00
Damien George 4f3d9429b5 py: Fix native functions so they run with their correct globals context.
Prior to this commit a function compiled with the native decorator
@micropython.native would not work correctly when accessing global
variables, because the globals dict was not being set upon function entry.

This commit fixes this problem by, upon function entry, setting as the
current globals dict the globals dict context the function was defined
within, as per normal Python semantics, and as bytecode does.  Upon
function exit the original globals dict is restored.

In order to restore the globals dict when an exception is raised the native
function must guard its internals with an nlr_push/nlr_pop pair.  Because
this push/pop is relatively expensive, in both C stack usage for the
nlr_buf_t and CPU execution time, the implementation here optimises things
as much as possible.  First, the compiler keeps track of whether a function
even needs to access global variables.  Using this information the native
emitter then generates three different kinds of code:

1. no globals used, no exception handlers: no nlr handling code and no
   setting of the globals dict.

2. globals used, no exception handlers: an nlr_buf_t is allocated on the
   C stack but it is not used if the globals dict is unchanged, saving
   execution time because nlr_push/nlr_pop don't need to run.

3. function has exception handlers, may use globals: an nlr_buf_t is
   allocated and nlr_push/nlr_pop are always called.

In the end, native functions that don't access globals and don't have
exception handlers will run more efficiently than those that do.

Fixes issue #1573.
2018-09-13 22:47:20 +10:00
Damien George f2de9d60f7 py/emitnative: Fix try-finally in outer scope, so finally is cancelled. 2018-09-11 15:33:25 +10:00
Paul Sokolovsky 674e069ba9 py/objarray: bytearray: Allow 2nd/3rd arg to constructor.
If bytearray is constructed from str, a second argument of encoding is
required (in CPython), and third arg of Unicode error handling is allowed,
e.g.:

bytearray("str", "utf-8", "strict")

This is similar to bytes:

bytes("str", "utf-8", "strict")

This patch just allows to pass 2nd/3rd arguments to bytearray, but
doesn't try to validate them to not impact code size. (This is also
similar to how bytes constructor is handled, though it does a bit
more validation, e.g. check that in case of str arg, encoding argument
is passed.)
2018-09-11 15:10:10 +10:00
Paul Sokolovsky b6ebb4f04e tests/extmod/uhashlib_md5: Add coverage tests for MD5 algorithm.
Based on tests/extmod/uhashlib_sha1.
2018-09-11 14:52:00 +10:00
Damien George e814db592d tests: Remove pyboard.py symlink and instead import from ../tools.
To eliminate the need for symlinks which don't work on systems like
Windows.
2018-09-05 15:36:33 +10:00
Damien George 5630f277bd tests/float: Test -inf and some larger values for special math funcs. 2018-09-04 17:03:37 +10:00
Damien George a111ca25ea tests/float/cmath_fun.py: Fix truncation of small real part of complex. 2018-09-04 17:02:36 +10:00
Damien George 4970e9bc8c tests/basics: Add test cases for context manager raising in enter/exit. 2018-09-04 14:37:30 +10:00
Damien George b14c705c18 tests/basics: Add more tests for return within try-finally. 2018-09-04 14:37:07 +10:00
Damien George 938daa4ff9 tests/run-tests: Enable native tests for unwinding jumps. 2018-09-04 14:33:43 +10:00
Damien George 3cd2c281d7 py/emitnative: Cancel caught exception once handled to prevent reraise.
The native emitter keeps the current exception in a slot in its C stack
(instead of on its Python value stack), so when it catches an exception it
must explicitly clear that slot so the same exception is not reraised later
on.
2018-09-03 17:41:02 +10:00
Damien George b735208403 py/vm: Fix handling of finally-return with complex nested finallys.
Back in 8047340d75 basic support was added in
the VM to handle return statements within a finally block.  But it didn't
cover all cases, in particular when some finally's were active and others
inactive when the "return" was executed.

This patch adds further support for return-within-finally by correctly
managing the currently_in_except_block flag, and should fix all cases.  The
main point is that finally handlers remain on the exception stack even if
they are active (currently being executed), and the unwind return code
should only execute those finally's which are inactive.

New tests are added for the cases which now pass.
2018-09-03 13:08:16 +10:00
Noralf Trønnes 39ee12d1ac Fix os.stat() to use 1970 epoch
Commit 95e70cd0ea 'time: Use 1970 epoch' changed epoch for the time
module, but not for other users. This patch does the same for the only
other core timeutils user: extmod/vfs_fat.c:fat_vfs_stat().
Other timeutils users: cc3200, esp8266 and stm32, are not changed.

Ports that don't use long ints, will still get wrong time values from
os.stat().
2018-08-25 20:43:02 +02:00
Damien George 828f771e32 tests/basics: Provide .exp files for generator tests that fail PEP479.
PEP479 (see https://www.python.org/dev/peps/pep-0479/) prohibited raising
StopIteration from within a generator (it is turned into a RuntimeError).
This behaviour was introduced in Python 3.5 and in 3.7 was made compulsory.
Until uPy implements PEP479, this patch adds .py.exp files for the relevant
tests so they can be run under Python 3.7.
2018-08-17 15:50:21 +10:00
Damien George 8979ce1671 tests: Modify tests that print repr of an exception with 1 arg.
In Python 3.7 the behaviour of repr() of an exception with one argument
changed: it no longer prints a trailing comma in the argument list.  See
https://bugs.python.org/issue30399

This patch modifies tests that rely on this behaviour to not rely on it.
And the python34.py test is updated to include a test for this behaviour
with a .exp file.
2018-08-17 15:46:04 +10:00
Damien George 0988b14cd6 tests/basics/int_big_error.py: Use bytearray to test for int overflow.
In Python 3.7 "1 >> (big int)" is now allowed, it no longer raises an
OverflowError.  So use bytearray to test big-int conversion overflow.
2018-08-17 15:43:47 +10:00
Damien George 96e1fd480d tests/basics/set_pop.py: Sort set before printing for consistent output. 2018-08-17 15:42:51 +10:00
Damien George f774614110 tests/micropython: Add tests for try and with blocks under native/viper. 2018-08-17 14:11:36 +10:00
stijn ca0d78cebb run-tests: Make .exp and .out file names unique by prefixing with dir.
Input files like basics/string_format.py and float/string_format.py have
the same basename so using that name for writing the output (.exp and .out
files) when both tests fail, results in the output of the first one being
overwritten.

Avoid this by using unique names for the output, replacing path characters
with underscores.
2018-08-10 16:33:42 +10:00
Damien George ce786da196 tests/run-tests: Enable bool1.py test with native emitter.
It should work reliably now.
2018-08-04 22:19:04 +10:00
Damien George 49529f22d4 tests/micropython/viper_cond: Add test for large int as bool. 2018-08-04 22:16:24 +10:00
Ayke van Laethem 6572029dc0 tests: Make tests work on targets without float support. 2018-08-04 15:14:23 +10:00
Damien George 571295d090 tests/extmod/ujson_dump_iobase.py: Return number of bytes written.
Otherwise returning None indicates that the write would block and nothing
was actually written.  Fixes issue #3990.
2018-07-30 12:05:48 +10:00
Dan Halbert f48b70050e merge finished 2018-07-28 13:29:47 -04:00
Ayke van Laethem 055ee18919 tests/run-tests: Improve crash reporting when running on remote targets.
It is very useful to know the actual error when debugging why a test fails.
2018-07-20 09:27:28 +10:00
Ayke van Laethem 1b88433f2d
tests/run-tests: Add nrf target. 2018-07-20 00:50:57 +02:00
Ayke van Laethem 7c98c6b053
tests: Improve feature detection for VFS. 2018-07-20 00:50:57 +02:00
Dan Halbert 7c219600a2 WIP: after merge; before testing 2018-07-11 16:45:30 -04:00
Damien George e2e22e3d7e py/objgenerator: Implement __name__ with normal fun attr accessor code.
With the recent change b488a4a848, a
generating function now has the same layout in memory as a normal bytecode
function, and so can reuse the latter's attribute accessor code to
implement __name__.
2018-07-10 16:33:57 +10:00
Damien George e30a5fc7bc extmod/modure: Add ure.sub() function and method, and tests.
This feature is controlled at compile time by MICROPY_PY_URE_SUB, disabled
by default.

Thanks to @dmazzella for the original patch for this feature; see #3770.
2018-07-02 14:55:02 +10:00
Damien George 1e9b871d29 extmod/modure: Add match.span(), start() and end() methods, and tests.
This feature is controlled at compile time by
MICROPY_PY_URE_MATCH_SPAN_START_END, disabled by default.

Thanks to @dmazzella for the original patch for this feature; see #3770.
2018-07-02 14:54:56 +10:00
Damien George 1f86460910 extmod/modure: Add match.groups() method, and tests.
This feature is controlled at compile time by MICROPY_PY_URE_MATCH_GROUPS,
disabled by default.

Thanks to @dmazzella for the original patch for this feature; see #3770.
2018-07-02 14:53:30 +10:00
Damien George d8dc918deb py/compile: Handle return/break/continue correctly in async with.
Before this patch the context manager's __aexit__() method would not be
executed if a return/break/continue statement was used to exit an async
with block.  async with now has the same semantics as normal with.

The fix here applies purely to the compiler, and does not modify the
runtime at all. It might (eventually) be better to define new bytecode(s)
to handle async with (and maybe other async constructs) in a cleaner, more
efficient way.

One minor drawback with addressing this issue purely in the compiler is
that it wasn't possible to get 100% CPython semantics.  The thing that is
different here to CPython is that the __aexit__ method is not looked up in
the context manager until it is needed, which is after the body of the
async with statement has executed.  So if a context manager doesn't have
__aexit__ then CPython raises an exception before the async with is
executed, whereas uPy will raise it after it is executed.  Note that
__aenter__ is looked up at the beginning in uPy because it needs to be
called straightaway, so if the context manager isn't a context manager then
it'll still raise an exception at the same location as CPython.  The only
difference is if the context manager has the __aenter__ method but not the
__aexit__ method, then in that case uPy has different behaviour.  But this
is a very minor, and acceptable, difference.
2018-06-27 16:57:42 +10:00
Damien George 726804ea40 tests: Move non-filesystem io tests to basics dir with io_ prefix. 2018-06-27 16:55:05 +10:00
Paul Sokolovsky bdceea1d12 tests/basics/namedtuple*: Import ucollections first.
Otherwise, test may have artefacts in the presence of the micropython-lib
module.
2018-06-27 14:58:14 +10:00
Paul Sokolovsky bb634115fc tests/extmod/ucryptolib*: Add into and inplace tests for ucryptolib.
Tests for separate input and output buffer (alloc-free operation) and
the same writable buffer used as input and output (inplace operation).
2018-06-27 14:56:46 +10:00