In commit 86ce442607 the '.frozen' entry was
added at the start of sys.path, to allow control over when frozen modules
are searched during import, and retain existing behaviour whereby frozen
was searched before the filesystem.
But Python semantics of sys.path require sys.path[0] to be the directory of
the currently executing script, or ''.
This commit moves the '.frozen' entry to second place in sys.path, so
sys.path[0] retains its correct value (described above).
Signed-off-by: Damien George <damien@micropython.org>
This commit makes sure that the value zero is always encoded in an mpz_t as
neg=0 and len=0 (previously it was just len=0).
This invariant is needed for some of the bitwise operations that operate on
negative numbers, because they cannot handle -0. For example
(-((1<<100)-(1<<100)))|1 was being computed as -65535, instead of 1.
Fixes issue #8042.
Signed-off-by: Damien George <damien@micropython.org>
This feature is not enabled on any port, it's not in CPython's io module,
and functionality is better suited to the micropython-lib implementation of
pkg_resources.
Expected result of const.py will be matched only when MICROPY_COMP_CONST is
enabled. For easy understanding, added description at the first of the
test code.
This commit removes all parts of code associated with the existing
MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE optimisation option, including the
-mcache-lookup-bc option to mpy-cross.
This feature originally provided a significant performance boost for Unix,
but wasn't able to be enabled for MCU targets (due to frozen bytecode), and
added significant extra complexity to generating and distributing .mpy
files.
The equivalent performance gain is now provided by the combination of
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE (which has
been enabled on the unix port in the previous commit).
It's hard to provide precise performance numbers, but tests have been run
on a wide variety of architectures (x86-64, ARM Cortex, Aarch64, RISC-V,
xtensa) and they all generally agree on the qualitative improvements seen
by the combination of MICROPY_OPT_LOAD_ATTR_FAST_PATH and
MICROPY_OPT_MAP_LOOKUP_CACHE.
For example, on a "quiet" Linux x64 environment (i3-5010U @ 2.10GHz) the
change from CACHE_MAP_LOOKUP_IN_BYTECODE, to LOAD_ATTR_FAST_PATH combined
with MAP_LOOKUP_CACHE is:
diff of scores (higher is better)
N=2000 M=2000 bccache -> attrmapcache diff diff% (error%)
bm_chaos.py 13742.56 -> 13905.67 : +163.11 = +1.187% (+/-3.75%)
bm_fannkuch.py 60.13 -> 61.34 : +1.21 = +2.012% (+/-2.11%)
bm_fft.py 113083.20 -> 114793.68 : +1710.48 = +1.513% (+/-1.57%)
bm_float.py 256552.80 -> 243908.29 : -12644.51 = -4.929% (+/-1.90%)
bm_hexiom.py 521.93 -> 625.41 : +103.48 = +19.826% (+/-0.40%)
bm_nqueens.py 197544.25 -> 217713.12 : +20168.87 = +10.210% (+/-3.01%)
bm_pidigits.py 8072.98 -> 8198.75 : +125.77 = +1.558% (+/-3.22%)
misc_aes.py 17283.45 -> 16480.52 : -802.93 = -4.646% (+/-0.82%)
misc_mandel.py 99083.99 -> 128939.84 : +29855.85 = +30.132% (+/-5.88%)
misc_pystone.py 83860.10 -> 82592.56 : -1267.54 = -1.511% (+/-2.27%)
misc_raytrace.py 21490.40 -> 22227.23 : +736.83 = +3.429% (+/-1.88%)
This shows that the new optimisations are at least as good as the existing
inline-bytecode-caching, and are sometimes much better (because the new
ones apply caching to a wider variety of map lookups).
The new optimisations can also benefit code generated by the native
emitter, because they apply to the runtime rather than the generated code.
The improvement for the native emitter when LOAD_ATTR_FAST_PATH and
MAP_LOOKUP_CACHE are enabled is (same Linux environment as above):
diff of scores (higher is better)
N=2000 M=2000 native -> nat-attrmapcache diff diff% (error%)
bm_chaos.py 14130.62 -> 15464.68 : +1334.06 = +9.441% (+/-7.11%)
bm_fannkuch.py 74.96 -> 76.16 : +1.20 = +1.601% (+/-1.80%)
bm_fft.py 166682.99 -> 168221.86 : +1538.87 = +0.923% (+/-4.20%)
bm_float.py 233415.23 -> 265524.90 : +32109.67 = +13.756% (+/-2.57%)
bm_hexiom.py 628.59 -> 734.17 : +105.58 = +16.796% (+/-1.39%)
bm_nqueens.py 225418.44 -> 232926.45 : +7508.01 = +3.331% (+/-3.10%)
bm_pidigits.py 6322.00 -> 6379.52 : +57.52 = +0.910% (+/-5.62%)
misc_aes.py 20670.10 -> 27223.18 : +6553.08 = +31.703% (+/-1.56%)
misc_mandel.py 138221.11 -> 152014.01 : +13792.90 = +9.979% (+/-2.46%)
misc_pystone.py 85032.14 -> 105681.44 : +20649.30 = +24.284% (+/-2.25%)
misc_raytrace.py 19800.01 -> 23350.73 : +3550.72 = +17.933% (+/-2.79%)
In summary, compared to MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE, the new
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE options:
- are simpler;
- take less code size;
- are faster (generally);
- work with code generated by the native emitter;
- can be used on embedded targets with a small and constant RAM overhead;
- allow the same .mpy bytecode to run on all targets.
See #7680 for further discussion. And see also #7653 for a discussion
about simplifying mpy-cross options.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This feature {x=} was introduced in Python 3.8 so needs a separate .exp
file to run on earlier Python versions.
See https://bugs.python.org/issue36817
Signed-off-by: Damien George <damien@micropython.org>
This achieves a substantial performance improvement when rendering glyphs
to color displays, the benefit increasing proportional to the number of
pixels in the glyph.
Prevents the finaliser from being missed if there's a dangling reference
on the stack to one of the blocks for the files (that this test checks
that they get finalised).
See github.com/micropython/micropython/pull/7659#issuecomment-899479793
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This tests both sending indications/notifications from a server to
subscribed clients via gatts_write(...,send_update=True) and subscribing
from a client.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This implements (most of) the PEP-498 spec for f-strings and is based on
https://github.com/micropython/micropython/pull/4998 by @klardotsh.
It is implemented in the lexer as a syntax translation to `str.format`:
f"{a}" --> "{}".format(a)
It also supports:
f"{a=}" --> "a={}".format(a)
This is done by extracting the arguments into a temporary vstr buffer,
then after the string has been tokenized, the lexer input queue is saved
and the contents of the temporary vstr buffer are injected into the lexer
instead.
There are four main limitations:
- raw f-strings (`fr` or `rf` prefixes) are not supported and will raise
`SyntaxError: raw f-strings are not supported`.
- literal concatenation of f-strings with adjacent strings will fail
"{}" f"{a}" --> "{}{}".format(a) (str.format will incorrectly use
the braces from the non-f-string)
f"{a}" f"{a}" --> "{}".format(a) "{}".format(a) (cannot concatenate)
- PEP-498 requires the full parser to understand the interpolated
argument, however because this entirely runs in the lexer it cannot
resolve nested braces in expressions like
f"{'}'}"
- The !r, !s, and !a conversions are not supported.
Includes tests and cpydiffs.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The correct callback-deregister functions must be called dependent on the
socket type, otherwise resources may not be freed correctly.
Signed-off-by: Damien George <damien@micropython.org>
Test instances can now use the following methods to synchronise their
execution:
multitest.broadcast("sync message")
multitest.wait("sync message")
Signed-off-by: Damien George <damien@micropython.org>
Previously a subclass of a type that didn't implement unary_op, or didn't
handle MP_UNARY_OP_BOOL, would raise TypeError on bool conversion.
Fixes#5677.
Replace "master" with "controller" and "slave" with "peripheral" in
comments, errors, and debug messages.
Add CONTROLLER and PERIPHERAL constants to pyb.SPI and pyb.I2C classes;
retain MASTER and SLAVE constants for backward compatiblity.
The MP_OBJ_STOP_ITERATION optimisation is a shortcut for creating a
StopIteration() exception object, and means that heap memory does not need
to be allocated for the exception (in cases where it can be used). This
commit allows this optimised object to take an optional argument (before,
it could only have no argument).
The commit also adds some new tests to cover corner cases with
StopIteration and generators that previously did not work.
Signed-off-by: Damien George <damien@micropython.org>
This test snuck through without proper formatting and is causing CI for
other unrelated changes to fail.
Signed-off-by: David Lechner <david@pybricks.com>
Currently when using uasyncio.start_server() the socket configuration is
done inside a uasyncio.create_task() background function. If the address
and port are already in use however this throws an OSError which cannot be
cleanly caught behind the create_task().
This commit moves the getaddrinfo and socket binding to the start_server()
function, and only creates the task if that succeeds. This means that any
OSError from the initial socket configuration is propagated directly up the
call stack, compatible with CPython behaviour.
See #7444.
Signed-off-by: Damien George <damien@micropython.org>
2.5 can be represented correctly in object representation C, but 2.3 cannot
(it is slightly truncated).
Signed-off-by: Damien George <damien@micropython.org>
This commit fixes a problem with a race between cancellation of task A and
completion of task B, when A waits on B. If task B completes just before
task A is cancelled then the cancellation of A does not work. Instead,
the CancelledError meant to cancel A gets passed through to B (that's
expected behaviour) but B handles it as a "Task exception wasn't retrieved"
scenario, printing out such a message (this is because finished tasks point
their "coro" attribute to themselves to indicate they are done, and
implement the throw() method, but that method inadvertently catches the
CancelledError). The correct behaviour is for B to bounce that
CancelledError back out.
This bug is mainly seen when wait_for() is used, and in that context the
symptoms are:
- occurs when using wait_for(T, S), if the task T being waited on finishes
at exactly the same time as the wait-for timeout S expires
- task T will have run to completion
- the "Task exception wasn't retrieved message" is printed with
"<class 'CancelledError'>" as the error (ie no traceback)
- the wait_for(T, S) call never returns (it's never put back on the
uasyncio run queue) and all tasks waiting on this are blocked forever
from running
- uasyncio otherwise continues to function and other tasks continue to be
scheduled as normal
The fix here reworks the "waiting" attribute of Task to be called "state"
and uses it to indicate whether a task is: running and not awaited on,
running and awaited on, finished and not awaited on, or finished and
awaited on. This means the task does not need to point "coro" to itself to
indicate finished, and also allows removal of the throw() method.
A benefit of this is that "Task exception wasn't retrieved" messages can go
back to being able to print the name of the coroutine function.
Fixes issue #7386.
Signed-off-by: Damien George <damien@micropython.org>
With docs and a multi-test using TCP server/client.
This method is a MicroPython extension, although there is discussion of
adding it to CPython: https://bugs.python.org/issue41305
Signed-off-by: Mike Teachman <mike.teachman@gmail.com>
The random module's getrandbits() method didn't give a proper error message
when calling it with a value that was outside of the range of 1-32, which
can lead to confusion using this function (which under CPython can accept
numbers larger than 32). Now instead of simply giving a ValueError it
gives an error message that states that the number of bits is constrained.
Also, since the random module's functions getrandbits() and randint()
differ from CPython, tests have been added to describe these differences.
For getrandbits the relevant documentation is shown and added to the docs.
The same is given for randint method so that the information is more easily
found.
Finally, since the int object lacks the bit_length() method there is a test
for that method also to include within the docs, showing the difference to
CPython.
If digest is called then the hash object is put in a "final" state and
calling update() or digest() again will raise a ValueError (instead of
silently producing the wrong result).
See issue #4119.
Signed-off-by: Damien George <damien@micropython.org>
