Commit Graph

13434 Commits

Author SHA1 Message Date
Damien George 079f3e5e5b py/parse: Allow all constant objects to be used in "X = const(o)".
Now that constant tuples are supported in the parser, eg (1, True, "str"),
it's a small step to allow anything that is a constant to be used with the
pattern:

    from micropython import const

    X = const(obj)

This commit makes the required changes to allow the following types of
constants:

    from micropython import const

    _INT = const(123)
    _FLOAT = const(1.2)
    _COMPLEX = const(3.4j)
    _STR = const("str")
    _BYTES = const(b"bytes")
    _TUPLE = const((_INT, _STR, _BYTES))
    _TUPLE2 = const((None, False, True, ..., (), _TUPLE))

Prior to this, only integers could be used in const(...).

Signed-off-by: Damien George <damien@micropython.org>
2022-05-18 16:18:35 +10:00
Damien George 761d2f6741 tests/micropython: Add more test cases for native generators.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-18 15:23:11 +10:00
Damien George 80938839c1 tests/extmod: Use bytearray instead of bytes for uctypes test.
Because the test modifies the (now) bytearray object, and if it's a bytes
object it's not guaranteed that it can be modified, or that this constant
object isn't used elsewhere.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-18 15:23:11 +10:00
Damien George 8588525868 py/compile: De-duplicate constant objects in module's constant table.
The recent rework of bytecode made all constants global with respect to the
module (previously, each function had its own constant table).  That means
the constant table for a module is shared among all functions/methods/etc
within the module.

This commit add support to the compiler to de-duplicate constants in this
module constant table.  So if a constant is used more than once -- eg 1.0
or (None, None) -- then the same object is reused for all instances.

For example, if there is code like `print(1.0, 1.0)` then the parser will
create two independent constants 1.0 and 1.0.  The compiler will then (with
this commit) notice they are the same and only put one of them in the
constant table.  The bytecode will then reuse that constant twice in the
print expression.  That allows the second 1.0 to be reclaimed by the GC,
also means the constant table has one less entry so saves a word.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-18 15:23:11 +10:00
Damien George b3d0f5f67c tests/micropython: Fully unlink nested list in extreme exc test.
To make sure there are no dangling references to the lists, and the GC can
reclaim heap memory.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-18 09:16:33 +10:00
Damien George 90682f43af py/compile: Allow new qstrs to be allocated at all compiler passes.
Prior to this commit, all qstrs were required to be allocated (by calling
mp_emit_common_use_qstr) in the MP_PASS_SCOPE pass (the first one).  But
this is an unnecessary restriction, which is lifted by this commit.
Lifting the restriction simplifies the compiler because it can allocate
qstrs in later passes.

This also generates better code, because in some cases (eg when a variable
is closed over) the scope of an identifier is not known until a bit later
and then the identifier no longer needs its qstr allocated in the global
table.

Code size is reduced for all ports with this commit.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 23:39:22 +10:00
Damien George 1fb01bd6c5 py/emitnative: Put a pointer to the native prelude in child_table array.
Some architectures (like esp32 xtensa) cannot read byte-wise from
executable memory.  This means the prelude for native functions -- which is
usually located after the machine code for the native function -- must be
placed in separate memory that can be read byte-wise.  Prior to this commit
this was achieved by enabling N_PRELUDE_AS_BYTES_OBJ for the emitter and
MICROPY_EMIT_NATIVE_PRELUDE_AS_BYTES_OBJ for the runtime.  The prelude was
then placed in a bytes object, pointed to by the module's constant table.

This behaviour is changed by this commit so that a pointer to the prelude
is stored either in mp_obj_fun_bc_t.child_table, or in
mp_obj_fun_bc_t.child_table[num_children] if num_children > 0.  The reasons
for doing this are:

1. It decouples the native emitter from runtime requirements, the emitted
   code no longer needs to know if the system it runs on can/can't read
   byte-wise from executable memory.

2. It makes all ports have the same emitter behaviour, there is no longer
   the N_PRELUDE_AS_BYTES_OBJ option.

3. The module's constant table is now used only for actual constants in the
   Python code.  This allows further optimisations to be done with the
   constants (eg constant deduplication).

Code size change for those ports that enable the native emitter:
   unix x64:   +80 +0.015%
      stm32:   +24 +0.004% PYBV10
    esp8266:   +88 +0.013% GENERIC
      esp32:   -20 -0.002% GENERIC[incl -112(data)]
        rp2:   +32 +0.005% PICO

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 16:44:49 +10:00
Damien George 8725a32f41 tools/ci.sh: For esp8266 CI install esptool 3.3.1.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 16:01:56 +10:00
Damien George f8d3956c63 tools/ci.sh: Run full test suite on stackless and settrace builds.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 14:25:51 +10:00
Damien George acfc3bbdf8 tests/multi_net: Skip SSL test if relevant modules aren't available.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 14:25:51 +10:00
Damien George ab0a8f3086 tests/run-tests.py: Exclude settrace tests when using native emitter.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 14:25:51 +10:00
Damien George 5f650b7b7a tests/thread: Use less resources for stress_aes if settrace enabled.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 14:25:51 +10:00
Damien George 1762990579 py/bc: Provide separate code-state setup funcs for bytecode and native.
mpy-cross will now generate native code based on the size of
mp_code_state_native_t, and the runtime will use this struct to calculate
the offset of the .state field.  This makes native code generation and
execution (which rely on this struct) independent to the settings
MICROPY_STACKLESS and MICROPY_PY_SYS_SETTRACE, both of which change the
size of the mp_code_state_t struct.

Fixes issue #5059.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 14:25:51 +10:00
Damien George 8e1db993cd py/asmx64: Support full range of regs in asm_x64_lea_disp_to_r64.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 14:25:51 +10:00
Damien George 6f68a8c240 tests/run-perfbench.py: Return error code if any test fails on target.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 14:06:41 +10:00
Damien George d7cf8a3b9d tests/perf_bench: Update .mpy file header to remove old unicode flag.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 14:06:41 +10:00
Damien George 2a366e183e docs/reference/mpyfiles: Remove docs about mpy flags and qstr win size.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 12:51:54 +10:00
Damien George c49d5207e9 py/persistentcode: Remove unicode feature flag from .mpy file.
Prior to this commit, even with unicode disabled .py and .mpy files could
contain unicode characters, eg by entering them directly in a string as
utf-8 encoded.

The only thing the compiler disallowed (with unicode disabled) was using
\uxxxx and \Uxxxxxxxx notation to specify a character within a string with
value >= 0x100; that would give a SyntaxError.

With this change mpy-cross will now accept \u and \U notation to insert a
character with value >= 0x100 into a string (because the -mno-unicode
option is now gone, there's no way to forbid this).  The runtime will
happily work with strings with such characters, just like it already works
with strings with characters that were utf-8 encoded directly.

This change simplifies things because there are no longer any feature
flags in .mpy files, and any bytecode .mpy will now run on any target.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 12:51:54 +10:00
Damien George b295b6f1f3 py/persistentcode: Remove obsolete comment about qstr window size.
This was made obsolete in f2040bfc7e

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 12:51:54 +10:00
Damien George 2ed4f7a130 tools/mpy_ld.py: Remove obsolete QSTR_WINDOW_SIZE constant.
This was made obsolete in f2040bfc7e

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 12:51:54 +10:00
Damien George 8aa254c369 tests: Fix tests to use sys.implementation._mpy.
The field was renamed to _mpy in 59c5d41611

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 09:46:40 +10:00
Takeo Takahashi e3c880a569 tools/autobuild: Update for ports/renesas-ra.
* Add build_renesas_ra_boards call in autobuild.sh
* Add build_renesas_ra_boards function to generate firmware.hex.

Signed-off-by: Takeo Takahashi <takeo.takahashi.xv@renesas.com>
2022-05-06 17:15:21 +09:00
Damien George 1e72580fd8 top: Update .git-blame-ignore-revs for latest formatting commit.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-05 13:31:04 +10:00
Damien George 5b700b0af9 all: Reformat remaining C code that doesn't have a space after a comma.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-05 13:30:40 +10:00
Damien George aec8db3cd1 tools/uncrustify: Make sure a space exists after a comma.
This will add a space after a comma if it doesn't have one, but will allow
more than one space if the spaces are already there.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-05 13:30:40 +10:00
Damien George b0a1b60a9b extmod: Move font_petme128_8x8.h from ports/stm32 to extmod.
And add spaces after commas so it is consistently formatted.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-05 13:30:40 +10:00
Damien George 1216c9fffa py/objmodule: Move stray #include to top of file.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-05 11:02:38 +10:00
Damien George da31ad7aad mimxrt/mbedtls: Use core-provided tracked alloc instead of custom funcs.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-05 10:31:50 +10:00
Damien George 39c96b543f stm32/mbedtls: Use core-provided tracked alloc instead of custom funcs.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-05 10:31:50 +10:00
Damien George fca5701f74 py/malloc: Introduce m_tracked_calloc, m_tracked_free functions.
Enabled by MICROPY_TRACKED_ALLOC.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-05 10:31:50 +10:00
Takeo Takahashi 965747bd97 renesas-ra: Rename pyb_rtc_ to machine_rtc_.
Signed-off-by: Takeo Takahashi <takeo.takahashi.xv@renesas.com>
2022-05-04 19:45:16 +09:00
Takeo Takahashi 5f57ec464a renesas-ra: Rename pyb_uart_ to machine_uart_.
Signed-off-by: Takeo Takahashi <takeo.takahashi.xv@renesas.com>
2022-05-04 19:45:16 +09:00
stijn 20028c7c80 windows: Avoid busy loop when using scheduler. 2022-05-04 10:22:29 +10:00
stijn 8aa79c95bd windows: Consolidate all sleep-related functions into windows_mphal.c.
Replace the timer-based sleep with the standard win32 call since the former
has no benefits: even though it allows specifying the time in 100uSec
chunks, the actual resolution is still limited by the OS and is never
better than 1mSec.

For clarity move all of this next to the mp_hal_delay_ms definition so all
related functions are in one place.
2022-05-04 10:22:09 +10:00
Damien George c90f097519 tests/extmod: Increase timing on uasyncio tests to make more reliable.
Non-real-time systems like Windows, Linux and macOS do not have reliable
timing, so increase the sleep intervals to make these tests more likely to
pass.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-03 22:53:12 +10:00
Jim Mussared 0e7bfc88c6 all: Use mp_obj_malloc everywhere it's applicable.
This replaces occurences of

    foo_t *foo = m_new_obj(foo_t);
    foo->base.type = &foo_type;

with

    foo_t *foo = mp_obj_malloc(foo_t, &foo_type);

Excludes any places where base is a sub-field or when new0/memset is used.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-03 22:28:14 +10:00
Jim Mussared 6a3bc0e1a1 py/objfloat: Explain why mp_obj_malloc isn't used.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-03 22:25:40 +10:00
Jim Mussared 709e8328d9 py/obj: Introduce mp_obj_malloc macro to allocate, and set object type.
This is to replace the following:

    mp_foo_obj_t *self = m_new_obj(mp_foo_obj_t);
    self->base.type = &mp_type_foo;

with:

    mp_foo_obj_t *self = mp_obj_malloc(mp_foo_obj_t, &mp_type_foo);

Calling the function is less code than inlining setting the type
everywhere, adds up to ~100 bytes on PYBV11.

It also helps to avoid an easy mistake of forgetting to set the type.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-03 22:23:46 +10:00
Damien George 590de399f0 py/emitcommon: Don't implicitly close class vars that are assigned to.
When in a class body or at the module level don't implicitly close over
variables that have been assigned to.

Fixes issue #8603.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-03 16:38:43 +10:00
Takeo Takahashi a21fd7cc21 docs/renesas-ra: Remove unused image files.
Signed-off-by: Takeo Takahashi <takeo.takahashi.xv@renesas.com>
2022-05-03 13:22:01 +10:00
Takeo Takahashi a204365df1 renesas-ra/boards: Change device name information to show properly.
* Change device name table to list style to show properly.
* Change the link of cable connection information to the latest.

Signed-off-by: Takeo Takahashi <takeo.takahashi.xv@renesas.com>
2022-05-02 12:02:32 +09:00
Takeo Takahashi 52add7b45c docs: Update files for renesas-ra port.
Signed-off-by: Takeo Takahashi <takeo.takahashi.xv@renesas.com>
2022-04-29 12:29:08 +09:00
Takeo Takahashi 1b61800530 docs/renesas-ra: Add renesas-ra docs files.
Signed-off-by: Takeo Takahashi <takeo.takahashi.xv@renesas.com>
2022-04-29 12:29:08 +09:00
Takeo Takahashi 3717d599e2 tests/run-tests.py: Update for renesas-ra port.
Signed-off-by: Takeo Takahashi <takeo.takahashi.xv@renesas.com>
2022-04-29 12:29:08 +09:00
Takeo Takahashi 4753913253 tests/renesas-ra: Add tests for renesas-ra port.
Signed-off-by: Takeo Takahashi <takeo.takahashi.xv@renesas.com>
2022-04-29 12:29:07 +09:00
Takeo Takahashi 3a941cce51 tools/ci.sh: Update for ports/renesas-ra.
Signed-off-by: Takeo Takahashi <takeo.takahashi.xv@renesas.com>
2022-04-29 12:29:07 +09:00
Takeo Takahashi e214ae0e64 github/workflows: Add workflow for renesas-ra port.
Signed-off-by: Takeo Takahashi <takeo.takahashi.xv@renesas.com>
2022-04-29 12:29:07 +09:00
Takeo Takahashi 86e35178e6 drivers/dht/dht.py: Change default import module as machine.
Signed-off-by: Takeo Takahashi <takeo.takahashi.xv@renesas.com>
2022-04-29 12:29:07 +09:00
Takeo Takahashi 0f57ccf7f2 renesas-ra: Add new port to Renesas RA.
Signed-off-by: Takeo Takahashi <takeo.takahashi.xv@renesas.com>
2022-04-29 12:29:06 +09:00
TakeoTakahashi2020 4c9b00b425 lib/fsp: Add renesas fsp git repository as submodule.
Signed-off-by: Takeo Takahashi <takeo.takahashi.xv@renesas.com>
2022-04-29 10:55:58 +09:00