Commit Graph

13465 Commits

Author SHA1 Message Date
Damien George 7883ae413d py/emitnative: Provide dedicated local for exception unwind handler ptr.
This eliminates the need to save and restore the exception unwind handler
pointer when calling nlr_push.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-19 17:31:56 +10:00
Damien George b608964804 py/emitnative: Simplify generation of code that loads prelude pointer.
It's possible to use REG_PARENT_ARG_1 instead of REG_LOCAL_3.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-19 17:31:56 +10:00
Damien George 56f2d3c2e5 py/asmthumb: Fix PC relative load by sign extending the constant.
PC relative offsets can be negative, in which case the movw result must be
sign extended.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-19 17:31:56 +10:00
Damien George c70f96f1c5 esp32: Track allocated iRAM and free it on soft reset.
This makes sure all iRAM allocated for native code is freed on soft reset.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-19 17:31:56 +10:00
Damien George 54ab9d23e9 tests/run-perfbench.py: Allow running tests via mpy and native emitter.
The performance benchmark tests now support `--via-mpy` and `--emit native`
on remote targets.  For example:

    $ ./run-perfbench.py -p --via-mpy --emit native 100 100

Signed-off-by: Damien George <damien@micropython.org>
2022-05-19 17:31:56 +10:00
Damien George 1786dacc83 tests/run-tests.py: Allow running tests via mpy-cross on remote targets.
This adds support for the `--via-mpy` and `--emit native` options when
running tests on remote targets (via pyboard.py).  It's now possible to do:

    $ ./run-tests.py --target pyboard --via-mpy
    $ ./run-tests.py --target pyboard --via-mpy --emit native

Signed-off-by: Damien George <damien@micropython.org>
2022-05-19 17:31:56 +10:00
Daniël van de Giessen ef16796f49 esp32/machine_pin: Fix ESP32C3 IDFv4.4.1 build.
Use new constants as seen in ESP-IDF commit b25fb1.

Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
2022-05-19 16:47:13 +10:00
Algy Tynan 5dac142a1a esp32/boards/LILYGO_TTGO_LORA32: Remove ID from TTGO Lora32 board json.
Generic ID creates an incorrect link on the MicroPython download page.
Remove the ID to fix the link.

Signed-off-by: Algy Tynan <algy@tynan.io>
2022-05-19 16:45:18 +10:00
Michael Himing a1abf51a9d esp32/boards/UM_FEATHERS3: Use correct sdkconfig.board. 2022-05-19 16:43:59 +10:00
Christian Zietz e92c22a839 esp8266/etshal.h: Remove unneeded function declaration. 2022-05-19 16:40:39 +10:00
Christian Zietz ea1c0557e5 esp8266/machine_wdt: Remove deinit method for watchdog.
Reasons for removal:
- It did not work properly because it stopped the hardware watchdog
  timer while keeping the software watchdog running (issue #8597).
- There isn't a deinit method for the WDT in any other port.
- "The watchdog is not intended to be stopped. That is a feature."
  (See #8600.)
2022-05-19 16:40:39 +10:00
Jim Mussared 8b201dc4c3 py: Remove support for MICROPY_PORT_BUILTIN_MODULES.
This functionality is now replaced with MP_REGISTER_MODULE.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:57:09 +10:00
Jim Mussared c2d889fc1e renesas-ra: Make port-specific modules use MP_REGISTER_MODULE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:57:09 +10:00
Jim Mussared 1d33ceb0d0 cc3200: Make port-specific modules use MP_REGISTER_MODULE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:57:09 +10:00
Jim Mussared 9e4da2320a javascript: Make port-specific modules use MP_REGISTER_MODULE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:57:09 +10:00
Jim Mussared ec2fe5d6ca zephyr: Make port-specific modules use MP_REGISTER_MODULE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:57:09 +10:00
Jim Mussared 63431a5536 unix: Make port-specific modules use MP_REGISTER_MODULE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:57:09 +10:00
Jim Mussared d0307e79bb teensy: Make port-specific modules use MP_REGISTER_MODULE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:57:09 +10:00
Jim Mussared c3f7424a57 samd: Make port-specific modules use MP_REGISTER_MODULE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:57:09 +10:00
Jim Mussared 62f00a43d5 rp2: Make port-specific modules use MP_REGISTER_MODULE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:57:09 +10:00
Jim Mussared 60c5968f99 pic16bit: Make port-specific modules use MP_REGISTER_MODULE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:57:09 +10:00
Jim Mussared f67ac90fa9 nrf: Make port-specific modules use MP_REGISTER_MODULE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:57:09 +10:00
Jim Mussared 7d79fd7e8b mimxrt: Make port-specific modules use MP_REGISTER_MODULE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:56:30 +10:00
Jim Mussared 4274b34d6b esp8266: Make port-specific modules use MP_REGISTER_MODULE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:56:30 +10:00
Jim Mussared 24127a4ac4 esp32: Make port-specific modules use MP_REGISTER_MODULE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:56:30 +10:00
Jim Mussared 0a92469c10 stm32: Make port-specific modules use MP_REGISTER_MODULE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:56:30 +10:00
Jim Mussared bb794f05b7 extmod: Make port-included extmod modules use MP_REGISTER_MODULES.
_onewire, socket, and network were previously added by the port rather
than objmodule.c.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:49:12 +10:00
Jim Mussared d8d3e6ae78 py: Make builtin modules use MP_REGISTER_MODULE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:49:12 +10:00
Jim Mussared 4eab44a1ec extmod: Make extmod modules use MP_REGISTER_MODULE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:49:12 +10:00
Jim Mussared 469450171f py/makemoduledefs.py: Allow multiple ways to register a module.
For example, ussl can come from axtls or mbedtls. If neither are enabled
then don't try and set an empty definition twice, and only include it
once in MICROPY_REGISTERED_MODULES.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:49:12 +10:00
Jim Mussared bc42db4496 github/ISSUE_TEMPLATE: Add GitHub issue templates and external links.
When opening a new issue the following selection is now shown:
- Bug reports
- Feature requests
- Security issue
- Documentation issue
- Link to forum
- Link to docs
- Link to downloads
2022-05-18 16:49:11 +10:00
Damien George 079f3e5e5b py/parse: Allow all constant objects to be used in "X = const(o)".
Now that constant tuples are supported in the parser, eg (1, True, "str"),
it's a small step to allow anything that is a constant to be used with the
pattern:

    from micropython import const

    X = const(obj)

This commit makes the required changes to allow the following types of
constants:

    from micropython import const

    _INT = const(123)
    _FLOAT = const(1.2)
    _COMPLEX = const(3.4j)
    _STR = const("str")
    _BYTES = const(b"bytes")
    _TUPLE = const((_INT, _STR, _BYTES))
    _TUPLE2 = const((None, False, True, ..., (), _TUPLE))

Prior to this, only integers could be used in const(...).

Signed-off-by: Damien George <damien@micropython.org>
2022-05-18 16:18:35 +10:00
Damien George 761d2f6741 tests/micropython: Add more test cases for native generators.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-18 15:23:11 +10:00
Damien George 80938839c1 tests/extmod: Use bytearray instead of bytes for uctypes test.
Because the test modifies the (now) bytearray object, and if it's a bytes
object it's not guaranteed that it can be modified, or that this constant
object isn't used elsewhere.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-18 15:23:11 +10:00
Damien George 8588525868 py/compile: De-duplicate constant objects in module's constant table.
The recent rework of bytecode made all constants global with respect to the
module (previously, each function had its own constant table).  That means
the constant table for a module is shared among all functions/methods/etc
within the module.

This commit add support to the compiler to de-duplicate constants in this
module constant table.  So if a constant is used more than once -- eg 1.0
or (None, None) -- then the same object is reused for all instances.

For example, if there is code like `print(1.0, 1.0)` then the parser will
create two independent constants 1.0 and 1.0.  The compiler will then (with
this commit) notice they are the same and only put one of them in the
constant table.  The bytecode will then reuse that constant twice in the
print expression.  That allows the second 1.0 to be reclaimed by the GC,
also means the constant table has one less entry so saves a word.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-18 15:23:11 +10:00
Damien George b3d0f5f67c tests/micropython: Fully unlink nested list in extreme exc test.
To make sure there are no dangling references to the lists, and the GC can
reclaim heap memory.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-18 09:16:33 +10:00
Damien George 90682f43af py/compile: Allow new qstrs to be allocated at all compiler passes.
Prior to this commit, all qstrs were required to be allocated (by calling
mp_emit_common_use_qstr) in the MP_PASS_SCOPE pass (the first one).  But
this is an unnecessary restriction, which is lifted by this commit.
Lifting the restriction simplifies the compiler because it can allocate
qstrs in later passes.

This also generates better code, because in some cases (eg when a variable
is closed over) the scope of an identifier is not known until a bit later
and then the identifier no longer needs its qstr allocated in the global
table.

Code size is reduced for all ports with this commit.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 23:39:22 +10:00
Damien George 1fb01bd6c5 py/emitnative: Put a pointer to the native prelude in child_table array.
Some architectures (like esp32 xtensa) cannot read byte-wise from
executable memory.  This means the prelude for native functions -- which is
usually located after the machine code for the native function -- must be
placed in separate memory that can be read byte-wise.  Prior to this commit
this was achieved by enabling N_PRELUDE_AS_BYTES_OBJ for the emitter and
MICROPY_EMIT_NATIVE_PRELUDE_AS_BYTES_OBJ for the runtime.  The prelude was
then placed in a bytes object, pointed to by the module's constant table.

This behaviour is changed by this commit so that a pointer to the prelude
is stored either in mp_obj_fun_bc_t.child_table, or in
mp_obj_fun_bc_t.child_table[num_children] if num_children > 0.  The reasons
for doing this are:

1. It decouples the native emitter from runtime requirements, the emitted
   code no longer needs to know if the system it runs on can/can't read
   byte-wise from executable memory.

2. It makes all ports have the same emitter behaviour, there is no longer
   the N_PRELUDE_AS_BYTES_OBJ option.

3. The module's constant table is now used only for actual constants in the
   Python code.  This allows further optimisations to be done with the
   constants (eg constant deduplication).

Code size change for those ports that enable the native emitter:
   unix x64:   +80 +0.015%
      stm32:   +24 +0.004% PYBV10
    esp8266:   +88 +0.013% GENERIC
      esp32:   -20 -0.002% GENERIC[incl -112(data)]
        rp2:   +32 +0.005% PICO

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 16:44:49 +10:00
Damien George 8725a32f41 tools/ci.sh: For esp8266 CI install esptool 3.3.1.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 16:01:56 +10:00
Damien George f8d3956c63 tools/ci.sh: Run full test suite on stackless and settrace builds.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 14:25:51 +10:00
Damien George acfc3bbdf8 tests/multi_net: Skip SSL test if relevant modules aren't available.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 14:25:51 +10:00
Damien George ab0a8f3086 tests/run-tests.py: Exclude settrace tests when using native emitter.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 14:25:51 +10:00
Damien George 5f650b7b7a tests/thread: Use less resources for stress_aes if settrace enabled.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 14:25:51 +10:00
Damien George 1762990579 py/bc: Provide separate code-state setup funcs for bytecode and native.
mpy-cross will now generate native code based on the size of
mp_code_state_native_t, and the runtime will use this struct to calculate
the offset of the .state field.  This makes native code generation and
execution (which rely on this struct) independent to the settings
MICROPY_STACKLESS and MICROPY_PY_SYS_SETTRACE, both of which change the
size of the mp_code_state_t struct.

Fixes issue #5059.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 14:25:51 +10:00
Damien George 8e1db993cd py/asmx64: Support full range of regs in asm_x64_lea_disp_to_r64.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 14:25:51 +10:00
Damien George 6f68a8c240 tests/run-perfbench.py: Return error code if any test fails on target.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 14:06:41 +10:00
Damien George d7cf8a3b9d tests/perf_bench: Update .mpy file header to remove old unicode flag.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 14:06:41 +10:00
Damien George 2a366e183e docs/reference/mpyfiles: Remove docs about mpy flags and qstr win size.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 12:51:54 +10:00
Damien George c49d5207e9 py/persistentcode: Remove unicode feature flag from .mpy file.
Prior to this commit, even with unicode disabled .py and .mpy files could
contain unicode characters, eg by entering them directly in a string as
utf-8 encoded.

The only thing the compiler disallowed (with unicode disabled) was using
\uxxxx and \Uxxxxxxxx notation to specify a character within a string with
value >= 0x100; that would give a SyntaxError.

With this change mpy-cross will now accept \u and \U notation to insert a
character with value >= 0x100 into a string (because the -mno-unicode
option is now gone, there's no way to forbid this).  The runtime will
happily work with strings with such characters, just like it already works
with strings with characters that were utf-8 encoded directly.

This change simplifies things because there are no longer any feature
flags in .mpy files, and any bytecode .mpy will now run on any target.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 12:51:54 +10:00
Damien George b295b6f1f3 py/persistentcode: Remove obsolete comment about qstr window size.
This was made obsolete in f2040bfc7e

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 12:51:54 +10:00