Commit Graph

469 Commits

Author SHA1 Message Date
David Lechner
d42d35f56d tests/run-tests.py: Enable -X realtime option for macOS tests.
This enables the new `-X realtime` runtime option when running tests on
macOS.  This causes MicroPython to configure all threads to be high
priority so that they are allowed to use high precision timers.  This
makes tests that depend on the passage of time more likely to succeed.

CI tests that were disabled because of this are now enabled again.

Signed-off-by: David Lechner <david@pybricks.com>
2022-05-24 00:52:44 +10:00
Damien George
c1b9d2259e py/dynruntime.mk: Add basic support for armv6m architecture.
The examples/natmod features0 and features1 examples now build and run on
ARMv6-M platforms.  More complicated examples are not yet supported because
the compiler emits references to built-in functions like __aeabi_uidiv.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-23 23:01:25 +10:00
Damien George
d4d53e9e11 py/emitnative: Access qstr values using indirection table qstr_table.
This changes the native emitter to access qstr values using the qstr
indirection table qstr_table, but only when generating native code that
will be saved to a .mpy file.  This makes the resulting native code fully
static, ie it does not require any fix-ups or rewriting when it is
imported.

The performance of native code is more or less unchanged.  Benchmark
results on PYBv1.0 (using --via-mpy and --emit native) are:

N=100 M=100          baseline -> this-commit     diff      diff% (error%)
bm_chaos.py            407.16 ->     411.85 :   +4.69 =  +1.152% (+/-0.01%)
bm_fannkuch.py         100.89 ->     101.20 :   +0.31 =  +0.307% (+/-0.01%)
bm_fft.py             3521.17 ->    3441.72 :  -79.45 =  -2.256% (+/-0.00%)
bm_float.py           6707.29 ->    6644.83 :  -62.46 =  -0.931% (+/-0.00%)
bm_hexiom.py            55.91 ->      55.41 :   -0.50 =  -0.894% (+/-0.00%)
bm_nqueens.py         5343.54 ->    5326.17 :  -17.37 =  -0.325% (+/-0.00%)
bm_pidigits.py         603.89 ->     632.79 :  +28.90 =  +4.786% (+/-0.33%)
core_qstr.py            64.18 ->      64.09 :   -0.09 =  -0.140% (+/-0.01%)
core_yield_from.py     313.61 ->     311.11 :   -2.50 =  -0.797% (+/-0.03%)
misc_aes.py            654.29 ->     659.75 :   +5.46 =  +0.834% (+/-0.02%)
misc_mandel.py        4205.10 ->    4272.08 :  +66.98 =  +1.593% (+/-0.01%)
misc_pystone.py       3077.79 ->    3128.39 :  +50.60 =  +1.644% (+/-0.01%)
misc_raytrace.py       388.45 ->     393.71 :   +5.26 =  +1.354% (+/-0.01%)
viper_call0.py         576.83 ->     566.76 :  -10.07 =  -1.746% (+/-0.05%)
viper_call1a.py        550.39 ->     540.12 :  -10.27 =  -1.866% (+/-0.11%)
viper_call1b.py        438.32 ->     432.09 :   -6.23 =  -1.421% (+/-0.11%)
viper_call1c.py        442.96 ->     436.11 :   -6.85 =  -1.546% (+/-0.08%)
viper_call2a.py        536.31 ->     527.37 :   -8.94 =  -1.667% (+/-0.04%)
viper_call2b.py        378.99 ->     377.50 :   -1.49 =  -0.393% (+/-0.08%)

Signed-off-by: Damien George <damien@micropython.org>
2022-05-23 15:43:06 +10:00
Damien George
1fb01bd6c5 py/emitnative: Put a pointer to the native prelude in child_table array.
Some architectures (like esp32 xtensa) cannot read byte-wise from
executable memory.  This means the prelude for native functions -- which is
usually located after the machine code for the native function -- must be
placed in separate memory that can be read byte-wise.  Prior to this commit
this was achieved by enabling N_PRELUDE_AS_BYTES_OBJ for the emitter and
MICROPY_EMIT_NATIVE_PRELUDE_AS_BYTES_OBJ for the runtime.  The prelude was
then placed in a bytes object, pointed to by the module's constant table.

This behaviour is changed by this commit so that a pointer to the prelude
is stored either in mp_obj_fun_bc_t.child_table, or in
mp_obj_fun_bc_t.child_table[num_children] if num_children > 0.  The reasons
for doing this are:

1. It decouples the native emitter from runtime requirements, the emitted
   code no longer needs to know if the system it runs on can/can't read
   byte-wise from executable memory.

2. It makes all ports have the same emitter behaviour, there is no longer
   the N_PRELUDE_AS_BYTES_OBJ option.

3. The module's constant table is now used only for actual constants in the
   Python code.  This allows further optimisations to be done with the
   constants (eg constant deduplication).

Code size change for those ports that enable the native emitter:
   unix x64:   +80 +0.015%
      stm32:   +24 +0.004% PYBV10
    esp8266:   +88 +0.013% GENERIC
      esp32:   -20 -0.002% GENERIC[incl -112(data)]
        rp2:   +32 +0.005% PICO

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 16:44:49 +10:00
Damien George
8725a32f41 tools/ci.sh: For esp8266 CI install esptool 3.3.1.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 16:01:56 +10:00
Damien George
f8d3956c63 tools/ci.sh: Run full test suite on stackless and settrace builds.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 14:25:51 +10:00
Damien George
c49d5207e9 py/persistentcode: Remove unicode feature flag from .mpy file.
Prior to this commit, even with unicode disabled .py and .mpy files could
contain unicode characters, eg by entering them directly in a string as
utf-8 encoded.

The only thing the compiler disallowed (with unicode disabled) was using
\uxxxx and \Uxxxxxxxx notation to specify a character within a string with
value >= 0x100; that would give a SyntaxError.

With this change mpy-cross will now accept \u and \U notation to insert a
character with value >= 0x100 into a string (because the -mno-unicode
option is now gone, there's no way to forbid this).  The runtime will
happily work with strings with such characters, just like it already works
with strings with characters that were utf-8 encoded directly.

This change simplifies things because there are no longer any feature
flags in .mpy files, and any bytecode .mpy will now run on any target.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 12:51:54 +10:00
Damien George
2ed4f7a130 tools/mpy_ld.py: Remove obsolete QSTR_WINDOW_SIZE constant.
This was made obsolete in f2040bfc7e

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 12:51:54 +10:00
Takeo Takahashi
e3c880a569 tools/autobuild: Update for ports/renesas-ra.
* Add build_renesas_ra_boards call in autobuild.sh
* Add build_renesas_ra_boards function to generate firmware.hex.

Signed-off-by: Takeo Takahashi <takeo.takahashi.xv@renesas.com>
2022-05-06 17:15:21 +09:00
Damien George
aec8db3cd1 tools/uncrustify: Make sure a space exists after a comma.
This will add a space after a comma if it doesn't have one, but will allow
more than one space if the spaces are already there.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-05 13:30:40 +10:00
Takeo Takahashi
3a941cce51 tools/ci.sh: Update for ports/renesas-ra.
Signed-off-by: Takeo Takahashi <takeo.takahashi.xv@renesas.com>
2022-04-29 12:29:07 +09:00
Damien George
81c9219375 tools/ci.sh: Don't run uasyncio_gather test on macOS.
The timing on macOS makes this test unreliable.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-21 14:25:17 +10:00
Damien George
fb10d15d47 tools/codeformat.py: Include more ports .c/.h files in formatting.
This adds a rule to cover all ports/**/*.[ch] file to the code formatting
list.  Explicit exclusions are also added for code in ports/ which is third
party, or which requires a lot of reformatting.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-20 16:02:11 +10:00
Damien George
71a64545a8 tools/codeformat.py: Remove ports/unix/*.py from exclusion list.
These .py files were removed from the .gitignore in commit
3c2aa5ff93

Signed-off-by: Damien George <damien@micropython.org>
2022-04-20 16:01:11 +10:00
Damien George
07f526067e tools/mpy-tool.py: Intern more strings when freezing.
Signed-off-by: Damien George <damien@micropython.org>
2022-04-14 23:52:14 +10:00
Damien George
40d431d1bb tools/mpy-tool.py: Optimise freezing of str when str data is a qstr.
Signed-off-by: Damien George <damien@micropython.org>
2022-04-14 23:52:14 +10:00
Damien George
e647966fc9 tools/mpy-tool.py: Make global qstr list a dedicated class.
Signed-off-by: Damien George <damien@micropython.org>
2022-04-14 23:52:14 +10:00
Damien George
dfc6c6299c tools/mpy-tool.py: Optimise freezing of empty str and bytes objects.
Signed-off-by: Damien George <damien@micropython.org>
2022-04-14 23:52:14 +10:00
Damien George
9c8a56343f tools/mpy-tool.py: Optimise freezing of ints that can fit a small int.
Signed-off-by: Damien George <damien@micropython.org>
2022-04-14 23:52:14 +10:00
Damien George
68b3aeeb57 tools/mpy-tool.py: Support freezing tuples and other consts.
This also simplifies how constants are frozen.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-14 23:52:14 +10:00
Damien George
2a075cc8a9 tools/mpy-tool.py: Support loading tuples from .mpy files.
Signed-off-by: Damien George <damien@micropython.org>
2022-04-14 23:52:14 +10:00
Damien George
42d0bd2c17 py/persistentcode: Define enum values for obj types instead of letters.
To keep the separate parts of the code that use these values in sync.  And
make it easier to add new object types.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-14 22:44:04 +10:00
Damien George
be25e333df esp32/boards: Add test manifest to test freezing during build.
Includes tests for freezing @micropython.native and @micropython.viper
code.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-14 14:06:38 +10:00
Rob Knegjens
d3d862769a tools/mpremote: Add option to mount cmd to allow "unsafe" symlinks.
Specifying the option `--unsafe-links` (or `-l`) to `mpremote mount` will
allow symlinks to be followed in the local directory that point outside of
the base directory path.

For the unsafe case the `path_check()` method of `PyboardCommand` still
checks for a common path but without expanding symlinks.  While this check
is currently redundant, it makes the purpose of the method clearer for
possible future uses or extensions.
2022-04-11 15:15:04 +10:00
Rob Knegjens
56978c3dde tools/mpremote: Show progress indicator when copying large files.
When copying large files (> 2048 bytes) to or from a device with
`mpremote cp` a progress bar and percentage counter are temporarily shown.
2022-04-11 15:04:20 +10:00
Damien George
b59989f40e rp2/Makefile: Add mbedtls to GIT_SUBMODULES list and use it in CI func.
Signed-off-by: Damien George <damien@micropython.org>
2022-04-11 13:05:25 +10:00
Damien George
86e6744ff5 rp2: Enable ucryptolib, using mbedtls for backend functions.
Signed-off-by: Damien George <damien@micropython.org>
2022-04-04 23:15:01 +10:00
Andrew Leech
79c05bd522 tools/mpremote: Improve reliability of mount after soft reboot.
With the existing code problems can occur with remounting, the "if t -
t_last_activity > QUIET_TIMEOUT:" check can be triggered early before the
REPL string comes through, meaning that the remount doesn't happen.

On certain boards the "MPY: soft reboot" line comes through immediately
(getting the routine past initial timeout) but then there's a slightly
longer delay while the board restarts before it prints out the startup
header and the REPL prompt.

This commit adds some extra pattern monitoring during the timeout loop to
track the state if a soft restart is actually started.
2022-04-04 16:49:38 +10:00
Waterlens
4c252ae067 tools/mpremote: Allow running mpremote with python -m.
This is helpful because some scripts are likely to use mpremote with a
specific python path.
2022-03-30 15:29:26 +11:00
Damien George
35dbde163a tools/mpremote: Support any prompt string when detecting soft reset.
The prompt may be changed by sys.ps1.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-28 16:47:50 +11:00
Damien George
6d11c69983 py: Change jump-if-x-or-pop opcodes to have unsigned offset argument.
These jumps are always forwards, and it's more efficient in the VM to
decode an unsigned argument.  These opcodes are already optimised versions
of the sequence "dup-top pop-jump-if-x pop" so it doesn't hurt generality
to optimise them further.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-28 15:43:09 +11:00
Damien George
538c3c0a55 py: Change jump opcodes to emit 1-byte jump offset when possible.
This commit introduces changes:

- All jump opcodes are changed to have variable length arguments, of either
  1 or 2 bytes (previously they were fixed at 2 bytes).  In most cases only
  1 byte is needed to encode the short jump offset, saving bytecode size.

- The bytecode emitter now selects 1 byte jump arguments when the jump
  offset is guaranteed to fit in 1 byte.  This is achieved by checking if
  the code size changed during the last pass and, if it did (if it shrank),
  then requesting that the compiler make another pass to get the correct
  offsets of the now-smaller code.  This can continue multiple times until
  the code stabilises.  The code can only ever shrink so this iteration is
  guaranteed to complete.  In most cases no extra passes are needed, the
  original 4 passes are enough to get it right by the 4th pass (because the
  2nd pass computes roughly the correct labels and the 3rd pass computes
  the correct size for the jump argument).

This change to the jump opcode encoding reduces .mpy files and RAM usage
(when bytecode is in RAM) by about 2% on average.

The performance of the VM is not impacted, at least within measurment of
the performance benchmark suite.

Code size is reduced for builds that include a decent amount of frozen
bytecode.  ARM Cortex-M builds without any frozen code increase by about
350 bytes.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-28 15:41:38 +11:00
David Lechner
e7f6b9f4f7 tools/gen-cpydiff: Skip Black fmt comments.
Since cpydiff is code used as documentation, there are cases where we may
want to use Black's `fmt: on/off/skip` comments to avoid automatic
formatting.  However, we don't want these comments to be distracting in the
generated documentation.

This rewrites the code to omit these comments when generating the docs.

Signed-off-by: David Lechner <david@pybricks.com>
2022-03-25 12:13:00 +11:00
Damien George
e316306546 stm32/mboot: Add support for reading from SD card.
Tested on PYBV10 and PYBD_SF6, with MBOOT_FSLOAD enabled and programming
new firmware from a .dfu.gz file stored on the SD card.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-22 13:33:43 +11:00
Maureen Helm
474d288e55 zephyr: Upgrade to Zephyr v3.0.0.
Updates the Zephyr port build instructions and CI to use the latest
Zephyr release tag.

Signed-off-by: Maureen Helm <maureen.helm@intel.com>
2022-03-07 16:48:35 +11:00
Damien George
bf01671a96 tools/mpremote: Bump version to 0.2.0.
Signed-off-by: Damien George <damien@micropython.org>
2022-03-07 16:02:55 +11:00
Damien George
8626dcd623 tools/ci.sh: Run performance benchmark as part of all unix test runs.
This tests that the performance benchmarks run without error.

Signed-off-by: Damien George <damien@micropython.org>
2022-02-28 19:02:58 +11:00
robert-hh
5c46721a1c tools/mpy-tool.py: Fix frozen comment generation to escape chars.
That caused the compile of frozen_content.c to fail if characters like
backslash were in a short string.  Thanks to @hippy for identifying the
spot to change.
2022-02-28 18:47:24 +11:00
Damien George
0a217624e1 tools/upip.py: Remove unused op_basename() function.
It seems this was never used, at least not since its inclusion in this
repository.

Signed-off-by: Damien George <damien@micropython.org>
2022-02-25 18:24:03 +11:00
Damien George
d6564a3159 tools/mpremote: Add "umount" command.
Signed-off-by: Damien George <damien@micropython.org>
2022-02-25 13:10:45 +11:00
Damien George
ad1f523e7e tools/mpremote: Add "resume" and "soft-reset" commands.
This makes the auto soft-reset behaviour of mpremote more logical, and now
configurable via these new commands.

Signed-off-by: Damien George <damien@micropython.org>
2022-02-25 13:09:28 +11:00
Damien George
66fc0f45c1 tools/ci.sh: Run urandom test scripts as part of native module tests.
Signed-off-by: Damien George <damien@micropython.org>
2022-02-24 18:29:02 +11:00
Damien George
73a1927fce github/workflows: Add new workflow to test .mpy file format and tools.
Signed-off-by: Damien George <damien@micropython.org>
2022-02-24 18:29:02 +11:00
Damien George
f2040bfc7e py: Rework bytecode and .mpy file format to be mostly static data.
Background: .mpy files are precompiled .py files, built using mpy-cross,
that contain compiled bytecode functions (and can also contain machine
code). The benefit of using an .mpy file over a .py file is that they are
faster to import and take less memory when importing.  They are also
smaller on disk.

But the real benefit of .mpy files comes when they are frozen into the
firmware.  This is done by loading the .mpy file during compilation of the
firmware and turning it into a set of big C data structures (the job of
mpy-tool.py), which are then compiled and downloaded into the ROM of a
device.  These C data structures can be executed in-place, ie directly from
ROM.  This makes importing even faster because there is very little to do,
and also means such frozen modules take up much less RAM (because their
bytecode stays in ROM).

The downside of frozen code is that it requires recompiling and reflashing
the entire firmware.  This can be a big barrier to entry, slows down
development time, and makes it harder to do OTA updates of frozen code
(because the whole firmware must be updated).

This commit attempts to solve this problem by providing a solution that
sits between loading .mpy files into RAM and freezing them into the
firmware.  The .mpy file format has been reworked so that it consists of
data and bytecode which is mostly static and ready to run in-place.  If
these new .mpy files are located in flash/ROM which is memory addressable,
the .mpy file can be executed (mostly) in-place.

With this approach there is still a small amount of unpacking and linking
of the .mpy file that needs to be done when it's imported, but it's still
much better than loading an .mpy from disk into RAM (although not as good
as freezing .mpy files into the firmware).

The main trick to make static .mpy files is to adjust the bytecode so any
qstrs that it references now go through a lookup table to convert from
local qstr number in the module to global qstr number in the firmware.
That means the bytecode does not need linking/rewriting of qstrs when it's
loaded.  Instead only a small qstr table needs to be built (and put in RAM)
at import time.  This means the bytecode itself is static/constant and can
be used directly if it's in addressable memory.  Also the qstr string data
in the .mpy file, and some constant object data, can be used directly.
Note that the qstr table is global to the module (ie not per function).

In more detail, in the VM what used to be (schematically):

    qst = DECODE_QSTR_VALUE;

is now (schematically):

    idx = DECODE_QSTR_INDEX;
    qst = qstr_table[idx];

That allows the bytecode to be fixed at compile time and not need
relinking/rewriting of the qstr values.  Only qstr_table needs to be linked
when the .mpy is loaded.

Incidentally, this helps to reduce the size of bytecode because what used
to be 2-byte qstr values in the bytecode are now (mostly) 1-byte indices.
If the module uses the same qstr more than two times then the bytecode is
smaller than before.

The following changes are measured for this commit compared to the
previous (the baseline):
- average 7%-9% reduction in size of .mpy files
- frozen code size is reduced by about 5%-7%
- importing .py files uses about 5% less RAM in total
- importing .mpy files uses about 4% less RAM in total
- importing .py and .mpy files takes about the same time as before

The qstr indirection in the bytecode has only a small impact on VM
performance.  For stm32 on PYBv1.0 the performance change of this commit
is:

diff of scores (higher is better)
N=100 M=100             baseline -> this-commit  diff      diff% (error%)
bm_chaos.py               371.07 ->  357.39 :  -13.68 =  -3.687% (+/-0.02%)
bm_fannkuch.py             78.72 ->   77.49 :   -1.23 =  -1.563% (+/-0.01%)
bm_fft.py                2591.73 -> 2539.28 :  -52.45 =  -2.024% (+/-0.00%)
bm_float.py              6034.93 -> 5908.30 : -126.63 =  -2.098% (+/-0.01%)
bm_hexiom.py               48.96 ->   47.93 :   -1.03 =  -2.104% (+/-0.00%)
bm_nqueens.py            4510.63 -> 4459.94 :  -50.69 =  -1.124% (+/-0.00%)
bm_pidigits.py            650.28 ->  644.96 :   -5.32 =  -0.818% (+/-0.23%)
core_import_mpy_multi.py  564.77 ->  581.49 :  +16.72 =  +2.960% (+/-0.01%)
core_import_mpy_single.py  68.67 ->   67.16 :   -1.51 =  -2.199% (+/-0.01%)
core_qstr.py               64.16 ->   64.12 :   -0.04 =  -0.062% (+/-0.00%)
core_yield_from.py        362.58 ->  354.50 :   -8.08 =  -2.228% (+/-0.00%)
misc_aes.py               429.69 ->  405.59 :  -24.10 =  -5.609% (+/-0.01%)
misc_mandel.py           3485.13 -> 3416.51 :  -68.62 =  -1.969% (+/-0.00%)
misc_pystone.py          2496.53 -> 2405.56 :  -90.97 =  -3.644% (+/-0.01%)
misc_raytrace.py          381.47 ->  374.01 :   -7.46 =  -1.956% (+/-0.01%)
viper_call0.py            576.73 ->  572.49 :   -4.24 =  -0.735% (+/-0.04%)
viper_call1a.py           550.37 ->  546.21 :   -4.16 =  -0.756% (+/-0.09%)
viper_call1b.py           438.23 ->  435.68 :   -2.55 =  -0.582% (+/-0.06%)
viper_call1c.py           442.84 ->  440.04 :   -2.80 =  -0.632% (+/-0.08%)
viper_call2a.py           536.31 ->  532.35 :   -3.96 =  -0.738% (+/-0.06%)
viper_call2b.py           382.34 ->  377.07 :   -5.27 =  -1.378% (+/-0.03%)

And for unix on x64:

diff of scores (higher is better)
N=2000 M=2000        baseline -> this-commit     diff      diff% (error%)
bm_chaos.py          13594.20 ->  13073.84 :  -520.36 =  -3.828% (+/-5.44%)
bm_fannkuch.py          60.63 ->     59.58 :    -1.05 =  -1.732% (+/-3.01%)
bm_fft.py           112009.15 -> 111603.32 :  -405.83 =  -0.362% (+/-4.03%)
bm_float.py         246202.55 -> 247923.81 : +1721.26 =  +0.699% (+/-2.79%)
bm_hexiom.py           615.65 ->    617.21 :    +1.56 =  +0.253% (+/-1.64%)
bm_nqueens.py       215807.95 -> 215600.96 :  -206.99 =  -0.096% (+/-3.52%)
bm_pidigits.py        8246.74 ->   8422.82 :  +176.08 =  +2.135% (+/-3.64%)
misc_aes.py          16133.00 ->  16452.74 :  +319.74 =  +1.982% (+/-1.50%)
misc_mandel.py      128146.69 -> 130796.43 : +2649.74 =  +2.068% (+/-3.18%)
misc_pystone.py      83811.49 ->  83124.85 :  -686.64 =  -0.819% (+/-1.03%)
misc_raytrace.py     21688.02 ->  21385.10 :  -302.92 =  -1.397% (+/-3.20%)

The code size change is (firmware with a lot of frozen code benefits the
most):

       bare-arm:  +396 +0.697%
    minimal x86: +1595 +0.979% [incl +32(data)]
       unix x64: +2408 +0.470% [incl +800(data)]
    unix nanbox: +1396 +0.309% [incl -96(data)]
          stm32: -1256 -0.318% PYBV10
         cc3200:  +288 +0.157%
        esp8266:  -260 -0.037% GENERIC
          esp32:  -216 -0.014% GENERIC[incl -1072(data)]
            nrf:  +116 +0.067% pca10040
            rp2:  -664 -0.135% PICO
           samd:  +844 +0.607% ADAFRUIT_ITSYBITSY_M4_EXPRESS

As part of this change the .mpy file format version is bumped to version 6.
And mpy-tool.py has been improved to provide a good visualisation of the
contents of .mpy files.

In summary: this commit changes the bytecode to use qstr indirection, and
reworks the .mpy file format to be simpler and allow .mpy files to be
executed in-place.  Performance is not impacted too much.  Eventually it
will be possible to store such .mpy files in a linear, read-only, memory-
mappable filesystem so they can be executed from flash/ROM.  This will
essentially be able to replace frozen code for most applications.

Signed-off-by: Damien George <damien@micropython.org>
2022-02-24 18:08:43 +11:00
Damien George
b1afbe3336 tools/ci.sh: Update IDF v4.4 build to use v4.4 tag.
Signed-off-by: Damien George <damien@micropython.org>
2022-02-24 15:23:32 +11:00
David Lechner
5d6408f8f9 tools/verifygitlog.py: Ignore line length in body if it's a URL.
This changes the git commit message line length check to ignore lines that
contain URLs, since these cannot be wrapped without breaking tools that
detect URLs and create a link.

Signed-off-by: David Lechner <david@pybricks.com>
2022-02-18 14:33:33 +11:00
Artyom Skrobov
f46a7140f5 py/qstr: Use const consistently to avoid a cast.
Originally at adafruit#4707

Signed-off-by: Artyom Skrobov <tyomitch@gmail.com>
2022-02-11 22:55:02 +11:00
Artyom Skrobov
18b1ba086c py/qstr: Separate hash and len from string data.
This allows the compiler to merge strings: e.g. "update",
"difference_update" and "symmetric_difference_update" will all point to the
same memory.

No functional change.

The size reduction depends on the number of qstrs in the build.  The change
this commit brings is:

   bare-arm:    -4 -0.007%
minimal x86:  +150 +0.092% [incl +48(data)]
   unix x64:  -608 -0.118%
unix nanbox:  -572 -0.126% [incl +32(data)]
      stm32: -1392 -0.352% PYBV10
     cc3200:  -448 -0.244%
    esp8266: -1208 -0.173% GENERIC
      esp32: -1028 -0.068% GENERIC[incl -1020(data)]
        nrf:  -440 -0.252% pca10040
        rp2: -1072 -0.217% PICO
       samd:  -368 -0.264% ADAFRUIT_ITSYBITSY_M4_EXPRESS

Performance is also improved (on bare metal at least) for the
core_import_mpy_multi.py, core_import_mpy_single.py and core_qstr.py
performance benchmarks.

Originally at adafruit#4583

Signed-off-by: Artyom Skrobov <tyomitch@gmail.com>
2022-02-11 22:52:32 +11:00
Damien George
fecfd52696 tools/mpremote: Fix special handling of ctrl-D when host FS is mounted.
Changes are:
- decision to remount local filesystem on remote device is made only if
  "MPY: soft reboot" is seen in the output after sending a ctrl-D
- a nice message is printed to the user when the remount occurs
- soft reset during raw REPL is now handled correctly

Fixes issue #7731.

Signed-off-by: Damien George <damien@micropython.org>
2022-02-07 13:21:29 +11:00
Andrew Leech
203ec8ca7f tools/mpremote: Correctly manage mounted flag during soft-reset. 2022-02-04 16:19:25 +11:00