There is no release of IDF v4.4 yet but master is now on v5.0-dev so a
specific commit must be chosen to stick to v4.4.
Signed-off-by: Damien George <damien@micropython.org>
IDF v4.4 does not have an official release so for now use the latest
master. Also remove building GENERIC with no options (all the other boards
are no-option builds), to keep CI time reasonable.
Signed-off-by: Damien George <damien@micropython.org>
This commit removes all parts of code associated with the existing
MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE optimisation option, including the
-mcache-lookup-bc option to mpy-cross.
This feature originally provided a significant performance boost for Unix,
but wasn't able to be enabled for MCU targets (due to frozen bytecode), and
added significant extra complexity to generating and distributing .mpy
files.
The equivalent performance gain is now provided by the combination of
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE (which has
been enabled on the unix port in the previous commit).
It's hard to provide precise performance numbers, but tests have been run
on a wide variety of architectures (x86-64, ARM Cortex, Aarch64, RISC-V,
xtensa) and they all generally agree on the qualitative improvements seen
by the combination of MICROPY_OPT_LOAD_ATTR_FAST_PATH and
MICROPY_OPT_MAP_LOOKUP_CACHE.
For example, on a "quiet" Linux x64 environment (i3-5010U @ 2.10GHz) the
change from CACHE_MAP_LOOKUP_IN_BYTECODE, to LOAD_ATTR_FAST_PATH combined
with MAP_LOOKUP_CACHE is:
diff of scores (higher is better)
N=2000 M=2000 bccache -> attrmapcache diff diff% (error%)
bm_chaos.py 13742.56 -> 13905.67 : +163.11 = +1.187% (+/-3.75%)
bm_fannkuch.py 60.13 -> 61.34 : +1.21 = +2.012% (+/-2.11%)
bm_fft.py 113083.20 -> 114793.68 : +1710.48 = +1.513% (+/-1.57%)
bm_float.py 256552.80 -> 243908.29 : -12644.51 = -4.929% (+/-1.90%)
bm_hexiom.py 521.93 -> 625.41 : +103.48 = +19.826% (+/-0.40%)
bm_nqueens.py 197544.25 -> 217713.12 : +20168.87 = +10.210% (+/-3.01%)
bm_pidigits.py 8072.98 -> 8198.75 : +125.77 = +1.558% (+/-3.22%)
misc_aes.py 17283.45 -> 16480.52 : -802.93 = -4.646% (+/-0.82%)
misc_mandel.py 99083.99 -> 128939.84 : +29855.85 = +30.132% (+/-5.88%)
misc_pystone.py 83860.10 -> 82592.56 : -1267.54 = -1.511% (+/-2.27%)
misc_raytrace.py 21490.40 -> 22227.23 : +736.83 = +3.429% (+/-1.88%)
This shows that the new optimisations are at least as good as the existing
inline-bytecode-caching, and are sometimes much better (because the new
ones apply caching to a wider variety of map lookups).
The new optimisations can also benefit code generated by the native
emitter, because they apply to the runtime rather than the generated code.
The improvement for the native emitter when LOAD_ATTR_FAST_PATH and
MAP_LOOKUP_CACHE are enabled is (same Linux environment as above):
diff of scores (higher is better)
N=2000 M=2000 native -> nat-attrmapcache diff diff% (error%)
bm_chaos.py 14130.62 -> 15464.68 : +1334.06 = +9.441% (+/-7.11%)
bm_fannkuch.py 74.96 -> 76.16 : +1.20 = +1.601% (+/-1.80%)
bm_fft.py 166682.99 -> 168221.86 : +1538.87 = +0.923% (+/-4.20%)
bm_float.py 233415.23 -> 265524.90 : +32109.67 = +13.756% (+/-2.57%)
bm_hexiom.py 628.59 -> 734.17 : +105.58 = +16.796% (+/-1.39%)
bm_nqueens.py 225418.44 -> 232926.45 : +7508.01 = +3.331% (+/-3.10%)
bm_pidigits.py 6322.00 -> 6379.52 : +57.52 = +0.910% (+/-5.62%)
misc_aes.py 20670.10 -> 27223.18 : +6553.08 = +31.703% (+/-1.56%)
misc_mandel.py 138221.11 -> 152014.01 : +13792.90 = +9.979% (+/-2.46%)
misc_pystone.py 85032.14 -> 105681.44 : +20649.30 = +24.284% (+/-2.25%)
misc_raytrace.py 19800.01 -> 23350.73 : +3550.72 = +17.933% (+/-2.79%)
In summary, compared to MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE, the new
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE options:
- are simpler;
- take less code size;
- are faster (generally);
- work with code generated by the native emitter;
- can be used on embedded targets with a small and constant RAM overhead;
- allow the same .mpy bytecode to run on all targets.
See #7680 for further discussion. And see also #7653 for a discussion
about simplifying mpy-cross options.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
- Moves definition of BOARD_FLASH_SIZE and other header files related to
flash configuration into the Makefile.
- Adds board specific clock_config.h.
- Adds board.h, pin_mux.h, and peripherals.h as they are
required by NXP MCU SDK in order to use our own clock_config.h.
- Renames board specific FlexSPI configuration files.
- Updates flash frequency of MIMXRT1020_EVK
- Creates separated flash_config files for QSPI NOR and
QSPI Hyper flash.
- Unifies VFS start address to be @ 1M for 1010 and 1020 boards.
- Unifies 1050EVK boards
- Adds support to both NOR and HyperFlash on boards with
both capabilities.
- Adds automatic FlexRAM initialization to start-up code based on
linker script and NXP HAL.
- Applies code formatting to all files in mimxrt port.
With this change the flash configuration is restructured and
organized. This simplifies the configuration process and
provides a better overview of each board's settings. With the integration
of clock_config.h, board.h, pin_mux.h, and peripherals.h we gain better
control of the settings and clock configurations. Furthermore the
implementation of an explicit FlexRAM setup improves the system
performance and allows for performance tuning.
Signed-off-by: Philipp Ebensberger
To keep things neat and tidy, we ensure that each file has 1 and only 1
newline at the end of each file.
Signed-off-by: David Lechner <david@pybricks.com>
This feature {x=} was introduced in Python 3.8 so needs a separate .exp
file to run on earlier Python versions.
See https://bugs.python.org/issue36817
Signed-off-by: Damien George <damien@micropython.org>
This makes it work like --no-follow and --no-exclusive using a mutex group
and dest. Although the current implementation with BooleanOptionAction is
neater it requires Python 3.9, so don't use this feature.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The --no-exclusive flag was accidentally added to the mutex group in
178198a01d.
The --soft-reset flag was accidentally added to the mutex group in
41adf17830.
These flags can be specified independently to --[no-]follow so should not
be in that mutex group.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
And using "-B" means mpy-cross is forcefully rebuilt, sometimes with
invalid CFLAGS_EXTRA options which makes the auto-build fail.
Signed-off-by: Damien George <damien@micropython.org>
The sys.stdin.buffer and sys.stdout.buffer streams work just as well (and
are just as fast) as pyb.USB_VCP on stm32 devices, so there's no need to
have the USB_VCP specialisation code, which just adds complexity.
Also, on stm32 devices with both USB and UART (or other serial interface),
if something other than the USB_VCP port is used for the serial connection
then mpremote mount will not work because it will default to reading and
writing on USB_VCP instead of the other connected serial stream.
As part of this simplification, support for a second port as input is
removed (this feature was never exposed to the user).
Signed-off-by: Damien George <damien@micropython.org>
This implements (most of) the PEP-498 spec for f-strings and is based on
https://github.com/micropython/micropython/pull/4998 by @klardotsh.
It is implemented in the lexer as a syntax translation to `str.format`:
f"{a}" --> "{}".format(a)
It also supports:
f"{a=}" --> "a={}".format(a)
This is done by extracting the arguments into a temporary vstr buffer,
then after the string has been tokenized, the lexer input queue is saved
and the contents of the temporary vstr buffer are injected into the lexer
instead.
There are four main limitations:
- raw f-strings (`fr` or `rf` prefixes) are not supported and will raise
`SyntaxError: raw f-strings are not supported`.
- literal concatenation of f-strings with adjacent strings will fail
"{}" f"{a}" --> "{}{}".format(a) (str.format will incorrectly use
the braces from the non-f-string)
f"{a}" f"{a}" --> "{}".format(a) "{}".format(a) (cannot concatenate)
- PEP-498 requires the full parser to understand the interpolated
argument, however because this entirely runs in the lexer it cannot
resolve nested braces in expressions like
f"{'}'}"
- The !r, !s, and !a conversions are not supported.
Includes tests and cpydiffs.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
For consistency with other board-level config macros that begin with
MICROPY_HW_USB.
Also allow boards in the mimxrt, nrf and samd ports to configure these
values.
Signed-off-by: Damien George <damien@micropython.org>
The firmware for Teensy 4.0, Teensy 4.1 and MIMXRT1020_EVK are created.
Users of other MIMXRT10xx_EVK boards should be able to build the firmware
themselves, they might need specific DEBUG settings.
The Makefile had to be changed in order to build the .bin file as well.
Coverage calculated by Codecov has the same reliability/deterministic
issues as Coveralls did, so the problem is likely to do with the output of
lcov/gcov, rather than the analysis and display of the data.
Switch from lcov to gcov for data generation to try and simplify this
process of computing coverage.
Signed-off-by: Damien George <damien@micropython.org>
Now a ctrl-C will not stop mpremote, rather this character will be passed
through to the attached device.
The mpremote version is also increased to 0.0.5.
Signed-off-by: Damien George <damien@micropython.org>
Using just the list of available ports, instead of a hard-coded list of
possible ports, means that all ports will be available for auto connection.
And the order that they will be attempted in will match what's printed by
"mpremote connect list" (and will be the same as before, trying ACMx before
USBx). Auto-connect will also now work on Mac, and will allow all COM
ports on Windows.
Signed-off-by: Damien George <damien@micropython.org>
Following on from ef16834887, this adds a
coverage build and running of the test suite on an ARM 32-bit Linux-based
architecture.
Signed-off-by: Damien George <damien@micropython.org>
This adds a coverage build and running of the test suite on a MIPS 32-bit
big endian architecture. It uses the feature of qemu to execute foreign
code as though it were native to the system (using qemu user mode). The
code compiled for MIPS will run under the qemu VM, but all syscalls made by
this code go to the host (Linux) system.
See related #7268 and #7273.
Signed-off-by: Damien George <damien@micropython.org>
As the new default behaviour, this allows PyDFU to be used with all
devices, not just the ones matching a specific set of VID/PID values. But
it's still possible to specify VID/PID if needed to narrow down the
selection of the USB device.
Signed-off-by: Tobias Thyrrestrup <tt@LEGO.com>
This can be treated by the linker the same as R_X86_64_REX_GOTPCRELX,
according to https://reviews.llvm.org/D18301.
Signed-off-by: Damien George <damien@micropython.org>
Since version 21.4b0, Black now processes one-line docstrings by stripping
leading and trailing spaces, and adding a padding space when needed to
break up """"; see https://github.com/psf/black/pull/1740
This commit makes the Python code in this repository conform to this rule.
This is now the default, but can be overridden with CLI `--no-exclusive`,
or constructing `Pyboard(..., exclusive=False)`.
Signed-off-by: Damien George <damien@micropython.org>
It's a bit of a pitfall with user C modules that including them in the
build does not automatically enable them. This commit changes the docs and
examples for user C modules to encourage writers of user C modules to
enable them unconditionally. This makes things simpler and covers most use
cases.
See discussion in issue #6960, and also #7086.
Signed-off-by: Damien George <damien@micropython.org>
Instead of raising a ZeroDivisionError, this tool now just skips any
elements in the DFU file that have zero size.
Signed-off-by: Damien George <damien@micropython.org>
Updates the zephyr docker image to the latest, v0.11.13. This updates CI
to use zephyr SDK v0.12.2 and GCC v10.2.0 for the zephyr port.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
Refactors the zephyr build infrastructure to build MicroPython as a
cmake target, using the recently introduced core cmake rules.
This change makes it possible to build the zephyr port like most other
zephyr applications using west or cmake directly. It simplifies building
with extra cmake arguments, such as specifying an alternate conf file or
adding an Arduino shield. It also enables building the zephyr port
anywhere in the host file system, which will allow regressing across
multiple boards with the zephyr twister script.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
This allows customising which features can be enabled in a frozen library.
e.g. `include("path.py", extra_features=True)`
in path.py:
options.defaults(standard_features=True)
if options.standard_features:
# freeze standard modules.
if options.extra_features:
# freeze extra modules.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
So that all MicroPython ports that use tinyusb use the same version. Also
requires fewer submodule checkouts when building rp2 along with other ports
that use tinyusb.
Signed-off-by: Damien George <damien@micropython.org>
The upip module is frozen into ports supporting it, and it is included in
the source tree, so there is no need to get it from PyPi. Moreover the
PyPi package referred to is an out-of-date version of upip which is
basically unrelated to our upip.py because the source is taken from a fork
of micropython-lib instead of this repository.
The main rules enforced are:
- At most 72 characters in the subject line, with a ": " in it.
- At most 75 characters per line in the body.
- No "noreply" email addresses.
Because the version included in xtensa-lx106-elf-standalone.tar.gz needs
Python 2 (and pyserial for Python 2).
Signed-off-by: Damien George <damien@micropython.org>
The -Og optimisation level produces a more realistic build, gives a better
debugging experience, and generates smaller code than -O0, allowing debug
builds to fit in flash.
This commit also assigns variables in can.c to prevent warnings when -Og is
used, and builds a board in CI with DEBUG=1 enabled.
Signed-off-by: Damien George <damien@micropython.org>
This commit adds support to pyboard.py for the new raw REPL paste mode.
Note that this new pyboard.py is fully backwards compatible with old
devices (it detects if the device supports the new raw REPL paste mode).
Signed-off-by: Damien George <damien@micropython.org>
Damien George
Dave Hylands
Jim Mussared
robert-hh
Philipp Ebensberger
Patrick Van Oosterwijck
Seon Rozenblum
David Lechner
Glenn Ruben Bakke
Tobias Thyrrestrup
Michel Bouwmans
Andrew Scheller
Frank Pilhofer
Yonatan Goldschmidt
Maureen Helm
Steve App
iabdalkader
David Michieli
Michael O'Cleirigh
Phil Howard
iTitou
Brianna Laugher
stijn