This will be used by https://micropython.org/download/ to generate the
full listing of boards and firmware files.
Optionally supports a board.md for additional customisation of the
download page, as well as deploy.md for flashing instructions.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This commit removes all parts of code associated with the existing
MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE optimisation option, including the
-mcache-lookup-bc option to mpy-cross.
This feature originally provided a significant performance boost for Unix,
but wasn't able to be enabled for MCU targets (due to frozen bytecode), and
added significant extra complexity to generating and distributing .mpy
files.
The equivalent performance gain is now provided by the combination of
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE (which has
been enabled on the unix port in the previous commit).
It's hard to provide precise performance numbers, but tests have been run
on a wide variety of architectures (x86-64, ARM Cortex, Aarch64, RISC-V,
xtensa) and they all generally agree on the qualitative improvements seen
by the combination of MICROPY_OPT_LOAD_ATTR_FAST_PATH and
MICROPY_OPT_MAP_LOOKUP_CACHE.
For example, on a "quiet" Linux x64 environment (i3-5010U @ 2.10GHz) the
change from CACHE_MAP_LOOKUP_IN_BYTECODE, to LOAD_ATTR_FAST_PATH combined
with MAP_LOOKUP_CACHE is:
diff of scores (higher is better)
N=2000 M=2000 bccache -> attrmapcache diff diff% (error%)
bm_chaos.py 13742.56 -> 13905.67 : +163.11 = +1.187% (+/-3.75%)
bm_fannkuch.py 60.13 -> 61.34 : +1.21 = +2.012% (+/-2.11%)
bm_fft.py 113083.20 -> 114793.68 : +1710.48 = +1.513% (+/-1.57%)
bm_float.py 256552.80 -> 243908.29 : -12644.51 = -4.929% (+/-1.90%)
bm_hexiom.py 521.93 -> 625.41 : +103.48 = +19.826% (+/-0.40%)
bm_nqueens.py 197544.25 -> 217713.12 : +20168.87 = +10.210% (+/-3.01%)
bm_pidigits.py 8072.98 -> 8198.75 : +125.77 = +1.558% (+/-3.22%)
misc_aes.py 17283.45 -> 16480.52 : -802.93 = -4.646% (+/-0.82%)
misc_mandel.py 99083.99 -> 128939.84 : +29855.85 = +30.132% (+/-5.88%)
misc_pystone.py 83860.10 -> 82592.56 : -1267.54 = -1.511% (+/-2.27%)
misc_raytrace.py 21490.40 -> 22227.23 : +736.83 = +3.429% (+/-1.88%)
This shows that the new optimisations are at least as good as the existing
inline-bytecode-caching, and are sometimes much better (because the new
ones apply caching to a wider variety of map lookups).
The new optimisations can also benefit code generated by the native
emitter, because they apply to the runtime rather than the generated code.
The improvement for the native emitter when LOAD_ATTR_FAST_PATH and
MAP_LOOKUP_CACHE are enabled is (same Linux environment as above):
diff of scores (higher is better)
N=2000 M=2000 native -> nat-attrmapcache diff diff% (error%)
bm_chaos.py 14130.62 -> 15464.68 : +1334.06 = +9.441% (+/-7.11%)
bm_fannkuch.py 74.96 -> 76.16 : +1.20 = +1.601% (+/-1.80%)
bm_fft.py 166682.99 -> 168221.86 : +1538.87 = +0.923% (+/-4.20%)
bm_float.py 233415.23 -> 265524.90 : +32109.67 = +13.756% (+/-2.57%)
bm_hexiom.py 628.59 -> 734.17 : +105.58 = +16.796% (+/-1.39%)
bm_nqueens.py 225418.44 -> 232926.45 : +7508.01 = +3.331% (+/-3.10%)
bm_pidigits.py 6322.00 -> 6379.52 : +57.52 = +0.910% (+/-5.62%)
misc_aes.py 20670.10 -> 27223.18 : +6553.08 = +31.703% (+/-1.56%)
misc_mandel.py 138221.11 -> 152014.01 : +13792.90 = +9.979% (+/-2.46%)
misc_pystone.py 85032.14 -> 105681.44 : +20649.30 = +24.284% (+/-2.25%)
misc_raytrace.py 19800.01 -> 23350.73 : +3550.72 = +17.933% (+/-2.79%)
In summary, compared to MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE, the new
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE options:
- are simpler;
- take less code size;
- are faster (generally);
- work with code generated by the native emitter;
- can be used on embedded targets with a small and constant RAM overhead;
- allow the same .mpy bytecode to run on all targets.
See #7680 for further discussion. And see also #7653 for a discussion
about simplifying mpy-cross options.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The zephyr port doesn't support SoftI2C so it's not enabled, and the legacy
I2C constructor check can be removed.
Signed-off-by: Damien George <damien@micropython.org>
To keep things neat and tidy, we ensure that each file has 1 and only 1
newline at the end of each file.
Signed-off-by: David Lechner <david@pybricks.com>
This function can be used to enable and disable the DC/DC converter with or
without the Bluetooth stack enabled. It can also be used to query the
current state of the DC/DC.
This commit also adds a definition of ARRAY_SIZE needed by nrfx HAL-layer.
extmod/vfs_lfs.c needs to resolve `mp_hal_time_ns()` in order to calculate
a timestamp from 1970 epoch. A wall clock is not available in the nrf
port, hence the function is implemented to resolve compilation linkage
error. The function always return 0.
Set the default manifest to "modules/manifest.py". This includes files
from the folder "modules/scripts". The manifest default value is overriden
by all nrf51 boards that have SoftDevice present (SD=s110) to save flash.
Also add "modules/manifest.py" which is set to freeze
"modules/scripts/_mkfs.py".
Add a helper script _mkfs.py which automatically formats the file system if
nrf.Flash() is located and a VFS file system has been included in the
compilation.
The precedence is: first LFS1, LFS2 then FAT.
Update the Makefile to handle FROZEN_MANIFEST, and the README with some
small samples on how to use freeze manifests. And add BOARD_DIR to the
Makefile which can be referenced in boards/<board>/mpconfigboard.mk to
include a board specific manifest.
Enable the following features for all boards except
nrf51 boards with SoftDevice present:
- MICROPY_VFS
- MICROPY_PY_NRF
- MICROPY_HW_ENABLE_INTERNAL_FLASH_STORAGE
Add posibility to override linker script "_fs_size" from make by adding the
FS_SIZE parameter. The syntax of value is linker script syntax. For
example, the value of 131072 bytes can be written as 128K like this:
FS_SIZE=128K.
If not set, default value for "_fs_size" from linker script will be used.
Disable MICROPY_FATFS_MULTI_PARTITION configuration because there is no
partition table in the flash for FATFS to read.
Also, set MICROPY_FATFS_MAX_SS to the size of a flash page. For nrf51 the
value 1024 is set. For nrf52/nrf91 the value 4096 is set.
This documents parameters that can be passed to make to enable a specific
file system to included in the build. Also, document the Makefile override
parameter "FS_SIZE" that can be used to tune the size of the flash region
to use as internal flash file system.
Update flash.c to also be compiled in when
MICROPY_HW_ENABLE_INTERNAL_FLASH_STORAGE is enabled and SoftDevice is
present.
Update bluetooth/ble_drv.c to forward flash events to flash.c when
MICROPY_HW_ENABLE_INTERNAL_FLASH_STORAGE is enabled.
This commit adds the "nrf" module for port specific modules and objects.
Included in it is the "Flash" object which exposes a block device
implementation to access internal SoC flash.
Thanks to @aykevl aka Ayke van Laethem for the initial implementation.
Calculate the unused flash area on the target device. The values will be
exposed by _unused_flash_start and _unused_flash_length. The start address
and the length are not aligned to either word or pages.
For consistency with other board-level config macros that begin with
MICROPY_HW_USB.
Also allow boards in the mimxrt, nrf and samd ports to configure these
values.
Signed-off-by: Damien George <damien@micropython.org>
This moves mp_pending_exception from mp_state_vm_t to mp_state_thread_t.
This allows exceptions to be scheduled on a specific thread.
Signed-off-by: David Lechner <david@pybricks.com>
This commit adds a few math functions to the source list in the Makefile,
and implements the log2f function, so that ulab can be compiled on the nrf
boards. It also addresses part of #5162.
The "word" referred to by BYTES_PER_WORD is actually the size of mp_obj_t
which is not always the same as the size of a pointer on the target
architecture. So rename this config value to better reflect what it
measures, and also prefix it with MP_.
For uses of BYTES_PER_WORD in setting the stack limit this has been
changed to sizeof(void *), because the stack usually grows with
machine-word sized values (eg an nlr_buf_t has many machine words in it).
Signed-off-by: Damien George <damien@micropython.org>
To simplify config, there's no need to specify MP_PLAT_PRINT_STRN if it's
the same as the default definition in py/mpconfig.h.
Signed-off-by: Damien George <damien@micropython.org>
Add "make submodules" to commands when building for the first time.
Otherwise, on a first time build, the submodules have not been checked out
and a lot of `fatal error: nrfx.h: No such file or directory` errors are
printed.
The nrf52840-mdk-usb-dongle and pca10050 comes with a pre-flashed
bootloader (OpenBootloader).
This commit updates the boards "mpconfigboard.mk" to use DFU as
default flashing method and set the corresponding BOOTLOADER
settings such that nrf52840_open_bootloader_1.2.x.ld linker
script is used.
The default DFU flashing method can be disabled by issuing "DFU=0"
when invoking make. This will lead to "segger" being used as default
flashing tool. When using "DFU=0", the linker scripts will not
compensate for any MBR and Bootloader region being present, and might
overwrite them if they were present.
The commit also removes the custom linker script specific to
nrf52840-mdk-usb-dongle as it now points to a generic.
Updated nrf52840-mdk-usb-dongle's README.md to be more clear on
how to deploy the built firmware.
The port README.md has also been updated. In the list of target
boards a new column has been added to indicate which bootloader
is present on the target board. And for consistency, changed all
examples in the README.md to use "deploy" instead of "flash".
An additional Makefile parameter NRFUTIL_PORT can be set in order
to define the serial port to used for the DFU (Default: /dev/ttyACM0).
The "nrfutil" that is used as flasher towards OpenBootloader is
available for installation through Python "pip".
In case of SD=s140, SoftDevice ID 0xB6 is passed to nrfutil's package
generation which corresponds to SoftDevice s140 v6.1.1.
Add the option for "mpconfigboard.mk" to define whether the
board hosts a bootloader or not. The BOOTLOADER make variable
must be set to the name of the bootloader.
When the BOOTLOADER name is set it is also required to supply
the BOOTLOADER_VERSION_MAJOR and the BOOTLOADER_VERSION_MINOR
from the "mpconfigboards.mk". These will be used to resolve which
bootloader linker script that should be passed to the linker.
The BOOTLOADER section also supplies the C-compiler with
BOOTLOADER_<bootloader name>=<version major><version minor>
as a compiler define. This is for future use in case a bootloader
needs to do modification to the startup files or similar (like
setting the VTOR specific to a version of a bootloader).
Adding variables that can be set from other linker scripts:
- _bootloader_head_size:
Bootloader flash offset in front of the application.
- _bootloader_tail_size:
Bootloader offset from the tail of the flash.
In case the bootloader is located at the end.
- _bootloader_head_ram_size:
Bootloader RAM usage in front of the application.
Updated calculations of application flash and RAM.
Hardware I2C implementations must provide a .init() protocol method if they
want to support reconfiguration. Otherwise the default is that i2c.init()
raises an OSError (currently the case for all ports).
mp_machine_soft_i2c_locals_dict is renamed to mp_machine_i2c_locals_dict to
match the generic SPI bindings.
Fixes issue #6623 (where calling .init() on a HW I2C would crash).
Signed-off-by: Damien George <damien@micropython.org>
The last argument of TUD_CDC_DESCRIPTOR() is the endpoint size (or
wMaxPacketSize), not the CDC RX buffer size (which can be larger than the
endpoint size).
Signed-off-by: Damien George <damien@micropython.org>
With a warning that this way of constructing software I2C/SPI is
deprecated. The check and warning will be removed in a future release.
This should help existing code to migrate to the new SoftI2C/SoftSPI types.
Signed-off-by: Damien George <damien@micropython.org>
Previous commits removed the ability for one I2C/SPI constructor to
construct both software- or hardware-based peripheral instances. Such
construction is now split to explicit soft and non-soft types.
This commit makes both types available in all ports that previously could
create both software and hardware peripherals: machine.I2C and machine.SPI
construct hardware instances, while machine.SoftI2C and machine.SoftSPI
create software instances.
This is a breaking change for use of software-based I2C and SPI. Code that
constructed I2C/SPI peripherals in the following way will need to be
changed:
machine.I2C(-1, ...) -> machine.SoftI2C(...)
machine.I2C(scl=scl, sda=sda) -> machine.SoftI2C(scl=scl, sda=sda)
machine.SPI(-1, ...) -> machine.SoftSPI(...)
machine.SPI(sck=sck, mosi=mosi, miso=miso)
-> machine.SoftSPI(sck=sck, mosi=mosi, miso=miso)
Code which uses machine.I2C and machine.SPI classes to access hardware
peripherals does not need to change.
Signed-off-by: Damien George <damien@micropython.org>
The SoftI2C constructor is now used soley to create SoftI2C instances, it
can no longer delegate to create a hardware-based I2C instance.
Signed-off-by: Damien George <damien@micropython.org>
Also rename machine_i2c_type to mp_machine_soft_i2c_type. These changes
make it clear that it's a soft-I2C implementation, and match SoftSPI.
Signed-off-by: Damien George <damien@micropython.org>
The mpconfigport.h file is an internal header and should only ever be
included once by mpconfig.h.
Signed-off-by: Damien George <damien@micropython.org>
Updating to Black v20.8b1 there are two changes that affect the code in
this repository:
- If there is a trailing comma in a list (eg [], () or function call) then
that list is now written out with one line per element. So remove such
trailing commas where the list should stay on one line.
- Spaces at the start of """ doc strings are removed.
Signed-off-by: Damien George <damien@micropython.org>
Enabling the following features for all targets, except for nrf51
targets compiled to be used with SoftDevice:
- MICROPY_PY_ARRAY_SLICE_ASSIGN
- MICROPY_PY_SYS_STDFILES
- MICROPY_PY_UBINASCII
Splitting mpconfigport.h into multiple device specific
files in order to facilitate variations between devices.
Due to the fact that the devices might have variations in
features and also variations in flash size it makes sense
that some devices offers more functionality than others
without being limited by restricted devices.
For example more micropython features can be activated for
nrf52840 with 1MB flash, compared to nrf51 with 256KB.
This commit adds time.ticks_ms/us support using RTC1 as the timebase. It
also adds the time.ticks_add/diff helper functions. This feature can be
enabled using MICROPY_PY_TIME_TICKS. If disabled the system uses the
legacy sleep methods and does not have any ticks functions.
In addition support for MICROPY_EVENT_POLL_HOOK was added to the
time.sleep_ms(x) function, making this function more power efficient and
allows support for select.poll/asyncio. To support this, the RTC's CCR0
was used to schedule a ~1msec event to wakeup the CPU.
Some important notes about the RTC timebase:
- Since the granularity of RTC1's ticks are approx 30usec, time.ticks_us is
not perfect, does not have 1us resolution, but is otherwise quite usable.
For tighter measurments the ticker's 1MHz counter should be used.
- time.ticks_ms(x) should *not* be called in an IRQ with higher prio than
the RTC overflow irq (3). If so it introduces a race condition and
possibly leads to wrong tick calculations.
See #6171 and #6202.
When compiling for debug (-O0) the .text segment cannot fit the flash
region when MICROPY_ROM_TEXT_COMPRESSION=1, because the compiler does not
optimise away the large if-else chain used to select the correct compressed
string.
This commit enforces MICROPY_ROM_TEXT_COMPRESSION=0 when compiling for
debug (DEBUG=1).
The storage space of the advertisement name is not declared static, leading
to a random advertisement name. This commit fixes the issue by declaring
it static.
The Bluetooth link gets disconnected when connecting from a PC after 30-40
seconds. This commit adds handling of the data length update request. The
data length parameter pointer is set to NULL in the reply, letting the
SoftDevice automatically set values and use them in the data length update
procedure.
mp_keyboard_interrupt() triggers a compiler error because the function is
implicitly declared. This commit adds "py/runtime.h" to the includes.
Fixes issue #5732.
Commit 6cea369b89 updated the TinyUSB
submodule to a version based on nrfx v2.0.0. This commit updates the nrf
port to work with the latest TinyUSB and nrfx v2.0.0.
Because it can confuse older versions of gcc. Instead use the correct
instruction for Thumb vs Thumb-2 (sub vs subs) so the assembler emits the
2-byte instruction.
Related to commit 1aa9ff9141.
Note: the uncrustify configuration is explicitly set to 'add' instead of
'force' in order not to alter the comments which use extra spaces after //
as a means of indenting text for clarity.
Now that error string compression is supported it's more important to have
consistent error string formatting (eg all lowercase English words,
consistent contractions). This commit cleans up some of the strings to
make them more consistent.
This function is not used by the core but having it as part of the build
allows it to be used by user C modules, or board extensions. The linker
won't include it in the final firmware if it remains unused.
This string is recognised by uncrustify, to disable formatting in the
region marked by these comments. This is necessary in the qstrdef*.h files
to prevent modification of the strings within the Q(...). In other places
it is used to prevent excessive reformatting that would make the code less
readable.
The "random" module no longer uses the hardware RNG (the extmod version of
this module has a pseudo-random number generator), so the config option
MICROPY_PY_RANDOM_HW_RNG is no longer meaningful. This commit replaces it
with MICROPY_HW_ENABLE_RNG, which controls whether the hardware RNG is
included in the build.
This provides a more consistent C-level API to raise exceptions, ie moving
away from nlr_raise towards mp_raise_XXX. It also reduces code size by a
small amount on some ports.
If the exception doesn't need printf-style formatting then calling
mp_raise_msg is more efficient. Also shorten exception messages to match
style in core and other ports.
For the 3 ports that already make use of this feature (stm32, nrf and
teensy) this doesn't make any difference, it just allows to disable it from
now on.
For other ports that use pyexec, this decreases code size because the debug
printing code is dead (it can't be enabled) but the compiler can't deduce
that, so code is still emitted.
This commit removes the Makefile-level MICROPY_FATFS config and moves the
MICROPY_VFS_FAT config to the Makefile level to replace it. It also moves
the include of the oofatfs source files in the build from each port to a
central place in extmod/extmod.mk.
For a port to enabled VFS FAT support it should now set MICROPY_VFS_FAT=1
at the level of the Makefile. This will include the relevant oofatfs files
in the build and set MICROPY_VFS_FAT=1 at the C (preprocessor) level.
This commit implements automatic module weak links for all built-in
modules, by searching for "ufoo" in the built-in module list if "foo"
cannot be found. This means that all modules named "ufoo" are always
available as "foo". Also, a port can no longer add any other weak links,
which makes strict the definition of a weak link.
It saves some code size (about 100-200 bytes) on ports that previously had
lots of weak links.
Some changes from the previous behaviour:
- It doesn't intern the non-u module names (eg "foo" is not interned),
which saves code size, but will mean that "import foo" creates a new qstr
(namely "foo") in RAM (unless the importing module is frozen).
- help('modules') no longer lists non-u module names, only the u-variants;
this reduces duplication in the help listing.
Weak links are effectively the same as having a set of symbolic links on
the filesystem that is searched last. So an "import foo" will search
built-in modules first, then all paths in sys.path, then weak links last,
importing "ufoo" if it exists. Thus a file called "foo.py" somewhere in
sys.path will still have precedence over the weak link of "foo" to "ufoo".
See issues: #1740, #4449, #5229, #5241.
This patch add basic building blocks for nrf9P60.
It also includes a secure bootloader which forwards all
possible peripherals that are user selectable to become
non-secure. After configuring Flash, RAM and peripherals
the secure bootloader will jump to the non-secure domain
where MicroPython is placed.
The minimum size of a secure boot has to be a flash
block of 32Kb, hence why the linker scripts are
offsetting the main application this much.
The RAM offset is set to 128K, to allow for later
integration of Nordic Semiconductor's BSD socket
library which reserves the range 0x20010000 - 0x2001FFFF.
Add support for pca10059 with REPL over tinyusb USB CDC.
The board also includes a board specific module that will
recover UICR->REGOUT0 in case this has been erased.
This initial support does not preserve any existing bootloader
on the pca10090 in case this was present, and expects to use all
available flash on the device.
Add nrf-port finyusb driver files. USB CDC can be activated
by board configuration files using the MICROPY_HW_USB_CDC.
Updating BLE driver, Makefile, nrfx-glue and main.c to plug
in the tinyusb stack.
The the nrfx driver is aware of chip specific registers, while
the raw HAL abstraction is not. This driver enables use of NVMC
in non-secure domain for nrf9160.
This patch moves the check for MICROPY_PY_MACHINE_TEMP to come
before the inclusion of nrf_temp.h. The nrf_temp.h depends on
the NRF_TEMP_Type which might not be defined for all nRF devices.
It was previously not taking into account that the list of pins was sparse,
so using the wrong index. The boards/X/pins.csv was generating the wrong
data for machine.Pin.board.
As part of this fix rename the variables to make it more clear what the
list contains (only board pins).
mp_compile no longer takes an emit_opt argument, rather this setting is now
provided by the global default_emit_opt variable.
Now, when -X emit=native is passed as a command-line option, the emitter
will be set for all compiled modules (included imports), not just the
top-level script.
In the future there could be a way to also set this variable from a script.
Fixes issue #4267.
Change static LED functions to lowercase names, and trim down source code
lines for variants of MICROPY_HW_LED_COUNT. Also rename configuration for
MICROPY_HW_LEDx_LEVEL to MICROPY_HW_LEDx_PULLUP to align with global PULLUP
configuration.
- IBK-BLYST-NANO: Breakout board
- IDK-BLYST-NANO: DevKit board with builtin IDAP-M CMSIS-DAP Debug JTAG,
RGB led
- BLUEIO-TAG-EVIM: Sensor tag board (environmental sensor
(T, H, P, Air quality) + 9 axis motion sensor)
Also, the LED module has been updated to support individual base level
configuration of each LED. If set, this will be used instead of the
common configuration, MICROPY_HW_LED_PULLUP. The new configuration,
MICROPY_HW_LEDX_LEVEL, where X is the LED number can be used to set
the base level of the specific LED.
The stm32 and nrf ports already had the behaviour that they would first
check if the script exists before executing it, and this patch makes all
other ports work the same way. This helps when developing apps because
it's hard to tell (when unconditionally trying to execute the scripts) if
the resulting OSError at boot up comes from missing boot.py or main.py, or
from some other error. And it's not really an error if these scripts don't
exist.
Adding section about on how to disable use of the linker flag
-flto, by setting the LTO=0 argument to Make. Also, added a
note on recommended toolchains to use that works well with
LTO enabled.
Removing linker script for nrf52840 s140 v6.0.0 as pca10056
target board now points to the new v6.1.1. Also, removing the
entry from the download_ble_stack.sh script.
Removing linker script for nrf52832 s132 v6.0.0 as all target
boards now points to the new v6.1.1. Also, removing the entry
from the download_ble_stack.sh script.
