This commit fixes the behaviour of socket.getaddrinfo on the ESP32 so it
raises an OSError when the name resolution fails instead of returning a []
or a resolution for 0.0.0.0.
Tests are added (generic and ESP32-specific) to verify behaviour consistent
with CPython, modulo the different types of exceptions per MicroPython
documentation.
The ones that are moved out of iRAM should not need to be there, because
either they call functions in iROM (eg mp_hal_stdout_tx_str), or they are
only ever called from a function in iROM and not from an interrupt (eg
ets_esf_free_bufs).
This frees up about 800 bytes of iRAM.
The zephyr build system supports merging application-level board
configurations, so there is no need to reproduce this functionality in
MicroPython.
If CONF_FILE is not explicitly set, then the zephyr build system looks for
prj.conf in the application directory. Therefore we rename the MicroPython
prj_base.conf to prj.conf.
Furthermore, if the zephyr build system finds boards/$(BOARD).conf in the
application directory, it merges that configuration with prj.conf.
Therefore we rename all the MicroPython board .conf files and move them
into a boards/ directory.
The minimal configuration, prj_minimal.conf, is left in the application
directory because it is used as an explicitly set CONF_FILE in
make-minimal.
This commit adds several small items to improve the support for OTA
updates on an esp32:
- a partition table for 4MB flash modules that has two OTA partitions ready
to go to do updates
- a GENERIC_OTA board that uses that partition table and that enables
automatic roll-back in the bootloader
- a new esp32.Partition.mark_app_valid_cancel_rollback() class-method to
signal that the boot is successful and should not be rolled back at the
next reset
- an automated test for doing an OTA update
- documentation updates
GPIO interrupts can occur when the flash ROM cache is in use and so the
GPIO interrupt handler must be in iRAM. This commit moves the handler to
iRAM, and also moves mp_sched_schedule to iRAM which is called by
pin_intr_handler.
As part of this fix the Pin class can no longer support hard=True in the
Pin.irq() method, because the VM and runtime are too big to put in iRAM.
Fixes#5714.
No functionality change is intended with this commit, it just consolidates
the separate implementations of GC helper code to the lib/utils/ directory
as a general set of helper functions useful for any port. This reduces
duplication of code, and makes it easier for future ports or embedders to
get the GC implementation correct.
Ports should now link against gchelper_native.c and either gchelper_m0.s or
gchelper_m3.s (currently only Cortex-M is supported but other architectures
can follow), or use the fallback gchelper_generic.c which will work on
x86/x64/ARM.
The gc_helper_get_sp function from gchelper_m3.s is not really GC related
and was only used by cc3200, so it has been moved to that port and renamed
to cortex_m3_get_sp.
But only when bluetooth is enabled, i.e. if building the dev or coverage
variants, and we have libusb available.
Update travis to match, i.e. specify the variant when doing
`make submodules`.
This commit adds full support to the unix port for Bluetooth using the
common extmod/modbluetooth Python bindings. This uses the libusb HCI
transport, which supports many common USB BT adaptors.
This change is made for two reasons:
1. A 3rd-party library (eg berkeley-db-1.xx, axtls) may use the system
provided errno for certain errors, and yet MicroPython stream objects
that it calls will be using the internal mp_stream_errno. So if the
library returns an error it is not known whether the corresponding errno
code is stored in the system errno or mp_stream_errno. Using the system
errno in all cases (eg in the mp_stream_posix_XXX wrappers) fixes this
ambiguity.
2. For systems that have threading the system-provided errno should always
be used because the errno value is thread-local.
For systems that do not have an errno, the new lib/embed/__errno.c file is
provided.
Note: the uncrustify configuration is explicitly set to 'add' instead of
'force' in order not to alter the comments which use extra spaces after //
as a means of indenting text for clarity.
This commit consolidates a number of check_esp_err functions that check
whether an ESP-IDF return code is OK and raises an exception if not. The
exception raised is an OSError with the error code as the first argument
(negative if it's ESP-IDF specific) and the ESP-IDF error string as the
second argument.
This commit also fixes esp32.Partition.set_boot to use check_esp_err, and
uses that function for a unit test.
This commit adds an idf_heap_info(capabilities) method to the esp32 module
which returns info about the ESP-IDF heaps. It's useful to get a bit of a
picture of what's going on when code fails because ESP-IDF can't allocate
memory anymore. Includes documentation and a test.
In mboot, the ability to override the USB vendor/product id's was added
back in 5688c9ba09. However, when the main
firmware is turned into a DFU file the default VID/PID are used there.
pydfu.py doesn't care about this but dfu-util does and prevents its use
when the VID/PID don't match.
This commit exposes BOOTLOADER_DFU_USB_VID/PID as make variables, for use
on either command line or mpconfigboard.mk, to set VID/PID in both mboot
and DFU files.
Add -Wdouble-promotion and -Wfloat-conversion for most ports to ban out
implicit floating point conversions, and add extra Travis builds using
MICROPY_FLOAT_IMPL_FLOAT to uncover warnings which weren't found
previously. For the unix port -Wsign-comparison is added as well but only
there since only clang supports this but gcc doesn't.
Initially some of these were found building the unix coverage variant on
MacOS because that build uses clang and has -Wdouble-promotion enabled, and
clang performs more vigorous promotion checks than gcc. Additionally the
codebase has been compiled with clang and msvc (the latter with warning
level 3), and with MICROPY_FLOAT_IMPL_FLOAT to find the rest of the
conversions.
Fixes are implemented either as explicit casts, or by using the correct
type, or by using one of the utility functions to handle floating point
casting; these have been moved from nativeglue.c to the public API.
Now that error string compression is supported it's more important to have
consistent error string formatting (eg all lowercase English words,
consistent contractions). This commit cleans up some of the strings to
make them more consistent.
This macro is used to implement global serialisation, typically by
disabling IRQs. On the unix port, if threading is enabled, use the
existing thread mutex (that protects the thread list structure) for this
purpose. Other places in the code (eg the scheduler) assume this macro
will provide serialisation.
Based on eg 1e6fd9f2b4, it's understood that
the intention for unix builds is that regular builds disable assert, but
the coverage build should set -O0 and enable asserts.
It looks like this didn't work (even before variants were introduced, eg at
v1.11) -- coverage always built with -Os and -DNDEBUG.
This commit makes it possible for variants to have finer-grained control
over COPT flags, and enables assert() and -O0 on coverage builds.
Other variants already match the defaults so they have been updated.
TimeoutError was added back in 077812b2ab for
the cc3200 port. In f522849a4d the cc3200
port enabled use of it in the socket module aliased to socket.timeout. So
it was never added to the builtins. Then it was replaced by
OSError(ETIMEDOUT) in 047af9b10b.
The esp32 port enables this exception, since the very beginning of that
port, but it could never be accessed because it's not in builtins.
It's being removed: 1) to not encourage its use; 2) because there are a lot
of other OSError subclasses which are not defined at all, and having
TimeoutError is a bit inconsistent.
Note that ports can add anything to the builtins via MICROPY_PORT_BUILTINS.
And they can also define their own exceptions using the
MP_DEFINE_EXCEPTION() macro.
This commit makes all functions and function wrappers in modubinascii.c
STATIC and conditional on the MICROPY_PY_UBINASCII setting, which will
exclude the file from qstr/ compressed-string searching when ubinascii is
not enabled. The now-unused modubinascii.h header file is also removed.
The cc3200 port is updated accordingly to use this module in its entirety
instead of providing its own top-level definition of ubinascii.
This was originally like this because the cc3200 port has its own ubinascii
module which referenced these methods. The plan appeared to be that the
API might diverge (e.g. hardware crc), but this should be done similar to
I2C/SPI via a port-specific handler, rather than the port having its own
definition of the module. Having a centralised module definition also
enforces consistency of the API among ports.
This commit changes the default filesystem type for esp32 to littlefs v2.
This port already enables both VfsFat and VfsLfs2, so either can be used
for the filesystem, and existing systems that use FAT will still work.
This commit changes the esp8266 boards to use littlefs v2 as the
filesystem, rather than FAT. Since the esp8266 doesn't expose the
filesystem to the PC over USB there's no strong reason to keep it as FAT.
Littlefs is smaller in code size, is more efficient in use of flash to
store data, is resilient over power failure, and using it saves about 4k of
heap RAM, which can now be used for other things.
This is a backwards incompatible change because all existing esp8266 boards
will need to update their filesystem after installing new firmware (eg
backup old files, install firmware, restore files to new filesystem).
As part of this commit the memory layout of the default board (GENERIC) has
changed. It now allocates all 1M of memory-mapped flash to the firmware,
so the filesystem area starts at the 2M point. This is done to allow more
frozen bytecode to be stored in the 1M of memory-mapped flash. This
requires an esp8266 module with 2M or more of flash to work, so a new board
called GENERIC_1M is added which has the old memory-mapping (but still
changed to use littlefs for the filesystem).
In summary there are now 3 esp8266 board definitions:
- GENERIC_512K: for 512k modules, doesn't have a filesystem.
- GENERIC_1M: for 1M modules, 572k for firmware+frozen code, 396k for
filesystem (littlefs).
- GENERIC: for 2M (or greater) modules, 968k for firmware+frozen code,
1M+ for filesystem (littlefs), FAT driver also included in firmware for
use on, eg, external SD cards.
Following up to 5e6cee07ab, some systems (eg
FreeBSD 12.0 64-bit) will crash if the stack-overflow margin is too small.
It seems the margin of 8192 bytes (or thereabouts) is always needed. This
commit adds this much margin if the requested stack size is too small.
Fixes issue #5824.
These were found by buiding the unix coverage variant on macOS (so clang
compiler). Mostly, these are fixing implicit cast of float/double to
mp_float_t which is one of those two and one mp_int_t to size_t fix for
good measure.
These are mainly used by the previous version of uasyncio which is now
replaced by a newer version, with built-in C module _uasyncio. Saves about
1300 bytes of flash.
https://www.python.org/dev/peps/pep-0475/
This implements something similar to PEP 475 on the unix port, and for the
VfsPosix class.
There are a few differences from the CPython implementation:
- Since we call mp_handle_pending() between any ENITR's, additional
functions could be called if MICROPY_ENABLE_SCHEDULER is enabled, not
just signal handlers.
- CPython only handles signal on the main thread, so other threads will
raise InterruptedError instead of retrying. On MicroPython,
mp_handle_pending() will currently raise exceptions on any thread.
A new macro MP_HAL_RETRY_SYSCALL is introduced to reduce duplicated code
and ensure that all instances behave the same. This will also allow other
ports that use POSIX-like system calls (and use, eg, VfsPosix) to provide
their own implementation if needed.
The stack size adjustment for detecting stack overflow in threads was not
taking into account that the requested stack size could be <= 8k, in which
case the subtraction would overflow. This is fixed in this commit by
ensuring that the adjustment can't be more than the available size.
This fixes the test tests/thread/thread_stacksize1.py which sometimes
crashes with a segmentation fault because of an uncaught NLR jump, which is
a "maximum recursion depth exceeded" exception.
Suggested-by: @dpgeorge
To enable lazy loading of submodules (among other things), which is very
useful for MicroPython libraries that want to have optional subcomponents.
Disabled explicitly on minimal ports.
This function is not used by the core but having it as part of the build
allows it to be used by user C modules, or board extensions. The linker
won't include it in the final firmware if it remains unused.
This removes the port-specific definition of MP_PLAT_PRINT_STRN on the
windows port, so that the default mp_hal_stdout_tx_strn_cooked() is always
used. This fixes releasing the GIL during the call to write() (this was
missed in bc3499f010).
Also, mp_hal_dupterm_tx_strn() was defined but never used anywhere so it is
safe to delete it.
This removes the port-specific definition of MP_PLAT_PRINT_STRN on the unix
port. Since fee7e5617f this is no longer a
single function call so we are not really optimising anything over using
the default definition of MP_PLAT_PRINT_STRN which calls
mp_hal_stdout_tx_strn_cooked().
Zephyr v2.2 reworked its gpio api to support linux device tree bindings and
pin logical levels. This commit updates the zephyr port's machine.Pin
class to replace the deprecated gpio api calls with the new supported gpio
api. This resolves several build warnings.
Tested on frdm_k64f and mimxrt1050_evk boards.
The "led" argument is always a pointer to the GPIO port, or'd with the pin
that the LED is on, so testing that it is "1" is unnecessary. The type of
"led" is also changed to uint32_t so it can properly hold a 32-bit pointer.
Updating the LED0 state from systick handler ensures LED0 is always
consistent with its flash rate regardless of other processing going on in
either interrupts or main. This improves the visible stability of the
bootloader, rather than LED0 flashing somewhat randomly at times.
This commit also changes the LED0 flash rate depending on the current state
of DFU, giving slightly more visual feedback on what the device is doing.
Adds support in the zephyr port to execute main.py if the file system is
enabled and the file exists. Existing support for executing a main.py
frozen module is preserved, since pyexec_file_if_exists() works just
like pyexec_frozen_module() if there's no vfs.
Enables the zephyr usb device stack and mass storage class on the
mimxrt1050_evk board. The mass storage class is backed by the sdhc disk
access driver, so it's now possible to browse and modify the contents of
the SD card from a USB host (your PC). This is in preparation to support
writing a main.py script to the SD card, and then executing it after the
next reset.
Adds support in the zephyr port to mount a file system if a block device
(sdhc disk access or flash area) is available. The mount point is either
"/sd" or "/flash" depending on the type of block device.
Tested with an sdhc disk access block device and fatfs on the
mimxrt1050_evk board.
Tested with a flash area block device and littlefs on the reel_board.
This commit adds micropython.heap_locked() which returns the current
lock-depth of the heap, and can be used by Python code to check if the heap
is locked or not. This new function is configured via
MICROPY_PY_MICROPYTHON_HEAP_LOCKED and is disabled by default.
This commit also changes the return value of micropython.heap_unlock() so
it returns the current lock-depth as well.
This is an extremely minimal port to the NXP i.MX RT, in the style of the
SAMD port It's largely based on the TinyUSB mimxrt implementation, using
the NXP SDK. It currently supports the Teensy 4.0 board with a REPL over
the USB-VCP interface.
This commit also adds the NXP SDK submodule (also from TinyUSB) to
lib/nxp_driver.
Note: if you already have the tinyusb submodule initialized recursively you
will need to run the following as the tinyusb sub-submodules have been
rearranged (upstream):
git submodule deinit lib/tinyusb
rm -rf .git/modules/lib/tinyusb
git submodule update --init lib/tinyusb
This makes a cleaner separation between the: driver, HCI UART and BT stack.
Also updated the naming to be more consistent (mp_bluetooth_hci_*).
Work done in collaboration with Jim Mussared aka @jimmo.
Move extmod/modbluetooth_nimble.* to extmod/nimble. And move common
Makefile lines to extmod/nimble/nimble.mk (which was previously only used
by stm32). This allows (upcoming) btstack to follow a similar structure.
Work done in collaboration with Jim Mussared aka @jimmo.
sys.stdout.flush() is needed on CPython to flush the output, and the change
in this commit makes such an expression also work on MicroPython (although
MicroPython doesn't actual need to do any flushing).
This string is recognised by uncrustify, to disable formatting in the
region marked by these comments. This is necessary in the qstrdef*.h files
to prevent modification of the strings within the Q(...). In other places
it is used to prevent excessive reformatting that would make the code less
readable.
If the built-in input() is enabled (which it is by default) then it needs
some form of readline, so supply it with one when MICROPY_USE_READLINE=0.
Fixes issue #5658.
This changes the signal used to trigger garbage collection from SIGUSR1 to
SIGRTMIN + 5. SIGUSR1 is quite common compared to SIGRTMIN (measured by
google search results) and is more likely to conflict with libraries that
may use the same signal.
POSIX specifies that there are at least 8 real-time signal so 5 was chosen
as a "random" number to further avoid potential conflict with libraries
that may use SIGRTMIN or SIGRTMAX.
Also, if we ever have a `usignal` module, it would be nice to leave SIGUSR1
and SIGUSR2 free for user programs.
The "random" module no longer uses the hardware RNG (the extmod version of
this module has a pseudo-random number generator), so the config option
MICROPY_PY_RANDOM_HW_RNG is no longer meaningful. This commit replaces it
with MICROPY_HW_ENABLE_RNG, which controls whether the hardware RNG is
included in the build.
The install target is current broken when PROG is used to override the
default executable name. This fixes it by removing the redundant TARGET
variable and uses PROG directly instead.
The install and uninstall targets are also moved to the common unix
Makefile so that all variants can be installed in the same way.
Currently it is not possible to override PREFIX when installing micropython
using the makefile. It is common practice to be able to run something like
this:
$ make install PREFIX=/usr DESTDIR=/tmp/staging
This fixes such usage.
This provides a more consistent C-level API to raise exceptions, ie moving
away from nlr_raise towards mp_raise_XXX. It also reduces code size by a
small amount on some ports.
The default value for MICROPYPATH used in unix/main.c is
"~/.micropython/lib:/usr/lib/micropython" which has 2 problems when used in
the Windows port:
- it has a ':' as path separator but the port uses ';' so the entire string
is effectively discarded since it gets interpreted as a single path which
doesn't exist
- /usr/lib/micropython is not a valid path in a standard Windows
environment
Override the value with a suitable default.
If the exception doesn't need printf-style formatting then calling
mp_raise_msg is more efficient. Also shorten exception messages to match
style in core and other ports.
It's not needed. The C integer implicit promotion rules mean that the
uint8_t of the incoming character is promoted to a (signed) int, matching
the type of interrupt_char. Thus the uint8_t incoming character can never
be equal to -1 (the value of interrupt_char that indicate that interruption
is disabled).
The mp_keyboard_interrupt() function does exactly what is needed here, and
using it gets ctrl-C working when MICROPY_ENABLE_SCHEDULER is enabled on
these ports (and MICROPY_ASYNC_KBD_INTR is disabled).
This is a more logical place to clear the KeyboardInterrupt traceback,
right before it is set as a pending exception. The clearing is also
optimised from a function call to a simple store of NULL.
It was originally in IRAM due to the linker script specification, but
since the function moved from lib/utils/interrupt_char.c to py/scheduler.c
it needs to be put back in IRAM.
This function is tightly coupled to the state and behaviour of the
scheduler, and is a core part of the runtime: to schedule a pending
exception. So move it there.
Pending exceptions would otherwise be handled later on where there may not
be an NLR handler in place.
A similar fix is also made to the unix port's REPL handler.
Fixes issues #4921 and #5488.
Previous behaviour is when this argument is set to "true", in which case
the function will raise any pending exception. Setting it to "false" will
cancel any pending exception.
Enables the littlefs (v1 and v2) filesystems in the zephyr port.
Example usage with the internal flash on the reel_board or the
rv32m1_vega_ri5cy board:
import os
from zephyr import FlashArea
bdev = FlashArea(FlashArea.STORAGE, 4096)
os.VfsLfs2.mkfs(bdev)
os.mount(bdev, '/flash')
with open('/flash/hello.txt','w') as f:
f.write('Hello world')
print(open('/flash/hello.txt').read())
Things get a little trickier with the frdm_k64f due to the micropython
application spilling into the default flash storage partition defined
for this board. The zephyr build system doesn't enforce the flash
partitioning when mcuboot is not enabled (which it is not for
micropython). For now we can demonstrate that the littlefs filesystem
works on frdm_k64f by constructing the FlashArea block device on the
mcuboot scratch partition instead of the storage partition. Do this by
replacing the FlashArea.STORAGE constant above with the value 4.
Introduces a new zephyr.FlashArea class that uses the zephyr flash map
api to implement the uos.AbstractBlockDev protocol. The flash driver is
enabled on the frdm_k64f board, reel_board, and rv32m1_vega_ri5cy board.
The standard and extended block device protocols are both supported,
therefore this class can be used with file systems like littlefs which
require the extended interface.
Enables the fatfs filesystem in the zephyr port.
Example usage with an SD card on the mimxrt1050_evk board:
import zephyr, os
bdev = zephyr.DiskAccess('SDHC')
os.VfsFat.mkfs(bdev)
os.mount(bdev, '/sd')
with open('/sd/hello.txt','w') as f:
f.write('Hello world')
print(open('/sd/hello.txt').read())
Introduces a new zephyr.DiskAccess class that uses the zephyr disk
access api to implement the uos.AbstractBlockDev protocol. This can be
used with any type of zephyr disk access driver, which currently
includes SDHC, RAM, and FLASH implementations. The SDHC driver is
enabled on the mimxrt1050_evk board.
Only the standard block device protocol (without the offset parameter)
can be supported with the zephyr disk access api, therefore this class
cannot be used with file systems like littlefs which require the
extended interface. In the future it may be possible to implement the
extended interface in a new class using the zephyr flash api.
CPython also has os.environ, which should be used instead of os.getenv()
due to caching in the os.environ mapping. But for MicroPython it makes
sense to only implement the basic underlying methods, ie getenv/putenv/
unsetenv.
This adds a -h option to print the usage help text and adds a new, shorter
error message that is printed when invalid arguments are given. This
behaviour follows CPython (and other tools) more closely.
This commit modifies the usage() function to only print the -v option help
text when MICROPY_DEBUG_PRINTERS is enabled. The -v option requires this
build option to be enabled for it to have any effect.
The usage text is also modified to show the -i and -m options, and also
show that running a command, module or file are mutually exclusive.
This adds support for a MICROPYINSPECT environment variable that works
exactly like PYTHONINSPECT; per CPython docs:
If this is set to a non-empty string it is equivalent to specifying the
-i option.
This variable can also be modified by Python code using os.environ to
force inspect mode on program termination.
Zephyr removed the build target syscall_macros_h_target in commit
f4adf107f31674eb20881531900ff092cc40c07f. Removes reference from
MicroPython to fix build errors in the zephyr port.
This change is not compatible with zephyr v2.1 or earlier. It will be
compatible with Zephyr v2.2 when released.
The SYS_CLOCK_HW_CYCLES_TO_NS macro was deprecated in zephyr commit
8892406c1de21bd5de5877f39099e3663a5f3af1. This commit updates MicroPython
to use the new k_cyc_to_ns_floor64 api and fix build warnings in the zephyr
port.
This change is compatible with Zephyr v2.1 and later.
Zephyr restructured its includes in v2.0 and removed compatibility shims
after two releases in commit 1342dadc365ee22199e51779185899ddf7478686.
Updates include paths in MicroPython accordingly to fix build errors in
the zephyr port.
These changes are compatible with Zephyr v2.0 and later.
When stdout is redirected it is useful to have errors printed to stderr
instead of being redirected.
mp_stderr_print() can't be used in these two instances since the
MicroPython runtime is not running so we use fprintf(stderr) instead.
The ability to change the host is a frequently requested feature, so
explicitly document how it can be achieved using the existing code.
See issues #2121, #4385, #4622, #5122, #5536.
This commit improves pllvalues.py to generate PLL values for H7 MCUs that
are valid (VCO in and out are in range) and extend for the entire range of
SYSCLK values up to 400MHz (up to 480MHz is currently unsupported).
This board now has the following 3 build configurations:
- mboot + external QSPI in XIP mode + internal filesystem
- mboot + external QSPI with filesystem (the default)
- no mboot + external QSPI with filesystem
With a SPI flash that has more than 16MB, 32-bit addressing is required
rather than the standard 24-bit. This commit adds support for 32-bit
addressing so that the SPI flash commands (read/write/erase) are selected
automatically depending on the size of the address being used at each
operation.
This modifies the signature of mp_thread_set_state() to use
mp_state_thread_t* instead of void*. This matches the return type of
mp_thread_get_state(), which returns the same value.
`struct _mp_state_thread_t;` had to be moved before
`#include <mpthreadport.h>` since the stm32 port uses it in its
mpthreadport.h file.
PLLM is shared among all PLL blocks on F7 MCUs, and this calculation to
configure PLLSAI to have 48MHz on the P output previously assumed that PLLM
is equal to HSE (eg PLLM=25 for HSE=25MHz). This commit relaxes this
assumption to allow other values of PLLM.
It is not safe to enable MICROPY_ASYNC_KBD_INTR and MICROPY_PY_THREAD_GIL
at the same time. This will trigger a compiler error to ensure that it
is not possible to make this mistake.
Addition of GIL EXIT/ENTER pairs are:
- modos: release the GIL during system calls. CPython does this as well.
- moduselect: release the GIL during the poll() syscall. This call can be
blocking, so it is important to allow other threads to run at this time.
- modusocket: release the GIL during system calls. Many of these calls can
be blocking, so it is important to allow other threads to run.
- unix_mphal: release the GIL during the read and write syscalls in
mp_hal_stdin_rx_chr and mp_hal_stdout_tx_strn. If we don't do this
threads are blocked when the REPL or the builtin input function are used.
- file, main, mpconfigport.h: release GIL during syscalls in built-in
functions that could block.
When CFLAGS_EXTRA/LDFLAGS_EXTRA (or anything) is set on the command line of
a make invocation then it will completely override any setting or appending
of these variables in the makefile(s). This means builds like the coverage
variant will have their mpconfigvariant.mk settings overridden. Fix this
by using CFLAGS/LDFLAGS exclusively in the makefile(s), reserving the
CFLAGS_EXTRA/LDFLAGS_EXTRA variables for external command-line use only.
Translate common Ctrl-Left/Right/Delete/Backspace to the EMACS-style
sequences (i.e. Alt key based) for forward-word, backward-word, forwad-kill
and backward-kill. Requires MICROPY_REPL_EMACS_WORDS_MOVE to be defined so
the readline implementation interprets these.
Prior to this commit, if the flash filesystem was not formatted then it
would error: "AttributeError: 'FlashBdev' object has no attribute 'mount'".
That is due to it not being able to detect the filesystem on the block
device and just trying to mount the block device directly.
This commit fixes the issue by just catching all exceptions. Also it's not
needed to try the mount if `flashbdev.bdev` is None.
This commit adds backward-word, backward-kill-word, forward-word,
forward-kill-word sequences for the REPL, with bindings to Alt+F, Alt+B,
Alt+D and Alt+Backspace respectively. It is disabled by default and can be
enabled via MICROPY_REPL_EMACS_WORDS_MOVE.
Further enabling MICROPY_REPL_EMACS_EXTRA_WORDS_MOVE adds extra bindings
for these new sequences: Ctrl+Right, Ctrl+Left and Ctrl+W.
The features are enabled on unix micropython-coverage and micropython-dev.
Invoking "make" will still build the standard "micropython" executable, but
other variants are now build using, eg, "make VARIANT=minimal". This
follows how bare-metal ports specify a particular board, and allows running
any make target (eg clean, test) with any variant.
Convenience targets (eg "make coverage") are provided to retain the old
behaviour, at least for now.
See issue #3043.
Most types are in rodata/ROM, and mp_obj_base_t.type is a constant pointer,
so enforce this const-ness throughout the code base. If a type ever needs
to be modified (eg a user type) then a simple cast can be used.
This change has the following effects:
- Reduces the resolution of the RTC sub-second counter from 30.52us to
122.07us.
- Allows RTC.calibration() to now support positive values (as well as
negative values).
- Reduces VBAT current consumption in standby mode by a small amount.
For general purpose use 122us resolution of the sub-second counter is
good enough, and the benefits of full range calibration and minor reduction
in VBAT consumption are worth the change.
Make version 4.1 and lower does not allow $call as the main expression on a
line, so assign the result of the $call to a dummy variable.
Fixes issue #5426.
Instances of the slice class are passed to __getitem__() on objects when
the user indexes them with a slice. In practice the majority of the time
(other than passing it on untouched) is to work out what the slice means in
the context of an array dimension of a particular length. Since Python 2.3
there has been a method on the slice class, indices(), that takes a
dimension length and returns the real start, stop and step, accounting for
missing or negative values in the slice spec. This commit implements such
a indices() method on the slice class.
It is configurable at compile-time via MICROPY_PY_BUILTINS_SLICE_INDICES,
disabled by default, enabled on unix, stm32 and esp32 ports.
This commit also adds new tests for slice indices and for slicing unicode
strings.
The existing uos.remove cannot be used to remove directories, instead
uos.rmdir is needed. And also provide uos.rename to get a good set of
filesystem functionality without requiring additional Python-level os
functions (eg using ffi).
For the 3 ports that already make use of this feature (stm32, nrf and
teensy) this doesn't make any difference, it just allows to disable it from
now on.
For other ports that use pyexec, this decreases code size because the debug
printing code is dead (it can't be enabled) but the compiler can't deduce
that, so code is still emitted.
Most stm32 boards can now be built in nan-boxing mode via:
$ make NANBOX=1
Note that if float is enabled then it will be forced to double-precision.
Also, native emitters will be disabled.
Move webrepl support code from ports/esp8266/modules into extmod/webrepl
(to be alongside extmod/modwebrepl.c), and use frozen manifests to include
it in the build on esp8266 and esp32.
A small modification is made to webrepl.py to make it work on non-ESP
ports, i.e. don't call dupterm_notify if not available.
- Corrected pin assignments and checked with CubeMX.
- Added additional I2C and UARTs.
- Added Ethernet interface definitions with lwIP and SSL support (but
Ethernet is currently unsupported on H7 MCUs so not fully enabled).
- Removed remarks on DFU/OCD in mpconfigboard.h because deploy-stlink works
fine too.
- Added more UARTs, I2C, corrected SPI, CAN, etc; verified against CubeMX.
- Adapted pins.csv to remove errors, add omissions, etc. according to
NUCLEO-144 User Manual.
- Changed linker file stm32f767.ld to reflect correct size of the Flash.
- Tested with LAN and SD card.
The Nucleo board does not have an SD card slot but does have the requisite
pins next to each other and labelled, so provide the configuration for
convenience.
Implements text, rodata and bss generalised relocations, as well as generic
qstr-object linking. This allows importing dynamic native modules on all
supported architectures in a unified way.
The default protection for the BLE ringbuf is to use
MICROPY_BEGIN_ATOMIC_SECTION, which disables all interrupts. On stm32 it
only needs to disable the lowest priority IRQ, pendsv, because that's the
IRQ level at which the BLE stack is driven.
qstrs in this file are always included in all builds, even if not used
anywhere. So remove those that are never needed, and make USB names
conditional on having USB enabled.
And return -MP_EIO if calling storage_read_block/storage_write_block fails.
This lines up with the return type and value (negative for error) of the
calls to MICROPY_HW_BDEV_READBLOCKS (and WRITEBLOCKS, and BDEV2 versions).
The pyb.Flash() class can now be used to construct objects which reference
sections of the flash storage, starting at a certain offset and going for a
certain length. Such objects also support the extended block protocol.
The signature for the constructor is: pyb.Flash(start=-1, len=-1).
This commit refactors and generalises the boot-mount routine on stm32 so
that it can mount filesystems of arbitrary type. That is, it no longer
assumes that the filesystem is FAT. It does this by using mp_vfs_mount()
which does auto-detection of the filesystem type.
Using mp_hal_delay_ms allows the scheduler to run, which might result in
another transmit operation happening, which would bypass the sleep (and
fail). Use mp_hal_delay_us instead.
The compile-time configuration value MICROPY_HW_RTC_USER_MEM_MAX can now be
used to define the amount of memory set aside for RTC.memory(). If this
value is configured to zero then the RTC.memory functionality is not
included in the build.
The IDF heap is more fragmented with IDF 4 and mbedtls cannot allocate
enough RAM with 16+16kiB for both in and out buffers, so reduce output
buffer size.
Fixes issue #5303.
This commit removes the Makefile-level MICROPY_FATFS config and moves the
MICROPY_VFS_FAT config to the Makefile level to replace it. It also moves
the include of the oofatfs source files in the build from each port to a
central place in extmod/extmod.mk.
For a port to enabled VFS FAT support it should now set MICROPY_VFS_FAT=1
at the level of the Makefile. This will include the relevant oofatfs files
in the build and set MICROPY_VFS_FAT=1 at the C (preprocessor) level.
This commit adds support for littlefs (v2) on all esp32 boards.
The original FAT filesystem still works and any board with a preexisting
FAT filesystem will still work as normal. It's possible to switch to
littlefs by reformatting the block device using:
import uos, flashbdev
uos.VfsLfs2.mkfs(flashbdev.bdev)
Then when the board reboots (soft or hard) the new littlefs filesystem will
be mounted. It's possible to switch back to a FAT filesystem by formatting
with uos.VfsFat.mkfs(flashbdev.bdev).
This commit adds an implementation of machine.Timer backed by the soft
timer mechanism. It allows an arbitrary number of timers with 1ms
resolution, with an associated Python callback. The Python-level API
matches existing ports that have a soft timer, and is used as:
from machine import Timer
t = Timer(freq=10, callback=lambda t:print(t))
...
t = Timer(mode=Timer.ONE_SHOT, period=2000, callback=lambda t:print(t))
...
t.deinit()
This commit adds an implementation of a "software timer" with a 1ms
resolution, using SysTick. It allows unlimited number of concurrent
timers (limited only by memory needed for each timer entry). They can be
one-shot or periodic, and associated with a Python callback.
There is a very small overhead added to the SysTick IRQ, which could be
further optimised in the future, eg by patching SysTick_Handler code
dynamically.
The MP_STATE_THREAD(stack_top) is always available so use it instead of
creating a separate variable. This also allows gc_collect() to be used as
an independent function, without real_main() being called.
When a SPI bus is initialized with a SPI host that is currently in use the
exception msg incorrectly indicates "SPI device already in use". The
mention of "device" in the exception msg is confusing because the error is
about trying to use a SPI host that is already claimed. A better exception
msg is "SPI host already in use".
For consistency with "umachine". Now that weak links are enabled
by default for built-in modules, this should be a no-op, but allows
extension of the bluetooth module by user code.
Also move registration of ubluetooth to objmodule rather than
port-specific.
This commit implements automatic module weak links for all built-in
modules, by searching for "ufoo" in the built-in module list if "foo"
cannot be found. This means that all modules named "ufoo" are always
available as "foo". Also, a port can no longer add any other weak links,
which makes strict the definition of a weak link.
It saves some code size (about 100-200 bytes) on ports that previously had
lots of weak links.
Some changes from the previous behaviour:
- It doesn't intern the non-u module names (eg "foo" is not interned),
which saves code size, but will mean that "import foo" creates a new qstr
(namely "foo") in RAM (unless the importing module is frozen).
- help('modules') no longer lists non-u module names, only the u-variants;
this reduces duplication in the help listing.
Weak links are effectively the same as having a set of symbolic links on
the filesystem that is searched last. So an "import foo" will search
built-in modules first, then all paths in sys.path, then weak links last,
importing "ufoo" if it exists. Thus a file called "foo.py" somewhere in
sys.path will still have precedence over the weak link of "foo" to "ufoo".
See issues: #1740, #4449, #5229, #5241.
When loading a manifest file, e.g. by include(), it will chdir first to the
directory of that manifest. This means that all file operations within a
manifest are relative to that manifest's location.
As a consequence of this, additional environment variables are needed to
find absolute paths, so the following are added: $(MPY_LIB_DIR),
$(PORT_DIR), $(BOARD_DIR). And rename $(MPY) to $(MPY_DIR) to be
consistent.
Existing manifests are updated to match.
Prior to this commit the systick IRQ priority was set at lowest priority on
F0/L0/WB MCUs, because it was left at the default and never configured.
This commit ensures the priority is configured and sets it to the highest
priority.
Remove the 240MHz CPU config option from sdkconfig.base and create a new
sdkconfig.240mhz file for those boards that want to use 240MHz on boot.
The default CPU frequency is now 160MHz (was 240MHz), to align with the ESP
IDF and support more boards (eg those with D2WD chips).
Fixes issue #5169.
This prevents issues with concurrent access to the ringbuf.
MICROPY_BEGIN_ATOMIC_SECTION is only atomic to the same core. We could
address this with a mutex, but it's also not safe to call mp_sched_schedule
across cores.
This avoids a confusing ENOMEM raised from gap_advertise if there is
currently an active connection. This refers to the static connection
buffer pre-allocated by Nimble (nothing to do with MicroPython heap
memory).
This patch add basic building blocks for nrf9P60.
It also includes a secure bootloader which forwards all
possible peripherals that are user selectable to become
non-secure. After configuring Flash, RAM and peripherals
the secure bootloader will jump to the non-secure domain
where MicroPython is placed.
The minimum size of a secure boot has to be a flash
block of 32Kb, hence why the linker scripts are
offsetting the main application this much.
The RAM offset is set to 128K, to allow for later
integration of Nordic Semiconductor's BSD socket
library which reserves the range 0x20010000 - 0x2001FFFF.
Add support for pca10059 with REPL over tinyusb USB CDC.
The board also includes a board specific module that will
recover UICR->REGOUT0 in case this has been erased.
This initial support does not preserve any existing bootloader
on the pca10090 in case this was present, and expects to use all
available flash on the device.
Add nrf-port finyusb driver files. USB CDC can be activated
by board configuration files using the MICROPY_HW_USB_CDC.
Updating BLE driver, Makefile, nrfx-glue and main.c to plug
in the tinyusb stack.
The specific board can be selected with the BOARD makefile variable. This
defaults (if not specified) to BOARD=GENERIC, which is the original default
firmware build. For the 512k target use BOARD=GENERIC_512K.
On other ports (e.g. ESP32) they provide a complete Nimble implementation
(i.e. we don't need to use the code in extmod/nimble). This change
extracts out the bits that we don't need to use in other ports:
- malloc/free/realloc for Nimble memory.
- pendsv poll handler
- depowering the cywbt
Also cleans up the root pointer management.
STM32F0 has PCLK=48MHz and maximum ADC clock is 14MHz so use PCLK/4=12MHz
to stay within spec of the ADC peripheral. In pyb.ADC set common sampling
time to approx 4uS for internal and external sources. In machine.ADC
reduce sample time to approx 1uS for external source, leave internal at
maximum sampling time.
This commit adds the option to use HSE or MSI system clock, and LSE or LSI
RTC clock, on L4 MCUs.
Note that prior to this commit the default clocks on an L4 part were MSI
and LSE. The defaults are now MSI and LSI.
In mpconfigboard.h select the clock source via:
#define MICROPY_HW_RTC_USE_LSE (0) or (1)
#define MICROPY_HW_CLK_USE_HSE (0) or (1)
and the PLLSAI1 N,P,Q,R settings:
#define MICROPY_HW_CLK_PLLSAIN (12)
#define MICROPY_HW_CLK_PLLSAIP (RCC_PLLP_DIV7)
#define MICROPY_HW_CLK_PLLSAIQ (RCC_PLLQ_DIV2)
#define MICROPY_HW_CLK_PLLSAIR (RCC_PLLR_DIV2)
The the nrfx driver is aware of chip specific registers, while
the raw HAL abstraction is not. This driver enables use of NVMC
in non-secure domain for nrf9160.
This patch moves the check for MICROPY_PY_MACHINE_TEMP to come
before the inclusion of nrf_temp.h. The nrf_temp.h depends on
the NRF_TEMP_Type which might not be defined for all nRF devices.
For use with F0 MCUs that don't have HSI48. Select the clock source
explicitly in mpconfigboard.h.
On the NUCLEO_F091RC board use HSE bypass when HSE is chosen because the
NUCLEO clock source is STLINK not a crystal.
Before this patch the UART baudrate on F0 MCUs was wrong because the
stm32lib SystemCoreClockUpdate sets SystemCoreClock to 8MHz instead of
48MHz if HSI48 is routed directly to SYSCLK.
The workaround is to use HSI48 -> PREDIV (/2) -> PLL (*2) -> SYSCLK.
Fixes issue #5049.
Enabled by default, but disabled when REPL is connected to the VCP (this is
the existing behaviour). Can be configured at run-time with, eg:
pyb.USB_VCP().init(flow=pyb.USB_VCP.RTS | pyb.USB_VCP.CTS)
The new fdcan.c file provides the low-level C interface to the FDCAN
peripheral, and pyb_can.c is updated to support both traditional CAN and
FDCAN, depending on the MCU being compiled for.
Add the project file to the mpy-cross directory, which is also where the
executable ends up, and change the Appveyor settings to build mpy-cross
with both msvc and mingw-w64 and verify this all works by running tests
with --via-mpy.
If this is not set it might default to calls to open() to use text mode
which is usually not wanted, and even wrong and leading to incorrect
results when loading binary .mpy files.
This also means that text files written and read will not have line-ending
translation from \n to \r\n and vice-versa anymore. This shouldn't be much
of a problem though since most tools dealing with text files adapt
automatically to any of the 2 formats.
Reserve sources.props for listing just the MicroPython core and extmod
files, similar to how py.mk lists port-independent source files. This
allows reusing the source list, for instance for building mpy-cross. The
sources for building the executable itself are listed in the corresponding
project file, similar to how the other ports specify the source files in
their Makefile.
Append to PyIncDirs, used to define include directories specific to
MicroPython, instead of just overwriting it so project files importing this
file can define additional directories. And allow defining the target
directory for the executable instead of hardcoding it to the windows
directory. Main reason for this change is that it will allow building
mpy-cross with msvc.
We want the .vcxproj to be just a container with the minimum content for
making it work as a project file for Visual Studio and MSBuild, whereas the
actual build options and actions get placed in separate reusable files.
This was roughly the case already except some compiler options were
overlooked; fix this here: we'll need those common options when adding a
project file for building mpy-cross.
These were probably added to detect more qstrs but as long as the
micropython executable itself doesn't use the same build options the qstrs
would be unused anyway. Furthermore these definitions are for internal use
and get enabled when corresponding MICROPY_EMIT_XXX are defined, in which
case the compiler would warn about symbol redefinitions since they'd be
defined both here and in the source.
This commit adds support for a second supported hash (currently set to the
4.0-beta1 tag). When this hash is detected, the relevant changes are
applied.
This allows to start using v4 features (e.g. BLE with Nimble), and also
start doing testing, while still supporting the original, stable, v3.3 IDF.
Note: this feature is experimental, not well tested, and network.LAN and
network.PPP are currently unsupported.
This option affects py/vm.c and py/gc.c and using -Os gets them compiling a
bit smaller, and small firmware is the aim of these two ports. Also,
having these files compiled with -Os on these ports, and -O3 as the default
on other ports, gives a better understanding of code-size changes when
making changes to these files.
According to the schematic, the SDRAM part on this board is a
MT48LC4M32B2B5-6A, with "Row addressing 4K A[11:0]" (per datasheet). This
commit updates mpconfigboard.h from 13 to 12 to match.
This patch uses the newly-added esp32.Partition class to replace the
existing FlashBdev class. Partition objects implement the block protocol
so can be directly mounted via uos.mount(). This has the following
benefits:
- allows the filesystem partition location and size to be specified in
partitions.csv, and overridden by a particular board
- very easily allows to have multiple filesystems by simply adding extra
entries to partitions.csv
- improves efficiency/speed of filesystem operations because the block
device is implemented fully in C
- opens the possibility to have encrypted flash storage (since Partitions
can be encrypted)
Note that this patch is fully backwards compatible: existing filesystems
remain untouched and work with this new code.
- STM32F407VGT6 (1MB of Flash, 192+4 Kbytes of SRAM)
- 5V (via USB) or Li-Polymer Battery (3.7V) power input
- 2 x LEDs
- 2 x user switches
- 2 x mikroBUS sockets
- 2 x 1x26 mikromedia-compatible headers (52 pins)
https://www.mikroe.com/clicker-2-stm32f4
Mboot currently requires at least three LEDs to display each of the four
states. However, since there are only four possible states, the states can
be displayed via binary counting on only 2 LEDs (if only 2 are available).
The existing patterns are still used for 3 or 4 LEDs.
It was previously not taking into account that the list of pins was sparse,
so using the wrong index. The boards/X/pins.csv was generating the wrong
data for machine.Pin.board.
As part of this fix rename the variables to make it more clear what the
list contains (only board pins).
This commit adds support for sys.settrace, allowing to install Python
handlers to trace execution of Python code. The interface follows CPython
as closely as possible. The feature is disabled by default and can be
enabled via MICROPY_PY_SYS_SETTRACE.
mp_compile no longer takes an emit_opt argument, rather this setting is now
provided by the global default_emit_opt variable.
Now, when -X emit=native is passed as a command-line option, the emitter
will be set for all compiled modules (included imports), not just the
top-level script.
In the future there could be a way to also set this variable from a script.
Fixes issue #4267.
- Split 'qemu-arm' from 'unix' for generating tests.
- Add frozen module to the qemu-arm test build.
- Add test that reproduces the requirement to half-word align native
function data.
Enabled via MICROPY_PY_URE_DEBUG, disabled by default (but enabled on unix
coverage build). This is a rarely used feature that costs a lot of code
(500-800 bytes flash). Debugging of regular expressions can be done
offline with other tools.
As per PEP 485, this function appeared in for Python 3.5. Configured via
MICROPY_PY_MATH_ISCLOSE which is disabled by default, but enabled for the
ports which already have MICROPY_PY_MATH_SPECIAL_FUNCTIONS enabled.
Before this patch I2C transactions using a hardware I2C peripheral on F0/F7
MCUs would not correctly generate the I2C restart condition, and instead
would generate a stop followed by a start. This is because the CR2 AUTOEND
bit was being set before CR2 START when the peripheral already had the I2C
bus from a previous transaction that did not generate a stop.
As a consequence all combined transactions, eg read-then-write for an I2C
memory transfer, generated a stop condition after the first transaction and
didn't generate a stop at the very end (but still released the bus). Some
I2C devices require a repeated start to function correctly.
This patch fixes this by making sure the CR2 AUTOEND bit is set after the
start condition and slave address have been fully transferred out.
Replaces the `SDKCONFIG` makefile variable with `BOARD`. Defaults to
BOARD=GENERIC. spiram can be enabled with `BOARD=GENERIC_SPIRAM`
Add example definition for TINYPICO, currently identical to GENERIC_SPIRAM
but with custom board/SoC names for the uPy banner.
They are both enabled by default, but can be disabled by defining
MICROPY_HW_ENABLE_MDNS_QUERIES and/or MICROPY_HW_ENABLE_MDNS_RESPONDER to
0. The hostname for the responder is currently taken from
tcpip_adapter_get_hostname() but should eventually be configurable.
This commit adds the connect() method to the PPP interface and requires
that connect() be called after active(1). This is a breaking change for
the PPP API.
With the connect() method it's now possible to pass in authentication
information for PAP/CHAP, eg:
ppp.active(1)
ppp.connect(authmode=ppp.AUTH_PAP, username="user", "password="password")
If no authentication is needed simply call connect() without any
parameters. This will get the original behaviour of calling active(1).
Some SD/MMC breakout boards don't support 4-bit bus mode. This adds a new
macro MICROPY_HW_SDMMC_BUS_WIDTH that allows each board to define the width
of the SD/MMC bus interface used on that board, defaulting to 4 bits.
The previous version did not work on MCUs that only had USB device mode
(compared to OTG) because of the handling of NAK. And this previous
handling of NAK had a race condition where a new packet could come in
before USBD_HID_SetNAK was called (since USBD_HID_ReceivePacket clears NAK
as part of its operation). Furthermore, the double buffering of incoming
reports was not working, only one buffer could be used at a time.
This commit rewrites the HID interface code to have a single incoming
buffer, and only calls USBD_HID_ReceivePacket after the user has read the
incoming report (similar to how the VCP does its flow control). As such,
USBD_HID_SetNAK and USBD_HID_ClearNAK are no longer needed.
API functionality from the user's point of view should be unchanged with
this commit.
On this port the GIL is enabled and everything works under the assumption
of the GIL, ie that a given task has exclusive access to the uPy state, and
any ISRs interrupt the current task and therefore the ISR inherits
exclusive access to the uPy state for the duration of its execution.
If the MicroPython tasks are not pinned to a specific core then an ISR may
be executed on a different core to the task, making it possible for the
main task and an ISR to execute in parallel, breaking the assumption of the
GIL.
The easiest and safest fix for this is to pin all MicroPython related code
to the same CPU core, as done by this patch. Then any ISR that accesses
MicroPython state must be registered from a MicroPython task, to ensure it
is invoked on the same core.
See issue #4895.
The C++ standard forbids redefining keywords, like inline and alignof, so
guard these definitions to avoid that, allowing to include the MicroPython
headers by C++ code.
This new series of MCUs is similar to the L4 series with an additional
Cortex-M0 coprocessor. The firmware for the wireless stack must be managed
separately and MicroPython does not currently interface to it. Supported
features so far include: RTC, UART, USB, internal flash filesystem.
