Commit Graph

14843 Commits

Author SHA1 Message Date
Brian 'redbeard' Harrington
ea8f0fd896 rp2/CMake: Normalize MICROPY_PORT_DIR.
In 5fe2a3f1 the ESP32 port underwent a change to how `MICROPY_PORT_DIR`
is defined.  This commit normalizes the `rp2` port to use the same
underlying variable mechanism (`CMAKE_CURRENT_LIST_DIR`).

Signed-off-by: Brian 'redbeard' Harrington <redbeard@dead-city.org>
2023-06-13 00:11:06 -07:00
Brian 'redbeard' Harrington
5fe2a3f14f esp32/CMake: Change PROJECT_DIR to CMAKE_CURRENT_LIST_DIR.
This migrates the CMake variable `MICROPY_PORT_DIR` from the ESP-IDF
defined project to the component. Previously used instances of the variable
within the project definition have been migrated to
`CMAKE_CURRENT_LIST_DIR`. Within the component (the `main` subdirectory in
the ESP32 port) we define `MICROPY_PORT_DIR` using `CMAKE_CURRENT_LIST_DIR`
and subsequently use the `MICROPY_PORT_DIR` value in all locations where
`PROJECT` had previously been used.

Context:

In commit 9b90882146, initial support was added for building with the newly
introduced CMake support provided by the ESP-IDF.

Specifically, the commit message states:

> This commit adds support for building the esp32 port with CMake, and in
particular, it builds MicroPython as a component within the ESP-IDF. Using
CMake and the ESP-IDF build infrastructure makes it much easier to maintain
the port, especially with the various new ESP32 MCUs and their required
toolchains.

`PROJECT_DIR` is a variable populated by the ESP-IDF specifically and is
not stable when used with "[Pure CMake components][1]" as documented in the
ESP-IDF. It is intended to be used in the scope of the parent of the
current file (the "project") as opposed to the current file ("the
component"). Crossing into the parent scope like this works solely when the
"project" is MicroPython, but not when used as a component by other ESP-IDF
projects.

Analyzing this file, the intention is to reference the "Project" which in
the example is the parent directory. Within the [CMake variables][2]
documentation, there is one specifically defined for referencing the
directory for the CMake listfile currently being processed:
[`CMAKE_CURRENT_LIST_DIR`][3].

After making the change from `PROJECT_DIR` to `CMAKE_CURRENT_LIST_DIR`, the
reach into the parent scope defined by the ESP-IDF and the resulting CMake
interface violation is removed.

Similar to the component definition, the project `CMakeLists.txt` uses the
variable `CMAKE_SOURCE_DIR` which CMake defines as "The path to the top
level of the source tree."  This commit changes the variable to
`CMAKE_CURRENT_LIST_DIR` for the reasons cited above.

[1]: https://docs.espressif.com/projects/esp-idf/en/latest/esp32s2/api-guides/build-system.html#writing-pure-cmake-components
[2]: https://cmake.org/cmake/help/latest/manual/cmake-variables.7.html
[3]: https://cmake.org/cmake/help/latest/variable/CMAKE_CURRENT_LIST_DIR.html

Signed-off-by: Brian 'redbeard' Harrington <redbeard@dead-city.org>
2023-06-13 00:11:05 -07:00
Glenn Moloney
fd277704c4 esp32/modespnow: Change name of buffer size config option to "rxbuf".
Rename "buffer" option to "rxbuf" which accords with docs and esp8266.

Signed-off-by: Glenn Moloney <glenn.moloney@gmail.com>
2023-06-10 12:16:26 +10:00
Jim Mussared
b4de697ad1 tools/mpremote: Fix exec_ -> exec in commands.py.
This was missed in the pyboard refactor and is preventing `cp -r` from
working.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-09 13:36:12 +10:00
Damien George
f01d5fb657 py/mkrules.mk: Automatically configure frozen options when manifest set.
Following how mkrules.cmake works.  This makes it easy for a port to enable
frozen code, by defining FROZEN_MANIFEST in its Makefile.

Signed-off-by: Damien George <damien@micropython.org>
2023-06-08 23:12:56 +10:00
Damien George
30100e9b3d stm32/boards: Use default spibdev config where appropriate.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-08 23:01:03 +10:00
Damien George
3b370157d0 stm32/mpconfigboard_common: Provide default spidev config.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-08 23:01:03 +10:00
Damien George
0832fc53ce tools/ci.sh: Add mimxrt and samd ports to code size build.
The automatic code size build and GitHub comment is a really useful
feature.  This commit adds a few more builds to it (mimxrt and samd).

Signed-off-by: Damien George <damien@micropython.org>
2023-06-08 22:47:03 +10:00
Jim Mussared
a1fbb1980c extmod/modtimeq: Remove timeq module.
This is a MicroPython-specific module that existed to support the old
version of uasyncio.  It's undocumented and not enabled on all ports and
takes up code size unnecessarily.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:28 +10:00
Jim Mussared
8211d56712 docs/library/index: Update docs after umodule rename.
- Update guide for extending built-in modules.
- Remove any last trace of umodule in other docs.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:28 +10:00
Jim Mussared
5fd042e7d1 all: Replace all uses of umodule in Python code.
Applies to drivers/examples/extmod/port-modules/tools.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared
9d7eac0713 tests/run-natmodtests.py: Don't allow imports from the cwd.
Make tests run in an isolated environment (i.e. `import io` would
otherwise get the `tests/io` directory).

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared
339f02a594 tests/run-perfbench.py: Don't allow imports from the cwd.
Make tests run in an isolated environment (i.e. `import io` would
otherwise get the `tests/io` directory).

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared
109717457e tests/run-multitests.py: Don't allow imports from the cwd.
Make tests run in an isolated environment (i.e. `import io` would
otherwise get the `tests/io` directory).

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared
4216bc7d13 tests: Replace umodule with module everywhere.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared
5e50975a6d py/modsys: Allow sys.path to be assigned to.
Previously sys.path could be modified by append/pop or slice assignment.

This allows `sys.path = [...]`, which can be simpler in many cases, but
also improves CPython compatibility.

It also allows sys.path to be set to a tuple which means that you can
clear sys.path (e.g. temporarily) with no allocations.

This also makes sys.path (and sys.argv for consistency) able to be disabled
via mpconfig. The unix port (and upytesthelper) require them, so they
explicitly verify that they're enabled.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared
7d2ee8aed0 py/mpconfig: Enable module delegation if sys needs it.
Otherwise you can get into the confusing state where e.g. sys.ps1 is
enabled in config (via `MICROPY_PY_SYS_PS1_PS2`) but still doesn't actually
get enabled.

Also verify that the required delegation options are enabled in modsys.c.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared
e6926d6021 py/objmodule: Workaround for MSVC with no module delegation.
When compiling mpy-cross, there is no `sys` module, and so there will
be no entries in the `mp_builtin_module_delegation_table`.

MSVC doesn't like this, so instead pretend as if the feature isn't
enabled at all.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared
13c817e61c py/objmodule: Add a table of built-in modules with delegation.
This replaces the previous QSTR_null entry in the globals dict which could
leak out to Python (e.g. via iteration of mod.__dict__) and could lead to
crashes.

It results in smaller code size at the expense of turning a lookup into a
loop, but the list it is looping over likely only contains one or two
elements.

To allow a module to register its custom attr function it can use the new
`MP_REGISTER_MODULE_DELEGATION` macro.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared
eb85f4d4c9 examples/natmod: Rename umodule to module.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared
2eba98f1e0 all: Use MP_REGISTER_EXTENSIBLE_MODULE for overrideable built-ins.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:21 +10:00
Jim Mussared
24c02c4eb5 py/makemoduledefs.py: Add a way to register extensible built-in modules.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:21 +10:00
Jim Mussared
45ac651d1a all: Rename *umodule*.c to remove the "u" prefix.
Updates any includes, and references from Makefiles/CMake.

This essentially reverts what was done long ago in commit
136b5cbd76

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:17 +10:00
Jim Mussared
0ceccd4cf8 all: Rename *umodule*.h to remove the "u" prefix.
This work was funded through GitHub Sponsors.

Also updates #includes.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:14 +10:00
Jim Mussared
f5f9edf645 all: Rename UMODULE to MODULE in preprocessor/Makefile vars.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:11 +10:00
Jim Mussared
7f5d5c7271 all: Rename mod_umodule*, ^umodule* to remove the "u" prefix.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:07 +10:00
Jim Mussared
1bf2dcb15e all: Rename mp_umodule*, mp_module_umodule* to remove the "u" prefix.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:04 +10:00
Jim Mussared
dfe232d000 py/builtinimport: Remove weak links.
In order to keep "import umodule" working, the existing mechanism is
replaced with a simple fallback to drop the "u".

This makes importing of built-ins no longer touch the filesystem, which
makes a typical built-in import take ~0.15ms rather than 3-5ms.

(Weak links were added in c14a81662c)

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:04 +10:00
Jim Mussared
30628d1bb7 all: Rename MP_QSTR_umodule to MP_QSTR_module everywhere.
This renames the builtin-modules, such that help('modules') and printing
the module object will show "module" rather than "umodule".

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:53:57 +10:00
Yaroslav Halchenko
d080d427eb top: Add "mis" to list of ignore words for codespell.
Observed with codespell 2.2.5.dev57+gdc7e98d9:

    $ codespell
    ./ports/rp2/machine_uart.c:163: mis ==> miss, mist
    ./ports/rp2/machine_uart.c:168: mis ==> miss, mist
    2
2023-06-06 10:23:44 +10:00
robert-hh
db5444f68a samd/boards: Extend the code size limit for boards with external flash.
Code size limits are charged to:
- SAMD21:    184K -> 248K
- SAMD51x19: 368K -> 496K
- SAMD51x20: 368K -> 1008K

Signed-off-by: robert-hh <robert@hammelrath.com>
2023-06-06 00:49:36 +10:00
robert-hh
69cb5e8f2a samd: Adapt existing samd.Flash and integrate with (Q)SPI flash in boot.
Checks are added to ensure, that only one of the flash drivers is selected.

Signed-off-by: robert-hh <robert@hammelrath.com>
2023-06-06 00:49:36 +10:00
robert-hh
2b5a5a0f35 samd/samd_qspiflash: Add QSPI flash driver and configure it accordingly.
The QSPI driver provides the interface for using an on-board QSPI flash for
the filesystem.  It provides the same methods as the driver for the
internal flash and uses the same name.  Therefore, only one of the drivers
for internal flash, SPI flash and QSPI flash must be enabled at a time.

Signed-off-by: robert-hh <robert@hammelrath.com>
2023-06-06 00:49:36 +10:00
robert-hh
5561130c3f samd/samd_spiflash: Add SPI flash driver and configure it accordingly.
The SPI flash driver includes the block device for being used as a
filesystem.  It provides the same methods as the driver for the internal
flash.

Signed-off-by: robert-hh <robert@hammelrath.com>
2023-06-06 00:49:36 +10:00
robert-hh
bf7d3ad8c6 samd/boards: Rename flash pins consistently for QSPI and SPI.
For SAMD21 devices, the board flash signals must be named in pins.csv as
FLASH_MOSI, FLASH_MISO, FLASH_SCK, FLASH_CS for creating the SPI object.

And rename the QSPI pins to QSPI_xxxx instead of FLASH_xxx.

Signed-off-by: robert-hh <robert@hammelrath.com>
2023-06-06 00:42:33 +10:00
David Grayson
a79a6ab364 py/builtinimport: Remove partially-loaded modules from sys.modules.
Prior to this commit, importing a module that exists but has a syntax error
or some other problem that happens at import time would result in a
potentially-incomplete module object getting added to sys.modules.
Subsequent imports would use that object, resulting in confusing error
messages that hide the root cause of the problem.

This commit fixes that issue by removing the failed module from sys.modules
using the new NLR callback mechanism.

Note that it is still important to add the module to sys.modules while the
import is happening so that we can support circular imports just like
CPython does.

Fixes issue #967.

Signed-off-by: David Grayson <davidegrayson@gmail.com>
2023-06-05 23:21:52 +10:00
Damien George
ce31e5a2dc py: Use nlr jump callbacks to optimise compile/execute functions.
The changed functions now use less stack, and don't have any issues with
local variables needing to be declared volatile.

Testing on a PYBv1.0, imports (of .py, .mpy and frozen code) now use 64
less bytes of C stack per import depth.

Signed-off-by: Damien George <damien@micropython.org>
2023-06-02 21:59:47 +10:00
Damien George
2757acf6ed py/nlr: Implement jump callbacks.
NLR buffers are usually quite large (use lots of C stack) and expensive to
push and pop.  Some of the time they are only needed to perform clean up if
an exception happens, and then they re-raise the exception.

This commit allows optimizing that scenario by introducing a linked-list of
NLR callbacks that are called automatically when an exception is raised.
They are essentially a light-weight NLR handler that can implement a
"finally" block, i.e. clean-up when an exception is raised, or (by passing
`true` to nlr_pop_jump_callback) when execution leaves the scope.

Signed-off-by: Damien George <damien@micropython.org>
2023-06-02 21:50:57 +10:00
Damien George
f36ae5edcb py/nlr: Remove commented-out debugging code.
Also remove the unnecessary include of mpstate.h.

Signed-off-by: Damien George <damien@micropython.org>
2023-06-02 21:47:34 +10:00
Damien George
24aa81e1da py/nlrsetjmp: Use MP_NLR_JUMP_HEAD macro to simplify code.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-02 21:47:34 +10:00
Damien George
a61d40e5e8 tools/mpremote: Fix use of stdout_write_bytes function.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-02 18:03:16 +10:00
Damien George
ef497dc924 LICENSE: Move wiznet5k entry from drivers to lib.
It was removed from drivers in 71dcb21e24

Signed-off-by: Damien George <damien@micropython.org>
2023-06-02 17:48:31 +10:00
Jim Mussared
b4d785fa20 tools/mpremote: Detach mpremote from pyboard.py.
This commit just takes the necessary parts of pyboard.py and merges them
with pyboardextended.py to make a new transport_serial.py, and updates the
rest of mpremote to use this instead.

It is difficult to continue to add features to mpremote (which usually
requires modification to pyboard.py) while also maintaining backwards
compatibility for pyboard.py.

The idea is that this provides a starting point for further refactoring of
mpremote to allow different transports (webrepl, BLE, etc).

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 17:42:13 +10:00
Jim Mussared
bd5d0163c4 docs/reference/mpremote.rst: Extend the mpremote guide.
Changes in this commit:
- Add a extra detail to each of the commands.
- Add more about handling options and arguments.
- Include shortcut commands that behave like real commands to the command
  list (e.g. bootloader, rtc).
- Add extra information and reword to address common misconceptions, in
  particular how commands chain together.
- Add additional examples showing some more interesting combinations.
- Add descriptions to each of the examples.
- Add pipx installation instructions.
- Describe how user-configuration works.

This work was sponsored by Google Season of Docs.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 16:16:28 +10:00
Jim Mussared
b6b19798c2 tools/mpremote: Handle cp without destination.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 16:16:24 +10:00
Jim Mussared
46715e370d tools/mpremote: Add rtc commands to get and set the RTC.
Replaces the existing functionality provided by the `setrtc` alias to use
the current time, rather than a hard-coded date/time.

Use `rtc` to query the current time.  Use `rtc --set` to set it.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 16:15:48 +10:00
Jim Mussared
d736a2f3f3 tools/mpremote: Allow terminator for shortcut commands.
For example, the `reset` shortcut previously allowed an optional delay, but
the argument handling cannot handle `reset next-command` as `next-command`
will be interpreted as the delay argument.  The fix in this commit allows
`reset + next-command`.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 16:14:26 +10:00
Jim Mussared
7c2c9ea21c tools/mpremote: Add sleep command.
This allows the sequence to be paused (e.g. wait for device, etc).

Also removes the t_ms arg in reset/bootloader, because these arguments
don't really need to be changed, and keeping them would mean inconsistent
units used for time delays.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 16:11:29 +10:00
Damien George
e4886dda85 stm32/help: Exclude legacy entries from help, and adjust constant names.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-02 12:43:58 +10:00
Jim Mussared
aac8744671 stm32/help: Don't include unavailable features in help().
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 11:49:13 +10:00