Commit Graph

12605 Commits

Author SHA1 Message Date
Jim Mussared
692d36d779 py: Implement partial PEP-498 (f-string) support.
This implements (most of) the PEP-498 spec for f-strings and is based on
https://github.com/micropython/micropython/pull/4998 by @klardotsh.

It is implemented in the lexer as a syntax translation to `str.format`:
  f"{a}" --> "{}".format(a)

It also supports:
  f"{a=}" --> "a={}".format(a)

This is done by extracting the arguments into a temporary vstr buffer,
then after the string has been tokenized, the lexer input queue is saved
and the contents of the temporary vstr buffer are injected into the lexer
instead.

There are four main limitations:
- raw f-strings (`fr` or `rf` prefixes) are not supported and will raise
  `SyntaxError: raw f-strings are not supported`.

- literal concatenation of f-strings with adjacent strings will fail
    "{}" f"{a}" --> "{}{}".format(a)    (str.format will incorrectly use
                                         the braces from the non-f-string)
    f"{a}" f"{a}" --> "{}".format(a) "{}".format(a) (cannot concatenate)

- PEP-498 requires the full parser to understand the interpolated
  argument, however because this entirely runs in the lexer it cannot
  resolve nested braces in expressions like
    f"{'}'}"

- The !r, !s, and !a conversions are not supported.

Includes tests and cpydiffs.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-14 16:58:40 +10:00
Damien George
162bf3c5d8 tools/mpremote: Add "devs" shortcut for "connect list".
See issue #7480.

Signed-off-by: Damien George <damien@micropython.org>
2021-08-14 00:13:35 +10:00
Damien George
1f48934312 tools/mpremote: Fix connect-list in case VID/PID are None.
Which can be the case on Windows and macOS for certain serial devices.

Fixes issue #7636.

Signed-off-by: Damien George <damien@micropython.org>
2021-08-14 00:13:35 +10:00
Damien George
8fcdb5490c extmod/modlwip: Fix close and clean up of UDP and raw sockets.
The correct callback-deregister functions must be called dependent on the
socket type, otherwise resources may not be freed correctly.

Signed-off-by: Damien George <damien@micropython.org>
2021-08-13 23:46:11 +10:00
Damien George
90d47ee34d tests/run-multitests.py: Add broadcast and wait facility.
Test instances can now use the following methods to synchronise their
execution:

    multitest.broadcast("sync message")
    multitest.wait("sync message")

Signed-off-by: Damien George <damien@micropython.org>
2021-08-13 23:26:34 +10:00
Patrick Van Oosterwijck
e49f609186 tools/autobuild: Add auto build for Silicognition wESP32. 2021-08-13 23:25:24 +10:00
Daniel Mizyrycki
8f45f5ee4f nrf: Set .mpy features consistent with documentation and other ports.
This allows nrf devices to load .mpy files.  And nrf52840 and nrf9160 based
boards also support compiling and loading native code.
2021-08-13 23:22:54 +10:00
Jim Mussared
d7fbc755dc docs/library/index.rst: Clarify module naming and purpose.
Adds section about extending built-in modules from Python.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-13 23:14:08 +10:00
Jim Mussared
ee549d725a tools/gen-cpydiff.py: Don't rename foo to ufoo in diff output.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-13 23:14:08 +10:00
Jim Mussared
c737cde947 docs: Replace ufoo with foo in all docs.
Anywhere a module is mentioned, use its "non-u" name for consistency.

The "import module" vs "import umodule" is something of a FAQ, and this
commit intends to help clear that up.  As a first approximation MicroPython
is Python, and so imports should work the same as Python and use the same
name, to a first approximation.  The u-version of a module is a detail that
can be learned later on, when the user wants to understand more and have
finer control over importing.

Existing Python code should just work, as much as it is possible to do that
within the constraints of embedded systems, and the MicroPython
documentation should match the idiomatic way to write Python code.

With universal weak links for modules (via MICROPY_MODULE_WEAK_LINKS) users
can consistently use "import foo" across all ports (with the exception of
the minimal ports).  And the ability to override/extend via "foo.py"
continues to work well.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-13 22:53:29 +10:00
Jim Mussared
218606351c docs: Rename ufoo.rst to foo.rst.
This is a simple rename of the files, no content changes
(other than updating index.rst to use the new paths)

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-13 20:31:32 +10:00
Damien George
671f012306 docs/templates: Add unix and zephyr quickref links to top-index.
Signed-off-by: Damien George <damien@micropython.org>
2021-08-13 20:21:21 +10:00
Julia Hathaway
6ed6990625 docs/library/zephyr: Add libraries specific to the Zephyr port.
Includes documentation for Zephyr specific modules (zephyr and zsensor),
classes (DiskAccess and FlashArea), and functions.

Signed-off-by: Julia Hathaway <julia.hathaway@nxp.com>
2021-08-13 20:21:14 +10:00
Julia Hathaway
333e16521b docs/zephyr: Add quick reference for the Zephyr port.
Includes an introduction to using the Zephyr port on MicroPython.  The
quickref details examples of how to use each module the port currently
supports.  The tutorial provides additional details for Zephyr specific
modules.

Signed-off-by: Julia Hathaway <julia.hathaway@nxp.com>
2021-08-13 20:20:57 +10:00
Ned Konz
42d1a1635c stm32/mbedtls: Fix compile warning about uninitialized val. 2021-08-10 23:43:25 +10:00
Ned Konz
a0cd18c1a5 stm32/boards/NUCLEO_H743ZI2: Add modified version of NUCLEO_H743ZI.
This commit creates a new stm32 board for the NUCLEO_H743ZI2, which is the
current version of this from ST.  This is a modified copy of the
NUCLEO_H743ZI board, and the ZI2 board differs in a few minor ways:

- LED2 has moved from PB7 to PE1 and is now yellow rather than blue
- the USB power enable has moved from PG6 to PG10
- the USER button is now pulled down
2021-08-10 23:38:05 +10:00
iabdalkader
23531bca74 rp2/CMakeLists.txt: Allow a board's cmake to set the manifest path.
This allows boards to add frozen modules, or bypass the port manifest
entirely.
2021-08-10 11:05:33 +10:00
Damien George
3835f5f597 esp32/makeimg.py: Get bootloader and partition offset from sdkconfig.
So that it works on ESP32C3, which has the bootloader at 0x0.

Fixes issue #7565.

Signed-off-by: Damien George <damien@micropython.org>
2021-08-10 00:19:49 +10:00
Daniel Mizyrycki
8645b7c23b nrf: Enable source line on tracebacks. 2021-08-09 15:09:33 +10:00
Glenn Ruben Bakke
77b4cfcbc9 nrf/modules/nrf: Add function to enable/disable DCDC.
This function can be used to enable and disable the DC/DC converter with or
without the Bluetooth stack enabled.  It can also be used to query the
current state of the DC/DC.

This commit also adds a definition of ARRAY_SIZE needed by nrfx HAL-layer.
2021-08-09 14:55:59 +10:00
Damien George
e5e0553224 nrf/modules/uos: Add ilistdir to uos module.
This was missed in the initial implementation of the uos module.

Signed-off-by: Damien George <damien@micropython.org>
2021-08-08 23:17:55 +10:00
Glenn Ruben Bakke
85f0ce214e tools/codeformat.py: Include ports/nrf/modules/nrf in code formatting. 2021-08-08 23:17:55 +10:00
Glenn Ruben Bakke
55d4321c3e nrf/boards: Enable MICROPY_VFS_LFS2 for all target boards.
Enable LittleFS v2 for all targets, except nrf51 targets when SoftDevice is
present.
2021-08-08 23:17:55 +10:00
Glenn Ruben Bakke
23e8729d3e nrf/mphalport: Add dummy function for mp_hal_time_ns().
extmod/vfs_lfs.c needs to resolve `mp_hal_time_ns()` in order to calculate
a timestamp from 1970 epoch.  A wall clock is not available in the nrf
port, hence the function is implemented to resolve compilation linkage
error.  The function always return 0.
2021-08-08 23:17:55 +10:00
Glenn Ruben Bakke
aa857eb65e nrf/Makefile: Set default manifest file for all targets.
Set the default manifest to "modules/manifest.py".  This includes files
from the folder "modules/scripts".  The manifest default value is overriden
by all nrf51 boards that have SoftDevice present (SD=s110) to save flash.

Also add "modules/manifest.py" which is set to freeze
"modules/scripts/_mkfs.py".
2021-08-08 23:17:55 +10:00
Glenn Ruben Bakke
7b6ad0ce2e nrf/modules/scripts: Add file system formatting script.
Add a helper script _mkfs.py which automatically formats the file system if
nrf.Flash() is located and a VFS file system has been included in the
compilation.

The precedence is: first LFS1, LFS2 then FAT.
2021-08-08 23:17:55 +10:00
Glenn Ruben Bakke
ffc636de2f nrf/boards: Set FROZEN_MANIFEST blank when SD present on nrf51 targets. 2021-08-08 23:17:55 +10:00
Glenn Ruben Bakke
4f76f66185 tools/ci.sh: Add mpy-cross build to nrf port. 2021-08-08 23:17:55 +10:00
Glenn Ruben Bakke
990341d18e nrf: Facilitate use of freeze manifest.
Update the Makefile to handle FROZEN_MANIFEST, and the README with some
small samples on how to use freeze manifests.  And add BOARD_DIR to the
Makefile which can be referenced in boards/<board>/mpconfigboard.mk to
include a board specific manifest.
2021-08-08 23:17:55 +10:00
Glenn Ruben Bakke
4326e08802 nrf/boards: Enable needed features for FAT/LFS1/LFS2.
Enable the following features for all boards except
nrf51 boards with SoftDevice present:

- MICROPY_VFS
- MICROPY_PY_NRF
- MICROPY_HW_ENABLE_INTERNAL_FLASH_STORAGE
2021-08-08 23:17:55 +10:00
Glenn Ruben Bakke
b0fd4372c4 nrf/main: Add auto mount and auto format hook for internal flash FS. 2021-08-08 23:17:55 +10:00
Glenn Ruben Bakke
6ff3a2afef nrf/qstrdefsport.h: Add entries for in-built FS mount points. 2021-08-08 23:17:49 +10:00
Glenn Ruben Bakke
f99aa82e85 nrf/mpconfigport.h: Enable MICROPY_PY_IO_FILEIO when an FS is enabled.
Enable MICROPY_PY_IO_FILEIO if MICROPY_VFS_FAT, MICROPY_VFS_LFS1
or MICROPY_VFS2 has been enabled.
2021-08-08 23:09:26 +10:00
Glenn Ruben Bakke
7a833edf37 nrf/modules/uos: Allow a board to configure MICROPY_VFS_FAT/LFS1/LFS2. 2021-08-08 23:09:26 +10:00
Glenn Ruben Bakke
0bde907a8b nrf/Makefile: Add _fs_size linker script override from make.
Add posibility to override linker script "_fs_size" from make by adding the
FS_SIZE parameter.  The syntax of value is linker script syntax.  For
example, the value of 131072 bytes can be written as 128K like this:
FS_SIZE=128K.

If not set, default value for "_fs_size" from linker script will be used.
2021-08-08 23:09:26 +10:00
Glenn Ruben Bakke
c9b72ba694 nrf/mpconfigport.h: Tune FAT FS configuration.
Disable MICROPY_FATFS_MULTI_PARTITION configuration because there is no
partition table in the flash for FATFS to read.

Also, set MICROPY_FATFS_MAX_SS to the size of a flash page.  For nrf51 the
value 1024 is set.  For nrf52/nrf91 the value 4096 is set.
2021-08-08 23:09:26 +10:00
Glenn Ruben Bakke
b40dfa961d nrf/README: Update README.md to reflect internal file systems.
This documents parameters that can be passed to make to enable a specific
file system to included in the build.  Also, document the Makefile override
parameter "FS_SIZE" that can be used to tune the size of the flash region
to use as internal flash file system.
2021-08-08 23:09:26 +10:00
Glenn Ruben Bakke
85cad50266 nrf/mpconfigport.h: Expose nrf module when MICROPY_PY_NRF is set. 2021-08-08 23:09:26 +10:00
Glenn Ruben Bakke
5a873e27eb nrf/drivers: Add support for using flash block device with SoftDevice.
Update flash.c to also be compiled in when
MICROPY_HW_ENABLE_INTERNAL_FLASH_STORAGE is enabled and SoftDevice is
present.

Update bluetooth/ble_drv.c to forward flash events to flash.c when
MICROPY_HW_ENABLE_INTERNAL_FLASH_STORAGE is enabled.
2021-08-08 23:09:26 +10:00
Glenn Ruben Bakke
127cec8cae nrf/modules/nrf: Add new nrf module with flash block device.
This commit adds the "nrf" module for port specific modules and objects.
Included in it is the "Flash" object which exposes a block device
implementation to access internal SoC flash.

Thanks to @aykevl aka Ayke van Laethem for the initial implementation.
2021-08-08 23:09:26 +10:00
Glenn Ruben Bakke
3b594f7b27 nrf/boards/common.ld: Calculate unused flash region.
Calculate the unused flash area on the target device.  The values will be
exposed by _unused_flash_start and _unused_flash_length.  The start address
and the length are not aligned to either word or pages.
2021-08-08 23:09:26 +10:00
Damien George
f834fef6bb stm32/powerctrl: Support changing frequency on WB MCUs.
This allows changing the frequency to: 100kHz, 200kHz, 400kHz, 800kHz,
1MHz, 2MHz, 4MHz, 8MHz, 16MHz, 32MHz, 64MHz.  For frequencies 2MHz and
below, low power run (LPR) mode is enabled automatically.

Signed-off-by: Damien George <damien@micropython.org>
2021-08-07 23:41:20 +10:00
Damien George
3b32b3d1b3 stm32/usbd_cdc_interface: Rename USBD_CDC_xx opts to MICROPY_HW_USB_xx.
Signed-off-by: Damien George <damien@micropython.org>
2021-08-07 23:13:58 +10:00
Damien George
f28e8b9073 stm32/usbd_desc: Rename USBD_xxx descriptor opts to MICROPY_HW_USB_xxx.
Signed-off-by: Damien George <damien@micropython.org>
2021-08-07 23:13:58 +10:00
Damien George
96c6b8cae3 ports: Rename USBD_VID/PID config macros to MICROPY_HW_USB_VID/PID.
For consistency with other board-level config macros that begin with
MICROPY_HW_USB.

Also allow boards in the mimxrt, nrf and samd ports to configure these
values.

Signed-off-by: Damien George <damien@micropython.org>
2021-08-07 23:13:55 +10:00
Damien George
45f9a38451 docs/library/machine.I2S.rst: Fix use of sd pin in examples.
Signed-off-by: Damien George <damien@micropython.org>
2021-08-07 20:40:54 +10:00
Damien George
78718fffb1 py/mkrules: Automatically build mpy-cross if it doesn't exist.
Commit 4173950658 removed automatic building
of mpy-cross, which rebuilt it whenever any of its dependent source files
changed.

But needing to build mpy-cross, and not knowing how, is a frequent issue.
This commit aims to help by automatically building mpy-cross only if it
doesn't exist.  For Makefiles it uses an order-only prerequisite, while
for CMake it uses a custom command.

If MICROPY_MPYCROSS (which is what makemanifest.py uses to locate the
mpy-cross executable) is defined in the environment then automatic build
will not be attempted, allowing a way to prevent this auto-build if needed.

Thanks to Trammell Hudson aka @osresearch for the original idea; see #5760.

Signed-off-by: Damien George <damien@micropython.org>
2021-08-07 20:25:32 +10:00
Peter Züger
d290f369d0 tests/extmod/ujson: Add tests for dump/dumps separators argument.
Basically just copied ujson_dump(s).py and added various valid/invalid
separator tests.

Signed-off-by: Peter Züger <zueger.peter@icloud.com>
2021-08-07 13:52:16 +10:00
Peter Züger
ffc854f17f extmod/modujson: Add support for dump/dumps separators keyword-argument.
Optionally enabled via MICROPY_PY_UJSON_SEPARATORS.  Enabled by default.

For dump, make sure mp_get_stream_raise is called after
mod_ujson_separators since CPython does it in this order (if both
separators and stream are invalid, separators will raise an exception
first).

Add separators argument in the docs as well.

Signed-off-by: Peter Züger <zueger.peter@icloud.com>
Signed-off-by: Damien George <damien@micropython.org>
2021-08-07 13:52:16 +10:00
Damien George
8616129f2e esp8266,esp32: Include hidden networks in WLAN.scan results.
Addresses issues #2697 and #5329.

Signed-off-by: Damien George <damien@micropython.org>
2021-08-07 13:28:34 +10:00