Commit Graph

11959 Commits

Author SHA1 Message Date
stijn 37c2f507a0 github/workflows: Add workflow to verify commit message format.
Using the new tools/verifygitlog.py script.
2021-01-30 14:09:21 +11:00
stijn d48860c7dd tools/verifygitlog.py: Add script for verifying commit message format.
The main rules enforced are:
- At most 72 characters in the subject line, with a ": " in it.
- At most 75 characters per line in the body.
- No "noreply" email addresses.
2021-01-30 14:08:29 +11:00
stijn fca2730ea0 lib/utils/pyexec: Remove obsolete LCD initialization.
This was added a long time ago in 75abee206d
when USB host support was added to the stm (now stm32) port, and when this
pyexec code was actually part of the stm port.  It's unlikely to work as
intended anymore.  If it is needed in the future then generic hook macros
can be added in pyexec.
2021-01-30 13:41:36 +11:00
stijn b9a35bebf7 py/qstr.h: Remove QSTR_FROM_STR_STATIC macro.
It practically does the same as qstr_from_str and was only used in one
place, which should actually use the compile-time MP_QSTR_XXX form for
consistency; qstr_from_str is for runtime strings only.
2021-01-30 13:40:48 +11:00
Jim Mussared 47d02b3104 extmod/nimble: Improve the flow control for l2cap recv path.
If the _IRQ_L2CAP_RECV handler does the actual consumption of the incoming
data (i.e. via l2cap_recvinto), rather than setting a flag for
non-scheduler-context to handle it later, then two things can happen:

- It can starve the VM (i.e. the scheduled task never terminates).  This is
  because calling l2cap_recvinto will empty the rx buffer, which will grant
  more credits to the channel (an HCI command), meaning more data can
  arrive.  This means that the loop in hal_uart.c that keeps reading HCI
  data from the uart and executing NimBLE events as they are created will
  not terminate, preventing other VM code from running.

- There's no flow control (i.e. data will arrive too quickly).  The channel
  shouldn't be given credits until after we return from scheduler context.

It's preferable that no work is done in scheduler/IRQ context.  But to
prevent this being a problem this commit changes l2cap_recvinto so that if
it is called in IRQ context, and the Python handler empties the rx buffer,
then don't grant credits until the Python handler is complete.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-01-30 13:22:40 +11:00
Jim Mussared 0f9a9129da stm32/rfcore: Fix flow control for IPCC RX IRQ.
Don't clear the IPCC channel flag until we've actually handled the incoming
data, or else the wireless firmware may clobber the IPCC buffer if more
data arrives.  This requires masking the IRQ until the data is handled.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-01-30 13:21:04 +11:00
Damien George b8f4c623f9 github/workflows: Add CI workflow for rp2 port.
Signed-off-by: Damien George <damien@micropython.org>
2021-01-30 00:42:29 +11:00
Damien George 469345e728 rp2: Add new port to Raspberry Pi RP2 microcontroller.
This commit adds a new port "rp2" which targets the new Raspberry Pi RP2040
microcontroller.

The build system uses pure cmake (with a small Makefile wrapper for
convenience).  The USB driver is TinyUSB, and there is a machine module
with most of the standard classes implemented.  Some examples are provided
in the examples/rp2/ directory.

Work done in collaboration with Graham Sanderson.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-30 00:42:29 +11:00
Damien George ef3ee7aa10 lib/pico-sdk: Add new pico-sdk submodule, for the rp2 port.
Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 23:57:10 +11:00
Damien George ec0503bd0c extmod/modonewire: Use pin_od_high/pin_od_low instead of pin_write.
The pin is configured in open-drain mode so these od_high/od_low methods
should be used.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 23:57:10 +11:00
graham sanderson 794df0f1d5 py/emitnative: Support binary ops on ARMv6M without use of ite instr. 2021-01-29 23:57:10 +11:00
Damien George 15ac5a3df9 extmod/modframebuf: Change int to unsigned int in format methods args.
These args are already bounds checked and clipped, and using unsigned ints
can be more efficient.  It also eliminates possible issues and compiler
warnings with shifting of signed integers.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 23:57:10 +11:00
Damien George 33f10381d6 lib/timeutils: Provide simple impl of extra funcs when Epoch is 1970.
Dates/times must be post 2000/1/1 to work correctly with these simple
implementations.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 23:57:10 +11:00
Damien George 75fea330bf py/emitinlinethumb: Exclude code using #if when ARMV7M disabled.
So there are no references to undeclared asm_thumb_mov_reg_i16().

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 23:57:10 +11:00
Damien George 7a97e4351b tests: Move native for test from pybnative to micropython.
And make it generic so it can be run on any target.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 23:57:10 +11:00
Damien George c9f4c5acd6 py/emitnative: Ensure encoding to load prelude_offset doesn't change sz.
Based on change made by Graham Sanderson.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 23:57:10 +11:00
graham sanderson 40d2010882 py/asmthumb: Add support for ARMv6M in native emitter.
Adds a new compile-time option MICROPY_EMIT_THUMB_ARMV7M which is enabled
by default (to get existing behaviour) and which should be disabled (set to
0) when building native emitter support (@micropython.native) on ARMv6M
targets.
2021-01-29 23:57:10 +11:00
Damien George fe16e785fe tools/mpy-tool.py: List frozen modules in MICROPY_FROZEN_LIST_ITEM.
Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 23:57:10 +11:00
Damien George 925bd67cfb py/objfun: Support fun.__globals__ attribute.
This returns a reference to the globals dict associated with the function,
ie the global scope that the function was defined in.  This attribute is
read-only but the dict itself is modifiable, per CPython behaviour.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 23:57:10 +11:00
Damien George 5d68b5e22c tools/ci.sh: For ci_code_size_setup, update apt to install gcc-multilib.
Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 23:57:10 +11:00
Damien George d1f120b142 stm32/main: Introduce MICROPY_HW_FLASH_MOUNT_AT_BOOT config option.
It's enabled by default to retain the existing behaviour.  A board can
disable this option if it manages mounting the filesystem itself, for
example in frozen code.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 15:02:55 +11:00
Damien George 8f211df360 stm32/mboot/fwupdate.py: Refactor update_mpy with support for STATUS.
Changes are:
- refactor to use new _create_element function
- support extended version of MOUNT element with block size
- support STATUS element

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 15:02:55 +11:00
Damien George bd7110a3d5 stm32/mboot: Introduce MBOOT_ERRNO_xxx constants and use them.
So that a failed update via fsload can be more easily diagnosed.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 15:02:55 +11:00
Damien George 0efa0b5437 stm32/mboot: Add ELEM_TYPE_STATUS element so application can get status.
This new element takes the form: (ELEM_TYPE_STATUS, 4, <address>).  If this
element is present in the mboot command then mboot will store to the given
address the result of the filesystem firmware update process.  The address
can for example be an RTC backup register.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 15:02:55 +11:00
Damien George c1eb292927 stm32/mboot: Don't auto-detect littlefs block size.
Instead it is now passed in as an optional parameter to the ELEM_MOUNT
element, with a compile-time configurable default.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 15:02:55 +11:00
Damien George d1945cc2b5 stm32/main: Check block 0 and 1 when auto-detecting littlefs.
The superblock for littlefs is in block 0 and 1, but block 0 may be erased
or partially written, so block 1 must be checked if block 0 does not have a
valid littlefs superblock in it.

Prior to this commit, if block 0 did not contain a valid littlefs
superblock (but block 1 did) then the auto-detection would fail, mounting a
FAT filesystem would also fail, and the system would reformat the flash,
even though it may have contained a valid littlefs filesystem.  This is now
fixed.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 15:02:55 +11:00
Damien George 71ea438561 extmod/vfs: Check block 0 and 1 when auto-detecting littlefs.
The superblock for littlefs is in block 0 and 1, but block 0 may be erased
or partially written, so block 1 must be checked if block 0 does not have a
valid littlefs superblock in it.

Prior to this commit, the mount of a block device which auto-detected the
filysystem type would fail for littlefs if block 0 did not contain a valid
superblock.  That is now fixed.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 15:02:55 +11:00
nanjekyejoannah 4eaebc1988 docs/develop: Add MicroPython Internals chapter.
This commit adds many new sections to the existing "Developing and building
MicroPython" chapter to make it all about the internals of MicroPython.

This work was done as part of Google's Season of Docs 2020.
2021-01-27 16:59:58 +11:00
Damien George 203e1d2a65 tools/ci.sh: For code size build, fetch history of master branch only.
It's not necessary to fetch all branches.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-24 15:02:20 +11:00
Damien George 0a079155e4 github/workflows: Fix code-size CI workflow.
Changes are:
- Use ubuntu-20.04 so that gcc-multilib installs without error.
- Use "fetch-depth: 100" to get history prior to pull request.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-24 14:56:40 +11:00
Vincent Duvert 45f0b6ab63 cc3200: Fix debug build.
* Fix a typo in the Makefile that prevented the debug build to be actually
  enabled when BTYPE=debug is used.

* Add a missing header in modmachine.c that is used when a debug build is
  created.
2021-01-23 17:44:27 +11:00
Vincent Duvert 342dc61784 cc3200/ftp: Add quotes to PWD response and allow FEAT prior to login.
This commit improves some FTP implementation details for better
compatibility with FTP clients:

* The PWD command now puts quotes around the directory name before
  returning it.  This fixes BBEdit’s FTP client, which performs a PWD after
  each CWD and gets confused if the returned directory path is not
  surrounded by quotes.

* The FEAT command is now allowed before logging in. This fixes the lftp
  client, which send FEAT first and gets confused (tries to use TLS) if the
  server responds with 332.
2021-01-23 17:40:53 +11:00
Oliver Joos 290dc1d5ee unix/modtime: Fix time() precision on unix ports with non-double floats.
With MICROPY_FLOAT_IMPL_FLOAT the results of utime.time(), gmtime() and
localtime() change only every 129 seconds.  As one consequence
tests/extmod/vfs_lfs_mtime.py will fail on a unix port with LFS support.

With this patch these functions only return floats if
MICROPY_FLOAT_IMPL_DOUBLE is used.  Otherwise they return integers.
2021-01-23 16:55:24 +11:00
Oliver Joos 419134bea4 tests/extmod: Add test for the precision of utime functions.
According to documentation time() has a precision of at least 1 second.
This test runs for 2.5 seconds and calls all utime functions every 100ms.
Then it checks if they returned enough different results.  All functions
with sub-second precision will return ~25 results.  This test passes with
15 results or more.  Functions that do not exist are skipped silently.
2021-01-23 16:54:57 +11:00
IhorNehrutsa 769e822f19 esp32/modnetwork: Synchronize WiFi AUTH_xxx constants with IDF values. 2021-01-23 16:49:16 +11:00
Yonatan Schachter 063d7cc0e2 zephyr: Add basic UART functionality to machine module.
Currently supports only polling read and write.

Signed-off-by: Yonatan Schachter <yonatan.schachter@gmail.com>
2021-01-23 16:31:00 +11:00
Jim Mussared aa136b4d78 extmod/modbluetooth: Add ble.hci_cmd(ogf, ocf, req, resp) function.
This allows sending arbitrary HCI commands and getting the response.  The
return value of the function is the status of the command.

This is intended for debugging and not to be a part of the public API, and
must be enabled via mpconfigboard.h.  It's currently only implemented for
NimBLE bindings.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-01-22 18:15:12 +11:00
Damien George 49dd9ba1a5 stm32/Makefile: Use MBOOT_PACK_KEYS_FILE as depedency of .pack.dfu.
To match the definition of GENERATE_PACK_DFU, so a board can customise the
location/name of this file if needed.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-18 13:53:54 +11:00
Damien George de2374cdc6 tools/ci.sh: Pip install pyhy for stm32 builds.
Signed-off-by: Damien George <damien@micropython.org>
2021-01-18 12:43:01 +11:00
Damien George f6e6ef69e0 stm32/boards/NUCLEO_WB55: Enable MBOOT with packing mode.
To have at least one board configured with MBOOT_ENABLE_PACKING, for CI
testing purposes and demonstration of the feature.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-18 12:43:01 +11:00
Damien George c6f334272a stm32/mboot: Add support for signed and encrypted firmware updates.
This commit adds support to stm32's mboot for signe, encrypted and
compressed DFU updates.  It is based on inital work done by Andrew Leech.

The feature is enabled by setting MBOOT_ENABLE_PACKING to 1 in the board's
mpconfigboard.mk file, and by providing a header file in the board folder
(usually called mboot_keys.h) with a set of signing and encryption keys
(which can be generated by mboot_pack_dfu.py).  The signing and encryption
is provided by libhydrogen.  Compression is provided by uzlib.  Enabling
packing costs about 3k of flash.

The included mboot_pack_dfu.py script converts a .dfu file to a .pack.dfu
file which can be subsequently deployed to a board with mboot in packing
mode.  This .pack.dfu file is created as follows:
- the firmware from the original .dfu is split into chunks (so the
  decryption can fit in RAM)
- each chunk is compressed, encrypted, a header added, then signed
- a special final chunk is added with a signature of the entire firmware
- all chunks are concatenated to make the final .pack.dfu file

The .pack.dfu file can be deployed over USB or from the internal filesystem
on the device (if MBOOT_FSLOAD is enabled).

See #5267 and #5309 for additional discussion.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-18 12:43:01 +11:00
Damien George 09e67de327 stm32/mboot/gzstream: Fix lost data decompressing final part of file.
Prior to this fix, the final piece of data in a compressed file may have
been lost when decompressing.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-18 12:43:01 +11:00
Damien George 0ce6948653 lib/libhydrogen: Add new libhydrogen submodule.
This library is a small and easy-to-use cryptographic library which is well
suited to embedded systems.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-18 12:43:01 +11:00
Jim Mussared f7aafc0628 extmod/nimble: Don't assert on save-IRK failure. 2020-12-23 10:08:00 +11:00
Jim Mussared f42a190247 extmod/nimble: Reset NimBLE BSS in mp_bluetooth_init.
Without this fix, each time service registration happened it would do an
increasingly large malloc() for service state.

See https://github.com/apache/mynewt-nimble/issues/896.
2020-12-23 10:07:49 +11:00
stijn 069557edef tests/misc/sys_settrace_features.py: Fix running with non-dflt encoding.
Notably git-cmd which comes with git installations on Windows alters the
encoding resulting in CPython tracing encodings/cp1252.py calls.
2020-12-18 13:57:17 +11:00
stijn 108183fcc0 tests/misc/sys_settrace: Make test output independent of invoked path.
The original logic of reducing a full path to a relative one assumes
"tests/misc" is in the filename which is limited in usage: it never works
for CPython on Windows since that will use a backslash as path separator,
and also won't work when the filename is a path not relative to the tests
directory which happens for example in the common case of running
"./run-tests -d misc".

Fix all cases by printing only the bare filename, which requires them all
to start with sys_settrace_ hence the renaming.
2020-12-18 13:56:45 +11:00
Damien George 505a1853b9 teensy: Fix build errors and warnings and enable -Werror.
Changes are:
- Remove include of stm32's adc.h because it was recently changed and is
  no longer compatible with teensy (and not used anyway).
- Remove define of __disable_irq in mpconfigport.h because it was clashing
  with an equivalent definition in core/mk20dx128.h.
- Add -Werror to CFLAGS, and change -std=gnu99 to -std=c99.

Signed-off-by: Damien George <damien@micropython.org>
2020-12-18 13:48:57 +11:00
Damien George e715a8fb9b stm32/boards/PYBD_SF2: Disable SPIFLASH_ENABLE_CACHE for mboot builds.
Mboot builds do not use the external SPI flash in caching mode, and
explicitly disabling it saves RAM and a small bit of flash.

Signed-off-by: Damien George <damien@micropython.org>
2020-12-18 13:39:32 +11:00
Damien George e43a74a4db drivers/memory/spiflash: Add MICROPY_HW_SPIFLASH_ENABLE_CACHE option.
This only needs to be enabled if a board uses FAT FS on external SPI flash.
When disabled (and using external SPI flash) 4k of RAM can be saved.

Signed-off-by: Damien George <damien@micropython.org>
2020-12-18 13:39:32 +11:00