Commit Graph

13280 Commits

Author SHA1 Message Date
Daniel Jour 8baf05af8c py/makeqstrdefs: Cleanup and extend source file classification.
- The classification of source files in makeqstrdefs.py has been moved into
  functions to consolidate the logic for that classification into a single
  place.
- Classification of source files (into C or C++ or "other" files) is based
  on the filename extension.
- For C++ there are many more common filename extensions than just ".cpp";
  see "Options Controlling the Kind of Output" in man gcc for example.  All
  common extensions for C++ source files which need preprocessing have been
  added.
2022-04-01 15:03:21 +11:00
Damien George 1dbf393962 tests/basics/fun_callstardblstar: Add test for large arg allocation.
Signed-off-by: Damien George <damien@micropython.org>
2022-04-01 09:20:42 +11:00
Damien George 40f5c743db py/runtime: Remove unnecessary check for kw_value == MP_OBJ_NULL.
The values are always real objects, only the key can be MP_OBJ_NULL to
indicate a **kwargs entry.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-01 09:20:42 +11:00
Damien George bd556b6996 py: Fix compiling and decoding of *args at large arg positions.
There were two issues with the existing code:

1. "1 << i" is computed as a 32-bit number so would overflow when
   executed on 64-bit machines (when mp_uint_t is 64-bit).  This meant that
   *args beyond 32 positions would not be handled correctly.

2. star_args must fit as a positive small int so that it is encoded
   correctly in the emitted code.  MP_SMALL_INT_BITS is too big because it
   overflows a small int by 1 bit.  MP_SMALL_INT_BITS - 1 does not work
   because it produces a signed small int which is then sign extended when
   extracted (even by mp_obj_get_int_truncated), and this sign extension
   means that any position arg after *args is also treated as a star-arg.
   So the maximum bit position is MP_SMALL_INT_BITS - 2.  This means that
   MP_OBJ_SMALL_INT_VALUE() can be used instead of
   mp_obj_get_int_truncated() to get the value of star_args.

These issues are fixed by this commit, and a test added.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-01 09:20:42 +11:00
Damien George e3de723e2d py/emitbc: Assert that a small int fits its encoding when emitting one.
Signed-off-by: Damien George <damien@micropython.org>
2022-03-31 23:59:10 +11:00
David Lechner 2e3f2045f9 py/runtime: Use size_t/ssize_t instead of uint/int.
This replaces instances of uint with size_t and int with ssize_t in
the mp_call_prepare_args_n_kw_var() function since all of the variables
are used as array offsets.

Also sort headers while we are touching this.

Signed-off-by: David Lechner <david@pybricks.com>
2022-03-31 17:01:25 +11:00
David Lechner 47685180f0 tests/basics/fun_callstardblstar: Add coverage test.
This fixes code coverage for the case where a *arg without __len__ is
unpacked and uses exactly the amount of memory that was allocated for
kw args. This triggers the code branch where the memory for the kw args
gets reallocated since it was used already by the *arg unpacking.

Signed-off-by: David Lechner <david@pybricks.com>
2022-03-31 17:01:15 +11:00
David Lechner 9b74d71aa7 py/runtime: Drop new_alloc < 4 check.
To reach this check, n_kw has to be >= 1 and therefore args2_alloc has
to be >= 2. Therefore new_alloc will always be >= 4. So this check will
never be true and can be removed.

Signed-off-by: David Lechner <david@pybricks.com>
2022-03-31 17:01:03 +11:00
David Lechner 3679a47eb0 py/runtime: Do not overallocate when len is known.
This fixes overallocating an extra mp_obj_t when the length of *args and
**args is known. Previously we were allocating 1 mp_obj_t for each
n_args and n_kw plus the length of each *arg and **arg (if they are
known). Since n_args includes *args and n_kw includes **args, this was
allocating an extra mp_obj_t in addition to the length of these args
when unpacked.

To fix this, we just subtract 1 from the length to account for the 1
already implicitly allocated by n_args and n_kw.

Signed-off-by: David Lechner <david@pybricks.com>
2022-03-31 17:00:50 +11:00
David Lechner 783b1a868f py/runtime: Allow multiple *args in a function call.
This is a partial implementation of PEP 448 to allow unpacking multiple
star args in a function or method call.

This is implemented by changing the emitted bytecodes so that both
positional args and star args are stored as positional args.  A bitmap is
added to indicate if an argument at a given position is a positional
argument or a star arg.

In the generated code, this new bitmap takes the place of the old star arg.
It is stored as a small int, so this means only the first N arguments can
be star args where N is the number of bits in a small int.

The runtime is modified to interpret this new bytecode format while still
trying to perform as few memory reallocations as possible.

Signed-off-by: David Lechner <david@pybricks.com>
2022-03-31 16:59:30 +11:00
David Lechner 1e99d29f36 py/runtime: Allow multiple **args in a function call.
This is a partial implementation of PEP 448 to allow multiple ** unpackings
when calling a function or method.

The compiler is modified to encode the argument as a None: obj key-value
pair (similar to how regular keyword arguments are encoded as str: obj
pairs).  The extra object that was pushed on the stack to hold a single **
unpacking object is no longer used and is removed.

The runtime is modified to decode this new format.

Signed-off-by: David Lechner <david@pybricks.com>
2022-03-31 16:54:00 +11:00
Damien George bb70874111 py/vm: Prevent array bound warning when using -MP_OBJ_ITER_BUF_NSLOTS.
This warning can happen on clang 13.0.1 building mpy-cross:

../py/vm.c:748:25: error: array index -3 refers past the last possible
  element for an array in 64-bit address space containing 64-bit (8-byte)
  elements (max possible 2305843009213693952 elements)
  [-Werror,-Warray-bounds]
                        sp[-MP_OBJ_ITER_BUF_NSLOTS + 1] = MP_OBJ_NULL;
                        ^  ~~~~~~~~~~~~~~~~~~~~~~~~~~~

Using pointer access instead of array access works around this warning.

Fixes issue #8467.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-31 10:59:55 +11:00
Damien George 7e8222ae06 py/emitnative: Don't store prelude at end of machine code if not needed.
Signed-off-by: Damien George <damien@micropython.org>
2022-03-30 16:32:17 +11:00
Damien George bf3585b33c py/asmxtensa: Fix use of l32i/s32i when offset won't fit in encoding.
This commit adds optimised l32i/s32i functions that select the best load/
store encoding based on the size of the offset, and uses the function when
necessary in code generation.

Without this, ASM_LOAD_REG_REG_OFFSET() could overflow the word offset
(using a narrow encoding), for example when loading the prelude from the
constant table when there are many (>16) constants.

Fixes issue #8458.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-30 16:32:17 +11:00
Damien George df9a412206 py/compile: Only show raw code that is bytecode.
Signed-off-by: Damien George <damien@micropython.org>
2022-03-30 16:31:53 +11:00
stijn 594c753c27 py/bc.h: Fix C++20 compilation with "volatile".
C++20 is deprecating several usages of the volatile keyword so remove this
one affected case in the codebase which causes such warning.
2022-03-30 16:29:25 +11:00
Damien George 90aaf2dbef extmod/uasyncio: Fix gather cancelling and handling of exceptions.
The following fixes are made:
- cancelling a gather now cancels all sub-tasks of the gather (previously
  it would only cancel the first)
- if any sub-task of a gather raises an exception then the gather finishes
  (previously it would only finish if the first sub-task raised)

Fixes issues #5798, #7807, #7901.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-30 16:07:44 +11:00
Damien George 335002a4c0 extmod/uasyncio: Allow task state to be a callable.
This implements a form of CPython's "add_done_callback()", but at this
stage it is a hidden feature and only intended to be used internally.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-30 16:07:44 +11:00
Andrew Leech a41bc5a7ca stm32/modmachine: Add deepsleep support to reset_cause() for WB55. 2022-03-30 15:56:26 +11:00
Meriç SARIIŞIK 6b23f7d301 stm32/sdio: Use runtime calculation for clock divider of sdio on H7.
STM32H7 family has a different calculation compared to the current one for
the SDMMC clock divider configuration.
2022-03-30 15:40:59 +11:00
Andrew Leech 303e222f70 nrf/modules: Include uasyncio in default board manifest. 2022-03-30 15:37:49 +11:00
Andrew Leech f92da1adc4 nrf/drivers/usb: Fix MP_STREAM_POLL_RD support on USB CDC.
This gets ipoll working on USB CDC stdin.
2022-03-30 15:34:25 +11:00
Andrew Leech dc8b43adc7 nrf/drivers/usb: Fix background events/scheduling while at USB REPL. 2022-03-30 15:33:29 +11:00
Waterlens 4c252ae067 tools/mpremote: Allow running mpremote with `python -m`.
This is helpful because some scripts are likely to use mpremote with a
specific python path.
2022-03-30 15:29:26 +11:00
Damien George 3e70be8ee9 tests/extmod: Update I2S rate test to work on mimxrt.
Tested on Teensy 4.0.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-30 14:14:57 +11:00
MikeTeachman 1f6cb8f047 mixmrt/machine_i2s: Add I2S protocol support.
This commit adds support for machine.I2S on the mimxrt port.  The I2S API
is consistent with the existing stm32, esp32, and rp2 implementations.

I2S features:
- controller transmit and controller receive
- 16-bit and 32-bit sample sizes
- mono and stereo formats
- sampling frequencies from 8kHz to 48kHz
- 3 modes of operation:
  - blocking
  - non-blocking with callback
  - uasyncio
- configurable internal buffer
- optional MCK

Tested with the following development boards:
- MIMXRT1010_EVK, MIMXRT1015_EVK, MIMXRT1020_EVK, MIMXRT1050_EVK
- Teensy 4.0, Teensy 4.1
- Olimex RT1010
- Seeed ARCH MIX

Tested with the following I2S hardware peripherals:
- UDA1334
- GY-SPH0645LM4H
- WM8960 codec on board the MIMXRT boards and separate breakout board
- INMP441
- PCM5102
- SGTL5000 on the Teensy audio shield

Signed-off-by: Mike Teachman <mike.teachman@gmail.com>
2022-03-30 14:12:40 +11:00
Damien George 5e685a9c6f docs/library/machine.I2S: Clarify what rate refers to.
Signed-off-by: Damien George <damien@micropython.org>
2022-03-29 11:44:08 +11:00
Damien George 7266285845 tests/extmod: Add test for machine.I2S data rate.
Signed-off-by: Damien George <damien@micropython.org>
2022-03-29 11:44:08 +11:00
Damien George 6804a8a891 stm32/machine_i2s: Fix 16-bit stereo i2s_frame_map.
Signed-off-by: Damien George <damien@micropython.org>
2022-03-29 11:44:08 +11:00
Damien George 2a91c8a888 stm32/machine_i2s: Allow I2S.deinit to be called multiple times.
In particular, it is called by the constructor if the instance already
exists.  So if the previous instance was deinit'd then it will be deinit'd
a second time.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-29 11:44:08 +11:00
Damien George c90dfba04c stm32/machine_i2s: Set FullDuplexMode to disabled on F4.
Signed-off-by: Damien George <damien@micropython.org>
2022-03-29 11:44:08 +11:00
Damien George 35dbde163a tools/mpremote: Support any prompt string when detecting soft reset.
The prompt may be changed by sys.ps1.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-28 16:47:50 +11:00
Damien George b312a7abf5 py/builtinimport: Alias sys to usys if import weak links aren't enabled.
The sys module should always be available (if it's compiled in), eg to
change sys.path for importing.  So provide an explicit alias from "sys" to
"usys" so that "import sys" can always work.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-28 16:09:58 +11:00
Damien George 6d11c69983 py: Change jump-if-x-or-pop opcodes to have unsigned offset argument.
These jumps are always forwards, and it's more efficient in the VM to
decode an unsigned argument.  These opcodes are already optimised versions
of the sequence "dup-top pop-jump-if-x pop" so it doesn't hurt generality
to optimise them further.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-28 15:43:09 +11:00
Damien George acd2c5c834 py/emitbc: Add check for bytecode jump offset overflow.
Signed-off-by: Damien George <damien@micropython.org>
2022-03-28 15:41:51 +11:00
Damien George 538c3c0a55 py: Change jump opcodes to emit 1-byte jump offset when possible.
This commit introduces changes:

- All jump opcodes are changed to have variable length arguments, of either
  1 or 2 bytes (previously they were fixed at 2 bytes).  In most cases only
  1 byte is needed to encode the short jump offset, saving bytecode size.

- The bytecode emitter now selects 1 byte jump arguments when the jump
  offset is guaranteed to fit in 1 byte.  This is achieved by checking if
  the code size changed during the last pass and, if it did (if it shrank),
  then requesting that the compiler make another pass to get the correct
  offsets of the now-smaller code.  This can continue multiple times until
  the code stabilises.  The code can only ever shrink so this iteration is
  guaranteed to complete.  In most cases no extra passes are needed, the
  original 4 passes are enough to get it right by the 4th pass (because the
  2nd pass computes roughly the correct labels and the 3rd pass computes
  the correct size for the jump argument).

This change to the jump opcode encoding reduces .mpy files and RAM usage
(when bytecode is in RAM) by about 2% on average.

The performance of the VM is not impacted, at least within measurment of
the performance benchmark suite.

Code size is reduced for builds that include a decent amount of frozen
bytecode.  ARM Cortex-M builds without any frozen code increase by about
350 bytes.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-28 15:41:38 +11:00
Damien George 9e3e67b1d8 py/objgenerator: Fix unused variables when native gen extracts prelude.
Some compilers will warn about unused variables like scope_flags.  So use
MP_BC_PRELUDE_SIG_DECODE() which will silence these warnings.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-25 12:35:49 +11:00
David Lechner 768879f999 py/smallint: Introduce MP_SMALL_INT_BITS macro.
This adds a new MP_SMALL_INT_BITS macro that is a compile-time constant
that contains the number of bits available in an MP_SMALL_INT.

We can use this in place of the runtime function mp_small_int_bits().

Signed-off-by: David Lechner <david@pybricks.com>
2022-03-25 12:23:43 +11:00
David Lechner e7f6b9f4f7 tools/gen-cpydiff: Skip Black fmt comments.
Since cpydiff is code used as documentation, there are cases where we may
want to use Black's `fmt: on/off/skip` comments to avoid automatic
formatting.  However, we don't want these comments to be distracting in the
generated documentation.

This rewrites the code to omit these comments when generating the docs.

Signed-off-by: David Lechner <david@pybricks.com>
2022-03-25 12:13:00 +11:00
David Lechner e7a92c0e69 tests/cmdline/cmd_showbc: Fix spelling of sequence. 2022-03-25 12:11:17 +11:00
Damien George b083cdba2b examples/embedding: Fix build with updated sys and os modules.
Signed-off-by: Damien George <damien@micropython.org>
2022-03-25 11:47:30 +11:00
Damien George 73623d04d5 github/workflows: Add new workflow to test embedding example.
Signed-off-by: Damien George <damien@micropython.org>
2022-03-25 11:39:00 +11:00
iabdalkader df86cef59a rp2: Allow Overriding cmake frozen manifest from the command line.
If MICROPY_FROZEN_MANIFEST is set from the cmake command line, then it will
override the default and any manifest set by the board.
2022-03-24 22:10:01 +11:00
iabdalkader fe8b47e29f ports: Allow boards to define additional network interfaces. 2022-03-24 17:23:28 +11:00
Damien George 66fe3d5cb5 stm32: Support building for STM32F745.
Signed-off-by: Damien George <damien@micropython.org>
2022-03-22 16:01:09 +11:00
Andrew Leech bb0ca00b77 stm32/mboot: Verify CRC32 of fsload DFU files before writing. 2022-03-22 15:04:41 +11:00
Andrew Leech bc856a1e29 stm32/mboot: Verify signature of fsload packed DFU files before writing.
When verifying the DFU contents, the signature of signed/encrypted files is
also now checked in this initial, dry-run stage.
2022-03-22 15:04:37 +11:00
Damien George 80055c2cdc stm32/mboot/fwupdate.py: Simplify calculation of CRC32.
Signed-off-by: Damien George <damien@micropython.org>
2022-03-22 14:14:31 +11:00
Damien George bf89e336b9 stm32/boards/PYBD_SF2: Turn on SD card in mboot init if SD enabled.
Signed-off-by: Damien George <damien@micropython.org>
2022-03-22 13:33:43 +11:00
Damien George e316306546 stm32/mboot: Add support for reading from SD card.
Tested on PYBV10 and PYBD_SF6, with MBOOT_FSLOAD enabled and programming
new firmware from a .dfu.gz file stored on the SD card.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-22 13:33:43 +11:00