Commit Graph

14040 Commits

Author SHA1 Message Date
Jim Mussared
d94141e147 py/persistentcode: Introduce .mpy sub-version.
The intent is to allow us to make breaking changes to the native ABI (e.g.
changes to dynruntime.h) without needing the bytecode version to increment.

With this commit the two bits previously used for the feature flags (but
now unused as of .mpy version 6) encode a sub-version.  A bytecode-only
.mpy file can be loaded as long as MPY_VERSION matches, but a native .mpy
(i.e. one with an arch set) must also match MPY_SUB_VERSION.  This allows 3
additional updates to the native ABI per bytecode revision.

The sub-version is set to 1 because the previous commits that changed the
layout of mp_obj_type_t have changed the native ABI.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
2022-09-19 23:19:55 +10:00
Jim Mussared
b41aaaa8a9 py/obj: Optimise code size and performance for make_new as a slot.
The check for make_new (i.e. used to determine something's type) is now
more complicated due to the slot access.  This commit changes the inlining
of a few frequently-used helpers to overall improve code size and
performance.
2022-09-19 19:06:16 +10:00
Jim Mussared
94beeabd2e py/obj: Convert make_new into a mp_obj_type_t slot.
Instead of being an explicit field, it's now a slot like all the other
methods.

This is a marginal code size improvement because most types have a make_new
(100/138 on PYBV11), however it improves consistency in how types are
declared, removing the special case for make_new.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:15 +10:00
Jim Mussared
6da41b5900 py/obj: Merge getiter and iternext mp_obj_type_t slots.
The goal here is to remove a slot (making way to turn make_new into a slot)
as well as reduce code size by the ~40 references to mp_identity_getiter
and mp_stream_unbuffered_iter.

This introduces two new type flags:
- MP_TYPE_FLAG_ITER_IS_ITERNEXT: This means that the "iter" slot in the
  type is "iternext", and should use the identity getiter.
- MP_TYPE_FLAG_ITER_IS_CUSTOM: This means that the "iter" slot is a pointer
  to a mp_getiter_iternext_custom_t instance, which then defines both
  getiter and iternext.

And a third flag that is the OR of both, MP_TYPE_FLAG_ITER_IS_STREAM: This
means that the type should use the identity getiter, and
mp_stream_unbuffered_iter as iternext.

Finally, MP_TYPE_FLAG_ITER_IS_GETITER is defined as a no-op flag to give
the default case where "iter" is "getiter".

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:13 +10:00
Jim Mussared
3c6127dfcf py/objnamedtuple: Optimise slot RAM usage for namedtuple.
Rather than reserving a full 12-slot mp_obj_type_t, reserve enough room for
seven and cast as necessary.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:13 +10:00
Jim Mussared
165388e4eb py/objtype: Optimise slot RAM usage for instance types.
In all cases other than where you have a native base with a protocol, it
now fits into 4 GC blocks (like it did before the slots representation).

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:13 +10:00
Jim Mussared
cb0ffdd2bf py/obj: Remove basic mp_obj_type_t sparse representation.
This makes the slots-based representation the only option.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:12 +10:00
Jim Mussared
3ac8b5851e py/obj: Add slot-index mp_obj_type_t representation.
The existings mp_obj_type_t uses a sparse representation for slots for the
capability methods of the type (eg print, make_new).  This commit adds a
compact slot-index representation.  The basic idea is that where the
mp_obj_type_t struct used to have 12 pointer fields, it now has 12 uint8_t
indices, and a variable-length array of pointers.  So in the best case (no
fields used) it saves 12x4-12=36 bytes (on a 32-bit machine) and in the
common case (three fields used) it saves 9x4-12=24 bytes.

Overall with all associated changes, this slot-index representation reduces
code size by 1000 to 3000 bytes on bare-metal ports.  Performance is
marginally better on a few tests (eg about 1% better on misc_pystone.py and
misc_raytrace.py on PYBv1.1), but overall marginally worse by a percent or
so.

See issue #7542 for further analysis and discussion.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:09 +10:00
Jim Mussared
a52cd5b07d py/obj: Add accessors for type slots and use everywhere.
This is a no-op, but sets the stage for changing the mp_obj_type_t
representation.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:07 +10:00
Jim Mussared
e8355eb163 py/obj: Add "full" and "empty" non-variable-length mp_obj_type_t.
This will always have the maximum/minimum size of a mp_obj_type_t
representation and can be used as a member in other structs.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:04 +10:00
Jim Mussared
5ddf671944 py/objexcept: Make MP_DEFINE_EXCEPTION use MP_DEFINE_CONST_OBJ_TYPE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:01 +10:00
Jim Mussared
9dce82776d all: Remove unnecessary locals_dict cast.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:01 +10:00
Jim Mussared
b7d6ee9b75 all: Fix #if inside MP_DEFINE_CONST_OBJ_TYPE for msvc.
Changes:

    MP_DEFINE_CONST_OBJ_TYPE(
       ...
       #if FOO
       ...
       #endif
       ...
    );

to:

    MP_DEFINE_CONST_OBJ_TYPE(
       ...
       FOO_TYPE_ATTR
       ...
    );

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:01 +10:00
Jim Mussared
662b9761b3 all: Make all mp_obj_type_t defs use MP_DEFINE_CONST_OBJ_TYPE.
In preparation for upcoming rework of mp_obj_type_t layout.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:01 +10:00
Jim Mussared
cdb880789f py/obj: Add macro to declare ROM mp_obj_type_t instances.
This will allow the structure of mp_obj_type_t to change while keeping the
definition code the same.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:05:50 +10:00
Jim Mussared
42587c7870 all: Standardise mp_obj_type_t initialisation.
Remove setting unused slots.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 18:41:29 +10:00
Jim Mussared
fb2a57800a all: Simplify buffer protocol to just a "get buffer" callback.
The buffer protocol type only has a single member, and this existing layout
creates problems for the upcoming split/slot-index mp_obj_type_t layout
optimisations.

If we need to make the buffer protocol more sophisticated in the future
either we can rely on the mp_obj_type_t optimisations to just add
additional slots to mp_obj_type_t or re-visit the buffer protocol then.

This change is a no-op in terms of generated code.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 18:40:39 +10:00
Jim Mussared
ca51d63c37 rp2/Makefile: Add link to build troubleshooting on failure.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-15 12:22:18 +10:00
Jim Mussared
18d0e6d0db esp32/Makefile: Add link to build troubleshooting on failure.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-15 12:22:18 +10:00
Jim Mussared
45972fa548 py/mkrules.mk: Add link to build troubleshooting on failure.
Also update the submodules help text to match.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-15 12:22:18 +10:00
Damien George
0e8c2204da esp32/mphalport: Fix calculation of large sleep by using 64-bit arith.
Fixes issue #9304.

Signed-off-by: Damien George <damien@micropython.org>
2022-09-14 14:03:37 +10:00
Damien George
74805435f9 py/objpolyiter: Fix comment about finaliser method.
Signed-off-by: Damien George <damien@micropython.org>
2022-09-13 21:14:22 +10:00
robert-hh
6472348c1b mimxrt/machine_uart: Fix a bug in UART.write().
Causing an incomplete send if the data size was longer than the buffer
size.
2022-09-13 18:35:48 +10:00
robert-hh
443fbcf81c mimxrt/machine_uart: Set the UART ioctl write poll flag properly.
It was always set to True.  The change adds a check to the tx status flag
which is set when all data is transferred.
2022-09-13 18:35:48 +10:00
robert-hh
0f048a5a2a mimxrt/machine_spi: Allow a setting of -1 for cs in the constructor.
In that case, no Pin will be configured for the CS signal, even if it is
internally still generated.  That setting allows to use any pin for CS,
which then must be controlled by the Python script.

Also make the default cs=-1 to match other ports (software CS).
2022-09-13 18:35:48 +10:00
robert-hh
8e54225140 mimxrt: Format the firmware image to match the new teensy loader.
The new teensy loader keeps the file system under certain conditions:
- The file size is properly set in the file header.
- The header version is 4.3

These changes are implemented here, requiring a backport of
fsl_flexspi_nor_boot.c.  There is still a problem with the command line
version of the teensy loader, which fails on the first attempt.  At the
second attempt it works.  The GUI version of the teensy loader is fine.
2022-09-13 18:35:48 +10:00
robert-hh
65ce7d7ade mimxrt/machine_uart: Drop a few commented lines about TX ring buffer. 2022-09-13 18:35:48 +10:00
Jim Mussared
6c376a9306 tests/extmod/uasyncio_heaplock.py: Force SKIP on stackless.
This is a latent issue that wasn't caught by CI because there was no
configuration that had both stackless+uasyncio.

The previous check to skip with stackless builds only worked when the
bytecode emitter was used by default.  Force the check to use the bytecode
emitter.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-13 17:41:02 +10:00
Jim Mussared
51b054dd66 unix: Refactor mpconfigvariant.mk.
All variants (except minimal) enable text compression and fat/lfs, so move
them to the common mpconfigport.mk.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-13 17:40:55 +10:00
Jim Mussared
3e5b1be8ca py/mpconfig: Add "everything" features from unix coverage.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-13 17:40:33 +10:00
Jim Mussared
c1530a0ce8 unix: Refactor mpconfigport.h and mpconfigvariant.h.
This is a no-op for coverage and minimal.

The standard and dev variants have been merged and enable the same feature
set as a typical bare-metal board.  And remove the CI for the dev build.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-13 17:39:03 +10:00
Jim Mussared
f4fed02537 unix/variants: Remove freedos and fast variants.
The freedos variant is untested by CI and is difficult to maintain.  The
fast variant is not a good name for what it does.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-13 17:37:55 +10:00
Jim Mussared
605266ee9a py/mpconfig: Make feature levels available to mpconfigport.h.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-13 17:37:49 +10:00
Jim Mussared
89a0fefb6c py/mpconfig: Add LFS1/LFS2 options to match FAT/posix.
Also fixes the #ifndef for FAT & posix.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-13 17:37:32 +10:00
Wind-stormger
57fd66b80f tools/pyboard.py: Support Windows pathname separators.
Addresses issue #9132.
2022-09-13 13:28:39 +10:00
Andrew Leech
2d4e7e99bf samd/mpconfigport: Enable MICROPY_ENABLE_FINALISER when VFS is used. 2022-09-13 13:08:32 +10:00
Andrew Leech
afc7e1d298 zephyr/mpconfigport: Enable MICROPY_ENABLE_FINALISER when VFS is used. 2022-09-13 13:08:03 +10:00
Andrew Leech
86676a43aa unix/mpconfigport: Enable MICROPY_ENABLE_FINALISER when VFS is used. 2022-09-13 13:06:56 +10:00
Andrew Leech
4e0964b59f extmod/vfs: Add finaliser to ilistdir to close directory handle.
When iterating over filesystem/folders with os.iterdir(), an open file
(directory) handle is used internally.  Currently this file handle is only
closed once the iterator is completely drained, eg. once all entries have
been looped over / converted into list etc.

If a program opens an iterdir but does not loop over it, or starts to loop
over the iterator but breaks out of the loop, then the handle never gets
closed.  In this state, when the iter object is cleaned up by the garbage
collector this open handle can cause corruption of the filesystem.

Fixes issues #6568 and #8506.
2022-09-13 13:00:42 +10:00
Andrew Leech
582b3e4e78 py/objpolyiter: Add a new polyiter type with finaliser support. 2022-09-13 13:00:25 +10:00
Jim Mussared
4903e48e34 tools/makemanifest.py: Force the repo version of the mpy_cross package.
In case the version from pypi is installed or some other version is
available in sys.path, prepend `$(TOP)/mpy-cross` to sys.path instead.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-09 18:12:55 +10:00
Damien George
1d4228ccc1 zephyr/boards: Add config for bbc_microbit_v2.
This enables the bluetooth module.  GAP scanning and advertising works.

Signed-off-by: Damien George <damien@micropython.org>
2022-09-09 14:14:31 +10:00
Jim Mussared
cacc96d98c extmod/modbluetooth: Replace def_handle with end_handle in char IRQ.
This is technically a breaking change, but:
a) We need the end handle to do descriptor discovery properly.
b) We have no possible use for the existing definition handle in the
characteristic result IRQ. None of the methods can use it, and therefore
no existing code should be using it in a way that changing it to a
different integer value should break.

Unfortunately NimBLE doesn't make it easy to get the end handle, so also
implement a mechanism to use the following characteristic to calculate
the previous characteristic's end handle.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-09 11:24:24 +10:00
Jim Mussared
82fc16f298 extmod/modbluetooth: Fix descriptor registration with empty tuple.
Incorrect use of "continue" when the tuple was length zero meant it
broke the rest of the argument handling.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-09 11:24:21 +10:00
Jim Mussared
6e75d177e7 stm32/boards/PYB: Fix handling of BOARD_VARIANT selection.
The matches should not have been quoted.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-08 14:54:48 +10:00
Jim Mussared
9070a24940 tools/autobuild: Use distinct directory for building stm32 variants.
Previous the build directory just used the board name, now make it use the
variant name too.

This shouldn't have any change because the existing directory should not
exist (all builds run by these scripts remove their build directory after
completion), but it makes debugging easier.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-08 14:54:11 +10:00
Jim Mussared
dec0ff7a10 rp2/boards: Remove all I2C,SPI pin defs that already match pico-sdk.
I.e. for whichever SPI/I2C instance is PICO_DEFAULT_I2C, there's no need to
set MICROPY_HW_SPIn_SCK.

The only ones remaining are for the non-default instance.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-08 14:51:15 +10:00
Jim Mussared
315e74236f rp2/machine_i2c: Use pico-sdk's default pins for I2C.
Inherits the default values for whichever instance is PICO_DEFAULT_I2C.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-08 14:51:11 +10:00
Jim Mussared
29437205f2 rp2/machine_spi: Use pico-sdk's default pins for SPI.
Rather than hardcoding the defaults, use pico-sdk's board definition.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-08 14:51:04 +10:00
Jim Mussared
c364301817 rp2/boards: Set PICO_BOARD correctly for each board.
In most cases, it's calculated automatically from the board name, and so
doesn't need to be set at all.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-08 14:50:54 +10:00