Prevent handle leaks when file objects aren't closed explicitly and
fix some MICROPY_CPYTHON_COMPAT issues: this wasn't properly adhered
to because #ifdef was used so it was always on, and closing files
multiple times should be avoided unconditionally.
This improves error messages in mpy-cross:
- When loading a .py file that doesn't exist (or can't be opened) it now
includes the filename in the OSError.
- When saving a .mpy file that can't be opened it now raises an exception
(prior, it would silently fail), and includes the filename in the
OSError.
Signed-off-by: Damien George <damien@micropython.org>
This matches class `__dict__`, and is similarly gated on
MICROPY_CPYTHON_COMPAT. Unlike class though, because modules's globals are
actually dict instances, the result is a mutable dictionary.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The intent is to allow us to make breaking changes to the native ABI (e.g.
changes to dynruntime.h) without needing the bytecode version to increment.
With this commit the two bits previously used for the feature flags (but
now unused as of .mpy version 6) encode a sub-version. A bytecode-only
.mpy file can be loaded as long as MPY_VERSION matches, but a native .mpy
(i.e. one with an arch set) must also match MPY_SUB_VERSION. This allows 3
additional updates to the native ABI per bytecode revision.
The sub-version is set to 1 because the previous commits that changed the
layout of mp_obj_type_t have changed the native ABI.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
The check for make_new (i.e. used to determine something's type) is now
more complicated due to the slot access. This commit changes the inlining
of a few frequently-used helpers to overall improve code size and
performance.
Instead of being an explicit field, it's now a slot like all the other
methods.
This is a marginal code size improvement because most types have a make_new
(100/138 on PYBV11), however it improves consistency in how types are
declared, removing the special case for make_new.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The goal here is to remove a slot (making way to turn make_new into a slot)
as well as reduce code size by the ~40 references to mp_identity_getiter
and mp_stream_unbuffered_iter.
This introduces two new type flags:
- MP_TYPE_FLAG_ITER_IS_ITERNEXT: This means that the "iter" slot in the
type is "iternext", and should use the identity getiter.
- MP_TYPE_FLAG_ITER_IS_CUSTOM: This means that the "iter" slot is a pointer
to a mp_getiter_iternext_custom_t instance, which then defines both
getiter and iternext.
And a third flag that is the OR of both, MP_TYPE_FLAG_ITER_IS_STREAM: This
means that the type should use the identity getiter, and
mp_stream_unbuffered_iter as iternext.
Finally, MP_TYPE_FLAG_ITER_IS_GETITER is defined as a no-op flag to give
the default case where "iter" is "getiter".
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Rather than reserving a full 12-slot mp_obj_type_t, reserve enough room for
seven and cast as necessary.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
In all cases other than where you have a native base with a protocol, it
now fits into 4 GC blocks (like it did before the slots representation).
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The existings mp_obj_type_t uses a sparse representation for slots for the
capability methods of the type (eg print, make_new). This commit adds a
compact slot-index representation. The basic idea is that where the
mp_obj_type_t struct used to have 12 pointer fields, it now has 12 uint8_t
indices, and a variable-length array of pointers. So in the best case (no
fields used) it saves 12x4-12=36 bytes (on a 32-bit machine) and in the
common case (three fields used) it saves 9x4-12=24 bytes.
Overall with all associated changes, this slot-index representation reduces
code size by 1000 to 3000 bytes on bare-metal ports. Performance is
marginally better on a few tests (eg about 1% better on misc_pystone.py and
misc_raytrace.py on PYBv1.1), but overall marginally worse by a percent or
so.
See issue #7542 for further analysis and discussion.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This will always have the maximum/minimum size of a mp_obj_type_t
representation and can be used as a member in other structs.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This will allow the structure of mp_obj_type_t to change while keeping the
definition code the same.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The buffer protocol type only has a single member, and this existing layout
creates problems for the upcoming split/slot-index mp_obj_type_t layout
optimisations.
If we need to make the buffer protocol more sophisticated in the future
either we can rely on the mp_obj_type_t optimisations to just add
additional slots to mp_obj_type_t or re-visit the buffer protocol then.
This change is a no-op in terms of generated code.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
In that case, no Pin will be configured for the CS signal, even if it is
internally still generated. That setting allows to use any pin for CS,
which then must be controlled by the Python script.
Also make the default cs=-1 to match other ports (software CS).
The new teensy loader keeps the file system under certain conditions:
- The file size is properly set in the file header.
- The header version is 4.3
These changes are implemented here, requiring a backport of
fsl_flexspi_nor_boot.c. There is still a problem with the command line
version of the teensy loader, which fails on the first attempt. At the
second attempt it works. The GUI version of the teensy loader is fine.
This is a latent issue that wasn't caught by CI because there was no
configuration that had both stackless+uasyncio.
The previous check to skip with stackless builds only worked when the
bytecode emitter was used by default. Force the check to use the bytecode
emitter.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
All variants (except minimal) enable text compression and fat/lfs, so move
them to the common mpconfigport.mk.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This is a no-op for coverage and minimal.
The standard and dev variants have been merged and enable the same feature
set as a typical bare-metal board. And remove the CI for the dev build.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The freedos variant is untested by CI and is difficult to maintain. The
fast variant is not a good name for what it does.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
When iterating over filesystem/folders with os.iterdir(), an open file
(directory) handle is used internally. Currently this file handle is only
closed once the iterator is completely drained, eg. once all entries have
been looped over / converted into list etc.
If a program opens an iterdir but does not loop over it, or starts to loop
over the iterator but breaks out of the loop, then the handle never gets
closed. In this state, when the iter object is cleaned up by the garbage
collector this open handle can cause corruption of the filesystem.
Fixes issues #6568 and #8506.
In case the version from pypi is installed or some other version is
available in sys.path, prepend `$(TOP)/mpy-cross` to sys.path instead.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This is technically a breaking change, but:
a) We need the end handle to do descriptor discovery properly.
b) We have no possible use for the existing definition handle in the
characteristic result IRQ. None of the methods can use it, and therefore
no existing code should be using it in a way that changing it to a
different integer value should break.
Unfortunately NimBLE doesn't make it easy to get the end handle, so also
implement a mechanism to use the following characteristic to calculate
the previous characteristic's end handle.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Incorrect use of "continue" when the tuple was length zero meant it
broke the rest of the argument handling.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Previous the build directory just used the board name, now make it use the
variant name too.
This shouldn't have any change because the existing directory should not
exist (all builds run by these scripts remove their build directory after
completion), but it makes debugging easier.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
