5633 Commits

Author SHA1 Message Date
Jim Mussared
6da41b5900 py/obj: Merge getiter and iternext mp_obj_type_t slots.
The goal here is to remove a slot (making way to turn make_new into a slot)
as well as reduce code size by the ~40 references to mp_identity_getiter
and mp_stream_unbuffered_iter.

This introduces two new type flags:
- MP_TYPE_FLAG_ITER_IS_ITERNEXT: This means that the "iter" slot in the
  type is "iternext", and should use the identity getiter.
- MP_TYPE_FLAG_ITER_IS_CUSTOM: This means that the "iter" slot is a pointer
  to a mp_getiter_iternext_custom_t instance, which then defines both
  getiter and iternext.

And a third flag that is the OR of both, MP_TYPE_FLAG_ITER_IS_STREAM: This
means that the type should use the identity getiter, and
mp_stream_unbuffered_iter as iternext.

Finally, MP_TYPE_FLAG_ITER_IS_GETITER is defined as a no-op flag to give
the default case where "iter" is "getiter".

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:13 +10:00
Jim Mussared
3c6127dfcf py/objnamedtuple: Optimise slot RAM usage for namedtuple.
Rather than reserving a full 12-slot mp_obj_type_t, reserve enough room for
seven and cast as necessary.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:13 +10:00
Jim Mussared
165388e4eb py/objtype: Optimise slot RAM usage for instance types.
In all cases other than where you have a native base with a protocol, it
now fits into 4 GC blocks (like it did before the slots representation).

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:13 +10:00
Jim Mussared
cb0ffdd2bf py/obj: Remove basic mp_obj_type_t sparse representation.
This makes the slots-based representation the only option.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:12 +10:00
Jim Mussared
3ac8b5851e py/obj: Add slot-index mp_obj_type_t representation.
The existings mp_obj_type_t uses a sparse representation for slots for the
capability methods of the type (eg print, make_new).  This commit adds a
compact slot-index representation.  The basic idea is that where the
mp_obj_type_t struct used to have 12 pointer fields, it now has 12 uint8_t
indices, and a variable-length array of pointers.  So in the best case (no
fields used) it saves 12x4-12=36 bytes (on a 32-bit machine) and in the
common case (three fields used) it saves 9x4-12=24 bytes.

Overall with all associated changes, this slot-index representation reduces
code size by 1000 to 3000 bytes on bare-metal ports.  Performance is
marginally better on a few tests (eg about 1% better on misc_pystone.py and
misc_raytrace.py on PYBv1.1), but overall marginally worse by a percent or
so.

See issue #7542 for further analysis and discussion.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:09 +10:00
Jim Mussared
a52cd5b07d py/obj: Add accessors for type slots and use everywhere.
This is a no-op, but sets the stage for changing the mp_obj_type_t
representation.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:07 +10:00
Jim Mussared
e8355eb163 py/obj: Add "full" and "empty" non-variable-length mp_obj_type_t.
This will always have the maximum/minimum size of a mp_obj_type_t
representation and can be used as a member in other structs.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:04 +10:00
Jim Mussared
5ddf671944 py/objexcept: Make MP_DEFINE_EXCEPTION use MP_DEFINE_CONST_OBJ_TYPE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:01 +10:00
Jim Mussared
9dce82776d all: Remove unnecessary locals_dict cast.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:01 +10:00
Jim Mussared
b7d6ee9b75 all: Fix #if inside MP_DEFINE_CONST_OBJ_TYPE for msvc.
Changes:

    MP_DEFINE_CONST_OBJ_TYPE(
       ...
       #if FOO
       ...
       #endif
       ...
    );

to:

    MP_DEFINE_CONST_OBJ_TYPE(
       ...
       FOO_TYPE_ATTR
       ...
    );

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:01 +10:00
Jim Mussared
662b9761b3 all: Make all mp_obj_type_t defs use MP_DEFINE_CONST_OBJ_TYPE.
In preparation for upcoming rework of mp_obj_type_t layout.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:01 +10:00
Jim Mussared
cdb880789f py/obj: Add macro to declare ROM mp_obj_type_t instances.
This will allow the structure of mp_obj_type_t to change while keeping the
definition code the same.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:05:50 +10:00
Jim Mussared
fb2a57800a all: Simplify buffer protocol to just a "get buffer" callback.
The buffer protocol type only has a single member, and this existing layout
creates problems for the upcoming split/slot-index mp_obj_type_t layout
optimisations.

If we need to make the buffer protocol more sophisticated in the future
either we can rely on the mp_obj_type_t optimisations to just add
additional slots to mp_obj_type_t or re-visit the buffer protocol then.

This change is a no-op in terms of generated code.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 18:40:39 +10:00
Jeff Epler
f11ef4cc4e
makeversionhdr should be unforgiving
Recently(?) github started making it the default to only copy a single
branch (e.g., main) and NO TAGS into new forks.

This makes the step of the build process that determines the CircuitPython
version not work, because tags are expected to be present. When tags are
not present, the version number is only a git hash. The version number
ends up being 0.0.0.

This causes problems with libraries that check for CircuitPython version
to determine compatibility, among other things.

We'll do other things to improve the situation, such as document it.
But it'd also be good if the build stopped when this detectable condition
occurs.
2022-09-16 10:28:53 -05:00
Dan Halbert
6bb47038d5 Merge remote-tracking branch 'adafruit/main' into adcdma 2022-09-15 20:42:51 -04:00
Jim Mussared
45972fa548 py/mkrules.mk: Add link to build troubleshooting on failure.
Also update the submodules help text to match.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-15 12:22:18 +10:00
Damien George
74805435f9 py/objpolyiter: Fix comment about finaliser method.
Signed-off-by: Damien George <damien@micropython.org>
2022-09-13 21:14:22 +10:00
Jim Mussared
3e5b1be8ca py/mpconfig: Add "everything" features from unix coverage.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-13 17:40:33 +10:00
Jim Mussared
605266ee9a py/mpconfig: Make feature levels available to mpconfigport.h.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-13 17:37:49 +10:00
Jim Mussared
89a0fefb6c py/mpconfig: Add LFS1/LFS2 options to match FAT/posix.
Also fixes the #ifndef for FAT & posix.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-13 17:37:32 +10:00
Andrew Leech
582b3e4e78 py/objpolyiter: Add a new polyiter type with finaliser support. 2022-09-13 13:00:25 +10:00
Jeff Epler
56112e00a1 Give fontio its own enable
.. the default is intended to be the equivalent of the original,
implementing `DISPLAYIO && TERMINALIO`.

This is a possible alternative to #6889, if I understand the intent.
2022-09-11 08:09:14 -05:00
Lee Atkinson
f279a2dbb2
Merge branch 'adafruit:main' into adcdma 2022-09-08 11:46:01 -04:00
latkinso42
a1856ea3e9 Renaming module from adcbuffer to analogbufio 2022-09-05 12:11:49 -04:00
Dan Halbert
52080e24eb status bar control 2022-08-30 15:23:44 -04:00
Andrew Leech
d521899e18 py/persistentcode: Clarify ValueError when native emitter disabled. 2022-08-29 12:38:49 +10:00
Jim Mussared
6c3d8d38bf py/objstr: Always validate utf-8 for mp_obj_new_str.
All uses of this are either tiny strings or not-known-to-be-safe.

Update comments for mp_obj_new_str_copy and mp_obj_new_str_of_type.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-26 16:45:46 +10:00
Jim Mussared
3a910b1565 py/objstr: Optimise mp_obj_new_str_from_vstr for known-safe strings.
The new `mp_obj_new_str_from_utf8_vstr` can be used when you know you
already have a unicode-safe string.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-26 16:44:35 +10:00
Jim Mussared
88864587f5 py/objstr: Always ensure mp_obj_str_from_vstr is unicode-safe.
Now that we have `mp_obj_new_str_type_from_vstr` (private helper used by
objstr.c) split from the public API (`mp_obj_new_str_from_vstr`), we can
enforce a unicode check at the public API without incurring a performance
cost on the various objstr.c methods (which are already working on known
unicode-safe strings).

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-26 16:44:20 +10:00
Jim Mussared
8a0ee5a5c0 py/objstr: Split mp_obj_str_from_vstr into bytes/str versions.
Previously the desired output type was specified.  Now make the type part
of the function name.  Because this function is used in a few places this
saves code size due to smaller call-site.

This makes `mp_obj_new_str_type_from_vstr` a private function of objstr.c
(which is almost the only place where the output type isn't a compile-time
constant).

This saves ~140 bytes on PYBV11.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-26 16:43:55 +10:00
Lee Atkinson
6fd08483e2 Tidying code for PR/ Minor Issues 2022-08-24 17:41:51 -04:00
Laurens Valk
d8ad87843a py/builtinimport: Allow overriding of mp_builtin___import__.
This allows ports to override mp_builtin___import__.

This can be useful in MicroPython applications where
MICROPY_ENABLE_EXTERNAL_IMPORT has to be disabled due to its impact on
build size (2% to 2.5% of the minimal port). By overriding the otherwise
very minimal mp_builtin___import__, ports can still allow limited forms
of application-specific imports.

Signed-off-by: Laurens Valk <laurens@pybricks.com>
2022-08-23 13:34:06 +10:00
Damien George
3d65101a8a py: Clean up formatting of union definitions.
Signed-off-by: Damien George <damien@micropython.org>
2022-08-23 13:09:57 +10:00
Lee Atkinson
f5655dd48c Fixing Build errors/ DMA Xfer Rate 2022-08-21 11:44:40 -04:00
Lee Atkinson
61591ac136 adding __init.c to circuitpy_defs.mk 2022-08-19 13:17:37 -04:00
Jim Mussared
af1f167820 py/dynruntime: Add mp_obj_is_true.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-19 23:31:11 +10:00
Lee Atkinson
d3c3c9eac4 Tidy a few things 2022-08-18 19:10:16 -04:00
Lee Atkinson
f91af513b7 Introduced odule adcbuffer / removed analogio/AnalogFastIn 2022-08-18 16:23:17 -04:00
Lee Atkinson
34b8fbaf14
Merge branch 'adafruit:main' into adcdma 2022-08-18 09:08:10 -04:00
Damien George
8f4c108025 all: Remove MICROPY_PY_IO_FILEIO config option.
Since commit e65d1e69e88268145ff0e7e73240f028885915be there is no longer an
io.FileIO class, so this option is no longer needed.

This option also controlled whether or not files supported being opened in
binary mode (eg 'rb'), and could, if disabled, lead to confusion as to why
opening a file in binary mode silently did the wrong thing (it would just
open in text mode if MICROPY_PY_IO_FILEIO was disabled).

The various VFS implementations (POSIX, FAT, LFS) were the only places
where enabling this option made a difference, and in almost all cases where
one of these filesystems were enabled, MICROPY_PY_IO_FILEIO was also
enabled.  So it makes sense to just unconditionally enable this feature
(ability to open a file in binary mode) in all cases, and so just remove
this config option altogether.  That makes configuration simpler and means
binary file support always exists (and opening a file in binary mode is
arguably more fundamental than opening in text mode, so if anything should
be configurable then it should be the ability to open in text mode).

Signed-off-by: Damien George <damien@micropython.org>
2022-08-18 11:54:17 +10:00
Scott Shawcroft
7717ab8e13
Merge pull request #6754 from jepler/check-read-utf8
When reading data from a file into a str, check if it's utf-8
2022-08-15 13:48:43 -07:00
Lee Atkinson
c53c1c0927 Adding py/circuitpy_mpconfig.mk 2022-08-14 18:49:59 -04:00
Lee Atkinson
b2c6bcdf6d Fixing py/circuitpy_defns.mk 2022-08-14 14:22:22 -04:00
Lee Atkinson
4542c801b0 Tidying up loose endson draft PR 2022-08-14 13:03:33 -04:00
Dan Ellis
6f4d424f46 py/formatfloat: Use pow(10, e) instead of pos/neg_pow lookup tables.
Rework the conversion of floats to decimal strings so it aligns precisely
with the conversion of strings to floats in parsenum.c.  This is to avoid
rendering 1eX as 9.99999eX-1 etc.  This is achieved by removing the power-
of-10 tables and using pow() to compute the exponent directly, and that's
done efficiently by first estimating the power-of-10 exponent from the
power-of-2 exponent in the floating-point representation.

Code size is reduced by roughly 100 to 200 bytes by this commit.

Signed-off-by: Dan Ellis <dan.ellis@gmail.com>
2022-08-12 23:53:34 +10:00
Dan Ellis
6cd2e41918 py/parsenum: Ensure that trailing zeros lead to identical results.
Prior to this commit, parsenum would calculate "1e-20" as 1.0*pow(10, -20),
and "1.000e-20" as 1000.0*pow(10, -23); in certain cases, this could make
seemingly-identical values compare as not equal.  This commit watches for
trailing zeros as a special case, and ignores them when appropriate, so
"1.000e-20" is also calculated as 1.0*pow(10, -20).

Fixes issue #5831.
2022-08-12 23:44:11 +10:00
Jeff Epler
06f22cb59c
When reading data from a file into a str, check if it's utf-8
Otherwise, weird stuff can happen down the line when it is print()ed,
especially as it can break the webrepl of circuitpython.
2022-08-12 08:25:27 -05:00
Damien George
cf90e24335 py/mkrules: Use abspath to find directory for mpy-cross dependency.
Otherwise if the `mpy-cross/build/` directory doesn't exist then
`mpy-cross/build/..` won't work.

Signed-off-by: Damien George <damien@micropython.org>
2022-08-12 16:38:24 +10:00
Damien George
945f377b43 py/objstr: Remove str function object declarations from header file.
Since f7f56d42851aaff2027e23a8ca45c1f1973f1aca consolidated all uses of
these to a single locals dict, they no longer need to be made public.

Signed-off-by: Damien George <damien@micropython.org>
2022-08-12 16:38:22 +10:00
Jim Mussared
28aaab9590 py/objstr: Add hex/fromhex to bytes/memoryview/bytearray.
These were added in Python 3.5.

Enabled via MICROPY_PY_BUILTINS_BYTES_HEX, and enabled by default for all
ports that currently have ubinascii.

Rework ubinascii to use the implementation of these methods.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-12 12:44:30 +10:00