5462 Commits

Author SHA1 Message Date
Scott Shawcroft
a6b60d2083
Merge remote-tracking branch 'adafruit/main' into dotenv 2022-05-23 11:23:20 -07:00
Damien George
c1b9d2259e py/dynruntime.mk: Add basic support for armv6m architecture.
The examples/natmod features0 and features1 examples now build and run on
ARMv6-M platforms.  More complicated examples are not yet supported because
the compiler emits references to built-in functions like __aeabi_uidiv.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-23 23:01:25 +10:00
Damien George
17ac68770c py/persistentcode: Select ARMV6M as maximum when __thumb2__ not defined.
If __thumb2__ is defined by the compiler then .mpy files marked as ARMV6M
and above (up to ARMV7EMDP) are supported.  If it's not defined then only
ARMV6M .mpy files are supported.  This makes sure that on CPUs like
Cortex-M0+ (where __thumb2__ is not defined) only .mpy files marked as
ARMV6M can be imported.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-23 23:01:25 +10:00
Damien George
a5324a1074 py/asmthumb: Make ARMv7-M instruction use dynamically selectable.
This commit adjusts the asm_thumb_xxx functions so they can be dynamically
configured to use ARMv7-M instructions or not.  This is available when
MICROPY_DYNAMIC_COMPILER is enabled, and then controlled by the value of
mp_dynamic_compiler.native_arch.

If MICROPY_DYNAMIC_COMPILER is disabled the previous behaviour is retained:
the functions emit ARMv7-M instructions only if MICROPY_EMIT_THUMB_ARMV7M
is enabled.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-23 23:01:12 +10:00
Damien George
d4d53e9e11 py/emitnative: Access qstr values using indirection table qstr_table.
This changes the native emitter to access qstr values using the qstr
indirection table qstr_table, but only when generating native code that
will be saved to a .mpy file.  This makes the resulting native code fully
static, ie it does not require any fix-ups or rewriting when it is
imported.

The performance of native code is more or less unchanged.  Benchmark
results on PYBv1.0 (using --via-mpy and --emit native) are:

N=100 M=100          baseline -> this-commit     diff      diff% (error%)
bm_chaos.py            407.16 ->     411.85 :   +4.69 =  +1.152% (+/-0.01%)
bm_fannkuch.py         100.89 ->     101.20 :   +0.31 =  +0.307% (+/-0.01%)
bm_fft.py             3521.17 ->    3441.72 :  -79.45 =  -2.256% (+/-0.00%)
bm_float.py           6707.29 ->    6644.83 :  -62.46 =  -0.931% (+/-0.00%)
bm_hexiom.py            55.91 ->      55.41 :   -0.50 =  -0.894% (+/-0.00%)
bm_nqueens.py         5343.54 ->    5326.17 :  -17.37 =  -0.325% (+/-0.00%)
bm_pidigits.py         603.89 ->     632.79 :  +28.90 =  +4.786% (+/-0.33%)
core_qstr.py            64.18 ->      64.09 :   -0.09 =  -0.140% (+/-0.01%)
core_yield_from.py     313.61 ->     311.11 :   -2.50 =  -0.797% (+/-0.03%)
misc_aes.py            654.29 ->     659.75 :   +5.46 =  +0.834% (+/-0.02%)
misc_mandel.py        4205.10 ->    4272.08 :  +66.98 =  +1.593% (+/-0.01%)
misc_pystone.py       3077.79 ->    3128.39 :  +50.60 =  +1.644% (+/-0.01%)
misc_raytrace.py       388.45 ->     393.71 :   +5.26 =  +1.354% (+/-0.01%)
viper_call0.py         576.83 ->     566.76 :  -10.07 =  -1.746% (+/-0.05%)
viper_call1a.py        550.39 ->     540.12 :  -10.27 =  -1.866% (+/-0.11%)
viper_call1b.py        438.32 ->     432.09 :   -6.23 =  -1.421% (+/-0.11%)
viper_call1c.py        442.96 ->     436.11 :   -6.85 =  -1.546% (+/-0.08%)
viper_call2a.py        536.31 ->     527.37 :   -8.94 =  -1.667% (+/-0.04%)
viper_call2b.py        378.99 ->     377.50 :   -1.49 =  -0.393% (+/-0.08%)

Signed-off-by: Damien George <damien@micropython.org>
2022-05-23 15:43:06 +10:00
Damien George
94955e8e3d py/asm: Add ASM_LOAD16_REG_REG_OFFSET macro for load-u16 with offset.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-23 14:21:16 +10:00
Damien George
8af5e2551f py/asmarm: Add asm_arm_ldrh_reg_reg_offset() helper func.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-23 14:21:16 +10:00
Damien George
94ae023136 py/asmthumb: Add asm_thumb_ldrh_reg_reg_i12_optimised() helper func.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-23 14:21:14 +10:00
Damien George
689138d484 py/asmthumb: Fix offset variable name in ldr, ldrh and strh functions.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-23 14:20:10 +10:00
Dan Halbert
9c643aa907 Merge branch '7.3.x' into merge-7.3.0-rc.1-to-main 2022-05-21 12:57:46 -04:00
Dan Halbert
199e2ef2fc Turn on f-strings for all builds 2022-05-21 00:04:53 -04:00
Dan Halbert
309dab4c75 merge 7.3.0-rc.1 changes to main 2022-05-20 15:37:56 -04:00
Dan Halbert
f63b26c534 address jepler's comments and further squeezes 2022-05-20 10:10:55 -04:00
Dan Halbert
a01dec1df9 message consolidation and more use of validators 2022-05-19 15:38:37 -04:00
Damien George
7883ae413d py/emitnative: Provide dedicated local for exception unwind handler ptr.
This eliminates the need to save and restore the exception unwind handler
pointer when calling nlr_push.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-19 17:31:56 +10:00
Damien George
b608964804 py/emitnative: Simplify generation of code that loads prelude pointer.
It's possible to use REG_PARENT_ARG_1 instead of REG_LOCAL_3.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-19 17:31:56 +10:00
Damien George
56f2d3c2e5 py/asmthumb: Fix PC relative load by sign extending the constant.
PC relative offsets can be negative, in which case the movw result must be
sign extended.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-19 17:31:56 +10:00
Scott Shawcroft
a30de85d57
Add dotenv read support
os.getenv() will use it (when available) to load variables from
/.env

This will also be useful when we need secrets or config for
CircuitPython outside of the VM (like WiFi credentials.)

Fixes #4212
2022-05-18 14:01:35 -07:00
Dan Halbert
4487f61404 Restore automatic gc_collect() after an import 2022-05-18 11:37:13 -04:00
Jim Mussared
8b201dc4c3 py: Remove support for MICROPY_PORT_BUILTIN_MODULES.
This functionality is now replaced with MP_REGISTER_MODULE.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:57:09 +10:00
Jim Mussared
d8d3e6ae78 py: Make builtin modules use MP_REGISTER_MODULE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:49:12 +10:00
Jim Mussared
4eab44a1ec extmod: Make extmod modules use MP_REGISTER_MODULE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:49:12 +10:00
Jim Mussared
469450171f py/makemoduledefs.py: Allow multiple ways to register a module.
For example, ussl can come from axtls or mbedtls. If neither are enabled
then don't try and set an empty definition twice, and only include it
once in MICROPY_REGISTERED_MODULES.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:49:12 +10:00
Damien George
079f3e5e5b py/parse: Allow all constant objects to be used in "X = const(o)".
Now that constant tuples are supported in the parser, eg (1, True, "str"),
it's a small step to allow anything that is a constant to be used with the
pattern:

    from micropython import const

    X = const(obj)

This commit makes the required changes to allow the following types of
constants:

    from micropython import const

    _INT = const(123)
    _FLOAT = const(1.2)
    _COMPLEX = const(3.4j)
    _STR = const("str")
    _BYTES = const(b"bytes")
    _TUPLE = const((_INT, _STR, _BYTES))
    _TUPLE2 = const((None, False, True, ..., (), _TUPLE))

Prior to this, only integers could be used in const(...).

Signed-off-by: Damien George <damien@micropython.org>
2022-05-18 16:18:35 +10:00
Damien George
8588525868 py/compile: De-duplicate constant objects in module's constant table.
The recent rework of bytecode made all constants global with respect to the
module (previously, each function had its own constant table).  That means
the constant table for a module is shared among all functions/methods/etc
within the module.

This commit add support to the compiler to de-duplicate constants in this
module constant table.  So if a constant is used more than once -- eg 1.0
or (None, None) -- then the same object is reused for all instances.

For example, if there is code like `print(1.0, 1.0)` then the parser will
create two independent constants 1.0 and 1.0.  The compiler will then (with
this commit) notice they are the same and only put one of them in the
constant table.  The bytecode will then reuse that constant twice in the
print expression.  That allows the second 1.0 to be reclaimed by the GC,
also means the constant table has one less entry so saves a word.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-18 15:23:11 +10:00
Damien George
90682f43af py/compile: Allow new qstrs to be allocated at all compiler passes.
Prior to this commit, all qstrs were required to be allocated (by calling
mp_emit_common_use_qstr) in the MP_PASS_SCOPE pass (the first one).  But
this is an unnecessary restriction, which is lifted by this commit.
Lifting the restriction simplifies the compiler because it can allocate
qstrs in later passes.

This also generates better code, because in some cases (eg when a variable
is closed over) the scope of an identifier is not known until a bit later
and then the identifier no longer needs its qstr allocated in the global
table.

Code size is reduced for all ports with this commit.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 23:39:22 +10:00
Damien George
1fb01bd6c5 py/emitnative: Put a pointer to the native prelude in child_table array.
Some architectures (like esp32 xtensa) cannot read byte-wise from
executable memory.  This means the prelude for native functions -- which is
usually located after the machine code for the native function -- must be
placed in separate memory that can be read byte-wise.  Prior to this commit
this was achieved by enabling N_PRELUDE_AS_BYTES_OBJ for the emitter and
MICROPY_EMIT_NATIVE_PRELUDE_AS_BYTES_OBJ for the runtime.  The prelude was
then placed in a bytes object, pointed to by the module's constant table.

This behaviour is changed by this commit so that a pointer to the prelude
is stored either in mp_obj_fun_bc_t.child_table, or in
mp_obj_fun_bc_t.child_table[num_children] if num_children > 0.  The reasons
for doing this are:

1. It decouples the native emitter from runtime requirements, the emitted
   code no longer needs to know if the system it runs on can/can't read
   byte-wise from executable memory.

2. It makes all ports have the same emitter behaviour, there is no longer
   the N_PRELUDE_AS_BYTES_OBJ option.

3. The module's constant table is now used only for actual constants in the
   Python code.  This allows further optimisations to be done with the
   constants (eg constant deduplication).

Code size change for those ports that enable the native emitter:
   unix x64:   +80 +0.015%
      stm32:   +24 +0.004% PYBV10
    esp8266:   +88 +0.013% GENERIC
      esp32:   -20 -0.002% GENERIC[incl -112(data)]
        rp2:   +32 +0.005% PICO

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 16:44:49 +10:00
Damien George
1762990579 py/bc: Provide separate code-state setup funcs for bytecode and native.
mpy-cross will now generate native code based on the size of
mp_code_state_native_t, and the runtime will use this struct to calculate
the offset of the .state field.  This makes native code generation and
execution (which rely on this struct) independent to the settings
MICROPY_STACKLESS and MICROPY_PY_SYS_SETTRACE, both of which change the
size of the mp_code_state_t struct.

Fixes issue #5059.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 14:25:51 +10:00
Damien George
8e1db993cd py/asmx64: Support full range of regs in asm_x64_lea_disp_to_r64.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 14:25:51 +10:00
Damien George
c49d5207e9 py/persistentcode: Remove unicode feature flag from .mpy file.
Prior to this commit, even with unicode disabled .py and .mpy files could
contain unicode characters, eg by entering them directly in a string as
utf-8 encoded.

The only thing the compiler disallowed (with unicode disabled) was using
\uxxxx and \Uxxxxxxxx notation to specify a character within a string with
value >= 0x100; that would give a SyntaxError.

With this change mpy-cross will now accept \u and \U notation to insert a
character with value >= 0x100 into a string (because the -mno-unicode
option is now gone, there's no way to forbid this).  The runtime will
happily work with strings with such characters, just like it already works
with strings with characters that were utf-8 encoded directly.

This change simplifies things because there are no longer any feature
flags in .mpy files, and any bytecode .mpy will now run on any target.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 12:51:54 +10:00
Damien George
b295b6f1f3 py/persistentcode: Remove obsolete comment about qstr window size.
This was made obsolete in f2040bfc7ee033e48acef9f289790f3b4e6b74e5

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 12:51:54 +10:00
Jeff Epler
016242aa26
Get rid of a memset() for the case where finalisers are enabled
This saves 24 bytes of flash on trinket m0
2022-05-14 06:37:18 -05:00
Scott Shawcroft
269d51d023
Add two stage reset for BLE
This lets the BLE stack run through the wait period after a VM run
when it may be waiting for more writes due to an auto-reload.

User BLE functionality will have their events stopped. Scanning and
advertising is also stopped.
2022-05-12 13:20:44 -07:00
Dan Halbert
58646a936e Correct errno messages 2022-05-10 14:01:30 -04:00
Damien George
5b700b0af9 all: Reformat remaining C code that doesn't have a space after a comma.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-05 13:30:40 +10:00
Damien George
1216c9fffa py/objmodule: Move stray #include to top of file.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-05 11:02:38 +10:00
Damien George
fca5701f74 py/malloc: Introduce m_tracked_calloc, m_tracked_free functions.
Enabled by MICROPY_TRACKED_ALLOC.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-05 10:31:50 +10:00
Jeff Epler
510890b207
Enable squeezing property storage in flash
.. and enable it on atmel-samd and raspberrypi. On trinket_m0 this saves
96 net bytes of flash. There are 216 bytes actually saved by reducing
the flash storage size of the property descriptors, but added code in
several paths takes back over half of the 'raw savings'.

By organizing the "get-only" and "get-set" (but no delete) properties
each in a different section, we can represent then more efficiently.

Testing performed: that a get-only property can still be gotten but
can't be set or deleted; that a get-set property can sill be gotten or
set but can't be deleted.  Tested on pygamer.

Because this requires linker file support, I only enabled it on two of
the ports.
2022-05-03 08:48:53 -05:00
Jeff Epler
78cf0a90af
Make MP_PROPERTY_GETTER / _GETSET fully declare the property
This will enable setting data attributes, namely, the section of the
symbol.
2022-05-03 08:48:53 -05:00
Jeff Epler
5b9f0681ad
Get some getter props defined through another macro 2022-05-03 08:48:53 -05:00
Jeff Epler
31da335cac
Introduce, use MP_PROPERTY_GETTER, _GETSET
Later, these can be changed in cunning ways to save flash storage.
2022-05-03 08:48:52 -05:00
Jim Mussared
0e7bfc88c6 all: Use mp_obj_malloc everywhere it's applicable.
This replaces occurences of

    foo_t *foo = m_new_obj(foo_t);
    foo->base.type = &foo_type;

with

    foo_t *foo = mp_obj_malloc(foo_t, &foo_type);

Excludes any places where base is a sub-field or when new0/memset is used.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-03 22:28:14 +10:00
Jim Mussared
6a3bc0e1a1 py/objfloat: Explain why mp_obj_malloc isn't used.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-03 22:25:40 +10:00
Jim Mussared
709e8328d9 py/obj: Introduce mp_obj_malloc macro to allocate, and set object type.
This is to replace the following:

    mp_foo_obj_t *self = m_new_obj(mp_foo_obj_t);
    self->base.type = &mp_type_foo;

with:

    mp_foo_obj_t *self = mp_obj_malloc(mp_foo_obj_t, &mp_type_foo);

Calling the function is less code than inlining setting the type
everywhere, adds up to ~100 bytes on PYBV11.

It also helps to avoid an easy mistake of forgetting to set the type.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-03 22:23:46 +10:00
Damien George
590de399f0 py/emitcommon: Don't implicitly close class vars that are assigned to.
When in a class body or at the module level don't implicitly close over
variables that have been assigned to.

Fixes issue #8603.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-03 16:38:43 +10:00
Damien George
402df833fe py/modsys: Introduce sys.implementation._machine constant.
This contains a string useful for identifying the underlying machine.  This
string is kept consistent with the second part of the REPL banner via the
new config option MICROPY_BANNER_MACHINE.

This makes os.uname() more or less redundant, as all the information in
os.uname() is now available in the sys module.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-28 17:23:03 +10:00
Damien George
59c5d41611 py/modsys: Rename sys.implementation.mpy to sys.implementation._mpy.
Per CPython docs, non-standard attributes must begin with an underscore.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-28 17:23:03 +10:00
Damien George
40047823bc py/modsys: Append MicroPython git version and build date to sys.version.
This commit adds the git hash and build date to sys.version.  This is
allowed according to CPython docs, and is what PyPy does.  The docs state:

    A string containing the version number of the Python interpreter plus
    additional information on the build number and compiler used.

Eg on CPython:

    Python 3.10.4 (main, Mar 23 2022, 23:05:40) [GCC 11.2.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import sys
    >>> sys.version
    '3.10.4 (main, Mar 23 2022, 23:05:40) [GCC 11.2.0]'

and PyPy:

    Python 2.7.12 (5.6.0+dfsg-4, Nov 20 2016, 10:43:30)
    [PyPy 5.6.0 with GCC 6.2.0 20161109] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>>> import sys
    >>>> sys.version
    '2.7.12 (5.6.0+dfsg-4, Nov 20 2016, 10:43:30)\n[PyPy 5.6.0 with GCC ...

With this commit on MicroPython we now have:

    MicroPython v1.18-371-g9d08eb024 on 2022-04-28; linux [GCC 11.2.0] v...
    Use Ctrl-D to exit, Ctrl-E for paste mode
    >>> import sys
    >>> sys.version
    '3.4.0; MicroPython v1.18-371-g9d08eb024 on 2022-04-28'

Note that the start of the banner is the same as the end of sys.version.
This helps to keep code size under control because the string can be reused
by the compiler.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-28 15:23:17 +10:00
Dan Halbert
3a8fb4e956
Merge pull request #5852 from jepler/floppy
Add floppyio
2022-04-26 11:40:19 -04:00
Damien George
a8f23f6366 shared/readline: Make tab insert an indent when it follows whitespace.
Entering tab at the REPL will now make it insert an indent (4 spaces) in
the following cases:
- after any whitespace on a line
- at the start of a line that is not the first line

This changes the existing behaviour where a tab would insert an indent only
if there were no matches in the auto-complete search, and it was the start
of the line.  This means, if there were any symbols in the global
namespace, tab could never be used to indent.

Note that entering tab at the start of the first line will still do
auto-completion, but will now do nothing if there are no symbols in the
global namespace, which is more consistent than before.

Signed-off-by: Damien George <damien@micropython.org>
2022-04-22 17:00:16 +10:00