Commit Graph

3618 Commits

Author SHA1 Message Date
David Lechner
b72cb0ca1b py/mpthread.h: Use strong type for mp_thread_set_state() argument.
This modifies the signature of mp_thread_set_state() to use
mp_state_thread_t* instead of void*.  This matches the return type of
mp_thread_get_state(), which returns the same value.

`struct _mp_state_thread_t;` had to be moved before
`#include <mpthreadport.h>` since the stm32 port uses it in its
mpthreadport.h file.
2020-01-29 17:10:32 +11:00
Jim Mussared
c3095b37e9 py/nativeglue: Fix typo about where the native fun table enum is. 2020-01-27 13:22:19 +11:00
Jim Mussared
888ddb81dd py/emitnative: Stop after finding an unwind target.
The loop searches backwards for a target, but doesn't stop after finding
the first result, meaning that it'll always end up at the outermost
exception handler.
2020-01-27 13:22:03 +11:00
Jim Mussared
0de304e7da py/emitnative: Use NULL for pending exception (not None).
This previously made the native emitter incompatible with the bytecode
emitter, and mp_resume (and subsequently mp_obj_generator_resume) expects
the bytecode emitter behavior (i.e. throw==NULL).
2020-01-27 13:21:49 +11:00
David Lechner
62537a18e3 py: Release GIL during syscalls in reader and writer code.
This releases the GIL during POSIX system calls that could block.
2020-01-26 23:26:31 +11:00
Yonatan Goldschmidt
d9433d3e94 py/obj.h: Add and use mp_obj_is_bool() helper.
Commit d96cfd13e3 introduced a regression in
testing for bool objects, that such objects were in some cases no longer
recognised and bools, eg when using mp_obj_is_type(o, &mp_type_bool), or
mp_obj_is_integer(o).

This commit fixes that problem by adding mp_obj_is_bool(o).  Builds with
MICROPY_OBJ_IMMEDIATE_OBJS enabled check if the object is any of the const
True or False objects.  Builds without it use the old method of ->type
checking, which compiles to smaller code (compared with the former
mentioned method).

Fixes #5538.
2020-01-24 10:53:45 +11:00
Damien George
e2c1226da4 py/objexcept: Optimise mp_obj_new_exception[_arg1/_args] functions.
This reduces code size by 10-70 bytes on all ports (except cc3200 which has
no change).
2020-01-23 13:37:25 +11:00
Damien George
edc7a8bf1d py/objgenerator: Use mp_obj_new_exception_arg1 to make StopIteration. 2020-01-23 13:37:25 +11:00
David Lechner
edbb73a411 py/qstr: Don't include or init qstr_mutex when GIL is enabled.
When threads and the GIL are enabled, then the qstr mutex is not needed.
The qstr_mutex field is never used in this case because of:

    #if MICROPY_PY_THREAD && !MICROPY_PY_THREAD_GIL
    #define QSTR_ENTER() mp_thread_mutex_lock(&MP_STATE_VM(qstr_mutex), 1)
    #define QSTR_EXIT() mp_thread_mutex_unlock(&MP_STATE_VM(qstr_mutex))
    #else
    #define QSTR_ENTER()
    #define QSTR_EXIT()
    #endif

So, we can completely remove qstr_mutex everywhere when MICROPY_PY_THREAD
&& !MICROPY_PY_THREAD_GIL.
2020-01-23 13:29:11 +11:00
David Lechner
ccc18f047d py/gc: Don't include or init gc_mutex when GIL is enabled.
When threads and the GIL are enabled, then the GC mutex is not needed.  The
gc_mutex field is never used in this case because of:

    #if MICROPY_PY_THREAD && !MICROPY_PY_THREAD_GIL
    #define GC_ENTER() mp_thread_mutex_lock(&MP_STATE_MEM(gc_mutex), 1)
    #define GC_EXIT() mp_thread_mutex_unlock(&MP_STATE_MEM(gc_mutex))
    #else
    #define GC_ENTER()
    #define GC_EXIT()
    #endif

So, we can completely remove gc_mutex everywhere when MICROPY_PY_THREAD
&& !MICROPY_PY_THREAD_GIL.
2020-01-23 13:28:42 +11:00
Damien George
fe203bb3e2 py/pairheap: Add generic implementation of pairing heap data structure. 2020-01-22 17:31:18 +11:00
Yonatan Goldschmidt
176ab99180 py/objint: Add mp_obj_int_get_uint_checked() helper.
Can be used where mp_obj_int_get_checked() will overflow due to the
sign-bit solely.  This returns an mp_uint_t, so it also verifies the given
integer is not negative.

Currently implemented only for mpz configurations.
2020-01-14 23:35:22 +11:00
Yonatan Goldschmidt
1c849d63a8 py/mpconfig.h: Define BITS_PER_BYTE only if not already defined.
It's a common macro that is possibly defined in headers of systems/SDKs
MicroPython is embedded into.
2020-01-12 20:57:01 +02:00
Damien George
40057600b8 py/obj: Optimise mp_obj_get_type for immediate objs with repr A and C.
This function is called often and with immediate objects enabled it has
more cases, so optimise it for speed.  With this optimisation the runtime
is now slightly faster with immediate objects enabled than with them
disabled.
2020-01-13 01:01:45 +11:00
Damien George
d96cfd13e3 py/obj: Add MICROPY_OBJ_IMMEDIATE_OBJS option to reduce code size.
This option (enabled by default for object representation A, B, C) makes
None/False/True objects immediate objects, ie they are no longer a concrete
object in ROM but are rather just values, eg None=0x6 for representation A.

Doing this saves a considerable amount of code size, due to these objects
being widely used:

   bare-arm:  -392 -0.591%
minimal x86:  -252 -0.170% [incl +52(data)]
   unix x64:  -624 -0.125% [incl -128(data)]
unix nanbox:    +0 +0.000%
      stm32: -1940 -0.510% PYBV10
     cc3200: -1216 -0.659%
    esp8266:  -404 -0.062% GENERIC
      esp32:  -732 -0.064% GENERIC[incl +48(data)]
        nrf:  -988 -0.675% pca10040
       samd:  -564 -0.556% ADAFRUIT_ITSYBITSY_M4_EXPRESS

Thanks go to @Jongy aka Yonatan Goldschmidt for the idea.
2020-01-13 01:01:45 +11:00
Damien George
6f0c83f6e1 py/obj.h: Redefine qstr object encoding to add immediate obj encoding.
This commit adjusts the definition of qstr encoding in all object
representations by taking a single bit from the qstr space and using it to
distinguish between qstrs and a new kind of literal object: immediate
objects.  In other words, the qstr space is divided in two pieces, one half
for qstrs and the other half for immediate objects.

There is still enough room for qstr values (29 bits in representation A on
a 32-bit architecture, and 19 bits in representation C) and the new
immediate objects can be used for things like None, False and True.
2020-01-13 01:01:45 +11:00
Damien George
ecdb30ea64 py/nativeglue: Use mp_const_X instead of &mp_const_X_obj. 2020-01-12 13:51:09 +11:00
David Lechner
339d0816c5 py/runtime: Move MICROPY_PORT_INIT_FUNC near the end of mp_init().
This moves the MICROPY_PORT_INIT_FUNC hook to the end of mp_init(), just
before MP_THREAD_GIL_ENTER(), so that everything (in particular the GIL
mutex) is intialized before the hook is called.  MICROPY_PORT_DEINIT_FUNC
is also moved to be symmetric (but there is no functional change there).

If a port needs to perform initialisation earlier than
MICROPY_PORT_INIT_FUNC then it can do it before calling mp_init().
2020-01-12 13:27:04 +11:00
Yonatan Goldschmidt
853aaa06f2 lib/mp-readline: Add word-based move/delete EMACS key sequences.
This commit adds backward-word, backward-kill-word, forward-word,
forward-kill-word sequences for the REPL, with bindings to Alt+F, Alt+B,
Alt+D and Alt+Backspace respectively.  It is disabled by default and can be
enabled via MICROPY_REPL_EMACS_WORDS_MOVE.

Further enabling MICROPY_REPL_EMACS_EXTRA_WORDS_MOVE adds extra bindings
for these new sequences: Ctrl+Right, Ctrl+Left and Ctrl+W.

The features are enabled on unix micropython-coverage and micropython-dev.
2020-01-12 13:09:27 +11:00
Yonatan Goldschmidt
df5c3bd976 py/unicode: Add unichar_isalnum(). 2020-01-12 13:03:57 +11:00
Jim Mussared
7d2ccd027f py/mkenv.mk: Move usage of 32-bit flags to py.mk.
This allows ports/variants to configure MICROPY_FORCE_32BIT after including
mkenv.mk, but before py.mk.
2020-01-12 10:34:10 +11:00
Damien George
bfbd94401d py: Make mp_obj_get_type() return a const ptr to mp_obj_type_t.
Most types are in rodata/ROM, and mp_obj_base_t.type is a constant pointer,
so enforce this const-ness throughout the code base.  If a type ever needs
to be modified (eg a user type) then a simple cast can be used.
2020-01-09 11:25:26 +11:00
Nicko van Someren
10709846f3 py/objslice: Inline fetching of slice paramters in str_subscr().
To reduce code size.
2019-12-29 00:06:02 +11:00
Nicko van Someren
4c93955b7b py/objslice: Add support for indices() method on slice objects.
Instances of the slice class are passed to __getitem__() on objects when
the user indexes them with a slice.  In practice the majority of the time
(other than passing it on untouched) is to work out what the slice means in
the context of an array dimension of a particular length.  Since Python 2.3
there has been a method on the slice class, indices(), that takes a
dimension length and returns the real start, stop and step, accounting for
missing or negative values in the slice spec.  This commit implements such
a indices() method on the slice class.

It is configurable at compile-time via MICROPY_PY_BUILTINS_SLICE_INDICES,
disabled by default, enabled on unix, stm32 and esp32 ports.

This commit also adds new tests for slice indices and for slicing unicode
strings.
2019-12-28 23:55:15 +11:00
Damien George
b23bd6433c py: Clean up commented-out code and comments about exception hierarchy.
In CPython, EnvironmentError and IOError are now aliases of OSError so no
need to have them listed in the code.  OverflowError inherits from
ArithmeticError because it's intended to be raised "when the result of an
arithmetic operation is too large to be represented" (per CPython docs),
and MicroPython aims to match the CPython exception hierarchy.
2019-12-28 01:01:36 +11:00
Yonatan Goldschmidt
61d2b40ad5 lib/utils/pyexec: Introduce MICROPY_REPL_INFO, wrap debug prints in it.
For the 3 ports that already make use of this feature (stm32, nrf and
teensy) this doesn't make any difference, it just allows to disable it from
now on.

For other ports that use pyexec, this decreases code size because the debug
printing code is dead (it can't be enabled) but the compiler can't deduce
that, so code is still emitted.
2019-12-28 00:05:39 +11:00
Damien George
d56bc6e03d py/obj.h: Use 32-bit shift in MP_OBJ_NEW_QSTR calc for obj-repr D.
The qst value is always small enough to fit in 31-bits (even less) and
using a 32-bit shift rather than a 64-bit shift reduces code size by about
300 bytes.
2019-12-27 23:33:34 +11:00
Damien George
4c0176d13f py/objstr: Don't use inline GET_STR_DATA_LEN for object-repr D.
Changing to use the helper function mp_obj_str_get_data_no_check() reduces
code size of nan-boxing builds by about 1000 bytes.
2019-12-27 23:15:52 +11:00
Damien George
1f499ad2fe py/objobject: Fix __setattr__/__delattr__ to build in nanbox mode. 2019-12-27 22:54:53 +11:00
Damien George
d97b40bdaa py: Introduce MP_ROM_FALSE/MP_ROM_TRUE for ROM to refer to bool objects.
This helps to prevent mistakes, and allows easily changing the ROM value of
False/True if needed.
2019-12-27 22:54:20 +11:00
Damien George
09376f0e47 py: Introduce MP_ROM_NONE macro for ROM to refer to None object.
This helps to prevent mistakes, and allows easily changing the ROM value of
None if needed.
2019-12-27 22:51:17 +11:00
Damien George
f5eec903fa py/objsingleton: Use mp_generic_unary_op for singleton objects.
So these types more closely match NoneType, eg they can be hashed, like in
CPython.
2019-12-27 12:53:36 +11:00
Damien George
aacd618939 py/runtime: Don't allocate iter buf for user-defined types.
A user-defined type that defines __iter__ doesn't need any memory to be
pre-allocated for its iterator (because it can't use such memory).  So
optimise for this case by not allocating the iter-buf.
2019-12-27 12:34:22 +11:00
Damien George
ab75210e33 py/asmx86: Remove unused 5th argument facility.
In commit 71a3d6ec3b mp_setup_code_state was
changed from a 5-arg function to a 4-arg function, and at that point 5-arg
calls in native code were no longer needed.  See also commit
4f9842ad80.
2019-12-27 12:30:51 +11:00
Damien George
b97fb683d0 py/asmx86: Fix stack to be 16-byte aligned for entry and sub-call. 2019-12-27 12:30:51 +11:00
Damien George
300eb65ae7 py/nlrx86: Silence possible warnings about unused nlr argument. 2019-12-23 00:07:03 +11:00
Yonatan Goldschmidt
42e45bd694 py/objobject: Add object.__delattr__ function.
Similar to object.__setattr__.
2019-12-21 00:14:22 +11:00
Yonatan Goldschmidt
07ccb5588c py/objobject: Add object.__setattr__ function.
Allows assigning attributes on class instances that implement their own
__setattr__.  Both object.__setattr__ and super(A, b).__setattr__ will work
with this commit.
2019-12-21 00:12:08 +11:00
Damien George
5e431188db py/obj.h: Remove comments about additional mp_buffer_info_t entries.
These entries are unlikely to be needed, so remove them to clean up the
struct definition.
2019-12-20 23:36:17 +11:00
Damien George
035180ca01 py: Remove commented-out debug printf's from emitbc and objlist.
Any debugging prints should use a macro like DEBUG_printf.
2019-12-20 23:34:46 +11:00
Damien George
1f37194730 all: Bump version to 1.12. 2019-12-20 16:58:17 +11:00
Damien George
073c5f3a40 py/profile: Fix debug opcode decoding of MP_BC_RAISE_xxx opcodes. 2019-12-20 14:57:44 +11:00
Damien George
95473980ef py/vm: Fix comment to refer to MP_BC_RAISE_OBJ instead of RAISE_VARARGS. 2019-12-20 14:57:06 +11:00
Damien George
0bd7d1f7f0 py/persistentcode: Move loading of rodata/bss to before obj/raw-code.
This makes the loading of viper-code-with-relocations a bit neater and
easier to understand, by treating the rodata/bss like a special object to
be loaded into the constant table (which is how it behaves).
2019-12-17 13:22:11 +11:00
Damien George
48e9262f55 py/dynruntime: Implement uint new/get, mp_obj_len and mp_obj_subscr. 2019-12-13 13:29:11 +11:00
Damien George
abc642973d py/dynruntime: Add support for float API to make/get floats.
We don't want to add a feature flag to .mpy files that indicate float
support because it will get complex and difficult to use.  Instead the .mpy
is built using whatever precision it chooses (float or double) and the
native glue API will convert between this choice and what the host runtime
actually uses.
2019-12-12 20:15:28 +11:00
Damien George
ff58961944 py/nativeglue: Add float new/get functions with both single and double. 2019-12-12 20:15:28 +11:00
Damien George
9ac949cdbd py/persistentcode: Make ARM Thumb archs support multiple sub-archs. 2019-12-12 20:15:28 +11:00
Damien George
aad79adab7 tools/mpy_ld.py: Add new mpy_ld.py tool and associated build files.
This commit adds a new tool called mpy_ld.py which is essentially a linker
that builds .mpy files directly from .o files.  A new header file
(dynruntime.h) and makefile fragment (dynruntime.mk) are also included
which allow building .mpy files from C source code.  Such .mpy files can
then be dynamically imported as though they were a normal Python module,
even though they are implemented in C.

Converting .o files directly (rather than pre-linked .elf files) allows the
resulting .mpy to be more efficient because it has more control over the
relocations; for example it can skip PLT indirection.  Doing it this way
also allows supporting more architectures, such as Xtensa which has
specific needs for position-independent code and the GOT.

The tool supports targets of x86, x86-64, ARM Thumb and Xtensa (windowed
and non-windowed).  BSS, text and rodata sections are supported, with
relocations to all internal sections and symbols, as well as relocations to
some external symbols (defined by dynruntime.h), and linking of qstrs.
2019-12-12 20:15:28 +11:00
Damien George
3690f79afc py/nativeglue: Add funcs/types to native glue table for dynamic runtime.
These allow discovery of symbols by native code that is loaded dynamically.
2019-12-12 20:15:28 +11:00