Commit Graph

3650 Commits

Author SHA1 Message Date
Andrew Leech 86bfabec11 py/modmicropython: Add heap_locked function to test state of heap.
This commit adds micropython.heap_locked() which returns the current
lock-depth of the heap, and can be used by Python code to check if the heap
is locked or not.  This new function is configured via
MICROPY_PY_MICROPYTHON_HEAP_LOCKED and is disabled by default.

This commit also changes the return value of micropython.heap_unlock() so
it returns the current lock-depth as well.
2020-03-11 16:54:16 +11:00
Andrew Leech ed93778e00 py/objstringio: Expose tell() on StringIO and BytesIO objects.
To match file objects.

Fixes issue #5581.
2020-03-11 14:43:03 +11:00
David Lechner 8a4ce6b79a tools/codeformat.py: Eliminate need for sizeof fixup.
This eliminates the need for the sizeof regex fixup by rearranging things a
bit.  All other bitfields already use the parentheses around expressions
with sizeof, so one case is fixed by following this convention.

VM_MAX_STATE_ON_STACK is the only remaining problem and it can be worked
around by changing the order of the operands.
2020-03-11 14:34:40 +11:00
Tom Collins fccf17521a py/objstr: Remove duplicate % in error string.
The double-% was added in 11de8399fe (Jun
2014) when such errors were formatted with printf.  But then
55830dd9bf (Dec 2018) changed
mp_obj_new_exception_msg() to not format the message, as discussed
in #3004.  So such error strings are no longer formatted and a % is just
that.
2020-03-11 14:31:29 +11:00
Damien George 1993c8cf9a py/builtinevex: Support passing in a bytearray/buffer to eval/exec.
CPython allows this and it's a simple generalisation of the existing code
which just supported str/bytes.

Fixes issue #5704.
2020-02-28 12:45:36 +11:00
Damien George 69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
Damien George 3f39d18c2b all: Add *FORMAT-OFF* in various places.
This string is recognised by uncrustify, to disable formatting in the
region marked by these comments.  This is necessary in the qstrdef*.h files
to prevent modification of the strings within the Q(...).  In other places
it is used to prevent excessive reformatting that would make the code less
readable.
2020-02-28 10:31:07 +11:00
Damien George b86075ef1f py/parse: Add parenthesis around calculated bit-width in struct.
To improve interaction with uncrustify formatter.
2020-02-28 10:30:52 +11:00
Damien George 17aeb43e18 py: Un-nest configuration #if/#endif's for selection of complex code.
Because un-nested #if's are simpler to handle with formatting tools.
2020-02-28 10:30:34 +11:00
Damien George 7768a8c38d py/malloc: Put { on separate line for funcs that have selective sigs.
To ensure there are balanced {}'s in the file, and to help with formatting.
2020-02-28 10:30:29 +11:00
Damien George a642241a12 py/builtinimport: Adjust if-block order in find_file to clean up #if's. 2020-02-28 10:30:20 +11:00
Damien George 2b50afaee4 py/bc0.h: Shift comment to start of line to improve format consistency. 2020-02-28 10:29:32 +11:00
Damien George a1b18b3ba7 py: Removing dangling "else" to improve code format consistency. 2020-02-28 10:29:27 +11:00
Jim Mussared 3ccce89b83 py/objarray: Turn on MP_TYPE_FLAG_EQ_CHECKS_OTHER_TYPE for memoryview.
And add corresponding tests.

Fixes #5674 (comparison of memoryview against bytes).
2020-02-21 14:24:07 +11:00
Damien George f6d99bc795 py/dynruntime.h: Add implementation of mp_obj_cast_to_native_base. 2020-02-21 13:32:48 +11:00
Damien George 9344e876bb py/objtype: Allow mp_instance_cast_to_native_base to take native obj.
And rename it to mp_obj_cast_to_native_base() to indicate this.  This
allows users of this function to easily support native and native-subclass
objects in the same way (by just passing the object through this function).
2020-02-21 13:20:12 +11:00
Damien George d3b2c6e44c py/objtuple: Remove code that handles tuple-subclass equality test.
Since commit 3aab54bf43 this piece of code is
no longer needed because the top-level function mp_obj_equal_not_equal()
now handles the case of user types, and will never call tuple's binary_op
function with MP_BINARY_OP_EQUAL and a non-tuple on the RHS.
2020-02-20 10:48:03 +11:00
Damien George 1fccda049f py/objexcept: Rename mp_obj_new_exception_msg_varg2 to ..._vlist.
Follow up to recent commit ad7213d3c3, the
name "varg2" is misleading, vlist describes better that the argument is a
va_list.  This name also matches CircuitPython, which already has such
helper functions.
2020-02-18 21:00:42 +11:00
Damien George ce39c958ef py: Factor out definition of mp_float_union_t to one location. 2020-02-18 13:04:36 +11:00
David Lechner f020eac6a8 py/obj.h: Remove TODO idea comment about truncated mp_map_t.
It was suggested to move this to a GitHub issue rather than keep it in the
code, which isn't really sustainable for all ideas.
2020-02-13 13:14:17 +11:00
Damien George ad7213d3c3 py: Add mp_raise_msg_varg helper and use it where appropriate.
This commit adds mp_raise_msg_varg(type, fmt, ...) as a helper for
nlr_raise(mp_obj_new_exception_msg_varg(type, fmt, ...)).  It makes the
C-level API for raising exceptions more consistent, and reduces code size
on most ports:

   bare-arm:   +28 +0.042%
minimal x86:  +100 +0.067%
   unix x64:   -56 -0.011%
unix nanbox:  -300 -0.068%
      stm32:  -204 -0.054% PYBV10
     cc3200:    +0 +0.000%
    esp8266:   -64 -0.010% GENERIC
      esp32:  -104 -0.007% GENERIC
        nrf:  -136 -0.094% pca10040
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2020-02-13 11:52:40 +11:00
Damien George 97eca38c4f py: Add mp_raise_type helper macro and use it where appropriate.
This provides a more consistent C-level API to raise exceptions, ie moving
away from nlr_raise towards mp_raise_XXX.  It also reduces code size by a
small amount on some ports.
2020-02-13 11:03:37 +11:00
Damien George 7679e3be96 py/objmodule.h: Remove obsolete mp_builtin_module_weak_links_map decl.
It was made obsolete in d2384efa80
2020-02-11 15:43:13 +11:00
Damien George 9ec1caf42e py: Expand type equality flags to 3 separate ones, fix bool/namedtuple.
Both bool and namedtuple will check against other types for equality; int,
float and complex for bool, and tuple for namedtuple.  So to make them work
after the recent commit 3aab54bf43 they would
need MP_TYPE_FLAG_NEEDS_FULL_EQ_TEST set.  But that makes all bool and
namedtuple equality checks less efficient because mp_obj_equal_not_equal()
could no longer short-cut x==x, and would need to try __ne__.  To improve
this, this commit splits the MP_TYPE_FLAG_NEEDS_FULL_EQ_TEST flags into 3
separate flags to give types more fine-grained control over how their
equality behaves.  These new flags are then used to fix bool and namedtuple
equality.

Fixes issue #5615 and #5620.
2020-02-11 11:06:00 +11:00
Damien George abe2caf6df py/scheduler: Move clearing of kbd traceback to mp_keyboard_interrupt.
This is a more logical place to clear the KeyboardInterrupt traceback,
right before it is set as a pending exception.  The clearing is also
optimised from a function call to a simple store of NULL.
2020-02-07 16:08:31 +11:00
Damien George 8fb5c8fdd5 py/scheduler: Allow a port to specify attrs for mp_keyboard_interrupt.
Functions like mp_keyboard_interrupt() may need to be called from an IRQ
handler and may need to be in a special memory section, so provide a
generic wrapping macro for a port to do this.  The macro name is chosen to
be MICROPY_WRAP_<function name in uppercase> so that (in the future with
more wrappers) each function could potentially be handled separately.
2020-02-07 16:08:29 +11:00
Damien George 9efb36bfa6 py/scheduler: Move mp_keyboard_interrupt from lib/utils to py core.
This function is tightly coupled to the state and behaviour of the
scheduler, and is a core part of the runtime: to schedule a pending
exception.  So move it there.
2020-02-07 16:08:26 +11:00
Damien George 98a3911c43 py/scheduler: Add "raise_exc" argument to mp_handle_pending.
Previous behaviour is when this argument is set to "true", in which case
the function will raise any pending exception.  Setting it to "false" will
cancel any pending exception.
2020-02-07 16:08:20 +11:00
Petr Viktorin e6c9800645 py/compile: Allow 'return' outside function in minimal builds.
A 'return' statement on module/class level is not correct Python, but
nothing terribly bad happens when it's allowed.  So remove the check unless
MICROPY_CPYTHON_COMPAT is on.

This is similar to MicroPython's treatment of 'import *' in functions
(except 'return' has unsurprising behavior if it's allowed).
2020-02-06 00:41:55 +11:00
David Lechner 3e1bbeabaf py/modthread: Fix spelling error in comment. 2020-01-31 23:57:25 +11:00
Nicko van Someren 3aab54bf43 py: Support non-boolean results for equality and inequality tests.
This commit implements a more complete replication of CPython's behaviour
for equality and inequality testing of objects.  This addresses the issues
discussed in #5382 and a few other inconsistencies.  Improvements over the
old code include:

- Support for returning non-boolean results from comparisons (as used by
  numpy and others).
- Support for non-reflexive equality tests.
- Preferential use of __ne__ methods and MP_BINARY_OP_NOT_EQUAL binary
  operators for inequality tests, when available.
- Fallback to op2 == op1 or op2 != op1 when op1 does not implement the
  (in)equality operators.

The scheme here makes use of a new flag, MP_TYPE_FLAG_NEEDS_FULL_EQ_TEST,
in the flags word of mp_obj_type_t to indicate if various shortcuts can or
cannot be used when performing equality and inequality tests.  Currently
four built-in classes have the flag set: float and complex are
non-reflexive (since nan != nan) while bytearray and frozenszet instances
can equal other builtin class instances (bytes and set respectively).  The
flag is also set for any new class defined by the user.

This commit also includes a more comprehensive set of tests for the
behaviour of (in)equality operators implemented in special methods.
2020-01-30 14:53:07 +11:00
Damien George c3450effd4 py/objtype: Make mp_obj_type_t.flags constants public, moved to obj.h. 2020-01-30 14:53:07 +11:00
David Lechner b72cb0ca1b py/mpthread.h: Use strong type for mp_thread_set_state() argument.
This modifies the signature of mp_thread_set_state() to use
mp_state_thread_t* instead of void*.  This matches the return type of
mp_thread_get_state(), which returns the same value.

`struct _mp_state_thread_t;` had to be moved before
`#include <mpthreadport.h>` since the stm32 port uses it in its
mpthreadport.h file.
2020-01-29 17:10:32 +11:00
Jim Mussared c3095b37e9 py/nativeglue: Fix typo about where the native fun table enum is. 2020-01-27 13:22:19 +11:00
Jim Mussared 888ddb81dd py/emitnative: Stop after finding an unwind target.
The loop searches backwards for a target, but doesn't stop after finding
the first result, meaning that it'll always end up at the outermost
exception handler.
2020-01-27 13:22:03 +11:00
Jim Mussared 0de304e7da py/emitnative: Use NULL for pending exception (not None).
This previously made the native emitter incompatible with the bytecode
emitter, and mp_resume (and subsequently mp_obj_generator_resume) expects
the bytecode emitter behavior (i.e. throw==NULL).
2020-01-27 13:21:49 +11:00
David Lechner 62537a18e3 py: Release GIL during syscalls in reader and writer code.
This releases the GIL during POSIX system calls that could block.
2020-01-26 23:26:31 +11:00
Yonatan Goldschmidt d9433d3e94 py/obj.h: Add and use mp_obj_is_bool() helper.
Commit d96cfd13e3 introduced a regression in
testing for bool objects, that such objects were in some cases no longer
recognised and bools, eg when using mp_obj_is_type(o, &mp_type_bool), or
mp_obj_is_integer(o).

This commit fixes that problem by adding mp_obj_is_bool(o).  Builds with
MICROPY_OBJ_IMMEDIATE_OBJS enabled check if the object is any of the const
True or False objects.  Builds without it use the old method of ->type
checking, which compiles to smaller code (compared with the former
mentioned method).

Fixes #5538.
2020-01-24 10:53:45 +11:00
Damien George e2c1226da4 py/objexcept: Optimise mp_obj_new_exception[_arg1/_args] functions.
This reduces code size by 10-70 bytes on all ports (except cc3200 which has
no change).
2020-01-23 13:37:25 +11:00
Damien George edc7a8bf1d py/objgenerator: Use mp_obj_new_exception_arg1 to make StopIteration. 2020-01-23 13:37:25 +11:00
David Lechner edbb73a411 py/qstr: Don't include or init qstr_mutex when GIL is enabled.
When threads and the GIL are enabled, then the qstr mutex is not needed.
The qstr_mutex field is never used in this case because of:

    #if MICROPY_PY_THREAD && !MICROPY_PY_THREAD_GIL
    #define QSTR_ENTER() mp_thread_mutex_lock(&MP_STATE_VM(qstr_mutex), 1)
    #define QSTR_EXIT() mp_thread_mutex_unlock(&MP_STATE_VM(qstr_mutex))
    #else
    #define QSTR_ENTER()
    #define QSTR_EXIT()
    #endif

So, we can completely remove qstr_mutex everywhere when MICROPY_PY_THREAD
&& !MICROPY_PY_THREAD_GIL.
2020-01-23 13:29:11 +11:00
David Lechner ccc18f047d py/gc: Don't include or init gc_mutex when GIL is enabled.
When threads and the GIL are enabled, then the GC mutex is not needed.  The
gc_mutex field is never used in this case because of:

    #if MICROPY_PY_THREAD && !MICROPY_PY_THREAD_GIL
    #define GC_ENTER() mp_thread_mutex_lock(&MP_STATE_MEM(gc_mutex), 1)
    #define GC_EXIT() mp_thread_mutex_unlock(&MP_STATE_MEM(gc_mutex))
    #else
    #define GC_ENTER()
    #define GC_EXIT()
    #endif

So, we can completely remove gc_mutex everywhere when MICROPY_PY_THREAD
&& !MICROPY_PY_THREAD_GIL.
2020-01-23 13:28:42 +11:00
Damien George fe203bb3e2 py/pairheap: Add generic implementation of pairing heap data structure. 2020-01-22 17:31:18 +11:00
Yonatan Goldschmidt 176ab99180 py/objint: Add mp_obj_int_get_uint_checked() helper.
Can be used where mp_obj_int_get_checked() will overflow due to the
sign-bit solely.  This returns an mp_uint_t, so it also verifies the given
integer is not negative.

Currently implemented only for mpz configurations.
2020-01-14 23:35:22 +11:00
Yonatan Goldschmidt 1c849d63a8 py/mpconfig.h: Define BITS_PER_BYTE only if not already defined.
It's a common macro that is possibly defined in headers of systems/SDKs
MicroPython is embedded into.
2020-01-12 20:57:01 +02:00
Damien George 40057600b8 py/obj: Optimise mp_obj_get_type for immediate objs with repr A and C.
This function is called often and with immediate objects enabled it has
more cases, so optimise it for speed.  With this optimisation the runtime
is now slightly faster with immediate objects enabled than with them
disabled.
2020-01-13 01:01:45 +11:00
Damien George d96cfd13e3 py/obj: Add MICROPY_OBJ_IMMEDIATE_OBJS option to reduce code size.
This option (enabled by default for object representation A, B, C) makes
None/False/True objects immediate objects, ie they are no longer a concrete
object in ROM but are rather just values, eg None=0x6 for representation A.

Doing this saves a considerable amount of code size, due to these objects
being widely used:

   bare-arm:  -392 -0.591%
minimal x86:  -252 -0.170% [incl +52(data)]
   unix x64:  -624 -0.125% [incl -128(data)]
unix nanbox:    +0 +0.000%
      stm32: -1940 -0.510% PYBV10
     cc3200: -1216 -0.659%
    esp8266:  -404 -0.062% GENERIC
      esp32:  -732 -0.064% GENERIC[incl +48(data)]
        nrf:  -988 -0.675% pca10040
       samd:  -564 -0.556% ADAFRUIT_ITSYBITSY_M4_EXPRESS

Thanks go to @Jongy aka Yonatan Goldschmidt for the idea.
2020-01-13 01:01:45 +11:00
Damien George 6f0c83f6e1 py/obj.h: Redefine qstr object encoding to add immediate obj encoding.
This commit adjusts the definition of qstr encoding in all object
representations by taking a single bit from the qstr space and using it to
distinguish between qstrs and a new kind of literal object: immediate
objects.  In other words, the qstr space is divided in two pieces, one half
for qstrs and the other half for immediate objects.

There is still enough room for qstr values (29 bits in representation A on
a 32-bit architecture, and 19 bits in representation C) and the new
immediate objects can be used for things like None, False and True.
2020-01-13 01:01:45 +11:00
Damien George ecdb30ea64 py/nativeglue: Use mp_const_X instead of &mp_const_X_obj. 2020-01-12 13:51:09 +11:00
David Lechner 339d0816c5 py/runtime: Move MICROPY_PORT_INIT_FUNC near the end of mp_init().
This moves the MICROPY_PORT_INIT_FUNC hook to the end of mp_init(), just
before MP_THREAD_GIL_ENTER(), so that everything (in particular the GIL
mutex) is intialized before the hook is called.  MICROPY_PORT_DEINIT_FUNC
is also moved to be symmetric (but there is no functional change there).

If a port needs to perform initialisation earlier than
MICROPY_PORT_INIT_FUNC then it can do it before calling mp_init().
2020-01-12 13:27:04 +11:00