Commit Graph

132 Commits

Author SHA1 Message Date
stijn 57a7d5be9a py: Fix msvc C++ compiler warnings with MP_OBJ_FUN_MAKE_SIG macro.
When obj.h is compiled as C++ code, the cl compiler emits a warning about
possibly unsafe mixing of size_t and bool types in the or operation in
MP_OBJ_FUN_MAKE_SIG.  Similarly there's an implicit narrowing integer
conversion in runtime.h.  This commit fixes this by being explicit.
2018-09-26 15:34:59 +10:00
Damien George 9f241ef398 py: Optimise call to mp_arg_check_num by compressing fun signature.
With 5 arguments to mp_arg_check_num(), some architectures need to pass
values on the stack.  So compressing n_args_min, n_args_max, takes_kw into
a single word and passing only 3 arguments makes the call more efficient,
because almost all calls to this function pass in constant values.  Code
size is also reduced by a decent amount:

   bare-arm:  -116
minimal x86:   -64
   unix x64:  -256
unix nanbox:  -112
      stm32:  -324
     cc3200:  -192
    esp8266:  -192
      esp32:  -144
2018-09-14 13:39:17 +10:00
Damien George 4f3d9429b5 py: Fix native functions so they run with their correct globals context.
Prior to this commit a function compiled with the native decorator
@micropython.native would not work correctly when accessing global
variables, because the globals dict was not being set upon function entry.

This commit fixes this problem by, upon function entry, setting as the
current globals dict the globals dict context the function was defined
within, as per normal Python semantics, and as bytecode does.  Upon
function exit the original globals dict is restored.

In order to restore the globals dict when an exception is raised the native
function must guard its internals with an nlr_push/nlr_pop pair.  Because
this push/pop is relatively expensive, in both C stack usage for the
nlr_buf_t and CPU execution time, the implementation here optimises things
as much as possible.  First, the compiler keeps track of whether a function
even needs to access global variables.  Using this information the native
emitter then generates three different kinds of code:

1. no globals used, no exception handlers: no nlr handling code and no
   setting of the globals dict.

2. globals used, no exception handlers: an nlr_buf_t is allocated on the
   C stack but it is not used if the globals dict is unchanged, saving
   execution time because nlr_push/nlr_pop don't need to run.

3. function has exception handlers, may use globals: an nlr_buf_t is
   allocated and nlr_push/nlr_pop are always called.

In the end, native functions that don't access globals and don't have
exception handlers will run more efficiently than those that do.

Fixes issue #1573.
2018-09-13 22:47:20 +10:00
Damien George bc87b862fd py/runtime: Add mp_load_method_protected helper which catches exceptions
This new helper function acts like mp_load_method_maybe but is wrapped in
an NLR handler so it can catch exceptions.  It prevents AttributeError from
propagating out, and optionally all other exceptions.  This helper can be
used to fully implement hasattr (see follow-up commit), and also for cases
where mp_load_method_maybe is used but it must now raise an exception.
2018-05-10 23:00:04 +10:00
Damien George 02d830c035 py: Introduce a Python stack for scoped allocation.
This patch introduces the MICROPY_ENABLE_PYSTACK option (disabled by
default) which enables a "Python stack" that allows to allocate and free
memory in a scoped, or Last-In-First-Out (LIFO) way, similar to alloca().

A new memory allocation API is introduced along with this Py-stack.  It
includes both "local" and "nonlocal" LIFO allocation.  Local allocation is
intended to be equivalent to using alloca(), whereby the same function must
free the memory.  Nonlocal allocation is where another function may free
the memory, so long as it's still LIFO.

Follow-up patches will convert all uses of alloca() and VLA to the new
scoped allocation API.  The old behaviour (using alloca()) will still be
available, but when MICROPY_ENABLE_PYSTACK is enabled then alloca() is no
longer required or used.

The benefits of enabling this option are (or will be once subsequent
patches are made to convert alloca()/VLA):
- Toolchains without alloca() can use this feature to obtain correct and
  efficient scoped memory allocation (compared to using the heap instead
  of alloca(), which is slower).
- Even if alloca() is available, enabling the Py-stack gives slightly more
  efficient use of stack space when calling nested Python functions, due to
  the way that compilers implement alloca().
- Enabling the Py-stack with the stackless mode allows for even more
  efficient stack usage, as well as retaining high performance (because the
  heap is no longer used to build and destroy stackless code states).
- With Py-stack and stackless enabled, Python-calling-Python is no longer
  recursive in the C mp_execute_bytecode function.

The micropython.pystack_use() function is included to measure usage of the
Python stack.
2017-12-11 13:49:09 +11:00
Damien George 5b8998da6d py/runtime: Move mp_exc_recursion_depth to runtime and rename to raise.
For consistency this helper function is renamed to match the other
exception helpers, and moved to their location in runtime.c.
2017-12-11 13:49:09 +11:00
Paul Sokolovsky 62b96147e6 py: mp_call_function_*_protected(): Pass-thru return value if possible.
Return the result of called function. If exception happened, return
MP_OBJ_NULL. Allows to use mp_call_function_*_protected() with callbacks
returning values, etc.
2017-12-05 00:38:41 +02:00
Paul Sokolovsky 9956fd0710 py/objtype: Fit qstrs for special methods in byte type.
Update makeqstrdata.py to sort strings starting with "__" to the beginning
of qstr list, so they get low qstr id's, guaranteedly fitting in 8 bits.
Then use this property to further compact op_id => qstr mapping arrays.
2017-10-21 11:06:32 +03:00
Damien George a3dc1b1957 all: Remove inclusion of internal py header files.
Header files that are considered internal to the py core and should not
normally be included directly are:
    py/nlr.h - internal nlr configuration and declarations
    py/bc0.h - contains bytecode macro definitions
    py/runtime0.h - contains basic runtime enums

Instead, the top-level header files to include are one of:
    py/obj.h - includes runtime0.h and defines everything to use the
        mp_obj_t type
    py/runtime.h - includes mpstate.h and hence nlr.h, obj.h, runtime0.h,
        and defines everything to use the general runtime support functions

Additional, specific headers (eg py/objlist.h) can be included if needed.
2017-10-04 12:37:50 +11:00
Damien George 6c82cfc089 py/objtype: Change type of enum-to-qstr table to uint16_t to save space.
Qstr values fit in 16-bits (and this fact is used elsewhere in the code) so
no need to use more than that for the large lookup tables.  The compiler
will anyway give a warning if the qstr values don't fit in 16 bits.  Saves
around 80 bytes of code space for Thumb2 archs.
2017-10-04 11:31:05 +11:00
Damien George 89f657f073 py/runtime.h: Change empty mp_warning macro so var-args are non empty.
Variable arguments in a macro should take at least 1 argument.
2017-09-13 20:33:55 +10:00
Damien George 58321dd985 all: Convert mp_uint_t to mp_unary_op_t/mp_binary_op_t where appropriate
The unary-op/binary-op enums are already defined, and there are no
arithmetic tricks used with these types, so it makes sense to use the
correct enum type for arguments that take these values.  It also reduces
code size quite a bit for nan-boxing builds.
2017-08-29 13:16:30 +10:00
Javier Candeira 35a1fea90b all: Raise exceptions via mp_raise_XXX
- Changed: ValueError, TypeError, NotImplementedError
  - OSError invocations unchanged, because the corresponding utility
    function takes ints, not strings like the long form invocation.
  - OverflowError, IndexError and RuntimeError etc. not changed for now
    until we decide whether to add new utility functions.
2017-08-13 22:52:33 +10:00
Alexander Steffen 55f33240f3 all: Use the name MicroPython consistently in comments
There were several different spellings of MicroPython present in comments,
when there should be only one.
2017-07-31 18:35:40 +10:00
Alexander Steffen 299bc62586 all: Unify header guard usage.
The code conventions suggest using header guards, but do not define how
those should look like and instead point to existing files. However, not
all existing files follow the same scheme, sometimes omitting header guards
altogether, sometimes using non-standard names, making it easy to
accidentally pick a "wrong" example.

This commit ensures that all header files of the MicroPython project (that
were not simply copied from somewhere else) follow the same pattern, that
was already present in the majority of files, especially in the py folder.

The rules are as follows.

Naming convention:
* start with the words MICROPY_INCLUDED
* contain the full path to the file
* replace special characters with _

In addition, there are no empty lines before #ifndef, between #ifndef and
one empty line before #endif. #endif is followed by a comment containing
the name of the guard macro.

py/grammar.h cannot use header guards by design, since it has to be
included multiple times in a single C file. Several other files also do not
need header guards as they are only used internally and guaranteed to be
included only once:
* MICROPY_MPHALPORT_H
* mpconfigboard.h
* mpconfigport.h
* mpthreadport.h
* pin_defs_*.h
* qstrdefs*.h
2017-07-18 11:57:39 +10:00
Damien George dd11af209d py: Add LOAD_SUPER_METHOD bytecode to allow heap-free super meth calls.
This patch allows the following code to run without allocating on the heap:

    super().foo(...)

Before this patch such a call would allocate a super object on the heap and
then load the foo method and call it right away.  The super object is only
needed to perform the lookup of the method and not needed after that.  This
patch makes an optimisation to allocate the super object on the C stack and
discard it right after use.

Changes in code size due to this patch are:

   bare-arm: +128
    minimal: +232
   unix x64: +416
unix nanbox: +364
     stmhal: +184
    esp8266: +340
     cc3200: +128
2017-04-22 23:39:20 +10:00
Damien George 64a4f11b2d py: Remove MP_STATE_CTX, use MP_STATE_THREAD instead (it's an alias).
MP_STATE_CTX was recently aliased to MP_STATE_THREAD and can now be
removed.
2017-03-24 18:43:28 +11:00
Damien George 6e74d24f30 py: Add micropython.schedule() function and associated runtime code. 2017-03-20 15:20:26 +11:00
Damien George ae8d867586 py: Add iter_buf to getiter type method.
Allows to iterate over the following without allocating on the heap:
- tuple
- list
- string, bytes
- bytearray, array
- dict (not dict.keys, dict.values, dict.items)
- set, frozenset

Allows to call the following without heap memory:
- all, any, min, max, sum

TODO: still need to allocate stack memory in bytecode for iter_buf.
2017-02-16 18:38:06 +11:00
Damien George 4e3bac2e42 py/runtime: Convert mp_uint_t to size_t where appropriate. 2017-02-16 16:51:13 +11:00
Paul Sokolovsky 037e6912c6 py/objtype: Implement __call__ handling for an instance w/o heap alloc.
By refactoring and reusing code from objboundmeth.
2016-11-22 01:33:55 +03:00
Damien George 3a0a771730 py: Add mp_raise_OSError(errno) helper function.
This is an often used code pattern, and its use reduces code size of the
core by about 100 bytes.
2016-10-07 13:31:59 +11:00
Damien George e97df97600 py: Shrink mp_arg_t struct by using reduced-size integer members.
qstrs ids are restricted to fit within 2 bytes already (eg in persistent
bytecode) so it's safe to use a uint16_t to store them in mp_arg_t.  And
the flags member only needs a maximum of 2 bytes so can also use uint16_t.

Savings in code size can be significant when many mp_arg_t structs are
used for argument parsing.  Eg, this patch reduces stmhal by 480 bytes.
2016-09-23 12:13:51 +10:00
Damien George 1a0d3fd632 py/runtime.h: Move comment about mp_not_implemented to correct place. 2016-08-14 16:35:10 +10:00
Paul Sokolovsky 8c50f93a41 py/runtime.h: Define mp_check_self(pred) helper macro.
Indended to replace raw asserts in bunch of files. Expands to empty
if MICROPY_BUILTIN_METHOD_CHECK_SELF_ARG is defined, otehrwise by
default still to assert, though a particular port may define it to
something else.
2016-08-12 21:58:56 +03:00
Paul Sokolovsky 9e1b61dedd py/runtime: Factor out exception raising helpers.
Introduce mp_raise_msg(), mp_raise_ValueError(), mp_raise_TypeError()
instead of previous pattern nlr_raise(mp_obj_new_exception_msg(...)).
Save few bytes on each call, which are many.
2016-08-12 21:28:45 +03:00
Paul Sokolovsky 6d103b6548 py: Move call_function_*_protected() functions to py/ for reuse.
They almost certainly needed by any C code which calls Python callbacks.
2016-04-25 19:31:17 +03:00
Damien George e7cd1699df py/argcheck: Use size_t instead of mp_uint_t to count number of args. 2016-03-14 22:35:48 +00:00
Damien George 999cedb90f py: Wrap all obj-ptr conversions in MP_OBJ_TO_PTR/MP_OBJ_FROM_PTR.
This allows the mp_obj_t type to be configured to something other than a
pointer-sized primitive type.

This patch also includes additional changes to allow the code to compile
when sizeof(mp_uint_t) != sizeof(void*), such as using size_t instead of
mp_uint_t, and various casts.
2015-11-29 14:25:35 +00:00
Damien George cbf7674025 py: Add MP_ROM_* macros and mp_rom_* types and use them. 2015-11-29 14:25:04 +00:00
Damien George 59fba2d6ea py: Remove mp_load_const_bytes and instead load precreated bytes object.
Previous to this patch each time a bytes object was referenced a new
instance (with the same data) was created.  With this patch a single
bytes object is created in the compiler and is loaded directly at execute
time as a true constant (similar to loading bignum and float objects).
This saves on allocating RAM and means that bytes objects can now be
used when the memory manager is locked (eg in interrupts).

The MP_BC_LOAD_CONST_BYTES bytecode was removed as part of this.

Generated bytecode is slightly larger due to storing a pointer to the
bytes object instead of the qstr identifier.

Code size is reduced by about 60 bytes on Thumb2 architectures.
2015-06-25 14:42:13 +00:00
Damien George ed570e4b2a py: Remove mp_load_const_str and replace uses with inlined version. 2015-06-25 13:58:41 +00:00
Damien George 7f9d1d6ab9 py: Overhaul and simplify printf/pfenv mechanism.
Previous to this patch the printing mechanism was a bit of a tangled
mess.  This patch attempts to consolidate printing into one interface.

All (non-debug) printing now uses the mp_print* family of functions,
mainly mp_printf.  All these functions take an mp_print_t structure as
their first argument, and this structure defines the printing backend
through the "print_strn" function of said structure.

Printing from the uPy core can reach the platform-defined print code via
two paths: either through mp_sys_stdout_obj (defined pert port) in
conjunction with mp_stream_write; or through the mp_plat_print structure
which uses the MP_PLAT_PRINT_STRN macro to define how string are printed
on the platform.  The former is only used when MICROPY_PY_IO is defined.

With this new scheme printing is generally more efficient (less layers
to go through, less arguments to pass), and, given an mp_print_t*
structure, one can call mp_print_str for efficiency instead of
mp_printf("%s", ...).  Code size is also reduced by around 200 bytes on
Thumb2 archs.
2015-04-16 14:30:16 +00:00
Damien George 12a5e17afb py: Add finer configuration of static funcs when not in stackless mode.
Also rename call_args_t to mp_call_args_t.
2015-04-02 22:56:58 +01:00
Paul Sokolovsky 7f1c98177b vm: Support strict stackless mode, with proper exception reporting.
I.e. in this mode, C stack will never be used to call a Python function,
but if there's no free heap for a call, it will be reported as
RuntimeError (as expected), not MemoryError.
2015-04-03 00:26:47 +03:00
Paul Sokolovsky e6c6fe3275 runtime: Split mp_call_prepare_args_n_kw_var() from mp_call_method_n_kw_var().
Allow for reuse for stackless design, where preparing args is separate from
calling.
2015-04-03 00:03:07 +03:00
Damien George 55b74d1ff5 py: Combine duplicated code that converts members from a lookup.
Despite initial guess, this code factoring does not hamper performance.
In fact it seems to improve speed by a little: running pystone(1.2) on
pyboard (which gives a very stable result) this patch takes pystones
from 1729.51 up to 1742.16.  Also, pystones on x64 increase by around
the same proportion (but it's much noisier).

Taking a look at the generated machine code, stack usage with this patch
is unchanged, and call is tail-optimised with all arguments in
registers.  Code size decreases by about 50 bytes on Thumb2 archs.
2015-03-21 14:21:54 +00:00
Damien George e1e359ff59 py: Put mp_sys_path, mp_sys_argv and gc_collected in mp_state_ctx_t.
Without mp_sys_path and mp_sys_argv in the root pointer section of the
state, their memory was being incorrectly collected by GC.
2015-02-07 17:24:10 +00:00
Damien George a5efcd4745 py: Specify unary/binary op name in TypeError error message.
Eg, "() + 1" now tells you that __add__ is not supported for tuple and
int types (before it just said the generic "binary operator").  We reuse
the table of names for slot lookup because it would be a waste of code
space to store the pretty name for each operator.
2015-01-27 18:02:25 +00:00
Damien George b6e6b5277f py: Implement proper re-raising in native codegen's finally handler.
This allows an exception to propagate correctly through a finally
handler.
2015-01-21 17:00:01 +00:00
Damien George 50912e7f5d py, unix, stmhal: Allow to compile with -Wshadow.
See issue #699.
2015-01-20 11:55:10 +00:00
Damien George b4b10fd350 py: Put all global state together in state structures.
This patch consolidates all global variables in py/ core into one place,
in a global structure.  Root pointers are all located together to make
GC tracing easier and more efficient.
2015-01-07 20:33:00 +00:00
Paul Sokolovsky 8a8c1fc82f py: Add basic framework for issuing compile/runtime warnings. 2015-01-01 22:09:18 +02:00
Damien George 7f23384d49 py: Make terse_arg_mismatch a global function and use it elsewhere.
Reduces code size when MICROPY_ERROR_REPORTING_TERSE is selected.
2015-01-01 15:33:50 +00:00
Damien George 9ddbe291c4 py: Add include guards to mpconfig,misc,qstr,obj,runtime,parsehelper. 2014-12-29 01:02:19 +00:00
Damien George b6b34cd3f6 py: Remove obsolute function declaration. 2014-11-06 17:31:18 +00:00
Damien George 124df6f8d0 py: Add mp_pending_exception global variable, for VM soft interrupt.
This allows to implement KeyboardInterrupt on unix, and a much safer
ctrl-C in stmhal port.  First ctrl-C is a soft one, with hope that VM
will notice it; second ctrl-C is a hard one that kills anything (for
both unix and stmhal).

One needs to check for a pending exception in the VM only for jump
opcodes.  Others can't produce an infinite loop (infinite recursion is
caught by stack check).
2014-10-25 23:37:57 +01:00
Damien George 4abff7500f py: Change uint to mp_uint_t in runtime.h, stackctrl.h, binary.h.
Part of code cleanup, working towards resolving issue #50.
2014-08-30 14:59:21 +01:00
Damien George 3c658a4e75 py: Fix bug where GC collected native/viper/asm function data.
Because (for Thumb) a function pointer has the LSB set, pointers to
dynamic functions in RAM (eg native, viper or asm functions) were not
being traced by the GC.  This patch is a comprehensive fix for this.

Addresses issue #820.
2014-08-24 16:28:17 +01:00
Damien George 86de21b810 py: Viper can call functions with native types, and raise exceptions. 2014-08-16 22:06:11 +01:00