Commit Graph

175 Commits

Author SHA1 Message Date
David Lechner 8a4ce6b79a tools/codeformat.py: Eliminate need for sizeof fixup.
This eliminates the need for the sizeof regex fixup by rearranging things a
bit.  All other bitfields already use the parentheses around expressions
with sizeof, so one case is fixed by following this convention.

VM_MAX_STATE_ON_STACK is the only remaining problem and it can be worked
around by changing the order of the operands.
2020-03-11 14:34:40 +11:00
Damien George 69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
Damien George a1b18b3ba7 py: Removing dangling "else" to improve code format consistency. 2020-02-28 10:29:27 +11:00
Damien George bfbd94401d py: Make mp_obj_get_type() return a const ptr to mp_obj_type_t.
Most types are in rodata/ROM, and mp_obj_base_t.type is a constant pointer,
so enforce this const-ness throughout the code base.  If a type ever needs
to be modified (eg a user type) then a simple cast can be used.
2020-01-09 11:25:26 +11:00
Damien George c8c0fd4ca3 py: Rework and compress second part of bytecode prelude.
This patch compresses the second part of the bytecode prelude which
contains the source file name, function name, source-line-number mapping
and cell closure information.  This part of the prelude now begins with a
single varible length unsigned integer which encodes 2 numbers, being the
byte-size of the following 2 sections in the header: the "source info
section" and the "closure section".  After decoding this variable unsigned
integer it's possible to skip over one or both of these sections very
easily.

This scheme saves about 2 bytes for most functions compared to the original
format: one in the case that there are no closure cells, and one because
padding was eliminated.
2019-10-01 12:26:22 +10:00
Damien George b5ebfadbd6 py: Compress first part of bytecode prelude.
The start of the bytecode prelude contains 6 numbers telling the amount of
stack needed for the Python values and exceptions, and the signature of the
function.  Prior to this patch these numbers were all encoded one after the
other (2x variable unsigned integers, then 4x bytes), but using so many
bytes is unnecessary.

An entropy analysis of around 150,000 bytecode functions from the CPython
standard library showed that the optimal Shannon coding would need about
7.1 bits on average to encode these 6 numbers, compared to the existing 48
bits.

This patch attempts to get close to this optimal value by packing the 6
numbers into a single, varible-length unsigned integer via bit-wise
interleaving.  The interleaving scheme is chosen to minimise the average
number of bytes needed, and at the same time keep the scheme simple enough
so it can be implemented without too much overhead in code size or speed.
The scheme requires about 10.5 bits on average to store the 6 numbers.

As a result most functions which originally took 6 bytes to encode these 6
numbers now need only 1 byte (in 80% of cases).
2019-10-01 12:26:22 +10:00
Damien George 81d04a0200 py: Add n_state to mp_code_state_t struct.
This value is used often enough that it is better to cache it instead of
decode it each time.
2019-10-01 12:26:22 +10:00
Damien George cd35dd9d9a py: Allow to pass in read-only buffers to viper and inline-asm funcs.
Fixes #4936.
2019-08-06 15:58:23 +10:00
Jun Wu 089c9b71d1 py: remove "if (0)" and "if (false)" branches.
Prior to this commit, building the unix port with `DEBUG=1` and
`-finstrument-functions` the compilation would fail with an error like
"control reaches end of non-void function".  This change fixes this by
removing the problematic "if (0)" branches.  Not all branches affect
compilation, but they are all removed for consistency.
2019-05-06 18:28:28 +10:00
Damien George 0e4c24ec08 py/nativeglue: Rename native convert funs to match other native helpers. 2019-03-14 12:22:25 +11:00
Damien George 7bc71f5446 py/objfun: Make fun_data arg of mp_obj_new_fun_asm() a const pointer. 2019-02-20 13:14:03 +11:00
Damien George eee1e8841a py: Downcase all MP_OBJ_IS_xxx macros to make a more consistent C API.
These macros could in principle be (inline) functions so it makes sense to
have them lower case, to match the other C API functions.

The remaining macros that are upper case are:
- MP_OBJ_TO_PTR, MP_OBJ_FROM_PTR
- MP_OBJ_NEW_SMALL_INT, MP_OBJ_SMALL_INT_VALUE
- MP_OBJ_NEW_QSTR, MP_OBJ_QSTR_VALUE
- MP_OBJ_FUN_MAKE_SIG
- MP_DECLARE_CONST_xxx
- MP_DEFINE_CONST_xxx

These must remain macros because they are used when defining const data (at
least, MP_OBJ_NEW_SMALL_INT is so it makes sense to have
MP_OBJ_SMALL_INT_VALUE also a macro).

For those macros that have been made lower case, compatibility macros are
provided for the old names so that users do not need to change their code
immediately.
2019-02-12 14:54:51 +11:00
Damien George afecc124e6 py: Fix location of VM returned exception in invalid opcode and comments
The location for a returned exception was changed to state[0] in
d95947b48a
2019-01-04 17:22:40 +11:00
Damien George 6d19934463 py: Get optional VM stack overflow check compiling and working again.
Changes to the layout of the bytecode header meant that this debug code was
no longer compiling.  This is now fixed and a new compile-time option is
introduced, MICROPY_DEBUG_VM_STACK_OVERFLOW, to turn on this feature (which
is disabled by default).  This option is needed because more than one file
needs to cooperate to make this check work.
2019-01-04 17:09:41 +11:00
Damien George cc2bd63c57 py/emitnative: Implement yield and yield-from in native emitter.
This commit adds first class support for yield and yield-from in the native
emitter, including send and throw support, and yields enclosed in exception
handlers (which requires pulling down the NLR stack before yielding, then
rebuilding it when resuming).

This has been fully tested and is working on unix x86 and x86-64, and
stm32.  Also basic tests have been done with the esp8266 port.  Performance
of existing native code is unchanged.
2018-10-01 13:31:11 +10:00
Damien George d95947b48a py/vm: When VM raises exception put exc obj at beginning of func state.
Instead of at end of state, n_state - 1.  It was originally (way back in
v1.0) put at the end of the state because the VM didn't have a pointer to
the start.  But now that the VM takes a mp_code_state_t pointer it does
have a pointer to the start of the state so can put the exception object
there.

This commit saves about 30 bytes of code on all architectures, and, more
importantly, reduces C-stack usage by a couple of words (8 bytes on Thumb2
and 16 bytes on x86-64) for every (non-generator) call of a bytecode
function because fun_bc_call no longer needs to remember the n_state
variable.
2018-09-29 23:25:08 +10:00
Damien George 43f1848bfa py: Make viper functions have the same entry signature as native.
This commit makes viper functions have the same signature as native
functions, at the level of the emitter/assembler.  This means that viper
functions can now be wrapped in the same uPy object as native functions.

Viper functions are now responsible for parsing their arguments (before it
was done by the runtime), and this makes calling them more efficient (in
most cases) because the viper entry code can be custom generated to suit
the signature of the function.

This change also opens the way forward for viper functions to take
arbitrary numbers of arguments, and for them to handle globals correctly,
among other things.
2018-09-15 22:39:27 +10:00
Damien George 9f241ef398 py: Optimise call to mp_arg_check_num by compressing fun signature.
With 5 arguments to mp_arg_check_num(), some architectures need to pass
values on the stack.  So compressing n_args_min, n_args_max, takes_kw into
a single word and passing only 3 arguments makes the call more efficient,
because almost all calls to this function pass in constant values.  Code
size is also reduced by a decent amount:

   bare-arm:  -116
minimal x86:   -64
   unix x64:  -256
unix nanbox:  -112
      stm32:  -324
     cc3200:  -192
    esp8266:  -192
      esp32:  -144
2018-09-14 13:39:17 +10:00
Damien George b630dfcc1d py: Fix compiling with debug enabled and make more use of DEBUG_printf.
DEBUG_printf and MICROPY_DEBUG_PRINTER is now used instead of normal
printf, and a fault is fixed in mp_obj_class_lookup with debugging enabled;
see issue #3999.  Debugging can now be enabled on all ports including when
nan-boxing is used.
2018-08-02 14:17:24 +10:00
Damien George e2e22e3d7e py/objgenerator: Implement __name__ with normal fun attr accessor code.
With the recent change b488a4a848, a
generating function now has the same layout in memory as a normal bytecode
function, and so can reuse the latter's attribute accessor code to
implement __name__.
2018-07-10 16:33:57 +10:00
Tom Collins a883fe12d9 py/objfun: Fix variable name in DECODE_CODESTATE_SIZE() macro.
This patch fixes the macro so you can pass any name in, and the macro will
make more sense if you're reading it on its own.  It worked previously
because n_state is always passed in as n_state_out_var.
2018-05-17 11:20:06 +10:00
Damien George 1e5a33df41 py: Convert all uses of alloca() to use new scoped allocation API. 2017-12-11 13:49:09 +11:00
Paul Sokolovsky 2b00181592 py/objfun: Factor out macro for initializing codestate.
This is second part of fun_bc_call() vs mp_obj_fun_bc_prepare_codestate()
common code refactor. This factors out code to initialize codestate
object. After this patch, mp_obj_fun_bc_prepare_codestate() is effectively
DECODE_CODESTATE_SIZE() followed by allocation followed by
INIT_CODESTATE(), and fun_bc_call() starts with that too.
2017-12-09 12:49:00 +02:00
Paul Sokolovsky d72370def7 py/objfun, vm: Add comments on codestate allocation in stackless mode. 2017-12-09 11:01:34 +02:00
Paul Sokolovsky fca1d1aa62 py/objfun: Factor out macro for decoding codestate size.
fun_bc_call() starts with almost the same code as
mp_obj_fun_bc_prepare_codestate(), the only difference is a way to
allocate the codestate object (heap vs stack with heap fallback).
Still, would be nice to avoid code duplication to make further
refactoring easier.

So, this commit factors out the common code before the allocation -
decoding and calculating codestate size. It produces two values,
so structured as a macro which writes to 2 variables passed as
arguments.
2017-12-09 09:19:34 +02:00
Damien George a3dc1b1957 all: Remove inclusion of internal py header files.
Header files that are considered internal to the py core and should not
normally be included directly are:
    py/nlr.h - internal nlr configuration and declarations
    py/bc0.h - contains bytecode macro definitions
    py/runtime0.h - contains basic runtime enums

Instead, the top-level header files to include are one of:
    py/obj.h - includes runtime0.h and defines everything to use the
        mp_obj_t type
    py/runtime.h - includes mpstate.h and hence nlr.h, obj.h, runtime0.h,
        and defines everything to use the general runtime support functions

Additional, specific headers (eg py/objlist.h) can be included if needed.
2017-10-04 12:37:50 +11:00
Stefan Naumann ace9fb5405 py: Add verbose debug compile-time flag MICROPY_DEBUG_VERBOSE.
It enables all the DEBUG_printf outputs in the py/ source code.
2017-08-15 11:53:36 +10:00
Alexander Steffen 55f33240f3 all: Use the name MicroPython consistently in comments
There were several different spellings of MicroPython present in comments,
when there should be only one.
2017-07-31 18:35:40 +10:00
Damien George a8a5d1e8c8 py: Provide mp_decode_uint_skip() to help reduce stack usage.
Taking the address of a local variable leads to increased stack usage, so
the mp_decode_uint_skip() function is added to reduce the need for taking
addresses.  The changes in this patch reduce stack usage of a Python call
by 8 bytes on ARM Thumb, by 16 bytes on non-windowing Xtensa archs, and by
16 bytes on x86-64.  Code size is also slightly reduced on most archs by
around 32 bytes.
2017-06-09 13:36:33 +10:00
Damien George 6b34107537 py: Change mp_uint_t to size_t for mp_obj_str_get_data len arg. 2017-03-29 12:56:45 +11:00
Damien George 6213ad7f46 py: Convert mp_uint_t to size_t for tuple/list accessors.
This patch changes mp_uint_t to size_t for the len argument of the
following public facing C functions:

mp_obj_tuple_get
mp_obj_list_get
mp_obj_get_array

These functions take a pointer to the len argument (to be filled in by the
function) and callers of these functions should update their code so the
type of len is changed to size_t.  For ports that don't use nan-boxing
there should be no change in generate code because the size of the type
remains the same (word sized), and in a lot of cases there won't even be a
compiler warning if the type remains as mp_uint_t.

The reason for this change is to standardise on the use of size_t for
variables that count memory (or memory related) sizes/lengths.  It helps
builds that use nan-boxing.
2017-03-29 12:56:17 +11:00
Damien George 71a3d6ec3b py: Reduce size of mp_code_state_t structure.
Instead of caching data that is constant (code_info, const_table and
n_state), store just a pointer to the underlying function object from which
this data can be derived.

This helps reduce stack usage for the case when the mp_code_state_t
structure is stored on the stack, as well as heap usage when it's stored
on the heap.

The downside is that the VM becomes a little more complex because it now
needs to derive the data from the underlying function object.  But this
doesn't impact the performance by much (if at all) because most of the
decoding of data is done outside the main opcode loop.  Measurements using
pystone show that little to no performance is lost.

This patch also fixes a nasty bug whereby the bytecode can be reclaimed by
the GC during execution.  With this patch there is always a pointer to the
function object held by the VM during execution, since it's stored in the
mp_code_state_t structure.
2017-03-17 16:39:13 +11:00
Krzysztof Blazewicz 7e480e8a30 py: Use mp_obj_get_array where sequence may be a tuple or a list. 2017-03-07 16:48:16 +11:00
Damien George dbcdb9f8d8 py/objfun: Convert mp_uint_t to size_t where appropriate. 2017-02-16 16:51:16 +11:00
Damien George ad297a1950 py: Allow inline-assembler emitter to be generic.
This patch refactors some code so that it is easier to integrate new
inline assemblers for different architectures other than ARM Thumb.
2016-12-09 17:06:21 +11:00
Damien George 571e6f26db py: Specialise builtin funcs to use separate type for fixed arg count.
Builtin functions with a fixed number of arguments (0, 1, 2 or 3) are
quite common.  Before this patch the wrapper for such a function cost
3 machine words.  After this patch it only takes 2, which can reduce the
code size by quite a bit (and pays off even more, the more functions are
added).  It also makes function dispatch slightly more efficient in CPU
usage, and furthermore reduces stack usage for these cases.  On x86 and
Thumb archs the dispatch functions are now tail-call optimised by the
compiler.

The bare-arm port has its code size increase by 76 bytes, but stmhal drops
by 904 bytes.  Stack usage by these builtin functions is decreased by 48
bytes on Thumb2 archs.
2016-10-21 16:26:01 +11:00
Damien George 0c595fa094 py/objfun: Use if instead of switch to check return value of VM execute.
It's simpler and improves code coverage.
2016-09-27 23:08:10 +10:00
Damien George c71edaed73 py/objfun: Remove unnecessary check for viper fun with 5 or more args.
The native emitter/compiler restricts viper functions to 4 args, so there
is no need for an extra check in the dynamic dispatch.
2016-09-27 23:05:51 +10:00
Damien George 581a59a456 py: Rename struct mp_code_state to mp_code_state_t.
Also at _t to mp_exc_stack pre-declaration in struct typedef.
2016-08-27 23:21:00 +10:00
Damien George 9a58316de2 py/objfun: Allow inline-asm functions to be called with 4 arguments. 2016-03-16 08:22:26 +00:00
Damien George 5f3e005b67 py: Extend native type-sig to use 4 bits, so uint is separate to ptr.
Before this patch, the native types for uint and ptr/ptr8/ptr16/ptr32
all overlapped and it was possible to make a mistake in casting.  Now,
these types are all separate and any coding mistakes will be raised
as runtime errors.
2016-02-02 23:16:05 +00:00
Damien George 8f54c08691 py/inlineasm: Add ability to specify return type of asm_thumb funcs.
Supported return types are: object, bool, int, uint.

For example:

@micropython.asm_thumb
def foo(r0, r1) -> uint:
    add(r0, r0, r1)
2016-01-27 14:27:10 +00:00
Damien George a0c97814df py: Change type of .make_new and .call args: mp_uint_t becomes size_t.
This patch changes the type signature of .make_new and .call object method
slots to use size_t for n_args and n_kw (was mp_uint_t.  Makes code more
efficient when mp_uint_t is larger than a machine word.  Doesn't affect
ports when size_t and mp_uint_t have the same size.
2016-01-11 00:48:41 +00:00
Damien George 1b0aab621b py: Change struct and macro for builtin fun so they can be type checked. 2016-01-03 11:53:44 +00:00
Damien George 1d899e1783 py/bc: Use size_t instead of mp_uint_t to count size of state and args. 2015-12-17 12:33:42 +00:00
Damien George 7a30e87d2b py: Fix MICROPY_STACKLESS mode to compile with MICROPY_OBJ_REPR_D. 2015-12-17 12:32:41 +00:00
Damien George 999cedb90f py: Wrap all obj-ptr conversions in MP_OBJ_TO_PTR/MP_OBJ_FROM_PTR.
This allows the mp_obj_t type to be configured to something other than a
pointer-sized primitive type.

This patch also includes additional changes to allow the code to compile
when sizeof(mp_uint_t) != sizeof(void*), such as using size_t instead of
mp_uint_t, and various casts.
2015-11-29 14:25:35 +00:00
Damien George 9f6976b74e py: Make mp_setup_code_state take concrete pointer for func arg. 2015-11-29 14:25:04 +00:00
Damien George c8e9c0d89a py: Add MICROPY_PERSISTENT_CODE so code can persist beyond the runtime.
Main changes when MICROPY_PERSISTENT_CODE is enabled are:

- qstrs are encoded as 2-byte fixed width in the bytecode
- all pointers are removed from bytecode and put in const_table (this
  includes const objects and raw code pointers)

Ultimately this option will enable persistence for not just bytecode but
also native code.
2015-11-13 12:49:18 +00:00
Damien George 713ea1800d py: Add constant table to bytecode.
Contains just argument names at the moment but makes it easy to add
arbitrary constants.
2015-11-13 12:49:18 +00:00