Commit Graph

179 Commits

Author SHA1 Message Date
Scott Shawcroft 966d25c6a5
Merge MicroPython v1.15 into CircuitPython 2021-05-12 17:51:42 -07:00
Scott Shawcroft f0bb26d70f
Merge MicroPython 1.13 into CircuitPython 2021-05-04 18:06:33 -07:00
Scott Shawcroft b35fa44c8a
Merge MicroPython 1.12 into CircuitPython 2021-05-03 14:01:18 -07:00
Scott Shawcroft 76033d5115
Merge MicroPython v1.11 into CircuitPython 2021-04-26 15:47:41 -07:00
Scott Shawcroft 09f7b43c64
Merge MicroPython 1.10 into CircuitPython 2021-04-21 15:59:17 -07:00
Scott Shawcroft b057fb8a4b
codeformat 2021-04-19 22:22:44 -07:00
microDev a52eb88031
run code formatting script 2021-03-15 19:27:36 +05:30
Damien George ad4656b861 all: Rename BYTES_PER_WORD to MP_BYTES_PER_OBJ_WORD.
The "word" referred to by BYTES_PER_WORD is actually the size of mp_obj_t
which is not always the same as the size of a pointer on the target
architecture.  So rename this config value to better reflect what it
measures, and also prefix it with MP_.

For uses of BYTES_PER_WORD in setting the stack limit this has been
changed to sizeof(void *), because the stack usually grows with
machine-word sized values (eg an nlr_buf_t has many machine words in it).

Signed-off-by: Damien George <damien@micropython.org>
2021-02-04 22:46:42 +11:00
Diego Elio Pettenò 34b4993d63 Add license to some obvious files. 2020-07-06 19:16:25 +01:00
stijn 84fa3312cf all: Format code to add space after C++-style comment start.
Note: the uncrustify configuration is explicitly set to 'add' instead of
'force' in order not to alter the comments which use extra spaces after //
as a means of indenting text for clarity.
2020-04-23 11:24:25 +10:00
Damien George 69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
Damien George 035180ca01 py: Remove commented-out debug printf's from emitbc and objlist.
Any debugging prints should use a macro like DEBUG_printf.
2019-12-20 23:34:46 +11:00
Damien George c8c0fd4ca3 py: Rework and compress second part of bytecode prelude.
This patch compresses the second part of the bytecode prelude which
contains the source file name, function name, source-line-number mapping
and cell closure information.  This part of the prelude now begins with a
single varible length unsigned integer which encodes 2 numbers, being the
byte-size of the following 2 sections in the header: the "source info
section" and the "closure section".  After decoding this variable unsigned
integer it's possible to skip over one or both of these sections very
easily.

This scheme saves about 2 bytes for most functions compared to the original
format: one in the case that there are no closure cells, and one because
padding was eliminated.
2019-10-01 12:26:22 +10:00
Damien George b5ebfadbd6 py: Compress first part of bytecode prelude.
The start of the bytecode prelude contains 6 numbers telling the amount of
stack needed for the Python values and exceptions, and the signature of the
function.  Prior to this patch these numbers were all encoded one after the
other (2x variable unsigned integers, then 4x bytes), but using so many
bytes is unnecessary.

An entropy analysis of around 150,000 bytecode functions from the CPython
standard library showed that the optimal Shannon coding would need about
7.1 bits on average to encode these 6 numbers, compared to the existing 48
bits.

This patch attempts to get close to this optimal value by packing the 6
numbers into a single, varible-length unsigned integer via bit-wise
interleaving.  The interleaving scheme is chosen to minimise the average
number of bytes needed, and at the same time keep the scheme simple enough
so it can be implemented without too much overhead in code size or speed.
The scheme requires about 10.5 bits on average to store the 6 numbers.

As a result most functions which originally took 6 bytes to encode these 6
numbers now need only 1 byte (in 80% of cases).
2019-10-01 12:26:22 +10:00
Damien George 02db91a7a3 py: Split RAISE_VARARGS opcode into 3 separate ones.
From the beginning of this project the RAISE_VARARGS opcode was named and
implemented following CPython, where it has an argument (to the opcode)
counting how many args the raise takes:

    raise # 0 args (re-raise previous exception)
    raise exc # 1 arg
    raise exc from exc2 # 2 args (chained raise)

In the bytecode this operation therefore takes 2 bytes, one for
RAISE_VARARGS and one for the number of args.

This patch splits this opcode into 3, where each is now a single byte.
This reduces bytecode size by 1 byte for each use of raise.  Every byte
counts!  It also has the benefit of reducing code size (on all ports except
nanbox).
2019-09-26 15:39:50 +10:00
Damien George 870e900d02 py: Introduce and use constants for multi-opcode sizes. 2019-09-26 15:27:11 +10:00
Milan Rossa 310b3d1b81 py: Integrate sys.settrace feature into the VM and runtime.
This commit adds support for sys.settrace, allowing to install Python
handlers to trace execution of Python code.  The interface follows CPython
as closely as possible.  The feature is disabled by default and can be
enabled via MICROPY_PY_SYS_SETTRACE.
2019-08-30 16:44:12 +10:00
Damien George 8e7745eb31 py/emitbc: Make all emit_write_bytecode_* funcs take a stack_adj arg.
This factoring of code gives significant code-size savings:

   bare-arm:  -456 -0.682%
minimal x86:  -844 -0.547%
   unix x64:  -472 -0.095%
unix nanbox: -1348 -0.303%
      stm32:  -472 -0.130% PYBV10
     cc3200:  -448 -0.242%
    esp8266:  -708 -0.108%
      esp32:  -400 -0.036% GENERIC
        nrf:  -520 -0.356% pca10040
       samd:  -456 -0.448% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2019-08-22 15:32:26 +10:00
Damien George 3d9bd80447 py/emitbc: Rewrite switch in load_const_tok to reduce code size. 2019-08-22 15:10:25 +10:00
Damien George d9d92f27d7 py/compile: Add support to select the native emitter at runtime. 2019-03-14 12:22:25 +11:00
Damien George 5a2599d962 py: Replace POP_BLOCK and POP_EXCEPT opcodes with POP_EXCEPT_JUMP.
POP_BLOCK and POP_EXCEPT are now the same, and are always followed by a
JUMP.  So this optimisation reduces code size, and RAM usage of bytecode by
two bytes for each try-except handler.
2019-03-05 16:09:58 +11:00
Damien George e1fb03f3e2 py: Fix VM crash with unwinding jump out of a finally block.
This patch fixes a bug in the VM when breaking within a try-finally.  The
bug has to do with executing a break within the finally block of a
try-finally statement.  For example:

    def f():
        for x in (1,):
            print('a', x)
            try:
                raise Exception
            finally:
                print(1)
                break
            print('b', x)
    f()

Currently in uPy the above code will print:

    a 1
    1
    1
    segmentation fault (core dumped)  micropython

Not only is there a seg fault, but the "1" in the finally block is printed
twice.  This is because when the VM executes a finally block it doesn't
really know if that block was executed due to a fall-through of the try (no
exception raised), or because an exception is active.  In particular, for
nested finallys the VM has no idea which of the nested ones have active
exceptions and which are just fall-throughs.  So when a break (or continue)
is executed it tries to unwind all of the finallys, when in fact only some
may be active.

It's questionable whether break (or return or continue) should be allowed
within a finally block, because they implicitly swallow any active
exception, but nevertheless it's allowed by CPython (although almost never
used in the standard library).  And uPy should at least not crash in such a
case.

The solution here relies on the fact that exception and finally handlers
always appear in the bytecode after the try body.

Note: there was a similar bug with a return in a finally block, but that
was previously fixed in b735208403
2019-03-05 16:05:05 +11:00
Damien George 6d19934463 py: Get optional VM stack overflow check compiling and working again.
Changes to the layout of the bytecode header meant that this debug code was
no longer compiling.  This is now fixed and a new compile-time option is
introduced, MICROPY_DEBUG_VM_STACK_OVERFLOW, to turn on this feature (which
is disabled by default).  This option is needed because more than one file
needs to cooperate to make this check work.
2019-01-04 17:09:41 +11:00
Damien George 80db30a510 py/emit: Completely remove set_native_type, arg type is set in compiler.
In viper mode, the type of the argument is now stored in id_info->flags.
2018-09-15 13:00:11 +10:00
Damien George 18e6358480 py/emit: Combine setup with/except/finally into one emit function.
This patch reduces code size by:

   bare-arm:   -16
minimal x86:  -156
   unix x64:  -288
unix nanbox:  -184
      stm32:   -48
     cc3200:   -16
    esp8266:   -96
      esp32:   -16

The last 10 patches combined reduce code size by:

   bare-arm:  -164
minimal x86: -1260
   unix x64: -3416
unix nanbox: -1616
      stm32:  -676
     cc3200:  -232
    esp8266: -1144
      esp32:  -268
2018-05-23 00:35:16 +10:00
Damien George 436e0d4c54 py/emit: Merge build set/slice into existing build emit function.
Reduces code size by:

   bare-arm:    +0
minimal x86:    +0
   unix x64:  -368
unix nanbox:  -248
      stm32:  -128
     cc3200:   -48
    esp8266:  -184
      esp32:   -40
2018-05-23 00:23:36 +10:00
Damien George d97906ca9a py/emit: Combine import from/name/star into one emit function.
Change in code size is:

   bare-arm:    +4
minimal x86:   -88
   unix x64:  -456
unix nanbox:   -88
      stm32:   -44
     cc3200:    +0
    esp8266:  -104
      esp32:    +8
2018-05-23 00:23:08 +10:00
Damien George 8a513da5a5 py/emit: Combine break_loop and continue_loop into one emit function.
Reduces code size by:

   bare-arm:    +0
minimal x86:    +0
   unix x64:   -80
unix nanbox:    +0
      stm32:   -12
     cc3200:    +0
    esp8266:   -28
      esp32:    +0
2018-05-23 00:23:04 +10:00
Damien George 6211d979ee py/emit: Combine load/store/delete attr into one emit function.
Reduces code size by:

   bare-arm:   -20
minimal x86:  -140
   unix x64:  -408
unix nanbox:  -140
      stm32:   -68
     cc3200:   -16
    esp8266:   -80
      esp32:   -32
2018-05-23 00:22:59 +10:00
Damien George a4941a8ba4 py/emit: Combine load/store/delete subscr into one emit function.
Reduces code size by:

   bare-arm:    -8
minimal x86:  -104
   unix x64:  -312
unix nanbox:  -120
      stm32:   -60
     cc3200:   -16
    esp8266:   -92
      esp32:   -24
2018-05-23 00:22:55 +10:00
Damien George d298013939 py/emit: Combine name and global into one func for load/store/delete.
Reduces code size by:

   bare-arm:   -56
minimal x86:  -300
   unix x64:  -576
unix nanbox:  -300
      stm32:  -164
     cc3200:   -56
    esp8266:  -236
      esp32:   -76
2018-05-23 00:22:47 +10:00
Damien George 26b5754092 py/emit: Combine build tuple/list/map emit funcs into one.
Reduces code size by:

   bare-arm:   -24
minimal x86:  -192
   unix x64:  -288
unix nanbox:  -184
      stm32:   -72
     cc3200:   -16
    esp8266:  -148
      esp32:   -32
2018-05-23 00:22:44 +10:00
Damien George e686c94052 py/emit: Combine yield value and yield-from emit funcs into one.
Reduces code size by:

   bare-arm:   -24
minimal x86:   -72
   unix x64:  -200
unix nanbox:   -72
      stm32:   -52
     cc3200:   -32
    esp8266:   -84
      esp32:   -24
2018-05-23 00:22:35 +10:00
Damien George 0a25fff956 py/emit: Combine fast and deref into one function for load/store/delete.
Reduces code size by:

   bare-arm:   -16
minimal x86:  -208
   unix x64:  -408
unix nanbox:  -248
      stm32:   -12
     cc3200:   -24
    esp8266:   -96
      esp32:   -44
2018-05-23 00:22:20 +10:00
Jeff Epler bc6c0b28bf py/emitbc: Avoid undefined behavior calling memset() with NULL 1st arg.
Calling memset(NULL, value, 0) is not standards compliant so we must add an
explicit check that emit->label_offsets is indeed not NULL before calling
memset (this pointer will be NULL on the first pass of the parse tree and
it's more logical / safer to check this pointer rather than check that the
pass is not the first one).

Code sanitizers will warn if NULL is passed as the first value to memset,
and compilers may optimise the code based on the knowledge that any pointer
passed to memset is guaranteed not to be NULL.
2018-05-21 12:04:20 +10:00
Damien George 5604b710c2 py/emitglue: When assigning bytecode only pass bytecode len if needed.
Most embedded targets will have this bit of the code disabled, saving a
small amount of code space.
2018-02-14 18:41:17 +11:00
Damien George f935bce3c5 py/{emitbc,asmbase}: Only clear emit labels to -1 when in debug mode.
Clearing the labels to -1 is purely a debugging measure.  For release
builds there is no need to do it as the label offset table should always
have the correct value assigned.
2017-12-08 18:23:23 +11:00
Damien George 280fb4d928 py/emitbc: Remove stray semicolon in outer scope. 2017-09-13 20:36:06 +10:00
Alexander Steffen 55f33240f3 all: Use the name MicroPython consistently in comments
There were several different spellings of MicroPython present in comments,
when there should be only one.
2017-07-31 18:35:40 +10:00
Tom Collins 145796f037 py,extmod: Some casts and minor refactors to quiet compiler warnings. 2017-07-07 11:32:22 +10:00
Damien George 8f064e469d py/emitbc: Fix bug with BC emitter computing Python stack size.
Previous to this patch the mp_emit_bc_adjust_stack_size function would
adjust the current stack size but would not increase the maximum stack size
if the current size went above it.  This meant that certain Python code
(eg a try-finally block with no statements inside it) would not have enough
Python stack allocated to it.

This patch fixes the problem by always checking if the current stack size
goes above the maximum, and adjusting the latter if it does.
2017-05-25 20:42:30 +10:00
Damien George dd11af209d py: Add LOAD_SUPER_METHOD bytecode to allow heap-free super meth calls.
This patch allows the following code to run without allocating on the heap:

    super().foo(...)

Before this patch such a call would allocate a super object on the heap and
then load the foo method and call it right away.  The super object is only
needed to perform the lookup of the method and not needed after that.  This
patch makes an optimisation to allocate the super object on the C stack and
discard it right after use.

Changes in code size due to this patch are:

   bare-arm: +128
    minimal: +232
   unix x64: +416
unix nanbox: +364
     stmhal: +184
    esp8266: +340
     cc3200: +128
2017-04-22 23:39:20 +10:00
Damien George 60656eaea4 py: Define and use MP_OBJ_ITER_BUF_NSLOTS to get size of stack iter buf.
It improves readability of code and reduces the chance to make a mistake.

This patch also fixes a bug with nan-boxing builds by rounding up the
calculation of the new NSLOTS variable, giving the correct number of slots
(being 4) even if mp_obj_t is larger than the native machine size.
2017-03-23 16:36:08 +11:00
Damien George 30b42dd72d py: Remove unused "use_stack" argument from for_iter_end emit function. 2017-02-16 18:38:06 +11:00
Damien George 088740ecc4 py: Optimise storage of iterator so it takes only 4 slots on Py stack. 2017-02-16 18:38:06 +11:00
Damien George 6e769da0da py: Make FOR_ITER opcode pop 1+4 slots from the stack when finished.
The extra 4 slots correspond to the iterator object stored on the stack.
2017-02-16 18:38:06 +11:00
Damien George f4df3aaa72 py: Allow bytecode/native to put iter_buf on stack for simple for loops.
So that the "for x in it: ..." statement can now work without using the
heap (so long as the iterator argument fits in an iter_buf structure).
2017-02-16 18:38:06 +11:00
Damien George cc2dbdd1fe py/emitbc: Produce correct line number info for large bytecode chunks.
Previous to this patch, for large chunks of bytecode that originated from
a single source-code line, the bytecode-line mapping would generate
something like (for 42 bytecode bytes and 1 line):

  BC_SKIP=31  LINE_SKIP=1
  BC_SKIP=11  LINE_SKIP=0

This would mean that any errors in the last 11 bytecode bytes would be
reported on the following line.  This patch fixes it to generate instead:

  BC_SKIP=31  LINE_SKIP=0
  BC_SKIP=11  LINE_SKIP=1
2017-02-10 11:58:10 +11:00
Pavol Rusnak 7ffc959c00 py: remove asserts that are always true in emitbc.c 2016-10-31 23:21:22 +03:00
Damien George 7385b018ed py/emitbc: Remove/refactor unreachable code, to improve coverage. 2016-09-27 15:46:50 +10:00