Commit Graph

142 Commits

Author SHA1 Message Date
Damien George 280fb4d928 py/emitbc: Remove stray semicolon in outer scope. 2017-09-13 20:36:06 +10:00
Alexander Steffen 55f33240f3 all: Use the name MicroPython consistently in comments
There were several different spellings of MicroPython present in comments,
when there should be only one.
2017-07-31 18:35:40 +10:00
Tom Collins 145796f037 py,extmod: Some casts and minor refactors to quiet compiler warnings. 2017-07-07 11:32:22 +10:00
Damien George 8f064e469d py/emitbc: Fix bug with BC emitter computing Python stack size.
Previous to this patch the mp_emit_bc_adjust_stack_size function would
adjust the current stack size but would not increase the maximum stack size
if the current size went above it.  This meant that certain Python code
(eg a try-finally block with no statements inside it) would not have enough
Python stack allocated to it.

This patch fixes the problem by always checking if the current stack size
goes above the maximum, and adjusting the latter if it does.
2017-05-25 20:42:30 +10:00
Damien George dd11af209d py: Add LOAD_SUPER_METHOD bytecode to allow heap-free super meth calls.
This patch allows the following code to run without allocating on the heap:

    super().foo(...)

Before this patch such a call would allocate a super object on the heap and
then load the foo method and call it right away.  The super object is only
needed to perform the lookup of the method and not needed after that.  This
patch makes an optimisation to allocate the super object on the C stack and
discard it right after use.

Changes in code size due to this patch are:

   bare-arm: +128
    minimal: +232
   unix x64: +416
unix nanbox: +364
     stmhal: +184
    esp8266: +340
     cc3200: +128
2017-04-22 23:39:20 +10:00
Damien George 60656eaea4 py: Define and use MP_OBJ_ITER_BUF_NSLOTS to get size of stack iter buf.
It improves readability of code and reduces the chance to make a mistake.

This patch also fixes a bug with nan-boxing builds by rounding up the
calculation of the new NSLOTS variable, giving the correct number of slots
(being 4) even if mp_obj_t is larger than the native machine size.
2017-03-23 16:36:08 +11:00
Damien George 30b42dd72d py: Remove unused "use_stack" argument from for_iter_end emit function. 2017-02-16 18:38:06 +11:00
Damien George 088740ecc4 py: Optimise storage of iterator so it takes only 4 slots on Py stack. 2017-02-16 18:38:06 +11:00
Damien George 6e769da0da py: Make FOR_ITER opcode pop 1+4 slots from the stack when finished.
The extra 4 slots correspond to the iterator object stored on the stack.
2017-02-16 18:38:06 +11:00
Damien George f4df3aaa72 py: Allow bytecode/native to put iter_buf on stack for simple for loops.
So that the "for x in it: ..." statement can now work without using the
heap (so long as the iterator argument fits in an iter_buf structure).
2017-02-16 18:38:06 +11:00
Damien George cc2dbdd1fe py/emitbc: Produce correct line number info for large bytecode chunks.
Previous to this patch, for large chunks of bytecode that originated from
a single source-code line, the bytecode-line mapping would generate
something like (for 42 bytecode bytes and 1 line):

  BC_SKIP=31  LINE_SKIP=1
  BC_SKIP=11  LINE_SKIP=0

This would mean that any errors in the last 11 bytecode bytes would be
reported on the following line.  This patch fixes it to generate instead:

  BC_SKIP=31  LINE_SKIP=0
  BC_SKIP=11  LINE_SKIP=1
2017-02-10 11:58:10 +11:00
Pavol Rusnak 7ffc959c00 py: remove asserts that are always true in emitbc.c 2016-10-31 23:21:22 +03:00
Damien George 7385b018ed py/emitbc: Remove/refactor unreachable code, to improve coverage. 2016-09-27 15:46:50 +10:00
Damien George f040685b0c py: Only store the exception instance on Py stack in bytecode try block.
When an exception is raised and is to be handled by the VM, it is stored
on the Python value stack so the bytecode can access it.  CPython stores
3 objects on the stack for each exception: exc type, exc instance and
traceback.  uPy followed this approach, but it turns out not to be
necessary.  Instead, it is enough to store just the exception instance on
the Python value stack.  The only place where the 3 values are needed
explicitly is for the __exit__ handler of a with-statement context, but
for these cases the 3 values can be extracted from the single exception
instance.

This patch removes the need to store 3 values on the stack, and instead
just stores the exception instance.

Code size is reduced by about 50-100 bytes, the compiler and VM are
slightly simpler, generate bytecode is smaller (by 2 bytes for each try
block), and the Python value stack is reduced in size for functions that
handle exceptions.
2016-09-27 12:37:21 +10:00
Damien George adaf0d865c py: Combine 3 comprehension opcodes (list/dict/set) into 1.
With the previous patch combining 3 emit functions into 1, it now makes
sense to also combine the corresponding VM opcodes, which is what this
patch does.  This eliminates 2 opcodes which simplifies the VM and reduces
code size, in bytes: bare-arm:44, minimal:64, unix(NDEBUG,x86-64):272,
stmhal:92, esp8266:200.  Profiling (with a simple script that creates many
list/dict/set comprehensions) shows no measurable change in performance.
2016-09-19 12:28:03 +10:00
Damien George a5624bf381 py: Combine 3 comprehension emit functions (list/dict/set) into 1.
The 3 kinds of comprehensions are similar enough that merging their emit
functions reduces code size.  Decreases in code size in bytes are:
bare-arm:24, minimal:96, unix(NDEBUG,x86-64):328, stmhal:80, esp8266:76.
2016-09-19 12:23:31 +10:00
Damien George ce8b4e8749 py: Combine continuous block of emit steps into with_cleanup emit call.
Because different emitters need to handle with-cleanup in different ways.
2016-04-07 08:50:38 +01:00
Damien George ea23520403 py: Add MICROPY_DYNAMIC_COMPILER option to config compiler at runtime.
This new compile-time option allows to make the bytecode compiler
configurable at runtime by setting the fields in the mp_dynamic_compiler
structure.  By using this feature, the compiler can generate bytecode
that targets any MicroPython runtime/VM, regardless of the host and
target compile-time settings.

Options so far that fall under this dynamic setting are:
- maximum number of bits that a small int can hold;
- whether caching of lookups is used in the bytecode;
- whether to use unicode strings or not (lexer behaviour differs, and
  therefore generated string constants differ).
2016-02-25 10:05:46 +00:00
Damien George dd5353a405 py: Add MICROPY_ENABLE_COMPILER and MICROPY_PY_BUILTINS_EVAL_EXEC opts.
MICROPY_ENABLE_COMPILER can be used to enable/disable the entire compiler,
which is useful when only loading of pre-compiled bytecode is supported.
It is enabled by default.

MICROPY_PY_BUILTINS_EVAL_EXEC controls support of eval and exec builtin
functions.  By default they are only included if MICROPY_ENABLE_COMPILER
is enabled.

Disabling both options saves about 40k of code size on 32-bit x86.
2015-12-18 12:35:44 +00:00
Damien George bdbe8c9ae2 py: Make UNARY_OP_NOT a first-class op, to agree with Py not semantics.
Fixes #1684 and makes "not" match Python semantics.  The code is also
simplified (the separate MP_BC_NOT opcode is removed) and the patch saves
68 bytes for bare-arm/ and 52 bytes for minimal/.

Previously "not x" was implemented as !mp_unary_op(x, MP_UNARY_OP_BOOL),
so any given object only needs to implement MP_UNARY_OP_BOOL (and the VM
had a special opcode to do the ! bit).

With this patch "not x" is implemented as mp_unary_op(x, MP_UNARY_OP_NOT),
but this operation is caught at the start of mp_unary_op and dispatched as
!mp_obj_is_true(x).  mp_obj_is_true has special logic to test for
truthness, and is the correct way to handle the not operation.
2015-12-10 22:19:48 +00:00
Damien George 999cedb90f py: Wrap all obj-ptr conversions in MP_OBJ_TO_PTR/MP_OBJ_FROM_PTR.
This allows the mp_obj_t type to be configured to something other than a
pointer-sized primitive type.

This patch also includes additional changes to allow the code to compile
when sizeof(mp_uint_t) != sizeof(void*), such as using size_t instead of
mp_uint_t, and various casts.
2015-11-29 14:25:35 +00:00
Damien George 5d66b427e2 py/emit: Change type of arg of load_const_obj from void* to mp_obj_t. 2015-11-29 14:25:04 +00:00
Damien George d8c834c95d py: Add MICROPY_PERSISTENT_CODE_LOAD/SAVE to load/save bytecode.
MICROPY_PERSISTENT_CODE must be enabled, and then enabling
MICROPY_PERSISTENT_CODE_LOAD/SAVE (either or both) will allow loading
and/or saving of code (at the moment just bytecode) from/to a .mpy file.
2015-11-13 12:49:18 +00:00
Damien George c8e9c0d89a py: Add MICROPY_PERSISTENT_CODE so code can persist beyond the runtime.
Main changes when MICROPY_PERSISTENT_CODE is enabled are:

- qstrs are encoded as 2-byte fixed width in the bytecode
- all pointers are removed from bytecode and put in const_table (this
  includes const objects and raw code pointers)

Ultimately this option will enable persistence for not just bytecode but
also native code.
2015-11-13 12:49:18 +00:00
Damien George 713ea1800d py: Add constant table to bytecode.
Contains just argument names at the moment but makes it easy to add
arbitrary constants.
2015-11-13 12:49:18 +00:00
Damien George 3a3db4dcf0 py: Put all bytecode state (arg count, etc) in bytecode. 2015-11-13 12:49:18 +00:00
Damien George 9b7f583b0c py: Reorganise bytecode layout so it's more structured, easier to edit. 2015-11-13 12:49:18 +00:00
Damien George fbcaf0ea18 py: Slightly simplify compile and emit of star/double-star arguments.
Saves a few bytes of code space and eliminates need for rot_two
bytecode (hence saving RAM and execution time, by a tiny bit).
2015-09-23 11:47:01 +01:00
Damien George 3a2171e406 py: Eliminate some cases which trigger unused parameter warnings. 2015-09-04 16:53:46 +01:00
Damien George 65dc960e3b unix-cpy: Remove unix-cpy. It's no longer needed.
unix-cpy was originally written to get semantic equivalent with CPython
without writing functional tests.  When writing the initial
implementation of uPy it was a long way between lexer and functional
tests, so the half-way test was to make sure that the bytecode was
correct.  The idea was that if the uPy bytecode matched CPython 1-1 then
uPy would be proper Python if the bytecodes acted correctly.  And having
matching bytecode meant that it was less likely to miss some deep
subtlety in the Python semantics that would require an architectural
change later on.

But that is all history and it no longer makes sense to retain the
ability to output CPython bytecode, because:

1. It outputs CPython 3.3 compatible bytecode.  CPython's bytecode
changes from version to version, and seems to have changed quite a bit
in 3.5.  There's no point in changing the bytecode output to match
CPython anymore.

2. uPy and CPy do different optimisations to the bytecode which makes it
harder to match.

3. The bytecode tests are not run.  They were never part of Travis and
are not run locally anymore.

4. The EMIT_CPYTHON option needs a lot of extra source code which adds
heaps of noise, especially in compile.c.

5. Now that there is an extensive test suite (which tests functionality)
there is no need to match the bytecode.  Some very subtle behaviour is
tested with the test suite and passing these tests is a much better
way to stay Python-language compliant, rather than trying to match
CPy bytecode.
2015-08-17 12:51:26 +01:00
Damien George 59fba2d6ea py: Remove mp_load_const_bytes and instead load precreated bytes object.
Previous to this patch each time a bytes object was referenced a new
instance (with the same data) was created.  With this patch a single
bytes object is created in the compiler and is loaded directly at execute
time as a true constant (similar to loading bignum and float objects).
This saves on allocating RAM and means that bytes objects can now be
used when the memory manager is locked (eg in interrupts).

The MP_BC_LOAD_CONST_BYTES bytecode was removed as part of this.

Generated bytecode is slightly larger due to storing a pointer to the
bytes object instead of the qstr identifier.

Code size is reduced by about 60 bytes on Thumb2 architectures.
2015-06-25 14:42:13 +00:00
Damien George 9a42eb541e py: Fix naming of function arguments when function is a closure.
Addresses issue #1226.
2015-05-06 13:55:33 +01:00
Damien George 8872abcbc4 py: Remove LOAD_CONST_ELLIPSIS bytecode, use LOAD_CONST_OBJ instead.
Ellipsis constant is rarely used so no point having an extra bytecode
for it.
2015-05-05 22:15:42 +01:00
Damien George 8c1d23a0e2 py: Modify bytecode "with" behaviour so it doesn't use any heap.
Before this patch a "with" block needed to create a bound method object
on the heap for the __exit__ call.  Now it doesn't because we use
load_method instead of load_attr, and save the method+self on the stack.
2015-04-24 01:52:28 +01:00
Damien George e72cda99fd py: Convert occurrences of non-debug printf to mp_printf. 2015-04-16 14:30:16 +00:00
Damien George 91bc32dc16 py: Provide typedefs for function types instead of writing them inline. 2015-04-09 15:31:53 +00:00
Damien George 4dea922610 py: Adjust some spaces in code style/format, purely for consistency. 2015-04-09 15:29:54 +00:00
Damien George c9aa1883ed py: Simplify bytecode prelude when encoding closed over variables. 2015-04-07 00:08:17 +01:00
Damien George 4112590a60 py, compiler: When just bytecode, make explicit calls instead of table.
When just the bytecode emitter is needed there is no need to have a
dynamic method table for the emitter back-end, and we can instead
directly call the mp_emit_bc_XXX functions.  This gives a significant
reduction in code size and a very slight performance boost for the
compiler.

This patch saves 1160 bytes code on Thumb2 and 972 bytes on x86, when
native emitters are disabled.

Overall savings in code over the last 3 commits are:

bare-arm: 1664 bytes.
minimal:  2136 bytes.
stmhal:    584 bytes (it has native emitter enabled).
cc3200:   1736 bytes.
2015-03-26 16:52:45 +00:00
Damien George a210c774f9 py, compiler: Remove emit_pass1 code, using emit_bc to do its job.
First pass for the compiler is computing the scope (eg if an identifier
is local or not) and originally had an entire table of methods dedicated
to this, most of which did nothing.  With changes from previous commit,
this set of methods can be removed and the methods from the bytecode
emitter used instead, with very little modification -- this is what is
done in this commit.

This factoring has little to no impact on the speed of the compiler
(tested by compiling 3763 Python scripts and timing it).

This factoring reduces code size by about 270-300 bytes on Thumb2 archs,
and 400 bytes on x86.
2015-03-26 16:52:45 +00:00
Damien George 542bd6b4a1 py, compiler: Refactor load/store/delete_id logic to reduce code size.
Saves around 230 bytes on Thumb2 and 750 bytes on x86.
2015-03-26 16:52:45 +00:00
Damien George 63f3832e81 py: Combine emit functions for jump true/false to reduce code size.
Saves 116 bytes for stmhal and 56 bytes for cc3200 port.
2015-02-28 15:04:06 +00:00
Damien George 7d414a1b52 py: Parse big-int/float/imag constants directly in parser.
Previous to this patch, a big-int, float or imag constant was interned
(made into a qstr) and then parsed at runtime to create an object each
time it was needed.  This is wasteful in RAM and not efficient.  Now,
these constants are parsed straight away in the parser and turned into
objects.  This allows constants with large numbers of digits (so
addresses issue #1103) and takes us a step closer to #722.
2015-02-08 01:57:40 +00:00
Damien George ff8dd3f486 py, unix: Allow to compile with -Wunused-parameter.
See issue #699.
2015-01-20 12:47:20 +00:00
Damien George 963a5a3e82 py, unix: Allow to compile with -Wsign-compare.
See issue #699.
2015-01-16 17:47:07 +00:00
Damien George 0abb5609b0 py: Remove unnecessary id_flags argument from emitter's load_fast.
Saves 24 bytes in bare-arm.
2015-01-16 12:24:49 +00:00
Damien George d2d64f00fb py: Add "default" to switches to allow better code flow analysis.
This helps compiler produce smaller code.  Saves 124 bytes on stmhal and
bare-arm.
2015-01-14 21:32:42 +00:00
Damien George dab1385177 py: Add load_const_obj to emitter, add LOAD_CONST_OBJ to bytecode.
This allows to directly load a Python object to the Python stack.  See
issue #722 for background.
2015-01-13 15:55:54 +00:00
Damien George 7ee91cf861 py: Add option to cache map lookup results in bytecode.
This is a simple optimisation inspired by JITing technology: we cache in
the bytecode (using 1 byte) the offset of the last successful lookup in
a map. This allows us next time round to check in that location in the
hash table (mp_map_t) for the desired entry, and if it's there use that
entry straight away.  Otherwise fallback to a normal map lookup.

Works for LOAD_NAME, LOAD_GLOBAL, LOAD_ATTR and STORE_ATTR opcodes.

On a few tests it gives >90% cache hit and greatly improves speed of
code.

Disabled by default.  Enabled for unix and stmhal ports.
2015-01-07 21:07:23 +00:00
Damien George b4b10fd350 py: Put all global state together in state structures.
This patch consolidates all global variables in py/ core into one place,
in a global structure.  Root pointers are all located together to make
GC tracing easier and more efficient.
2015-01-07 20:33:00 +00:00