Commit Graph

345 Commits

Author SHA1 Message Date
Damien George 253f2bd7be py/compile: Combine compiler-opt of 2 and 3 tuple-to-tuple assignment.
This patch combines the compiler optimisation code for double and triple
tuple-to-tuple assignment, taking it from two separate if-blocks to one
combined if-block.  This can be done because the code for both of these
optimisations has a lot in common.  Combining them together reduces code
size for ports that have the triple-tuple optimisation enabled (and doesn't
change code size for ports that have it disabled).
2018-02-04 13:35:21 +11:00
Damien George 1e5a33df41 py: Convert all uses of alloca() to use new scoped allocation API. 2017-12-11 13:49:09 +11:00
Damien George 487dbdb267 py/compile: Use alloca instead of qstr_build when compiling import name.
The technique of using alloca is how dotted import names are composed in
mp_import_from and mp_builtin___import__, so use the same technique in the
compiler.  This puts less pressure on the heap (only the stack is used if
the qstr already exists, and if it doesn't exist then the standard qstr
block memory is used for the new qstr rather than a separate chunk of the
heap) and reduces overall code size.
2017-11-01 13:16:16 +11:00
Damien George ad6aae13a4 py/compile: Remove unused pn_colon code when compiling func params. 2017-08-21 22:00:34 +10:00
Alexander Steffen 55f33240f3 all: Use the name MicroPython consistently in comments
There were several different spellings of MicroPython present in comments,
when there should be only one.
2017-07-31 18:35:40 +10:00
Damien George 0291a624cb py/compile: Fix enum variable declaration. 2017-07-09 13:18:14 +10:00
Krzysztof Blazewicz 91a385db98 py/compile: Use switch-case to match token and operator.
Reduces code size.
2017-07-05 15:50:36 +10:00
Krzysztof Blazewicz a040fb89e7 py/compile: Combine arith and bit-shift ops into 1 compile routine.
This refactoring saves code space.
2017-07-05 15:49:00 +10:00
Damien George d94bc675e8 py/compile: Optimise emitter label indices to save a word of heap.
Previous to this patch, a label with value "0" was used to indicate an
invalid label, but that meant a wasted word (at slot 0) in the array of
label offsets.  This patch adjusts the label indices so the first one
starts at 0, and the maximum value indicates an invalid label.
2017-06-22 15:05:58 +10:00
Damien George 4c5f108321 py/compile: Fix bug with break/continue in else of optimised for-range.
This patch fixes a bug whereby the Python stack was not correctly reset if
there was a break/continue statement in the else black of an optimised
for-range loop.

For example, in the following code the "j" variable from the inner for loop
was not being popped off the Python stack:

    for i in range(4):
        for j in range(4):
            pass
        else:
            continue

This is now fixed with this patch.
2017-06-22 13:50:33 +10:00
Damien George 1e70fda69f py/compile: Raise SyntaxError if positional args are given after */**.
In CPython 3.4 this raises a SyntaxError.  In CPython 3.5+ having a
positional after * is allowed but uPy has the wrong semantics and passes
the arguments in the incorrect order.  To prevent incorrect use of a
function going unnoticed it is important to raise the SyntaxError in uPy,
until the behaviour is fixed to follow CPython 3.5+.
2017-06-14 18:18:01 +10:00
Ville Skyttä ca16c38210 various: Spelling fixes 2017-05-29 11:36:05 +03:00
Damien George dd11af209d py: Add LOAD_SUPER_METHOD bytecode to allow heap-free super meth calls.
This patch allows the following code to run without allocating on the heap:

    super().foo(...)

Before this patch such a call would allocate a super object on the heap and
then load the foo method and call it right away.  The super object is only
needed to perform the lookup of the method and not needed after that.  This
patch makes an optimisation to allocate the super object on the C stack and
discard it right after use.

Changes in code size due to this patch are:

   bare-arm: +128
    minimal: +232
   unix x64: +416
unix nanbox: +364
     stmhal: +184
    esp8266: +340
     cc3200: +128
2017-04-22 23:39:20 +10:00
Damien George 5335942b59 py/compile: Refactor handling of special super() call.
This patch refactors the handling of the special super() call within the
compiler.  It removes the need for a global (to the compiler) state variable
which keeps track of whether the subject of an expression is super.  The
handling of super() is now done entirely within one function, which makes
the compiler a bit cleaner and allows to easily add more optimisations to
super calls.

Changes to the code size are:

   bare-arm: +12
    minimal:  +0
   unix x64: +48
unix nanbox: -16
     stmhal:  +4
     cc3200:  +0
    esp8266: -56
2017-04-22 21:46:32 +10:00
Damien George 0dd6a59c89 py/compile: Don't do unnecessary check if iter parse node is a struct.
If we get to this point in the code then pn_iter is guaranteed to be a
struct.
2017-04-22 21:43:42 +10:00
Damien George ae54fbf166 py/compile: Add COMP_RETURN_IF_EXPR option to enable return-if-else opt.
With this optimisation enabled the compiler optimises the if-else
expression within a return statement.  The optimisation reduces bytecode
size by 2 bytes for each use of such a return-if-else statement.  Since
such a statement is not often used, and costs bytes for the code, the
feature is disabled by default.

For example the following code:

    def f(x):
        return 1 if x else 2

compiles to this bytecode with the optimisation disabled (left column is
bytecode offset in bytes):

    00 LOAD_FAST 0
    01 POP_JUMP_IF_FALSE 8
    04 LOAD_CONST_SMALL_INT 1
    05 JUMP 9
    08 LOAD_CONST_SMALL_INT 2
    09 RETURN_VALUE

and to this bytecode with the optimisation enabled:

    00 LOAD_FAST 0
    01 POP_JUMP_IF_FALSE 6
    04 LOAD_CONST_SMALL_INT 1
    05 RETURN_VALUE
    06 LOAD_CONST_SMALL_INT 2
    07 RETURN_VALUE

So the JUMP to RETURN_VALUE is optimised and replaced by RETURN_VALUE,
saving 2 bytes and making the code a bit faster.
2017-04-22 14:58:01 +10:00
Damien George 40b40ffc98 py/compile: Extract parse-node kind at start of func for efficiency.
Otherwise the type of parse-node and its kind has to be re-extracted
multiple times.  This optimisation reduces code size by a bit (16 bytes on
bare-arm).
2017-04-22 14:23:47 +10:00
Damien George fa03bbf0fd py/compile: Don't do unnecessary check if parse node is a struct.
PN_atom_expr_normal parse nodes always have structs for their second
sub-node, so simplify the check for the sub-node kind to save code size.
2017-04-22 14:13:37 +10:00
Damien George de9b53695d py: Raise a ValueError if range() step is zero.
Following CPython.  Otherwise one gets either an infinite loop (if code is
optimised by the uPy compiler) or possibly a divide-by-zero CPU exception.
2017-04-05 10:50:26 +10:00
Damien George f9b0e644e5 py/compile: Provide terse error message for invalid dict/set literals. 2017-03-29 12:44:27 +11:00
Damien George 18c059febf py: Shorten a couple of error messages. 2017-03-29 12:36:46 +11:00
Damien George f55a059e7a py/compile: Simplify syntax-error messages for illegal assignments.
With this patch all illegal assignments are reported as "can't assign to
expression".  Before the patch there were special cases for a literal on
the LHS, and for augmented assignments (eg +=), but it seems a waste of
bytes (and there are lots of bytes used in error messages) to spend on
distinguishing such errors which a user will rarely encounter.
2017-03-29 12:28:33 +11:00
Damien George 40c1272e55 py/compile: When compiling super(), handle closed-over self variable.
The self variable may be closed-over in the function, and in that case the
call to super() should load the contents of the closure cell using
LOAD_DEREF (before this patch it would just load the cell directly).
2017-03-27 11:27:08 +11:00
Damien George 60656eaea4 py: Define and use MP_OBJ_ITER_BUF_NSLOTS to get size of stack iter buf.
It improves readability of code and reduces the chance to make a mistake.

This patch also fixes a bug with nan-boxing builds by rounding up the
calculation of the new NSLOTS variable, giving the correct number of slots
(being 4) even if mp_obj_t is larger than the native machine size.
2017-03-23 16:36:08 +11:00
Damien George 5255255fb9 py: Create str/bytes objects in the parser, not the compiler.
Previous to this patch any non-interned str/bytes objects would create a
special parse node that held a copy of the str/bytes data.  Then in the
compiler this data would be turned into a str/bytes object.  This actually
lead to 2 copies of the data, one in the parse node and one in the object.
The parse node's copy of the data would be freed at the end of the compile
stage but nevertheless it meant that the peak memory usage of the
parse/compile stage was higher than it needed to be (by an amount equal to
the number of bytes in all the non-interned str/bytes objects).

This patch changes the behaviour so that str/bytes objects are created
directly in the parser and the object stored in a const-object parse node
(which already exists for bignum, float and complex const objects).  This
reduces peak RAM usage of the parse/compile stage, simplifies the parser
and compiler, and reduces code size by about 170 bytes on Thumb2 archs,
and by about 300 bytes on Xtensa archs.
2017-02-24 13:43:43 +11:00
Damien George 534b7c368d py: Do adjacent str/bytes literal concatenation in lexer, not compiler.
It's much more efficient in RAM and code size to do implicit literal string
concatenation in the lexer, as opposed to the compiler.

RAM usage is reduced because the concatenation can be done right away in the
tokeniser by just accumulating the string/bytes literals into the lexer's
vstr.  Prior to this patch adjacent strings/bytes would create a parse tree
(one node per string/bytes) and then in the compiler a whole new chunk of
memory was allocated to store the concatenated string, which used more than
double the memory compared to just accumulating in the lexer.

This patch also significantly reduces code size:

bare-arm: -204
minimal:  -204
unix x64: -328
stmhal:   -208
esp8266:  -284
cc3200:   -224
2017-02-17 12:12:40 +11:00
Damien George 71019ae4f5 py/grammar: Group no-compile grammar rules together to shrink tables.
Grammar rules have 2 variants: ones that are attached to a specific
compile function which is called to compile that grammar node, and ones
that don't have a compile function and are instead just inspected to see
what form they take.

In the compiler there is a table of all grammar rules, with each entry
having a pointer to the associated compile function.  Those rules with no
compile function have a null pointer.  There are 120 such rules, so that's
120 words of essentially wasted code space.

By grouping together the compile vs no-compile rules we can put all the
no-compile rules at the end of the list of rules, and then we don't need
to store the null pointers.  We just have a truncated table and it's
guaranteed that when indexing this table we only index the first half,
the half with populated pointers.

This patch implements such a grouping by having a specific macro for the
compile vs no-compile grammar rules (DEF_RULE vs DEF_RULE_NC).  It saves
around 460 bytes of code on 32-bit archs.
2017-02-16 19:45:06 +11:00
Damien George 4d2bab1444 py/compile: Optimise list/dict/set comprehensions to use stack iter. 2017-02-16 18:38:07 +11:00
Damien George 30b42dd72d py: Remove unused "use_stack" argument from for_iter_end emit function. 2017-02-16 18:38:06 +11:00
Damien George 088740ecc4 py: Optimise storage of iterator so it takes only 4 slots on Py stack. 2017-02-16 18:38:06 +11:00
Damien George 6e769da0da py: Make FOR_ITER opcode pop 1+4 slots from the stack when finished.
The extra 4 slots correspond to the iterator object stored on the stack.
2017-02-16 18:38:06 +11:00
Damien George f4df3aaa72 py: Allow bytecode/native to put iter_buf on stack for simple for loops.
So that the "for x in it: ..." statement can now work without using the
heap (so long as the iterator argument fits in an iter_buf structure).
2017-02-16 18:38:06 +11:00
Damien George de9cd00b39 py/compile: Add an extra pass for Xtensa inline assembler.
It needs an extra pass to compute the size of the constant table for the
l32r instructions.
2016-12-19 17:42:25 +11:00
Damien George e920bab976 py/emitinline: Move common code for end of final pass to compiler.
This patch moves some common code from the individual inline assemblers to
the compiler, the code that calls the emit-glue to assign the machine code
to the functions scope.
2016-12-09 21:23:17 +11:00
Damien George dd53b12193 py/emitinline: Move inline-asm align and data methods to compiler.
These are generic methods that don't depend on the architecture and so
can be handled directly by the compiler.
2016-12-09 20:54:54 +11:00
Damien George f76b1bfa9f py: Add inline Xtensa assembler.
This patch adds the MICROPY_EMIT_INLINE_XTENSA option, which, when
enabled, allows the @micropython.asm_xtensa decorator to be used.

The following opcodes are currently supported (ax is a register, a0-a15):

    ret_n()
    callx0(ax)
    j(label)
    jx(ax)

    beqz(ax, label)
    bnez(ax, label)
    mov(ax, ay)
    movi(ax, imm) # imm can be full 32-bit, uses l32r if needed

    and_(ax, ay, az)
    or_(ax, ay, az)
    xor(ax, ay, az)
    add(ax, ay, az)
    sub(ax, ay, az)
    mull(ax, ay, az)

    l8ui(ax, ay, imm)
    l16ui(ax, ay, imm)
    l32i(ax, ay, imm)
    s8i(ax, ay, imm)
    s16i(ax, ay, imm)
    s32i(ax, ay, imm)
    l16si(ax, ay, imm)
    addi(ax, ay, imm)

    ball(ax, ay, label)
    bany(ax, ay, label)
    bbc(ax, ay, label)
    bbs(ax, ay, label)
    beq(ax, ay, label)
    bge(ax, ay, label)
    bgeu(ax, ay, label)
    blt(ax, ay, label)
    bnall(ax, ay, label)
    bne(ax, ay, label)
    bnone(ax, ay, label)

Upon entry to the assembly function the registers a0, a12, a13, a14 are
pushed to the stack and the stack pointer (a1) decreased by 16.  Upon
exit, these registers and the stack pointer are restored, and ret.n is
executed to return to the caller (caller address is in a0).

Note that the ABI for the Xtensa emitters is non-windowing.
2016-12-09 17:07:38 +11:00
Damien George ad297a1950 py: Allow inline-assembler emitter to be generic.
This patch refactors some code so that it is easier to integrate new
inline assemblers for different architectures other than ARM Thumb.
2016-12-09 17:06:21 +11:00
Damien George 8e5aced1fd py: Integrate Xtensa assembler into native emitter.
The config option MICROPY_EMIT_XTENSA can now be enabled to target the
Xtensa architecture with @micropython.native and @micropython.viper
decorators.
2016-12-09 16:51:49 +11:00
Damien George 080a78b15e py/compile: Simplify configuration of native emitter. 2016-12-07 11:17:17 +11:00
Damien George e6cf5fb2cc py/compile: Remove comment about TODO for short circuiting for if-stmt.
Short circuiting is handled correctly by c_if_cond, and constants within
short-circuit expressions are optimised by the parser.
2016-11-26 16:15:55 +11:00
Damien George ed9c93f0f1 py/parse: Make mp_parse_node_new_leaf an inline function.
It is split into 2 functions, one to make small ints and the other to make
a non-small-int leaf node.  This reduces code size by 32 bytes on
bare-arm, 64 bytes on unix (x64-64) and 144 bytes on stmhal.
2016-11-15 16:48:48 +11:00
Damien George b0cbfb0492 py/parse: Move function to check for const parse node to parse.[ch]. 2016-11-15 16:48:48 +11:00
Damien George deaa57acf3 py/compile: Remove debugging code for compiler dispatch.
It was a relic from the days of developing the compiler and is no longer
needed, and it's impossible to trigger via a test.
2016-10-12 10:20:48 +11:00
Damien George e49153fb98 py/compile: Remove unreachable code. 2016-10-11 12:29:54 +11:00
Damien George 216a711cd4 py/compile: Fix typo when checking for parse-node kind. 2016-09-30 14:48:06 +10:00
Damien George 0d10517a45 py/scope: Factor common code to find locals and close over them.
Saves 50-100 bytes of code.
2016-09-30 13:53:00 +10:00
Damien George 3dea8c9e92 py/scope: Use lookup-table to determine a scope's simple name.
Generates slightly smaller and more efficient code.
2016-09-30 12:34:05 +10:00
Damien George b32c01b748 py/compile: Fix async-for/async-with to work with simpler exc on stack.
There is now just the exception instance on the stack when an exception is
raised, not the full (type, exc, traceback).
2016-09-28 11:52:13 +10:00
Damien George f040685b0c py: Only store the exception instance on Py stack in bytecode try block.
When an exception is raised and is to be handled by the VM, it is stored
on the Python value stack so the bytecode can access it.  CPython stores
3 objects on the stack for each exception: exc type, exc instance and
traceback.  uPy followed this approach, but it turns out not to be
necessary.  Instead, it is enough to store just the exception instance on
the Python value stack.  The only place where the 3 values are needed
explicitly is for the __exit__ handler of a with-statement context, but
for these cases the 3 values can be extracted from the single exception
instance.

This patch removes the need to store 3 values on the stack, and instead
just stores the exception instance.

Code size is reduced by about 50-100 bytes, the compiler and VM are
slightly simpler, generate bytecode is smaller (by 2 bytes for each try
block), and the Python value stack is reduced in size for functions that
handle exceptions.
2016-09-27 12:37:21 +10:00
Damien George a5624bf381 py: Combine 3 comprehension emit functions (list/dict/set) into 1.
The 3 kinds of comprehensions are similar enough that merging their emit
functions reduces code size.  Decreases in code size in bytes are:
bare-arm:24, minimal:96, unix(NDEBUG,x86-64):328, stmhal:80, esp8266:76.
2016-09-19 12:23:31 +10:00
Damien George 24df30c133 py/compile: Don't compile assert statements when optimisations enabled.
As per CPython.
2016-08-26 22:28:22 +10:00
Damien George 3ff16ff52e py: Declare constant data as properly constant.
Otherwise some compilers (eg without optimisation) will put this read-only
data in RAM instead of ROM.
2016-05-20 12:46:20 +01:00
Damien George eacbd7aeba py: Fix constant folding and inline-asm to work with new async grammar. 2016-04-13 15:26:39 +01:00
pohmelie 81ebba7e02 py: add async/await/async for/async with syntax
They are sugar for marking function as generator, "yield from"
and pep492 python "semantically equivalents" respectively.

@dpgeorge was the original author of this patch, but @pohmelie made
changes to implement `async for` and `async with`.
2016-04-13 15:26:38 +01:00
Damien George 2c915e1ae6 py: Implement basic with support in native emitter. 2016-04-07 08:53:24 +01:00
Damien George ce8b4e8749 py: Combine continuous block of emit steps into with_cleanup emit call.
Because different emitters need to handle with-cleanup in different ways.
2016-04-07 08:50:38 +01:00
Damien George 3acaa28b52 py: Don't allocate an extra parse node for power exponent.
Previous to this patch, the "**b" in "a**b" had its own parse node with
just one item (the "b").  Now, the "b" is just the last element of the
power parse-node.  This saves (a tiny bit of) RAM when compiling.
2016-03-16 13:04:51 +00:00
Damien George ea23520403 py: Add MICROPY_DYNAMIC_COMPILER option to config compiler at runtime.
This new compile-time option allows to make the bytecode compiler
configurable at runtime by setting the fields in the mp_dynamic_compiler
structure.  By using this feature, the compiler can generate bytecode
that targets any MicroPython runtime/VM, regardless of the host and
target compile-time settings.

Options so far that fall under this dynamic setting are:
- maximum number of bits that a small int can hold;
- whether caching of lookups is used in the bytecode;
- whether to use unicode strings or not (lexer behaviour differs, and
  therefore generated string constants differ).
2016-02-25 10:05:46 +00:00
Damien George 8f54c08691 py/inlineasm: Add ability to specify return type of asm_thumb funcs.
Supported return types are: object, bool, int, uint.

For example:

@micropython.asm_thumb
def foo(r0, r1) -> uint:
    add(r0, r0, r1)
2016-01-27 14:27:10 +00:00
Damien George 93b3726240 py/parse: Optimise away parse node that's just parenthesis around expr.
Before this patch, (x+y)*z would be parsed to a tree that contained a
redundant identity parse node corresponding to the parenthesis.  With
this patch such nodes are optimised away, which reduces memory
requirements for expressions with parenthesis, and simplifies the
compiler because it doesn't need to handle this identity case.

A parenthesis parse node is still needed for tuples.
2016-01-07 13:07:52 +00:00
Damien George dd5353a405 py: Add MICROPY_ENABLE_COMPILER and MICROPY_PY_BUILTINS_EVAL_EXEC opts.
MICROPY_ENABLE_COMPILER can be used to enable/disable the entire compiler,
which is useful when only loading of pre-compiled bytecode is supported.
It is enabled by default.

MICROPY_PY_BUILTINS_EVAL_EXEC controls support of eval and exec builtin
functions.  By default they are only included if MICROPY_ENABLE_COMPILER
is enabled.

Disabling both options saves about 40k of code size on 32-bit x86.
2015-12-18 12:35:44 +00:00
Damien George a83124361e py/compile: Simplify compilation of comprehension iterators.
Saves 88 bytes on Thumb2, and 200 bytes on x86-64 archs.
2015-12-18 01:37:55 +00:00
Damien George 831137b807 py/compile: Use size_t or uintptr_t instead of mp_uint_t. 2015-12-17 13:13:18 +00:00
Damien George 29e9db0c58 py: Fix compiler to handle lambdas used as default arguments.
Addresses issue #1709.
2015-12-12 13:42:51 +00:00
Damien George 33ac0fd09f py: Don't try to optimise for+range when args are not simple expressions.
Addresses issue #1693.
2015-12-08 21:05:14 +00:00
Damien George b8cfb0d7b2 py: Add support for 64-bit NaN-boxing object model, on 32-bit machine.
To use, put the following in mpconfigport.h:

    #define MICROPY_OBJ_REPR (MICROPY_OBJ_REPR_D)
    #define MICROPY_FLOAT_IMPL (MICROPY_FLOAT_IMPL_DOUBLE)
    typedef int64_t mp_int_t;
    typedef uint64_t mp_uint_t;
    #define UINT_FMT "%llu"
    #define INT_FMT "%lld"

Currently does not work with native emitter enabled.
2015-11-29 14:25:36 +00:00
Damien George c3f64d9799 py: Change qstr_* functions to use size_t as the type for str len arg. 2015-11-29 14:25:04 +00:00
Damien George 9a56912ad1 py/compile: Do proper checking of * and ** in function definition.
This patch checks that there is only one *, and that ** is last in the
arg list.
2015-11-23 16:50:42 +00:00
Damien George d4dba88236 py/compile: Add mp_compile_to_raw_code() to return raw code object.
This can then be passed to mp_raw_code_save_file to save a .mpy file.
2015-11-20 12:30:37 +00:00
Damien George 2c83894257 py: Implement default and star args for lambdas. 2015-11-17 14:00:14 +00:00
Damien George cbd9ae5256 py/compile: Don't unnecessarily save state when compiling param list.
Parameter lists can't be nested so there is no need to save the global
state when compiling them.
2015-11-17 12:37:02 +00:00
Damien George 3a3db4dcf0 py: Put all bytecode state (arg count, etc) in bytecode. 2015-11-13 12:49:18 +00:00
Damien George d7e3b36a09 py/compile: Remove unnecessary label in compilation of for statement. 2015-10-14 15:51:12 +01:00
Damien George 64f2b213bb py: Move constant folding from compiler to parser.
It makes much more sense to do constant folding in the parser while the
parse tree is being built.  This eliminates the need to create parse
nodes that will just be folded away.  The code is slightly simpler and a
bit smaller as well.

Constant folding now has a configuration option,
MICROPY_COMP_CONST_FOLDING, which is enabled by default.
2015-10-12 12:58:45 +01:00
Damien George b948de36fb py: Don't generate unnecessary parse nodes for assignment or kwargs.
This patch eliminates the need for a nested parse node for assignments
and keyword arguments.  It saves a little bit of RAM when parsing.
2015-10-08 14:26:01 +01:00
Damien George 7e12a601b8 py/compile: Fix edge case when constant-folding negation of integer.
Also adds tests specifically for testing constant folding.
2015-10-08 13:02:00 +01:00
Damien George 0496de26d3 py: Allow to enable inline assembler without native emitter. 2015-10-03 17:07:54 +01:00
Damien George 58e0f4ac50 py: Allocate parse nodes in chunks to reduce fragmentation and RAM use.
With this patch parse nodes are allocated sequentially in chunks.  This
reduces fragmentation of the heap and prevents waste at the end of
individually allocated parse nodes.

Saves roughly 20% of RAM during parse stage.
2015-10-02 00:11:11 +01:00
Damien George e5635f4ab3 py: Catch all cases of integer (big and small) division by zero. 2015-10-01 22:48:48 +01:00
Damien George 9d5e5c08ab py/compile: Put compiler state on the C stack.
It's relatively small (between 44 and 56 bytes) and helps to reduce heap
pressure and fragmentation during compilation.
2015-09-24 13:15:57 +01:00
Damien George fbcaf0ea18 py: Slightly simplify compile and emit of star/double-star arguments.
Saves a few bytes of code space and eliminates need for rot_two
bytecode (hence saving RAM and execution time, by a tiny bit).
2015-09-23 11:47:01 +01:00
Delio Brignoli e6978a4e26 py: Fix call args when a stararg is followed by keyword args. 2015-09-23 11:37:00 +01:00
Damien George 558a016e2c py/compile: Refine SyntaxError for repeated use of global/nonlocal. 2015-09-07 16:55:02 +01:00
Damien George ea5b59bfe6 py/compile: Only compile function annotations if really needed.
Function annotations are only needed when the native emitter is enabled
and when the current scope is emitted in viper mode.  All other times
the annotations can be skipped completely.
2015-09-04 16:44:14 +01:00
Damien George 7f70b60f4d py: Remove unused compile scope flags, and irrelevant flag compute code. 2015-08-17 22:39:03 +01:00
Damien George 65dc960e3b unix-cpy: Remove unix-cpy. It's no longer needed.
unix-cpy was originally written to get semantic equivalent with CPython
without writing functional tests.  When writing the initial
implementation of uPy it was a long way between lexer and functional
tests, so the half-way test was to make sure that the bytecode was
correct.  The idea was that if the uPy bytecode matched CPython 1-1 then
uPy would be proper Python if the bytecodes acted correctly.  And having
matching bytecode meant that it was less likely to miss some deep
subtlety in the Python semantics that would require an architectural
change later on.

But that is all history and it no longer makes sense to retain the
ability to output CPython bytecode, because:

1. It outputs CPython 3.3 compatible bytecode.  CPython's bytecode
changes from version to version, and seems to have changed quite a bit
in 3.5.  There's no point in changing the bytecode output to match
CPython anymore.

2. uPy and CPy do different optimisations to the bytecode which makes it
harder to match.

3. The bytecode tests are not run.  They were never part of Travis and
are not run locally anymore.

4. The EMIT_CPYTHON option needs a lot of extra source code which adds
heaps of noise, especially in compile.c.

5. Now that there is an extensive test suite (which tests functionality)
there is no need to match the bytecode.  Some very subtle behaviour is
tested with the test suite and passing these tests is a much better
way to stay Python-language compliant, rather than trying to match
CPy bytecode.
2015-08-17 12:51:26 +01:00
Damien George cfc4c33801 py/compile: Give more precise line number for compile errors.
Previous to this patch there were some cases where line numbers for
errors were 0 (unknown).  Now the compiler attempts to give a better
line number where possible, in some cases giving the line number of the
closest statement, and other cases the line number of the inner-most
scope of the error (eg the line number of the start of the function).
This helps to give good (and sometimes exact) line numbers for
ViperTypeError exceptions.

This patch also makes sure that the first compile error (eg SyntaxError)
that is encountered is reported (previously it was the last one that was
reported).
2015-07-29 22:16:01 +00:00
Damien George 84d59c2873 py: For viper compile errors, add traceback with function and filename.
ViperTypeError now includes filename and function name where the error
occurred.  The line number is the line number of the start of the
function definition, which is the best that can be done without a lot
more work.

Partially addresses issue #1381.
2015-07-27 22:20:00 +01:00
Damien George 59fba2d6ea py: Remove mp_load_const_bytes and instead load precreated bytes object.
Previous to this patch each time a bytes object was referenced a new
instance (with the same data) was created.  With this patch a single
bytes object is created in the compiler and is loaded directly at execute
time as a true constant (similar to loading bignum and float objects).
This saves on allocating RAM and means that bytes objects can now be
used when the memory manager is locked (eg in interrupts).

The MP_BC_LOAD_CONST_BYTES bytecode was removed as part of this.

Generated bytecode is slightly larger due to storing a pointer to the
bytes object instead of the qstr identifier.

Code size is reduced by about 60 bytes on Thumb2 architectures.
2015-06-25 14:42:13 +00:00
Damien George c8b60f013b py: Make viper codegen raise proper exception (ViperTypeError) on error.
This fixes a long standing problem that viper code generation gave
terrible error messages, and actually no errors on pyboard where
assertions are disabled.

Now all compile-time errors are raised as proper Python exceptions, and
are of type ViperTypeError.

Addresses issue #940.
2015-04-20 13:29:31 +00:00
Damien George 55fe92bb8f py: Remove old debugging printf's in compile.c. 2015-04-11 21:52:27 +01:00
Damien George 91bc32dc16 py: Provide typedefs for function types instead of writing them inline. 2015-04-09 15:31:53 +00:00
Damien George 4dea922610 py: Adjust some spaces in code style/format, purely for consistency. 2015-04-09 15:29:54 +00:00
Damien George 4112590a60 py, compiler: When just bytecode, make explicit calls instead of table.
When just the bytecode emitter is needed there is no need to have a
dynamic method table for the emitter back-end, and we can instead
directly call the mp_emit_bc_XXX functions.  This gives a significant
reduction in code size and a very slight performance boost for the
compiler.

This patch saves 1160 bytes code on Thumb2 and 972 bytes on x86, when
native emitters are disabled.

Overall savings in code over the last 3 commits are:

bare-arm: 1664 bytes.
minimal:  2136 bytes.
stmhal:    584 bytes (it has native emitter enabled).
cc3200:   1736 bytes.
2015-03-26 16:52:45 +00:00
Damien George a210c774f9 py, compiler: Remove emit_pass1 code, using emit_bc to do its job.
First pass for the compiler is computing the scope (eg if an identifier
is local or not) and originally had an entire table of methods dedicated
to this, most of which did nothing.  With changes from previous commit,
this set of methods can be removed and the methods from the bytecode
emitter used instead, with very little modification -- this is what is
done in this commit.

This factoring has little to no impact on the speed of the compiler
(tested by compiling 3763 Python scripts and timing it).

This factoring reduces code size by about 270-300 bytes on Thumb2 archs,
and 400 bytes on x86.
2015-03-26 16:52:45 +00:00
Damien George 542bd6b4a1 py, compiler: Refactor load/store/delete_id logic to reduce code size.
Saves around 230 bytes on Thumb2 and 750 bytes on x86.
2015-03-26 16:52:45 +00:00
Damien George 44f65c0e2f py: Fix bug in compiler which allowed through illegal augmented assign.
It allowed such things as (a, b) += c.
2015-03-25 23:06:48 +00:00
Damien George aedf583af2 py: Simplify some logic in compiler; add comments about CPython compat. 2015-03-25 22:06:47 +00:00
Damien George 1ef26b35c1 py, extmod: Remove include of unnecessary system headers. 2015-03-14 23:11:25 +00:00
Damien George 42e0c59308 py: Add MICROPY_COMP_{DOUBLE,TRIPLE}_TUPLE_ASSIGN config options.
These allow to fine-tune the compiler to select whether it optimises
tuple assignments of the form a, b = c, d and a, b, c = d, e, f.
Sensible defaults are provided.
2015-03-14 13:11:35 +00:00