Commit Graph

385 Commits

Author SHA1 Message Date
Damien George 5042bce8fb py: Don't automatically intern strings in parser.
This completes non-automatic interning of strings in the parser, so that
doc strings don't take up RAM.  It complicates the parser and compiler,
and bloats stmhal by about 300 bytes.  It's complicated because now
there are 2 kinds of parse-nodes that can be strings: interned leaves
and non-interned structs.
2014-05-25 22:06:06 +01:00
Damien George 58ebde4664 Tidy up some configuration options.
MP_ALLOC_* -> MICROPY_ALLOC_*
MICROPY_PATH_MAX -> MICROPY_ALLOC_PATH_MAX
MICROPY_ENABLE_REPL_HELPERS -> MICROPY_HELPER_REPL
MICROPY_ENABLE_LEXER_UNIX -> MICROPY_HELPER_LEXER_UNIX
MICROPY_EXTRA_* -> MICROPY_PORT_*

See issue #35.
2014-05-21 20:32:59 +01:00
Damien George 96f137b24a py: Rename BYTE_CODE to BYTECODE (this was missed in previous rename). 2014-05-12 22:35:37 +01:00
Damien George 3417bc2f25 py: Rename byte_code to bytecode everywhere.
bytecode is the more widely used.  See issue #590.
2014-05-10 10:36:38 +01:00
Damien George ffae48d750 py, compiler: Add basic support for A=const(123).
You can now do:

    X = const(123)
    Y = const(456 + X)

and the compiler will replace X and Y with their values.

See discussion in issue #266 and issue #573.
2014-05-08 15:58:39 +00:00
Damien George 36db6bcf54 py, compiler: Improve passes; add an extra pass for native emitter. 2014-05-07 17:24:22 +01:00
Damien George ca25c15d56 py, compiler: Start adding support for compile-time constants.
Just a start, no working code yet.  As per issue #573.
2014-05-07 15:42:03 +01:00
Damien George 04b9147e15 Add license header to (almost) all files.
Blanket wide to all .c and .h files.  Some files originating from ST are
difficult to deal with (license wise) so it was left out of those.

Also merged modpyb.h, modos.h, modstm.h and modtime.h in stmhal/.
2014-05-03 23:27:38 +01:00
Damien George 708c073250 py: Add '*' qstr for 'import *'; use blank qstr for comprehension arg. 2014-04-27 19:23:46 +01:00
Damien George 968bf34c4c py: Remove unnecessary LOAD_CONST_ID bytecode.
It's the same as LOAD_CONST_STR.
2014-04-27 19:12:05 +01:00
Damien George 9d181f62dc py: Save some ROM by shortening compiler error messages.
Messages are still explanatory, while taking a little less ROM.
2014-04-27 16:55:27 +01:00
Damien George 2c0842b3c5 py: Change the way function arguments are compiled.
New way uses slightly less ROM and RAM, should be slightly faster, and,
most importantly, allows to catch the error "non-keyword arg following
keyword arg".

Addresses issue #466.
2014-04-27 16:46:51 +01:00
Damien George 2827d62e8b py: Implement keyword-only args.
Implements 'def f(*, a)' and 'def f(*a, b)', but not default
keyword-only args, eg 'def f(*, a=1)'.

Partially addresses issue #524.
2014-04-27 15:50:52 +01:00
Damien George 6d3c5e4301 Add ARRAY_SIZE macro, and use it where possible. 2014-04-26 10:47:29 +01:00
Damien George 1463c1fa82 py: Add MICROPY_ENABLE_DOC_STRING, disabled by default.
Also add a few STATIC's to some compile functions that should have them.

Addresses issue #521.
2014-04-25 23:52:57 +01:00
Damien George e5f8a77db6 py: Add 'align' and 'data' meta-instructions to inline assembler. 2014-04-21 13:33:15 +01:00
Damien George 3558f62fb5 py: Making closures now passes pointer to stack, not a tuple for vars.
Closed over variables are now passed on the stack, instead of creating a
tuple and passing that.  This way memory for the closed over variables
can be allocated within the closure object itself.  See issue #510 for
background.
2014-04-20 17:50:40 +01:00
Damien George 729f7b42d6 py: Merge BINARY_OP_SUBSCR and store_subscr (w/ delete) into subscr.
mp_obj_t->subscr now does load/store/delete.
2014-04-17 22:10:53 +01:00
Damien George df8127a17e py: Remove unique_codes from emitglue.c. Replace with pointers.
Attempt to address issue #386.  unique_code_id's have been removed and
replaced with a pointer to the "raw code" information.  This pointer is
stored in the actual byte code (aligned, so the GC can trace it), so
that raw code (ie byte code, native code and inline assembler) is kept
only for as long as it is needed.  In memory it's now like a tree: the
outer module's byte code points directly to its children's raw code.  So
when the outer code gets freed, if there are no remaining functions that
need the raw code, then the children's code gets freed as well.

This is pretty much like CPython does it, except that CPython stores
indexes in the byte code rather than machine pointers.  These indices
index the per-function constant table in order to find the relevant
code.
2014-04-13 11:04:33 +01:00
Damien George a26dc50968 py: Improve inline assembler; improve compiler constant folding. 2014-04-12 17:54:52 +01:00
Damien George 7b4330191f py, compiler: Fix up creation of default positionals tuple.
With new order of evaluation of defaults, creating the tuple was done in
the wrong spot.
2014-04-12 00:05:49 +01:00
Damien George 8b19db00aa py, compiler: Fix compiling of keyword args following named star. 2014-04-11 23:25:34 +01:00
Damien George 69b89d21b2 py: Change compile order for default positional and keyword args.
This simplifies the compiler a little, since now it can do 1 pass over
a function declaration, to determine default arguments.  I would have
done this originally, but CPython 3.3 somehow had the default keyword
args compiled before the default position args (even though they appear
in the other order in the text of the script), and I thought it was
important to have the same order of execution when evaluating default
arguments.  CPython 3.4 has changed the order to the more obvious one,
so we can also change.
2014-04-11 13:38:30 +00:00
Damien George 0e3329a6b8 py, compiler: Allow lambda's to yield. 2014-04-11 13:10:21 +00:00
Damien George 0288cf020e py: Implement compiling of *-expr within parenthesis. 2014-04-11 11:53:00 +00:00
Damien George 57e99ebc86 py: Add simple way of looking up constants in compiler.
Working towards trying to support compile-time constants (see discussion
in issue #227), this patch allows the compiler to look inside arbitrary
uPy objects at compile time.  The objects to search are given by the
macro MICROPY_EXTRA_CONSTANTS (so they must be constant/ROM objects),
and the constant folding occures on forms base.attr (both base and attr
must be id's).

It works, but it breaks strict CPython compatibility, since the lookup
will succeed even without importing the namespace.
2014-04-10 22:42:11 +01:00
Damien George d66ae18640 py: Simplify stack get/set to become stack adjust in emitters.
Can do this now that the stack size calculation is improved.
2014-04-10 17:28:54 +00:00
Damien George 069a35e3a5 py, compiler: Improve stack depth counting.
Much less of a hack now.  Hopefully it's correct!
2014-04-10 17:22:19 +00:00
Damien George 6f355fd3b9 py: Make labels unsigned ints (converted from int).
Labels should never be negative, and this modified type signature
reflects that.
2014-04-10 14:11:31 +01:00
Damien George 635543c72c py, compiler: Implement compiling of relative imports. 2014-04-10 12:56:52 +01:00
Damien George 2bf7c09222 py: Properly implement deletion of locals and derefs, and detect errors.
Needed to reinstate 2 delete opcodes, to specifically check that a local
is not deleted twice.
2014-04-09 15:26:46 +01:00
Damien George 11d8cd54c9 py, compiler: Turn id_info_t.param into a set of flags.
So we can add more flags.
2014-04-09 14:42:51 +01:00
Damien George b140bff877 py, compile: Simplify initialisation of compiler structure. 2014-04-09 12:54:21 +01:00
Damien George 02a4c05c5f py, compile: Reduce size of compiler structure. 2014-04-09 12:50:58 +01:00
Damien George 922ddd6415 py, compile: Combine have_star_arg, have_dbl_star_arg into star_flags.
Small reduction in ROM, heap and stack usage.
2014-04-09 12:43:17 +01:00
Damien George 78035b995f py, compiler: Clean up and compress scope/compile structures.
Convert int types to uint where sensible, and then to uint8_t or
uint16_t where possible to reduce RAM usage.
2014-04-09 12:27:39 +01:00
Damien George 495d781a36 py: implement UNPACK_EX byte code (for: a, *b, c = d) 2014-04-08 17:51:47 +01:00
Damien George b7ffdcc1c7 py: Improve compiler syntax errors; catch more errors. 2014-04-08 16:41:02 +01:00
Damien George cdd96dff2c py: Implement more features in native emitter.
On x64, native emitter now passes 70 of the tests.
2014-04-06 12:58:40 +01:00
Damien George 65cad12d38 py: Add option to compiler to specify default code emitter.
Also add command line option to unix port to select emitter.
2014-04-06 11:48:15 +01:00
Damien George af27259439 py: Enable optimisation of multiplying 2 small ints in compiler. 2014-04-04 11:21:58 +00:00
Damien George ecf5b77123 py: This time, real proper overflow checking of small int power.
Previous overflow test was inadequate.
2014-04-04 11:13:51 +00:00
Damien George 094d450003 py: Wrap compile_scope_inline_asm in #if; remove comment from misc.h. 2014-04-02 17:31:27 +01:00
Damien George af6edc61bd py: Enable a jump optimisation in the compiler. 2014-04-02 16:12:28 +01:00
Damien George 882b363564 py: Move to Python 3.4.0 compatibility.
Very little has changed.  In Python 3.4 they removed the opcode
STORE_LOCALS, but in Micro Python we only ever used this for CPython
compatibility, so it was a trivial thing to remove.  It also allowed to
clean up some dead code (eg the 0xdeadbeef in class construction), and
now class builders use 1 less stack word.

Python 3.4.0 introduced the LOAD_CLASSDEREF opcode, which I have not
yet understood.  Still, all tests (apart from bytecode test) still pass.
Bytecode tests needs some more attention, but they are not that
important anymore.
2014-04-02 15:56:31 +01:00
Damien George 3ff2d03891 py: Fix bug in optimised for .. range.
Don't store final, failing value to the loop variable.  This fix also
makes for .. range a bit more efficient, as it uses less store/load
pairs for the loop variable.
2014-03-31 18:02:22 +01:00
Damien George e337f1ef5e py: Towards default keyword arguments.
These are default arguments after a bare *.
2014-03-31 15:18:37 +01:00
Damien George 1aa2c10263 Merge branch 'master' of github.com:micropython/micropython 2014-03-31 12:01:09 +01:00
Damien George 3056509e00 py: Rename and reorder parameters in emit_make_function/closure.
In preparation for implementing default keyword arguments.
2014-03-31 11:30:17 +01:00
Paul Sokolovsky 96eec4f8a6 compile: Don't try to constant-fold division by zero.
The way it is, just crashes app. And optimizing to
"raise ZeroDivisionError" is probably too much.
2014-03-31 02:23:57 +03:00
Damien George 804760bfca py: Fix bug in compiler for empty class bases.
Eg class A(): pass would fail an assertion.
2014-03-30 23:06:37 +01:00
Damien George d17926db71 Rename rt_* to mp_*.
Mostly just a global search and replace.  Except rt_is_true which
becomes mp_obj_is_true.

Still would like to tidy up some of the names, but this will do for now.
2014-03-30 13:35:08 +01:00
Paul Sokolovsky 8d9cc2e669 compile: Print error messages on unimplemented relative imports. 2014-03-30 04:16:35 +03:00
Damien George 21a07dc50f Merge pull request #389 from pfalcon/with-statement
With statement implementation
2014-03-29 14:00:03 +00:00
Damien George d1e443d0bc py: Free unique_code slot for outer module.
Partly (very partly!) addresses issue #386.  Most importantly, at the
REPL command line, each invocation does not now lead to increased memory
usage (unless you define a function/lambda).
2014-03-29 11:39:36 +00:00
Paul Sokolovsky 44307d5ef8 vm: Implement "with" statement (SETUP_WITH and WITH_CLEANUP bytecodes). 2014-03-29 04:39:24 +02:00
Damien George 2326d52d20 py: Factor out code from runtime.c to emitglue.c. 2014-03-27 23:26:35 +00:00
Damien George 8dcc0c7924 py: Calculate maximum exception stack size in compiler. 2014-03-27 10:55:21 +00:00
Damien George bdcbf0fcd1 py: Restore CPython compatibility in compiler for closures with def args. 2014-03-26 23:15:35 +00:00
Paul Sokolovsky 2447a5b582 py: Support closures with default args. 2014-03-26 23:17:44 +02:00
Rachel Dowdall 56402796d8 Fixed floor division on mp ints and small ints. Added a floordivide test case. 2014-03-22 20:19:24 +00:00
Rachel Dowdall cde8631f15 Fixed modulo operator on ints and mp ints to agree with python. Added intdivmod.c and tests/basics/modulo.py. 2014-03-22 17:29:27 +00:00
xbe efe3422394 py: Clean up includes.
Remove unnecessary includes. Add includes that improve portability.
2014-03-17 02:43:40 -07:00
Damien George f41fdd05b0 py: Unify syntax error handling in compiler; check defualt arg syntax.
Checks for non-default args following default args, and errors out.
Addresses issue #328.
2014-03-03 23:19:11 +00:00
Damien George 1dc76af7bf py: Remove name of var arg from macros with var args. 2014-02-26 16:57:08 +00:00
Paul Sokolovsky 56e5ef203b parse: Refactor parse node encoding to support full range of small ints.
Based on suggestion by @dpgeorge at
https://github.com/micropython/micropython/pull/313
2014-02-22 16:39:45 +02:00
Paul Sokolovsky bbf0e2fe12 parse: Note that fact that parser's small ints are different than VM small int.
Specifically, VM's small ints are 31 bit, while parser's only 28. There's already
MP_OBJ_FITS_SMALL_INT(), so, for clarity, rename MP_FIT_SMALL_INT() to
MP_PARSE_FITS_SMALL_INT().
2014-02-21 03:27:09 +02:00
Paul Sokolovsky a1aba36feb compile: Add comments ergarding non-implemented relative imports. 2014-02-20 13:22:28 +02:00
Damien George 8725f8f7de py: Pass all scope flags through to runtime. 2014-02-15 19:33:11 +00:00
Paul Sokolovsky 520e2f58a5 Replace global "static" -> "STATIC", to allow "analysis builds". Part 2. 2014-02-12 18:31:30 +02:00
Paul Sokolovsky 2f0b026a44 Clean up handling of function return type annotation. 2014-02-10 02:04:26 +02:00
Damien George bbcd49a496 py: Fix compile of class with keyword arguments in bases. 2014-02-06 20:30:16 +00:00
Damien George 35e2a4e6bb py: Add built-in super. 2014-02-05 00:51:47 +00:00
Damien George 9aa2a527b5 py: Tidy up BINARY_OPs; negation done by special NOT bytecode.
IS_NOT and NOT_IN are now compiled to IS + NOT and IN + NOT, with a new
special NOT bytecode.
2014-02-01 23:04:09 +00:00
Damien George 7e5fb24e3b py: Reduce code size of compiler by a bit. 2014-02-01 22:18:47 +00:00
Damien George cbddb279bb py: Implement break/continue from an exception with finally.
Still todo: break/continue from within the finally block itself.
2014-02-01 20:08:18 +00:00
Paul Sokolovsky 90750029df Implement default function arguments (for Python functions).
TODO: Decide if we really need separate bytecode for creating functions
with default arguments - we would need same for closures, then there're
keywords arguments too. Having all combinations is a small exponential
explosion, likely we need just 2 cases - simplest (no defaults, no kw),
and full - defaults & kw.
2014-02-01 15:38:22 +02:00
Damien George 08d075592f py: Fix bug with LOAD_METHOD; fix int->machine_int_t for small int.
LOAD_METHOD bug was: emitbc did not correctly calculate the amount of
stack usage for a LOAD_METHOD operation.

small int bug was: int was being used to pass small ints, when it should
have been machine_int_t.
2014-01-29 18:58:52 +00:00
Damien George e4b6a079b3 py: Implement 'not' in compiler, and improve rt_is_true. 2014-01-28 23:27:35 +00:00
Damien George 41d02b654e py: Improve freeing of emitters in mp_compile.
There can be multiple emitters allocated during compile (eg byte code
and native).
2014-01-24 22:42:28 +00:00
Paul Sokolovsky f46d87a30d Add support for freeing code emitter objects at the end of compilation. 2014-01-24 16:31:20 +02:00
Paul Sokolovsky fd31358505 mp_compile(): Properly free module_scope and all nested scopes. 2014-01-23 23:16:18 +02:00
Damien George b979122dfb py: Use C99 way of variable macro arguments.
Addresses Issue #207.
2014-01-23 00:34:21 +00:00
Damien George 00208ce194 py: Change macro var args in parser to be C99 compliant. 2014-01-23 00:00:53 +00:00
Damien George 600ae734cf py: Implement break and continue byte codes, and add tests.
Also fixes a bug in the for-in-range optimiser.

I hope to remove break and continue byte codes in the future and just
use jump (if possible).
2014-01-21 23:48:04 +00:00
Damien George 55baff4c9b Revamp qstrs: they now include length and hash.
Can now have null bytes in strings.  Can define ROM qstrs per port using
qstrdefsport.h
2014-01-21 21:40:13 +00:00
Damien George cbd2f7482c py: Add module/function/class name to exceptions.
Exceptions know source file, line and block name.

Also tidy up some debug printing functions and provide a global
flag to enable/disable them.
2014-01-19 11:48:48 +00:00
Damien George 08335004cf Add source file name and line number to error messages.
Byte code has a map from byte-code offset to source-code line number,
used to give better error messages.
2014-01-18 23:24:36 +00:00
Damien George d02c6d8962 Implement eval. 2014-01-15 22:14:03 +00:00
Damien George e5863d9301 py: AssertionError is loaded from global, to match CPython. 2014-01-12 12:35:08 +00:00
John R. Lenton b8698fca75 unified the bops 2014-01-11 00:58:59 +00:00
Paul Sokolovsky 899c69f94c compile_for_stmt_optimised_range(): Properly handle negative & unknown steps.
If step is not constant, in first approximation, we can't apply optimization,
(well, we could, but need a special case for this).
2014-01-11 01:00:21 +02:00
Damien George 71c5181a8d Convert Python types to proper Python type hierarchy.
Now much more inline with how CPython does types.
2014-01-04 20:21:15 +00:00
Damien George eb7bfcb286 Split qstr into pools, and put initial pool in ROM.
Qstr's are now split into a linked-list of qstr pools.  This has 2
benefits: the first pool can be in ROM (huge benefit, since we no longer
use RAM for the core qstrs), and subsequent pools use m_new for the next
pool instead of m_renew (thus avoiding a huge single table for all the
qstrs).

Still would be better to use a hash table, but this scheme takes us part
of the way (eventually convert the pools to hash tables).

Also fixed bug with import.

Also improved the way the module code is referenced (not magic number 1
anymore).
2014-01-04 15:57:35 +00:00
Damien George e67ed5d285 Improve configurability for native x64/thumb emitter.
With MICROPY_EMIT_X64 and MICROPY_EMIT_THUMB disabled, the respective
emitters and assemblers will not be included in the code.  This can
significantly reduce binary size for unix version.
2014-01-04 13:55:24 +00:00
Damien George 1fb031744f Change mp_compile so that it returns a function object for the module. 2014-01-03 14:22:03 +00:00
Damien George fe8fb9165c py: remove depedence on strcat and stpcpy.
This fixes Issue #29, and means the core is no longer dependent on
string functions, except strlen.
2014-01-02 16:36:09 +00:00
Damien George 6baf76e28b py: make closures work. 2013-12-30 22:32:17 +00:00
Damien 732407f1bf Change memory allocation API to require size for free and realloc. 2013-12-29 19:33:23 +00:00
Damien d99b05282d Change object representation from 1 big union to individual structs.
A big change.  Micro Python objects are allocated as individual structs
with the first element being a pointer to the type information (which
is itself an object).  This scheme follows CPython.  Much more flexible,
not necessarily slower, uses same heap memory, and can allocate objects
statically.

Also change name prefix, from py_ to mp_ (mp for Micro Python).
2013-12-21 18:17:45 +00:00
Damien e2880aa2fd Fix a few compiler warnings. 2013-12-20 14:22:59 +00:00
Damien a1b2693161 py: remove further unnecessary emit_verbatim code. 2013-12-12 15:34:40 +00:00
Damien e388f1034e py: fix bug with doc string not recognised after first newline of file. 2013-12-12 15:24:38 +00:00
Damien 02f8941bf6 py: reduce use of emit_verbatim calls to minimum. 2013-12-12 15:13:36 +00:00
Damien 9ecbcfff99 py: work towards working closures. 2013-12-11 00:41:43 +00:00
Damien 318aec6ba9 py: emit correct id for closed over variables. 2013-12-10 18:28:17 +00:00
Damien 6332174ab4 py: compiler supports string juxtaposition=concatenation. 2013-12-10 17:41:49 +00:00
Damien db4c361f1c py: add skeletal import functionality. 2013-12-10 17:27:24 +00:00
Damien d79338969b py: restrict further when for-range optimisation is done. 2013-11-28 19:12:18 +00:00
Damien 0446a0d76d Change some debugging/output messages for native code generation. 2013-11-17 13:16:36 +00:00
Damien f3822fc34c Fix but with optimised range being 1 over. 2013-11-09 20:12:03 +00:00
Damien f72fd0e875 Add optimisation for "for x in range". 2013-11-06 20:20:49 +00:00
Damien 27fb45eb1c Add local_num skeleton framework to deref/closure emit calls. 2013-10-20 15:07:49 +01:00
Damien 5ac1b2efbd Implement REPL. 2013-10-18 19:58:12 +01:00
Damien ce89a21ea4 Implement basic exception framework, and simple for loop. 2013-10-15 22:25:17 +01:00
Damien 3ef4abb446 Change ifdef/if defined to simple if's. 2013-10-12 16:53:13 +01:00
Damien 0efb3a1b66 Tidy up SMALL_INT optimisations and CPython compatibility. 2013-10-12 16:16:56 +01:00
Damien 3a205179ea Option in compile.c to emit compatible or not with CPython. 2013-10-12 15:01:56 +01:00
Damien c025ebb2dc Separate out mpy core and unix version. 2013-10-12 14:30:21 +01:00
Damien 91d387de7d Improve indent/dedent error checking and reporting. 2013-10-09 15:09:52 +01:00
Damien 13ed3a658d Native Python and Viper support for x64 and thumb all together. 2013-10-08 09:05:10 +01:00
Damien 3410be8035 Merge viper types with standard native emitter. 2013-10-07 23:09:10 +01:00
Damien 7af3d19a3c Implement crude viper emit stage. 2013-10-07 00:02:49 +01:00
Damien dc83382903 Make runtime able to call inline asm with 1 argument. 2013-10-06 01:01:01 +01:00
Damien b14de21fc8 Optimise typedargslist_name to not create a node if just an id. 2013-10-06 00:28:28 +01:00
Damien a2f2f7db1f Almost supports arguments for inline asm functions. 2013-10-06 00:14:13 +01:00
Damien 826005c60b Add support for inline thumb assembly. 2013-10-05 23:17:28 +01:00
Damien 5bfb759980 Incorporate emit_thumb into new emit framework. 2013-10-05 18:41:24 +01:00
Damien 6cdd3af601 Implement built-in decorators to select emit type. 2013-10-05 18:08:26 +01:00
Damien 4b03e77d4a Factorise EMIT_COMMON calls, mostly into emit_pass1. 2013-10-05 14:17:09 +01:00
Damien 054848a1b8 Compiler computes labels and max_num_labels. 2013-10-05 13:44:41 +01:00
Damien b05d707b23 Further factorise PASS_1 out of specific emit code. 2013-10-05 13:37:10 +01:00
Damien 415eb6f850 Restructure emit so it goes through a method table. 2013-10-05 12:19:06 +01:00
Damien 492d082455 Use macro EMIT_COMMON for emit_common calls. 2013-10-04 20:35:08 +01:00
Damien 429d71943d Initial commit. 2013-10-04 19:53:11 +01:00