Commit Graph

2880 Commits

Author SHA1 Message Date
Damien George 9f85c4fe48 py/objstr: Catch case of negative "maxsplit" arg to str.rsplit().
Negative values mean no limit on the number of splits so should delegate to
the .split() method.
2017-06-02 13:07:22 +10:00
Damien George bc76302eab py/modbuiltins: Add core-provided version of input() function.
The implementation is taken from stmhal/input.c, with code added to handle
ctrl-C.  This built-in is controlled by MICROPY_PY_BUILTINS_INPUT and is
disabled by default.  It uses readline() to capture input but this can be
overridden by defining the mp_hal_readline macro.
2017-06-01 16:02:49 +10:00
Ville Skyttä ca16c38210 various: Spelling fixes 2017-05-29 11:36:05 +03:00
Tom Collins e26fb3ad73 py/objstringio: Catch mp_uint_t overflow of stream position in write(). 2017-05-26 13:40:08 +10:00
Damien George 85f7b0b468 py/mkrules.mk: Fix auto-qstr generation when "make -B" is used.
For make v3.81, using "make -B" can set $? to empty and in this case the
auto-qstr generation needs to pass all args (ie $^) to cpp.  The previous
fix for this (which was removed in 23a693ec2d)
used if statements in the shell command, which gave very long lines that
didn't work on certain systems (eg cygwin).

The fix in this patch is to use an $if(...) expression, which will evaluate
to $? (only newer prerequisites) if it's non empty, otherwise it will use
$^ (all prerequisites).
2017-05-26 13:12:42 +10:00
Damien George 8f064e469d py/emitbc: Fix bug with BC emitter computing Python stack size.
Previous to this patch the mp_emit_bc_adjust_stack_size function would
adjust the current stack size but would not increase the maximum stack size
if the current size went above it.  This meant that certain Python code
(eg a try-finally block with no statements inside it) would not have enough
Python stack allocated to it.

This patch fixes the problem by always checking if the current stack size
goes above the maximum, and adjusting the latter if it does.
2017-05-25 20:42:30 +10:00
Damien George 04d05db27e py/vm: Fix bug with unwind jump popping the iterator from a for loop.
This patch fixes a regression introduced by
088740ecc4
2017-05-25 20:39:08 +10:00
Damien George 68e71eacb8 py/vm: Fix bug with stackless mode and unwinding of exceptions.
This patch fixes a regression introduced by
71a3d6ec3b

Previous to this patch the n_state variable was referring to that computed
at the very start of the mp_execute_bytecode function.  This patch fixes it
so that n_state is recomputed when the code_state changes.
2017-05-25 20:35:57 +10:00
Damien George 338f0849d9 py/mkenv.mk: Use $(TOP) instead of ".." to reference tools, mpy-cross. 2017-05-19 15:53:55 +10:00
Damien George 03659c51ca py/objrange: Fix slicing of range when step of slice is negative. 2017-05-18 17:32:42 +10:00
Damien George eb4c37f7a4 py/sequence: Fix boundary errors when slicing with a negative step. 2017-05-18 17:32:42 +10:00
Tom Collins 53461deb04 py/objstringio: Fix StringIO reads at or beyond EOF.
Existing code failed if seek() went past EOF (which is acceptable when writing).
2017-05-15 23:58:04 +03:00
Tom Collins f06d0839bd py/modsys: update conditionals for code referencing sys.stdout
Working on a build with PY_IO enabled (for PY_UJSON support) but PY_SYS_STDFILES disabled (no filesystem).  There are multiple references to mp_sys_stdout_obj that should only be enabled if both PY_IO and PY_SYS_STDFILES are enabled.
2017-05-14 18:24:50 +03:00
Tom Collins 6f56412ec3 py/lexer: Process CR earlier to allow newlines checks on chr1.
Resolves an issue where lexer failed to accept CR after line continuation
character.  It also simplifies the code.
2017-05-12 15:14:24 +10:00
Damien George a1f254028d py/mkrules.mk: Add dependency of .mpy files upon mpy-cross.
This ensures that mpy-cross is automatically built (and is up-to-date) for
ports that use frozen bytecode.  It also makes sure that .mpy files are
re-built if mpy-cross is changed.
2017-05-11 23:40:16 +10:00
Paul Sokolovsky edc02bd952 unix/main: Implement -m option for packages. 2017-05-09 14:22:21 +03:00
Tom Collins 2998647c4e py/lexer: Simplify lexer startup by using dummy bytes and next_char().
Now consistently uses the EOL processing ("\r" and "\r\n" convert to "\n")
and EOF processing (ensure "\n" before EOF) provided by next_char().

In particular the lexer can now correctly handle input that starts with CR.
2017-05-09 14:43:23 +10:00
Damien George 6cfa61a4cc py/binary: Handle storing big-ints to all arrays types.
Prior to this patch only 'q' and 'Q' type arrays could store big-int
values.  With this patch any big int that is stored to an array is handled
by the big-int implementation, regardless of the typecode of the array.
This allows arrays to work with all type sizes on all architectures.
2017-05-09 10:41:00 +10:00
Paul Sokolovsky 4a4490ffcc py/modio: resource_stream: Implement "package" param handling. 2017-05-06 18:42:35 +03:00
Damien George 58bb73e010 py/objint: In int.from_bytes, only create big-int if really needed.
This patch ensures that int.from_bytes only creates a big-int if necessary,
by checking the value for a small-int overflow as it's being parsed.
2017-05-06 10:29:09 +10:00
Paul Sokolovsky d7da2dba07 py/modio: Implement uio.resource_stream(package, resource_path).
The with semantics of this function is close to
pkg_resources.resource_stream() function from setuptools, which
is the canonical way to access non-source files belonging to a package
(resources), regardless of what medium the package uses (e.g. individual
source files vs zip archive). In the case of MicroPython, this function
allows to access resources which are frozen into the executable, besides
accessing resources in the file system.

This is initial stage of the implementation, which actually doesn't
implement "package" part of the semantics, just accesses frozen resources
from "root", or filesystem resource - from current dir.
2017-05-03 01:47:08 +03:00
stijn 2f0ce2a6f5 py: Cleanup use of global DEBUG preprocessor definition
The standard preprocessor definition to differentiate debug and non-debug
builds is NDEBUG, not DEBUG, so don't rely on the latter:
- just delete the use of it in objint_longlong.c as it has been stale code
  for years anyway (since commit [c4029e5]): SUFFIX isn't used anywhere.
- replace DEBUG with MICROPY_DEBUG_NLR in nlr.h: it is rarely used anymore
  so can be off by default
2017-04-30 14:28:37 +03:00
Damien George f85fd79c6c py/mpz: In mpn_sub, use existing function to remove trailing zeros. 2017-04-25 12:22:04 +10:00
Damien George c7aa86ce6f py/mpz: Strip trailing zeros from mpz value when set from bytes. 2017-04-25 12:06:10 +10:00
Damien George dd11af209d py: Add LOAD_SUPER_METHOD bytecode to allow heap-free super meth calls.
This patch allows the following code to run without allocating on the heap:

    super().foo(...)

Before this patch such a call would allocate a super object on the heap and
then load the foo method and call it right away.  The super object is only
needed to perform the lookup of the method and not needed after that.  This
patch makes an optimisation to allocate the super object on the C stack and
discard it right after use.

Changes in code size due to this patch are:

   bare-arm: +128
    minimal: +232
   unix x64: +416
unix nanbox: +364
     stmhal: +184
    esp8266: +340
     cc3200: +128
2017-04-22 23:39:20 +10:00
Damien George 5335942b59 py/compile: Refactor handling of special super() call.
This patch refactors the handling of the special super() call within the
compiler.  It removes the need for a global (to the compiler) state variable
which keeps track of whether the subject of an expression is super.  The
handling of super() is now done entirely within one function, which makes
the compiler a bit cleaner and allows to easily add more optimisations to
super calls.

Changes to the code size are:

   bare-arm: +12
    minimal:  +0
   unix x64: +48
unix nanbox: -16
     stmhal:  +4
     cc3200:  +0
    esp8266: -56
2017-04-22 21:46:32 +10:00
Damien George 0dd6a59c89 py/compile: Don't do unnecessary check if iter parse node is a struct.
If we get to this point in the code then pn_iter is guaranteed to be a
struct.
2017-04-22 21:43:42 +10:00
Damien George ae54fbf166 py/compile: Add COMP_RETURN_IF_EXPR option to enable return-if-else opt.
With this optimisation enabled the compiler optimises the if-else
expression within a return statement.  The optimisation reduces bytecode
size by 2 bytes for each use of such a return-if-else statement.  Since
such a statement is not often used, and costs bytes for the code, the
feature is disabled by default.

For example the following code:

    def f(x):
        return 1 if x else 2

compiles to this bytecode with the optimisation disabled (left column is
bytecode offset in bytes):

    00 LOAD_FAST 0
    01 POP_JUMP_IF_FALSE 8
    04 LOAD_CONST_SMALL_INT 1
    05 JUMP 9
    08 LOAD_CONST_SMALL_INT 2
    09 RETURN_VALUE

and to this bytecode with the optimisation enabled:

    00 LOAD_FAST 0
    01 POP_JUMP_IF_FALSE 6
    04 LOAD_CONST_SMALL_INT 1
    05 RETURN_VALUE
    06 LOAD_CONST_SMALL_INT 2
    07 RETURN_VALUE

So the JUMP to RETURN_VALUE is optimised and replaced by RETURN_VALUE,
saving 2 bytes and making the code a bit faster.
2017-04-22 14:58:01 +10:00
Damien George 40b40ffc98 py/compile: Extract parse-node kind at start of func for efficiency.
Otherwise the type of parse-node and its kind has to be re-extracted
multiple times.  This optimisation reduces code size by a bit (16 bytes on
bare-arm).
2017-04-22 14:23:47 +10:00
Damien George fa03bbf0fd py/compile: Don't do unnecessary check if parse node is a struct.
PN_atom_expr_normal parse nodes always have structs for their second
sub-node, so simplify the check for the sub-node kind to save code size.
2017-04-22 14:13:37 +10:00
Damien George 4df013c8cc py/objtype: mp_obj_new_super doesn't need to be public, so inline it.
Saves code size (20 bytes on bare-arm) and makes it a tiny bit more
efficient.
2017-04-22 12:14:04 +10:00
Paul Sokolovsky 9e8f316392 extmod/moductypes: Fix bigint handling for 32-bit ports. 2017-04-21 16:43:21 +03:00
Damien George 7a72c0db5a py: Reduce str/repr precision of float numbers when floats are 30-bit.
With 30-bit floats there aren't enough bits to faithfully print 7 decimal
digits, so reduce the precision to 6 digits.
2017-04-21 16:21:56 +10:00
Damien George bbb4b9822f py/modmicropython: Add micropython.kbd_intr() function.
It controls the character that's used to (asynchronously) raise a
KeyboardInterrupt exception.  Passing "-1" allows to disable the
interception of the interrupt character (as long as a port allows such a
behaviour).
2017-04-18 17:24:30 +10:00
Damien George c7e8c6f7de py/gc: Execute finaliser code in a protected environment.
If a finaliser raises an exception then it must not propagate through the
GC sweep function.  This patch protects against such a thing by running
finaliser code via the mp_call_function_1_protected call.

This patch also adds scheduler lock/unlock calls around the finaliser
execution to further protect against any possible reentrancy issues: the
memory manager is already locked when doing a collection, but we also don't
want to allow any scheduled code to run, KeyboardInterrupts to interupt the
code, nor threads to switch.
2017-04-12 13:52:04 +10:00
Damien George 08242eed26 py/nlrsetjmp: Add check for failed NLR jump.
Also optimise the function so it only needs to call the MP_STATE_THREAD
macro once (following how other nlr code is written).
2017-04-12 13:50:31 +10:00
Damien George a73501b1d6 py/objfloat: Add implementation of high-quality float hashing.
Disabled by default.
2017-04-12 13:38:17 +10:00
Damien George 816413e4b2 py: Optimise types for common case where type has a single parent type.
The common cases for inheritance are 0 or 1 parent types, for both built-in
types (eg built-in exceptions) as well as user defined types.  So it makes
sense to optimise the case of 1 parent type by storing just the type and
not a tuple of 1 value (that value being the single parent type).

This patch makes such an optimisation.  Even though there is a bit more
code to handle the two cases (either a single type or a tuple with 2 or
more values) it helps reduce overall code size because it eliminates the
need to create a static tuple to hold single parents (eg for the built-in
exceptions).  It also helps reduce RAM usage for user defined types that
only derive from a single parent.

Changes in code size (in bytes) due to this patch:

    bare-arm:       -16
    minimal (x86): -176
    unix (x86-64): -320
    unix nanbox:   -384
    stmhal:         -64
    cc3200:         -32
    esp8266:       -108
2017-04-12 13:22:21 +10:00
Damien George fc710169b7 py/obj: Clean up and add comments describing mp_obj_type_t struct. 2017-04-12 13:20:26 +10:00
Damien George e31fbd9b41 py/objint: Use unsigned arithmetic when formatting an integer.
Otherwise the edge case of the most negative integer value will not convert
correctly.
2017-04-11 15:18:35 +10:00
Damien George f66df1efc8 py/objint: Extract small int value directly because type is known. 2017-04-11 15:16:09 +10:00
Damien George 79ce664952 py/runtime: When init'ing kbd intr exc, use tuple ptr instead of object. 2017-04-10 17:07:26 +10:00
Damien George ee86de1f1a py: Make sure that static emg-exc-buffer is aligned to size of mp_obj_t.
This buffer is used to allocate objects temporarily, and such objects
require that their underlying memory be correctly aligned for their data
type.  Aligning for mp_obj_t should be sufficient for emergency exceptions,
but in general the memory buffer should aligned to the maximum alignment of
the machine (eg on a 32-bit machine with mp_obj_t being 4 bytes, a double
may not be correctly aligned).

This patch fixes a bug for certain nan-boxing builds, where mp_obj_t is 8
bytes and must be aligned to 8 bytes (even though the machine is 32 bit).
2017-04-10 16:02:56 +10:00
Damien George 81d2ca2b12 py/objtuple: Add support for inplace add (same as normal add). 2017-04-05 12:38:02 +10:00
Damien George de9b53695d py: Raise a ValueError if range() step is zero.
Following CPython.  Otherwise one gets either an infinite loop (if code is
optimised by the uPy compiler) or possibly a divide-by-zero CPU exception.
2017-04-05 10:50:26 +10:00
Damien George fc245d1ca4 py/objint: Consolidate mp_obj_new_int_from_float to one implementation.
This reduces code duplication and allows to make mp_classify_fp_as_int
static, which reduces code size.
2017-04-04 16:45:49 +10:00
Damien George 19f2e47d59 py: Add very simple but correct hashing for float and complex numbers.
Hashing of float and complex numbers that are exact (real) integers should
return the same integer hash value as hashing the corresponding integer
value.  Eg hash(1), hash(1.0) and hash(1+0j) should all be the same (this
is how Python is specified: if x==y then hash(x)==hash(y)).

This patch implements the simplest way of doing float/complex hashing by
just converting the value to int and returning that value.
2017-04-04 11:57:21 +10:00
Paul Sokolovsky 9a973977bb py/objstr: Use MICROPY_FULL_CHECKS for range checking when constructing bytes.
Split this setting from MICROPY_CPYTHON_COMPAT. The idea is to be able to
keep MICROPY_CPYTHON_COMPAT disabled, but still pass more of regression
testsuite. In particular, this fixes last failing test in basics/ for
Zephyr port.
2017-04-02 21:20:07 +03:00
Damien George a5500a8aad py/obj.h: Make sequence grow more efficient and support overlapping.
The first memmove now copies less bytes in some cases (because len_adj <=
slice_len), and the memcpy is replaced with memmove to support the
possibility that dest and slice regions are overlapping.
2017-04-02 17:28:24 +10:00
Damien George 4c307bfba1 all: Move BYTES_PER_WORD definition from ports to py/mpconfig.h
It can still be overwritten by a port in mpconfigport.h but for almost
all cases one can use the provided default.
2017-04-01 11:39:38 +11:00