Commit Graph

2769 Commits

Author SHA1 Message Date
Damien George 4b8ec5256d py/objfloat: Raise ZeroDivisionError for 0 to negative power. 2017-02-03 00:01:37 +11:00
Damien George 30cf503e97 py/objset: Make inplace binary operators actually modify the set. 2017-02-02 23:33:49 +11:00
Paul Sokolovsky 50d3a9df67 py/objstringio: Allow to specify initial capacity by passing numeric argument.
E.g. uio.BytesIO(100) will allocate buffer with 100 bytes of space.
2017-02-02 00:33:43 +03:00
Dave Hylands aee74a1dae unix: Make stack be non-executable
This PR is to address issue #2812.
2017-02-01 23:03:10 +03:00
Damien George 1808b2e8d5 extmod: Remove MICROPY_FSUSERMOUNT and related files.
Replaced by MICROPY_VFS and the VFS sub-system.
2017-01-30 12:26:07 +11:00
Damien George 8beba7310f extmod/vfs_fat: Remove MICROPY_READER_FATFS component. 2017-01-30 12:26:07 +11:00
Paul Sokolovsky 7a7516d40d extmod/machine_signal: Implement "signal" abstraction for machine module.
A signal is like a pin, but ca also be inverted (active low). As such, it
abstracts properties of various physical devices, like LEDs, buttons,
relays, buzzers, etc. To instantiate a Signal:

pin = machine.Pin(...)
signal = machine.Signal(pin, inverted=True)

signal has the same .value() and __call__() methods as a pin.
2017-01-29 18:57:36 +03:00
Damien George 3f6b4e08e3 extmod/vfs: Expose mp_vfs_mount_t type.
It should only be used for low-level things and with caution, for example
putting mounted VFS data in ROM or the static data section.
2017-01-27 23:22:15 +11:00
Damien George dcb9ea7215 extmod: Add generic VFS sub-system.
This provides mp_vfs_XXX functions (eg mount, open, listdir) which are
agnostic to the underlying filesystem type, and just require an object with
the relevant filesystem-like methods (eg .mount, .open, .listidr) which can
then be mounted.

These mp_vfs_XXX functions would typically be used by a port to implement
the "uos" module, and mp_vfs_open would be the builtin open function.

This feature is controlled by MICROPY_VFS, disabled by default.
2017-01-27 17:19:06 +11:00
Damien George d4464b0050 py/py.mk: Add CFLAGS_MOD flag to set config file for FatFs. 2017-01-27 13:19:10 +11:00
Damien George cc4c1adf6e py/showbc: Make sure to set the const_table before printing bytecode. 2017-01-27 12:34:09 +11:00
Paul Sokolovsky e2e663291d py/objstr: Optimize string concatenation with empty string.
In this, don't allocate copy, just return non-empty string. This helps
with a standard pattern of buffering data in case of short reads:

    buf = b""
    while ...:
        s = f.read(...)
        buf += s
        ...

For a typical case when single read returns all data needed, there won't
be extra allocation. This optimization helps uasyncio.
2017-01-27 00:49:39 +03:00
Damien George e9cb1f8077 py/objmodule: Move module init/deinit code into runtime functions.
They are one-line functions and having them inline in mp_init/mp_deinit
eliminates the overhead of a function call, and matches how other state
is initialised in mp_init.
2017-01-26 23:30:38 +11:00
Damien George eaa77455c3 py/objint: Fix left-shift overflow in checking for large int. 2017-01-25 14:39:13 +11:00
Damien George f5172af1c4 py/builtinhelp: Implement help('modules') to list available modules.
This is how CPython does it, and it's very useful to help users discover
the available modules for a given port, especially built-in and frozen
modules.  The function does not list modules that are in the filesystem
because this would require a fair bit of work to do correctly, and is very
port specific (depending on the filesystem).
2017-01-22 12:12:54 +11:00
Damien George 9de91914fb py: Move weak-link map to objmodule.c, and expose module maps as public. 2017-01-22 11:59:29 +11:00
Damien George 9f04dfb568 py: Add builtin help function to core, with default help msg.
This builtin is configured using MICROPY_PY_BUILTINS_HELP, and is disabled
by default.
2017-01-22 11:56:16 +11:00
Paul Sokolovsky e6ab43e2c0 py/objint_longlong: Add stub for mp_obj_int_from_bytes_impl().
To be implemented later.
2017-01-21 20:15:05 +03:00
Paul Sokolovsky bec7bfb29d py/objint: from_bytes(): Implement "byteorder" param and arbitrary precision.
If result guaranteedly fits in a small int, it is handled in objint.c.
Otherwise, it is delegated to mp_obj_int_from_bytes_impl(), which should
be implemented by individual objint_*.c, similar to
mp_obj_int_to_bytes_impl().
2017-01-21 20:14:18 +03:00
Paul Sokolovsky 1b42f5251f py/mpz: Implement mpz_set_from_bytes() as a foundation for int.from_bytes(). 2017-01-21 20:07:50 +03:00
Damien George e873243aa3 py/objint_mpz: Refactor switch-statement to remove unreachable default. 2017-01-19 23:35:45 +11:00
Damien George 94a587a750 py/formatfloat: Remove unreachable code.
The if-block that this unreachable code is in has a condition "f>=5" so
"fp_isless1(f)" will always fail.
2017-01-19 23:32:16 +11:00
Paul Sokolovsky af90461931 py/binary: mp_binary_get_size: Raise error on unsupported typecodes.
Previouly, we had errors checked in callers, which led to duplicate code
or missing checks in some places.
2017-01-17 22:53:06 +03:00
Damien George d7150b09d7 py/runtime: Refactor default case of switch to remove assert(0). 2017-01-17 17:03:56 +11:00
Damien George 5bea080737 py/objexcept: Replace if-cond and assert(0) with simple assert. 2017-01-17 17:03:16 +11:00
Damien George f51f22dd42 py/emitnative: Remove assert(0)'s or replace with mp_not_implemented. 2017-01-17 17:02:21 +11:00
Damien George 86e942309a py/parse: Refactor code to remove assert(0)'s.
This helps to improve code coverage.  Note that most of the changes in
this patch are just de-denting the cases of the switch statements.
2017-01-17 17:00:55 +11:00
Damien George 239f920299 py/objgenerator: Don't raise RuntimeError if GeneratorExit ignored.
In this case it's allowed to be ignored.
2017-01-17 00:16:56 +11:00
Damien George 681994638b py/objgenerator: When throwing an object, don't make an exc instance.
Arguments to throw() for generators don't need to be exceptions.
2017-01-17 00:14:14 +11:00
Damien George aeb2655073 py/runtime: Fix handling of throw() when resuming generator.
If GeneratorExit is injected as a throw-value then that should lead to
the close() method being called, if it exists.  If close() does not exist
then throw() should not be called, and this patch fixes this.
2017-01-17 00:10:49 +11:00
Damien George 40863fce6f py/runtime: Refactor assert(0) to improve coverage. 2017-01-17 00:09:56 +11:00
Damien George d23834bc96 py/builtinimport: Remove unreachable code and change obj-import comment. 2017-01-16 16:41:43 +11:00
Damien George 63e291de70 py/builtinimport: Raise ValueError for bad relative import, per CPython. 2017-01-16 16:21:04 +11:00
Damien George b528e9a428 py/builtinimport: Fix bug when importing names from frozen packages.
The commit d9047d3c8a introduced a bug
whereby "from a.b import c" stopped working for frozen packages.  This is
because the path was not properly truncated and became "a//b".  Such a
path resolves correctly for a "real" filesystem, but not for a search in
the list of frozen modules.
2017-01-08 20:17:23 +11:00
Damien George 343b4189b0 py/mkrules.mk: Add MPY_CROSS_FLAGS option to pass flags to mpy-cross.
So that ports can pass their own custom options to mpy-cross.
2017-01-05 15:51:36 +11:00
Damien George 3f9c45efd1 py/asmarm: Fix assembler's PASS_EMIT constant name. 2017-01-03 15:40:50 +11:00
Damien George afc5063539 py/unicode: Comment-out unused function unichar_isprint. 2016-12-28 17:50:10 +11:00
Damien George ea6a958393 py/objint: Simplify mp_int_format_size and remove unreachable code.
One never needs to format integers with a base larger than 16 (but code
can be easily extended beyond this value if needed in the future).
2016-12-28 12:46:20 +11:00
Damien George 44bf8e1f2b py/mpprint: Add assertion for, and comment about, valid base values. 2016-12-28 12:45:33 +11:00
Damien George ca7af9a778 py/parsenum: Fix warning for signed/unsigned comparison. 2016-12-28 12:25:00 +11:00
Damien George 2d9440e2d1 py/mpz: Fix assertion in mpz_set_from_str which checks value of base. 2016-12-28 12:04:19 +11:00
Damien George c2dd494bd9 py/parsenum: Simplify and generalise decoding of digit values.
This function should be able to parse integers with any value for the
base, because it is called by int('xxx', base).
2016-12-28 12:02:49 +11:00
Paul Sokolovsky 25f44c19f1 cc3200: Re-add support for UART REPL (MICROPY_STDIO_UART setting).
UART REPL support was lost in os.dupterm() refactorings, etc. As
os.dupterm() is there, implement UART REPL support at the high level -
if MICROPY_STDIO_UART is set, make default boot.py contain os.dupterm()
call for a UART. This means that changing MICROPY_STDIO_UART value will
also require erasing flash on a module to force boot.py re-creation.
2016-12-27 01:05:37 +03:00
Paul Sokolovsky cf96be60dc py/misc.h: Typo fix in comment. 2016-12-27 01:05:30 +03:00
Damien George c305ae3243 py/lexer: Permanently disable the mp_lexer_show_token function.
The lexer is very mature and this debug function is no longer used.  If
it's really needed one can uncomment it and recompile.
2016-12-22 10:49:54 +11:00
Damien George f4aebafe7a py/lexer: Remove unnecessary check for EOF in lexer's next_char func.
This check always fails (ie chr0 is never EOF) because the callers of this
function never call it past the end of the input stream.  And even if they
did it would be harmless because 1) reader.readbyte must continue to
return an EOF char if the stream is exhausted; 2) next_char would just
count the subsequent EOF's as characters worth 1 column.
2016-12-22 10:39:06 +11:00
Damien George b9c4783273 py/lexer: Remove unreachable code in string tokeniser. 2016-12-22 10:37:13 +11:00
Damien George adccafb42a tests/basics/lexer: Add a test for newline-escaping within a string. 2016-12-22 10:32:06 +11:00
Paul Sokolovsky d02f6a9956 extmod/modutimeq: Refactor into optimized class.
import utimeq, utime
    # Max queue size, the queue allocated statically on creation
    q = utimeq.utimeq(10)
    q.push(utime.ticks_ms(), data1, data2)
    res = [0, 0, 0]
    # Items in res are filled up with results
    q.pop(res)
2016-12-22 00:29:32 +03:00
Damien George 46a6592f9a py/emitglue: Refactor to remove assert(0), to improve coverage. 2016-12-21 11:52:05 +11:00
Damien George e4af712125 py/objint: Rename mp_obj_int_as_float to mp_obj_int_as_float_impl.
And also simplify it to remove the check for small int.  This can be done
because this function is only ever called if the argument is not a small
int.
2016-12-21 11:46:27 +11:00
Damien George 7318949c46 py/modbuiltins: Remove unreachable code. 2016-12-20 14:00:59 +11:00
Damien George de9cd00b39 py/compile: Add an extra pass for Xtensa inline assembler.
It needs an extra pass to compute the size of the constant table for the
l32r instructions.
2016-12-19 17:42:25 +11:00
Damien George 7f1da0a03b py: Add MICROPY_KBD_EXCEPTION config option to provide mp_kbd_exception.
Defining and initialising mp_kbd_exception is boiler-plate code and so the
core runtime can provide it, instead of each port needing to do it
themselves.

The exception object is placed in the VM state rather than on the heap.
2016-12-15 13:00:19 +11:00
Paul Sokolovsky 403c93053e py/mpconfig.h: Enable MICROPY_PY_SYS_EXIT by default.
sys.exit() is an important function to terminate a program. In particular,
the testsuite relies on it to skip tests (i.e. any other functionality may
be disabled, but sys.exit() is required to at least report that properly).
2016-12-14 21:22:05 +03:00
Damien George e8f2db7da3 py/runtime: Zero out fs_user_mount array in mp_init.
There's no need to force ports to copy-and-paste this initialisation
code.  If FSUSERMOUNT is enabled then this zeroing out must be done.
2016-12-14 11:40:11 +11:00
Damien George e83f140463 py/mpz: Remove unreachable code in mpn_or_neg functions. 2016-12-14 10:39:41 +11:00
Damien George d9047d3c8a py/builtinimport: Support importing packages from compiled .mpy files.
This patch ensures that __init__.mpy files are imported if their
containing directory is imported as a package.
2016-12-13 15:09:48 +11:00
Damien George a3c61004c2 py/binary: Do zero extension when storing a value larger than word size. 2016-12-12 15:00:06 +11:00
Paul Sokolovsky 9d787de2a1 py/objint: from_bytes, to_bytes: Require byteorder arg, require "little".
CPython requires byteorder arg, make uPy compatible. As we support only
"little", error out on anything else.
2016-12-09 21:15:16 +03:00
Damien George 93ee6603b1 py/asm: Fix x86 and ARM assemblers due to recent code refactoring. 2016-12-09 22:54:45 +11:00
Damien George 155fdc74d5 py/asm: Remove need for dummy_data when doing initial assembler passes.
For all but the last pass the assembler only needs to count how much space
is needed for the machine code, it doesn't actually need to emit anything.
The dummy_data just uses unnecessary RAM and without it the code is not
any more complex (and code size does not increase for Thumb and Xtensa
archs).
2016-12-09 22:50:58 +11:00
Damien George e920bab976 py/emitinline: Move common code for end of final pass to compiler.
This patch moves some common code from the individual inline assemblers to
the compiler, the code that calls the emit-glue to assign the machine code
to the functions scope.
2016-12-09 21:23:17 +11:00
Damien George dd53b12193 py/emitinline: Move inline-asm align and data methods to compiler.
These are generic methods that don't depend on the architecture and so
can be handled directly by the compiler.
2016-12-09 20:54:54 +11:00
Damien George a7fd786a1f py/emitinline: Embed entire asm struct instead of a pointer to it.
This reduces fragmentation, and memory use by 1 word.  But more
importantly it means the emit_inline_asm_t struct now "derives" from
mp_asm_base.
2016-12-09 20:35:21 +11:00
Damien George f76b1bfa9f py: Add inline Xtensa assembler.
This patch adds the MICROPY_EMIT_INLINE_XTENSA option, which, when
enabled, allows the @micropython.asm_xtensa decorator to be used.

The following opcodes are currently supported (ax is a register, a0-a15):

    ret_n()
    callx0(ax)
    j(label)
    jx(ax)

    beqz(ax, label)
    bnez(ax, label)
    mov(ax, ay)
    movi(ax, imm) # imm can be full 32-bit, uses l32r if needed

    and_(ax, ay, az)
    or_(ax, ay, az)
    xor(ax, ay, az)
    add(ax, ay, az)
    sub(ax, ay, az)
    mull(ax, ay, az)

    l8ui(ax, ay, imm)
    l16ui(ax, ay, imm)
    l32i(ax, ay, imm)
    s8i(ax, ay, imm)
    s16i(ax, ay, imm)
    s32i(ax, ay, imm)
    l16si(ax, ay, imm)
    addi(ax, ay, imm)

    ball(ax, ay, label)
    bany(ax, ay, label)
    bbc(ax, ay, label)
    bbs(ax, ay, label)
    beq(ax, ay, label)
    bge(ax, ay, label)
    bgeu(ax, ay, label)
    blt(ax, ay, label)
    bnall(ax, ay, label)
    bne(ax, ay, label)
    bnone(ax, ay, label)

Upon entry to the assembly function the registers a0, a12, a13, a14 are
pushed to the stack and the stack pointer (a1) decreased by 16.  Upon
exit, these registers and the stack pointer are restored, and ret.n is
executed to return to the caller (caller address is in a0).

Note that the ABI for the Xtensa emitters is non-windowing.
2016-12-09 17:07:38 +11:00
Damien George ad297a1950 py: Allow inline-assembler emitter to be generic.
This patch refactors some code so that it is easier to integrate new
inline assemblers for different architectures other than ARM Thumb.
2016-12-09 17:06:21 +11:00
Damien George 8e5aced1fd py: Integrate Xtensa assembler into native emitter.
The config option MICROPY_EMIT_XTENSA can now be enabled to target the
Xtensa architecture with @micropython.native and @micropython.viper
decorators.
2016-12-09 16:51:49 +11:00
Damien George fcac4b07f1 py/asmxtensa: Add low-level Xtensa assembler. 2016-12-09 16:51:49 +11:00
Damien George 81316fa411 py/asmbase: Add MP_PLAT_COMMIT_EXEC option for handling exec code.
If a port defines MP_PLAT_COMMIT_EXEC then this function is used to turn
RAM data into executable code.  For example a port may want to write the
data to flash for execution.  The function must return a pointer to the
executable data.
2016-12-09 16:51:49 +11:00
Damien George c8746e1e72 py: Move arch-specific assembler macros from emitnative to asmXXX.h. 2016-12-09 16:51:49 +11:00
Damien George c2713592bc py/emit.h: Remove long-obsolete declarations for cpython emitter. 2016-12-09 13:28:25 +11:00
Damien George 080a78b15e py/compile: Simplify configuration of native emitter. 2016-12-07 11:17:17 +11:00
Damien George 304cfda8c4 py/stream: Move ad-hoc ioctl constants to stream.h and rename them.
The constants MP_IOCTL_POLL_xxx, which were stmhal-specific, are moved
from stmhal/pybioctl.h (now deleted) to py/stream.h.  And they are renamed
to MP_STREAM_POLL_xxx to be consistent with other such constants.

All uses of these constants have been updated.
2016-12-02 16:37:29 +11:00
Damien George 63e82dcdfe py/asmthumb: Fix build for F7 MCUs after recent code refactoring. 2016-11-30 00:11:25 +11:00
Damien George 612599587b py: Factor out common code from assemblers into asmbase.[ch].
All assemblers should "derive" from mp_asm_base_t.
2016-11-28 09:24:50 +11:00
Damien George e6cf5fb2cc py/compile: Remove comment about TODO for short circuiting for if-stmt.
Short circuiting is handled correctly by c_if_cond, and constants within
short-circuit expressions are optimised by the parser.
2016-11-26 16:15:55 +11:00
Paul Sokolovsky 037e6912c6 py/objtype: Implement __call__ handling for an instance w/o heap alloc.
By refactoring and reusing code from objboundmeth.
2016-11-22 01:33:55 +03:00
Paul Sokolovsky 8f5bc3ffc0 stmhal/moduselect: Move to extmod/ for reuse by other ports. 2016-11-21 00:05:56 +03:00
Damien George 5bdf1650de py/lexer: Make lexer use an mp_reader as its source. 2016-11-16 18:35:01 +11:00
Damien George 66d955c218 py/lexer: Rewrite mp_lexer_new_from_fd in terms of mp_reader. 2016-11-16 18:13:51 +11:00
Damien George e5ef15a9d7 py/lexer: Provide generic mp_lexer_new_from_file based on mp_reader.
If a port defines MICROPY_READER_POSIX or MICROPY_READER_FATFS then
lexer.c now provides an implementation of mp_lexer_new_from_file using
the mp_reader_new_file function.
2016-11-16 18:13:51 +11:00
Damien George 511c083811 py/lexer: Rewrite mp_lexer_new_from_str_len in terms of mp_reader_mem. 2016-11-16 18:13:50 +11:00
Damien George 6b239c271c py: Factor out persistent-code reader into separate files.
Implementations of persistent-code reader are provided for POSIX systems
and systems using FatFS.  Macros to use these are MICROPY_READER_POSIX and
MICROPY_READER_FATFS respectively.  If an alternative implementation is
needed then a port can define the function mp_reader_new_file.
2016-11-16 18:13:50 +11:00
Damien George 6810f2c134 py: Factor persistent code load/save funcs into persistentcode.[ch]. 2016-11-16 16:14:14 +11:00
Damien George 9b525134d1 py/parse: Add code to fold logical constants in or/and/not operations.
Adds about 200 bytes to the code size when constant folding is enabled.
2016-11-15 16:48:49 +11:00
Damien George ed9c93f0f1 py/parse: Make mp_parse_node_new_leaf an inline function.
It is split into 2 functions, one to make small ints and the other to make
a non-small-int leaf node.  This reduces code size by 32 bytes on
bare-arm, 64 bytes on unix (x64-64) and 144 bytes on stmhal.
2016-11-15 16:48:48 +11:00
Damien George b0cbfb0492 py/parse: Move function to check for const parse node to parse.[ch]. 2016-11-15 16:48:48 +11:00
Damien George 659b06b250 py/*.mk: Replace uses of 'sed' with $(SED). 2016-11-15 16:09:43 +11:00
Dave Hylands 0400fa45ba py/mkrules.mk: Rework find command so it works on OSX.
The Mac version of find doesn't support -printf, so this changes
things to use sed to strip off the leading path element instead.
2016-11-15 16:07:48 +11:00
Paul Sokolovsky a0b2c6ad32 py/runtime: mp_resume: Fix exception handling for nanbox port. 2016-11-15 01:41:22 +03:00
Paul Sokolovsky 79d996a57b py/runtime: mp_resume: Handle exceptions in Python __next__().
This includes StopIteration and thus are important to make Python-coded
iterables work with yield from/await.

Exceptions in Python send() are still not handled and left for future
consideration and optimization.
2016-11-15 01:10:34 +03:00
Paul Sokolovsky c3d96d387c py/objexcept: Allow clearing traceback with 'exc.__traceback__ = None'.
We allow 'exc.__traceback__ = None' assignment as a low-level optimization
of pre-allocating exception instance and raising it repeatedly - this
avoids memory allocation during raise. However, uPy will keep adding
traceback entries to such exception instance, so before throwing it,
traceback should be cleared like above.

'exc.__traceback__ = None' syntax is CPython compatible. However, unlike
it, reading that attribute or setting it to any other value is not
supported (and not intended to be supported, again, the only reason for
adding this feature is to allow zero-memalloc exception raising).
2016-11-14 02:29:09 +03:00
Paul Sokolovsky 59a1201da9 all: Remove readall() method, which is equivalent to read() w/o args.
Its addition was due to an early exploration on how to add CPython-like
stream interface. It's clear that it's not needed and just takes up
bytes in all ports.
2016-11-14 00:24:22 +03:00
Damien George 0f3388de1e py/emitnative: Fix native import emitter when in viper mode. 2016-11-10 22:53:04 +11:00
Damien George fcda6a2a78 py: Strip leading dirs from frozen mpy files, so any path can be used.
With this patch one can now do "make FROZEN_MPY_DIR=../../frozen" to
specify a directory containing scripts to be frozen (as well as absolute
paths).

The compiled .mpy files are now stored in $(BUILD)/frozen_mpy/.
2016-11-08 14:36:06 +11:00
Damien George bdf33bc136 py: Move frozen bytecode Makefile rules from ports to common mk files.
Now, to use frozen bytecode all a port needs to do is define
FROZEN_MPY_DIR to the directory containing the .py files to freeze, and
define MICROPY_MODULE_FROZEN_MPY and MICROPY_QSTR_EXTRA_POOL.
2016-11-08 14:28:30 +11:00
Damien George 561844f3ba py: Add MICROPY_FLOAT_CONST macro for defining float constants.
All float constants in the core should use this macro to prevent
unnecessary creation of double-precision floats, which makes code less
efficient.
2016-11-03 12:33:01 +11:00
Colin Hogben 828df54bfe py: Change config default so m_malloc0 uses memset if GC not enabled.
With MICROPY_ENABLE_GC set to false the alternate memory manager may not
clear all memory that is allocated, so it must be cleared in m_malloc0.
2016-11-03 10:16:31 +11:00
Colin Hogben f9b6b37cf6 py: Fix wrong assumption that m_renew will not move if shrinking
In both parse.c and qstr.c, an internal chunking allocator tidies up
by calling m_renew to shrink an allocated chunk to the size used, and
assumes that the chunk will not move.  However, when MICROPY_ENABLE_GC
is false, m_renew calls the system realloc, which does not guarantee
this behaviour.  Environments where realloc may return a different
pointer include:

(1) mbed-os with MBED_HEAP_STATS_ENABLED (which adds a wrapper around
malloc & friends; this is where I was hit by the bug);

(2) valgrind on linux (how I diagnosed it).

The fix is to call m_renew_maybe with allow_move=false.
2016-11-02 23:15:41 +11:00