Commit Graph

2768 Commits

Author SHA1 Message Date
Damien George
30cf503e97 py/objset: Make inplace binary operators actually modify the set. 2017-02-02 23:33:49 +11:00
Paul Sokolovsky
50d3a9df67 py/objstringio: Allow to specify initial capacity by passing numeric argument.
E.g. uio.BytesIO(100) will allocate buffer with 100 bytes of space.
2017-02-02 00:33:43 +03:00
Dave Hylands
aee74a1dae unix: Make stack be non-executable
This PR is to address issue #2812.
2017-02-01 23:03:10 +03:00
Damien George
1808b2e8d5 extmod: Remove MICROPY_FSUSERMOUNT and related files.
Replaced by MICROPY_VFS and the VFS sub-system.
2017-01-30 12:26:07 +11:00
Damien George
8beba7310f extmod/vfs_fat: Remove MICROPY_READER_FATFS component. 2017-01-30 12:26:07 +11:00
Paul Sokolovsky
7a7516d40d extmod/machine_signal: Implement "signal" abstraction for machine module.
A signal is like a pin, but ca also be inverted (active low). As such, it
abstracts properties of various physical devices, like LEDs, buttons,
relays, buzzers, etc. To instantiate a Signal:

pin = machine.Pin(...)
signal = machine.Signal(pin, inverted=True)

signal has the same .value() and __call__() methods as a pin.
2017-01-29 18:57:36 +03:00
Damien George
3f6b4e08e3 extmod/vfs: Expose mp_vfs_mount_t type.
It should only be used for low-level things and with caution, for example
putting mounted VFS data in ROM or the static data section.
2017-01-27 23:22:15 +11:00
Damien George
dcb9ea7215 extmod: Add generic VFS sub-system.
This provides mp_vfs_XXX functions (eg mount, open, listdir) which are
agnostic to the underlying filesystem type, and just require an object with
the relevant filesystem-like methods (eg .mount, .open, .listidr) which can
then be mounted.

These mp_vfs_XXX functions would typically be used by a port to implement
the "uos" module, and mp_vfs_open would be the builtin open function.

This feature is controlled by MICROPY_VFS, disabled by default.
2017-01-27 17:19:06 +11:00
Damien George
d4464b0050 py/py.mk: Add CFLAGS_MOD flag to set config file for FatFs. 2017-01-27 13:19:10 +11:00
Damien George
cc4c1adf6e py/showbc: Make sure to set the const_table before printing bytecode. 2017-01-27 12:34:09 +11:00
Paul Sokolovsky
e2e663291d py/objstr: Optimize string concatenation with empty string.
In this, don't allocate copy, just return non-empty string. This helps
with a standard pattern of buffering data in case of short reads:

    buf = b""
    while ...:
        s = f.read(...)
        buf += s
        ...

For a typical case when single read returns all data needed, there won't
be extra allocation. This optimization helps uasyncio.
2017-01-27 00:49:39 +03:00
Damien George
e9cb1f8077 py/objmodule: Move module init/deinit code into runtime functions.
They are one-line functions and having them inline in mp_init/mp_deinit
eliminates the overhead of a function call, and matches how other state
is initialised in mp_init.
2017-01-26 23:30:38 +11:00
Damien George
eaa77455c3 py/objint: Fix left-shift overflow in checking for large int. 2017-01-25 14:39:13 +11:00
Damien George
f5172af1c4 py/builtinhelp: Implement help('modules') to list available modules.
This is how CPython does it, and it's very useful to help users discover
the available modules for a given port, especially built-in and frozen
modules.  The function does not list modules that are in the filesystem
because this would require a fair bit of work to do correctly, and is very
port specific (depending on the filesystem).
2017-01-22 12:12:54 +11:00
Damien George
9de91914fb py: Move weak-link map to objmodule.c, and expose module maps as public. 2017-01-22 11:59:29 +11:00
Damien George
9f04dfb568 py: Add builtin help function to core, with default help msg.
This builtin is configured using MICROPY_PY_BUILTINS_HELP, and is disabled
by default.
2017-01-22 11:56:16 +11:00
Paul Sokolovsky
e6ab43e2c0 py/objint_longlong: Add stub for mp_obj_int_from_bytes_impl().
To be implemented later.
2017-01-21 20:15:05 +03:00
Paul Sokolovsky
bec7bfb29d py/objint: from_bytes(): Implement "byteorder" param and arbitrary precision.
If result guaranteedly fits in a small int, it is handled in objint.c.
Otherwise, it is delegated to mp_obj_int_from_bytes_impl(), which should
be implemented by individual objint_*.c, similar to
mp_obj_int_to_bytes_impl().
2017-01-21 20:14:18 +03:00
Paul Sokolovsky
1b42f5251f py/mpz: Implement mpz_set_from_bytes() as a foundation for int.from_bytes(). 2017-01-21 20:07:50 +03:00
Damien George
e873243aa3 py/objint_mpz: Refactor switch-statement to remove unreachable default. 2017-01-19 23:35:45 +11:00
Damien George
94a587a750 py/formatfloat: Remove unreachable code.
The if-block that this unreachable code is in has a condition "f>=5" so
"fp_isless1(f)" will always fail.
2017-01-19 23:32:16 +11:00
Paul Sokolovsky
af90461931 py/binary: mp_binary_get_size: Raise error on unsupported typecodes.
Previouly, we had errors checked in callers, which led to duplicate code
or missing checks in some places.
2017-01-17 22:53:06 +03:00
Damien George
d7150b09d7 py/runtime: Refactor default case of switch to remove assert(0). 2017-01-17 17:03:56 +11:00
Damien George
5bea080737 py/objexcept: Replace if-cond and assert(0) with simple assert. 2017-01-17 17:03:16 +11:00
Damien George
f51f22dd42 py/emitnative: Remove assert(0)'s or replace with mp_not_implemented. 2017-01-17 17:02:21 +11:00
Damien George
86e942309a py/parse: Refactor code to remove assert(0)'s.
This helps to improve code coverage.  Note that most of the changes in
this patch are just de-denting the cases of the switch statements.
2017-01-17 17:00:55 +11:00
Damien George
239f920299 py/objgenerator: Don't raise RuntimeError if GeneratorExit ignored.
In this case it's allowed to be ignored.
2017-01-17 00:16:56 +11:00
Damien George
681994638b py/objgenerator: When throwing an object, don't make an exc instance.
Arguments to throw() for generators don't need to be exceptions.
2017-01-17 00:14:14 +11:00
Damien George
aeb2655073 py/runtime: Fix handling of throw() when resuming generator.
If GeneratorExit is injected as a throw-value then that should lead to
the close() method being called, if it exists.  If close() does not exist
then throw() should not be called, and this patch fixes this.
2017-01-17 00:10:49 +11:00
Damien George
40863fce6f py/runtime: Refactor assert(0) to improve coverage. 2017-01-17 00:09:56 +11:00
Damien George
d23834bc96 py/builtinimport: Remove unreachable code and change obj-import comment. 2017-01-16 16:41:43 +11:00
Damien George
63e291de70 py/builtinimport: Raise ValueError for bad relative import, per CPython. 2017-01-16 16:21:04 +11:00
Damien George
b528e9a428 py/builtinimport: Fix bug when importing names from frozen packages.
The commit d9047d3c8a introduced a bug
whereby "from a.b import c" stopped working for frozen packages.  This is
because the path was not properly truncated and became "a//b".  Such a
path resolves correctly for a "real" filesystem, but not for a search in
the list of frozen modules.
2017-01-08 20:17:23 +11:00
Damien George
343b4189b0 py/mkrules.mk: Add MPY_CROSS_FLAGS option to pass flags to mpy-cross.
So that ports can pass their own custom options to mpy-cross.
2017-01-05 15:51:36 +11:00
Damien George
3f9c45efd1 py/asmarm: Fix assembler's PASS_EMIT constant name. 2017-01-03 15:40:50 +11:00
Damien George
afc5063539 py/unicode: Comment-out unused function unichar_isprint. 2016-12-28 17:50:10 +11:00
Damien George
ea6a958393 py/objint: Simplify mp_int_format_size and remove unreachable code.
One never needs to format integers with a base larger than 16 (but code
can be easily extended beyond this value if needed in the future).
2016-12-28 12:46:20 +11:00
Damien George
44bf8e1f2b py/mpprint: Add assertion for, and comment about, valid base values. 2016-12-28 12:45:33 +11:00
Damien George
ca7af9a778 py/parsenum: Fix warning for signed/unsigned comparison. 2016-12-28 12:25:00 +11:00
Damien George
2d9440e2d1 py/mpz: Fix assertion in mpz_set_from_str which checks value of base. 2016-12-28 12:04:19 +11:00
Damien George
c2dd494bd9 py/parsenum: Simplify and generalise decoding of digit values.
This function should be able to parse integers with any value for the
base, because it is called by int('xxx', base).
2016-12-28 12:02:49 +11:00
Paul Sokolovsky
25f44c19f1 cc3200: Re-add support for UART REPL (MICROPY_STDIO_UART setting).
UART REPL support was lost in os.dupterm() refactorings, etc. As
os.dupterm() is there, implement UART REPL support at the high level -
if MICROPY_STDIO_UART is set, make default boot.py contain os.dupterm()
call for a UART. This means that changing MICROPY_STDIO_UART value will
also require erasing flash on a module to force boot.py re-creation.
2016-12-27 01:05:37 +03:00
Paul Sokolovsky
cf96be60dc py/misc.h: Typo fix in comment. 2016-12-27 01:05:30 +03:00
Damien George
c305ae3243 py/lexer: Permanently disable the mp_lexer_show_token function.
The lexer is very mature and this debug function is no longer used.  If
it's really needed one can uncomment it and recompile.
2016-12-22 10:49:54 +11:00
Damien George
f4aebafe7a py/lexer: Remove unnecessary check for EOF in lexer's next_char func.
This check always fails (ie chr0 is never EOF) because the callers of this
function never call it past the end of the input stream.  And even if they
did it would be harmless because 1) reader.readbyte must continue to
return an EOF char if the stream is exhausted; 2) next_char would just
count the subsequent EOF's as characters worth 1 column.
2016-12-22 10:39:06 +11:00
Damien George
b9c4783273 py/lexer: Remove unreachable code in string tokeniser. 2016-12-22 10:37:13 +11:00
Damien George
adccafb42a tests/basics/lexer: Add a test for newline-escaping within a string. 2016-12-22 10:32:06 +11:00
Paul Sokolovsky
d02f6a9956 extmod/modutimeq: Refactor into optimized class.
import utimeq, utime
    # Max queue size, the queue allocated statically on creation
    q = utimeq.utimeq(10)
    q.push(utime.ticks_ms(), data1, data2)
    res = [0, 0, 0]
    # Items in res are filled up with results
    q.pop(res)
2016-12-22 00:29:32 +03:00
Damien George
46a6592f9a py/emitglue: Refactor to remove assert(0), to improve coverage. 2016-12-21 11:52:05 +11:00
Damien George
e4af712125 py/objint: Rename mp_obj_int_as_float to mp_obj_int_as_float_impl.
And also simplify it to remove the check for small int.  This can be done
because this function is only ever called if the argument is not a small
int.
2016-12-21 11:46:27 +11:00
Damien George
7318949c46 py/modbuiltins: Remove unreachable code. 2016-12-20 14:00:59 +11:00
Damien George
de9cd00b39 py/compile: Add an extra pass for Xtensa inline assembler.
It needs an extra pass to compute the size of the constant table for the
l32r instructions.
2016-12-19 17:42:25 +11:00
Damien George
7f1da0a03b py: Add MICROPY_KBD_EXCEPTION config option to provide mp_kbd_exception.
Defining and initialising mp_kbd_exception is boiler-plate code and so the
core runtime can provide it, instead of each port needing to do it
themselves.

The exception object is placed in the VM state rather than on the heap.
2016-12-15 13:00:19 +11:00
Paul Sokolovsky
403c93053e py/mpconfig.h: Enable MICROPY_PY_SYS_EXIT by default.
sys.exit() is an important function to terminate a program. In particular,
the testsuite relies on it to skip tests (i.e. any other functionality may
be disabled, but sys.exit() is required to at least report that properly).
2016-12-14 21:22:05 +03:00
Damien George
e8f2db7da3 py/runtime: Zero out fs_user_mount array in mp_init.
There's no need to force ports to copy-and-paste this initialisation
code.  If FSUSERMOUNT is enabled then this zeroing out must be done.
2016-12-14 11:40:11 +11:00
Damien George
e83f140463 py/mpz: Remove unreachable code in mpn_or_neg functions. 2016-12-14 10:39:41 +11:00
Damien George
d9047d3c8a py/builtinimport: Support importing packages from compiled .mpy files.
This patch ensures that __init__.mpy files are imported if their
containing directory is imported as a package.
2016-12-13 15:09:48 +11:00
Damien George
a3c61004c2 py/binary: Do zero extension when storing a value larger than word size. 2016-12-12 15:00:06 +11:00
Paul Sokolovsky
9d787de2a1 py/objint: from_bytes, to_bytes: Require byteorder arg, require "little".
CPython requires byteorder arg, make uPy compatible. As we support only
"little", error out on anything else.
2016-12-09 21:15:16 +03:00
Damien George
93ee6603b1 py/asm: Fix x86 and ARM assemblers due to recent code refactoring. 2016-12-09 22:54:45 +11:00
Damien George
155fdc74d5 py/asm: Remove need for dummy_data when doing initial assembler passes.
For all but the last pass the assembler only needs to count how much space
is needed for the machine code, it doesn't actually need to emit anything.
The dummy_data just uses unnecessary RAM and without it the code is not
any more complex (and code size does not increase for Thumb and Xtensa
archs).
2016-12-09 22:50:58 +11:00
Damien George
e920bab976 py/emitinline: Move common code for end of final pass to compiler.
This patch moves some common code from the individual inline assemblers to
the compiler, the code that calls the emit-glue to assign the machine code
to the functions scope.
2016-12-09 21:23:17 +11:00
Damien George
dd53b12193 py/emitinline: Move inline-asm align and data methods to compiler.
These are generic methods that don't depend on the architecture and so
can be handled directly by the compiler.
2016-12-09 20:54:54 +11:00
Damien George
a7fd786a1f py/emitinline: Embed entire asm struct instead of a pointer to it.
This reduces fragmentation, and memory use by 1 word.  But more
importantly it means the emit_inline_asm_t struct now "derives" from
mp_asm_base.
2016-12-09 20:35:21 +11:00
Damien George
f76b1bfa9f py: Add inline Xtensa assembler.
This patch adds the MICROPY_EMIT_INLINE_XTENSA option, which, when
enabled, allows the @micropython.asm_xtensa decorator to be used.

The following opcodes are currently supported (ax is a register, a0-a15):

    ret_n()
    callx0(ax)
    j(label)
    jx(ax)

    beqz(ax, label)
    bnez(ax, label)
    mov(ax, ay)
    movi(ax, imm) # imm can be full 32-bit, uses l32r if needed

    and_(ax, ay, az)
    or_(ax, ay, az)
    xor(ax, ay, az)
    add(ax, ay, az)
    sub(ax, ay, az)
    mull(ax, ay, az)

    l8ui(ax, ay, imm)
    l16ui(ax, ay, imm)
    l32i(ax, ay, imm)
    s8i(ax, ay, imm)
    s16i(ax, ay, imm)
    s32i(ax, ay, imm)
    l16si(ax, ay, imm)
    addi(ax, ay, imm)

    ball(ax, ay, label)
    bany(ax, ay, label)
    bbc(ax, ay, label)
    bbs(ax, ay, label)
    beq(ax, ay, label)
    bge(ax, ay, label)
    bgeu(ax, ay, label)
    blt(ax, ay, label)
    bnall(ax, ay, label)
    bne(ax, ay, label)
    bnone(ax, ay, label)

Upon entry to the assembly function the registers a0, a12, a13, a14 are
pushed to the stack and the stack pointer (a1) decreased by 16.  Upon
exit, these registers and the stack pointer are restored, and ret.n is
executed to return to the caller (caller address is in a0).

Note that the ABI for the Xtensa emitters is non-windowing.
2016-12-09 17:07:38 +11:00
Damien George
ad297a1950 py: Allow inline-assembler emitter to be generic.
This patch refactors some code so that it is easier to integrate new
inline assemblers for different architectures other than ARM Thumb.
2016-12-09 17:06:21 +11:00
Damien George
8e5aced1fd py: Integrate Xtensa assembler into native emitter.
The config option MICROPY_EMIT_XTENSA can now be enabled to target the
Xtensa architecture with @micropython.native and @micropython.viper
decorators.
2016-12-09 16:51:49 +11:00
Damien George
fcac4b07f1 py/asmxtensa: Add low-level Xtensa assembler. 2016-12-09 16:51:49 +11:00
Damien George
81316fa411 py/asmbase: Add MP_PLAT_COMMIT_EXEC option for handling exec code.
If a port defines MP_PLAT_COMMIT_EXEC then this function is used to turn
RAM data into executable code.  For example a port may want to write the
data to flash for execution.  The function must return a pointer to the
executable data.
2016-12-09 16:51:49 +11:00
Damien George
c8746e1e72 py: Move arch-specific assembler macros from emitnative to asmXXX.h. 2016-12-09 16:51:49 +11:00
Damien George
c2713592bc py/emit.h: Remove long-obsolete declarations for cpython emitter. 2016-12-09 13:28:25 +11:00
Damien George
080a78b15e py/compile: Simplify configuration of native emitter. 2016-12-07 11:17:17 +11:00
Damien George
304cfda8c4 py/stream: Move ad-hoc ioctl constants to stream.h and rename them.
The constants MP_IOCTL_POLL_xxx, which were stmhal-specific, are moved
from stmhal/pybioctl.h (now deleted) to py/stream.h.  And they are renamed
to MP_STREAM_POLL_xxx to be consistent with other such constants.

All uses of these constants have been updated.
2016-12-02 16:37:29 +11:00
Damien George
63e82dcdfe py/asmthumb: Fix build for F7 MCUs after recent code refactoring. 2016-11-30 00:11:25 +11:00
Damien George
612599587b py: Factor out common code from assemblers into asmbase.[ch].
All assemblers should "derive" from mp_asm_base_t.
2016-11-28 09:24:50 +11:00
Damien George
e6cf5fb2cc py/compile: Remove comment about TODO for short circuiting for if-stmt.
Short circuiting is handled correctly by c_if_cond, and constants within
short-circuit expressions are optimised by the parser.
2016-11-26 16:15:55 +11:00
Paul Sokolovsky
037e6912c6 py/objtype: Implement __call__ handling for an instance w/o heap alloc.
By refactoring and reusing code from objboundmeth.
2016-11-22 01:33:55 +03:00
Paul Sokolovsky
8f5bc3ffc0 stmhal/moduselect: Move to extmod/ for reuse by other ports. 2016-11-21 00:05:56 +03:00
Damien George
5bdf1650de py/lexer: Make lexer use an mp_reader as its source. 2016-11-16 18:35:01 +11:00
Damien George
66d955c218 py/lexer: Rewrite mp_lexer_new_from_fd in terms of mp_reader. 2016-11-16 18:13:51 +11:00
Damien George
e5ef15a9d7 py/lexer: Provide generic mp_lexer_new_from_file based on mp_reader.
If a port defines MICROPY_READER_POSIX or MICROPY_READER_FATFS then
lexer.c now provides an implementation of mp_lexer_new_from_file using
the mp_reader_new_file function.
2016-11-16 18:13:51 +11:00
Damien George
511c083811 py/lexer: Rewrite mp_lexer_new_from_str_len in terms of mp_reader_mem. 2016-11-16 18:13:50 +11:00
Damien George
6b239c271c py: Factor out persistent-code reader into separate files.
Implementations of persistent-code reader are provided for POSIX systems
and systems using FatFS.  Macros to use these are MICROPY_READER_POSIX and
MICROPY_READER_FATFS respectively.  If an alternative implementation is
needed then a port can define the function mp_reader_new_file.
2016-11-16 18:13:50 +11:00
Damien George
6810f2c134 py: Factor persistent code load/save funcs into persistentcode.[ch]. 2016-11-16 16:14:14 +11:00
Damien George
9b525134d1 py/parse: Add code to fold logical constants in or/and/not operations.
Adds about 200 bytes to the code size when constant folding is enabled.
2016-11-15 16:48:49 +11:00
Damien George
ed9c93f0f1 py/parse: Make mp_parse_node_new_leaf an inline function.
It is split into 2 functions, one to make small ints and the other to make
a non-small-int leaf node.  This reduces code size by 32 bytes on
bare-arm, 64 bytes on unix (x64-64) and 144 bytes on stmhal.
2016-11-15 16:48:48 +11:00
Damien George
b0cbfb0492 py/parse: Move function to check for const parse node to parse.[ch]. 2016-11-15 16:48:48 +11:00
Damien George
659b06b250 py/*.mk: Replace uses of 'sed' with $(SED). 2016-11-15 16:09:43 +11:00
Dave Hylands
0400fa45ba py/mkrules.mk: Rework find command so it works on OSX.
The Mac version of find doesn't support -printf, so this changes
things to use sed to strip off the leading path element instead.
2016-11-15 16:07:48 +11:00
Paul Sokolovsky
a0b2c6ad32 py/runtime: mp_resume: Fix exception handling for nanbox port. 2016-11-15 01:41:22 +03:00
Paul Sokolovsky
79d996a57b py/runtime: mp_resume: Handle exceptions in Python __next__().
This includes StopIteration and thus are important to make Python-coded
iterables work with yield from/await.

Exceptions in Python send() are still not handled and left for future
consideration and optimization.
2016-11-15 01:10:34 +03:00
Paul Sokolovsky
c3d96d387c py/objexcept: Allow clearing traceback with 'exc.__traceback__ = None'.
We allow 'exc.__traceback__ = None' assignment as a low-level optimization
of pre-allocating exception instance and raising it repeatedly - this
avoids memory allocation during raise. However, uPy will keep adding
traceback entries to such exception instance, so before throwing it,
traceback should be cleared like above.

'exc.__traceback__ = None' syntax is CPython compatible. However, unlike
it, reading that attribute or setting it to any other value is not
supported (and not intended to be supported, again, the only reason for
adding this feature is to allow zero-memalloc exception raising).
2016-11-14 02:29:09 +03:00
Paul Sokolovsky
59a1201da9 all: Remove readall() method, which is equivalent to read() w/o args.
Its addition was due to an early exploration on how to add CPython-like
stream interface. It's clear that it's not needed and just takes up
bytes in all ports.
2016-11-14 00:24:22 +03:00
Damien George
0f3388de1e py/emitnative: Fix native import emitter when in viper mode. 2016-11-10 22:53:04 +11:00
Damien George
fcda6a2a78 py: Strip leading dirs from frozen mpy files, so any path can be used.
With this patch one can now do "make FROZEN_MPY_DIR=../../frozen" to
specify a directory containing scripts to be frozen (as well as absolute
paths).

The compiled .mpy files are now stored in $(BUILD)/frozen_mpy/.
2016-11-08 14:36:06 +11:00
Damien George
bdf33bc136 py: Move frozen bytecode Makefile rules from ports to common mk files.
Now, to use frozen bytecode all a port needs to do is define
FROZEN_MPY_DIR to the directory containing the .py files to freeze, and
define MICROPY_MODULE_FROZEN_MPY and MICROPY_QSTR_EXTRA_POOL.
2016-11-08 14:28:30 +11:00
Damien George
561844f3ba py: Add MICROPY_FLOAT_CONST macro for defining float constants.
All float constants in the core should use this macro to prevent
unnecessary creation of double-precision floats, which makes code less
efficient.
2016-11-03 12:33:01 +11:00
Colin Hogben
828df54bfe py: Change config default so m_malloc0 uses memset if GC not enabled.
With MICROPY_ENABLE_GC set to false the alternate memory manager may not
clear all memory that is allocated, so it must be cleared in m_malloc0.
2016-11-03 10:16:31 +11:00
Colin Hogben
f9b6b37cf6 py: Fix wrong assumption that m_renew will not move if shrinking
In both parse.c and qstr.c, an internal chunking allocator tidies up
by calling m_renew to shrink an allocated chunk to the size used, and
assumes that the chunk will not move.  However, when MICROPY_ENABLE_GC
is false, m_renew calls the system realloc, which does not guarantee
this behaviour.  Environments where realloc may return a different
pointer include:

(1) mbed-os with MBED_HEAP_STATS_ENABLED (which adds a wrapper around
malloc & friends; this is where I was hit by the bug);

(2) valgrind on linux (how I diagnosed it).

The fix is to call m_renew_maybe with allow_move=false.
2016-11-02 23:15:41 +11:00
Jan Pochyla
ffb04a5845 unix: fix symbol references for x86 Mac 2016-11-02 00:42:04 +03:00