Commit Graph

3888 Commits

Author SHA1 Message Date
Damien George 09376f0e47 py: Introduce MP_ROM_NONE macro for ROM to refer to None object.
This helps to prevent mistakes, and allows easily changing the ROM value of
None if needed.
2019-12-27 22:51:17 +11:00
Damien George f5eec903fa py/objsingleton: Use mp_generic_unary_op for singleton objects.
So these types more closely match NoneType, eg they can be hashed, like in
CPython.
2019-12-27 12:53:36 +11:00
Damien George aacd618939 py/runtime: Don't allocate iter buf for user-defined types.
A user-defined type that defines __iter__ doesn't need any memory to be
pre-allocated for its iterator (because it can't use such memory).  So
optimise for this case by not allocating the iter-buf.
2019-12-27 12:34:22 +11:00
Damien George ab75210e33 py/asmx86: Remove unused 5th argument facility.
In commit 71a3d6ec3b mp_setup_code_state was
changed from a 5-arg function to a 4-arg function, and at that point 5-arg
calls in native code were no longer needed.  See also commit
4f9842ad80.
2019-12-27 12:30:51 +11:00
Damien George b97fb683d0 py/asmx86: Fix stack to be 16-byte aligned for entry and sub-call. 2019-12-27 12:30:51 +11:00
Damien George 300eb65ae7 py/nlrx86: Silence possible warnings about unused nlr argument. 2019-12-23 00:07:03 +11:00
Yonatan Goldschmidt 42e45bd694 py/objobject: Add object.__delattr__ function.
Similar to object.__setattr__.
2019-12-21 00:14:22 +11:00
Yonatan Goldschmidt 07ccb5588c py/objobject: Add object.__setattr__ function.
Allows assigning attributes on class instances that implement their own
__setattr__.  Both object.__setattr__ and super(A, b).__setattr__ will work
with this commit.
2019-12-21 00:12:08 +11:00
Damien George 5e431188db py/obj.h: Remove comments about additional mp_buffer_info_t entries.
These entries are unlikely to be needed, so remove them to clean up the
struct definition.
2019-12-20 23:36:17 +11:00
Damien George 035180ca01 py: Remove commented-out debug printf's from emitbc and objlist.
Any debugging prints should use a macro like DEBUG_printf.
2019-12-20 23:34:46 +11:00
Damien George 1f37194730 all: Bump version to 1.12. 2019-12-20 16:58:17 +11:00
Damien George 073c5f3a40 py/profile: Fix debug opcode decoding of MP_BC_RAISE_xxx opcodes. 2019-12-20 14:57:44 +11:00
Damien George 95473980ef py/vm: Fix comment to refer to MP_BC_RAISE_OBJ instead of RAISE_VARARGS. 2019-12-20 14:57:06 +11:00
Damien George 0bd7d1f7f0 py/persistentcode: Move loading of rodata/bss to before obj/raw-code.
This makes the loading of viper-code-with-relocations a bit neater and
easier to understand, by treating the rodata/bss like a special object to
be loaded into the constant table (which is how it behaves).
2019-12-17 13:22:11 +11:00
Damien George 48e9262f55 py/dynruntime: Implement uint new/get, mp_obj_len and mp_obj_subscr. 2019-12-13 13:29:11 +11:00
Damien George abc642973d py/dynruntime: Add support for float API to make/get floats.
We don't want to add a feature flag to .mpy files that indicate float
support because it will get complex and difficult to use.  Instead the .mpy
is built using whatever precision it chooses (float or double) and the
native glue API will convert between this choice and what the host runtime
actually uses.
2019-12-12 20:15:28 +11:00
Damien George ff58961944 py/nativeglue: Add float new/get functions with both single and double. 2019-12-12 20:15:28 +11:00
Damien George 9ac949cdbd py/persistentcode: Make ARM Thumb archs support multiple sub-archs. 2019-12-12 20:15:28 +11:00
Damien George aad79adab7 tools/mpy_ld.py: Add new mpy_ld.py tool and associated build files.
This commit adds a new tool called mpy_ld.py which is essentially a linker
that builds .mpy files directly from .o files.  A new header file
(dynruntime.h) and makefile fragment (dynruntime.mk) are also included
which allow building .mpy files from C source code.  Such .mpy files can
then be dynamically imported as though they were a normal Python module,
even though they are implemented in C.

Converting .o files directly (rather than pre-linked .elf files) allows the
resulting .mpy to be more efficient because it has more control over the
relocations; for example it can skip PLT indirection.  Doing it this way
also allows supporting more architectures, such as Xtensa which has
specific needs for position-independent code and the GOT.

The tool supports targets of x86, x86-64, ARM Thumb and Xtensa (windowed
and non-windowed).  BSS, text and rodata sections are supported, with
relocations to all internal sections and symbols, as well as relocations to
some external symbols (defined by dynruntime.h), and linking of qstrs.
2019-12-12 20:15:28 +11:00
Damien George 3690f79afc py/nativeglue: Add funcs/types to native glue table for dynamic runtime.
These allow discovery of symbols by native code that is loaded dynamically.
2019-12-12 20:15:28 +11:00
Damien George 360d972c16 py/nativeglue: Add new header file with native function table typedef. 2019-12-12 20:15:28 +11:00
Damien George b47e155bd0 py/persistentcode: Add ability to relocate loaded native code.
Implements text, rodata and bss generalised relocations, as well as generic
qstr-object linking.  This allows importing dynamic native modules on all
supported architectures in a unified way.
2019-12-12 20:15:28 +11:00
Emil Renner Berthing 4ebbacd65e py/objenumerate: Check for valid args in enumerate constructor.
For the case where MICROPY_CPYTHON_COMPAT is disabled.  This fix makes
basics/fun_error2.py pass and not crash the interpreter.
2019-12-09 14:28:24 +11:00
Damien George 01e5802ee3 py: Remove 3 obsolete commented-out lines from header files. 2019-11-26 21:36:41 +11:00
Yonatan Goldschmidt 4318a6d755 py/objstringio: Slightly optimize stringio_copy_on_write for code size.
With the memcpy() call placed last it avoids the effects of registers
clobbering.  It's definitely effective in non-inlined functions, but even
here it is still making a small difference.  For example, on stm32, this
saves an extra `ldr` instruction to load `o->vstr` after the memcpy()
returns.
2019-11-26 14:26:24 +11:00
Léa Saviot bc129f1b84 py/qstr: Raise exception in qstr_from_strn if str to intern is too long.
The string length being longer than the allowed qstr length can happen in
many locations, for example in the parser with very long variable names.
Without an explicit check that the length is within range (as done in this
patch) the code would exhibit crashes and strange behaviour with truncated
strings.
2019-11-26 10:51:47 +11:00
Léa Saviot a7bc4d1a14 py/builtinimport: Raise exception on empty module name.
To prevent a crash returning MP_OBJ_NULL.  A test is added for this case.
2019-11-26 00:28:32 +11:00
Petr Viktorin 57c18fdd38 py/compile: Coalesce error message for break/continue outside loop.
To reduce code size.
2019-11-21 12:13:11 +11:00
Jim Mussared 4f96689281 py/ringbuf: Add peek16 method. 2019-11-21 12:04:53 +11:00
Andrew Leech 1e87f11d3f py/objdict: Support ujson.dump() of OrderedDict objects.
Following CPython, OrderedDict are dumped with the syntax of dict.
2019-11-13 13:51:18 +11:00
Damien George 7e374d2317 py/emitnx86: Make mp_f_n_args table match order of mp_fun_kind_t. 2019-11-07 19:43:23 +11:00
Thea Flowers f0e4677f0d py/emitnative: Fix typo, REG_PARENT_ARG_RET should be REG_PARENT_RET. 2019-11-07 12:30:47 +11:00
Damien George c13f9f209d all: Convert nlr_raise(mp_obj_new_exception_msg(x)) to mp_raise_msg(x).
This helper function was added a while ago and these are the remaining
cases to convert, to save a bit of code size.
2019-11-05 11:35:45 +11:00
Damien George 80df377e95 py/modsys: Report .mpy version in sys.implementation.
This commit adds a sys.implementation.mpy entry when the system supports
importing .mpy files.  This entry is a 16-bit integer which encodes two
bytes of information from the header of .mpy files that are supported by
the system being run: the second and third bytes, .mpy version, and flags
and native architecture.  This allows determining the supported .mpy file
dynamically by code, and also for the user to find it out by inspecting
this value.  It's further possible to dynamically detect if the system
supports importing .mpy files by `hasattr(sys.implementation, 'mpy')`.
2019-11-04 16:00:41 +11:00
Damien George f4601af10a py/persistentcode: Move declarations for .mpy header from .c to .h file. 2019-11-04 16:00:05 +11:00
Jim Mussared 5578182ec9 py/objgenerator: Allow pend_throw to an unstarted generator.
Replace the is_running field with a tri-state variable to indicate
running/not-running/pending-exception.

Update tests to cover the various cases.

This allows cancellation in uasyncio even if the coroutine hasn't been
executed yet.  Fixes #5242
2019-11-04 15:51:16 +11:00
Jim Mussared 576ed89224 py/objgenerator: Remove globals from mp_obj_gen_instance_t.
This wasn't necessary as the wrapped function already has a reference to
its globals.  But it had a dual purpose of tracking whether the function
was currently running, so replace it with a bool.
2019-11-04 15:50:59 +11:00
Damien George f2ecfe8b83 py/nativeglue: Remove unused mp_obj_new_cell from mp_fun_table.
It has been unused since 9988618e0e
2019-11-01 17:26:10 +11:00
Damien George d3c383de79 py/stream.h: Add MP_STREAM_POLL_NVAL constant. 2019-10-31 12:54:37 +11:00
Damien George 323d47887f py/runtime: Reorder some binary ops so they don't require conditionals.
runtime0.h is part of the MicroPython ABI so it's simpler if it's
independent of config options, like MICROPY_PY_REVERSE_SPECIAL_METHODS.

What's effectively done here is to move MP_BINARY_OP_DIVMOD and
MP_BINARY_OP_CONTAINS up in the enum, then remove the #if
MICROPY_PY_REVERSE_SPECIAL_METHODS conditional.

Without this change .mpy files would need to have a feature flag for
MICROPY_PY_REVERSE_SPECIAL_METHODS (when embedding native code that uses
this enum).

This commit has no effect when MICROPY_PY_REVERSE_SPECIAL_METHODS is
disabled.  With this option enabled this commit reduces code size by about
60 bytes.
2019-10-29 23:13:51 +11:00
Damien George a099505420 extmod: Add VFS littlefs bindings.
Both LFS1 and LFS2 are supported at the same time.
2019-10-29 14:17:29 +11:00
Damien George e1c7b1cb43 extmod/vfs_blockdev: Factor out block device interface code. 2019-10-29 12:55:17 +11:00
Michael Neuling 079cc940a6 powerpc: Add initial port to bare metal PowerPC arch.
Runs in microwatt (GHDL and FPGA) and qemu.

Port done initially by Michael Neuling, with help from Anton Blanchard and
Jordan Niethe.
2019-10-22 22:45:33 +11:00
Jim Mussared b02d7e612d extmod/modbluetooth: Rename module to "ubluetooth".
For consistency with "umachine". Now that weak links are enabled
by default for built-in modules, this should be a no-op, but allows
extension of the bluetooth module by user code.

Also move registration of ubluetooth to objmodule rather than
port-specific.
2019-10-22 21:58:05 +11:00
Damien George 21a60935a5 py/modarray: Rename "array" module to "uarray".
Following the other modules like ustruct, ucollections.

See issues #4370 and #4449.
2019-10-22 16:35:46 +11:00
Damien George d2384efa80 py: Automatically provide weak links from "foo" to "ufoo" module name.
This commit implements automatic module weak links for all built-in
modules, by searching for "ufoo" in the built-in module list if "foo"
cannot be found.  This means that all modules named "ufoo" are always
available as "foo".  Also, a port can no longer add any other weak links,
which makes strict the definition of a weak link.

It saves some code size (about 100-200 bytes) on ports that previously had
lots of weak links.

Some changes from the previous behaviour:
- It doesn't intern the non-u module names (eg "foo" is not interned),
  which saves code size, but will mean that "import foo" creates a new qstr
  (namely "foo") in RAM (unless the importing module is frozen).
- help('modules') no longer lists non-u module names, only the u-variants;
  this reduces duplication in the help listing.

Weak links are effectively the same as having a set of symbolic links on
the filesystem that is searched last.  So an "import foo" will search
built-in modules first, then all paths in sys.path, then weak links last,
importing "ufoo" if it exists.  Thus a file called "foo.py" somewhere in
sys.path will still have precedence over the weak link of "foo" to "ufoo".

See issues: #1740, #4449, #5229, #5241.
2019-10-22 15:30:52 +11:00
Jim Mussared c7ae8c5a99 py/objstr: Size-optimise failure path for mp_obj_str_get_buffer.
These fields are never looked at if the function returns non-zero.
2019-10-22 13:54:09 +11:00
Jim Mussared 7662501d5b py/mkrules.mk: Add warning/error for invalid frozen config. 2019-10-21 23:21:04 +11:00
Jim Mussared 8ba963cfa3 tools/makemanifest.py: Eval relative paths w.r.t. current manifest file.
When loading a manifest file, e.g. by include(), it will chdir first to the
directory of that manifest.  This means that all file operations within a
manifest are relative to that manifest's location.

As a consequence of this, additional environment variables are needed to
find absolute paths, so the following are added: $(MPY_LIB_DIR),
$(PORT_DIR), $(BOARD_DIR).  And rename $(MPY) to $(MPY_DIR) to be
consistent.

Existing manifests are updated to match.
2019-10-21 23:01:41 +11:00
Josh Lloyd 8f9e2e325a py/objtype: Add type.__bases__ attribute.
Enabled as part of MICROPY_CPYTHON_COMPAT.
2019-10-18 15:20:56 +11:00
Damien George e81f538e25 tools: Add mechanism to provide a manifest of frozen files.
This introduces a new build variable FROZEN_MANIFEST which can be set to a
manifest listing (written in Python) that describes the set of files to be
frozen in to the firmware.
2019-10-15 21:34:23 +11:00
Damien George d7a9388fe0 ports: Add new make target "submodules" which inits required modules. 2019-10-15 17:14:41 +11:00
Damien George 23f0691fdd py/persistentcode: Make .mpy more compact with qstr directly in prelude.
Instead of encoding 4 zero bytes as placeholders for the simple_name and
source_file qstrs, and storing the qstrs after the bytecode, store the
qstrs at the location of these 4 bytes.  This saves 4 bytes per bytecode
function stored in a .mpy file (for example lcd160cr.mpy drops by 232
bytes, 4x 58 functions).  And resulting code size is slightly reduced on
ports that use this feature.
2019-10-15 16:56:27 +11:00
Damien George 9adedce42e py: Add new Xtensa-Windowed arch for native emitter.
Enabled via the configuration MICROPY_EMIT_XTENSAWIN.
2019-10-05 13:44:53 +10:00
Damien George f7ddc94166 py/asmxtensa: Add support for Xtensa with windowed registers.
Window-specific asm emit functions are added, along with a new macro option
GENERIC_ASM_API_WIN.
2019-10-05 13:44:08 +10:00
Damien George 306ec5369a py/emitnative: Add support for archs that cannot read executable data.
In which case place the native function prelude in a bytes object, linked
from the const_table of that function.  An architecture should define
N_PRELUDE_AS_BYTES_OBJ to 1 before including py/emitnative.c to emit
correct machine code, then enable MICROPY_EMIT_NATIVE_PRELUDE_AS_BYTES_OBJ
so the runtime can correctly handle the prelude being in a bytes object.
2019-10-05 13:42:39 +10:00
Damien George 3504edc804 py/emitnative: Add support for using setjmp with native emitter.
To enable this feature the N_NLR_SETJMP macro should be set to 1 before
including py/emitnative.c.
2019-10-05 13:41:58 +10:00
Damien George 4107597b84 py/emitnative: Add support for archs with windowed registers.
Such that args/return regs for the parent are different to args/return regs
for child calls.  For an architecture to use this feature it should define
the REG_PARENT_xxx macros before including py/emitnative.c.
2019-10-05 13:41:14 +10:00
Damien George 809d89c794 py/runtime: Fix PEP479 behaviour throwing StopIteration into yield from.
Commit 3f6ffe059f implemented PEP479 but did
not catch the case fixed in this commit.  Found by coverage analysis, that
the VM had uncovered code.
2019-10-04 23:27:00 +10:00
Damien George 82c494a97e py/vm: Fix handling of unwind jump out of active finally.
Prior to this commit, when unwinding through an active finally the stack
was not being correctly popped/folded, which resulting in the VM crashing
for complicated unwinding of nested finallys.

This should be fixed with this commit, and more tests for return/break/
continue within a finally have been added to exercise this.
2019-10-04 23:01:29 +10:00
Jim Mussared a09fd04758 py/makeqstrdefs.py: Remove unused blacklist.
As of 7d58a197cf, `NULL` should no longer be
here because it's allowed (MP_QSTRnull took its place).  This entry was
preventing the use of MP_QSTR_NULL to mean "NULL" (although this is not
currently used).

A blacklist should not be needed because it should be possible to intern
all strings.

Fixes issue #5140.
2019-10-04 17:18:56 +10:00
Jim Mussared 4ddd46e6cf docs/develop/qstr.rst: Add documentation for string interning. 2019-10-04 17:13:13 +10:00
Petr Viktorin 25a9bccdee py/compile: Disallow 'import *' outside module level.
This check follows CPython's behaviour, because 'import *' always populates
the globals with the imported names, not locals.

Since it's safe to do this (doesn't lead to a crash or undefined behaviour)
the check is only enabled for MICROPY_CPYTHON_COMPAT.

Fixes issue #5121.
2019-10-04 16:46:47 +10:00
Damien George 1d0423419b py/bc: Don't include mp_decode_uint funcs when not needed.
These are now only needed when persistent code is disabled.
2019-10-01 12:26:22 +10:00
Damien George c8c0fd4ca3 py: Rework and compress second part of bytecode prelude.
This patch compresses the second part of the bytecode prelude which
contains the source file name, function name, source-line-number mapping
and cell closure information.  This part of the prelude now begins with a
single varible length unsigned integer which encodes 2 numbers, being the
byte-size of the following 2 sections in the header: the "source info
section" and the "closure section".  After decoding this variable unsigned
integer it's possible to skip over one or both of these sections very
easily.

This scheme saves about 2 bytes for most functions compared to the original
format: one in the case that there are no closure cells, and one because
padding was eliminated.
2019-10-01 12:26:22 +10:00
Damien George b5ebfadbd6 py: Compress first part of bytecode prelude.
The start of the bytecode prelude contains 6 numbers telling the amount of
stack needed for the Python values and exceptions, and the signature of the
function.  Prior to this patch these numbers were all encoded one after the
other (2x variable unsigned integers, then 4x bytes), but using so many
bytes is unnecessary.

An entropy analysis of around 150,000 bytecode functions from the CPython
standard library showed that the optimal Shannon coding would need about
7.1 bits on average to encode these 6 numbers, compared to the existing 48
bits.

This patch attempts to get close to this optimal value by packing the 6
numbers into a single, varible-length unsigned integer via bit-wise
interleaving.  The interleaving scheme is chosen to minimise the average
number of bytes needed, and at the same time keep the scheme simple enough
so it can be implemented without too much overhead in code size or speed.
The scheme requires about 10.5 bits on average to store the 6 numbers.

As a result most functions which originally took 6 bytes to encode these 6
numbers now need only 1 byte (in 80% of cases).
2019-10-01 12:26:22 +10:00
Damien George 81d04a0200 py: Add n_state to mp_code_state_t struct.
This value is used often enough that it is better to cache it instead of
decode it each time.
2019-10-01 12:26:22 +10:00
Damien George 4c5e1a0368 py/bc: Change mp_code_state_t.exc_sp to exc_sp_idx.
Change from a pointer to an index, to make space in mp_code_state_t.
2019-10-01 12:26:22 +10:00
Damien George 1d7afcce49 py/bc: Remove comments referring to obsolete currently_in_except_block.
It was made obsolete in 6f9e3ff719
2019-10-01 12:26:22 +10:00
Jim Mussared 16f8ceeaaa extmod/modbluetooth: Add low-level Python BLE API. 2019-10-01 09:51:02 +10:00
Jim Mussared 42e9bdf19b py/ringbuf: Add helpers for put16/get16. 2019-10-01 09:51:02 +10:00
Damien George 5716c5cf65 py/persistentcode: Bump .mpy version to 5.
The bytecode opcodes have changed (there are more, and they have been
reordered).
2019-09-26 16:39:37 +10:00
Damien George 96f2a38075 py/nativeglue: Make mp_fun_table fixed size regardless of config.
So that mpy files with native code will always work correctly, and raise an
exception if a feature is used that is not supported by the runtime.
2019-09-26 16:24:06 +10:00
Josh Lloyd 7d58a197cf py: Rename MP_QSTR_NULL to MP_QSTRnull to avoid intern collisions.
Fixes #5140.
2019-09-26 16:04:56 +10:00
Damien George b596638b9b mpy-cross: Set number of registers in nlr_buf_t based on native arch.
Fixes #5059.  Done in collaboration with Jim Mussared.
2019-09-26 15:53:05 +10:00
Damien George 74503107a7 py/emitnative: Factor sizeof/offsetof calculations to macros. 2019-09-26 15:52:19 +10:00
Damien George a48cdb57b2 py/nlr.h: Factor out constants to specific macros. 2019-09-26 15:52:19 +10:00
Damien George 02db91a7a3 py: Split RAISE_VARARGS opcode into 3 separate ones.
From the beginning of this project the RAISE_VARARGS opcode was named and
implemented following CPython, where it has an argument (to the opcode)
counting how many args the raise takes:

    raise # 0 args (re-raise previous exception)
    raise exc # 1 arg
    raise exc from exc2 # 2 args (chained raise)

In the bytecode this operation therefore takes 2 bytes, one for
RAISE_VARARGS and one for the number of args.

This patch splits this opcode into 3, where each is now a single byte.
This reduces bytecode size by 1 byte for each use of raise.  Every byte
counts!  It also has the benefit of reducing code size (on all ports except
nanbox).
2019-09-26 15:39:50 +10:00
Damien George 870e900d02 py: Introduce and use constants for multi-opcode sizes. 2019-09-26 15:27:11 +10:00
Damien George 1f7202d122 py/bc: Replace big opcode format table with simple macro. 2019-09-26 15:27:11 +10:00
Damien George 5889cf58db py/bc0: Order opcodes into groups based on their size and format. 2019-09-26 15:27:10 +10:00
Damien George 2069c563f9 py: Add support for matmul operator @ as per PEP 465.
To make progress towards MicroPython supporting Python 3.5, adding the
matmul operator is important because it's a really "low level" part of the
language, being a new token and modifications to the grammar.

It doesn't make sense to make it configurable because 1) it would make the
grammar and lexer complicated/messy; 2) no other operators are
configurable; 3) it's not a feature that can be "dynamically plugged in"
via an import.

And matmul can be useful as a general purpose user-defined operator, it
doesn't have to be just for numpy use.

Based on work done by Jim Mussared.
2019-09-26 15:12:39 +10:00
Damien George 14e203282a py/compile: Use calculation instead of switch to convert token to op. 2019-09-26 14:37:26 +10:00
Damien George 9bf2feba63 py/parse: Use calculation instead of table to convert token to operator. 2019-09-26 14:37:26 +10:00
Damien George 6ce7c051e8 py/lexer: Reorder operator tokens to match corresponding binary ops. 2019-09-26 14:37:26 +10:00
stijn 22131a6738 py/persistentcode: Enable persistent code saving for Windows ports. 2019-09-18 22:15:36 +10:00
Damien George 6e07fde895 py/mkrules.mk: Add QSTR_GLOBAL_REQUIREMENTS variable for qstr auto-gen. 2019-09-12 18:16:16 +10:00
Damien George ea060a42e9 py/vm: Factor cached map lookup code to inline function.
To reduce code duplication and allow to more easily modify this function.
2019-09-10 11:23:52 +10:00
Damien George fb73bdae62 py/mkenv.mk: Add GDB variable. 2019-09-05 22:59:06 +10:00
Damien George b29fae0c56 py/bc: Fix size calculation of UNWIND_JUMP opcode in mp_opcode_format.
Prior to this patch mp_opcode_format would calculate the incorrect size of
the MP_BC_UNWIND_JUMP opcode, missing the additional byte.  But, because
opcodes below 0x10 are unused and treated as bytes in the .mpy load/save
and freezing code, this bug did not show any symptoms, since nested unwind
jumps would rarely (if ever) reach a depth of 16 (so the extra byte of this
opcode would be between 0x01 and 0x0f and be correctly loaded/saved/frozen
simply as an undefined opcode).

This patch fixes this bug by correctly accounting for the additional byte.
        .
2019-09-02 13:30:16 +10:00
Damien George c348e79187 py/binary: Change mp_uint_t to size_t for index, size, align args.
Reduces code size for nan-box builds, otherwise changes nothing.
2019-09-02 13:14:27 +10:00
Damien George 24c3e9b283 py/modstruct: Fix struct.pack_into with unaligned offset of native type.
Following the same fix for unpack.
2019-09-02 13:14:16 +10:00
Tom McDermott 1022f9cc35 py/modstruct: Fix struct.unpack with unaligned offset of native type.
With this patch alignment is done relative to the start of the buffer that
is being unpacked, not the raw pointer value, as per CPython.

Fixes issue #3314.
2019-09-02 13:10:55 +10:00
Jeff Epler 12f13ee634 py/objtuple: Allow compatible subclasses of tuple in mp_obj_tuple_get.
As part of this patch a private macro mp_obj_is_tuple_compatible is
introduced to encapsulate the check, which is used in two locations.

Fixes #5005.
2019-08-31 00:14:11 +10:00
Milan Rossa b295df4b08 py/profile: Add debugging for sys.settrace feature. 2019-08-30 16:48:47 +10:00
Milan Rossa 310b3d1b81 py: Integrate sys.settrace feature into the VM and runtime.
This commit adds support for sys.settrace, allowing to install Python
handlers to trace execution of Python code.  The interface follows CPython
as closely as possible.  The feature is disabled by default and can be
enabled via MICROPY_PY_SYS_SETTRACE.
2019-08-30 16:44:12 +10:00
Milan Rossa c96aedad46 py/profile: Add initial implementation of sys.settrace feature. 2019-08-30 16:43:56 +10:00
Damien George dbf35d3da3 py/bc: Factor out code to get bytecode line number info into new func. 2019-08-30 16:43:46 +10:00
Damien George c7c6703950 py/compile: Improve the line numbering precision for lambdas.
Prior to this patch the line number for a lambda would be "line 1" if the
body of the lambda contained only a simple expression (with no line number
stored in the parse node).  Now the line number is always reported
correctly.
2019-08-30 16:43:46 +10:00
Damien George af20c2ead3 py: Add global default_emit_opt variable to make emit kind persistent.
mp_compile no longer takes an emit_opt argument, rather this setting is now
provided by the global default_emit_opt variable.

Now, when -X emit=native is passed as a command-line option, the emitter
will be set for all compiled modules (included imports), not just the
top-level script.

In the future there could be a way to also set this variable from a script.

Fixes issue #4267.
2019-08-28 12:47:58 +10:00
Damien George 08c1fe5569 py/vm: Don't add traceback info for exceptions that are re-raised.
With this patch exceptions that are re-raised have improved tracebacks
(less confusing, match CPython), and it makes re-raise slightly more
efficient (in time and RAM) because they no longer need to add a traceback.
Also general VM performance is not measurably affected.

Partially fixes issue #2928.
2019-08-28 12:31:53 +10:00
Damien George 16f6169c88 py/vm: Don't add traceback info for exc's propagated through a finally.
With this patch exception tracebacks that go through a finally are improved
(less confusing, match CPython), and it makes finally's slightly more
efficient (in time and RAM) because they no longer need to add a traceback.

Partially fixes issue #2928.
2019-08-28 12:31:49 +10:00
Damien George 3d7455a0bb py/py.mk: Remove trailing spaces at end of line. 2019-08-27 16:37:43 +10:00
Damien George 2fca0d7f18 py/vm: Shorten error message for not-implemented opcode.
It's really an opcode that's not implemented, so use "opcode" instead of
"byte code".  And remove the redundant "not implemented" text because that
is already implied by the exception type.  There's no need to have a long
error message for an exception that is almost never encountered.  Saves
about 20 bytes of code size on most ports.
2019-08-22 16:13:05 +10:00
Damien George 53527138a9 py/bc0.h: Add comment that MP_BC_MAKE_CLOSURE/_DEFARGS take extra byte. 2019-08-22 16:07:28 +10:00
Damien George 973c87d8fa py/objgenerator: Move defn of mp_const_GeneratorExit_obj here.
Because the mp_obj_exception_t type is now globally available.
2019-08-22 16:05:05 +10:00
Damien George bc9b656f35 py/runtime: Remove obsolete comment about mp_parse_compile_execute.
mp_locals_get/set and mp_globals_get/set are now static-inline functions so
this comment is no longer correct.
2019-08-22 15:59:14 +10:00
Damien George 8e7745eb31 py/emitbc: Make all emit_write_bytecode_* funcs take a stack_adj arg.
This factoring of code gives significant code-size savings:

   bare-arm:  -456 -0.682%
minimal x86:  -844 -0.547%
   unix x64:  -472 -0.095%
unix nanbox: -1348 -0.303%
      stm32:  -472 -0.130% PYBV10
     cc3200:  -448 -0.242%
    esp8266:  -708 -0.108%
      esp32:  -400 -0.036% GENERIC
        nrf:  -520 -0.356% pca10040
       samd:  -456 -0.448% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2019-08-22 15:32:26 +10:00
Damien George 3d9bd80447 py/emitbc: Rewrite switch in load_const_tok to reduce code size. 2019-08-22 15:10:25 +10:00
Milan Rossa ae6fe8b43c py/compile: Improve the line numbering precision for comprehensions.
The line number for comprehensions is now always reported as the correct
global location in the script, instead of just "line 1".
2019-08-19 23:50:30 +10:00
Damien George 7d851a27f1 extmod/modure: Make regex dump-code debugging feature optional.
Enabled via MICROPY_PY_URE_DEBUG, disabled by default (but enabled on unix
coverage build).  This is a rarely used feature that costs a lot of code
(500-800 bytes flash).  Debugging of regular expressions can be done
offline with other tools.
2019-08-19 16:43:00 +10:00
Damien George 11ecdf2ec6 py/nlr: Use MP_UNREACHABLE at the end of arch-specific nlr_jump funcs.
Recent versions of gcc perform optimisations which can lead to the
following code from the MP_NLR_JUMP_HEAD macro being omitted:

    top->ret_val = val; \
    MP_NLR_RESTORE_PYSTACK(top); \
    *_top_ptr = top->prev; \

This is noticeable (at least) in the unix coverage on x86-64 built with gcc
9.1.0.  This is because the nlr_jump function is marked as no-return, so
gcc deduces that the above code has no effect.

Adding MP_UNREACHABLE tells the compiler that the asm code may branch
elsewhere, and so it cannot optimise away the code.
2019-08-19 16:14:57 +10:00
Damien George 0c80cb39af py: Introduce MP_UNREACHABLE macro to annotate unreachable code.
And use it to replace the same pattern at the end of nlrthumb.c:nlr_jump.
2019-08-19 16:14:33 +10:00
stijn af5c998f37 py/modmath: Implement math.isclose() for non-complex numbers.
As per PEP 485, this function appeared in for Python 3.5.  Configured via
MICROPY_PY_MATH_ISCLOSE which is disabled by default, but enabled for the
ports which already have MICROPY_PY_MATH_SPECIAL_FUNCTIONS enabled.
2019-08-17 23:23:17 +10:00
Damien George acfbb9febd py/objarray: Fix amount of free space in array when doing slice assign.
Prior to this patch the amount of free space in an array (including
bytearray) was not being maintained correctly for the case of slice
assignment which changed the size of the array.  Under certain cases (as
encoded in the new test) it was possible that the array could grow beyond
its allocated memory block and corrupt the heap.

Fixes issue #4127.
2019-08-15 23:02:04 +10:00
Milan Rossa cb3647004f py: Implement new sys.atexit feature.
This patch implements a new sys.atexit function which registers a function
that is later executed when the main script ends.  It is configurable via
MICROPY_PY_SYS_ATEXIT, disabled by default.

This is not compliant with CPython, rather it can be used to implement a
CPython compatible "atexit" module if desired (similar to how
sys.print_exception can be used to implement functionality of the
"traceback" module).
2019-08-15 17:30:50 +10:00
Milan Rossa efdcd6baa7 py/showbc: Fix off-by-one when showing address of unknown opcode. 2019-08-06 16:08:39 +10:00
Damien George cd35dd9d9a py: Allow to pass in read-only buffers to viper and inline-asm funcs.
Fixes #4936.
2019-08-06 15:58:23 +10:00
Paul m. p. P f60229e261 py/modio: Call mp_import_name to do resource stream import.
So code is not duplicated and it can take advantage of __import__ being
overridden.
2019-07-31 22:37:44 +10:00
Paul m. p. P 60f1063797 py/runtime: Allow to override builtins.__import__ with Python func.
This patch adds a simple but powerful hook into the import system, in a
CPython compatible way, by allowing to override builtins.__import__.

This does introduce some overhead to all imports but it's minor:
- the dict lookup of __import__ is bypassed if there are no modifications
  to the builtins module (which is the case at start up);
- imports are not performance critical, usually done just at the start of a
  script;
- compared to how much work is done in an import, looking up a value in a
  dict is a relatively small additional piece of work.
2019-07-31 22:36:00 +10:00
Paul m. p. P a8e3201b37 py/builtinimport: Populate __file__ when importing frozen or mpy files.
Note that bytecode already includes the source filename as a qstr so there
is no additional memory used by the interning operation here.
2019-07-31 17:00:11 +10:00
Eric Poulsen 01054f2092 py/objdict: Quote non-string types when used as keys in JSON output.
JSON requires that keys of objects be strings.  CPython will therefore
automatically quote simple types (NoneType, bool, int, float) when they are
used directly as keys in JSON output.  To prevent subtle bugs and emit
compliant JSON, MicroPython should at least test for such keys so they
aren't silently let through.  Then doing the actual quoting is a similar
cost to raising an exception, so that's what is implemented by this patch.

Fixes issue #4790.
2019-07-30 16:34:27 +10:00
Yonatan Goldschmidt e9593d5075 py/sequence: Fix grammar in comment about equality. 2019-07-25 16:37:25 +10:00
Paul m. p. P 3b3a4749ce py/objstringio: Guard bytesio_stream_p struct w/ MICROPY_PY_IO_BYTESIO.
It's static and can lead to a compilation warning/error when
MICROPY_PY_IO_BYTESIO is disabled.
2019-07-17 16:12:19 +10:00
Jim Mussared bc66fe9064 py/scheduler: Rename sched_stack to sched_queue.
Behaviour was changed from stack to queue in
8977c7eb58, and this updates variable names
to match.  Also updates other references (docs, error messages).
2019-07-17 16:09:32 +10:00
Paul m. p. P 154062d9cb py/makeqstrdata.py: Allow using \r\n as a qstr if a port requires it. 2019-07-12 17:10:11 +10:00
David Lechner 82dc9856b0 py/asmarm: Use __builtin___clear_cache instead of __clear_cache.
__clear_cache causes a compile error when using clang.  Instead use
__builtin___clear_cache which is available under both gcc and clang.

Also replace tabs with spaces in this section of code (introduced by a
previous commit).
2019-07-12 16:48:37 +10:00
Laurens Valk a73859d5af py/objgenerator: Add missing #if guard for PY_GENERATOR_PEND_THROW.
Without it, gen_instance_pend_throw_obj is defined but not used when
MICROPY_PY_GENERATOR_PEND_THROW is set to 0.
2019-07-09 13:24:23 +10:00
David Lechner f3a5b313e5 py/nlrthumb: Check __thumb2__ instead of __ARM_ARCH_6M__.
This fixes compiling for older architectures (e.g. armv5tej).

According to [1], the limit of R0-R7 for the STR and LDR instructions is
tied to the Thumb instruction set and not any specific processor
architectures.

[1]: http://www.keil.com/support/man/docs/armasm/armasm_dom1361289906890.htm
2019-07-03 01:24:22 +10:00
David Lechner 62b00dd5d8 py/asmarm: Use __clear_cache on Linux/GCC when creating new asm code.
Comes from https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/posts/caches-and-self-modifying-code

This fixes a crash when running MicroPython using qemu-arm.
2019-07-03 01:19:41 +10:00
Paul m. p. P 2920d26af5 py/persistentcode: Ensure prelude_offset is always initialised. 2019-07-01 23:46:49 +10:00
Damien George b7da67cdaa lib/utils/sys_stdio_mphal: Add support to poll sys.stdin and sys.stdout.
A port must provide the following function for this to work:

    uintptr_t mp_hal_stdio_poll(uintptr_t poll_flags);
2019-07-01 17:10:12 +10:00
Jun Wu d165a401dc py/persistentcode: Fix compilation with load and save both enabled.
With both MICROPY_PERSISTENT_CODE_SAVE and MICROPY_PERSISTENT_CODE_LOAD
enabled the code fails to compile, due to undeclared 'n_obj'.  If
MICROPY_EMIT_NATIVE is disabled there are more errors due to the use of
undefined fields in mp_raw_code_t.

This patch fixes such compilation by avoiding undefined fields.

MICROPY_EMIT_NATIVE was changed to MICROPY_EMIT_MACHINE_CODE in this file
to match the mp_raw_code_t definition.
2019-06-28 13:59:45 +10:00
Jun Wu b152bbddd1 py: Define EMIT_MACHINE_CODE as EMIT_NATIVE || EMIT_INLINE_ASM.
The combination MICROPY_EMIT_NATIVE || MICROPY_EMIT_INLINE_ASM is used in
many places, so define a new macro for it.
2019-06-28 13:54:45 +10:00
Paul m. p. P 862cc45a9c py/mkrules.mk: Use $(CPP) not $(CC) -E for preprocessor rule. 2019-06-25 15:03:41 +10:00
Damien George 34c04d2319 py/nlrthumb: Save and restore VFP registers s16-s21 when CPU has them.
These s16-s21 registers are used by gcc so need to be saved.  Future
versions of gcc (beyond v9.1.0), or other compilers, may eventually need
additional registers saved/restored.

See issue #4844.
2019-06-19 14:53:17 +10:00
Damien George cd6b115815 extmod: Factor out makefile rules from py.mk to new extmod.mk file.
To logically separate extmod related rules out, and prevent py.mk from
growing too large.
2019-06-05 14:23:12 +10:00
Yonatan Goldschmidt 7cf26ca4bd py/obj: Optimise small-int comparison to 0 in mp_obj_is_true.
Instead of converting to a small-int at runtime this can be done at compile
time, then we only have a simple comparison during runtime.  This reduces
code size on some ports (e.g -4 on qemu-arm, -52 on unix nanbox), and for
others at least doesn't increase code size.
2019-06-05 10:54:23 +10:00
Damien George 4173950658 mpy-cross: Do not automatically build mpy-cross, rather do it manually.
Building mpy-cross automatically leads to some issues with the build
process and slows it down.  Instead, require it to be built manually.
2019-06-03 14:44:44 +10:00
Damien George a4f1d82757 py/nativeglue: Remove dependency on mp_fun_table in dyn-compiler mode.
mpy-cross uses MICROPY_DYNAMIC_COMPILER and MICROPY_EMIT_NATIVE but does
not actually need to execute native functions, and does not need
mp_fun_table.  This commit makes it so mp_fun_table and all its entries are
not built when MICROPY_DYNAMIC_COMPILER is enabled, significantly reducing
the size of the mpy-cross executable and allowing it to be built on more
machines/OS's.
2019-05-29 21:17:29 +10:00
Damien George bff4e13009 py/nativeglue: Make private glue funs all static, remove commented code. 2019-05-29 21:14:24 +10:00
Damien George 6f75c4f3cd all: Bump version to 1.11. 2019-05-29 16:38:10 +10:00
Damien George ab26553759 py/vm: Remove obsolete comments about matching of exception opcodes.
These are incorrect since 5a2599d962
2019-05-27 11:58:32 +10:00
Sebastien Rinsoz 6cf4e9675b py/mkrules.mk: Remove unnecessary ; in makefile.
This ; make Windows compilation fail with GNU makefile 4.2.1.  It was added
in 0dc85c9f86 as part of a shell if-
statement, but this if-statement was subsequently removed in
23a693ec2d so the semicolon is not needed.
2019-05-22 12:57:22 +10:00
Sebastien Rinsoz a4f4239e95 py: Update makefiles to use $(TOUCH) instead of hard coded "touch".
The variable $(TOUCH) is initialized with the "touch" value in mkenv.mk
like for the other command line tools (rm, echo, cp, mkdir etc).  With
this, for example, Windows users can specify the path of touch.exe.
2019-05-22 12:56:40 +10:00
Sébastien Rinsoz c03f81c633 py: Update makefiles to use $(CAT) variable instead of hard coded "cat".
The variable $(CAT) is initialised with the "cat" value in mkenv.mk like
for the other command line tools (rm, echo, cp, mkdir etc).  With this,
for example, Windows users can specify the path of cat.exe.
2019-05-21 14:26:28 +10:00
stijn fb54736bdb py/objarray: Add decode method to bytearray.
Reuse the implementation for bytes since it works the same way regardless
of the underlying type.  This method gets added for CPython compatibility
of bytearray, but to keep the code simple and small array.array now also
has a working decode method, which is non-standard but doesn't hurt.
2019-05-21 14:24:04 +10:00
Damien George 653e1756c0 various: Update early copyright years to match actual edit history. 2019-05-17 18:06:11 +10:00
Paul Sokolovsky 016d9a40fe various: Add and update my copyright line based on git history.
For modules I initially created or made substantial contributions to.
2019-05-17 18:04:15 +10:00
stijn 90fae9172a py/objarray: Add support for memoryview.itemsize attribute.
This allows figuring out the number of bytes in the memoryview object as
len(memview) * memview.itemsize.

The feature is enabled via MICROPY_PY_BUILTINS_MEMORYVIEW_ITEMSIZE and is
disabled by default.
2019-05-14 17:15:17 +10:00
Henrik Vendelbo ab93321e31 py/persistentcode: Change "len" type to size_t for mp_obj_str_get_data. 2019-05-13 12:38:06 +10:00
Damien George c0a1de3c21 py/misc.h: Rename _MP_STRINGIFY to not use leading underscore in ident.
Macro identifiers with a leading underscore are reserved.
2019-05-09 17:11:33 +10:00
Damien George 4268d0e1ac py/objgenerator: Remove unneeded forward decl and clean up white space. 2019-05-09 13:49:07 +10:00
Damien George dac9d47671 py/objgenerator: Fix handling of None passed as 2nd arg to throw().
Fixes issue #4527.
2019-05-09 13:40:28 +10:00
Damien George 7e90e22ea5 mpy-cross: Add --version command line option to print version info.
Prints something like:

MicroPython v1.10-304-g8031b7a25 on 2019-05-02; mpy-cross emitting mpy v4
2019-05-07 13:54:20 +10:00
Jun Wu 089c9b71d1 py: remove "if (0)" and "if (false)" branches.
Prior to this commit, building the unix port with `DEBUG=1` and
`-finstrument-functions` the compilation would fail with an error like
"control reaches end of non-void function".  This change fixes this by
removing the problematic "if (0)" branches.  Not all branches affect
compilation, but they are all removed for consistency.
2019-05-06 18:28:28 +10:00
Yonatan Goldschmidt ef9843653b extmod/moducryptolib: Add AES-CTR support.
Selectable at compile time via MICROPY_PY_UCRYPTOLIB_CTR.  Disabled by
default.
2019-05-06 18:09:48 +10:00
Damien George 5ea38e4d74 py/native: Improve support for bool type in viper functions.
Variables with type bool now act more like an int, and there is proper
casting to/from Python objects.
2019-05-03 23:18:30 +10:00
Damien George 9ef784dcc6 py/asmthumb: Support asm_thumb code running on normal ARM processors.
With this change, @micropython.asm_thumb functions will work on standard
ARM processors (that are in ARM state by default), in scripts and
precompiled .mpy files.

Addresses issue #4675.
2019-05-01 15:24:21 +10:00
Damien George 27d22d8712 py/mpprint: Support printing %ld and %lu formats on 64-bit archs.
Fixes issue #4702.
2019-04-23 12:40:15 +10:00
Damien George 9ce25d7022 py/runtime: Fix mp_unpack_ex so seq can't be reclaimed by GC during use.
The issue described in the comment added here can be seen by forcing a
gc_collect() at the start of each call to gc_alloc().
2019-04-15 11:30:19 +10:00
Damien George 3fa06cf61e py/objset: Remove unused forward declaration and clean up whitespace. 2019-04-15 11:14:22 +10:00
Damien George 1754c71f45 py/runtime: Optimise to not create temp float for int to power negative. 2019-04-15 11:04:59 +10:00
Damien George 673e154dfe py/makedefs: Use io.open with utf-8 encoding when processing source.
In case (user) source code contains utf-8 encoded data and the default
locale is not utf-8.

See #4592.
2019-04-12 11:34:52 +10:00
Romain Goyet dce785cc3d py/nlrthumb: Add support for iOS where the C func is _nlr_push_tail. 2019-03-26 16:48:11 +11:00
Andrew Leech 8977c7eb58 py/scheduler: Convert micropythyon.schedule() to a circular buffer.
This means the schedule operates on a first-in, first-executed manner
rather than the current last-in, first executed.
2019-03-26 16:35:42 +11:00
Damien George 440462b18e py/runtime: Remove long-obsolete MICROPY_FSUSERMOUNT init code.
In 1808b2e8d5 it was replaced by MICROPY_VFS
and related code.
2019-03-20 00:16:37 +11:00
Damien George 5a6026c614 py/compile: Check that arch is set when compiling native, viper or asm. 2019-03-14 12:22:25 +11:00
Damien George 55fcb83a42 py/compile: Support multiple inline asm emitters. 2019-03-14 12:22:25 +11:00
Damien George d9d92f27d7 py/compile: Add support to select the native emitter at runtime. 2019-03-14 12:22:25 +11:00
Damien George 0e4c24ec08 py/nativeglue: Rename native convert funs to match other native helpers. 2019-03-14 12:22:25 +11:00
Damien George 3b973a5658 py: Move mp_native_type_from_qstr() from emitnative.c to nativeglue.c. 2019-03-14 12:22:25 +11:00
Andrew Leech 89ff506513 py: Update and rework build system for including external C modules.
How to use this feature is documented in docs/develop/cmodules.rst.
2019-03-08 22:58:42 +11:00
Ayke van Laethem 2e516074da py: Implement a module system for external, user C modules.
This system makes it a lot easier to include external libraries as static,
native modules in MicroPython.  Simply pass USER_C_MODULES (like
FROZEN_MPY_DIR) as a make parameter.
2019-03-08 22:49:00 +11:00
Andrew Leech cf22f4793c py: Allow registration of modules at their definition.
During make, makemoduledefs.py parses the current builds c files for
MP_REGISTER_MODULE(module_name, obj_module, enabled_define)

These are used to generate a header with the required entries for
"mp_rom_map_elem_t mp_builtin_module_table[]" in py/objmodule.c
2019-03-08 22:46:43 +11:00
Damien George 9a5f92ea72 py/persistentcode: Bump .mpy version to 4. 2019-03-08 15:53:05 +11:00
Damien George 1396a026be py: Add support to save native, viper and asm code to .mpy files.
This commit adds support for saving and loading .mpy files that contain
native code (native, viper and inline-asm).  A lot of the ground work was
already done for this in the form of removing pointers from generated
native code.  The changes here are mainly to link in qstr values to the
native code, and change the format of .mpy files to contain native code
blocks (possibly mixed with bytecode).

A top-level summary:

- @micropython.native, @micropython.viper and @micropython.asm_thumb/
  asm_xtensa are now allowed in .py files when compiling to .mpy, and they
  work transparently to the user.

- Entire .py files can be compiled to native via mpy-cross -X emit=native
  and for the most part the generated .mpy files should work the same as
  their bytecode version.

- The .mpy file format is changed to 1) specify in the header if the file
  contains native code and if so the architecture (eg x86, ARMV7M, Xtensa);
  2) for each function block the kind of code is specified (bytecode,
  native, viper, asm).

- When native code is loaded from a .mpy file the native code must be
  modified (in place) to link qstr values in, just like bytecode (see
  py/persistentcode.c:arch_link_qstr() function).

In addition, this now defines a public, native ABI for dynamically loadable
native code generated by other languages, like C.
2019-03-08 15:53:05 +11:00
Damien George 636ed0ff8d py/emitglue: Remove union in mp_raw_code_t to combine bytecode & native. 2019-03-08 15:53:04 +11:00
Damien George 3986820912 py/emitnative: Adjust accounting of size of const_table.
n_obj no longer includes a count for mp_fun_table to make it a bit simpler.
2019-03-08 15:53:04 +11:00
Damien George 205edb4305 py/emitnative: Provide concentrated points of qstr emit. 2019-03-08 15:53:04 +11:00
Damien George 01a1f31f67 py/emitnative: Consolidate where HASCONSTS is set to load-const-obj fun.
Simplifies the code and fixes handling of the Ellipsis const in native code
generation (which also needs the constant table so must set this flag).
2019-03-08 15:53:04 +11:00
Damien George 02cc288edb py: Add independent config for debugging sentinel object values.
The new compile-time option is MICROPY_DEBUG_MP_OBJ_SENTINELS, disabled by
default.  This is to allow finer control of whether this debugging feature
is enabled or not (because, for example, this setting must be the same for
mpy-cross and the MicroPython main code when using native code generation).
2019-03-08 15:53:04 +11:00
Damien George 4f0931b21f py/persistentcode: Define static qstr set to reduce size of mpy files.
When encoded in the mpy file, if qstr <= QSTR_LAST_STATIC then store two
bytes: 0, static_qstr_id.  Otherwise encode the qstr as usual (either with
string data or a reference into the qstr window).

Reduces mpy file size by about 5%.
2019-03-05 16:32:05 +11:00
Damien George 992a6e1dea py/persistentcode: Pack qstrs directly in bytecode to reduce mpy size.
Instead of emitting two bytes in the bytecode for where the linked qstr
should be written to, it is now replaced by the actual qstr data, or a
reference into the qstr window.

Reduces mpy file size by about 10%.
2019-03-05 16:27:34 +11:00
Damien George 5996eeb48f py/persistentcode: Add a qstr window to save mpy files more efficiently.
This is an implementation of a sliding qstr window used to reduce the
number of qstrs stored in a .mpy file.  The window size is configured to 32
entries which takes a fixed 64 bytes (16-bits each) on the C stack when
loading/saving a .mpy file.  It allows to remember the most recent 32 qstrs
so they don't need to be stored again in the .mpy file.  The qstr window
uses a simple least-recently-used mechanism to discard the least recently
used qstr when the window overflows (similar to dictionary compression).
This scheme only needs a single pass to save/load the .mpy file.

Reduces mpy file size by about 25% with a window size of 32.
2019-03-05 16:25:07 +11:00
Damien George 5a2599d962 py: Replace POP_BLOCK and POP_EXCEPT opcodes with POP_EXCEPT_JUMP.
POP_BLOCK and POP_EXCEPT are now the same, and are always followed by a
JUMP.  So this optimisation reduces code size, and RAM usage of bytecode by
two bytes for each try-except handler.
2019-03-05 16:09:58 +11:00
Damien George 6f9e3ff719 py/vm: Remove currently_in_except_block variable.
After the previous commit it is no longer needed.
2019-03-05 16:09:41 +11:00
Damien George e1fb03f3e2 py: Fix VM crash with unwinding jump out of a finally block.
This patch fixes a bug in the VM when breaking within a try-finally.  The
bug has to do with executing a break within the finally block of a
try-finally statement.  For example:

    def f():
        for x in (1,):
            print('a', x)
            try:
                raise Exception
            finally:
                print(1)
                break
            print('b', x)
    f()

Currently in uPy the above code will print:

    a 1
    1
    1
    segmentation fault (core dumped)  micropython

Not only is there a seg fault, but the "1" in the finally block is printed
twice.  This is because when the VM executes a finally block it doesn't
really know if that block was executed due to a fall-through of the try (no
exception raised), or because an exception is active.  In particular, for
nested finallys the VM has no idea which of the nested ones have active
exceptions and which are just fall-throughs.  So when a break (or continue)
is executed it tries to unwind all of the finallys, when in fact only some
may be active.

It's questionable whether break (or return or continue) should be allowed
within a finally block, because they implicitly swallow any active
exception, but nevertheless it's allowed by CPython (although almost never
used in the standard library).  And uPy should at least not crash in such a
case.

The solution here relies on the fact that exception and finally handlers
always appear in the bytecode after the try body.

Note: there was a similar bug with a return in a finally block, but that
was previously fixed in b735208403
2019-03-05 16:05:05 +11:00
Damien George 871954d75c py/py.mk: Update lwip build config to work with latest lwip version.
Also, to make it possible for ports to provide their own lwipopts.h, the
default include directory of extmod/lwip-include is no longer added and
instead a port should now make sure the correct include directory is
included in the list (can still use extmod/lwip-include).
2019-03-04 23:29:01 +11:00
Tom Collins 2d644ac455 py/objexcept: Fix hash of exc str created in mp_obj_new_exception_msg. 2019-03-04 12:07:03 +11:00
Damien George 0779693c23 py/compile: Add optimisation to compile OrderedDict inplace.
This optimisation eliminates the need to create a temporary normal dict.
The optimisation is enabled via MICROPY_COMP_CONST_LITERAL which is enabled
by default (although only has an effect if OrderdDict is enabled).

Thanks to @pfalcon for the initial idea and implementation.
2019-03-01 15:22:46 +11:00
Damien George 12ce9f2689 py/compile: Fix handling of unwinding BaseException in async with.
All exceptions that unwind through the async-with must be caught and
BaseException is the top-level class, which includes Exception and others.

Fixes issue #4552.
2019-02-26 23:52:10 +11:00
Damien George 4ee2c2a4cd py: Eliminate warnings about unused arguments when debugging disabled. 2019-02-25 14:52:36 +11:00
Damien George 7bc71f5446 py/objfun: Make fun_data arg of mp_obj_new_fun_asm() a const pointer. 2019-02-20 13:14:03 +11:00
Damien George bf352047de py/obj.h: Remove obsolete mp_obj_new_fun_viper() declaration. 2019-02-20 13:06:35 +11:00
Damien George 2b575418b6 py/qstr: Evaluate find_qstr only once then pass to Q_GET_HASH macro.
Q_GET_HASH may evaluate its argument more than once.
2019-02-19 23:44:01 +11:00
Yonatan Goldschmidt bc4f8b438b extmod/moduwebsocket: Refactor `websocket` to `uwebsocket`.
As mentioned in #4450, `websocket` was experimental with a single intended
user, `webrepl`. Therefore, we'll make this change without a weak
link `websocket` -> `uwebsocket`.
2019-02-14 00:35:45 +11:00
Damien George 5368210e36 py/mkenv.mk: Change default PYTHON variable from "python" to "python3".
This change makes it so that python3 is required by default to build
MicroPython. Python 2 can be used by specifying make PYTHON=python2.

This comes about due to a recent-ish change to PEP 394 that makes the
python command more optional than before (even with Python 2 installed);
see cd59ec03c8 (diff-1d22f7bd72cbc900670f058b1107d426)

Since the command python is no longer required to be provided by a
distribution we need to use either python2 or python3 as commands.  And
python3 seems the obvious choice.
2019-02-12 14:58:15 +11:00
Damien George 054dd33eba py: Downcase MP_xxx_SLOT_IS_FILLED inline functions. 2019-02-12 14:54:51 +11:00
Damien George eee1e8841a py: Downcase all MP_OBJ_IS_xxx macros to make a more consistent C API.
These macros could in principle be (inline) functions so it makes sense to
have them lower case, to match the other C API functions.

The remaining macros that are upper case are:
- MP_OBJ_TO_PTR, MP_OBJ_FROM_PTR
- MP_OBJ_NEW_SMALL_INT, MP_OBJ_SMALL_INT_VALUE
- MP_OBJ_NEW_QSTR, MP_OBJ_QSTR_VALUE
- MP_OBJ_FUN_MAKE_SIG
- MP_DECLARE_CONST_xxx
- MP_DEFINE_CONST_xxx

These must remain macros because they are used when defining const data (at
least, MP_OBJ_NEW_SMALL_INT is so it makes sense to have
MP_OBJ_SMALL_INT_VALUE also a macro).

For those macros that have been made lower case, compatibility macros are
provided for the old names so that users do not need to change their code
immediately.
2019-02-12 14:54:51 +11:00
Yonatan Goldschmidt 343401c6df py/mpconfig.h: Fix comments mentioning dangling file and variable names. 2019-02-06 00:25:30 +11:00
Yonatan Goldschmidt ec31438c54 py/builtinhelp: Only print help re FS modules if external import enabled 2019-02-06 00:23:16 +11:00
Paul Sokolovsky 8fea833e3f py: Update my copyright info on some files.
Based on git history.
2019-02-06 00:19:00 +11:00
Paul Sokolovsky 2f5d113fad py/warning: Support categories for warnings.
Python defines warnings as belonging to categories, where category is a
warning type (descending from exception type). This is useful, as e.g.
allows to disable warnings selectively and provide user-defined warning
types.  So, implement this in MicroPython, except that categories are
represented just with strings.  However, enough hooks are left to implement
categories differently per-port (e.g. as types), without need to patch each
and every usage.
2019-01-31 16:48:30 +11:00
Damien George deb67569ff py/compile: Swap order of pop_block/pop_except in "except as" handler.
To make the try-finally block self contained.
2019-01-27 14:09:44 +11:00
stijn 42863830be py: Add optional support for 2-argument version of built-in next().
Configurable via MICROPY_PY_BUILTINS_NEXT2, disabled by default.
2019-01-27 13:01:28 +11:00
Sean Burton e33bc59712 py: Remove calls to file reader functions when these are disabled.
If MICROPY_PERSISTENT_CODE_LOAD or MICROPY_ENABLE_COMPILER are enabled then
code gets enabled that calls file reading functions which may be disabled
if no readers have been implemented.

To fix this, introduce a MICROPY_HAS_FILE_READER variable, which is
automatically set if MICROPY_READER_POSIX or MICROPY_READER_VFS is set but
can also be manually set if a custom reader is being implemented.  Then
disable the file reading calls if this is not set.
2019-01-27 11:08:25 +11:00
Damien George 3e25d611ef all: Bump version to 1.10. 2019-01-26 00:56:48 +11:00
Damien George aba83e66d7 py/mpconfig.h: Remove parentheses from MICROPY_VERSION_xxx macros.
Otherwise MICROPY_VERSION_STRING includes these parentheses in the string.
2019-01-26 00:44:35 +11:00
Damien George 5089b3ffb6 py/obj.h: Explicitly cast args to uint32_t in MP_OBJ_FUN_MAKE_SIG.
For architectures where size_t is less than 32 bits (eg 16 bits) the args
must be casted to uint32_t so the left shift will work.  For architectures
where size_t is greater than 32 bits (eg 64 bits) this new casting will not
lose any bits because the end result must anyway fit in a uint32_t.
2019-01-25 16:03:05 +11:00
Damien George 529dcce2be py/modio: Make iobase_singleton object const so it goes in ROM. 2019-01-10 23:08:07 +11:00
Damien George afecc124e6 py: Fix location of VM returned exception in invalid opcode and comments
The location for a returned exception was changed to state[0] in
d95947b48a
2019-01-04 17:22:40 +11:00
Damien George 6d19934463 py: Get optional VM stack overflow check compiling and working again.
Changes to the layout of the bytecode header meant that this debug code was
no longer compiling.  This is now fixed and a new compile-time option is
introduced, MICROPY_DEBUG_VM_STACK_OVERFLOW, to turn on this feature (which
is disabled by default).  This option is needed because more than one file
needs to cooperate to make this check work.
2019-01-04 17:09:41 +11:00
Damien George fa50047bbc py/runtime: Unlock the GIL in mp_deinit function.
This mirrors what is done in mp_init.  Some RTOSs require this symmetry to
get back to a clean state (when doing a soft reset, for example).
2018-12-27 14:20:31 +11:00
Damien George 7cd59c5bc3 py/mpconfig: Move MICROPY_VERSION macros to static ones in mpconfig.h.
It's more robust to have the version defined statically in a header file,
rather than dynamically generating it via git using a git tag.  In case
git doesn't exist, or a different source control tool is used, it's
important to still have the uPy version number available.
2018-12-22 01:40:38 +11:00
Paul Sokolovsky 5ed578e5b4 py/gc: Adjust gc_alloc() signature to be able to accept multiple flags.
The older "bool has_finaliser" gets recast as GC_ALLOC_FLAG_HAS_FINALISER=1
so this is a backwards compatible change to the signature.  Since bool gets
implicitly converted to 1 this patch doesn't include conversion of all
calls.
2018-12-20 17:52:16 +11:00
Paul Sokolovsky a261d8b615 py/objarray: Introduce "memview_offset" alias for "free" field of object
Both mp_type_array and mp_type_memoryview use the same object structure,
mp_obj_array_t, but for the case of memoryview, some fields, e.g. "free",
have different meaning.  As the "free" field is also a bitfield, assume
that (anonymous) union can't be used here (for the concerns of possible
compatibility issues with wide array of toolchains), and just add a field
alias using a #define.  As it's a define, it should be a selective
identifier, so use verbose "memview_offset" to avoid any clashes.
2018-12-20 17:40:48 +11:00
Damien George 0d165fec9c py/qstr: Put a lower bound on new qstr pool allocation. 2018-12-15 14:32:09 +11:00
Damien George 6bf8ecfe3a py/bc: Fix calculation of opcode size for opcodes with map caching.
All 4 opcodes that can have caching bytes also have qstrs, so the test for
them must go in the qstr part of the code.  The reason this incorrect
calculation of the opcode size did not lead to a bug is because the caching
byte is at the end of the opcode (byte, qstr, qstr, cache) and is always
0x00 when saving/loading, so was just treated as a single byte no-op
opcode.  Hence these opcodes were being saved/loaded/decoded correctly.

Thanks to @malinah for finding the problem and providing the initial patch.
2018-12-13 01:26:55 +11:00
Paul Sokolovsky fbb8335084 py/objdict: Make .fromkeys() method configurable.
On by default, turned off for minimal/bare-arm. Saves 144 bytes on x86.
2018-12-13 01:20:55 +11:00
Damien George 55830dd9bf py/objexcept: Make sure mp_obj_new_exception_msg doesn't copy/format msg
mp_obj_new_exception_msg() assumes that the message passed to it is in ROM
and so can use its data directly to create the string object for the
argument of the exception, saving RAM.  At the same time, this approach
also makes sure that there is no attempt to format the message with printf,
which could lead to faults if the message contained % characters.

Fixes issue #3004.
2018-12-10 16:01:05 +11:00
Damien George bad4e15da5 py/objexcept: Use macros to make offsets in emergency exc buf clearer. 2018-12-10 15:53:38 +11:00
Paul Sokolovsky 38151f35c1 extmod/moductypes: Add aliases for native C types.
SHORT, INT, LONG, LONGLONG, and unsigned (U*) variants are being defined.
This is done at compile using GCC-style predefined macros like
__SIZEOF_INT__.  If the compiler doesn't have such defines, no such types
will be defined.
2018-12-10 14:40:43 +11:00
Paul Sokolovsky b1d08726ee py/obj: Add support for __int__ special method.
Based on the discussion, this special method is available unconditionally,
as converting to int is a common operation.
2018-12-07 17:28:04 +11:00
Damien George 113f00a9ab py/objboundmeth: Support loading generic attrs from the method.
Instead of assuming that the method is a bytecode object, and only
supporting load of __name__, make the operation generic by delegating the
load to the method object itself.  Saves a bit of code size and fixes the
case of attempting to load __name__ on a native method, see issue #4028.
2018-12-06 18:02:41 +11:00
Ayke van Laethem 31cf528c75 py: Add option to reduce GC stack integer size to save RAM.
A new option MICROPY_GC_STACK_ENTRY_TYPE is added to select a custom type
instead of size_t for the gc_stack array items.  This can be beneficial for
small devices, especially those that are low on memory anyway.  If a device
has 1MB or less of heap (and 16-byte GC blocks) then this type can be
uint16_t, saving 128 bytes of RAM.
2018-12-04 17:17:25 +11:00
Craig Younkins 7f948a5645 py/py.mk: Fix broken Gmane URL. 2018-12-04 01:03:44 +11:00
Damien George 7c85c7c210 py/unicode: Fix check for valid utf8 being stricter about contn chars. 2018-11-26 16:13:08 +11:00
Paul Sokolovsky 5c18730f28 py/runtime: Fix qstr assumptions when handling "import *".
There was an assumption that all names in a module dict are qstr's.
However, they can be dynamically generated (by assigning to globals()),
and in case of a long name, it won't be a qstr. Handle this situation
properly, including taking care of not creating superfluous qstr's for
names starting with "_" (which aren't imported by "import *").
2018-11-01 13:33:16 +11:00
Damien George e328a5d469 py/scope: Optimise scope_find_or_add_id to not need "added" arg.
Taking the address of a local variable is mildly expensive, in code size
and stack usage.  So optimise scope_find_or_add_id() to not need to take a
pointer to the "added" variable, and instead take the kind to use for newly
added identifiers.
2018-10-28 00:38:18 +11:00
Damien George ba92c79841 py/compile: Remove unneeded variable from global/nonlocal stmt helpers. 2018-10-28 00:38:18 +11:00
Damien George 9201f46cc8 py/compile: Fix case of eager implicit conversion of local to nonlocal.
This ensures that implicit variables are only converted to implicit
closed-over variables (nonlocals) at the very end of the function scope.
If variables are closed-over when first used (read from, as was done prior
to this commit) then this can be incorrect because the variable may be
assigned to later on in the function which means they are just a plain
local, not closed over.

Fixes issue #4272.
2018-10-28 00:33:08 +11:00
Damien George 746dbf78d3 py/py.mk: When building axtls use -Wno-all to prevent all warnings.
Building axtls gives a lot of warnings with -Wall enabled, and explicitly
disabling all of them cannot be done in a way compatible with gcc and
clang, and likely other compilers.  So just use -Wno-all to prevent all of
the extra warnings (in addition to the necessary -Wno-unused-parameter,
-Wno-uninitialized, -Wno-sign-compare and -Wno-old-style-definition).

Fixes issue #4182.
2018-10-27 23:53:08 +11:00
Paul m. p. P 454cca6016 py/objmodule: Implement PEP 562's __getattr__ for modules.
Configurable via MICROPY_MODULE_GETATTR, disabled by default.  Among other
things __getattr__ for modules can help to build lazy loading / code
unloading at runtime.
2018-10-23 11:22:50 +11:00
Paul Sokolovsky 5a91fce9f8 py/objstr: Make str.count() method configurable.
Configurable via MICROPY_PY_BUILTINS_STR_COUNT.  Default is enabled.
Disabled for bare-arm, minimal, unix-minimal and zephyr ports.  Disabling
it saves 408 bytes on x86.
2018-10-22 22:49:05 +11:00
Damien George 7eb29c2000 py/objtype: Remove comment about catching exc from user __getattr__.
Any exception raised in a user __getattr__ should be propagated out.  A
test is added to verify these semantics.
2018-10-18 12:15:16 +11:00
Damien George de71035e02 py/emitnative: Put None/False/True in global native const table.
So these constant objects can be loaded by dereferencing the REG_FUN_TABLE
pointer instead of loading immediate values.  This reduces the size of
generated native code (when such constants are used), and means that
pointers to these constants are no longer stored in the assembly code.
2018-10-15 00:20:49 +11:00
Damien George 6c6050ca43 py/emitnative: Push internal None rather than const obj where possible.
This shifts the work of loading the constant None object on to
load_reg_stack_imm(), making the handling of None more centralised.
2018-10-15 00:20:49 +11:00
Damien George 7c16bc0406 py/emitnative: Simplify viper mode handling in emit_native_import_name. 2018-10-15 00:20:49 +11:00
Damien George 175739cd37 py/emitnative: Consolidate use of stacked immediate values to one func.
This commit adds the helper function load_reg_stack_imm() which deals with
constant immediate values and converting them to Python objects if needed.
2018-10-15 00:20:49 +11:00
Damien George 6bda951d4d py/emitnative: Remove unused ptr argument from ASM_CALL_IND macro. 2018-10-13 15:16:33 +11:00
Damien George 25571800fc py/asmthumb: Remove unused fun_ptr arg from asm_thumb_bl_ind function. 2018-10-13 15:16:33 +11:00
Damien George 5f1dd5b86b py/asmarm: Simplify asm_arm_bl_ind to only load via index, not literal.
The maximum index into mp_fun_table is currently less than 1024 and should
stay that way to keep things efficient for all architectures, so there is
no need to handle loading the pointer directly via a literal in this
function.
2018-10-13 15:16:33 +11:00
Damien George 006671056d py/emitnative: Load native fun table ptr from const table for all archs.
All architectures now have a dedicated register to hold the pointer to the
native function table mp_fun_table, and so they all need to load this
register at the start of the native function.  This commit makes the
loading of this register uniform across architectures by passing the
pointer in the constant table for the native function, and then loading the
register from the constant table.  Doing it this way means that the pointer
is not stored in the assembly code, helping to make the code more portable.
2018-10-13 15:16:33 +11:00
Damien George 355eb8eafb py/asmx86: Change indirect calls to load fun ptr from the native table.
Instead of storing the function pointer directly in the assembly code.
This makes the generated code more independent of the runtime (so easier to
relocate the code), and reduces the generated code size.
2018-10-13 15:16:33 +11:00
Damien George b7c6f859d0 py/asmx86: Change stack management to reference locals by esp not ebp.
The esp register is always a fixed distance below ebp, and using esp to
reference locals on the stack frees up the ebp register for general purpose
use (which is important for an architecture with only 8 user registers).
2018-10-13 15:16:33 +11:00
Damien George 8e4b4bac70 py/asmx64: Change indirect calls to load fun ptr from the native table.
Instead of storing the function pointer directly in the assembly code.
This makes the generated code more independent of the runtime (so easier to
relocate the code), and reduces the generated code size.
2018-10-13 15:16:33 +11:00
Damien George 8941c63290 py/asmx64: Change stack management to reference locals by rsp not rbp.
The rsp register is always a fixed distance below rbp, and using rsp to
reference locals on the stack frees up the rbp register for general purpose
use.
2018-10-13 15:16:33 +11:00
Damien George 34af10d2ef py/emitnative: Clean up unused macro and forward function declarations. 2018-10-02 15:01:56 +10:00
Damien George 69e7903904 py/obj.h: Use uint64_t instead of mp_int_t in repr-D MP_OBJ_IS_x macros.
This follows how it's already done in MP_OBJ_IS_OBJ: the objects are
considered 64-bit unsigned ints for the purpose of bitwise manipulation.
2018-10-01 16:36:46 +10:00
Damien George a9237cee82 py/runtime: Remove comment in mp_import_name about level being 0.
A non-zero level has been supported for some time now.
2018-10-01 15:35:10 +10:00
Damien George 4ab397576f py/runtime: Use mp_import_name to implement tail of mp_import_from. 2018-10-01 15:22:03 +10:00
Damien George cc2bd63c57 py/emitnative: Implement yield and yield-from in native emitter.
This commit adds first class support for yield and yield-from in the native
emitter, including send and throw support, and yields enclosed in exception
handlers (which requires pulling down the NLR stack before yielding, then
rebuilding it when resuming).

This has been fully tested and is working on unix x86 and x86-64, and
stm32.  Also basic tests have been done with the esp8266 port.  Performance
of existing native code is unchanged.
2018-10-01 13:31:11 +10:00
Damien George 8fec6f5434 py/emitnative: Reorder native state on C stack so nlr_buf_t is first.
The nlr_buf_t doesn't need to be part of the Python value stack (as it was
before this commit), it's simpler to have it separated as auxiliary state
that lives on the C stack.  This will help adding yield support because in
that case the nlr_buf_t and Python value stack live in separate memory
areas (C stack and heap respectively).
2018-10-01 12:36:21 +10:00
Damien George 4fc437f1ef py/asmxtensa: Use proper calculation for const table offset.
Instead of hard-coding it to 4 bytes.  This allows for there to be other
data stored at the very start of the emitted native code.
2018-10-01 12:34:58 +10:00
Damien George 5b19916d6e py/asmx64: Extend asm_x64_mov_reg_pcrel to accept high registers. 2018-10-01 12:34:36 +10:00
Damien George 1dc720dc01 py/asmx86: Comment out unused asm_x86_nop to prevent compiler warnings. 2018-10-01 12:34:23 +10:00
Damien George 87231132d4 py/asmthumb: Extend asm entry/exit to handle stack larger than 508 bytes 2018-09-30 23:31:17 +10:00
Damien George ef9394e76a py/asmthumb: Clean up asm_thumb_bl_ind to use new optimised ldr helper. 2018-09-30 23:30:18 +10:00
Damien George 07ccb192c5 py/asmthumb: Add wide ldr to handle larger offsets.
In particular this allows native functions on Thumb2 to index more than 32
constants in the constant table.
2018-09-30 23:27:01 +10:00
Damien George d95947b48a py/vm: When VM raises exception put exc obj at beginning of func state.
Instead of at end of state, n_state - 1.  It was originally (way back in
v1.0) put at the end of the state because the VM didn't have a pointer to
the start.  But now that the VM takes a mp_code_state_t pointer it does
have a pointer to the start of the state so can put the exception object
there.

This commit saves about 30 bytes of code on all architectures, and, more
importantly, reduces C-stack usage by a couple of words (8 bytes on Thumb2
and 16 bytes on x86-64) for every (non-generator) call of a bytecode
function because fun_bc_call no longer needs to remember the n_state
variable.
2018-09-29 23:25:08 +10:00
Damien George dd288904db py/objtype: Support full object model for get/set/delitem special meths.
This makes these special methods have the same calling behaviour as other
methods in a class instance (mp_convert_member_lookup() is already called
by mp_obj_class_lookup()).
2018-09-28 23:22:34 +10:00
Damien George 2eb0170157 py/objtype: Remove TODO about storing attributes to classes.
This behaviour is tested in basics/class_store.py and follows CPython.
2018-09-28 23:15:12 +10:00
Damien George 2c7a3061d5 py/runtime: Remove nlr protection when calling __next__ in mp_resume.
And remove related comment about needing such protection when calling send.

Reasoning for removal is as follows:
- mp_resume is only called by the VM in YIELD_FROM opcode
- if send_value != MP_OBJ_NULL then throw_value == MP_OBJ_NULL
- so if __next__ or send are called then throw_value == MP_OBJ_NULL
- if __next__ or send raise an exception without nlr protection then the
  exception will be handled by the global exception handler of the VM
- this handler already has code to handle exceptions raised in YIELD_FROM,
  including correct handling of StopIteration
- this handler doesn't handle the case of injection of GeneratorExit, but
  this won't be needed because throw_value == MP_OBJ_NULL

Note that it's already possible for mp_resume() to raise an exception
(including StopIteration) from the unprotected call to type->iternext(), so
that's why the VM already has code to handle the case of exceptions coming
out of mp_resume().

This commit reduces code size by a bit, and significantly reduces C stack
usage when using yield-from, from 88 bytes down to 40 for Thumb2, and 152
down to 72 bytes for x86-64 (better than half).  (Note that gcc doesn't
seem to tail-call optimise the call from mp_resume() to mp_obj_gen_resume()
so this saving in C stack usage helps all uses of yield-from.)
2018-09-28 22:16:56 +10:00
Damien George 0c9d452370 py/vm: Fix case of throwing GeneratorExit type into yield-from.
mp_make_raise_obj must be used to convert a possible exception type to an
instance object, otherwise the VM may raise a non-exception object.

An existing test is adjusted to test this case, with the original test
already moved to generator_throw.py.
2018-09-28 11:39:35 +10:00
Damien George e9012a20f7 py/emitnative: Change type of const_table from uintptr_t to mp_uint_t.
This matches how bytecode does it, and matches the signature of
mp_emit_glue_assign_native.  Since the native emitter doesn't support
nan-boxing uintptr_t and mp_uint_t are anyway the same bit-width.
2018-09-28 00:04:10 +10:00
Damien George 2e86233263 py/asm*: Remove ASM_MOV_REG_ALIGNED_IMM emit macro, it's no longer used.
After the previous commit this macro is no longer needed by the native
emitter because live heap pointers are no longer stored in generated native
machine code.
2018-09-27 23:39:08 +10:00
Damien George 7d4b6cc868 py/emitnative: Place const objs for native code in separate const table.
This commit changes native code to handle constant objects like bytecode:
instead of storing the pointers inside the native code they are now stored
in a separate constant table (such pointers include objects like bignum,
bytes, and raw code for nested functions).  This removes the need for the
GC to scan native code for root pointers, and takes a step towards making
native code independent of the runtime (eg so it can be compiled offline by
mpy-cross).

Note that the changes to the struct scope_t did not increase its size: on a
32-bit architecture it is still 48 bytes, and on a 64-bit architecture it
decreased from 80 to 72 bytes.
2018-09-27 23:39:08 +10:00
Damien George b3eadf3f3d py/objfloat: Fix abs(-0.0) so it returns 0.0.
Nan and inf (signed and unsigned) are also handled correctly by using
signbit (they were also handled correctly with "val<0", but that didn't
handle -0.0 correctly).  A test case is added for this behaviour.
2018-09-27 15:21:25 +10:00
Damien George fc1bb51af5 py/objgenerator: Remove TODO about returning gen being called again.
The code implements correct behaviour, as tested by the new test case added
in this commit.
2018-09-27 15:18:24 +10:00
Damien George 6d20be31ae py/vm: Reword TODO about invalid ip/sp after an exception to a note. 2018-09-27 15:17:37 +10:00
Damien George 04f7da78db py/objmodule: Remove TODO about checking store attr to a module.
The code implements correct behaviour, as tested by basics/module1.py.
2018-09-27 15:16:24 +10:00
Damien George cc5c3c64ca py/objint: Remove TODO about checking of int() arg types with 2 args.
The arguments are checked by mp_obj_str_get_data and mp_obj_get_int.
2018-09-27 15:15:29 +10:00
Damien George 814f17a3a4 py/objdict: Reword TODO about inlining mp_obj_dict_get to a note. 2018-09-27 15:14:12 +10:00
Damien George baa83a0c6d py/objslice: Remove long-obsolete comment about enhancing slice object.
Commit afaaf535e6 made this comment obsolete.
2018-09-27 11:23:31 +10:00
Damien George 76355c8863 py/vm: Make small optimisation of BUILD_SLICE opcode.
No need to call DECODE_UINT since the value will always be either 2 or 3.
2018-09-27 11:22:33 +10:00
stijn 57a7d5be9a py: Fix msvc C++ compiler warnings with MP_OBJ_FUN_MAKE_SIG macro.
When obj.h is compiled as C++ code, the cl compiler emits a warning about
possibly unsafe mixing of size_t and bool types in the or operation in
MP_OBJ_FUN_MAKE_SIG.  Similarly there's an implicit narrowing integer
conversion in runtime.h.  This commit fixes this by being explicit.
2018-09-26 15:34:59 +10:00
Paul Sokolovsky a135bca4a1 py/objstr: format: Return bytes result for bytes format string.
This is an improvement over previous behavior when str was returned for
both str and bytes input format.  This new behaviour is also consistent
with how the % operator works, as well as many other str/bytes methods.

It should be noted that it's not how current versions of CPython work,
where there's a gap in the functionality and bytes.format() is not
supported.
2018-09-26 15:29:41 +10:00
Christopher Swenson 8c656754aa py/modmath: Add math.factorial, optimised and non-opt implementations.
This commit adds the math.factorial function in two variants:
- squared difference, which is faster than the naive version, relatively
  compact, and non-recursive;
- a mildly optimised recursive version, faster than the above one.

There are some more optimisations that could be done, but they tend to take
more code, and more storage space.  The recursive version seems like a
sensible compromise.

The new function is disabled by default, and uses the non-optimised version
by default if it is enabled.  The options are MICROPY_PY_MATH_FACTORIAL
and MICROPY_OPT_MATH_FACTORIAL.
2018-09-26 15:03:04 +10:00
Romain Goyet b768cc6ca8 py/parsenum: Avoid rounding errors with negative powers-of-10.
This patches avoids multiplying with negative powers-of-10 when parsing
floating-point values, when those powers-of-10 can be exactly represented
as a positive power.  When represented as a positive power and used to
divide, the resulting float will not have any rounding errors.

The issue is that mp_parse_num_decimal will sometimes not give the closest
floating representation of the input string.  Eg for "0.3", which can't be
represented exactly in floating point, mp_parse_num_decimal gives a
slightly high (by 1LSB) result.  This is because it computes the answer as
3 * 0.1, and since 0.1 also can't be represented exactly, multiplying by 3
multiplies up the rounding error in the 0.1.  Computing it as 3 / 10, as
now done by the change in this commit, gives an answer which is as close to
the true value of "0.3" as possible.
2018-09-20 22:06:41 +10:00
Damien George 3f6ffe059f py/objgenerator: Implement PEP479, StopIteration convs to RuntimeError.
This commit implements PEP479 which disallows raising StopIteration inside
a generator to signal that it should be finished.  Instead, the generator
should simply return when it is complete.

See https://www.python.org/dev/peps/pep-0479/ for details.
2018-09-20 15:36:59 +10:00
Paul Sokolovsky 93f29975db py/modbuiltins: Make oct/hex work when !MICROPY_PY_BUILTINS_STR_OP_MODULO
Instead of redirecting to str.__mod__(), use str.format() in this case.
2018-09-20 14:41:35 +10:00
Paul Sokolovsky 2da5d41350 py/objstr: Make % (__mod__) formatting operator configurable.
Default is enabled, disabled for minimal builds. Saves 1296 bytes on x86,
976 bytes on ARM.
2018-09-20 14:41:08 +10:00
Damien George b01f66c5f1 py: Shorten error messages by using contractions and some rewording. 2018-09-20 14:33:10 +10:00
Damien George 0a36a80f96 py/objtype: Clarify comment about configuring inplace op methods.
In 0e80f345f8 the inplace operations __iadd__
and __isub__ were made unconditionally available, so the comment about this
section is changed to reflect that.
2018-09-20 11:42:56 +10:00
Damien George 7e3dd9f8a3 py/asmthumb: Detect presence of I-cache using CMSIS macro.
Fixes issue #4113.
2018-09-16 01:50:45 +10:00
Damien George 30a45360e7 py/asmxtensa: Make indirect calls using func table, not raw pointers.
Loading a pointer by indexing into the native function table mp_fun_table,
rather than loading an immediate value (via a PC-relative load), uses less
code space.
2018-09-16 00:43:24 +10:00
Damien George 93d71c5436 py/emitnative: Make viper funcs run with their correct globals context.
Viper functions will now capture the globals at the point they were defined
and use these globals when executing.
2018-09-15 22:39:27 +10:00
Damien George f12e039c2b py/emitnative: Use macros instead of raw offsetof for slot locations.
Old globals are now stored in the second slot (ip in mp_code_state_t) to
make things simpler for viper.
2018-09-15 22:39:27 +10:00
Damien George a676b5acf6 py/emitnative: Support arbitrary number of arguments to viper functions. 2018-09-15 22:39:27 +10:00
Damien George 43f1848bfa py: Make viper functions have the same entry signature as native.
This commit makes viper functions have the same signature as native
functions, at the level of the emitter/assembler.  This means that viper
functions can now be wrapped in the same uPy object as native functions.

Viper functions are now responsible for parsing their arguments (before it
was done by the runtime), and this makes calling them more efficient (in
most cases) because the viper entry code can be custom generated to suit
the signature of the function.

This change also opens the way forward for viper functions to take
arbitrary numbers of arguments, and for them to handle globals correctly,
among other things.
2018-09-15 22:39:27 +10:00
Damien George 460954734e py/emitnative: Reuse mp_native_type_from_qstr when searching for a cast. 2018-09-15 13:52:58 +10:00
Damien George 9f2067288a py/compile: Factor code that compiles viper type annotations. 2018-09-15 13:44:39 +10:00
Damien George a169a5848c py/compile: Merge viper annotation and normal param compilation stages.
Now that the compiler can store the results of the viper types in the
scope, the viper parameter annotation compilation stage can be merged with
the normal parameter compilation stage.
2018-09-15 13:20:54 +10:00
Damien George 80db30a510 py/emit: Completely remove set_native_type, arg type is set in compiler.
In viper mode, the type of the argument is now stored in id_info->flags.
2018-09-15 13:00:11 +10:00
Damien George 07caf4f969 py/emit: Remove need to call set_native_type to set viper return type.
Instead this return type is now stored in the scope_flags.
2018-09-15 12:41:25 +10:00
Damien George 1d7c221b30 py/emit: Remove need to call set_native_type to set native/viper mode.
The native emitter can easily determine the mode via scope->emit_options.
2018-09-15 12:17:14 +10:00
Damien George 3751512e9d py/emit: Move MP_EMIT_OPT_xxx enums from compile.h to emitglue.h. 2018-09-15 12:17:09 +10:00
Damien George abb536da49 py/{asmx86,asmx64}: Extend test_r8_with_r8 to accept all 8 lower regs. 2018-09-14 17:38:09 +10:00
Damien George dd522d63b6 py/asmx64: Fix bug in assembler when creating disp with r13 and 0 offset 2018-09-14 17:36:09 +10:00
Damien George 9f241ef398 py: Optimise call to mp_arg_check_num by compressing fun signature.
With 5 arguments to mp_arg_check_num(), some architectures need to pass
values on the stack.  So compressing n_args_min, n_args_max, takes_kw into
a single word and passing only 3 arguments makes the call more efficient,
because almost all calls to this function pass in constant values.  Code
size is also reduced by a decent amount:

   bare-arm:  -116
minimal x86:   -64
   unix x64:  -256
unix nanbox:  -112
      stm32:  -324
     cc3200:  -192
    esp8266:  -192
      esp32:  -144
2018-09-14 13:39:17 +10:00
Damien George 4f3d9429b5 py: Fix native functions so they run with their correct globals context.
Prior to this commit a function compiled with the native decorator
@micropython.native would not work correctly when accessing global
variables, because the globals dict was not being set upon function entry.

This commit fixes this problem by, upon function entry, setting as the
current globals dict the globals dict context the function was defined
within, as per normal Python semantics, and as bytecode does.  Upon
function exit the original globals dict is restored.

In order to restore the globals dict when an exception is raised the native
function must guard its internals with an nlr_push/nlr_pop pair.  Because
this push/pop is relatively expensive, in both C stack usage for the
nlr_buf_t and CPU execution time, the implementation here optimises things
as much as possible.  First, the compiler keeps track of whether a function
even needs to access global variables.  Using this information the native
emitter then generates three different kinds of code:

1. no globals used, no exception handlers: no nlr handling code and no
   setting of the globals dict.

2. globals used, no exception handlers: an nlr_buf_t is allocated on the
   C stack but it is not used if the globals dict is unchanged, saving
   execution time because nlr_push/nlr_pop don't need to run.

3. function has exception handlers, may use globals: an nlr_buf_t is
   allocated and nlr_push/nlr_pop are always called.

In the end, native functions that don't access globals and don't have
exception handlers will run more efficiently than those that do.

Fixes issue #1573.
2018-09-13 22:47:20 +10:00
Damien George f2de9d60f7 py/emitnative: Fix try-finally in outer scope, so finally is cancelled. 2018-09-11 15:33:25 +10:00
Paul Sokolovsky 674e069ba9 py/objarray: bytearray: Allow 2nd/3rd arg to constructor.
If bytearray is constructed from str, a second argument of encoding is
required (in CPython), and third arg of Unicode error handling is allowed,
e.g.:

bytearray("str", "utf-8", "strict")

This is similar to bytes:

bytes("str", "utf-8", "strict")

This patch just allows to pass 2nd/3rd arguments to bytearray, but
doesn't try to validate them to not impact code size. (This is also
similar to how bytes constructor is handled, though it does a bit
more validation, e.g. check that in case of str arg, encoding argument
is passed.)
2018-09-11 15:10:10 +10:00
Paul Sokolovsky 5fe3730a30 extmod/moduhashlib: Add md5 implementation, using axTLS.
MD5 is still widely used, and may be important in some cases for networking
interoperability, e.g. HTTP Digest authentication.
2018-09-11 14:51:52 +10:00
stijn 89516b2b62 py/runtime: Fix incorrect test for MICROPY_PORT_DEINIT_FUNC. 2018-09-11 00:38:31 +10:00
Damien George 0be2ea50e9 py/py.mk: Build axtls library directly from its source files.
This removes the need for a separate axtls build stage, and builds all
axtls object files along with other code.  This simplifies and cleans up
the build process, automatically builds axtls when needed, and puts the
axtls object files in the correct $(BUILD) location.

The MicroPython axtls configuration file is provided in
extmod/axtls-include/config.h
2018-09-08 00:07:23 +10:00
Damien George 8014e7f15f py/compile: Factor code that compiles start/end of exception handler. 2018-09-04 16:06:22 +10:00
Damien George 4ae7111573 py/emitnative: Add support for return/break/continue in try and with.
This patch adds full support for unwinding jumps to the native emitter.
This means that return/break/continue can be used in try-except,
try-finally and with statements.  For code that doesn't use unwinding jumps
there is almost no overhead added to the generated code.
2018-09-04 14:31:28 +10:00
Damien George 3cd2c281d7 py/emitnative: Cancel caught exception once handled to prevent reraise.
The native emitter keeps the current exception in a slot in its C stack
(instead of on its Python value stack), so when it catches an exception it
must explicitly clear that slot so the same exception is not reraised later
on.
2018-09-03 17:41:02 +10:00
Damien George b735208403 py/vm: Fix handling of finally-return with complex nested finallys.
Back in 8047340d75 basic support was added in
the VM to handle return statements within a finally block.  But it didn't
cover all cases, in particular when some finally's were active and others
inactive when the "return" was executed.

This patch adds further support for return-within-finally by correctly
managing the currently_in_except_block flag, and should fix all cases.  The
main point is that finally handlers remain on the exception stack even if
they are active (currently being executed), and the unwind return code
should only execute those finally's which are inactive.

New tests are added for the cases which now pass.
2018-09-03 13:08:16 +10:00
Damien George 4f9842ad80 py/emitnx86: Fix number of args passed to mp_setup_code_state, 4 not 5. 2018-08-17 15:03:51 +10:00
Damien George 794c32102e py/asmxtensa: Use narrow version of add instr to reduce native code size 2018-08-17 14:53:58 +10:00
Damien George a0a29724c8 py/emitnative: Fix bug with store of 16 and 32 values in viper ARM mode. 2018-08-17 14:11:37 +10:00
Damien George 1ad44acb15 py/asmxtensa: Optimise loading local addr and support larger offsets. 2018-08-17 14:11:37 +10:00
Damien George fd10a11c6b py/asmxtensa: Fix bug with order of regs in addi encoding. 2018-08-17 14:11:37 +10:00
Damien George a3de776486 py/emitnative: Optimise and improve exception handling in native code.
Prior to this patch, native code would use a full nlr_buf_t for each
exception handler (try-except, try-finally, with).  For nested exception
handlers this would use a lot of C stack and be rather inefficient.

This patch changes how exceptions are handled in native code by setting up
only a single nlr_buf_t context for the entire function, and then manages a
state machine (using the PC) to work out which exception handler to run
when an exception is raised by an nlr_jump.  This keeps the C stack usage
at a constant level regardless of the depth of Python exception blocks.

The patch also fixes an existing bug when local variables are written to
within an exception handler, then their value was incorrectly restored if
an exception was raised (since the nlr_jump would restore register values,
back to the point of the nlr_push).

And it also gets nested try-finally+with working with the viper emitter.

Broadly speaking, efficiency of executing native code that doesn't use
any exception blocks is unchanged, and emitted code size is only slightly
increased for such function.  C stack usage of all native functions is
either equal or less than before.  Emitted code size for native functions
that use exception blocks is increased by roughly 10% (due in part to
fixing of above-mentioned bugs).

But, most importantly, this patch allows to implement more Python features
in native code, like unwind jumps and yielding from within nested exception
blocks.
2018-08-16 13:56:36 +10:00
Damien George 2964b41c28 py/asm*: Support assembling code to jump to a register, and get PC+off.
Useful for position independent code, and implementing state machines.
2018-08-16 13:45:24 +10:00
Damien George f7d6108d1a py/asmxtensa: Handle function entry/exit when stack use larger than 127. 2018-08-16 13:43:36 +10:00
Damien George 8c49995398 py/emitnative: Use small tables to simplify handling of local regs. 2018-08-15 10:55:11 +10:00
Damien George b8b2525576 extmod/modbtree: Update to work with new mp_stream_posix_XXX signatures. 2018-08-14 17:41:23 +10:00
Damien George 9ab816d676 py/stream: Adjust mp_stream_posix_XXX to take void*, not mp_obj_t.
These POSIX wrappers are assumed to be passed a concrete stream object so
it is more efficient (eg on nan-boxing builds) to pass in the pointer
rather than mp_obj_t, because then the users of these functions only need
to store a void* (and mp_obj_t may be wider than a pointer).  And things
would be further improved if the stream protocol functions eventually took
a pointer as their first argument (instead of an mp_obj_t).

This patch is a step to getting ussl/axtls compiling on nan-boxing builds.

See issue #3085.
2018-08-14 17:36:08 +10:00
Damien George a785a3dbfb py/objarray: Allow to build again when bytearray is disabled. 2018-08-14 16:23:21 +10:00
Damien George 91041945c9 py/gc: In gc_alloc, reset n_free var right before search for free mem.
Otherwise there is the possibility that n_free starts out non-zero from the
previous iteration, which may have found a few (but not enough) free blocks
at the end of the heap.  If this is the case, and if the very first blocks
that are scanned the second time around (starting at
gc_last_free_atb_index) are found to give enough memory (including the
blocks at the end of the heap from the previous iteration that left n_free
non-zero) then memory will be allocated starting before the location that
gc_last_free_atb_index points to, most likely leading to corruption.

This serious bug did not manifest itself in the past because a gc_collect
always resets gc_last_free_atb_index to point to the start of the GC heap,
and the first block there is almost always allocated to a long-lived
object (eg entries from sys.path, or mounted filesystem objects), which
means that n_free would be reset at the start of the search loop.

But with threading enabled with the GIL disabled it is possible to trigger
the bug via the following sequence of events:

1. Thread A runs gc_alloc, fails to find enough memory, and has a non-zero
   n_free at the end of the search.
2. Thread A calls gc_collect and frees a bunch of blocks on the GC heap.
3. Just after gc_collect finishes in thread A, thread B takes gc_mutex and
   does an allocation, moving gc_last_free_atb_index to point to the
   interior of the heap, to a place where there is most likely a run of
   available blocks.
4. Thread A regains gc_mutex and does its second search for free memory,
   starting with a non-zero n_free.  Since it's likely that the first block
   it searches is available it will allocate memory which overlaps with the
   memory before gc_last_free_atb_index.
2018-08-14 16:11:21 +10:00
Paul Sokolovsky bb28fe7b7b py/py.mk: Don't hardcode path to libaxtls.a.
Use -L$(BUILD), not -Lbuild. Otherwise, builds for different archs/subarchs
using different values of BUILD may fail.
2018-08-14 15:10:52 +10:00
Damien George cbec17f2cd py/compile: For dynamic compiler, widen literal 1 to get correct shift.
Without this patch, on 64-bit architectures the "1 << (small_int_bits - 1)"
is computed using only 32-bit values (since small_int_bits is a uint8_t)
and so will overflow (and give the wrong result) if small_int_bits is
larger than 32.
2018-08-13 23:34:47 +10:00
Damien George 17b512020b py/emitnative: Allocate space for local stack info as it's needed. 2018-08-07 16:19:38 +10:00
Damien George 652a58698e py/emitnative: Simplify handling of exception objects from nlr_buf_t.
There is no need to have three copies of the exception object on the top of
the native value stack.  Instead, the values on the stack should be the
first two items in an nlr_buf_t: the prev pointer and the ret_val pointer.
This is all that is needed and is what the rest of the native emitter
expects is on the stack.

This patch is essentially an optimisation.  Behaviour is unchanged,
although the stack layout for native exception handling now makes more
sense.
2018-08-06 14:44:33 +10:00
Damien George 3bef7bd782 py/emitnative: Fix native locals stack to start at correct location.
A native function allocates space on its C stack for mp_code_state_t,
followed by its Python stack, then its locals.  This patch makes sure that
the native function actually starts at the start of its Python stack,
rather than at the start of mp_code_state_t (which didn't lead to any
issues so far because the mp_code_state_t is unused after the native
function sets itself up).
2018-08-04 22:41:35 +10:00
Damien George 1c0bd46d1d py/asmx86: Use generic emit function to simplify cmp emit function. 2018-08-04 22:26:14 +10:00
Damien George 10830059c5 py/emitnative: Fix x86 native zero checks by comparing full word.
On x86 archs (both 32 and 64 bit) a bool return value only sets the 8-bit
al register, and the higher bits of the ax register have an undefined
value.  When testing the return value of such cases it is required to just
test al for zero/non-zero.  On the other hand, checking for truth or
zero/non-zero on an integer return value requires checking all bits of the
register.  These two cases must be distinguished and handled correctly in
generated native code.  This patch makes sure of this.

For other supported native archs (ARM, Thumb2, Xtensa) there is no such
distinction and this patch does not change anything for them.
2018-08-04 22:03:49 +10:00
Damien George 4b1e8bdebd py/emitnative: Factor common code for native jump helper. 2018-08-04 21:45:24 +10:00
Damien George b630dfcc1d py: Fix compiling with debug enabled and make more use of DEBUG_printf.
DEBUG_printf and MICROPY_DEBUG_PRINTER is now used instead of normal
printf, and a fault is fixed in mp_obj_class_lookup with debugging enabled;
see issue #3999.  Debugging can now be enabled on all ports including when
nan-boxing is used.
2018-08-02 14:17:24 +10:00
Damien George da2d2b6d88 py/mpconfig.h: Introduce MICROPY_DEBUG_PRINTER for debugging output.
This patch in effect renames MICROPY_DEBUG_PRINTER_DEST to
MICROPY_DEBUG_PRINTER, moving its default definition from
lib/utils/printf.c to py/mpconfig.h to make it official and documented, and
makes this macro a pointer rather than the actual mp_print_t struct.  This
is done to get consistency with MICROPY_ERROR_PRINTER, and provide this
macro for use outside just lib/utils/printf.c.

Ports are updated to use the new macro name.
2018-08-02 14:04:44 +10:00
Damien George 1e3a7f561f py/asmthumb: Optimise native code calling runtime glue functions.
This patch makes the Thumb-2 native emitter use wide ldr instructions to
call into the runtime, when the index into the native glue function table
is 32 or greater.  This reduces the generated assembler code from 10 bytes
to 6 bytes, saving RAM and making native code run about 0.8% faster.
2018-07-31 15:06:28 +10:00
Damien George aec6fa9160 py/objstr: In format error message, use common string with %s for type.
This error message did not consume all of its variable args, a bug
introduced long ago in baf6f14deb.  By fixing
it to use %s (instead of keeping the string as-is and deleting the last
arg) the same error message string is now reused three times in this format
function and gives a code size reduction of around 130 bytes.  It also now
gives a better error message when a non-string is passed in as an argument
to format, eg '{:d}'.format([]).
2018-07-30 12:46:47 +10:00
Damien George 7a4f1b00f6 py/stream: Introduce MP_STREAM_GET_FILENO ioctl request.
Can be used by POSIX-like systems that associate file numbers with a file.
2018-07-20 13:08:41 +10:00
Damien George e94d644a81 py/runtime: Use mp_obj_new_int_from_ll when return int is not small.
There's no need to call mp_obj_new_int() which will just fail the check for
small int and call mp_obj_new_int_from_ll() anyway.

Thanks to @Jongy for prompting this change.
2018-07-14 23:05:25 +10:00
Damien George 8c9c167dc6 py/emitnative: Optimise for iteration asm code for non-debug build.
In non-debug mode MP_OBJ_STOP_ITERATION is zero and comparing something to
zero can be done more efficiently in assembler than comparing to a non-zero
value.
2018-07-12 18:08:01 +10:00
Damien George e2e22e3d7e py/objgenerator: Implement __name__ with normal fun attr accessor code.
With the recent change b488a4a848, a
generating function now has the same layout in memory as a normal bytecode
function, and so can reuse the latter's attribute accessor code to
implement __name__.
2018-07-10 16:33:57 +10:00
Damien George fcf621b066 py/malloc: Give a compile warning if using finaliser without GC.
Fixes issue #3844.
2018-07-09 14:40:02 +10:00
Damien George 4a1edd8382 py/obj.h: Give compile error if using obj repr D with single-prec float.
Object representation D only works with no floats, or double precision
floats.
2018-07-08 23:45:05 +10:00
Damien George 4cd853fbd2 py/objmodule: Make mp_obj_module_get_globals an inline function.
Because this function is simple it saves code size to have it inlined.
Being an auxiliary helper function (and only used in the py/ core) the
argument should always be an mp_obj_module_t*, so there's no need for the
assert (and having it would require including assert.h in obj.h).
2018-07-08 22:27:39 +10:00
Damien George d9cdb880ff py/objdict: Make mp_obj_dict_get_map an inline function.
It's a very simple function and saves code, and improves efficiency, by
being inline.  Note that this is an auxiliary helper function and so
doesn't need mp_check_self -- that's used for functions that can be
accessed directly from Python code (eg from a method table).
2018-07-08 22:27:05 +10:00
Damien George a6ea6b08bc py: Simplify some cases of accessing the map of module and type dict.
mp_obj_module_get_globals() returns a mp_obj_dict_t*, and type->locals_dict
is a mp_obj_dict_t*, so access the map entry of the dict directly instead
of needing to cast this mp_obj_dict_t* up to an object and then calling the
mp_obj_dict_get_map() helper function.
2018-07-08 21:31:09 +10:00
Nicko van Someren d66c33cbd6 py/obj.h: Fix broken build for object repr C when float disabled.
Fixes issue #3914.
2018-07-03 09:51:08 +10:00
Damien George b488a4a848 py/objgenerator: Eliminate need for mp_obj_gen_wrap wrapper instances.
For generating functions there is no need to wrap the bytecode function in
a generator wrapper instance.  Instead the type of the bytecode function
can be changed to mp_type_gen_wrap.  This reduces code size and saves a
block of GC heap RAM for each generator.
2018-07-02 15:30:57 +10:00
Damien George e30a5fc7bc extmod/modure: Add ure.sub() function and method, and tests.
This feature is controlled at compile time by MICROPY_PY_URE_SUB, disabled
by default.

Thanks to @dmazzella for the original patch for this feature; see #3770.
2018-07-02 14:55:02 +10:00
Damien George 1e9b871d29 extmod/modure: Add match.span(), start() and end() methods, and tests.
This feature is controlled at compile time by
MICROPY_PY_URE_MATCH_SPAN_START_END, disabled by default.

Thanks to @dmazzella for the original patch for this feature; see #3770.
2018-07-02 14:54:56 +10:00
Damien George 1f86460910 extmod/modure: Add match.groups() method, and tests.
This feature is controlled at compile time by MICROPY_PY_URE_MATCH_GROUPS,
disabled by default.

Thanks to @dmazzella for the original patch for this feature; see #3770.
2018-07-02 14:53:30 +10:00