Commit Graph

3441 Commits

Author SHA1 Message Date
Damien George
8941c63290 py/asmx64: Change stack management to reference locals by rsp not rbp.
The rsp register is always a fixed distance below rbp, and using rsp to
reference locals on the stack frees up the rbp register for general purpose
use.
2018-10-13 15:16:33 +11:00
Damien George
34af10d2ef py/emitnative: Clean up unused macro and forward function declarations. 2018-10-02 15:01:56 +10:00
Damien George
69e7903904 py/obj.h: Use uint64_t instead of mp_int_t in repr-D MP_OBJ_IS_x macros.
This follows how it's already done in MP_OBJ_IS_OBJ: the objects are
considered 64-bit unsigned ints for the purpose of bitwise manipulation.
2018-10-01 16:36:46 +10:00
Damien George
a9237cee82 py/runtime: Remove comment in mp_import_name about level being 0.
A non-zero level has been supported for some time now.
2018-10-01 15:35:10 +10:00
Damien George
4ab397576f py/runtime: Use mp_import_name to implement tail of mp_import_from. 2018-10-01 15:22:03 +10:00
Damien George
cc2bd63c57 py/emitnative: Implement yield and yield-from in native emitter.
This commit adds first class support for yield and yield-from in the native
emitter, including send and throw support, and yields enclosed in exception
handlers (which requires pulling down the NLR stack before yielding, then
rebuilding it when resuming).

This has been fully tested and is working on unix x86 and x86-64, and
stm32.  Also basic tests have been done with the esp8266 port.  Performance
of existing native code is unchanged.
2018-10-01 13:31:11 +10:00
Damien George
8fec6f5434 py/emitnative: Reorder native state on C stack so nlr_buf_t is first.
The nlr_buf_t doesn't need to be part of the Python value stack (as it was
before this commit), it's simpler to have it separated as auxiliary state
that lives on the C stack.  This will help adding yield support because in
that case the nlr_buf_t and Python value stack live in separate memory
areas (C stack and heap respectively).
2018-10-01 12:36:21 +10:00
Damien George
4fc437f1ef py/asmxtensa: Use proper calculation for const table offset.
Instead of hard-coding it to 4 bytes.  This allows for there to be other
data stored at the very start of the emitted native code.
2018-10-01 12:34:58 +10:00
Damien George
5b19916d6e py/asmx64: Extend asm_x64_mov_reg_pcrel to accept high registers. 2018-10-01 12:34:36 +10:00
Damien George
1dc720dc01 py/asmx86: Comment out unused asm_x86_nop to prevent compiler warnings. 2018-10-01 12:34:23 +10:00
Damien George
87231132d4 py/asmthumb: Extend asm entry/exit to handle stack larger than 508 bytes 2018-09-30 23:31:17 +10:00
Damien George
ef9394e76a py/asmthumb: Clean up asm_thumb_bl_ind to use new optimised ldr helper. 2018-09-30 23:30:18 +10:00
Damien George
07ccb192c5 py/asmthumb: Add wide ldr to handle larger offsets.
In particular this allows native functions on Thumb2 to index more than 32
constants in the constant table.
2018-09-30 23:27:01 +10:00
Damien George
d95947b48a py/vm: When VM raises exception put exc obj at beginning of func state.
Instead of at end of state, n_state - 1.  It was originally (way back in
v1.0) put at the end of the state because the VM didn't have a pointer to
the start.  But now that the VM takes a mp_code_state_t pointer it does
have a pointer to the start of the state so can put the exception object
there.

This commit saves about 30 bytes of code on all architectures, and, more
importantly, reduces C-stack usage by a couple of words (8 bytes on Thumb2
and 16 bytes on x86-64) for every (non-generator) call of a bytecode
function because fun_bc_call no longer needs to remember the n_state
variable.
2018-09-29 23:25:08 +10:00
Damien George
dd288904db py/objtype: Support full object model for get/set/delitem special meths.
This makes these special methods have the same calling behaviour as other
methods in a class instance (mp_convert_member_lookup() is already called
by mp_obj_class_lookup()).
2018-09-28 23:22:34 +10:00
Damien George
2eb0170157 py/objtype: Remove TODO about storing attributes to classes.
This behaviour is tested in basics/class_store.py and follows CPython.
2018-09-28 23:15:12 +10:00
Damien George
2c7a3061d5 py/runtime: Remove nlr protection when calling __next__ in mp_resume.
And remove related comment about needing such protection when calling send.

Reasoning for removal is as follows:
- mp_resume is only called by the VM in YIELD_FROM opcode
- if send_value != MP_OBJ_NULL then throw_value == MP_OBJ_NULL
- so if __next__ or send are called then throw_value == MP_OBJ_NULL
- if __next__ or send raise an exception without nlr protection then the
  exception will be handled by the global exception handler of the VM
- this handler already has code to handle exceptions raised in YIELD_FROM,
  including correct handling of StopIteration
- this handler doesn't handle the case of injection of GeneratorExit, but
  this won't be needed because throw_value == MP_OBJ_NULL

Note that it's already possible for mp_resume() to raise an exception
(including StopIteration) from the unprotected call to type->iternext(), so
that's why the VM already has code to handle the case of exceptions coming
out of mp_resume().

This commit reduces code size by a bit, and significantly reduces C stack
usage when using yield-from, from 88 bytes down to 40 for Thumb2, and 152
down to 72 bytes for x86-64 (better than half).  (Note that gcc doesn't
seem to tail-call optimise the call from mp_resume() to mp_obj_gen_resume()
so this saving in C stack usage helps all uses of yield-from.)
2018-09-28 22:16:56 +10:00
Damien George
0c9d452370 py/vm: Fix case of throwing GeneratorExit type into yield-from.
mp_make_raise_obj must be used to convert a possible exception type to an
instance object, otherwise the VM may raise a non-exception object.

An existing test is adjusted to test this case, with the original test
already moved to generator_throw.py.
2018-09-28 11:39:35 +10:00
Damien George
e9012a20f7 py/emitnative: Change type of const_table from uintptr_t to mp_uint_t.
This matches how bytecode does it, and matches the signature of
mp_emit_glue_assign_native.  Since the native emitter doesn't support
nan-boxing uintptr_t and mp_uint_t are anyway the same bit-width.
2018-09-28 00:04:10 +10:00
Damien George
2e86233263 py/asm*: Remove ASM_MOV_REG_ALIGNED_IMM emit macro, it's no longer used.
After the previous commit this macro is no longer needed by the native
emitter because live heap pointers are no longer stored in generated native
machine code.
2018-09-27 23:39:08 +10:00
Damien George
7d4b6cc868 py/emitnative: Place const objs for native code in separate const table.
This commit changes native code to handle constant objects like bytecode:
instead of storing the pointers inside the native code they are now stored
in a separate constant table (such pointers include objects like bignum,
bytes, and raw code for nested functions).  This removes the need for the
GC to scan native code for root pointers, and takes a step towards making
native code independent of the runtime (eg so it can be compiled offline by
mpy-cross).

Note that the changes to the struct scope_t did not increase its size: on a
32-bit architecture it is still 48 bytes, and on a 64-bit architecture it
decreased from 80 to 72 bytes.
2018-09-27 23:39:08 +10:00
Damien George
b3eadf3f3d py/objfloat: Fix abs(-0.0) so it returns 0.0.
Nan and inf (signed and unsigned) are also handled correctly by using
signbit (they were also handled correctly with "val<0", but that didn't
handle -0.0 correctly).  A test case is added for this behaviour.
2018-09-27 15:21:25 +10:00
Damien George
fc1bb51af5 py/objgenerator: Remove TODO about returning gen being called again.
The code implements correct behaviour, as tested by the new test case added
in this commit.
2018-09-27 15:18:24 +10:00
Damien George
6d20be31ae py/vm: Reword TODO about invalid ip/sp after an exception to a note. 2018-09-27 15:17:37 +10:00
Damien George
04f7da78db py/objmodule: Remove TODO about checking store attr to a module.
The code implements correct behaviour, as tested by basics/module1.py.
2018-09-27 15:16:24 +10:00
Damien George
cc5c3c64ca py/objint: Remove TODO about checking of int() arg types with 2 args.
The arguments are checked by mp_obj_str_get_data and mp_obj_get_int.
2018-09-27 15:15:29 +10:00
Damien George
814f17a3a4 py/objdict: Reword TODO about inlining mp_obj_dict_get to a note. 2018-09-27 15:14:12 +10:00
Damien George
baa83a0c6d py/objslice: Remove long-obsolete comment about enhancing slice object.
Commit afaaf535e6 made this comment obsolete.
2018-09-27 11:23:31 +10:00
Damien George
76355c8863 py/vm: Make small optimisation of BUILD_SLICE opcode.
No need to call DECODE_UINT since the value will always be either 2 or 3.
2018-09-27 11:22:33 +10:00
stijn
57a7d5be9a py: Fix msvc C++ compiler warnings with MP_OBJ_FUN_MAKE_SIG macro.
When obj.h is compiled as C++ code, the cl compiler emits a warning about
possibly unsafe mixing of size_t and bool types in the or operation in
MP_OBJ_FUN_MAKE_SIG.  Similarly there's an implicit narrowing integer
conversion in runtime.h.  This commit fixes this by being explicit.
2018-09-26 15:34:59 +10:00
Paul Sokolovsky
a135bca4a1 py/objstr: format: Return bytes result for bytes format string.
This is an improvement over previous behavior when str was returned for
both str and bytes input format.  This new behaviour is also consistent
with how the % operator works, as well as many other str/bytes methods.

It should be noted that it's not how current versions of CPython work,
where there's a gap in the functionality and bytes.format() is not
supported.
2018-09-26 15:29:41 +10:00
Christopher Swenson
8c656754aa py/modmath: Add math.factorial, optimised and non-opt implementations.
This commit adds the math.factorial function in two variants:
- squared difference, which is faster than the naive version, relatively
  compact, and non-recursive;
- a mildly optimised recursive version, faster than the above one.

There are some more optimisations that could be done, but they tend to take
more code, and more storage space.  The recursive version seems like a
sensible compromise.

The new function is disabled by default, and uses the non-optimised version
by default if it is enabled.  The options are MICROPY_PY_MATH_FACTORIAL
and MICROPY_OPT_MATH_FACTORIAL.
2018-09-26 15:03:04 +10:00
Romain Goyet
b768cc6ca8 py/parsenum: Avoid rounding errors with negative powers-of-10.
This patches avoids multiplying with negative powers-of-10 when parsing
floating-point values, when those powers-of-10 can be exactly represented
as a positive power.  When represented as a positive power and used to
divide, the resulting float will not have any rounding errors.

The issue is that mp_parse_num_decimal will sometimes not give the closest
floating representation of the input string.  Eg for "0.3", which can't be
represented exactly in floating point, mp_parse_num_decimal gives a
slightly high (by 1LSB) result.  This is because it computes the answer as
3 * 0.1, and since 0.1 also can't be represented exactly, multiplying by 3
multiplies up the rounding error in the 0.1.  Computing it as 3 / 10, as
now done by the change in this commit, gives an answer which is as close to
the true value of "0.3" as possible.
2018-09-20 22:06:41 +10:00
Damien George
3f6ffe059f py/objgenerator: Implement PEP479, StopIteration convs to RuntimeError.
This commit implements PEP479 which disallows raising StopIteration inside
a generator to signal that it should be finished.  Instead, the generator
should simply return when it is complete.

See https://www.python.org/dev/peps/pep-0479/ for details.
2018-09-20 15:36:59 +10:00
Paul Sokolovsky
93f29975db py/modbuiltins: Make oct/hex work when !MICROPY_PY_BUILTINS_STR_OP_MODULO
Instead of redirecting to str.__mod__(), use str.format() in this case.
2018-09-20 14:41:35 +10:00
Paul Sokolovsky
2da5d41350 py/objstr: Make % (__mod__) formatting operator configurable.
Default is enabled, disabled for minimal builds. Saves 1296 bytes on x86,
976 bytes on ARM.
2018-09-20 14:41:08 +10:00
Damien George
b01f66c5f1 py: Shorten error messages by using contractions and some rewording. 2018-09-20 14:33:10 +10:00
Damien George
0a36a80f96 py/objtype: Clarify comment about configuring inplace op methods.
In 0e80f345f8 the inplace operations __iadd__
and __isub__ were made unconditionally available, so the comment about this
section is changed to reflect that.
2018-09-20 11:42:56 +10:00
Damien George
7e3dd9f8a3 py/asmthumb: Detect presence of I-cache using CMSIS macro.
Fixes issue #4113.
2018-09-16 01:50:45 +10:00
Damien George
30a45360e7 py/asmxtensa: Make indirect calls using func table, not raw pointers.
Loading a pointer by indexing into the native function table mp_fun_table,
rather than loading an immediate value (via a PC-relative load), uses less
code space.
2018-09-16 00:43:24 +10:00
Damien George
93d71c5436 py/emitnative: Make viper funcs run with their correct globals context.
Viper functions will now capture the globals at the point they were defined
and use these globals when executing.
2018-09-15 22:39:27 +10:00
Damien George
f12e039c2b py/emitnative: Use macros instead of raw offsetof for slot locations.
Old globals are now stored in the second slot (ip in mp_code_state_t) to
make things simpler for viper.
2018-09-15 22:39:27 +10:00
Damien George
a676b5acf6 py/emitnative: Support arbitrary number of arguments to viper functions. 2018-09-15 22:39:27 +10:00
Damien George
43f1848bfa py: Make viper functions have the same entry signature as native.
This commit makes viper functions have the same signature as native
functions, at the level of the emitter/assembler.  This means that viper
functions can now be wrapped in the same uPy object as native functions.

Viper functions are now responsible for parsing their arguments (before it
was done by the runtime), and this makes calling them more efficient (in
most cases) because the viper entry code can be custom generated to suit
the signature of the function.

This change also opens the way forward for viper functions to take
arbitrary numbers of arguments, and for them to handle globals correctly,
among other things.
2018-09-15 22:39:27 +10:00
Damien George
460954734e py/emitnative: Reuse mp_native_type_from_qstr when searching for a cast. 2018-09-15 13:52:58 +10:00
Damien George
9f2067288a py/compile: Factor code that compiles viper type annotations. 2018-09-15 13:44:39 +10:00
Damien George
a169a5848c py/compile: Merge viper annotation and normal param compilation stages.
Now that the compiler can store the results of the viper types in the
scope, the viper parameter annotation compilation stage can be merged with
the normal parameter compilation stage.
2018-09-15 13:20:54 +10:00
Damien George
80db30a510 py/emit: Completely remove set_native_type, arg type is set in compiler.
In viper mode, the type of the argument is now stored in id_info->flags.
2018-09-15 13:00:11 +10:00
Damien George
07caf4f969 py/emit: Remove need to call set_native_type to set viper return type.
Instead this return type is now stored in the scope_flags.
2018-09-15 12:41:25 +10:00
Damien George
1d7c221b30 py/emit: Remove need to call set_native_type to set native/viper mode.
The native emitter can easily determine the mode via scope->emit_options.
2018-09-15 12:17:14 +10:00
Damien George
3751512e9d py/emit: Move MP_EMIT_OPT_xxx enums from compile.h to emitglue.h. 2018-09-15 12:17:09 +10:00
Damien George
abb536da49 py/{asmx86,asmx64}: Extend test_r8_with_r8 to accept all 8 lower regs. 2018-09-14 17:38:09 +10:00
Damien George
dd522d63b6 py/asmx64: Fix bug in assembler when creating disp with r13 and 0 offset 2018-09-14 17:36:09 +10:00
Damien George
9f241ef398 py: Optimise call to mp_arg_check_num by compressing fun signature.
With 5 arguments to mp_arg_check_num(), some architectures need to pass
values on the stack.  So compressing n_args_min, n_args_max, takes_kw into
a single word and passing only 3 arguments makes the call more efficient,
because almost all calls to this function pass in constant values.  Code
size is also reduced by a decent amount:

   bare-arm:  -116
minimal x86:   -64
   unix x64:  -256
unix nanbox:  -112
      stm32:  -324
     cc3200:  -192
    esp8266:  -192
      esp32:  -144
2018-09-14 13:39:17 +10:00
Damien George
4f3d9429b5 py: Fix native functions so they run with their correct globals context.
Prior to this commit a function compiled with the native decorator
@micropython.native would not work correctly when accessing global
variables, because the globals dict was not being set upon function entry.

This commit fixes this problem by, upon function entry, setting as the
current globals dict the globals dict context the function was defined
within, as per normal Python semantics, and as bytecode does.  Upon
function exit the original globals dict is restored.

In order to restore the globals dict when an exception is raised the native
function must guard its internals with an nlr_push/nlr_pop pair.  Because
this push/pop is relatively expensive, in both C stack usage for the
nlr_buf_t and CPU execution time, the implementation here optimises things
as much as possible.  First, the compiler keeps track of whether a function
even needs to access global variables.  Using this information the native
emitter then generates three different kinds of code:

1. no globals used, no exception handlers: no nlr handling code and no
   setting of the globals dict.

2. globals used, no exception handlers: an nlr_buf_t is allocated on the
   C stack but it is not used if the globals dict is unchanged, saving
   execution time because nlr_push/nlr_pop don't need to run.

3. function has exception handlers, may use globals: an nlr_buf_t is
   allocated and nlr_push/nlr_pop are always called.

In the end, native functions that don't access globals and don't have
exception handlers will run more efficiently than those that do.

Fixes issue #1573.
2018-09-13 22:47:20 +10:00
Damien George
f2de9d60f7 py/emitnative: Fix try-finally in outer scope, so finally is cancelled. 2018-09-11 15:33:25 +10:00
Paul Sokolovsky
674e069ba9 py/objarray: bytearray: Allow 2nd/3rd arg to constructor.
If bytearray is constructed from str, a second argument of encoding is
required (in CPython), and third arg of Unicode error handling is allowed,
e.g.:

bytearray("str", "utf-8", "strict")

This is similar to bytes:

bytes("str", "utf-8", "strict")

This patch just allows to pass 2nd/3rd arguments to bytearray, but
doesn't try to validate them to not impact code size. (This is also
similar to how bytes constructor is handled, though it does a bit
more validation, e.g. check that in case of str arg, encoding argument
is passed.)
2018-09-11 15:10:10 +10:00
Paul Sokolovsky
5fe3730a30 extmod/moduhashlib: Add md5 implementation, using axTLS.
MD5 is still widely used, and may be important in some cases for networking
interoperability, e.g. HTTP Digest authentication.
2018-09-11 14:51:52 +10:00
stijn
89516b2b62 py/runtime: Fix incorrect test for MICROPY_PORT_DEINIT_FUNC. 2018-09-11 00:38:31 +10:00
Damien George
0be2ea50e9 py/py.mk: Build axtls library directly from its source files.
This removes the need for a separate axtls build stage, and builds all
axtls object files along with other code.  This simplifies and cleans up
the build process, automatically builds axtls when needed, and puts the
axtls object files in the correct $(BUILD) location.

The MicroPython axtls configuration file is provided in
extmod/axtls-include/config.h
2018-09-08 00:07:23 +10:00
Damien George
8014e7f15f py/compile: Factor code that compiles start/end of exception handler. 2018-09-04 16:06:22 +10:00
Damien George
4ae7111573 py/emitnative: Add support for return/break/continue in try and with.
This patch adds full support for unwinding jumps to the native emitter.
This means that return/break/continue can be used in try-except,
try-finally and with statements.  For code that doesn't use unwinding jumps
there is almost no overhead added to the generated code.
2018-09-04 14:31:28 +10:00
Damien George
3cd2c281d7 py/emitnative: Cancel caught exception once handled to prevent reraise.
The native emitter keeps the current exception in a slot in its C stack
(instead of on its Python value stack), so when it catches an exception it
must explicitly clear that slot so the same exception is not reraised later
on.
2018-09-03 17:41:02 +10:00
Damien George
b735208403 py/vm: Fix handling of finally-return with complex nested finallys.
Back in 8047340d75 basic support was added in
the VM to handle return statements within a finally block.  But it didn't
cover all cases, in particular when some finally's were active and others
inactive when the "return" was executed.

This patch adds further support for return-within-finally by correctly
managing the currently_in_except_block flag, and should fix all cases.  The
main point is that finally handlers remain on the exception stack even if
they are active (currently being executed), and the unwind return code
should only execute those finally's which are inactive.

New tests are added for the cases which now pass.
2018-09-03 13:08:16 +10:00
Damien George
4f9842ad80 py/emitnx86: Fix number of args passed to mp_setup_code_state, 4 not 5. 2018-08-17 15:03:51 +10:00
Damien George
794c32102e py/asmxtensa: Use narrow version of add instr to reduce native code size 2018-08-17 14:53:58 +10:00
Damien George
a0a29724c8 py/emitnative: Fix bug with store of 16 and 32 values in viper ARM mode. 2018-08-17 14:11:37 +10:00
Damien George
1ad44acb15 py/asmxtensa: Optimise loading local addr and support larger offsets. 2018-08-17 14:11:37 +10:00
Damien George
fd10a11c6b py/asmxtensa: Fix bug with order of regs in addi encoding. 2018-08-17 14:11:37 +10:00
Damien George
a3de776486 py/emitnative: Optimise and improve exception handling in native code.
Prior to this patch, native code would use a full nlr_buf_t for each
exception handler (try-except, try-finally, with).  For nested exception
handlers this would use a lot of C stack and be rather inefficient.

This patch changes how exceptions are handled in native code by setting up
only a single nlr_buf_t context for the entire function, and then manages a
state machine (using the PC) to work out which exception handler to run
when an exception is raised by an nlr_jump.  This keeps the C stack usage
at a constant level regardless of the depth of Python exception blocks.

The patch also fixes an existing bug when local variables are written to
within an exception handler, then their value was incorrectly restored if
an exception was raised (since the nlr_jump would restore register values,
back to the point of the nlr_push).

And it also gets nested try-finally+with working with the viper emitter.

Broadly speaking, efficiency of executing native code that doesn't use
any exception blocks is unchanged, and emitted code size is only slightly
increased for such function.  C stack usage of all native functions is
either equal or less than before.  Emitted code size for native functions
that use exception blocks is increased by roughly 10% (due in part to
fixing of above-mentioned bugs).

But, most importantly, this patch allows to implement more Python features
in native code, like unwind jumps and yielding from within nested exception
blocks.
2018-08-16 13:56:36 +10:00
Damien George
2964b41c28 py/asm*: Support assembling code to jump to a register, and get PC+off.
Useful for position independent code, and implementing state machines.
2018-08-16 13:45:24 +10:00
Damien George
f7d6108d1a py/asmxtensa: Handle function entry/exit when stack use larger than 127. 2018-08-16 13:43:36 +10:00
Damien George
8c49995398 py/emitnative: Use small tables to simplify handling of local regs. 2018-08-15 10:55:11 +10:00
Damien George
b8b2525576 extmod/modbtree: Update to work with new mp_stream_posix_XXX signatures. 2018-08-14 17:41:23 +10:00
Damien George
9ab816d676 py/stream: Adjust mp_stream_posix_XXX to take void*, not mp_obj_t.
These POSIX wrappers are assumed to be passed a concrete stream object so
it is more efficient (eg on nan-boxing builds) to pass in the pointer
rather than mp_obj_t, because then the users of these functions only need
to store a void* (and mp_obj_t may be wider than a pointer).  And things
would be further improved if the stream protocol functions eventually took
a pointer as their first argument (instead of an mp_obj_t).

This patch is a step to getting ussl/axtls compiling on nan-boxing builds.

See issue #3085.
2018-08-14 17:36:08 +10:00
Damien George
a785a3dbfb py/objarray: Allow to build again when bytearray is disabled. 2018-08-14 16:23:21 +10:00
Damien George
91041945c9 py/gc: In gc_alloc, reset n_free var right before search for free mem.
Otherwise there is the possibility that n_free starts out non-zero from the
previous iteration, which may have found a few (but not enough) free blocks
at the end of the heap.  If this is the case, and if the very first blocks
that are scanned the second time around (starting at
gc_last_free_atb_index) are found to give enough memory (including the
blocks at the end of the heap from the previous iteration that left n_free
non-zero) then memory will be allocated starting before the location that
gc_last_free_atb_index points to, most likely leading to corruption.

This serious bug did not manifest itself in the past because a gc_collect
always resets gc_last_free_atb_index to point to the start of the GC heap,
and the first block there is almost always allocated to a long-lived
object (eg entries from sys.path, or mounted filesystem objects), which
means that n_free would be reset at the start of the search loop.

But with threading enabled with the GIL disabled it is possible to trigger
the bug via the following sequence of events:

1. Thread A runs gc_alloc, fails to find enough memory, and has a non-zero
   n_free at the end of the search.
2. Thread A calls gc_collect and frees a bunch of blocks on the GC heap.
3. Just after gc_collect finishes in thread A, thread B takes gc_mutex and
   does an allocation, moving gc_last_free_atb_index to point to the
   interior of the heap, to a place where there is most likely a run of
   available blocks.
4. Thread A regains gc_mutex and does its second search for free memory,
   starting with a non-zero n_free.  Since it's likely that the first block
   it searches is available it will allocate memory which overlaps with the
   memory before gc_last_free_atb_index.
2018-08-14 16:11:21 +10:00
Paul Sokolovsky
bb28fe7b7b py/py.mk: Don't hardcode path to libaxtls.a.
Use -L$(BUILD), not -Lbuild. Otherwise, builds for different archs/subarchs
using different values of BUILD may fail.
2018-08-14 15:10:52 +10:00
Damien George
cbec17f2cd py/compile: For dynamic compiler, widen literal 1 to get correct shift.
Without this patch, on 64-bit architectures the "1 << (small_int_bits - 1)"
is computed using only 32-bit values (since small_int_bits is a uint8_t)
and so will overflow (and give the wrong result) if small_int_bits is
larger than 32.
2018-08-13 23:34:47 +10:00
Damien George
17b512020b py/emitnative: Allocate space for local stack info as it's needed. 2018-08-07 16:19:38 +10:00
Damien George
652a58698e py/emitnative: Simplify handling of exception objects from nlr_buf_t.
There is no need to have three copies of the exception object on the top of
the native value stack.  Instead, the values on the stack should be the
first two items in an nlr_buf_t: the prev pointer and the ret_val pointer.
This is all that is needed and is what the rest of the native emitter
expects is on the stack.

This patch is essentially an optimisation.  Behaviour is unchanged,
although the stack layout for native exception handling now makes more
sense.
2018-08-06 14:44:33 +10:00
Damien George
3bef7bd782 py/emitnative: Fix native locals stack to start at correct location.
A native function allocates space on its C stack for mp_code_state_t,
followed by its Python stack, then its locals.  This patch makes sure that
the native function actually starts at the start of its Python stack,
rather than at the start of mp_code_state_t (which didn't lead to any
issues so far because the mp_code_state_t is unused after the native
function sets itself up).
2018-08-04 22:41:35 +10:00
Damien George
1c0bd46d1d py/asmx86: Use generic emit function to simplify cmp emit function. 2018-08-04 22:26:14 +10:00
Damien George
10830059c5 py/emitnative: Fix x86 native zero checks by comparing full word.
On x86 archs (both 32 and 64 bit) a bool return value only sets the 8-bit
al register, and the higher bits of the ax register have an undefined
value.  When testing the return value of such cases it is required to just
test al for zero/non-zero.  On the other hand, checking for truth or
zero/non-zero on an integer return value requires checking all bits of the
register.  These two cases must be distinguished and handled correctly in
generated native code.  This patch makes sure of this.

For other supported native archs (ARM, Thumb2, Xtensa) there is no such
distinction and this patch does not change anything for them.
2018-08-04 22:03:49 +10:00
Damien George
4b1e8bdebd py/emitnative: Factor common code for native jump helper. 2018-08-04 21:45:24 +10:00
Damien George
b630dfcc1d py: Fix compiling with debug enabled and make more use of DEBUG_printf.
DEBUG_printf and MICROPY_DEBUG_PRINTER is now used instead of normal
printf, and a fault is fixed in mp_obj_class_lookup with debugging enabled;
see issue #3999.  Debugging can now be enabled on all ports including when
nan-boxing is used.
2018-08-02 14:17:24 +10:00
Damien George
da2d2b6d88 py/mpconfig.h: Introduce MICROPY_DEBUG_PRINTER for debugging output.
This patch in effect renames MICROPY_DEBUG_PRINTER_DEST to
MICROPY_DEBUG_PRINTER, moving its default definition from
lib/utils/printf.c to py/mpconfig.h to make it official and documented, and
makes this macro a pointer rather than the actual mp_print_t struct.  This
is done to get consistency with MICROPY_ERROR_PRINTER, and provide this
macro for use outside just lib/utils/printf.c.

Ports are updated to use the new macro name.
2018-08-02 14:04:44 +10:00
Damien George
1e3a7f561f py/asmthumb: Optimise native code calling runtime glue functions.
This patch makes the Thumb-2 native emitter use wide ldr instructions to
call into the runtime, when the index into the native glue function table
is 32 or greater.  This reduces the generated assembler code from 10 bytes
to 6 bytes, saving RAM and making native code run about 0.8% faster.
2018-07-31 15:06:28 +10:00
Damien George
aec6fa9160 py/objstr: In format error message, use common string with %s for type.
This error message did not consume all of its variable args, a bug
introduced long ago in baf6f14deb.  By fixing
it to use %s (instead of keeping the string as-is and deleting the last
arg) the same error message string is now reused three times in this format
function and gives a code size reduction of around 130 bytes.  It also now
gives a better error message when a non-string is passed in as an argument
to format, eg '{:d}'.format([]).
2018-07-30 12:46:47 +10:00
Damien George
7a4f1b00f6 py/stream: Introduce MP_STREAM_GET_FILENO ioctl request.
Can be used by POSIX-like systems that associate file numbers with a file.
2018-07-20 13:08:41 +10:00
Damien George
e94d644a81 py/runtime: Use mp_obj_new_int_from_ll when return int is not small.
There's no need to call mp_obj_new_int() which will just fail the check for
small int and call mp_obj_new_int_from_ll() anyway.

Thanks to @Jongy for prompting this change.
2018-07-14 23:05:25 +10:00
Damien George
8c9c167dc6 py/emitnative: Optimise for iteration asm code for non-debug build.
In non-debug mode MP_OBJ_STOP_ITERATION is zero and comparing something to
zero can be done more efficiently in assembler than comparing to a non-zero
value.
2018-07-12 18:08:01 +10:00
Damien George
e2e22e3d7e py/objgenerator: Implement __name__ with normal fun attr accessor code.
With the recent change b488a4a848, a
generating function now has the same layout in memory as a normal bytecode
function, and so can reuse the latter's attribute accessor code to
implement __name__.
2018-07-10 16:33:57 +10:00
Damien George
fcf621b066 py/malloc: Give a compile warning if using finaliser without GC.
Fixes issue #3844.
2018-07-09 14:40:02 +10:00
Damien George
4a1edd8382 py/obj.h: Give compile error if using obj repr D with single-prec float.
Object representation D only works with no floats, or double precision
floats.
2018-07-08 23:45:05 +10:00
Damien George
4cd853fbd2 py/objmodule: Make mp_obj_module_get_globals an inline function.
Because this function is simple it saves code size to have it inlined.
Being an auxiliary helper function (and only used in the py/ core) the
argument should always be an mp_obj_module_t*, so there's no need for the
assert (and having it would require including assert.h in obj.h).
2018-07-08 22:27:39 +10:00
Damien George
d9cdb880ff py/objdict: Make mp_obj_dict_get_map an inline function.
It's a very simple function and saves code, and improves efficiency, by
being inline.  Note that this is an auxiliary helper function and so
doesn't need mp_check_self -- that's used for functions that can be
accessed directly from Python code (eg from a method table).
2018-07-08 22:27:05 +10:00
Damien George
a6ea6b08bc py: Simplify some cases of accessing the map of module and type dict.
mp_obj_module_get_globals() returns a mp_obj_dict_t*, and type->locals_dict
is a mp_obj_dict_t*, so access the map entry of the dict directly instead
of needing to cast this mp_obj_dict_t* up to an object and then calling the
mp_obj_dict_get_map() helper function.
2018-07-08 21:31:09 +10:00
Nicko van Someren
d66c33cbd6 py/obj.h: Fix broken build for object repr C when float disabled.
Fixes issue #3914.
2018-07-03 09:51:08 +10:00
Damien George
b488a4a848 py/objgenerator: Eliminate need for mp_obj_gen_wrap wrapper instances.
For generating functions there is no need to wrap the bytecode function in
a generator wrapper instance.  Instead the type of the bytecode function
can be changed to mp_type_gen_wrap.  This reduces code size and saves a
block of GC heap RAM for each generator.
2018-07-02 15:30:57 +10:00
Damien George
e30a5fc7bc extmod/modure: Add ure.sub() function and method, and tests.
This feature is controlled at compile time by MICROPY_PY_URE_SUB, disabled
by default.

Thanks to @dmazzella for the original patch for this feature; see #3770.
2018-07-02 14:55:02 +10:00
Damien George
1e9b871d29 extmod/modure: Add match.span(), start() and end() methods, and tests.
This feature is controlled at compile time by
MICROPY_PY_URE_MATCH_SPAN_START_END, disabled by default.

Thanks to @dmazzella for the original patch for this feature; see #3770.
2018-07-02 14:54:56 +10:00
Damien George
1f86460910 extmod/modure: Add match.groups() method, and tests.
This feature is controlled at compile time by MICROPY_PY_URE_MATCH_GROUPS,
disabled by default.

Thanks to @dmazzella for the original patch for this feature; see #3770.
2018-07-02 14:53:30 +10:00
Damien George
d8dc918deb py/compile: Handle return/break/continue correctly in async with.
Before this patch the context manager's __aexit__() method would not be
executed if a return/break/continue statement was used to exit an async
with block.  async with now has the same semantics as normal with.

The fix here applies purely to the compiler, and does not modify the
runtime at all. It might (eventually) be better to define new bytecode(s)
to handle async with (and maybe other async constructs) in a cleaner, more
efficient way.

One minor drawback with addressing this issue purely in the compiler is
that it wasn't possible to get 100% CPython semantics.  The thing that is
different here to CPython is that the __aexit__ method is not looked up in
the context manager until it is needed, which is after the body of the
async with statement has executed.  So if a context manager doesn't have
__aexit__ then CPython raises an exception before the async with is
executed, whereas uPy will raise it after it is executed.  Note that
__aenter__ is looked up at the beginning in uPy because it needs to be
called straightaway, so if the context manager isn't a context manager then
it'll still raise an exception at the same location as CPython.  The only
difference is if the context manager has the __aenter__ method but not the
__aexit__ method, then in that case uPy has different behaviour.  But this
is a very minor, and acceptable, difference.
2018-06-27 16:57:42 +10:00
Yonatan Goldschmidt
473fe45da2 extmod/moducryptolib: Optionally export MODE_* constants to Python.
Allow including crypto consts based on compilation settings.  Disabled by
default to reduce code size; if one wants extra code readability, can
enable them.
2018-06-27 16:29:26 +10:00
Paul Sokolovsky
567bc2d6ce extmod/moducryptolib: Add ucryptolib module with crypto functions.
The API follows guidelines of https://www.python.org/dev/peps/pep-0272/,
but is optimized for code size, with the idea that full PEP 0272
compatibility can be added with a simple Python wrapper mode.

The naming of the module follows (u)hashlib pattern.

At the bare minimum, this module is expected to provide:

* AES128, ECB (i.e. "null") mode, encrypt only

Implementation in this commit is based on axTLS routines, and implements
following:

* AES 128 and 256
* ECB and CBC modes
* encrypt and decrypt
2018-06-27 14:54:40 +10:00
Damien George
25ae98f07c py/compile: Combine expr, xor_expr and and_expr into one function.
This and the previous 4 commits combined have change in code size of:

   bare-arm:   -92
minimal x86:  -544
   unix x64:  -544
unix nanbox:  -712
      stm32:  -116
     cc3200:  -128
    esp8266:  -348
      esp32:  -232
2018-06-22 17:00:29 +10:00
Damien George
36e474e83f py/compile: Combine or_test and and_test compile functions. 2018-06-22 17:00:29 +10:00
Damien George
1a7109d65a py/compile: Combine global and nonlocal statement compile functions. 2018-06-22 17:00:29 +10:00
Damien George
d23bec3fc8 py/compile: Combine subscript_2 and subscript_3 into one function. 2018-06-22 17:00:29 +10:00
Damien George
c149197928 py/compile: Combine break and continue compile functions. 2018-06-22 17:00:29 +10:00
Damien George
34344a413f py/stream: Remove stray empty line at start of file.
This was accidentally added in 6abede2ca9
2018-06-20 16:26:12 +10:00
Damien George
582b190764 py: Add checks for stream objects in print() and sys.print_exception(). 2018-06-20 15:57:10 +10:00
Damien George
2c8d130f70 py/stream: Update comment for mp_stream_write_adaptor. 2018-06-20 15:56:32 +10:00
Damien George
c00ee200ac py/objarray: Replace 0x80 with new MP_OBJ_ARRAY_TYPECODE_FLAG_RW macro. 2018-06-18 13:40:53 +10:00
Damien George
6abede2ca9 py/stream: Introduce and use efficient mp_get_stream to access stream_p.
The existing mp_get_stream_raise() helper does explicit checks that the
input object is a real pointer object, has a non-NULL stream protocol, and
has the desired stream C method (read/write/ioctl).  In most cases it is
not necessary to do these checks because it is guaranteed that the input
object has the stream protocol and desired C methods.  For example, native
objects that use the stream wrappers (eg mp_stream_readinto_obj) in their
locals dict always have the stream protocol (or else they shouldn't have
these wrappers in their locals dict).

This patch introduces an efficient mp_get_stream() which doesn't do any
checks and just extracts the stream protocol struct.  This should be used
in all cases where the argument object is known to be a stream.  The
existing mp_get_stream_raise() should be used primarily to verify that an
object does have the correct stream protocol methods.

All uses of mp_get_stream_raise() in py/stream.c have been converted to use
mp_get_stream() because the argument is guaranteed to be a proper stream
object.

This patch improves efficiency of stream operations and reduces code size.
2018-06-18 12:35:56 +10:00
Damien George
035906419d extmod/uos_dupterm: Use native C stream methods on dupterm object.
This patch changes dupterm to call the native C stream methods on the
connected stream objects, instead of calling the Python readinto/write
methods.  This is much more efficient for native stream objects like UART
and webrepl and doesn't require allocating a special dupterm array.

This change is a minor breaking change from the user's perspective because
dupterm no longer accepts pure user stream objects to duplicate on.  But
with the recent addition of uio.IOBase it is possible to still create such
classes just by inheriting from uio.IOBase, for example:

    import uio, uos

    class MyStream(uio.IOBase):
        def write(self, buf):
            # existing write implementation
        def readinto(self, buf):
            # existing readinto implementation

    uos.dupterm(MyStream())
2018-06-12 15:06:11 +10:00
Damien George
7ad04d17da py/mkrules.mk: Regenerate all qstrs when config files change.
A port can define QSTR_GLOBAL_DEPENDENCIES to add extra files.
2018-06-12 13:53:43 +10:00
Yonatan Goldschmidt
6630354ffe extmod/moduhashlib: Allow to disable the sha256 class.
Via the config value MICROPY_PY_UHASHLIB_SHA256.  Default to enabled to
keep backwards compatibility.

Also add default value for the sha1 class, to at least document its
existence.
2018-06-12 13:50:11 +10:00
Damien George
af0932a779 py/modio: Add uio.IOBase class to allow to define user streams.
A user class derived from IOBase and implementing readinto/write/ioctl can
now be used anywhere a native stream object is accepted.

The mapping from C to Python is:

    stream_p->read  --> readinto(buf)
    stream_p->write --> write(buf)
    stream_p->ioctl --> ioctl(request, arg)

Among other things it allows the user to:

- create an object which can be passed as the file argument to print:
  print(..., file=myobj), and then print will pass all the data to the
  object via the objects write method (same as CPython)
- pass a user object to uio.BufferedWriter to buffer the writes (same as
  CPython)
- use select.select on a user object
- register user objects with select.poll, in particular so user objects can
  be used with uasyncio
- create user files that can be returned from user filesystems, and import
  can import scripts from these user files

For example:

    class MyOut(io.IOBase):
        def write(self, buf):
            print('write', repr(buf))
            return len(buf)

    print('hello', file=MyOut())

The feature is enabled via MICROPY_PY_IO_IOBASE which is disabled by
default.
2018-06-12 12:29:26 +10:00
Damien George
6a445b60fa py/lexer: Add support for underscores in numeric literals.
This is a very convenient feature introduced in Python 3.6 by PEP 515.
2018-06-12 12:17:43 +10:00
Damien George
522ea80f06 py/gc: Add gc_sweep_all() function to run all remaining finalisers.
This patch adds the gc_sweep_all() function which does a garbage collection
without tracing any root pointers, so frees all the memory, and most
importantly runs any remaining finalisers.

This helps primarily for soft reset: it will close any open files, any open
sockets, and help to get the system back to a clean state upon soft reset.
2018-06-12 11:55:29 +10:00
Damien George
36c1052183 py/objtype: Optimise instance get/set/del by skipping special accessors.
This patch is a code optimisation, trading text bytes for speed.  On
pyboard it's an increase of 0.06% in code size for a gain (in pystone
performance) of roughly 6.5%.

The patch optimises load/store/delete of attributes in user defined classes
by not looking up special accessors (@property, __get__, __delete__,
__set__, __setattr__ and __getattr_) if they are guaranteed not to exist in
the class.

Currently, if you do my_obj.foo() then the runtime has to do a few checks
to see if foo is a property or has __get__, and if so delegate the call.
And for stores things like my_obj.foo = 1 has to first check if foo is a
property or has __set__ defined on it.

Doing all those checks each and every time the attribute is accessed has a
performance penalty.  This patch eliminates all those checks for cases when
it's guaranteed that the checks will always fail, ie no attributes are
properties nor have any special accessor methods defined on them.

To make this guarantee it checks all attributes of a user-defined class
when it is first created.  If any of the attributes of the user class are
properties or have special accessors, or any of the base classes of the
user class have them, then it sets a flag in the class to indicate that
special accessors must be checked for.  Then in the load/store/delete code
it checks this flag to see if it can take the shortcut and optimise the
lookup.

It's an optimisation that's pretty widely applicable because it improves
lookup performance for all methods of user defined classes, and stores of
attributes, at least for those that don't have special accessors.  And, it
allows to enable descriptors with minimal additional runtime overhead if
they are not used for a particular user class.

There is one restriction on dynamic class creation that has been introduced
by this patch: a user-defined class cannot go from zero special accessors
to one special accessor (or more) after that class has been subclassed.  If
the script attempts this an AttributeError is raised (see addition to
tests/misc/non_compliant.py for an example of this case).

The cost in code space bytes for the optimisation in this patch is:

   unix x64:  +528
unix nanbox:  +508
      stm32:  +192
     cc3200:  +200
    esp8266:  +332
      esp32:  +244

Performance tests that were done:

- on unix x86-64, pystone improved by about 5%
- on pyboard, pystone improved by about 6.5%, from 1683 up to 1794
- on pyboard, bm_chaos (from CPython benchmark suite) improved by about 5%
- on esp32, pystone improved by about 30% (but there are caching effects)
- on esp32, bm_chaos improved by about 11%
2018-06-08 12:12:08 +10:00
Damien George
bace1a16d0 py/objtype: Don't expose mp_obj_instance_attr().
mp_obj_is_instance_type() can be used instead to check for instance types.
2018-06-08 11:48:25 +10:00
Damien George
db5d8c97f1 py/obj.h: Introduce a "flags" entry in mp_obj_type_t. 2018-06-08 11:48:25 +10:00
Damien George
a8b9e71ac1 py/mpconfig.h: Add default MICROPY_VFS_FAT config value.
At least to document it's existence.
2018-06-06 14:33:42 +10:00
Damien George
a93144cb65 py/reader: Allow MICROPY_VFS_POSIX to work with MICROPY_READER_POSIX. 2018-06-06 14:28:23 +10:00
Damien George
8d82b0edbd extmod: Add VfsPosix filesystem component.
This VFS component allows to mount a host POSIX filesystem within the uPy
VFS sub-system.  All traditional POSIX file access then goes through the
VFS, allowing to sandbox a uPy process to a certain sub-dir of the host
system, as well as mount other filesystem types alongside the host
filesystem.
2018-06-06 14:28:23 +10:00
Damien George
1427f8f593 py/stream: Move definition of mp_stream_p_t from obj.h to stream.h.
Since a long time now, mp_obj_type_t no longer refers explicitly to
mp_stream_p_t but rather to an abstract "const void *protocol".  So there's
no longer any need to define mp_stream_p_t in obj.h and it can go with all
its associated definitions in stream.h.  Pretty much all users of this type
will already include the stream header.
2018-06-04 16:53:17 +10:00
Jeff Epler
c60589c02b py/objtype: Fix assertion failures in super_attr by checking type.
Fixes assertion failures and segmentation faults when making calls like:

    super(1, 1).x
2018-05-30 11:14:07 +10:00
Jeff Epler
05b13fd292 py/objtype: Fix assertion failures in mp_obj_new_type by checking types.
Fixes assertion failures when the arguments to type() were not of valid
types, e.g., when making calls like:

    type("", (), 3)
    type("", 3, {})
2018-05-30 11:11:24 +10:00
Damien George
dfeaea1441 py/objtype: Remove TODO comment about needing to check for property.
Instance members are always treated as values, even if they are properties.
A test is added to show this is the case.
2018-05-25 10:59:40 +10:00
Damien George
18e6358480 py/emit: Combine setup with/except/finally into one emit function.
This patch reduces code size by:

   bare-arm:   -16
minimal x86:  -156
   unix x64:  -288
unix nanbox:  -184
      stm32:   -48
     cc3200:   -16
    esp8266:   -96
      esp32:   -16

The last 10 patches combined reduce code size by:

   bare-arm:  -164
minimal x86: -1260
   unix x64: -3416
unix nanbox: -1616
      stm32:  -676
     cc3200:  -232
    esp8266: -1144
      esp32:  -268
2018-05-23 00:35:16 +10:00
Damien George
436e0d4c54 py/emit: Merge build set/slice into existing build emit function.
Reduces code size by:

   bare-arm:    +0
minimal x86:    +0
   unix x64:  -368
unix nanbox:  -248
      stm32:  -128
     cc3200:   -48
    esp8266:  -184
      esp32:   -40
2018-05-23 00:23:36 +10:00
Damien George
d97906ca9a py/emit: Combine import from/name/star into one emit function.
Change in code size is:

   bare-arm:    +4
minimal x86:   -88
   unix x64:  -456
unix nanbox:   -88
      stm32:   -44
     cc3200:    +0
    esp8266:  -104
      esp32:    +8
2018-05-23 00:23:08 +10:00
Damien George
8a513da5a5 py/emit: Combine break_loop and continue_loop into one emit function.
Reduces code size by:

   bare-arm:    +0
minimal x86:    +0
   unix x64:   -80
unix nanbox:    +0
      stm32:   -12
     cc3200:    +0
    esp8266:   -28
      esp32:    +0
2018-05-23 00:23:04 +10:00
Damien George
6211d979ee py/emit: Combine load/store/delete attr into one emit function.
Reduces code size by:

   bare-arm:   -20
minimal x86:  -140
   unix x64:  -408
unix nanbox:  -140
      stm32:   -68
     cc3200:   -16
    esp8266:   -80
      esp32:   -32
2018-05-23 00:22:59 +10:00
Damien George
a4941a8ba4 py/emit: Combine load/store/delete subscr into one emit function.
Reduces code size by:

   bare-arm:    -8
minimal x86:  -104
   unix x64:  -312
unix nanbox:  -120
      stm32:   -60
     cc3200:   -16
    esp8266:   -92
      esp32:   -24
2018-05-23 00:22:55 +10:00
Damien George
d298013939 py/emit: Combine name and global into one func for load/store/delete.
Reduces code size by:

   bare-arm:   -56
minimal x86:  -300
   unix x64:  -576
unix nanbox:  -300
      stm32:  -164
     cc3200:   -56
    esp8266:  -236
      esp32:   -76
2018-05-23 00:22:47 +10:00
Damien George
26b5754092 py/emit: Combine build tuple/list/map emit funcs into one.
Reduces code size by:

   bare-arm:   -24
minimal x86:  -192
   unix x64:  -288
unix nanbox:  -184
      stm32:   -72
     cc3200:   -16
    esp8266:  -148
      esp32:   -32
2018-05-23 00:22:44 +10:00
Damien George
e686c94052 py/emit: Combine yield value and yield-from emit funcs into one.
Reduces code size by:

   bare-arm:   -24
minimal x86:   -72
   unix x64:  -200
unix nanbox:   -72
      stm32:   -52
     cc3200:   -32
    esp8266:   -84
      esp32:   -24
2018-05-23 00:22:35 +10:00
Damien George
0a25fff956 py/emit: Combine fast and deref into one function for load/store/delete.
Reduces code size by:

   bare-arm:   -16
minimal x86:  -208
   unix x64:  -408
unix nanbox:  -248
      stm32:   -12
     cc3200:   -24
    esp8266:   -96
      esp32:   -44
2018-05-23 00:22:20 +10:00
Damien George
400273a799 py/objgenerator: Protect against reentering a generator.
Generators that are already executing cannot be reexecuted.  This patch
puts in a check for such a case.

Thanks to @jepler for finding the bug.
2018-05-22 16:54:03 +10:00
Damien George
771cb359af py/objgenerator: Save state in old_globals instead of local variable.
The code_state.old_globals variable is there to save the globals state so
should be used for this purpose, to avoid the need for additional local
variables on the C stack.
2018-05-22 16:39:19 +10:00
Jan Klusacek
b318ebf101 py/modbuiltins: Add support for rounding integers.
As per CPython semantics.  This feature is controlled by
MICROPY_PY_BUILTINS_ROUND_INT which is disabled by default.
2018-05-22 14:18:16 +10:00
Damien George
f2ec792554 py/parsenum: Adjust braces so they are balanced. 2018-05-22 13:20:00 +10:00
Damien George
6bd78741c1 py/gc: When GC threshold is hit don't unnecessarily collect twice.
Without this, if GC threshold is hit and there is not enough memory left to
satisfy the request, gc_collect() will run a second time and the search for
memory will happen again and will fail again.

Thanks to @adritium for pointing out this issue, see #3786.
2018-05-21 13:36:21 +10:00
Jeff Epler
95e43efc99 py/objfloat: Fix undefined integer behavior hashing negative zero.
Under ubsan, when evaluating hash(-0.) the following diagnostic occurs:

    ../../py/objfloat.c:102:15: runtime error: negation of
    -9223372036854775808 cannot be represented in type 'mp_int_t' (aka
    'long'); cast to an unsigned type to negate this value to itself

So do just that, to tell the compiler that we want to perform this
operation using modulo arithmetic rules.
2018-05-21 12:49:56 +10:00
Jeff Epler
c4dafcef4f py/mpz: Avoid undefined behavior at integer overflow in mpz_hash.
Before this, ubsan would detect a problem when executing
hash(006699999999999999999999999999999999999999999999999999999999999999999999)

    ../../py/mpz.c:1539:20: runtime error: left shift of 1067371580458 by
    32 places cannot be represented in type 'mp_int_t' (aka 'long')

When the overflow does occur it now happens as defined by the rules of
unsigned arithmetic.
2018-05-21 12:48:26 +10:00
Jeff Epler
60eb5305f6 py/objfloat: Fix undefined shifting behavior in high-quality float hash.
When computing e.g. hash(0.4e3) with ubsan enabled, a diagnostic like the
following would occur:

    ../../py/objfloat.c:91:30: runtime error: shift exponent 44 is too
    large for 32-bit type 'int'

By casting constant "1" to the right type the intended value is preserved.
2018-05-21 12:42:22 +10:00