Commit Graph

9055 Commits

Author SHA1 Message Date
Paul Sokolovsky
096e967aad Revert "py/nlr: Factor out common NLR code to generic functions."
This reverts commit 6a3a742a6c.

The above commit has number of faults starting from the motivation down
to the actual implementation.

1. Faulty implementation.

The original code contained functions like:

NORETURN void nlr_jump(void *val) {
    nlr_buf_t **top_ptr = &MP_STATE_THREAD(nlr_top);
    nlr_buf_t *top = *top_ptr;
...
     __asm volatile (
    "mov    %0, %%edx           \n" // %edx points to nlr_buf
    "mov    28(%%edx), %%esi    \n" // load saved %esi
    "mov    24(%%edx), %%edi    \n" // load saved %edi
    "mov    20(%%edx), %%ebx    \n" // load saved %ebx
    "mov    16(%%edx), %%esp    \n" // load saved %esp
    "mov    12(%%edx), %%ebp    \n" // load saved %ebp
    "mov    8(%%edx), %%eax     \n" // load saved %eip
    "mov    %%eax, (%%esp)      \n" // store saved %eip to stack
    "xor    %%eax, %%eax        \n" // clear return register
    "inc    %%al                \n" // increase to make 1, non-local return
     "ret                        \n" // return
    :                               // output operands
    : "r"(top)                      // input operands
    :                               // clobbered registers
     );
}

Which clearly stated that C-level variable should be a parameter of the
assembly, whcih then moved it into correct register.

Whereas now it's:

NORETURN void nlr_jump_tail(nlr_buf_t *top) {
    (void)top;

    __asm volatile (
    "mov    28(%edx), %esi      \n" // load saved %esi
    "mov    24(%edx), %edi      \n" // load saved %edi
    "mov    20(%edx), %ebx      \n" // load saved %ebx
    "mov    16(%edx), %esp      \n" // load saved %esp
    "mov    12(%edx), %ebp      \n" // load saved %ebp
    "mov    8(%edx), %eax       \n" // load saved %eip
    "mov    %eax, (%esp)        \n" // store saved %eip to stack
    "xor    %eax, %eax          \n" // clear return register
    "inc    %al                 \n" // increase to make 1, non-local return
    "ret                        \n" // return
    );

    for (;;); // needed to silence compiler warning
}

Which just tries to perform operations on a completely random register (edx
in this case). The outcome is the expected: saving the pure random luck of
the compiler putting the right value in the random register above, there's
a crash.

2. Non-critical assessment.

The original commit message says "There is a small overhead introduced
(typically 1 machine instruction)". That machine instruction is a call
if a compiler doesn't perform tail optimization (happens regularly), and
it's 1 instruction only with the broken code shown above, fixing it
requires adding more. With inefficiencies already presented in the NLR
code, the overhead becomes "considerable" (several times more than 1%),
not "small".

The commit message also says "This eliminates duplicated code.". An
obvious way to eliminate duplication would be to factor out common code
to macros, not introduce overhead and breakage like above.

3. Faulty motivation.

All this started with a report of warnings/errors happening for a niche
compiler. It could have been solved in one the direct ways: a) fixing it
just for affected compiler(s); b) rewriting it in proper assembly (like
it was before BTW); c) by not doing anything at all, MICROPY_NLR_SETJMP
exists exactly to address minor-impact cases like thar (where a) or b) are
not applicable). Instead, a backwards "solution" was put forward, leading
to all the issues above.

The best action thus appears to be revert and rework, not trying to work
around what went haywire in the first place.
2017-12-26 19:27:58 +02:00
Paul Sokolovsky
d9977a8ad9 zephyr/Makefile: clean: Clean libmicropython.a too. 2017-12-26 14:46:16 +02:00
Paul Sokolovsky
5de064fbd0 docs/library/index: Elaborate uPy libraries intro. 2017-12-23 21:21:08 +02:00
Damien George
b806889512 stm32/i2c: Support more I2C baudrates for F746, and more F7 MCUs. 2017-12-23 19:24:24 +11:00
Damien George
008e1788e8 stm32/i2c: Fix bug with I2C4 initialisation. 2017-12-23 19:22:52 +11:00
Damien George
d32417c096 stm32/uart: Support board configs with CTS/RTS on UART6. 2017-12-23 19:03:16 +11:00
Damien George
9bcdb0acd1 esp8266/Makefile: Remove commented-out unused lines.
These were copied from the stm32 port (then stmhal) at the very beginning
of this port, with the anticipation that the esp8266 port would have board
definition files with a list of valid pins and their names.  But that has
not been implemented and likely won't be, so remove the corresponding lines
from the Makefile.
2017-12-22 17:16:42 +11:00
Ayke van Laethem
b90f51f86a drivers/sdcard: Support old SD cards (<=2GB). 2017-12-22 16:49:58 +11:00
Ayke van Laethem
f16c775a07 esp32/README: Update toolchain setup. 2017-12-22 16:27:24 +11:00
Peter D. Gray
7a46d9ae73 stm32/uart: Add support for 7-bit modes: 7N1 and 7N2. 2017-12-22 15:37:17 +11:00
Damien George
c73360bfdb stm32: Allow to build a board without any hardware I2C ports defined.
This patch adds in internal config value MICROPY_HW_ENABLE_HW_I2C that is
automatically configured, and enabled only if one or more hardware I2C
ports are defined in the mpconfigboard.h file.  If none are defined then
the pyb.I2C class is excluded from the build, along with all supporting
code.  The machine.I2C class will still be available for software I2C.

Disabling all hardware I2C on an F4 board saves around 10,000 bytes of code
and 200 bytes of RAM.
2017-12-22 15:20:42 +11:00
Peter D. Gray
82dc5c1d8c stm32: Use corrected capitalization of HAL_SD_CardStateTypedef.
It was originally TypeDef.  STM32L4 only supports Typedef and F4/F7 have
legacy macros in stm32_hal_legacy.h to support both.
2017-12-22 14:49:31 +11:00
Damien George
a1d85d6199 tests/basics/memoryerror: Add test for out-of-memory using realloc. 2017-12-20 16:58:27 +11:00
Damien George
26d4a6fa45 py/malloc: Remove unneeded code checking m_malloc return value.
m_malloc already checks for a failed allocation so there's no need to check
for it in m_malloc0.
2017-12-20 16:55:42 +11:00
Damien George
d7a52e1539 qemu-arm/test_main: Include setjmp.h because it's used by gc_collect.
And it's no longer unconditionally included by nlr.h, only if NLR_SETJMP
is defined.
2017-12-20 15:42:06 +11:00
Damien George
6a3a742a6c py/nlr: Factor out common NLR code to generic functions.
Each NLR implementation (Thumb, x86, x64, xtensa, setjmp) duplicates a lot
of the NLR code, specifically that dealing with pushing and popping the NLR
pointer to maintain the linked-list of NLR buffers.  This patch factors all
of that code out of the specific implementations into generic functions in
nlr.c.  This eliminates duplicated code.

The factoring also allows to make the machine-specific NLR code pure
assembler code, thus allowing nlrthumb.c to use naked function attributes
in the correct way (naked functions can only have basic inline assembler
code in them).

There is a small overhead introduced (typically 1 machine instruction)
because now the generic nlr_jump() must call nlr_jump_tail() rather than
them being one combined function.
2017-12-20 15:42:06 +11:00
Damien George
d8d633f156 unix/mpconfigport_coverage.h: Enable MICROPY_PY_IO_RESOURCE_STREAM.
Where possible it's important to test all code in the code base.
2017-12-19 17:04:55 +11:00
Damien George
304a3bcc1c py/modio: Use correct config macro to enable resource_stream function. 2017-12-19 16:59:08 +11:00
Damien George
d35c6ffc84 tests/extmod: Add some uctypes tests to improve coverage of that module. 2017-12-19 16:48:41 +11:00
Damien George
35a759dc1d tests: Add some more tests to improve coverage of py/parse.c. 2017-12-19 16:13:00 +11:00
Damien George
ae1be76d40 py/mpz: Apply a small code-size optimisation. 2017-12-19 15:45:56 +11:00
Damien George
2bfa531798 tests/basics/builtin_pow3: Add tests for edge cases of pow3. 2017-12-19 15:44:10 +11:00
Damien George
374eaf5271 py/mpz: Fix pow3 function so it handles the case when 3rd arg is 1.
In this case the result should always be 0, even if 2nd arg is 0.
2017-12-19 15:42:58 +11:00
Damien George
8e6113a188 tests/basics/generator_pend_throw: Add test for just-started generator. 2017-12-19 15:02:34 +11:00
Damien George
e800e4463d tests/unix: Add test for printf with %lx format. 2017-12-19 15:01:17 +11:00
Damien George
7cae17fac7 tests/float/builtin_float_hash: Add test to improve objfloat.c coverage. 2017-12-19 14:50:33 +11:00
Damien George
251b00457c tests/extmod/uhashlib_sha256: Add test for hashing 56 bytes of data. 2017-12-19 14:46:31 +11:00
Damien George
7db79d8b03 py/objset: Remove unneeded check from set_equal.
set_equal is called only from set_binary_op, and this guarantees that the
second arg to set_equal is always a set or frozenset.  So there is no need
to do a further check.
2017-12-19 14:01:19 +11:00
Damien George
7208cad97a tests/basics: Add more set tests to improve coverage of py/objset.c. 2017-12-19 13:59:54 +11:00
Damien George
136cb7f27c py/map: Don't include ordered-dict mutating code when not needed. 2017-12-19 13:37:15 +11:00
Damien George
f5fb68e94f py/runtime: Remove unnecessary break statements from switch. 2017-12-19 13:13:21 +11:00
Damien George
d1fd889ad0 esp32/machine_hw_spi: Remove unnecessary white space for consistency. 2017-12-18 15:46:08 +11:00
Eric Poulsen
9123c8d946 esp32/machine_hw_spi: Fix large HW SPI transfers by splitting them up.
Breaks up HW SPI transfers into maximum chunks of 32736 bits (4092 bytes),
because this is the maximum that the underlying ESP IDF will accept.
2017-12-18 15:44:35 +11:00
Paul Sokolovsky
5455bf79c5 .travis.yml: Build and test strict stackless build.
Previously, testing of stackless build happened (manually) in
travis-stackless branch. However, stackless offers important
featureset, so it's worth to test it as a part of the main
CI. Strict stackless is used because it's the "real" stackless
build, which avoids using C stack as much as possible (non-strict
just prefers heap over C stack, but may end up using the latter).
2017-12-16 20:43:30 +02:00
Paul Sokolovsky
ea742085ed unix/mpconfigport.h: Allow to override stackless options from commandline. 2017-12-16 20:43:04 +02:00
Paul Sokolovsky
7f9a62408d unix/Makefile: coverage: Allow user to pass CFLAGS_EXTRA.
This build sets CFLAGS_EXTRA itself, but preserve user's value as passed
on make command line/etc.
2017-12-16 20:23:12 +02:00
Paul Sokolovsky
e37ccfe59b docs/packages: Explicitly recommend usage of setuptools instead of distutils. 2017-12-16 10:42:30 +02:00
Paul Sokolovsky
9251f1395e docs/packages: Use "install_dir/" in examples. 2017-12-16 10:37:36 +02:00
Paul Sokolovsky
02d2a0fb3a docs/conf: Reference CPython 3.5 docs.
CPython 3.6 contains some backward incompatible changes, and further
version(s) are expected to have more. As we anyway implemente 3.4 with
some features of 3.5, refer to 3.5 docs to avoid confusion.

Examples of 3.6 backward incompatibilities:

https://docs.python.org/3.6/library/json.html#json.dump
https://docs.python.org/3.6/library/json.html#json.load

> Changed in version 3.6: All optional parameters are now keyword-only.

https://docs.python.org/3.6/library/functions.html#type

> Changed in version 3.6: Subclasses of type which don’t override
> type.__new__ may no longer use the one-argument form to get the
> type of an object.

https://docs.python.org/3.6/library/collections.html#collections.namedtuple

> Changed in version 3.6: The verbose and rename parameters became
> keyword-only arguments.
2017-12-16 01:22:46 +02:00
Paul Sokolovsky
6364401666 py/objgenerator: Allow to pend an exception for next execution.
This implements .pend_throw(exc) method, which sets up an exception to be
triggered on the next call to generator's .__next__() or .send() method.
This is unlike .throw(), which immediately starts to execute the generator
to process the exception. This effectively adds Future-like capabilities
to generator protocol (exception will be raised in the future).

The need for such a method arised to implement uasyncio wait_for() function
efficiently (its behavior is clearly "Future" like, and normally would
require to introduce an expensive Future wrapper around all native
couroutines, like upstream asyncio does).

py/objgenerator: pend_throw: Return previous pended value.

This effectively allows to store an additional value (not necessary an
exception) in a coroutine while it's not being executed. uasyncio has
exactly this usecase: to mark a coro waiting in I/O queue (and thus
not executed in the normal scheduling queue), for the purpose of
implementing wait_for() function (cancellation of such waiting coro
by a timeout).
2017-12-15 20:20:36 +02:00
Paul Sokolovsky
f4ed2dfa94 lib/tinytest: Clean up test reporting in the presence of stdout output.
tinytest is written with the idea that tests won't write to stdout, so it
prints test name witjout newline, then executes test, then writes status.
But MicroPython tests write to stdout, so the test output becomes a mess.
So, instead print it like:

    # starting basics/andor.py
    ... test output ...
    basics/andor.py: OK
2017-12-15 19:41:08 +02:00
Paul Sokolovsky
dd35fe7ca0 zephyr/prj_base.conf: Bump MAIN_STACK_SIZE to let builtin testsuite run. 2017-12-15 18:17:00 +02:00
Paul Sokolovsky
6b19520a74 zephyr: Add support for binary with builtin testsuite.
If TEST is defined, file it refers to will be used as the testsuite
source (should be generated with tools/tinytest-codegen.py).

"make-bin-testsuite" script is introduce to build such a binary.
2017-12-15 12:10:39 +02:00
Paul Sokolovsky
103eeffcd9 tests/run-tests: Skip running feature checks for --list-tests/--write-exp.
The whole idea of --list-tests is that we prepare a list of tests to run
later, and currently don't have a connection to target board. Similarly
for --write-exp - only "python3" binary would be required for this operation,
not "micropython".
2017-12-15 12:07:09 +02:00
Paul Sokolovsky
4475f32420 tools/tinytest-codegen: Ignore system locale, write output in UTF-8.
Way to reproduce a UnicodeEncodeError without this patch:

    LC_ALL=C tinytest-codegen.py ...
2017-12-15 12:04:10 +02:00
Damien George
cf8e8c29e7 py/emitglue: Change type of bit-field to explicitly unsigned mp_uint_t.
Some compilers can treat enum types as signed, in which case 3 bits is not
enough to encode all mp_raw_code_kind_t values.  So change the type to
mp_uint_t.
2017-12-15 10:21:10 +11:00
Paul Sokolovsky
448d93a04a docs/glossary: micropython-lib: Clarify wording. 2017-12-15 00:11:02 +02:00
Paul Sokolovsky
bf73ee114f docs/packages: mpy_bin2res no longer required to create resources.
Everything happens automagically with overridden "sdist" from
sdist_upip.py.
2017-12-14 18:28:10 +02:00
Paul Sokolovsky
3233537a15 tests/run-tests: Don't test for --target=unix with "pyb is None".
If we test for unix target, do that explicitly. pyb var will be None
for commands like --list-tests too.
2017-12-14 13:36:06 +02:00
Paul Sokolovsky
325d0fc74b tools/tinytest-codegen: Add --stdin switch instead of recently added --target.
Instead of passing thru more and more options from tinytest-codegen to
run-tests --list-tests, pipe output of run-tests --list-tests into
tinytest-codegen.
2017-12-14 12:26:59 +02:00