Commit Graph

33 Commits

Author SHA1 Message Date
Damien George cf31d384f1 py/stream: Switch stream close operation from method to ioctl.
This patch moves the implementation of stream closure from a dedicated
method to the ioctl of the stream protocol, for each type that implements
closing.  The benefits of this are:

1. Rounds out the stream ioctl function, which already includes flush,
   seek and poll (among other things).

2. Makes calling mp_stream_close() on an object slightly more efficient
   because it now no longer needs to lookup the close method and call it,
   rather it just delegates straight to the ioctl function (if it exists).

3. Reduces code size and allows future types that implement the stream
   protocol to be smaller because they don't need a dedicated close method.

Code size reduction is around 200 bytes smaller for x86 archs and around
30 bytes smaller for the bare-metal archs.
2018-04-10 13:41:32 +10:00
Damien George 0b88a9f02e unix/coverage: Allow coverage tests to pass with debugging disabled. 2018-03-08 12:49:31 +11:00
Damien George c607b58efe tests: Move heap-realloc-while-locked test from C to Python.
This test for calling gc_realloc() while the GC is locked can be done in
pure Python, so better to do it that way since it can then be tested on
more ports.
2018-03-02 10:59:09 +11:00
Damien George c3f1b22338 tests/unix: Add coverage tests for various GC calls. 2018-03-01 22:49:15 +11:00
Damien George d3cac18d49 tests/unix: Add coverage test for VM executing invalid bytecode. 2018-02-27 16:18:11 +11:00
Damien George 62be14d77c tests/unix: Add coverage tests for mpz_set_from_float, mpz_mul_inpl.
These new tests cover cases that can't be reached from Python and get
coverage of py/mpz.c to 100%.

These "unreachable from Python" pieces of code could be removed but they
form an integral part of the mpz C API and may be useful for non-Python
usage of mpz.
2018-02-25 23:43:16 +11:00
Damien George 82828340a0 ports: Enable ucollections.deque on relevant ports.
These ports are all capable of running uasyncio.
2018-02-21 22:55:13 +11:00
Damien George ab7819c314 unix/mpconfigport_coverage: Enable range (in)equality comparison. 2018-02-14 23:22:02 +11:00
Damien George 24c513cbc3 unix/Makefile,embedding/Makefile: Remove obsolete use of STMHAL_SRC_C. 2018-02-14 15:24:21 +11:00
Damien George 0b12cc8feb .travis.yml,ports/unix/Makefile: Add coverage test for script via stdin. 2018-02-08 11:30:19 +11:00
Damien George 923ebe767d tests/unix: Add coverage test for calling mp_obj_new_bytearray. 2018-02-08 11:14:30 +11:00
Damien George d8d633f156 unix/mpconfigport_coverage.h: Enable MICROPY_PY_IO_RESOURCE_STREAM.
Where possible it's important to test all code in the code base.
2017-12-19 17:04:55 +11:00
Damien George e800e4463d tests/unix: Add test for printf with %lx format. 2017-12-19 15:01:17 +11:00
Paul Sokolovsky ea742085ed unix/mpconfigport.h: Allow to override stackless options from commandline. 2017-12-16 20:43:04 +02:00
Paul Sokolovsky 7f9a62408d unix/Makefile: coverage: Allow user to pass CFLAGS_EXTRA.
This build sets CFLAGS_EXTRA itself, but preserve user's value as passed
on make command line/etc.
2017-12-16 20:23:12 +02:00
Damien George 357486d9b4 unix: Add support for using the Python stack. 2017-12-11 13:49:09 +11:00
Paul Sokolovsky e7fc765880 unix/mpconfigport: Disable uio.resource_stream().
This function was implemented as an experiment, and was enabled only in
unix port. To remind, it allows to access arbitrary files frozen as
source modules (vs bytecode).

However, further experimentation showed that the same functionality can
be implemented with frozen bytecode. The process requires more steps, but
with suitable toolset it doesn't matter patch. This process is:

1. Convert binary files into "Python resource module" with
tools/mpy_bin2res.py.
2. Freeze as the bytecode.
3. Use micropython-lib's pkg_resources.resource_stream() to access it.

In other words, the extra step is using tools/mpy_bin2res.py (because
there would be wrapper for uio.resource_stream() anyway).

Going frozen bytecode route allows more flexibility, and same/additional
efficiency:

1. Frozen source support can be disabled altogether for additional code
savings.
2. Resources could be also accessed as a buffer, not just as a stream.

There're few caveats too:

1. It wasn't actually profiled the overhead of storing a resource in
"Python resource module" vs storing it directly, but it's assumed that
overhead is small.
2. The "efficiency" claim above applies to the case when resource
file is frozen as the bytecode. If it's not, it actually will take a
lot of RAM on loading. But in this case, the resource file should not
be used (i.e. generated) in the first place, and micropython-lib's
pkg_resources.resource_stream() implementation has the appropriate
fallback to read the raw files instead. This still poses some distribution
issues, e.g. to deployable to baremetal ports (which almost certainly
would require freezeing as the bytecode), a distribution package should
include the resource module. But for non-freezing deployment, presense
of resource module will lead to memory inefficiency.

All the discussion above reminds why uio.resource_stream() was implemented
in the first place - to address some of the issues above. However, since
then, frozen bytecode approach seems to prevail, so, while there're still
some issues to address with it, this change is being made.

This change saves 488 bytes for the unix x86_64 port.
2017-12-10 02:38:23 +02:00
Damien George 4601759bf5 py/objstr: Remove "make_qstr_if_not_already" arg from mp_obj_new_str.
This patch simplifies the str creation API to favour the common case of
creating a str object that is not forced to be interned.  To force
interning of a new str the new mp_obj_new_str_via_qstr function is added,
and should only be used if warranted.

Apart from simplifying the mp_obj_new_str function (and making it have the
same signature as mp_obj_new_bytes), this patch also reduces code size by a
bit (-16 bytes for bare-arm and roughly -40 bytes on the bare-metal archs).
2017-11-16 13:17:51 +11:00
Christopher Cooper 7413b3ce3e extmod/moduhashlib: Enable SHA1 hashing when using "mbedtls" library.
The SHA1 hashing functionality is provided via the "axtls" library's
implementation, and hence is unavailable when the "axtls" library is not being
used.  This change provides the same SHA1 hashing functionality when using the
"mbedtls" library by using its implementation instead.
2017-11-12 21:46:23 +02:00
stijn 79ed58f87b py/objnamedtuple: Add _asdict function if OrderedDict is supported 2017-11-12 14:16:54 +02:00
Paul Sokolovsky b9580b85a8 unix/moduselect: Fix nanbox build after adding .dump() method. 2017-11-07 01:13:19 +02:00
Paul Sokolovsky cb910c6a0c unix/moduselect: Add .dump() method for debugging.
Commented out by default.
2017-11-07 00:43:21 +02:00
Paul Sokolovsky 24c8eda744 unix: Enable MICROPY_PY_REVERSE_SPECIAL_METHODS.
With inplace methods now disabled by default, it makes sense to enable
reverse methods, as they allow for more useful features, e.g. allow
for datetime module to implement both 2 * HOUR and HOUR * 2 (where
HOUR is e.g. timedelta object).
2017-10-28 13:05:57 +03:00
Paul Sokolovsky 9a7e3469b2 unix/modusocket: Remove #if MICROPY_SOCKET_EXTRA code blocks.
These defined couple of functions added during initial experimentation,
which aren't part of MicroPython API and no longer used or needed.
2017-10-24 23:13:19 +03:00
Damien George f4059dcc0c all: Use NULL instead of "" when calling mp_raise exception helpers.
This is the established way of doing it and reduces code size by a little
bit.
2017-10-24 22:39:36 +11:00
Paul Sokolovsky cfff12612f unix: Rename modsocket.c to modusocket.c.
Unix naming is historical, before current conventions were established.
All other ports however have it as "modusocket.c", so rename for
consistency and to avoid confusion.
2017-10-23 12:09:37 +03:00
Damien George 37282f8fc1 extmod/uos_dupterm: Update uos.dupterm() and helper funcs to have index.
The uos.dupterm() signature and behaviour is updated to reflect the latest
enhancements in the docs.  It has minor backwards incompatibility in that
it no longer accepts zero arguments.

The dupterm_rx helper function is moved from esp8266 to extmod and
generalised to support multiple dupterm slots.

A port can specify multiple slots by defining the MICROPY_PY_OS_DUPTERM
config macro to an integer, being the number of slots it wants to have;
0 means to disable the dupterm feature altogether.

The unix and esp8266 ports are updated to work with the new interface and
are otherwise unchanged with respect to functionality.
2017-10-13 20:01:57 +11:00
Damien George a3dc1b1957 all: Remove inclusion of internal py header files.
Header files that are considered internal to the py core and should not
normally be included directly are:
    py/nlr.h - internal nlr configuration and declarations
    py/bc0.h - contains bytecode macro definitions
    py/runtime0.h - contains basic runtime enums

Instead, the top-level header files to include are one of:
    py/obj.h - includes runtime0.h and defines everything to use the
        mp_obj_t type
    py/runtime.h - includes mpstate.h and hence nlr.h, obj.h, runtime0.h,
        and defines everything to use the general runtime support functions

Additional, specific headers (eg py/objlist.h) can be included if needed.
2017-10-04 12:37:50 +11:00
David Lechner 62849b7010 py: Add config option to print warnings/errors to stderr.
This adds a new configuration option to print runtime warnings and errors to
stderr. On Unix, CPython prints warnings and unhandled exceptions to stderr,
so the unix port here is configured to use this option.

The unix port already printed unhandled exceptions on the main thread to
stderr. This patch fixes unhandled exceptions on other threads and warnings
(issue #2838) not printing on stderr.

Additionally, a couple tests needed to be fixed to handle this new behavior.
This is done by also capturing stderr when running tests.
2017-09-26 11:59:11 +10:00
Damien George ede8a0235b py/vstr: Raise a RuntimeError if fixed vstr buffer overflows.
Current users of fixed vstr buffers (building file paths) assume that there
is no overflow and do not check for overflow after building the vstr.  This
has the potential to lead to NULL pointer dereferences
(when vstr_null_terminated_str returns NULL because it can't allocate RAM
for the terminating byte) and stat'ing and loading invalid path names (due
to the path being truncated).  The safest and simplest thing to do in these
cases is just raise an exception if a write goes beyond the end of a fixed
vstr buffer, which is what this patch does.  It also simplifies the vstr
code.
2017-09-21 20:29:41 +10:00
Paul Sokolovsky eb84a830df py/runtime: Implement dispatch for "reverse op" special methods.
If, for class X, X.__add__(Y) doesn't exist (or returns NotImplemented),
try Y.__radd__(X) instead.

This patch could be simpler, but requires undoing operand swap and
operation switch to get non-confusing error message in case __radd__
doesn't exist.
2017-09-10 17:05:57 +03:00
Damien George 4a93801c12 all: Update Makefiles and others to build with new ports/ dir layout.
Also renames "stmhal" to "stm32" in documentation and everywhere else.
2017-09-06 14:09:13 +10:00
Damien George 01dd7804b8 ports: Make new ports/ sub-directory and move all ports there.
This is to keep the top-level directory clean, to make it clear what is
core and what is a port, and to allow the repository to grow with new ports
in a sustainable way.
2017-09-06 13:40:51 +10:00