splitlines() occurs ~179 times in CPython3 standard library, so was
deemed worthy to implement. The method has subtle semantic differences
from just .split("\n"). It is also defined as working for any end-of-line
combination, but this is currently not implemented - it works only with
LF line-endings (which should be OK for text strings on any platforms,
but not OK for bytes).
Given that there's already support for "fixed table" maps, which are
essentially ordered maps, the implementation of OrderedDict just extends
"fixed table" maps by adding an "is ordered" flag and add/remove
operations, and reuses 95% of objdict code, just making methods tolerant
to both dict and OrderedDict.
Some things are missing so far, like CPython-compatible repr and comparison.
OrderedDict is Disabled by default; enabled on unix and stmhal ports.
These allow to fine-tune the compiler to select whether it optimises
tuple assignments of the form a, b = c, d and a, b, c = d, e, f.
Sensible defaults are provided.
This is rarely used feature which takes enough code to implement, so is
controlled by MICROPY_PY_ARRAY_SLICE_ASSIGN config setting, default off.
But otherwise it may be useful, as allows to update arbitrary-sized data
buffers in-place.
Slice is yet to implement, and actually, slice assignment implemented in
such a way that RHS of assignment should be array of the exact same item
typecode as LHS. CPython has it more relaxed, where RHS can be any sequence
of compatible types (e.g. it's possible to assign list of int's to a
bytearray slice).
Overall, when all "slice write" features are implemented, it may cost ~1KB
of code.
The implementation of these functions is very large (order 4k) and they
are rarely used, so we don't enable them by default.
They are however enabled in stmhal and unix, since we have the room.
To enable parsing constants more efficiently, mp_parse should be allowed
to raise an exception, and mp_compile can already raise a MemoryError.
So these functions need to be protected by an nlr push/pop block.
This patch adds that feature in all places. This allows to simplify how
mp_parse and mp_compile are called: they now raise an exception if they
have an error and so explicit checking is not needed anymore.
Native code has GC-heap pointers in it so it must be scanned. But on
unix port memory for native functions is mmap'd, and so it must have
explicit code to scan it for root pointers.
Compiler optimises lookup of module.CONST when enabled (an existing
feature). Disabled by default; enabled for unix, windows, stmhal.
Costs about 100 bytes ROM on stmhal.
This allows to enable mem-info functions in micropython module, even if
MICROPY_MEM_STATS is not enabled. In this case, you get mem_info and
qstr_info but not mem_{total,current,peak}.
GC for unix/windows builds doesn't make use of the bss section anymore,
so we do not need the (sometimes complicated) build features and code related to it
This is a simple optimisation inspired by JITing technology: we cache in
the bytecode (using 1 byte) the offset of the last successful lookup in
a map. This allows us next time round to check in that location in the
hash table (mp_map_t) for the desired entry, and if it's there use that
entry straight away. Otherwise fallback to a normal map lookup.
Works for LOAD_NAME, LOAD_GLOBAL, LOAD_ATTR and STORE_ATTR opcodes.
On a few tests it gives >90% cache hit and greatly improves speed of
code.
Disabled by default. Enabled for unix and stmhal ports.
This patch consolidates all global variables in py/ core into one place,
in a global structure. Root pointers are all located together to make
GC tracing easier and more efficient.
This patch makes MICROPY_PY_BUILTINS_SET compile-time option fully
disable the builtin set object (when set to 0). This includes removing
set constructor/comprehension from the grammar, the compiler and the
emitters. Now, enabling set costs 8168 bytes on unix x64, and 3576
bytes on stmhal.
system() is the basic function to support automation of tasks, so have it
available builtin, for example, for bootstrapping rest of micropython
environment.
This patch adds a configuration option (MICROPY_CAN_OVERRIDE_BUILTINS)
which, when enabled, allows to override all names within the builtins
module. A builtins override dict is created the first time the user
assigns to a name in the builtins model, and then that dict is searched
first on subsequent lookups. Note that this implementation doesn't
allow deleting of names.
This patch also does some refactoring of builtins code, creating the
modbuiltins.c file.
Addresses issue #959.
The function is modeled after traceback.print_exception(), but unbloated,
and put into existing module to save overhead on adding another module.
Compliant traceback.print_exception() is intended to be implemented in
micropython-lib in terms of sys.print_exception().
This change required refactoring mp_obj_print_exception() to take pfenv_t
interface arguments.
Addresses #751.
mp_obj_int_get_truncated is used as a "fast path" int accessor that
doesn't check for overflow and returns the int truncated to the machine
word size, ie mp_int_t.
Use mp_obj_int_get_truncated to fix struct.pack when packing maximum word
sized values.
Addresses issues #779 and #998.
mp_lexer_t type is exposed, mp_token_t type is removed, and simple lexer
functions (like checking current token kind) are now inlined.
This saves 784 bytes ROM on 32-bit unix, 348 bytes on stmhal, and 460
bytes on bare-arm. It also saves a tiny bit of RAM since mp_lexer_t
is a bit smaller. Also will run a bit more efficiently.
The specifier should go after the number, before size suffix like 'k' or 'm'.
E.g.: "-X heapsize=100wk" will use 100K heap on 32-bit system and 200K - on
64-bit.
This build is primarily intended for benchmarking, and may have random
features enabled/disabled to get high scores in synthetic benchmarks.
The intent is to show/prove that MicroPython codebase can compete with
CPython, when configured appropriately. But the main MicroPython aim
still remains to optimize for memory usage (which inevitibly leads to
performance degradation in some areas on some workloads).
gc.enable/disable are now the same as CPython: they just control whether
automatic garbage collection is enabled or not. If disabled, you can
still allocate heap memory, and initiate a manual collection.
Support for packages as argument not implemented, but otherwise error and
exit handling should be correct. This for example will allow to do:
pip-micropython install micropython-test.pystone
micropython -m test.pystone
This allows to implement KeyboardInterrupt on unix, and a much safer
ctrl-C in stmhal port. First ctrl-C is a soft one, with hope that VM
will notice it; second ctrl-C is a hard one that kills anything (for
both unix and stmhal).
One needs to check for a pending exception in the VM only for jump
opcodes. Others can't produce an infinite loop (infinite recursion is
caught by stack check).
This makes open() and _io.FileIO() more CPython compliant.
The mode kwarg is fully iplemented.
The encoding kwarg is allowed but not implemented; mainly to allow
the tests to specify encoding for CPython, see #874
Also, usocket.readinto(). Known issue is that .readinto() should be available
only for binary files, but micropython uses single method table for both
binary and text files.
Per new conventions, we'd like to consistently use "u*" naming conventions
for modules which don't offer complete CPython compatibility, while offer
subset or similar API.
It seems most sensible to use size_t for measuring "number of bytes" in
malloc and vstr functions (since that's what size_t is for). We don't
use mp_uint_t because malloc and vstr are not Micro Python specific.
Force OSX to compile with clang even if gcc is available
Change LDFLAGS syntax to be compatible with clang
Fix questionable syntax on line 90
Remove extraneous tab character
sys.exit always raises SystemExit so doesn't need a special
implementation for each port. If C exit() is really needed, use the
standard os._exit function.
Also initialise mp_sys_path and mp_sys_argv in teensy port.
For the sake of older versions of gcc (and other compilers), don't use
the #warning CPP directive, nor the -Wno-error=cpp option.
Also, fix a strict alias warning in modffi.c for older compilers, and
add a test for ffi module.
Addresses issue #847.
Because (for Thumb) a function pointer has the LSB set, pointers to
dynamic functions in RAM (eg native, viper or asm functions) were not
being traced by the GC. This patch is a comprehensive fix for this.
Addresses issue #820.
Also provides setraw() function from "tty" module (which in CPython is
implemented in Python). The idea here is that 95% of "termios" module usage
is to set raw mode to allow access to normal serial devices. Then, instead
of exporting gazillion termios symbols, it's better to implement it in C,
and export minimal number of symbols (mostly baud rates and drain values).
qstr_init is always called exactly before mp_init, so makes sense to
just have mp_init call it. Similarly with
mp_init_emergency_exception_buf. Doing this makes the ports simpler and
less error prone (ie they can no longer forget to call these).
The user code should call micropython.alloc_emergency_exception_buf(size)
where size is the size of the buffer used to print the argument
passed to the exception.
With the test code from #732, and a call to
micropython.alloc_emergenncy_exception_buf(100) the following error is
now printed:
```python
>>> import heartbeat_irq
Uncaught exception in Timer(4) interrupt handler
Traceback (most recent call last):
File "0://heartbeat_irq.py", line 14, in heartbeat_cb
NameError: name 'led' is not defined
```
With unicode enabled, this patch allows reading a fixed number of
characters from text-mode streams; eg file.read(5) will read 5 unicode
chars, which can made of more than 5 bytes.
For an ASCII stream (ie no chars > 127) it only needs to do 1 read. If
there are lots of non-ASCII chars in a stream, then it needs multiple
reads of the underlying object.
Adds a new test for this case. Enables unicode support by default on
unix and stmhal ports.
- rearrange/add definitions that were not there so it's easier to compare both
- use MICROPY_PY_SYS_PLATFORM in main.c since it's available anyway
- define EWOULDBLOCK, it is missing from ingw32
As stack checking is enabled by default, ports which don't call
stack_ctrl_init() are broken now (report RuntimeError on startup). Save
them trouble and just init stack control framework in interpreter init.
Such mechanism is important to get stable Python functioning, because Python
function calling is handled with C stack. The idea is to sprinkle
STACK_CHECK() calls in places where there can be C recursion.
TODO: Add more STACK_CHECK()'s.
The idea is that it should be possible to pass any additional params for
experimentation without need to patch sources (and without need to deviate
from or repeat baseline options).
Some people want to enable even more warnings. Let them do it without putting
burden on everyone. Some people vice versa think that current settings should
be relaxed. In this regard, -Werror is the most problematic, it disallows to
use #warning directive, and disallows to pass configuration settings on make
command lines. Again, until decided how to deal with these globally, allow to
work around these problems locally.
cast error in MP_OBJ_NEW_SMALL_INT(). This is necessary for FreeBSD, where
st_ino is of different size
- If MP_CLOCKS_PER_SEC is defined on the target host, simply define CLOCK_DIV
as a fraction, regardless of the value of MP_CLOCKS_PER_SEC.
FreeBSD uses a non-POSIX compliant value of 128 for CLOCKS_PER_SEC
there are special tweaks and paths to be considered. Just provide some
defaults, in case the values are undefined.
- py-version.sh does not need any bash specific features.
- Use libdl only on Linux for now. FreeBSD provides dl*() calls from libc.
As I suspected for a long time, for x86, register helper doesn't really make
any difference - there's simply not enough register to keep anything in
them for any prolonged time. Anything gets pushed on stack anyway. So, on
x86, uPy passed all tests even with empty reg helper. So, this setjmp
implementation goes as "untested".
This reverts commit 6e76f7bc90.
This patch tries to workaround a previous clang workaround. Instead of going
into workaround of workaround spiral, the original workaround should be tamed.
Without this fix, I get the following error:
CC gccollect.c
gccollect.c: In function ‘gc_helper_get_regs’:
gccollect.c:63:1: error: bp cannot be used in asm here
io.FileIO is binary I/O, ans actually optional. Default file type is
io.TextIOWrapper, which provides str results. CPython3 explicitly describes
io.TextIOWrapper as buffered I/O, but we don't have buffering support yet
anyway.
stat() is bad function to use using FFI, because its ABI is largely private.
To start with, Glibc .so doesn't even have "stat" symbol. Then, layout of
struct stat is too implementation-dependent. So, introduce _os to deal
with stat() and other similar cases.
When disabling these via mpconfigport.mk or on the commandline,
the correspoding build options are not set and the sources are not
built so the modules should not be added to the
MICROPY_EXTRA_BUILTIN_MODULES list since they are undefined.
This will work if MICROPY_DEBUG_PRINTERS is defined, which is only for
unix/windows ports. This makes it convenient to user uPy normally, but
easily get bytecode dump on the spot if needed, without constant recompiles
back and forth.
TODO: Add more useful debug output, adjust verbosity level on which
specifically bytecode dump happens.
Blanket wide to all .c and .h files. Some files originating from ST are
difficult to deal with (license wise) so it was left out of those.
Also merged modpyb.h, modos.h, modstm.h and modtime.h in stmhal/.
The mingw port used _fullpath() until now, but the behaviour is not exactly
the same as realpath()'s on unix; major difference being that it doesn't
return an error for non-existing files, which would bypass main's error
checking and bail out without any error message.
Also realpath() will return forward slashes only since main() relies on that.
Some BSD socket functions don't return error numbers in errno namespace, but
rather in other error namespaces. CPython resolves this by using OSError
subclasses for them. We don't do that so far, so there's ambiguity here.
On stmhal, computed gotos make the binary about 1k bigger, but makes it
run faster, and we have the room, so why not. All tests pass on
pyboard using computed gotos.
The autogenerated header files have been moved about, and an extra
include dir has been added, which means you can give a custom
BUILD=newbuilddir option to make, and everything "just works"
Also tidied up the way the different Makefiles build their include-
directory flags
In conjunction with #504 this allows you to do things like:
```shell
make -C unix clean && make -C unix test CC=gcc-4.7
```
all from the top-level micropython directory :-)
Something similar could probably be done for windows/Makefile too, but I don't have a cygwin setup to test with.
ffi is needed to use micropython-lib, so let's have it enabled by default,
then folks who have troubles with libffi can disable it, instead of everyone
doing manual actions again and again.
Full CPython compatibility with this requires actually parsing the
input so far collected, and if it fails parsing due to lack of tokens,
then continue collecting input. It's not worth doing it this way. Not
having compatibility at this level does not hurt the goals of Micro
Python.
This is to reduce ROM usage. stream_p is used in file and socket types
only (at the moment), so seems a good idea to make the protocol
functions a pointer instead of the actual structure.
It saves 308 bytes of ROM in the stmhal/ port, 928 in unix/.
Finishes addressing issue #424.
In the end this was a very neat refactor that now makes things a lot
more consistent across the py code base. It allowed some
simplifications in certain places, now that everything is a dict object.
Also converted builtins tables to dictionaries. This will be useful
when we need to turn builtins into a proper module.
Pretty much everyone needs to include map.h, since it's such an integral
part of the Micro Python object implementation. Thus, the definitions
are now in obj.h instead. map.h is removed.
Mostly just a global search and replace. Except rt_is_true which
becomes mp_obj_is_true.
Still would like to tidy up some of the names, but this will do for now.
Originally, .methods was used for methods in a ROM class, and
locals_dict for methods in a user-created class. That distinction is
unnecessary, and we can use locals_dict for ROM classes now that we have
ROMable maps.
This removes an entry in the bloated mp_obj_type_t struct, saving a word
for each ROM object and each RAM object. ROM objects that have a
methods table (now a locals_dict) need an extra word in total (removed
the methods pointer (1 word), no longer need the sentinel (2 words), but
now need an mp_obj_dict_t wrapper (4 words)). But RAM objects save a
word because they never used the methods entry.
Overall the ROM usage is down by a few hundred bytes, and RAM usage is
down 1 word per user-defined type/class.
There is less code (no need to check 2 tables), and now consistent with
the way ROM modules have their tables initialised.
Efficiency is very close to equivaluent.
mp_module_obj_t can now be put in ROM.
Configuration of float type is now similar to longint: can now choose
none, float or double as the implementation.
math module has basic math functions. For STM port, these are not yet
implemented (they are just stub functions).
Each built-in exception is now a type, with base type BaseException.
C exceptions are created by passing a pointer to the exception type to
make an instance of. When raising an exception from the VM, an
instance is created automatically if an exception type is raised (as
opposed to an exception instance).
Exception matching (RT_BINARY_OP_EXCEPTION_MATCH) is now proper.
Handling of parse error changed to match new exceptions.
mp_const_type renamed to mp_type_type for consistency.
Ultimately all static strings should be qstr. This entry in the type
structure is only used for printing error messages (to tell the type of
the bad argument), and printing objects that don't supply a .print method.
Unlike CPython socket, microsocket object already implements stream protocol
(read/write methods), so makefile() just returns object itself. TODO: this
doesn't take care of arguments CPython's makefile() may accept.
It's no longer intended to provide just "raw" socket interface, may include
some convenience methods for compatibility with CPython socket - but anyway
just minimal set required to deal with socket client and servers, not wider
network functionality.
