This saves a lot of RAM for 2 reasons:
1. For functions that don't have default values, var args or var kw
args (which is a large number of functions in the general case), the
mp_obj_fun_bc_t type now fits in 1 GC block (previously needed 2 because
of the extra pointer to point to the arg_names array). So this saves 16
bytes per function (32 bytes on 64-bit machines).
2. Combining separate memory regions generally saves RAM because the
unused bytes at the end of the GC block are saved for 1 of the blocks
(since that block doesn't exist on its own anymore). So generally this
saves 8 bytes per function.
Tested by importing lots of modules:
- 64-bit Linux gave about an 8% RAM saving for 86k of used RAM.
- pyboard gave about a 6% RAM saving for 31k of used RAM.
In CPython IOError (and EnvironmentError) is deprecated and aliased to
OSError. All modules that used to raise IOError now raise OSError (or a
derived exception).
In Micro Python we never used IOError (except 1 place, incorrectly) and
so don't need to keep it.
See http://legacy.python.org/dev/peps/pep-3151/ for background.
Because (for Thumb) a function pointer has the LSB set, pointers to
dynamic functions in RAM (eg native, viper or asm functions) were not
being traced by the GC. This patch is a comprehensive fix for this.
Addresses issue #820.
reversed function now implemented, and works for tuple, list, str, bytes
and user objects with __len__ and __getitem__.
Renamed mp_builtin_len to mp_obj_len to make it publically available (eg
for reversed).
This happens for example for zero-size arrays. As .get_buffer() method now
has explicit return value, it's enough to distinguish success vs failure
of getting buffer.
The user code should call micropython.alloc_emergency_exception_buf(size)
where size is the size of the buffer used to print the argument
passed to the exception.
With the test code from #732, and a call to
micropython.alloc_emergenncy_exception_buf(100) the following error is
now printed:
```python
>>> import heartbeat_irq
Uncaught exception in Timer(4) interrupt handler
Traceback (most recent call last):
File "0://heartbeat_irq.py", line 14, in heartbeat_cb
NameError: name 'led' is not defined
```
This will allow roughly the same behavior as Python3 for non-ASCII strings,
for example, print("<phrase in non-Latin script>".split()) will print list
of words, not weird hex dump (like Python2 behaves). (Of course, that it
will print list of words, if there're "words" in that phrase at all, separated
by ASCII-compatible whitespace; that surely won't apply to every human
language in existence).
Two things are handled here: allow to compare native subtypes of tuple,
e.g. namedtuple (TODO: should compare type too, currently compared
duck-typedly by content). Secondly, allow user sunclasses of tuples
(and its subtypes) be compared either. "Magic" I did previously in
objtype.c covers only one argument (lhs is many), so we're in trouble
when lhs is native type - there's no other option besides handling
rhs in special manner. Fortunately, this patch outlines approach with
fast path for native types.
Blanket wide to all .c and .h files. Some files originating from ST are
difficult to deal with (license wise) so it was left out of those.
Also merged modpyb.h, modos.h, modstm.h and modtime.h in stmhal/.
Closed over variables are now passed on the stack, instead of creating a
tuple and passing that. This way memory for the closed over variables
can be allocated within the closure object itself. See issue #510 for
background.
When querying an object that supports the buffer protocol, that object
must now return a typecode (as per binary.[ch]). This does not have to
be honoured by the caller, but can be useful for determining element
size.
This is to reduce ROM usage. stream_p is used in file and socket types
only (at the moment), so seems a good idea to make the protocol
functions a pointer instead of the actual structure.
It saves 308 bytes of ROM in the stmhal/ port, 928 in unix/.
One of the reason for separate "message" (besides still unfulfilled desire to
optimize memory usage) was apparent special handling of exception with
messages by CPython. Well, the message is still just an exception argument,
it just printed specially. Implement that with PRINT_EXC printing format.
Pretty much everyone needs to include map.h, since it's such an integral
part of the Micro Python object implementation. Thus, the definitions
are now in obj.h instead. map.h is removed.
Mostly just a global search and replace. Except rt_is_true which
becomes mp_obj_is_true.
Still would like to tidy up some of the names, but this will do for now.
Rationale: setting up the stack (state for locals and exceptions) is
really part of the "code", it's the prelude of the function. For
example, native code adjusts the stack pointer on entry to the function.
Native code doesn't need to know n_state for any other reason. So
putting the state size in the bytecode prelude is sensible.
It reduced ROM usage on STM by about 30 bytes :) And makes it easier to
pass information about the bytecode between functions.
Originally, .methods was used for methods in a ROM class, and
locals_dict for methods in a user-created class. That distinction is
unnecessary, and we can use locals_dict for ROM classes now that we have
ROMable maps.
This removes an entry in the bloated mp_obj_type_t struct, saving a word
for each ROM object and each RAM object. ROM objects that have a
methods table (now a locals_dict) need an extra word in total (removed
the methods pointer (1 word), no longer need the sentinel (2 words), but
now need an mp_obj_dict_t wrapper (4 words)). But RAM objects save a
word because they never used the methods entry.
Overall the ROM usage is down by a few hundred bytes, and RAM usage is
down 1 word per user-defined type/class.
There is less code (no need to check 2 tables), and now consistent with
the way ROM modules have their tables initialised.
Efficiency is very close to equivaluent.
This gets "value" of exceptions in the sense as it's defined for
StopIteration.value (i.e. args[0] or None).
TODO: This really should be inline function.
Return with value gets converted to StopIteration(value). Implementation
keeps optimizing against creating of possibly unneeded exception objects,
so there're considerable refactoring to implement these features.
mp_module_obj_t can now be put in ROM.
Configuration of float type is now similar to longint: can now choose
none, float or double as the implementation.
math module has basic math functions. For STM port, these are not yet
implemented (they are just stub functions).
Each built-in exception is now a type, with base type BaseException.
C exceptions are created by passing a pointer to the exception type to
make an instance of. When raising an exception from the VM, an
instance is created automatically if an exception type is raised (as
opposed to an exception instance).
Exception matching (RT_BINARY_OP_EXCEPTION_MATCH) is now proper.
Handling of parse error changed to match new exceptions.
mp_const_type renamed to mp_type_type for consistency.
Ultimately all static strings should be qstr. This entry in the type
structure is only used for printing error messages (to tell the type of
the bad argument), and printing objects that don't supply a .print method.
Unlike CPython socket, microsocket object already implements stream protocol
(read/write methods), so makefile() just returns object itself. TODO: this
doesn't take care of arguments CPython's makefile() may accept.
TODO: Decide if we really need separate bytecode for creating functions
with default arguments - we would need same for closures, then there're
keywords arguments too. Having all combinations is a small exponential
explosion, likely we need just 2 cases - simplest (no defaults, no kw),
and full - defaults & kw.
We still have FAST_[0,1,2] byte codes, but they now just access the
fastn array (before they had special local variables). It's now
simpler, a bit faster, and uses a bit less stack space (on STM at least,
which is most important).
The only reason now to keep FAST_[0,1,2] byte codes is for compressed
byte code size.
In Python, importing module several times returns same underlying module
object. This also fixes import statement handling for builtin modules.
There're still issues:
1. CPython exposes set of loaded modules as sys.modules, we may want to
do that either.
2. Builtin modules are implicitly imported, which is not really correct.
We should separate registering a (builtin) module and importing a module.
CPython keeps builtin module names in sys.builtin_module_names .
mp_obj_int_get() can be used when just full resolution of C machine_int_t
is required (returns truncated value of long int). mp_obj_int_get_checked()
will throw exception if Python int value not representable in machine_int_t.
Change state layout in VM so the stack starts at state[0] and grows
upwards. Locals are at the top end of the state and number downwards.
This cleans up a lot of the interface connecting the VM to C: now all
functions that take an array of Micro Python objects are in order (ie no
longer in reverse).
Also clean up C API with keyword arguments (call_n and call_n_kw
replaced with single call method that takes keyword arguments). And now
make_new takes keyword arguments.
emitnative.c has not yet been changed to comply with the new order of
stack layout.
We likely should make mp_obj_new_int() inline, and rely on its
encapsulated check rather than inline checks everywhere explicitly.
Also, parser for big small int values is still broken.
Stream protocol is abstraction of serial I/O. Buffer protocol is
abstraction of random-access I/O. These protocols are defined down
to C level, to allow generic, while still efficient algorithms
to be coded in C (like, buffered transfer between 2 stream objects,
saving/loading of buffer object to/from stream, etc). (Note that CPython
define buffer protocol on C level, but apparently not stream protocol).
It's not really about that, though; it's about me figuring out a sane
way forward for keyword-argument functions (and function
metadata). But it's useful as is, and shouldn't break any existing
code, so here you have it; I'm going to park it in my mind for a bit
while sorting out the rest of the dict branch.
A big change. Micro Python objects are allocated as individual structs
with the first element being a pointer to the type information (which
is itself an object). This scheme follows CPython. Much more flexible,
not necessarily slower, uses same heap memory, and can allocate objects
statically.
Also change name prefix, from py_ to mp_ (mp for Micro Python).
