When built for Linux, libffi includes very bloated and workaround exec-alloc
implementation required to work around SELinux and other "sekuritee" features
which real people don't use. MicroPython has own alloc-exec implementation,
used to alloc memory for @micropython.native code. With this option enabled,
uPy's implementation will override libffi's. This saves 11K on x86_64 (and
that accounts for more than half of the libffi code size).
TODO: Possibly, we want to refactor this option to allow either use uPy's
implementation even for libffi, or allow to use libffi's implementation even
for uPy.
Disabled by default, enabled in unix port. Need for this method easily
pops up when working with text UI/reporting, and coding workalike
manually again and again counter-productive.
To use frozen bytecode make a subdirectory under the unix/ directory
(eg frozen/), put .py files there, then run:
make FROZEN_MPY_DIR=frozen
Be sure to build from scratch. The .py files will then be available for
importing.
The config variable MICROPY_MODULE_FROZEN is now made of two separate
parts: MICROPY_MODULE_FROZEN_STR and MICROPY_MODULE_FROZEN_MPY. This
allows to have none, either or both of frozen strings and frozen mpy
files (aka frozen bytecode).
Calling it from mp_init() is too late for some ports (like Unix), and leads
to incomplete stack frame being captured, with following GC issues. So, now
each port should call mp_stack_ctrl_init() on its own, ASAP after startup,
and taking special precautions so it really was called before stack variables
get allocated (because if such variable with a pointer is missed, it may lead
to over-collecting (typical symptom is segfaulting)).
Seedable and reproducible pseudo-random number generator. Implemented
functions are getrandbits(n) (n <= 32) and seed().
The algorithm used is Yasmarang by Ilya Levin:
http://www.literatecode.com/yasmarang
POSIX doesn't guarantee something like that to work, but it works on any
system with careful signal implementation. Roughly, the requirement is
that signal handler is executed in the context of the process, its main
thread, etc. This is true for Linux. Also tested to work without issues
on MacOSX.
This basically introduces the MICROPY_MACHINE_MEM_GET_READ_ADDR
and MICROPY_MACHINE_MEM_GET_WRITE_ADDR macros. If one of them is
not defined, then a default identity function is provided.
To let unix port implement "machine" functionality on Python level, and
keep consistent naming in other ports (baremetal ports will use magic
module "symlinking" to still load it on "import machine").
Fixes#1701.
ilistdir() returns iterator which yields triples of (name, type, ino)
where ino is inode number for entry's data, type of entry (file/dir/etc.),
and name of file/dir. listdir() can be easily implemented in terms of this
iterator (which is otherwise more efficient in terms of memory use and may
save expensive call to stat() for each returned entry).
CPython has os.scandir() which also returns an iterator, but it yields
more complex objects of DirEntry type. scandir() can also be easily
implemented in terms of ilistdir().
THis is required to deal well with signals, signals being the closest
analogue of hardware interrupts for POSIX. This is also CPython 3.5
compliant behavior (PEP 475).
The main problem implementing this is to figure out how much time was
spent in waiting so far/how much is remaining. It's well-known fact that
Linux updates select()'s timeout value when returning with EINTR to the
remaining wait time. Here's what POSIX-based standards say about this:
(http://pubs.opengroup.org/onlinepubs/9699919799/functions/pselect.html):
"Upon successful completion, the select() function may modify the object
pointed to by the timeout argument."
I.e. it allows to modify timeout value, but doesn't say how exactly it is
modified. And actually, it allows such modification only "upon successful
completion", which returning with EINTR error hardly is.
POSIX also allows to request automatic EINTR restart for system calls using
sigaction call with SA_RESTART flag, but here's what the same document says
about it:
"If SA_RESTART has been set for the interrupting signal, it is
implementation-defined whether the function restarts or returns with
[EINTR]."
In other words, POSIX doesn't leave room for both portable and efficient
handling of this matter, so the code just allows to manually select
Linux-compatible behavior with MICROPY_SELECT_REMAINING_TIME option,
or otherwise will just raise OSError. When systems with non-Linux behavior
are found, they can be handled separately.
In other words, unix port now uses overriden printf(), instead of using
libc's. This should remove almost all dependency on libc stdio (which
is bloated).
py/mphal.h contains declarations for generic mp_hal_XXX functions, such
as stdio and delay/ticks, which ports should provide definitions for. A
port will also provide mphalport.h with further HAL declarations.
Another function (like stat) which is problematic to deal with on ABI level
(FFI), as struct statvfs layout may differ unpredictably between OSes and
even different versions of a same OS. So, implement it in C, returning a
10-element tuple of f_bsize, f_frsize, f_blocks, f_bfree, f_bavail, f_files,
f_ffree, f_favail, f_flag, f_namemax. This is exactly the order described
in Python3 docs, https://docs.python.org/3/library/os.html#os.statvfs
(but note that os.statvfs() should make these values available as
attributes).
This requires root access. And on recent Linux kernels, with
CONFIG_STRICT_DEVMEM option enabled, only address ranges listed in
/proc/iomem can be accessed. The above compiled-time option can be
however overriden with boot-time option "iomem=relaxed".
This also removed separate read/write paths - there unlikely would
be a case when they're different.
MicroPython doesn't come with standard library included, so it is important
to be able to easily install needed package in a seamless manner. Bundling
package manager (upip) inside an executable solves this issue.
upip is bundled only with standard executable, not "minimal" or "fast"
builds.
