CPython also has os.environ, which should be used instead of os.getenv()
due to caching in the os.environ mapping. But for MicroPython it makes
sense to only implement the basic underlying methods, ie getenv/putenv/
unsetenv.
This adds a -h option to print the usage help text and adds a new, shorter
error message that is printed when invalid arguments are given. This
behaviour follows CPython (and other tools) more closely.
This commit modifies the usage() function to only print the -v option help
text when MICROPY_DEBUG_PRINTERS is enabled. The -v option requires this
build option to be enabled for it to have any effect.
The usage text is also modified to show the -i and -m options, and also
show that running a command, module or file are mutually exclusive.
This adds support for a MICROPYINSPECT environment variable that works
exactly like PYTHONINSPECT; per CPython docs:
If this is set to a non-empty string it is equivalent to specifying the
-i option.
This variable can also be modified by Python code using os.environ to
force inspect mode on program termination.
When stdout is redirected it is useful to have errors printed to stderr
instead of being redirected.
mp_stderr_print() can't be used in these two instances since the
MicroPython runtime is not running so we use fprintf(stderr) instead.
This modifies the signature of mp_thread_set_state() to use
mp_state_thread_t* instead of void*. This matches the return type of
mp_thread_get_state(), which returns the same value.
`struct _mp_state_thread_t;` had to be moved before
`#include <mpthreadport.h>` since the stm32 port uses it in its
mpthreadport.h file.
It is not safe to enable MICROPY_ASYNC_KBD_INTR and MICROPY_PY_THREAD_GIL
at the same time. This will trigger a compiler error to ensure that it
is not possible to make this mistake.
Addition of GIL EXIT/ENTER pairs are:
- modos: release the GIL during system calls. CPython does this as well.
- moduselect: release the GIL during the poll() syscall. This call can be
blocking, so it is important to allow other threads to run at this time.
- modusocket: release the GIL during system calls. Many of these calls can
be blocking, so it is important to allow other threads to run.
- unix_mphal: release the GIL during the read and write syscalls in
mp_hal_stdin_rx_chr and mp_hal_stdout_tx_strn. If we don't do this
threads are blocked when the REPL or the builtin input function are used.
- file, main, mpconfigport.h: release GIL during syscalls in built-in
functions that could block.
When CFLAGS_EXTRA/LDFLAGS_EXTRA (or anything) is set on the command line of
a make invocation then it will completely override any setting or appending
of these variables in the makefile(s). This means builds like the coverage
variant will have their mpconfigvariant.mk settings overridden. Fix this
by using CFLAGS/LDFLAGS exclusively in the makefile(s), reserving the
CFLAGS_EXTRA/LDFLAGS_EXTRA variables for external command-line use only.
This commit adds backward-word, backward-kill-word, forward-word,
forward-kill-word sequences for the REPL, with bindings to Alt+F, Alt+B,
Alt+D and Alt+Backspace respectively. It is disabled by default and can be
enabled via MICROPY_REPL_EMACS_WORDS_MOVE.
Further enabling MICROPY_REPL_EMACS_EXTRA_WORDS_MOVE adds extra bindings
for these new sequences: Ctrl+Right, Ctrl+Left and Ctrl+W.
The features are enabled on unix micropython-coverage and micropython-dev.
Invoking "make" will still build the standard "micropython" executable, but
other variants are now build using, eg, "make VARIANT=minimal". This
follows how bare-metal ports specify a particular board, and allows running
any make target (eg clean, test) with any variant.
Convenience targets (eg "make coverage") are provided to retain the old
behaviour, at least for now.
See issue #3043.
Instances of the slice class are passed to __getitem__() on objects when
the user indexes them with a slice. In practice the majority of the time
(other than passing it on untouched) is to work out what the slice means in
the context of an array dimension of a particular length. Since Python 2.3
there has been a method on the slice class, indices(), that takes a
dimension length and returns the real start, stop and step, accounting for
missing or negative values in the slice spec. This commit implements such
a indices() method on the slice class.
It is configurable at compile-time via MICROPY_PY_BUILTINS_SLICE_INDICES,
disabled by default, enabled on unix, stm32 and esp32 ports.
This commit also adds new tests for slice indices and for slicing unicode
strings.
The existing uos.remove cannot be used to remove directories, instead
uos.rmdir is needed. And also provide uos.rename to get a good set of
filesystem functionality without requiring additional Python-level os
functions (eg using ffi).
This commit removes the Makefile-level MICROPY_FATFS config and moves the
MICROPY_VFS_FAT config to the Makefile level to replace it. It also moves
the include of the oofatfs source files in the build from each port to a
central place in extmod/extmod.mk.
For a port to enabled VFS FAT support it should now set MICROPY_VFS_FAT=1
at the level of the Makefile. This will include the relevant oofatfs files
in the build and set MICROPY_VFS_FAT=1 at the C (preprocessor) level.
When loading a manifest file, e.g. by include(), it will chdir first to the
directory of that manifest. This means that all file operations within a
manifest are relative to that manifest's location.
As a consequence of this, additional environment variables are needed to
find absolute paths, so the following are added: $(MPY_LIB_DIR),
$(PORT_DIR), $(BOARD_DIR). And rename $(MPY) to $(MPY_DIR) to be
consistent.
Existing manifests are updated to match.
This commit adds support for sys.settrace, allowing to install Python
handlers to trace execution of Python code. The interface follows CPython
as closely as possible. The feature is disabled by default and can be
enabled via MICROPY_PY_SYS_SETTRACE.
mp_compile no longer takes an emit_opt argument, rather this setting is now
provided by the global default_emit_opt variable.
Now, when -X emit=native is passed as a command-line option, the emitter
will be set for all compiled modules (included imports), not just the
top-level script.
In the future there could be a way to also set this variable from a script.
Fixes issue #4267.
Enabled via MICROPY_PY_URE_DEBUG, disabled by default (but enabled on unix
coverage build). This is a rarely used feature that costs a lot of code
(500-800 bytes flash). Debugging of regular expressions can be done
offline with other tools.
As per PEP 485, this function appeared in for Python 3.5. Configured via
MICROPY_PY_MATH_ISCLOSE which is disabled by default, but enabled for the
ports which already have MICROPY_PY_MATH_SPECIAL_FUNCTIONS enabled.
This allows figuring out the number of bytes in the memoryview object as
len(memview) * memview.itemsize.
The feature is enabled via MICROPY_PY_BUILTINS_MEMORYVIEW_ITEMSIZE and is
disabled by default.
The original code called setsockopt(SO_RCVTIMEO/SO_SNDTIMEO) with NULL
timeout structure argument, which is an illegal usage of that function.
The old code also didn't validate the return value of setsockopt, missing
the bug completely.
When building with link time optimization enabled it is possible both
gc_collect() and gc_collect_regs_and_stack() get inlined into gc_alloc()
which can result in the regs variable being pushed on the stack earlier
than some of the registers. Depending on the calling convention, those
registers might however contain pointers to blocks which have just been
allocated in the caller of gc_alloc(). Then those pointers end up higher on
the stack than regs, aren't marked by gc_collect_root() and hence get
sweeped, even though they're still in use.
As reported in #4652 this happened for in 32-bit msvc release builds:
mp_lexer_new() does two consecutive allocations and the latter triggered a
gc_collect() which would sweep the memory of the first allocation again.
As mentioned in #4450, `websocket` was experimental with a single intended
user, `webrepl`. Therefore, we'll make this change without a weak
link `websocket` -> `uwebsocket`.
If opening of /dev/mem has failed an `OSError` is appropriately raised, but
the next time `mem8/16/32` is accessed the invalid file descriptor is used
and the program gets a SIGSEGV.
Python defines warnings as belonging to categories, where category is a
warning type (descending from exception type). This is useful, as e.g.
allows to disable warnings selectively and provide user-defined warning
types. So, implement this in MicroPython, except that categories are
represented just with strings. However, enough hooks are left to implement
categories differently per-port (e.g. as types), without need to patch each
and every usage.
One can't use pthread calls in a signal handler because they are not
async-signal-safe (see man signal-safety). Instead, sem_post can be used
to post from within a signal handler and this should be more efficient than
using a busy wait loop, waiting on a volatile variable.
We standardized to provide uos.remove() as a more obvious and user-friendly
name. That's what written in the docs. The Unix port implementation
predates this convention, so update it now.
Configurable via MICROPY_MODULE_GETATTR, disabled by default. Among other
things __getattr__ for modules can help to build lazy loading / code
unloading at runtime.
Configurable via MICROPY_PY_BUILTINS_STR_COUNT. Default is enabled.
Disabled for bare-arm, minimal, unix-minimal and zephyr ports. Disabling
it saves 408 bytes on x86.
1. Return correct error code for non-blocking vs timed out socket
(POSIX returns EAGAIN for both, we want ETIMEDOUT in case of timed
out socket). To achieve this, blocking/non-blocking flag is added
to the mp_obj_socket_t, to avoid issuing fcntl() syscall each time
EAGAIN occurs. (mp_obj_socket_t used to be 8 bytes, having some room
in a standard 16-byte alloc block.)
2. Handle socket.settimeout(0) properly - in Python, that means
non-blocking mode, but SO_RCVTIMEO/SO_SNDTIMEO of 0 is infinite
timeout.
3. Overall, make sure that socket.settimeout() call switches blocking
state as expected.
This will allow to e.g. implement HTTP Digest authentication.
Adds 540 bytes for x86_32, 332 for arm_thumb2 (for Unix port, which already
includes axTLS library).
This commit adds the math.factorial function in two variants:
- squared difference, which is faster than the naive version, relatively
compact, and non-recursive;
- a mildly optimised recursive version, faster than the above one.
There are some more optimisations that could be done, but they tend to take
more code, and more storage space. The recursive version seems like a
sensible compromise.
The new function is disabled by default, and uses the non-optimised version
by default if it is enabled. The options are MICROPY_PY_MATH_FACTORIAL
and MICROPY_OPT_MATH_FACTORIAL.
Changes made:
- make use of MP_OBJ_TO_PTR and MP_OBJ_FROM_PTR where necessary
- fix shadowing of index variable i, renamed to j
- fix type of above variable to size_t to prevent comparison warning
- fix shadowing of res variable
- use "(void)" instead of "()" for functions that take no arguments
If DTTOIF() macro is not defined, the code refers to MP_S_IFDIR, etc.
symbols defined in extmod/vfs.h, so should include it.
This fixes build for Android.
"coverage" build uses different BUILD directory comparing to the normal
build. Previously, any build picked up libaxtls.a from normal build's
directory, but that was fixed recently. So, for each build, we must
build axtls explicitly.
This fixes Travis build in particular.
This patch in effect renames MICROPY_DEBUG_PRINTER_DEST to
MICROPY_DEBUG_PRINTER, moving its default definition from
lib/utils/printf.c to py/mpconfig.h to make it official and documented, and
makes this macro a pointer rather than the actual mp_print_t struct. This
is done to get consistency with MICROPY_ERROR_PRINTER, and provide this
macro for use outside just lib/utils/printf.c.
Ports are updated to use the new macro name.
This mechanism will scale to to an arbitrary number of pollable objects, so
long as they implement the MP_STREAM_GET_FILENO ioctl. Since ussl objects
pass through ioctl requests transparently to the underlying socket object,
it will allow ussl sockets to be polled. And a user object with uio.IOBase
as a base could support polling.
The API follows guidelines of https://www.python.org/dev/peps/pep-0272/,
but is optimized for code size, with the idea that full PEP 0272
compatibility can be added with a simple Python wrapper mode.
The naming of the module follows (u)hashlib pattern.
At the bare minimum, this module is expected to provide:
* AES128, ECB (i.e. "null") mode, encrypt only
Implementation in this commit is based on axTLS routines, and implements
following:
* AES 128 and 256
* ECB and CBC modes
* encrypt and decrypt
This behaviour of a NULL write C method on a stream that uses the write
adaptor objects is no longer supported. It was only ever used by the
coverage build for testing the fail path of mp_get_stream_raise().
Now that the coverage build has fully switched to the VFS sub-system these
functions were no longer available, so add them to the uos_vfs module.
Also, vfs_open is no longer needed, it's available as the built-in open.
The unix coverage build is now switched fully to the VFS implementation, ie
the uos module is the uos_vfs module. For example, one can now sandbox uPy
to their home directory via:
$ ./micropython_coverage
>>> import uos
>>> uos.umount('/') # unmount existing root VFS
>>> vfs = uos.VfsPosix('/home/user') # create new POSIX VFS
>>> uos.mount(vfs, '/') # mount new POSIX VFS at root
Some filesystem/OS features may no longer work with the coverage build due
to this change, and these need to be gradually fixed.
The standard unix port remains unchanged, it still uses the traditional uos
module which directly accesses the underlying host filesystem.
This patch moves the implementation of stream closure from a dedicated
method to the ioctl of the stream protocol, for each type that implements
closing. The benefits of this are:
1. Rounds out the stream ioctl function, which already includes flush,
seek and poll (among other things).
2. Makes calling mp_stream_close() on an object slightly more efficient
because it now no longer needs to lookup the close method and call it,
rather it just delegates straight to the ioctl function (if it exists).
3. Reduces code size and allows future types that implement the stream
protocol to be smaller because they don't need a dedicated close method.
Code size reduction is around 200 bytes smaller for x86 archs and around
30 bytes smaller for the bare-metal archs.
This test for calling gc_realloc() while the GC is locked can be done in
pure Python, so better to do it that way since it can then be tested on
more ports.
