The msvc compiler doesn't accept zero-sized arrays so let the freezing
process generate compatible C code in case no modules are found and the
involved arrays are all empty. This doesn't affect the functionality in
any way because those arrays only get accessed when mp_frozen_mpy_names
contains names, i.e. when modules are actually found.
MP_BC_CALL_FUNCTION will leave the result on the Python stack, so that
result must be discarded by MP_BC_POP_TOP.
Signed-off-by: Damien George <damien@micropython.org>
Updating to Black v20.8b1 there are two changes that affect the code in
this repository:
- If there is a trailing comma in a list (eg [], () or function call) then
that list is now written out with one line per element. So remove such
trailing commas where the list should stay on one line.
- Spaces at the start of """ doc strings are removed.
Signed-off-by: Damien George <damien@micropython.org>
The existing implementation of mkdir() in this file is not sophisticated
enough to work correctly on all operating systems (eg Mac can raise
EISDIR). Using the standard os.makedirs() function handles all cases
correctly.
Signed-off-by: Damien George <damien@micropython.org>
Prior to this commit, pyboard.py used eval() to "parse" file data received
from the board. Using eval() on received data from a device is dangerous,
because a malicious device may inject arbitrary code execution on the PC
that is doing the operation.
Consider the following scenario:
Eve may write a malicious script to Bob's board in his absence. On return
Bob notices that something is wrong with the board, because it doesn't work
as expected anymore. He wants to read out boot.py (or any other file) to
see what is wrong. What he gets is a remote code execution on his PC.
Proof of concept:
Eve:
$ cat boot.py
_print = print
print = lambda *x, **y: _print("os.system('ls /; echo Pwned!')", end="\r\n\x04")
$ ./pyboard.py -f cp boot.py :
cp boot.py :boot.py
Bob:
$ ./pyboard.py -f cp :boot.py /tmp/foo
cp :boot.py /tmp/foo
bin chroot dev home lib32 media opt root sbin sys usr
boot config etc lib lib64 mnt proc run srv tmp var
Pwned!
There's also the possibility that the device is malfunctioning and sends
random and possibly dangerous data back to the PC, to be eval'd.
Fix this problem by using ast.literal_eval() to parse the received bytes,
instead of eval().
Signed-off-by: Michael Buesch <m@bues.ch>
.. however, the number of endpoints is only set for SAMD (8).
Other ports need to set the value. Otherwise, the build will show
the message
```
Unable to check whether maximum number of endpoints is respected
```
Since we made the decision to allow translations which do not have coverage in
the terminal font, these routinely occur and are expected. The message is
unhelpful and due to their voume make it harder to find relevant information
particularly in github actions results.
If mpy-cross exits with an error be sure to print that error in a way that
is readable, instead of a long bytes object.
Signed-off-by: Damien George <damien@micropython.org>
The file `mbedtls_errors/mp_mbedtls_errors.c` can be used instead of
`mbedtls/library/error.c` to give shorter error strings, reducing the build
size of the error strings from about 12-16kB down to about 2-5kB.
Also:
- Remove download count update because the files are no longer
on GitHub.
- Add "extensions" and "languages" to each board dictionary so we
can stop using "files" entirely.
This reverts commit 4d6f60d428.
This implementation used the timeout as a maximum amount of time needed for
the operation, when actually the spec and other tools suggest that it's the
minumum delay needed between subsequent USB transfers.
This adds support for freezing an entire directory while keeping the
directory as part of the import path. For example
freeze("path/to/library", "module")
will recursively freeze all scripts in "path/to/library/module" and have
them importable as "from module import ...".
Signed-off-by: Damien George <damien@micropython.org>
With only `sp_func_proto_paren = remove` set there are some cases where
uncrustify misses removing a space between the function name and the
opening '('. This sets all of the related options to `force` as well.
This is the result of running...
uncrustify -c tools/uncrustify.cfg --update-config-with-doc -o tools/uncrustify.cfg
...with some manual fixups to correct places where it changed things it
should not have.
Essentially it just adds new config parameters introduced in v0.71.0
with their default values.
Signed-off-by: David Lechner <david@pybricks.com>
Formatting for `* sizeof` was fixed in uncrustify v0.71, so we no longer
need the fixups for it. Also, there was one file where the updated
uncrustify caught a problem that the regex didn't pick up, which is updated
in this commit.
Signed-off-by: David Lechner <david@pybricks.com>
This adds a new command line option `-v` to `tools/codeformat.py` to enable
verbose printing of all files that are scanned.
Normally `uncrustify` and `black` are called with the `-q` option so
setting verbose suppresses the `-q` option and on `black` also enables the
`-v` option which makes it print out all file names matching the filter
similar to how `uncrustify` does by default.
This suppresses the Parsing: <file> as language C lines. This makes
parsing run a bit faster and on CI it makes for less scrolling through logs
(and black already uses the -q option).
Note: the uncrustify configuration is explicitly set to 'add' instead of
'force' in order not to alter the comments which use extra spaces after //
as a means of indenting text for clarity.
The HID descriptor reported by circuitpython erroneously limited the
maximum keycode to 101, which prevented circuitpython from sending a
number of otherwise valid keycodes.
Closes#274
The decompression of error-strings is only done if the string is accessed
via printing or via er.args. Tests are added for this feature to ensure
the decompression works.
This adds the Python files in the tests/ directory to be formatted with
./tools/codeformat.py. The basics/ subdirectory is excluded for now so we
aren't changing too much at once.
In a few places `# fmt: off`/`# fmt: on` was used where the code had
special formatting for readability or where the test was actually testing
the specific formatting.
This eliminates the need for the sizeof regex fixup by rearranging things a
bit. All other bitfields already use the parentheses around expressions
with sizeof, so one case is fixed by following this convention.
VM_MAX_STATE_ON_STACK is the only remaining problem and it can be worked
around by changing the order of the operands.
When using a manifest on Windows the reference to mpy-cross compiler was
missing the .exe file extension, so add it when appropriate.
Also allow the default path to mpy-cross to be overridden by the (optional)
MICROPY_MPYCROSS environment variable, to allow full flexibility on any OS.
This commit adds a tool, codeformat.py, which will reformat C and Python
code to fit a certain style. By default the tool will reformat (almost)
all the original (ie not 3rd-party) .c, .h and .py files in this
repository. Passing filenames on the command-line to codeformat.py will
reformat only those. Reformatting is done in-place.
uncrustify is used for C reformatting, which is available for many
platforms and can be easily built from source, see
https://github.com/uncrustify/uncrustify. The configuration for uncrustify
is also added in this commit and values are chosen to best match the
existing code style. A small post-processing stage on .c and .h files is
done by codeformat.py (after running uncrustify) to fix up some minor
items:
- space inserted after * when used as multiplication with sizeof
- #if/ifdef/ifndef/elif/else/endif are dedented by one level when they are
configuring if-blocks and case-blocks.
For Python code, the formatter used is black, which can be pip-installed;
see https://github.com/psf/black. The defaults are used, except for line-
length which is set at 99 characters to match the "about 100" line-length
limit used in C code.
The formatting tools used and their configuration were chosen to strike a
balance between keeping existing style and not changing too many lines of
code, and enforcing a relatively strict style (especially for Python code).
This should help to keep the code consistent across everything, and reduce
cognitive load when writing new code to match the style.
This option makes pyboard.py exit as soon as the script/command is
successfully sent to the device, ie it does not wait for any output. This
can help to avoid hangs if the board is being rebooted with --comman (for
example).
Example usage:
$ python3 ./tools/pyboard.py --device /dev/ttyUSB0 --no-follow \
--command 'import machine; machine.reset()'
Some parts of code have been aligned to increase readability. In general
'' instead of "" were used wherever possible to keep the same convention
for entire file. Import inspect line has been moved to the top according
to hints reported by pep8 tools. A few extra spaces were removed, a few
missing spaces were added. Comments have been updated, mostly in
"read_dfu_file" function. Some other comments have been capitalized and/or
slightly updated. A few docstrings were fixed as well. No real code
changes intended.
Introduces a way to place CircuitPython code and data into
tightly coupled memory (TCM) which is accessible by the CPU in a
single cycle. It also frees up room in the corresponding cache for
intermittent data. Loading from external flash is slow!
The data cache is also now enabled.
Adds support for the iMX RT 1021 chip. Adds three new boards:
* iMX RT 1020 EVK
* iMX RT 1060 EVK
* Teensy 4.0
Related to #2492, #2472 and #2477. Fixes#2475.
It was intended that the `f.load_glyphs` line was fast and did most of
the work. However, it actually didn't, because it's necessary to pass
in a code point by number, not by string.
Additionally, a little light layer violation is needed to make the check
for missing characters fast. This used to be less important, as no
fonts had missing characters. However, it would take an appreciable
length of time on the Korean translation when failing to find hundreds
of different code points.
Testing performed: built
build-circuitplayground_express_displayio/autogen_display_resources.c with ko
translation before and after change. verified the file content was identical.
Time went from about 7s on my machine to way under 1 second.
This is an alternative to f4ed2df that adds a newline so that the output of
the test starts on a new line and the result of the test is prefixed with
"result: " to distinguish it from the test output.
Suggested-by: @dpgeorge
This patch allows executing .mpy files (including native ones) directly on
a target, eg a board over a serial connection. So there's no need to copy
the file to its filesystem to test it.
For example:
$ mpy-cross foo.py
$ pyboard.py foo.mpy
This makes the loading of viper-code-with-relocations a bit neater and
easier to understand, by treating the rodata/bss like a special object to
be loaded into the constant table (which is how it behaves).
We don't want to add a feature flag to .mpy files that indicate float
support because it will get complex and difficult to use. Instead the .mpy
is built using whatever precision it chooses (float or double) and the
native glue API will convert between this choice and what the host runtime
actually uses.
This commit adds a new tool called mpy_ld.py which is essentially a linker
that builds .mpy files directly from .o files. A new header file
(dynruntime.h) and makefile fragment (dynruntime.mk) are also included
which allow building .mpy files from C source code. Such .mpy files can
then be dynamically imported as though they were a normal Python module,
even though they are implemented in C.
Converting .o files directly (rather than pre-linked .elf files) allows the
resulting .mpy to be more efficient because it has more control over the
relocations; for example it can skip PLT indirection. Doing it this way
also allows supporting more architectures, such as Xtensa which has
specific needs for position-independent code and the GOT.
The tool supports targets of x86, x86-64, ARM Thumb and Xtensa (windowed
and non-windowed). BSS, text and rodata sections are supported, with
relocations to all internal sections and symbols, as well as relocations to
some external symbols (defined by dynruntime.h), and linking of qstrs.
Usage:
mpy-tool.py -o merged.mpy --merge mod1.mpy mod2.mpy
The constituent .mpy files are executed sequentially when the merged file
is imported, and they all use the same global namespace.
In cases where more than one board is connected to a single computer it can become pretty hard to figure out which board you're actually talking to. For example, if you have several MIDI-compatible boards they all show up as "CircuitPython MIDI". This change allows boards to replace the "CircuitPython" part of their USB descriptors with more specific text, for example, "CircuitPython Feather" or just "Feather". This will let folks more easily tell boards apart.
The new option is named `USB_INTERFACE_NAME` and is available in `mkconfigboard.mk`. For example:
```
USB_INTERFACE_NAME = "Feather"
```
While the new manifest.py style got introduced for freezing python code
into the resulting binary, the old way - where files and modules within
ports/*/modules where baked into the resulting binary - was still
supported via `freeze('$(PORT_DIR)/modules')` within manifest.py.
However behaviour changed for symlinked directories (=modules), as those
links weren't followed anymore.
This commit restores the original behaviour by explicitly following
symlinks within a modules/ directory
When loading a manifest file, e.g. by include(), it will chdir first to the
directory of that manifest. This means that all file operations within a
manifest are relative to that manifest's location.
As a consequence of this, additional environment variables are needed to
find absolute paths, so the following are added: $(MPY_LIB_DIR),
$(PORT_DIR), $(BOARD_DIR). And rename $(MPY) to $(MPY_DIR) to be
consistent.
Existing manifests are updated to match.
This introduces a new build variable FROZEN_MANIFEST which can be set to a
manifest listing (written in Python) that describes the set of files to be
frozen in to the firmware.
Instead of encoding 4 zero bytes as placeholders for the simple_name and
source_file qstrs, and storing the qstrs after the bytecode, store the
qstrs at the location of these 4 bytes. This saves 4 bytes per bytecode
function stored in a .mpy file (for example lcd160cr.mpy drops by 232
bytes, 4x 58 functions). And resulting code size is slightly reduced on
ports that use this feature.
This patch compresses the second part of the bytecode prelude which
contains the source file name, function name, source-line-number mapping
and cell closure information. This part of the prelude now begins with a
single varible length unsigned integer which encodes 2 numbers, being the
byte-size of the following 2 sections in the header: the "source info
section" and the "closure section". After decoding this variable unsigned
integer it's possible to skip over one or both of these sections very
easily.
This scheme saves about 2 bytes for most functions compared to the original
format: one in the case that there are no closure cells, and one because
padding was eliminated.
The start of the bytecode prelude contains 6 numbers telling the amount of
stack needed for the Python values and exceptions, and the signature of the
function. Prior to this patch these numbers were all encoded one after the
other (2x variable unsigned integers, then 4x bytes), but using so many
bytes is unnecessary.
An entropy analysis of around 150,000 bytecode functions from the CPython
standard library showed that the optimal Shannon coding would need about
7.1 bits on average to encode these 6 numbers, compared to the existing 48
bits.
This patch attempts to get close to this optimal value by packing the 6
numbers into a single, varible-length unsigned integer via bit-wise
interleaving. The interleaving scheme is chosen to minimise the average
number of bytes needed, and at the same time keep the scheme simple enough
so it can be implemented without too much overhead in code size or speed.
The scheme requires about 10.5 bits on average to store the 6 numbers.
As a result most functions which originally took 6 bytes to encode these 6
numbers now need only 1 byte (in 80% of cases).
Prior to this patch mp_opcode_format would calculate the incorrect size of
the MP_BC_UNWIND_JUMP opcode, missing the additional byte. But, because
opcodes below 0x10 are unused and treated as bytes in the .mpy load/save
and freezing code, this bug did not show any symptoms, since nested unwind
jumps would rarely (if ever) reach a depth of 16 (so the extra byte of this
opcode would be between 0x01 and 0x0f and be correctly loaded/saved/frozen
simply as an undefined opcode).
This patch fixes this bug by correctly accounting for the additional byte.
.
- Split 'qemu-arm' from 'unix' for generating tests.
- Add frozen module to the qemu-arm test build.
- Add test that reproduces the requirement to half-word align native
function data.
Use "-f" to select filesystem mode, followed by the command to execute.
Optionally put ":" at the start of a filename to indicate that it's on the
remote device, if it would otherwise be ambiguous.
Examples:
$ pyboard.py -f ls
$ pyboard.py -f cat main.py
$ pyboard.py -f cp :main.py . # get from device
$ pyboard.py -f cp main.py : # put to device
$ pyboard.py -f rm main.py
This also improves Palette so it stores the original RGB888 colors.
Lastly, it adds I2CDisplay as a display bus to talk over I2C. Particularly
useful for the SSD1306.
Fixes#1828. Fixes#1956
This fixes the bug that bitmap changes do not cause screen updates
and optimizes the refresh when the bitmap is simply shown on the
screen. If the bitmap is used in tiles, then changing it will
cause all TileGrids using it to do a full refresh.
Fixes#1981
Instead of iterating over all the glyphs and calculating the maximum
width and height, use the FONTBOUNDINGBOX to determine the size of a
tile for terminalio.
This works better with fonts such as generated by FontForge, that don't
include the empty space in the glyph bitmap itself. It also lets the
font author specify vertical spacing they want.
I only tested this with the default font and with one I generated with
FontForge.
Different operations to the display tree have different costs. Be
aware of these costs when optimizing your code.
* Changing tiles indices in a TileGrid will update an area
covering them all.
* Changing a palette will refresh every object that references it.
* Moving a TileGrid will update both where it was and where it moved to.
* Adding something to a Group will refresh each individual area it
covers.
* Removing things from a Group will refresh one area that covers all
previous locations. (Not separate areas like add.)
* Setting a new top level Group will refresh the entire display.
Only TileGrid moves are optimized for overlap. All other overlaps
cause sending of duplicate pixels.
This also adds flip_x, flip_y and transpose_xy to TileGrid. They
change the direction of the pixels but not the location.
Fixes#1169. Fixes#1705. Fixes#1923.
Previously, when linking qstr objects in native code for ARM Thumb, the
index into the machine code was being incremented by 4, not 8. It should
be 8 to account for the size of the two machine instructions movw and movt.
This patch makes sure the index into the machine code is incremented by the
correct amount for all variations of qstr linking.
See issue #4829.
Fixes errors in the tool when 1) linking qstrs in native ARM-M code; 2)
freezing multiple files some of which use native code and some which don't.
Fixes issue #4829.
This changes the displayio pixel computation from per-pixel to
per-area. This is precursor work to updating portions of the screen
(#1169). It should provide mild speedups because bounds checks are
done once per area rather than once per pixel. Filling by area also
allows TileGrid to maintain a row-associative fill pattern even when
the display's refresh is orthogonal to it.
The user can now select their own package index by either passing the "-i"
command line option, or setting the upip.index_urls variable (before doing
an install).
The https://micropython.org/pi package index hosts packages from
micropython-lib and will be searched first when installing a package. If a
package is not found here then it will fallback to PyPI.
Prior to this patch, when a lot of data was output by a running script
pyboard.py would try to capture all of this output into the "data"
variable, which would gradually slow down pyboard.py to the point where it
would have large CPU and memory usage (on the host) and potentially lose
data.
This patch fixes this problem by not accumulating the data in the case that
the data is not needed, which is when "data_consumer" is used.
The qstr window size is not log-2 encoded, it's just the actual number (but
in mpy-tool.py this didn't lead to an error because the size is just used
to truncate the window so it doesn't grow arbitrarily large in memory).
Addresses issue #4635.
This system makes it a lot easier to include external libraries as static,
native modules in MicroPython. Simply pass USER_C_MODULES (like
FROZEN_MPY_DIR) as a make parameter.
When encoded in the mpy file, if qstr <= QSTR_LAST_STATIC then store two
bytes: 0, static_qstr_id. Otherwise encode the qstr as usual (either with
string data or a reference into the qstr window).
Reduces mpy file size by about 5%.
Instead of emitting two bytes in the bytecode for where the linked qstr
should be written to, it is now replaced by the actual qstr data, or a
reference into the qstr window.
Reduces mpy file size by about 10%.
This is an implementation of a sliding qstr window used to reduce the
number of qstrs stored in a .mpy file. The window size is configured to 32
entries which takes a fixed 64 bytes (16-bits each) on the C stack when
loading/saving a .mpy file. It allows to remember the most recent 32 qstrs
so they don't need to be stored again in the .mpy file. The qstr window
uses a simple least-recently-used mechanism to discard the least recently
used qstr when the window overflows (similar to dictionary compression).
This scheme only needs a single pass to save/load the .mpy file.
Reduces mpy file size by about 25% with a window size of 32.
POP_BLOCK and POP_EXCEPT are now the same, and are always followed by a
JUMP. So this optimisation reduces code size, and RAM usage of bytecode by
two bytes for each try-except handler.
This changes a number of things in displayio:
* Introduces BuiltinFont and Glyph so the built in font can be used by libraries. For boards with
a font it is available as board.TERMINAL_FONT. Fixes#1172
* Remove _load_row from Bitmap in favor of bitmap[] access. Index can be x/y tuple or overall index. Fixes#1191
* Add width and height properties to Bitmap.
* Add insert and [] access to Group. Fixes#1518
* Add index param to pop on Group.
* Terminal no longer takes unicode character info. It takes a BuiltinFont instead.
* Fix Terminal's handling of [###D vt100 commands used when up arrowing into repl history.
* Add x and y positions to Group plus scale as well.
* Add bitmap accessor for BuiltinFont
Under python3 (tested with 3.6.7) bytes with a list of integers as an
argument returns a different result than under python 2.7 (tested with
2.7.15rc1) which causes pydfu.py to fail when run under 2.7. Changing
bytes to bytearray makes pydfu work properly under both Python 2.7 and
Python 3.6.
If you happen to only have a really simple frozen file that doesn't contain
any new qstrs then the generated frozen_mpy.c file contains an empty
enumeration which causes a C compile time error.
Following an equivalent fix to py/bc.c. The reason the incorrect values
for the opcode constants were not previously causing a bug is because they
were never being used: these opcodes always have qstr arguments so the part
of the code that was comparing them would never be reached.
Thanks to @malinah for finding the problem and providing the initial patch.
A DFU device must be in the idle state before it can be programmed, and
this requires either clearing the status or aborting, depending on its
current state. Code is added to do this. And the USB transfer size is now
automatically detected so devices with a size less than 2048 bytes work
correctly.
This started while adding USB MIDI support (and descriptor support is
in this change.) When seeing that I'd have to implement the MIDI class
logic twice, once for atmel-samd and once for nrf, I decided to refactor
the USB stack so its shared across ports. This has led to a number of
changes that remove items from the ports folder and move them into
supervisor.
Furthermore, we had external SPI flash support for nrf pending so I
factored out the connection between the usb stack and the flash API as
well. This PR also includes the QSPI support for nRF.
Some Python linters don't like unconditional except clauses because they
catch SystemExit and KeyboardInterrupt, which usually is not the intended
behaviour.
There appears to be an issue on Windows with CPython >= 3.6,
sys.stdout.flush() raises an exception:
OSError: [WinError 87] The parameter is incorrect
It works fine to just catch and ignore the error on the flush line. Tested
on Windows 10 x64 1803 (Build 17134.228), Python 3.6.4 amd64.
This saves code space in builds which use link-time optimization.
The optimization drops the untranslated strings and replaces them
with a compressed_string_t struct. It can then be decompressed to
a c string.
Builds without LTO work as well but include both untranslated
strings and compressed strings.
This work could be expanded to include QSTRs and loaded strings if
a compress method is added to C. Its tracked in #531.
The Python documentation recommends to pass the command as a string when
using Popen(..., shell=True). This is because "sh -c <string>" is used to
execute the command and additional arguments after the command string are
passed to the shell itself (not the executing command).
https://docs.python.org/3.5/library/subprocess.html#subprocess.Popen
The first dynamic qstr pool is double the size of the 'alloc' field of
the last const qstr pool. The built in const qstr pool
(mp_qstr_const_pool) has a hardcoded alloc size of 10, meaning that the
first dynamic pool is allocated space for 20 entries. The alloc size
must be less than or equal to the actual number of qstrs in the pool
(the 'len' field) to ensure that the first dynamically created qstr
triggers the creation of a new pool.
When modules are frozen a second const pool is created (generally
mp_qstr_frozen_const_pool) and linked to the built in pool. However,
this second const pool had its 'alloc' field set to the number of qstrs
in the pool. When freezing a large quantity of modules this can result
in thousands of qstrs being in the pool. This means that the first
dynamically created qstr results in a massive allocation. This commit
sets the alloc size of the frozen qstr pool to 10 or less (if the number
of qstrs in the pool is less than 10). The result of this is that the
allocation behaviour when a dynamic qstr is created is identical with an
without frozen code.
Note that there is the potential for a slight memory inefficiency if the
frozen modules have less than 10 qstrs, as the first few dynamic
allocations will have quite a large overhead, but the geometric growth
soon deals with this.
The ST DFU bootloader supports a transfer size up to 2048 bytes, so send
that much data on each download (to device) packet. This almost halves
total download time.
- moving Makefile to parent folder. This resolves some of the weird path/build issues
- remove trace references to cpydiff and original file (no longer used anywhere)
- converting SOURCEDIR to a changeable variable, passed through all sphinx-build calls
- adding path to conf.py, in case it moves again
- making `-v` default with VERBOSE
- making `-E` default with FORCE
- creating BASEOPTS to store all the dirs, paths, and settings, passing them to sphinx-build in one long chain, instead of individually
- updating README to use the make command as default. Also added text explaining some of the customization you can add into a make command
1. Make _extext and _sidata coincide. Old _etext location did not include
.ARM.exidx sections, which were usually absent but not always. So flash
data was copied to RAM in wrong place.
2. Use decimal constants with "K" and "M" suffixes in .ld files instead
of hex constants, to make them easier to read and check for accuracy.
Instead of passing thru more and more options from tinytest-codegen to
run-tests --list-tests, pipe output of run-tests --list-tests into
tinytest-codegen.
Gets passed to run-tests --list-tests to get actual list of tests to use.
If --target= is not given, legacy set hardcoded in tinytest-codegen itself
is used.
Also, get rid of tinytest test groups - they aren't really used for
anything, and only complicate processing. Besides, one of the next
step is to limit number of tests per a generated file to control
the binary size, which also will require "flat" list of tests.
The way tinytest was used in qemu-arm test target is that it didn't test
much. MicroPython tests are based on matching the test output against
reference output, but qemu-arm's implementation didn't do that, it
effectively tested just that there was no exception during test
execution. "upytesthelper" wrapper was introduce to fix it, and so
test generator is now switched to generate test code for it.
Also, fix PEP8 and other codestyle issues.
This does two major things:
1. It stores the arm gcc in the repo. The download from the PPA was taking between 2 and 7 minutes.
2. It splits each board into a separate job so they are built in parallel.
Introduction of ports subdirectory where all ports are moved to
The main change in this release is the introduction of a "ports/"
subdirectory at the top-level of the repository, and all of the ports are
moved here. In the process the "stmhal" port is renamed to "stm32" to
better reflect the MCU that it targets. In addition, the STM32 CMSIS and
HAL sources are moved to a new submodule called "stm32lib".
The bytecode has changed in this release, compared to the previous release,
and as a consequence the .mpy version number has increased to version 3.
This means that scripts compiled with the previous mpy-cross must be
recompiled to work with this new version.
There have also been various enhancements and optimisations, such as:
check for valid UTF-8 when creating str objects, support for reverse
special binary operations like __radd__, full domain checking in the math
module, support for floor-division and modulo in the viper emitter,
and addition of stack overflow checking when executing a regex.
The stm32 port sees improved support for F7 MCUs, addition of a new board
B_L475E_IOT01A based on the STM32L475, and support for the Wiznet W5500
chipset along with improved socket behaviour.
A detailed list of changes follows.
py core:
- objstr: startswith, endswith: check arg to be a string
- nlrx86,x64: replace #define of defined() with portable macro usage
- objtype: handle NotImplemented return from binary special methods
- objtype: mp_obj_class_lookup: improve debug logging
- map: remove unused new/free functions
- make m_malloc_fail() have void return type, since it doesn't return
- modstruct: in struct.pack, stop converting if there are no args left
- modstruct: check and prevent buffer-read overflow in struct unpacking
- modstruct: check and prevent buffer-write overflow in struct packing
- nlrthumb: get working again on standard Thumb arch (ie not Thumb2)
- objfloat: fix binary ops with incompatible objects
- obj: fix comparison of float/complex NaN with itself
- objtype: implement fallback for instance inplace special methods
- objtuple: properly implement comparison with incompatible types
- objstr: add check for valid UTF-8 when making a str from bytes
- objlist: properly implement comparison with incompatible types
- runtime0.h: move relational ops to the beginning of mp_binary_op_t
- runtime0.h: move MP_BINARY_OP_DIVMOD to the end of mp_binary_op_t
- objtype: make sure mp_binary_op_method_name has full size again
- runtime0.h: put inplace arith ops in front of normal operations
- builtinhelp: simplify code slightly by extracting object type
- runtime: implement dispatch for "reverse op" special methods
- nlrx86: fix building for Android/x86
- builtinhelp: change signature of help text var from pointer to array
- runtime.h: change empty mp_warning macro so var-args are non empty
- modbuiltins: implement abs() by dispatching to MP_UNARY_OP_ABS
- {objfloat,objcomplex}: optimise MP_UNARY_OP_ABS by reusing variables
- mpconfig.h: add note that using computed gotos in VM is not C99
- objstr: strip: don't strip "\0" by default
- objexcept: prevent infinite recursion when allocating exceptions
- stream: remove unnecessary checks for NULL return from vstr_add_len
- vstr: raise a RuntimeError if fixed vstr buffer overflows
- vm: use lowercase letter at start of exception message
- persistentcode: define mp_raw_code_save_file() for any unix target
- add config option to print warnings/errors to stderr
- objfloat: support raising a negative number to a fractional power
- objset: simplify set and frozenset by separating their locals dicts
- objset: check that RHS of a binary op is a set/frozenset
- objset: include the failed key in a KeyError raised from set.remove
- objtype: change type of enum-to-qstr table to uint16_t to save space
- objstr: make empty bytes object have a null-terminating byte
- mpprint: only check for null string printing when NDEBUG not defined
- objtype: clean up unary- and binary-op enum-to-qstr mapping tables
- persistentcode: bump .mpy version number to version 3
- bc: update opcode_format_table to match the bytecode
- modmath: add full checks for math domain errors
- modmath: convert log2 macro into a function
- formatfloat: don't print the negative sign of a NaN value
- formatfloat: use standard isinf, isnan funcs instead of custom ones
- modbuiltins: use existing utf8_get_char helper in builtin ord func
- emitnative: implement floor-division and modulo for viper emitter
- objtype: use CPython compatible method name for sizeof
- objtype: fit qstrs for special methods in byte type
- objtype: define all special methods if requested
- objtype: introduce MICROPY_PY_ALL_INPLACE_SPECIAL_METHODS
extmod:
- modubinascii: only include uzlib/tinf.h when it's really needed
- modussl_mbedtls: allow to compile with MBEDTLS_DEBUG_C disabled
- machine_pinbase: put PinBase singleton in ROM
- re1.5: upgrade to v0.8.2, adds hook for stack overflow checking
- modure: add stack overflow checking when executing a regex
- uos_dupterm: update uos.dupterm() and helper funcs to have index
- uos_dupterm: swallow any errors from dupterm closing the stream
- vfs: replace VLA in proxy func with small, static sized array
- modussl: add finaliser support for ussl objects
- modussl_mbedtls: allow to compile with unix coverage build
lib:
- add new submodule, stm32lib containing STM32 CMSIS and HAL source
- embed/abort_: use mp_raise_msg helper function
- libm: fix tanhf so that it correctly handles +/- infinity args
- libm: remove implementation of log2f, use MP_NEED_LOG2 instead
- axtls: update, support for SSL_EAGAIN return code
- berkeley-db-1.xx: update, allow to override MINCACHE, DEFPSIZE
drivers:
- memory/spiflash: change from hard-coded soft SPI to generic SPI
- display/ssd1306.py: improve performance of graphics methods
- nrf24l01: make nRF24L01 test script more portable
- display/ssd1306: implement SSD1306_I2C poweron method
- display/ssd1306: make poweron() work the same with SSD1306_SPI
- wiznet5k: improve the performance of socket ops with threading
- wiznet5k: get low-level W5500 driver working
tools:
- upip: upgrade to 1.2.2
- pyboard: use repr() when quoting data in error messages
- pyboard: update docstring for additional device support
tests:
- object_new: better messages, check user __new__() method
- class_new: add checks for __init__ being called and other improvements
- class_new: add another testcase for __new__/__init__ interaction
- class_inplace_op: test for inplace op fallback to normal one
- run-bench-tests: update locations of executables, now in ports/
- class_reverse_op: test for reverse arith ops special methods
- run-tests: skip class_inplace_op for minimal profile
- run-tests: fix copy-paste mistake in var name
- cpydiff: add cases for locals() discrepancies
- extmod: add test for ure regexes leading to infinite recursion
- extmod: add test for '-' in character class in regex
- run-tests: close device under test using "finally"
- net_inet: update tls test to work with CPython and incl new site
unix port:
- rename modsocket.c to modusocket.c
- modusocket: remove #if MICROPY_SOCKET_EXTRA code blocks
- enable MICROPY_PY_REVERSE_SPECIAL_METHODS
stm32 port:
- modmachine: make machine.bootloader() work when MPU is enabled
- modmachine: improve support for sleep/deepsleep on F7 MCUs
- compute PLL freq table during build instead of at run time
- modmachine: for F7 MCU, save power by reducing internal volt reg
- boards/pllvalues.py: make script work with both Python 2 and 3
- Makefile: use lib/stm32lib instead of local cmsis and hal files
- remove cmsis and hal files, they are now a submodule
- Makefile: automatically fetch stm32lib submodule if needed
- update to new STM Cube HAL library
- fix clock initialisation of L4 MCUs
- rename stmhal port directory to stm32
- remove unused usbd_msc.c file
- boards: change remaining stm32f4xx_hal_conf.h to unix line ending
- boards: change linker scripts to use "K" instead of hex byte size
- boards: fix I2C1 pin mapping on NUCLEO_F401RE/F411RE boards
- i2c: when scanning for I2C devices only do 1 probe per address
- modnwwiznet5k: release the GIL on blocking network operations
- boards: add new board B_L475E_IOT01A based on STM32L475
- make-stmconst.py: make sure mpz const data lives in ROM
- timer: make pyb.Timer() instances persistent
- mpconfigport.h: add configuration for max periphs on L4 series
- usbdev: make the USBD callback struct const so it can go in ROM
- usbdev: change static function variable to non-static
- usbdev: put all CDC state in a struct
- usbdev: put all HID state in a struct
- usbdev: simplify CDC tx/rx buffer passing
- usbdev: simplify HID tx/rx buffer passing
- usbdev/core: add state parameter to all callback functions
- usbdev: put all state for the USB device driver in a struct
- usbdev: simplify pointers to MSC state and block dev operations
- usbdev: merge all global USB device state into a single struct
- usbdev: make device descriptor callbacks take a state pointer
- usbdev: move all the USB device descriptor state into its struct
- timer: enable ARPE so that timer freq can be changed smoothly
- modnwwiznet5k: get the IP address of an established socket
- boards: fix typos in stm32f767_af.csv table
- usbd_cdc_interface: don't reset CDC output buf on initialisation
- modnwwiznet5k: implement WIZNET5K.isconnected() method
- modusocket: make getaddrinfo() work when passed an IP address
- modusocket: return OSError(-2) if getaddrinfo fails
- mpconfigport.h: add MICROPY_THREAD_YIELD() macro
- modnwwiznet5k: add support for W5500 Ethernet chip
- modnwwiznet5k: increase SPI bus speed to 42MHz
- modnwwiznet5k: implement stream ioctl for the Wiznet driver
- mphalport: improve efficiency of mp_hal_stdout_tx_strn_cooked
- make uos.dupterm() conform to specs by using extmod version
cc3200 port:
- enable micropython.kbd_intr() method
- use standard implementation of keyboard interrupt
esp8266 port:
- rename axtls_helpers.c to posix_helpers.c
- posix_helpers: set ENOMEM on memory alloc failure
- set DEFPSIZE=1024, MINCACHE=3 for "btree" module
- esp_mphal: send data in chunks to mp_uos_dupterm_tx_strn
- modnetwork: add "bssid" keyword arg to WLAN.connect() method
- modules/webrepl_setup: add info about allowed password length
zephyr port:
- Makefile: revamp "test" target after ports were moved to ports/
- use CONFIG_NET_APP_SETTINGS to setup initial network addresses
- switch to interrupt-driven pull-style console
pic16bit port:
- add definition of SEEK_SET to unistd.h
docs:
- pyboard/tutorial: add "timeout=0" to UART in pass-through example
- more xrefs to "MicroPython port" in glossary
- library/network: fix ref to "socket" module (should be "usocket")
- machine.Signal: improve style/grammar and add usage example
- library: add description of "index" parameter to uos.dupterm()
- library/micropython: fix typo in RST formatting
- library/framebuf.rst: generalise constructor to all colour formats
- btree: describe page caching policy of the underlying implementation
- esp8266/tutorial: update neopixel with example of using 4 bbp
- library/network: clarify usage of "bssid" arg in connect() method
- pyboard/quickref: add info for Switch, RTC, CAN, Accel classes
- pyboard/tutorial: update now that yellow LED also supports PWM
- esp8266/quickref: add quickref info for RTC class
- library: add missing cross-ref links for classes in pyb module
- library/network: update docs to state that W5500 is supported
- uselect: document one-shot polling mode
- usocket: elaborate descriptions
- usocket: document inet_ntop(), inet_pton()
- library/network: add dhcp_hostname parameter
- reference/isr_rules: minor typo correction
- ussl: fix module name refs and use "MicroPython port" term
- esp8266/general: add section on TLS limitations
- usocket: document that settimeout() isn't supported by all ports
- ure: add "|" (alternative) to the list of supported operators
- reference/isr_rules.rst: add tutorial on use of micropython.schedule()
travis:
- use --upgrade when pip is installing cpp-coveralls
- update build command now that stm32 Wiznet config has changed
examples:
- hwconfig_console: add .on()/.off() methods
all:
- convert mp_uint_t to mp_unary_op_t/mp_binary_op_t where appropriate
- convert remaining "mp_uint_t n_args" to "size_t n_args"
- make new ports/ sub-directory and move all ports there
- update Makefiles and others to build with new ports/ dir layout
- remove inclusion of internal py header files
- use NULL instead of "" when calling mp_raise exception helpers
README:
- update "Dependencies" section
- add explicit section on contributing
- add gcc and arm-none-eabi-newlib to list of required components
.gitattributes:
- remove obsolete entries for stmhal/hal, stmhal/cmsis
- add entries for files that will move to ports/ dir
* Introduce a python script to generate the USB descriptor instead of
a bunch of C macros. In the future, we can use this dynamically in
CircuitPython.
* Add support for detecting read-only mass storage mounts.
Fixes#377
* atmel-samd: Remove ASF3. This will break builds.
* atmel-samd: Add ASF4 for the SAMD21 and SAMD51.
* Introduce the supervisor concept to facilitate porting.
The supervisor is the code which runs individual MicroPython VMs. By
splitting it out we make it more consistent and easier to find.
This also adds very basic SAMD21 and SAMD51 support using the
supervisor. Only the REPL currently works.
This begins the work for #178.
This introduces a skip_if module that can be used by tests to
determine when they should be skipped due to the environment.
Some tests have been split in order to have finer grained skip
control.
Fixes for stmhal USB mass storage, lwIP bindings and VFS regressions
This release provides an important fix for the USB mass storage device in
the stmhal port by implementing the SCSI SYNCHRONIZE_CACHE command, which
is now require by some Operating Systems. There are also fixes for the
lwIP bindings to improve non-blocking sockets and error codes. The VFS has
some regressions fixed including the ability to statvfs the root.
All changes are listed below.
py core:
- modbuiltins: add core-provided version of input() function
- objstr: catch case of negative "maxsplit" arg to str.rsplit()
- persistentcode: allow to compile with complex numbers disabled
- objstr: allow to compile with obj-repr D, and unicode disabled
- modsys: allow to compile with obj-repr D and PY_ATTRTUPLE disabled
- provide mp_decode_uint_skip() to help reduce stack usage
- makeqstrdefs.py: make script run correctly with Python 2.6
- objstringio: if created from immutable object, follow copy on write policy
extmod:
- modlwip: connect: for non-blocking mode, return EINPROGRESS
- modlwip: fix error codes for duplicate calls to connect()
- modlwip: accept: fix error code for non-blocking mode
- vfs: allow to statvfs the root directory
- vfs: allow "buffering" and "encoding" args to VFS's open()
- modframebuf: fix signed/unsigned comparison pendantic warning
lib:
- libm: use isfinite instead of finitef, for C99 compatibility
- utils/interrupt_char: remove support for KBD_EXCEPTION disabled
tests:
- basics/string_rsplit: add tests for negative "maxsplit" argument
- float: convert "sys.exit()" to "raise SystemExit"
- float/builtin_float_minmax: PEP8 fixes
- basics: convert "sys.exit()" to "raise SystemExit"
- convert remaining "sys.exit()" to "raise SystemExit"
unix port:
- convert to use core-provided version of built-in import()
- Makefile: replace references to make with $(MAKE)
windows port:
- convert to use core-provided version of built-in import()
qemu-arm port:
- Makefile: adjust object-file lists to get correct dependencies
- enable micropython.mem_*() functions to allow more tests
stmhal port:
- boards: enable DAC for NUCLEO_F767ZI board
- add support for NUCLEO_F446RE board
- pass USB handler as parameter to allow more than one USB handler
- usb: use local USB handler variable in Start-of-Frame handler
- usb: make state for USB device private to top-level USB driver
- usbdev: for MSC implement SCSI SYNCHRONIZE_CACHE command
- convert from using stmhal's input() to core provided version
cc3200 port:
- convert from using stmhal's input() to core provided version
teensy port:
- convert from using stmhal's input() to core provided version
esp8266 port:
- Makefile: replace references to make with $(MAKE)
- Makefile: add clean-modules target
- convert from using stmhal's input() to core provided version
zephyr port:
- modusocket: getaddrinfo: Fix mp_obj_len() usage
- define MICROPY_PY_SYS_PLATFORM (to "zephyr")
- machine_pin: use native Zephyr types for Zephyr API calls
docs:
- machine.Pin: remove out_value() method
- machine.Pin: add on() and off() methods
- esp8266: consistently replace Pin.high/low methods with .on/off
- esp8266/quickref: polish Pin.on()/off() examples
- network: move confusingly-named cc3200 Server class to its reference
- uos: deconditionalize, remove minor port-specific details
- uos: move cc3200 port legacy VFS mounting functions to its ref doc
- machine: sort machine classes in logical order, not alphabetically
- network: first step to describe standard network class interface
examples:
- embedding: use core-provided KeyboardInterrupt object
This patch allows the following code to run without allocating on the heap:
super().foo(...)
Before this patch such a call would allocate a super object on the heap and
then load the foo method and call it right away. The super object is only
needed to perform the lookup of the method and not needed after that. This
patch makes an optimisation to allocate the super object on the C stack and
discard it right after use.
Changes in code size due to this patch are:
bare-arm: +128
minimal: +232
unix x64: +416
unix nanbox: +364
stmhal: +184
esp8266: +340
cc3200: +128
This allows to execute a command and communicate with its stdin/stdout
via pipes ("exec") or with command-created pseudo-terminal ("execpty"),
to emulate serial access. Immediate usecase is controlling a QEMU process
which emulates board's serial via normal console, but it could be used
e.g. with helper binaries to access real board over other hadware
protocols, etc.
An example of device specification for these cases is:
--device exec:../zephyr/qemu.sh
--device execpty:../zephyr/qemu2.sh
Where qemu.sh contains long-long qemu startup line, or calls another
command. There's a special support in this patch for running the command
in a new terminal session, to support shell wrappers like that (without
new terminal session, only wrapper script would be terminated, but its
child processes would continue to run).
This patch introduces the a small framework to track differences between
uPy and CPython. The framework consists of:
- A set of "tests" which test for an individual feature that differs between
uPy and CPy. Each test is like a normal uPy test in the test suite, but
has a special comment at the start with some meta-data: a category (eg
syntax, core language), a human-readable description of the difference, a
cause, and a workaround. Following the meta-data there is a short code
snippet which demonstrates the difference. See tests/cpydiff directory
for the initial set of tests.
- A program (this patch) which runs all the tests (on uPy and CPy) and
generates nicely-formated .rst documenting the differences.
- Integration into the docs build so that everything is automatic, and the
differences appear in a way that is easy for users to read/reference (see
latter commits).
The idea with using this new framework is:
- When a new difference is found it's easy to write a short test for it,
along with a description, and add it to the existing ones. It's also easy
for contributors to submit tests for differences they find.
- When something is no longer different the tool will give an error and
difference can be removed (or promoted to a proper feature test).
Previous to this patch the qemu-arm tests were compiled with is_relp=true
meaning that the __repl_print__ function was called for all lines of code
in the outer scope. This is not the right behaviour for scripts that are
executed as though they were a file (eg tests).
With this fix the micropython/heapalloc_str.py test now works so it is
removed from the test blacklist.
Support for Xtensa emitter and assembler, and upgraded F4 and F7 STM HAL
This release adds support for the Xtensa architecture as a target for the
native emitter, as well as Xtensa inline assembler. The int.from_bytes
and int.to_bytes methods now require a second argument (the byte order)
per CPython (only "little" is supported at this time). The "readall"
method has been removed from all stream classes that used it; "read" with
no arguments should be used instead. There is now support for importing
packages from compiled .mpy files. Test coverage is increased to 96%.
The generic I2C driver has improvements: configurable clock stretching
timeout, "stop" argument added to readfrom/writeto methods, "nack"
argument added to readinto, and write[to] now returns num of ACKs
received. The framebuf module now handles 16-bit depth (generic colour
format) and has hline, vline, rect, line methods. A new utimeq module is
added for efficient queue ordering defined by modulo time (to be
compatible with time.ticks_xxx functions). The pyboard.py script has been
modified so that the target board is not reset between scripts or commands
that are given on a single command line.
For the stmhal port the STM Cube HAL has been upgraded: Cube F4 HAL to
v1.13.1 (CMSIS 2.5.1, HAL v1.5.2) and Cube F7 HAL to v1.1.2. There is a
more robust pyb.I2C implementation (DMA is now disabled by default, can be
enabled via an option), and there is an implementation of machine.I2C with
robust error handling and hardware acceleration on F4 MCUs. It is now
recommended to use machine.I2C instead of pyb.I2C. The UART class is now
more robust with better handling of errors/timeouts. There is also more
accurate VBAT and VREFINT measurements for the ADC. New boards that are
supported include: NUCLEO_F767ZI, STM32F769DISC and NUCLEO_L476RG.
For the esp8266 port select/poll is now supported for sockets using the
uselect module. There is support for native and viper emitters, as well
as an inline assembler (with limited iRAM for storage of native functions,
or the option to store code to flash). There is improved software I2C
with a slight API change: scl/sda pins can be specified as positional only
when "-1" is passed as the first argument to indicate the use of software
I2C. It is recommended to use keyword arguments for scl/sda. There is
very early support for over-the-air (OTA) updates using the yaota8266
project.
A detailed list of changes follows.
py core:
- emitnative: fix native import emitter when in viper mode
- remove readall() method, which is equivalent to read() w/o args
- objexcept: allow clearing traceback with 'exc.__traceback__ = None'
- runtime: mp_resume: handle exceptions in Python __next__()
- mkrules.mk: rework find command so it works on OSX
- *.mk: replace uses of 'sed' with $(SED)
- parse: move function to check for const parse node to parse.[ch]
- parse: make mp_parse_node_new_leaf an inline function
- parse: add code to fold logical constants in or/and/not operations
- factor persistent code load/save funcs into persistentcode.[ch]
- factor out persistent-code reader into separate files
- lexer: rewrite mp_lexer_new_from_str_len in terms of mp_reader_mem
- lexer: provide generic mp_lexer_new_from_file based on mp_reader
- lexer: rewrite mp_lexer_new_from_fd in terms of mp_reader
- lexer: make lexer use an mp_reader as its source
- objtype: implement __call__ handling for an instance w/o heap alloc
- factor out common code from assemblers into asmbase.[ch]
- stream: move ad-hoc ioctl constants to stream.h and rename them
- compile: simplify configuration of native emitter
- emit.h: remove long-obsolete declarations for cpython emitter
- move arch-specific assembler macros from emitnative to asmXXX.h
- asmbase: add MP_PLAT_COMMIT_EXEC option for handling exec code
- asmxtensa: add low-level Xtensa assembler
- integrate Xtensa assembler into native emitter
- allow inline-assembler emitter to be generic
- add inline Xtensa assembler
- emitinline: embed entire asm struct instead of a pointer to it
- emitinline: move inline-asm align and data methods to compiler
- emitinline: move common code for end of final pass to compiler
- asm: remove need for dummy_data when doing initial assembler passes
- objint: from_bytes, to_bytes: require byteorder arg, require "little"
- binary: do zero extension when storing a value larger than word size
- builtinimport: support importing packages from compiled .mpy files
- mpz: remove unreachable code in mpn_or_neg functions
- runtime: zero out fs_user_mount array in mp_init
- mpconfig.h: enable MICROPY_PY_SYS_EXIT by default
- add MICROPY_KBD_EXCEPTION config option to provide mp_kbd_exception
- compile: add an extra pass for Xtensa inline assembler
- modbuiltins: remove unreachable code
- objint: rename mp_obj_int_as_float to mp_obj_int_as_float_impl
- emitglue: refactor to remove assert(0), to improve coverage
- lexer: remove unreachable code in string tokeniser
- lexer: remove unnecessary check for EOF in lexer's next_char func
- lexer: permanently disable the mp_lexer_show_token function
- parsenum: simplify and generalise decoding of digit values
- mpz: fix assertion in mpz_set_from_str which checks value of base
- mpprint: add assertion for, and comment about, valid base values
- objint: simplify mp_int_format_size and remove unreachable code
- unicode: comment-out unused function unichar_isprint
- consistently update signatures of .make_new and .call methods
- mkrules.mk: add MPY_CROSS_FLAGS option to pass flags to mpy-cross
- builtinimport: fix bug when importing names from frozen packages
extmod:
- machine_i2c: make the clock stretching timeout configurable
- machine_i2c: raise an error when clock stretching times out
- machine_i2c: release SDA on bus error
- machine_i2c: add a C-level I2C-protocol, refactoring soft I2C
- machine_i2c: add argument to C funcs to control stop generation
- machine_i2c: rewrite i2c.scan in terms of C-level protocol
- machine_i2c: rewrite mem xfer funcs in terms of C-level protocol
- machine_i2c: remove unneeded i2c_write_mem/i2c_read_mem funcs
- machine_i2c: make C-level functions return -errno on I2C error
- machine_i2c: add 'nack' argument to i2c.readinto
- machine_i2c: make i2c.write[to] methods return num of ACKs recvd
- machine_i2c: add 'stop' argument to i2c readfrom/writeto meths
- machine_i2c: remove trivial function wrappers
- machine_i2c: expose soft I2C obj and readfrom/writeto funcs
- machine_i2c: add hook to constructor to call port-specific code
- modurandom: allow to build with float disabled
- modframebuf: make FrameBuffer handle 16bit depth
- modframebuf: add back legacy FrameBuffer1 "class"
- modframebuf: optimise fill and fill_rect methods
- vfs_fat: implement POSIX behaviour of rename, allow to overwrite
- moduselect: use stream helper function instead of ad-hoc code
- moduselect: use configurable EVENT_POLL_HOOK instead of WFI
- modlwip: add ioctl method to socket, with poll implementation
- vfs_fat_file: allow file obj to respond to ioctl flush request
- modbtree: add method to sync the database
- modbtree: rename "sync" method to "flush" for consistency
- modframebuf: add hline, vline, rect and line methods
- machine_spi: provide reusable software SPI class
- modframebuf: make framebuf implement the buffer protocol
- modframebuf: store underlying buffer object to prevent GC free
- modutimeq: copy of current moduheapq with timeq support for refactoring
- modutimeq: refactor into optimized class
- modutimeq: make time_less_than be actually "less than", not less/eq
lib:
- utils/interrupt_char: use core-provided mp_kbd_exception if enabled
drivers:
- display/ssd1306.py: update to use FrameBuffer not FrameBuffer1
- onewire: enable pull up on data pin
- onewire/ds18x20: fix negative temperature calc for DS18B20
tools:
- tinytest-codegen: blacklist recently added uheapq_timeq test (qemu-arm)
- pyboard.py: refactor so target is not reset between scripts/cmd
- mpy-tool.py: add support for OPT_CACHE_MAP_LOOKUP_IN_BYTECODE
tests:
- micropython: add test for import from within viper function
- use read() instead of readall()
- basics: add test for logical constant folding
- micropython: add test for creating traceback without allocation
- micropython: move alloc-less traceback test to separate test file
- extmod: improve ujson coverage
- basics: improve user class coverage
- basics: add test for dict.fromkeys where arg is a generator
- basics: add tests for if-expressions
- basics: change dict_fromkeys test so it doesn't use generators
- basics: enable tests for list slice getting with 3rd arg
- extmod/vfs_fat_fileio: add test for constructor of FileIO type
- extmod/btree1: exercise btree.flush()
- extmod/framebuf1: add basics tests for hline, vline, rect, line
- update for required byteorder arg for int.from_bytes()/to_bytes()
- extmod: improve moductypes test coverage
- extmod: improve modframebuf test coverage
- micropython: get heapalloc_traceback test running on baremetal
- struct*: make skippable
- basics: improve mpz test coverage
- float/builtin_float_round: test round() with second arg
- basics/builtin_dir: add test for dir() of a type
- basics: add test for builtin locals()
- basics/set_pop: improve coverage of set functions
- run-tests: for REPL tests make sure the REPL is exited at the end
- basics: improve test coverage for generators
- import: add a test which uses ... in from-import statement
- add tests to improve coverage of runtime.c
- add tests to improve coverage of objarray.c
- extmod: add test for utimeq module
- basics/lexer: add a test for newline-escaping within a string
- add a coverage test for printing the parse-tree
- utimeq_stable: test for partial stability of utimeq queuing
- heapalloc_inst_call: test for no alloc for simple object calls
- basics: add tests for parsing of ints with base 36
- basics: add tests to improve coverage of binary.c
- micropython: add test for micropython.stack_use() function
- extmod: improve ubinascii.c test coverage
- thread: improve modthread.c test coverage
- cmdline: improve repl.c autocomplete test coverage
- unix: improve runtime_utils.c test coverage
- pyb/uart: update test to match recent change to UART timeout_char
- run-tests: allow to skip set tests
- improve warning.c test coverage
- float: improve formatfloat.c test coverage using Python
- unix: improve formatfloat.c test coverage using C
- unix/extra_coverage: add basic tests to import frozen str and mpy
- types1: split out set type test to set_types
- array: allow to skip test if "array" is unavailable
- unix/extra_coverage: add tests for importing frozen packages
unix port:
- rename define for unix moduselect to MICROPY_PY_USELECT_POSIX
- Makefile: update freedos target for change of USELECT config name
- enable utimeq module
- main: allow to print the parse tree in coverage build
- Makefile: make "coverage_test" target mirror Travis test actions
- moduselect: if file object passed to .register(), return it in .poll()
- Makefile: split long line for coverage target, easier to modify
- enable and add basic frozen str and frozen mpy in coverage build
- Makefile: allow cache-map-lookup optimisation with frozen bytecode
windows port:
- enable READER_POSIX to get access to lexer_new_from_file
stmhal port:
- dma: de-init the DMA peripheral properly before initialising
- i2c: add option to I2C to enable/disable use of DMA transfers
- i2c: reset the I2C peripheral if there was an error on the bus
- rename mp_hal_pin_set_af to _config_alt, to simplify alt config
- upgrade to STM32CubeF4 v1.13.0 - CMSIS/Device 2.5.1
- upgrade to STM32CubeF4 v1.13.0 - HAL v1.5.1
- apply STM32CubeF4 v1.13.1 patch - upgrade HAL driver to v1.5.2
- hal/i2c: reapply HAL commit ea040a4 for f4
- hal/sd: reapply HAL commit 1d7fb82 for f4
- hal: reapply HAL commit 9db719b for f4
- hal/rcc: reapply HAL commit c568a2b for f4
- hal/sd: reapply HAL commit 09de030 for f4
- boards: configure all F4 boards to work with new HAL
- make-stmconst.py: fix regex's to work with current CMSIS
- i2c: handle I2C IRQs
- dma: precalculate register base and bitshift on handle init
- dma: mark DMA sate as READY even if HAL_DMA_Init is skipped
- can: clear FIFO flags in IRQ handler
- i2c: provide custom IRQ handlers
- hal: do not include <stdio.h> in HAL headers
- mphalport.h: use single GPIOx->BSRR register
- make-stmconst.py: add support for files with invalid utf8 bytes
- update HALCOMMITS due to change to hal
- make-stmconst.py: restore Python 2 compatibility
- update HALCOMMITS due to change to hal
- moduselect: move to extmod/ for reuse by other ports
- i2c: use the HAL's I2C IRQ handler for F7 and L4 MCUs
- updates to get F411 MCUs compiling with latest ST HAL
- i2c: remove use of legacy I2C_NOSTRETCH_DISABLED option
- add beginnings of port-specific machine.I2C implementation
- i2c: add support for I2C4 hardware block on F7 MCUs
- i2c: expose the pyb_i2c_obj_t struct and some relevant functions
- machine_i2c: provide HW implementation of I2C peripherals for F4
- add support for flash storage on STM32F415
- add back GPIO_BSRRL and GPIO_BSRRH constants to stm module
- add OpenOCD configuration for STM32L4
- add address parameters to openocd config files
- adc: add "mask" selection parameter to pyb.ADCAll constructor
- adc: provide more accurate measure of VBAT and VREFINT
- adc: make ADCAll.read_core_temp return accurate float value
- adc: add ADCAll.read_vref method, returning "3.3v" value
- adc: add support for F767 MCU
- adc: make channel "16" always map to the temperature sensor
- sdcard: clean/invalidate cache before DMA transfers with SD card
- moduos: implement POSIX behaviour of rename, allow to overwrite
- adc: use constants from new HAL version
- refactor UART configuration to use pin objects
- uart: add support for UART7 and UART8 on F7 MCUs
- uart: add check that UART id is valid for the given board
- cmsis: update STM32F7 CMSIS device include files to V1.1.2
- hal: update ST32CubeF7 HAL files to V1.1.2
- port of f4 hal commit c568a2b to updated f7 hal
- port of f4 hal commit 09de030 to updated f7 hal
- port of f4 hal commit 1d7fb82 to updated f7 hal
- declare and initialise PrescTables for F7 MCUs
- boards/STM32F7DISC: define LSE_STARTUP_TIMEOUT
- hal: update HALCOMMITS due to change in f7 hal files
- refactor to use extmod implementation of software SPI class
- cmsis: add CMSIS file stm32f767xx.h, V1.1.2
- add NUCLEO_F767ZI board, with openocd config for stm32f7
- cmsis: add CMSIS file stm32f769xx.h, V1.1.2
- add STM32F769DISC board files
- move PY_SYS_PLATFORM config from board to general config file
- mpconfigport: add weak-module links for io, collections, random
- rename mp_const_vcp_interrupt to mp_kbd_exception
- usb: always use the mp_kbd_exception object for VCP interrupt
- use core-provided keyboard exception object
- led: properly initialise timer handle to zero before using it
- mphalport.h: explicitly use HAL's GPIO constants for pull modes
- usrsw: use mp_hal_pin_config function instead of HAL_GPIO_Init
- led: use mp_hal_pin_config function instead of HAL_GPIO_Init
- sdcard: use mp_hal_pin_config function instead of HAL_GPIO_Init
- add support for STM32 Nucleo64 L476RG
- uart: provide a custom function to transmit over UART
- uart: increase inter-character timeout by 1ms
- enable utimeq module
cc3200 port:
- tools/smoke.py: change readall() to read()
- pybspi: remove static mode=SPI.MASTER parameter for latest HW API
- mods/pybspi: remove SPI.MASTER constant, it's no longer needed
- update for moduselect moved to extmod/
- re-add support for UART REPL (MICROPY_STDIO_UART setting)
- enable UART REPL by default
- README: (re)add information about accessing REPL on serial
- make: rename "deploy" target to "deploy-ota"
- add targets to erase flash, deploy firmware using cc3200tool
- README: reorganize and update to the current state of affairs
- modwlan: add network.WLAN.print_ver() diagnostic function
esp8266 port:
- enable uselect module
- move websocket_helper.py from scripts to modules for frozen BC
- refactor to use extmod implementation of software SPI class
- mpconfigport_512k: disable framebuf module for 512k build
- enable native emitter for Xtensa arch
- enable inline Xtensa assembler
- add "ota" target to produce firmware binary for use with yaota8266
- use core-provided keyboard exception object
- add "erase" target to Makefile, to erase entire flash
- when doing GC be sure to trace the memory holding native code
- modesp: flash_user_start(): support configuration with yaota8266
- force relinking OTA firmware image if built after normal one
- scripts/inisetup: dump FS starting sector/size on error
- Makefile: produce OTA firmware as firmware-ota.bin
- modesp: make check_fw() work with OTA firmware
- enable utimeq module
- Makefile: put firmware-ota.bin in build/, for consistency
- modules/flashbdev: add RESERVED_SECS before the filesystem
- modules/flashbdev: remove code to patch bootloader flash size
- modules/flashbdev: remove now-unused function set_bl_flash_size
- modules/flashbdev: change RESERVED_SECS to 0
zephyr port:
- add .gitignore to ignore Zephyr's "outdir" directory
- zephyr_getchar: update to Zephyr 1.6 unified kernel API
- switch to Zephyr 1.6 unified kernel API
- support raw REPL
- implement soft reset feature
- main: initialize sys.path and sys.argv
- use core-provided keyboard exception object
- uart_core: access console UART directly instead of printk() hack
- enable slice subscription
docs:
- remove references to readall() and update stream read() docs
- library/index: elaborate on u-modules
- library/machine.I2C: refine definitions of I2C methods
- library/pyb.Accel: add hardware note about pins used by accel
- library/pyb.UART: added clarification about timeouts
- library/pyb.UART: moved writechar doc to sit with other writes
- esp8266/tutorial: update intro to add Getting the firmware section
- library/machine.I2C: fix I2C constructor docs to match impl
- esp8266/tutorial: close socket after reading page content
- esp8266/general: add "Scarcity of runtime resources" section
- library/esp: document esp.set_native_code_location() function
- library/esp: remove para and add further warning about flash
- usocket: clarify that socket timeout raises OSError exception
travis:
- build STM32 F7 and L4 boards under Travis CI
- include persistent bytecode with floats in coverage tests
examples:
- hwapi: button_led: Add GPIO pin read example
- hwapi: add soft_pwm example converted to uasyncio
- http_client: use read() instead of readall()
- hwapi: add uasyncio example of fading 2 LEDs in parallel
- hwapi: add example for machine.time_pulse_us()
- hwapi: add hwconfig for console tracing of LED operations
- accellog.py: change 1: to /sd/, and update comment about FS
- hwapi/hwconfig_console: don't alloc memory in value()
With caching of map lookups in the bytecode, frozen bytecode can still
work but must be stored in RAM, not ROM. This patch allows mpy-tool.py to
generate code that works with this optimisation, but it's not recommended
to use it on embedded targets (because of lack of RAM).
Previous to this patch pyboard.py would open a new serial connection to
the target for each script that was run, and for any command that was run.
Apart from being inefficient, this meant that the board was soft-reset
between scripts/commands, which precludes scripts from accessing variables
set in a previous one.
This patch changes the behaviour of pyboard.py so that the connection to
the target is created only once, and it's not reset between scripts or any
command that is sent with the -c option.
To make its inclusion as frozen modules in multiple ports less magic.
Ports are just expected to symlink 2 files into their scripts/modules
subdirs.
Unix port updated to use this and in general follow frozen modules setup
tested and tried on baremetal ports, where there's "scripts" predefined
dir (overridable with FROZEN_DIR make var), and a user just drops Python
files there.
This helps to test floating point code on Cortex-M hardware.
As part of this patch the link-time-optimisation was disabled because it
wasn't compatible with software FP support. In particular, the linker
could not find the __aeabi_f2d, __aeabi_d2f etc functions even though they
were provided by lib/libm/math.c.
Frozen modules are now stored with extensions and with '/' as path
separator. In other words, frozen modules paths stored as they are
in normal filesystem.
When an mpy file is frozen it must know the values of certain
configuration variables. This patch provides an explicit check in the
generated C file that the configuration variables are what they are
supposed to be.
The config variable MICROPY_MODULE_FROZEN is now made of two separate
parts: MICROPY_MODULE_FROZEN_STR and MICROPY_MODULE_FROZEN_MPY. This
allows to have none, either or both of frozen strings and frozen mpy
files (aka frozen bytecode).
Scott Shawcroft
sw23
awachtler
Scott Shawcroft
stijn
Damien George
Dan Halbert
Jeff Epler
Lucian Copeland
Martin Milata
Michael Buesch
Yihui Xiong
Taku Fukada
hathach
Jeff Epler
lady ada
Kenny
dherrada
Thorsten von Eicken
Diego Elio Pettenò
dherrada
Andrew Leech
sommersoft
David Lechner
dherrada
dherrada
Mike Weiblen
Damien George
Lars Kellogg-Stedman
Jim Mussared
Sean Cross
David Lechner
MikeTeachman
Dan Halbert
c0rejump
arturo182
Jason Neal
Jeff Epler
Melissa LeBlanc-Williams
Yonatan Goldschmidt
Thea Flowers
Mirko Vogt
Kamil Tomaszewski
kamtom480
Josh Lloyd
Milan Rossa
sommersoft
C47D
Tavish Naruka
Radomir Dopieralski
Jun Wu
hexthat
rhubarbdog
Ayke van Laethem
Gregory P. Smith
vgoodwinv
Dave Hylands
Carl Karsten
sabas1080
Josh Klar
Martin Dybdal
Rich Barlow
roland
Scott Shawcroft
Noralf Trønnes
Keith Wiley
Karin Hawley
Tony DiCola
Kevin Townsend
Paul Sokolovsky
Alexander Steffen
Ville Skyttä
Krzysztof Blazewicz
Rami Ali
Alex March