Hybrid allocation is now part of the infrastructure. Moving memory contents would not be necessary because displayio can recreate them, but does not hurt.
Hybrid allocation is now part of the infrastructure. Moving memory contents would not be necessary because displayio can recreate them, but does not hurt.
This allows calls to `allocate_memory()` while the VM is running, it will then allocate from the GC heap (unless there is a suitable hole among the supervisor allocations), and when the VM exits and the GC heap is freed, the allocation will be moved to the bottom of the former GC heap and transformed into a proper supervisor allocation. Existing movable allocations will also be moved to defragment the supervisor heap and ensure that the next VM run gets as much memory as possible for the GC heap.
By itself this breaks terminalio because it violates the assumption that supervisor_display_move_memory() still has access to an undisturbed heap to copy the tilegrid from. It will work in many cases, but if you're unlucky you will get garbled terminal contents after exiting from the vm run that created the display. This will be fixed in the following commit, which is separate to simplify review.
* Initialize the EPaper display on the MagTag at start.
* Tweak the display send to take a const buffer.
* Correct Luma math
* Multiply the blue component, not add.
* Add all of the components together before dividing. This
reduces the impact of truncated division.
After calling board.SPI().deinit(), calling board.SPI() again would return the unusable deinited object and there was no way of getting it back into an initialized state until the end of the session.
Fixes#3581.
Pins were marked as never_reset by common_hal_displayio_fourwire_construct() and common_hal_sharpdisplay_framebuffer_construct(), but these marks were never removed, so at the end of a session after displayio.release_displays(), {spi|i2c}_singleton would be set to NULL but the pins would not be reset. In the next session, board.SPI() and board.I2C() were unable to reconstruct the object because the pins were still in use.
For symmetry with creation of the singleton, add deinitialization before setting it to NULL in reset_board_busses(). This makes the pins resettable, so that reset_port(), moved behind it, then resets them.
At the end of a session that called displayio.release_displays() (and did not initialize a new display), a board.I2C() bus that was previously used by a display would wrongly be considered still in use. While I can’t think of any unrecoverable problem this would cause in the next session, it violates the assumption that a soft reboot resets everything not needed by displays, potentially leading to confusion.
By itself, this change does not fix the problem yet - rather, it introduces the same issue as in #3581 for SPI. This needs to be solved in the same way for I2C and SPI.
This is enabled by #3482
I was unable to determine why previously I had added sizeof(void*)
to the GC heap allocation, so I removed that code as a mistake.
@cwalther determined that for boards with 2 displays (monster m4sk),
start_terminal would be called for each one, leaking supervisor heap
entries.
Determine, by comparing addresses, whether the display being acted on
is the first display (number zero) and do (or do not) call start_terminal.
stop_terminal can safely be called multiple times, so there's no need
to guard against calling it more than once.
Slight behavioral change: The terminal size would follow the displays[0]
size, not the displays[1] size
If the display is paused, `_PM_swapbuffer_maybe` will never return.
So, when brightness is 0, refresh does nothing. This makes it necessary
to update the display when unpausing.
Closes: #3524
A call to supervisor_start_terminal remained in
common_hal_displayio_display_construct and was copied to other display
_construct functions, even though it was also being done in
displayio_display_core_construct when that was factored out.
Originally, this was harmless, except it created an extra allocation.
When investigating #3482, I found that this bug became harmful,
especially for displays that were created in Python code, because it
caused a supervisor allocation to leak.
I believe that it is safe to merge #3482 after this PR is merged.
An RGBMatrix has no bus and no bus_free method. It is always possible
to refresh the display.
This was not a problem before, but the fix I suggested (#3449) added
a call to core_bus_free when a FramebufferDisplay was being refreshed.
This was not caught during testing.
This is a band-aid fix and it brings to light a second problem in which
a SharpDisplay + FrameBuffer will not have a 'bus' object, and yet does
operate using a shared SPI bus. This kind of display will need a
"bus-free" like function to be added, or it can have problems like
#3309.
It was incorrect to NULL out the pointer to our heap allocated buffer in
`reset`, because subsequent to framebuffer_reset, but while
the heap was still active, we could call `get_bufinfo` again,
leading to a fresh allocation on the heap that is about to be destroyed.
Typical stack trace:
```
#1 0x0006c368 in sharpdisplay_framebuffer_get_bufinfo
#2 0x0006ad6e in _refresh_display
#3 0x0006b168 in framebufferio_framebufferdisplay_background
#4 0x00069d22 in displayio_background
#5 0x00045496 in supervisor_background_tasks
#6 0x000446e8 in background_callback_run_all
#7 0x00045546 in supervisor_run_background_tasks_if_tick
#8 0x0005b042 in common_hal_neopixel_write
#9 0x00044c4c in clear_temp_status
#10 0x000497de in spi_flash_flush_keep_cache
#11 0x00049a66 in supervisor_external_flash_flush
#12 0x00044b22 in supervisor_flash_flush
#13 0x0004490e in filesystem_flush
#14 0x00043e18 in cleanup_after_vm
#15 0x0004414c in run_repl
#16 0x000441ce in main
```
When this happened -- which was inconsistent -- the display would keep
some heap allocation across reset which is exactly what we need to avoid.
NULLing the pointer in reconstruct follows what RGBMatrix does, and that
code is a bit more battle-tested anyway.
If I had a motivation for structuring the SharpMemory code differently,
I can no longer recall it.
Testing performed: Ran my complicated calculator program over multiple
iterations without observing signs of heap corruption.
Closes: #3473
This already begins obscuring things, because now there are two sets of
shared-module functions for manipulating the same structure, e.g.,
common_hal_canio_remote_transmission_request_get_id and
common_hal_canio_message_get_id
Tested & working:
* Send standard packets
* Receive standard packets (1 FIFO, no filter)
Interoperation between SAM E54 Xplained running this tree and
MicroPython running on STM32F405 Feather with an external
transceiver was also tested.
Many other aspects of a full implementation are not yet present,
such as error detection and recovery.
The expectations of displayio.Display and frambufferio.FramebufferDisplay
are different when it comes to rotation.
In displayio.Display, if you call core_construct with a WxH = 64x32
and rotation=90, you get something that is 32 pixels wide and 64 pixels
tall in the LCD's coordinate system.
This is fine, as the existing definitions were written to work like this.
With framebuffer displays, however, the underlying framebuffer (such as
RGBMatrix) says "I am WxH pixels wide in my coordinate system" and the
constructor is given a rotation; when the rotation indicates a transpose
that means "exchange rows and columns, so that to the Groups displayed
on it, there is an effectively HxW pixel region for use".
Happily, we already have a set_rotation method. Thus (modulo the time
spent debugging things anyway:) the fix is simple: Always request no
rotation from core_construct, then immediately fix up the rotation
to match what was requested.
Testing performed: 32x16 RGBMatrix on Metro M4 Express (but using
the Airlift firmware, as this is the configuration the error was reported
on):
* initially construct display at 0, 90, 180, 270 degrees
* later change angle to 0, 90, 180, 270 degrees
* no garbled display
* no safe mode crashes
e.g., allocating a 192x32x6bpp matrix would be enough to trigger this
reliably on a Metro M4 Express using the "memory hogging" layout.
Allocating 64x32x6bpp could trigger it, but somewhat unreliably.
There are several things going on here:
* we make the failing call with interrupts off
* we were throwing an exception with interrupts off
* protomatter failed badly in _PM_free when it was partially-initialized
Incorporate the fix from protomatter, switch to a non-throwing malloc
variant, and ensure that interrupts get turned back on.
This decreases the quality of the MemoryError (it cannot report the size
of the failed allocation) but allows CircuitPython to survive, rather
than faulting.
this flips the bottom-right style to top-left which is at least
kind of normal. A 2x2 square at (0,0) would be defined like
(0,0), (3,0), (3,3), (0,3)
Which seems kind of surprising but at least less bonkers than
that square being defined at (1,1), which is the current behavior.
The getter for vectorio.Polygon#points was not updated with the data type change of the stored points list.
This moves the implementation to shared_module and updates the data type to reflect the actual state.
Before this, the mp3 file would be read into the in-memory buffer
only when new samples were actually needed. This meant that the time
to read mp3 content always counted against the ~22ms audio buffer length.
Now, when there's at least 1 full disk block of free space in the input
buffer, we can request that the buffer be filled _after_ returning from
audiomp3_mp3file_get_buffer and actually filling the DMA pointers. In
this way, the time taken for reading MP3 data from flash/SD is less
likely to cause an underrun of audio DMA.
The existing calls to fill the inbuf remain, but in most cases during
streaming these become no-ops because the buffer will be over half full.
Testing performed: That a card is successfully mounted on Pygamer with
the built in SD card slot
This module is enabled for most FULL_BUILD boards, but is disabled for
samd21 ("M0"), litex, and pca10100 for various reasons.
I noticed that this code was referring to samd-specific functionality,
and isn't enabled except in one samd board (pewpew10). Move it.
There is incomplte support for _pew in mimxrt10xx which then caused build
errors; adding a #if guard to check for _pew being enabled fixes it.
The _pew module is not likely to be important on mimxrt but I'll leave the
choice to remove it to someone else.
There were two main problems
- word_buffer was being filled as though with unsigned samples,
but during mixing all samples are kept in signed mode
- If the first buffer was stopped, the voices_active flag got set
anyway, even though the output buffer wasn't initialized yet,
so the samples were mixed with indeterminate data
We also cover the case where no buffer was playing, and ensure
the output buffer is filled.
This now works much better. Tested on neotrellis m4 playing back
4 mp3 streams at a time in signed-16, 22050Hz
When handling negative steps, start and stop need to be mp_int_t so they
can be checked against a potential negative value during the for loop
used to set the slice values.
This change takes polygon from 126k pixels per second fill to 240k pps fill
on a reference 5 point star 50x66px polygon, updating both location and shape
at 10hz. Tested on an m4 express feather.
As a curiosity, the flat-out fill rate of a shape whose get_pixel is `return 0;`
fills just shy of 375k pixels per second.
vectorio builds on m4 express feather
Concrete shapes are composed into a VectorShape which is put into a displayio Group for display.
VectorShape provides transpose and x/y positioning for shape implementations.
Included Shapes:
* Circle
- A radius; Circle is positioned at its axis in the VectorShape.
- You can freely modify the radius to grow and shrink the circle in-place.
* Polygon
- An ordered list of points.
- Beteween each successive point an edge is inferred. A final edge closing the shape is inferred between the last
point and the first point.
- You can modify the points in a Polygon. The points' coordinate system is relative to (0, 0) so if you'd like a
top-center justified 10x20 rectangle you can do points [(-5, 0), (5, 0), (5, 20), (0, 20)] and your VectorShape
x and y properties will position the rectangle relative to its top center point
* Rectangle
A width and a height.
This adds initial support for an AES module named aesio. This
implementation supports only a subset of AES modes, namely
ECB, CBC, and CTR modes.
Example usage:
```
>>> import aesio
>>>
>>> key = b'Sixteen byte key'
>>> cipher = aesio.AES(key, aesio.MODE_ECB)
>>> output = bytearray(16)
>>> cipher.encrypt_into(b'Circuit Python!!', output)
>>> output
bytearray(b'E\x14\x85\x18\x9a\x9c\r\x95>\xa7kV\xa2`\x8b\n')
>>>
```
This key is 16-bytes, so it uses AES128. If your key is 24- or 32-
bytes long, it will switch to AES192 or AES256 respectively.
This has been tested with many of the official NIST test vectors,
such as those used in `pycryptodome` at
39626a5b01/lib/Crypto/SelfTest/Cipher/test_vectors/AES
CTR has not been tested as NIST does not provide test vectors for it.
Signed-off-by: Sean Cross <sean@xobs.io>
This gets all the purely internal references. Some uses of
protomatter/Protomatter/PROTOMATTER remain, as they are references
to symbols in the Protomatter C library itself.
I originally believed that there would be a wrapper library around it,
like with _pixelbuf; but this proves not to be the case, as there's
too little for the library to do.
- bump supervisor alloc count by 4 (we actually use 5)
- move reconstruct to after gc heap is reset
- destroy protomatter object entirely if not used by a FramebufferDisplay
- ensure previous supervisor allocations are released
- zero out pointers so GC can collect them
It was fixed as 0/0 even though it used to get it from the current
SPI state. This makes it more explicit with kwargs.
Thanks to magpie_lark and kmatocha on the Adafruit Support forum
for finding the issue: https://forums.adafruit.com/viewtopic.php?f=60&t=162515
This removes downscaling (halving-add) when multiple voices are
being mixed. To avoid clipping, and get similar behavior to before,
set the "level" of each voice to (1/voice_count).
Slow paths that were applicable to only M0 chips were removed.
As a side effect, the internal volume representation is now 0 ..
0x8000 (inclusive), which additionally makes a level of exactly 0.5
representable.
Testing performed, on PyGamer: For all 4 data cases, for stereo and
mono, for 1 and 2 voices, play pure sign waves represented as
RawSamples and view the result on a scope and through headphones.
Also, scope the amount of time spent in background tasks.
Code size: growth of +272 bytes
Performance (time in background task when mixing 2 stereo 16-bit voices):
76us per down from 135us (once per ~2.9ms long term average)
(Decrease from 4.7% to 2.4% of all CPU time)
After adding the ability to change files in an existing MP3File object,
it became apparent that at the beginning of a track some part of an
existing buffer was playing first.
I noticed that in get_buffer, the just-populated buffer wasn't being
returned, but the other one was. But still after fixing this, I heard
wrong audio at the beginning of a track, so I took the heavy duty approach
and zeroed the buffers out. That means there's a remaining bug to chase,
which is merely hidden by the memset()s.
This enables jeplayer to allocate just one MP3File at startup, rather
than have to make repeated large allocations while the application is
running.
The buffers have to be allocated their theoretical maximum, but that
doesn't matter much as all the real-life MP3 files I checked needed
that much allocation anyway.
Apparently sometimes, a proper "frame info" block is not found after
a "sync word". Keep looking for one as needed, instead of giving up
after one try.
This was one reason that the "bartlebeats" mp3s would not play.
In conversion I missed these arguments were being passed, but noticed
it when an implausible value for audio_channel was sent to mp3's
reset_buffer method.
It's not clear whether there was any negative impact to this, but it
should be fixed!
There were several problems with the way this worked -- the read_count
approach was too complicated and I made a mistake "simplifying" it from
WaveFile. And when the right channel was returned, it was off by 1 byte,
making it into static.
Instead, directly track which is the "other" channel that has data
available, and by using the right data type make the "+ channel"
arithmetic give the right result.
This requires a double cast (int16_t*)(void*) due to an alignment warning;
the alignment is now ensured manually, but the compiler doesn't make the
necessary inference that the low address bit must be clear.
When a playing mp3 is deinitted, it's possible to reach get_buffer,
but all the internal pointers are NULL. This would lead to a hard fault.
Avoid it by returning GET_BUFFER_ERROR instead.
We weren't correctly collecting the start and stop sequences. As
a result, the GC would free the space and allocate other info
there.
Thanks to JacobT on Discord for the bug report!
This code is shared by most parts, except where not all the #ifdefs
inside the tick function were present in all ports. This mostly would
have broken gamepad tick support on non-samd ports.
The "ms32" and "ms64" variants of the tick functions are introduced
because there is no 64-bit atomic read. Disabling interrupts avoids
a low probability bug where milliseconds could be off by ~49.5 days
once every ~49.5 days (2^32 ms).
Avoiding disabling interrupts when only the low 32 bits are needed is a minor
optimization.
Testing performed: on metro m4 express, USB still works and
time.monotonic_ns() still counts up
This PR refines the _bleio API. It was originally motivated by
the addition of a new CircuitPython service that enables reading
and modifying files on the device. Moving the BLE lifecycle outside
of the VM motivated a number of changes to remove heap allocations
in some APIs.
It also motivated unifying connection initiation to the Adapter class
rather than the Central and Peripheral classes which have been removed.
Adapter now handles the GAP portion of BLE including advertising, which
has moved but is largely unchanged, and scanning, which has been enhanced
to return an iterator of filtered results.
Once a connection is created (either by us (aka Central) or a remote
device (aka Peripheral)) it is represented by a new Connection class.
This class knows the current connection state and can discover and
instantiate remote Services along with their Characteristics and
Descriptors.
Relates to #586
While finding sources of clicks and buzzes in nrf i2sout, I identified
this site as one which could be long running. Reproducer code was to
play a 22.05kHz sample and repeatedly print `os.listdir('')`
These arguments are constrained to be compile-time constants, a fact
that gcc complains about under "-Og" optimization, but not in normal
builds. Declare them as enumerated types
Also, move the rendering setup code to shared-module from
shared-bindings.
In CP 5.0, displayio_display_core_set_region_to_update now starts
its own transaction, so it has to be moved outside of the transaction
started by the render call.
Otherwise, examples like the one attached to the related issue fail
because tud_hid_ready never returns true.
Testing performed: Adapted the example to nrf particle xenon (it was
handy), removed dependency on IR, verified that the problem occurred
before this change, and that it was fixed after this change.
Closes: #2048
It lets us re-use the same buffer for playing multiple files.
This also allows us to control the size of the buffer. Half of the
buffer will be used for the fist, and half for the second internal
buffer.
When nrf pwm audio is introduced, it will be called `audiopwmio`. To
enable code sharing with the existing (dac-based) `audioio`, factor
the sample and mixer types to `audiocore`.
INCOMPATIBLE CHANGE: Now, `Mixer`, `RawSample` and `WaveFile` must
be imported from `audiocore`, not `audioio`.
This also improves Palette so it stores the original RGB888 colors.
Lastly, it adds I2CDisplay as a display bus to talk over I2C. Particularly
useful for the SSD1306.
Fixes#1828. Fixes#1956
This fixes the bug that bitmap changes do not cause screen updates
and optimizes the refresh when the bitmap is simply shown on the
screen. If the bitmap is used in tiles, then changing it will
cause all TileGrids using it to do a full refresh.
Fixes#1981
Different operations to the display tree have different costs. Be
aware of these costs when optimizing your code.
* Changing tiles indices in a TileGrid will update an area
covering them all.
* Changing a palette will refresh every object that references it.
* Moving a TileGrid will update both where it was and where it moved to.
* Adding something to a Group will refresh each individual area it
covers.
* Removing things from a Group will refresh one area that covers all
previous locations. (Not separate areas like add.)
* Setting a new top level Group will refresh the entire display.
Only TileGrid moves are optimized for overlap. All other overlaps
cause sending of duplicate pixels.
This also adds flip_x, flip_y and transpose_xy to TileGrid. They
change the direction of the pixels but not the location.
Fixes#1169. Fixes#1705. Fixes#1923.
This changes the displayio pixel computation from per-pixel to
per-area. This is precursor work to updating portions of the screen
(#1169). It should provide mild speedups because bounds checks are
done once per area rather than once per pixel. Filling by area also
allows TileGrid to maintain a row-associative fill pattern even when
the display's refresh is orthogonal to it.
If one of the default pins was already in use it would crash.
The internal API has been refined to allow us to get the value
without causing an init of the singleton.
Fixes#1753
The previous code assumed HID report ids were consecutive. This is
not true in the CircuitPython descriptor where report ids are fixed
for each report type.
Fixes#1617
This brings it inline with Group. Also fixes#1613
This also includes a number of fixes for where a method is called
through a subclass. We now correctly get the native object.
Fixes#1567
Lastly, this adds subscript support to TileGrid for changing tile
indices. Similar to Bitmap, it accepts ints or 2-tuples.
* Fixes safe mode on the SAMD51. The "preserved" value was being
clobbered by the bootloader.
* Fixes auto-reload loop when in safe mode.
* Fixes reading Group children with [].
* Check that a TileGrid actually moves before queueing a refresh.
This changes a number of things in displayio:
* Introduces BuiltinFont and Glyph so the built in font can be used by libraries. For boards with
a font it is available as board.TERMINAL_FONT. Fixes#1172
* Remove _load_row from Bitmap in favor of bitmap[] access. Index can be x/y tuple or overall index. Fixes#1191
* Add width and height properties to Bitmap.
* Add insert and [] access to Group. Fixes#1518
* Add index param to pop on Group.
* Terminal no longer takes unicode character info. It takes a BuiltinFont instead.
* Fix Terminal's handling of [###D vt100 commands used when up arrowing into repl history.
* Add x and y positions to Group plus scale as well.
* Add bitmap accessor for BuiltinFont
Display rotation is relative to the scan order of the display.
The scan order can be found by scrolling the display with command
0x37 `display_bus.send(0x37, struct.pack(">H", i % 128))`
Fixes#1504
* Fix Hallowing.
* Fix builds without displayio.
* Fix y bounds that appears as untrollable row of pixels.
* Add scrolling to TileGrid.
* Remove Sprite to save space. TileGrid is a drop in replacement.
This fixes commit a99f9427420d("'/' and '\' are also acceptable ends of the path now") which broke mkdir.
The problem is where the directory name is a single letter like this:
>>> os.mkdir('a')
>>> os.mkdir('a/b')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OSError: [Errno 17] File exists
>>> os.mkdir('a/bb')
>>>
I wasn't smart enough to fix this in the oofatfs library, so I did it in the os shared module by
creating a path lookup function for the os methods that only deals with directories. I reverted
the library change introduced by the aforementioned commit.
This means that os.stat and os.rename can't handle trailing slashes. This is to avoid allowing
filenames with trailing slashes to pass through. In order to handle trailing slashes for these
it would be necessary to check if it really is a directory before stripping. I didn't do this
since the original issue was to make os.chdir tolerate trailing slashes.
There's an open MicroPython issue #2929 wrt. trailing slashes and mkdir.
This started while adding USB MIDI support (and descriptor support is
in this change.) When seeing that I'd have to implement the MIDI class
logic twice, once for atmel-samd and once for nrf, I decided to refactor
the USB stack so its shared across ports. This has led to a number of
changes that remove items from the ports folder and move them into
supervisor.
Furthermore, we had external SPI flash support for nrf pending so I
factored out the connection between the usb stack and the flash API as
well. This PR also includes the QSPI support for nRF.
This was the last class from ubluepy and so that module is now gone.
The Device class offers both Peripheral and Central functionality.
See the inline docs for more info.
Also, renamed Sprite's palette to pixel_shader so it can be
anything that produces colors based on values (including color values).
Added a ColorConverter that converts RGB888 (found in bitmaps) to
RGB565 for the display.
Fixes#1182
It's designed to minimize RAM footprint by using Sprites to
represent objects on the screen. The object model also facilitates
partial screen updating which reduces the bandwidth needed to display.
This is all handled in C. Python simply manipulates the objects with
the ability to synchronize to frame timing.
This saves code space in builds which use link-time optimization.
The optimization drops the untranslated strings and replaces them
with a compressed_string_t struct. It can then be decompressed to
a c string.
Builds without LTO work as well but include both untranslated
strings and compressed strings.
This work could be expanded to include QSTRs and loaded strings if
a compress method is added to C. Its tracked in #531.
Don't check the pin's pull direction on every tick, instead cache it
at the beginning. Also avoid a "can't get pull of output pin" error
when one of the pins passed is in output mode.
This evolves the API from 2.x (and breaks it). Playback devices are now
separate from the samples themselves. This allows for greater playback
flexibility. Two sample sources are audioio.RawSample and audioio.WaveFile.
They can both be mono or stereo. They can be output to audioio.AudioOut or
audiobusio.I2SOut.
Internally, the dma tracking has changed from a TC counting block transfers
to an interrupt generated by the block event sent to the EVSYS. This reduces
the overhead of each DMA transfer so multiple can occure without using up TCs.
Fixes#652. Fixes#522. Huge progress on #263
While it is traditional to have buttons on pins that are pulled up, and
have the button connect them to the ground, some CircuitPython boards
(notably the CPX) have the button pins pulled low and the button
connects them to VCC.
This patch makes the gamepad only change the pin's pull if it wasn't
already set when passed to the constructor, and also makes it consider
a button pressed when its value is the opposite of its pull.
.. defaulting to off for circuitpython-supported boards, on for others.
.. fixing up the tests that fail when it is turned off, so that they skip
instead of failing
Introduction of ports subdirectory where all ports are moved to
The main change in this release is the introduction of a "ports/"
subdirectory at the top-level of the repository, and all of the ports are
moved here. In the process the "stmhal" port is renamed to "stm32" to
better reflect the MCU that it targets. In addition, the STM32 CMSIS and
HAL sources are moved to a new submodule called "stm32lib".
The bytecode has changed in this release, compared to the previous release,
and as a consequence the .mpy version number has increased to version 3.
This means that scripts compiled with the previous mpy-cross must be
recompiled to work with this new version.
There have also been various enhancements and optimisations, such as:
check for valid UTF-8 when creating str objects, support for reverse
special binary operations like __radd__, full domain checking in the math
module, support for floor-division and modulo in the viper emitter,
and addition of stack overflow checking when executing a regex.
The stm32 port sees improved support for F7 MCUs, addition of a new board
B_L475E_IOT01A based on the STM32L475, and support for the Wiznet W5500
chipset along with improved socket behaviour.
A detailed list of changes follows.
py core:
- objstr: startswith, endswith: check arg to be a string
- nlrx86,x64: replace #define of defined() with portable macro usage
- objtype: handle NotImplemented return from binary special methods
- objtype: mp_obj_class_lookup: improve debug logging
- map: remove unused new/free functions
- make m_malloc_fail() have void return type, since it doesn't return
- modstruct: in struct.pack, stop converting if there are no args left
- modstruct: check and prevent buffer-read overflow in struct unpacking
- modstruct: check and prevent buffer-write overflow in struct packing
- nlrthumb: get working again on standard Thumb arch (ie not Thumb2)
- objfloat: fix binary ops with incompatible objects
- obj: fix comparison of float/complex NaN with itself
- objtype: implement fallback for instance inplace special methods
- objtuple: properly implement comparison with incompatible types
- objstr: add check for valid UTF-8 when making a str from bytes
- objlist: properly implement comparison with incompatible types
- runtime0.h: move relational ops to the beginning of mp_binary_op_t
- runtime0.h: move MP_BINARY_OP_DIVMOD to the end of mp_binary_op_t
- objtype: make sure mp_binary_op_method_name has full size again
- runtime0.h: put inplace arith ops in front of normal operations
- builtinhelp: simplify code slightly by extracting object type
- runtime: implement dispatch for "reverse op" special methods
- nlrx86: fix building for Android/x86
- builtinhelp: change signature of help text var from pointer to array
- runtime.h: change empty mp_warning macro so var-args are non empty
- modbuiltins: implement abs() by dispatching to MP_UNARY_OP_ABS
- {objfloat,objcomplex}: optimise MP_UNARY_OP_ABS by reusing variables
- mpconfig.h: add note that using computed gotos in VM is not C99
- objstr: strip: don't strip "\0" by default
- objexcept: prevent infinite recursion when allocating exceptions
- stream: remove unnecessary checks for NULL return from vstr_add_len
- vstr: raise a RuntimeError if fixed vstr buffer overflows
- vm: use lowercase letter at start of exception message
- persistentcode: define mp_raw_code_save_file() for any unix target
- add config option to print warnings/errors to stderr
- objfloat: support raising a negative number to a fractional power
- objset: simplify set and frozenset by separating their locals dicts
- objset: check that RHS of a binary op is a set/frozenset
- objset: include the failed key in a KeyError raised from set.remove
- objtype: change type of enum-to-qstr table to uint16_t to save space
- objstr: make empty bytes object have a null-terminating byte
- mpprint: only check for null string printing when NDEBUG not defined
- objtype: clean up unary- and binary-op enum-to-qstr mapping tables
- persistentcode: bump .mpy version number to version 3
- bc: update opcode_format_table to match the bytecode
- modmath: add full checks for math domain errors
- modmath: convert log2 macro into a function
- formatfloat: don't print the negative sign of a NaN value
- formatfloat: use standard isinf, isnan funcs instead of custom ones
- modbuiltins: use existing utf8_get_char helper in builtin ord func
- emitnative: implement floor-division and modulo for viper emitter
- objtype: use CPython compatible method name for sizeof
- objtype: fit qstrs for special methods in byte type
- objtype: define all special methods if requested
- objtype: introduce MICROPY_PY_ALL_INPLACE_SPECIAL_METHODS
extmod:
- modubinascii: only include uzlib/tinf.h when it's really needed
- modussl_mbedtls: allow to compile with MBEDTLS_DEBUG_C disabled
- machine_pinbase: put PinBase singleton in ROM
- re1.5: upgrade to v0.8.2, adds hook for stack overflow checking
- modure: add stack overflow checking when executing a regex
- uos_dupterm: update uos.dupterm() and helper funcs to have index
- uos_dupterm: swallow any errors from dupterm closing the stream
- vfs: replace VLA in proxy func with small, static sized array
- modussl: add finaliser support for ussl objects
- modussl_mbedtls: allow to compile with unix coverage build
lib:
- add new submodule, stm32lib containing STM32 CMSIS and HAL source
- embed/abort_: use mp_raise_msg helper function
- libm: fix tanhf so that it correctly handles +/- infinity args
- libm: remove implementation of log2f, use MP_NEED_LOG2 instead
- axtls: update, support for SSL_EAGAIN return code
- berkeley-db-1.xx: update, allow to override MINCACHE, DEFPSIZE
drivers:
- memory/spiflash: change from hard-coded soft SPI to generic SPI
- display/ssd1306.py: improve performance of graphics methods
- nrf24l01: make nRF24L01 test script more portable
- display/ssd1306: implement SSD1306_I2C poweron method
- display/ssd1306: make poweron() work the same with SSD1306_SPI
- wiznet5k: improve the performance of socket ops with threading
- wiznet5k: get low-level W5500 driver working
tools:
- upip: upgrade to 1.2.2
- pyboard: use repr() when quoting data in error messages
- pyboard: update docstring for additional device support
tests:
- object_new: better messages, check user __new__() method
- class_new: add checks for __init__ being called and other improvements
- class_new: add another testcase for __new__/__init__ interaction
- class_inplace_op: test for inplace op fallback to normal one
- run-bench-tests: update locations of executables, now in ports/
- class_reverse_op: test for reverse arith ops special methods
- run-tests: skip class_inplace_op for minimal profile
- run-tests: fix copy-paste mistake in var name
- cpydiff: add cases for locals() discrepancies
- extmod: add test for ure regexes leading to infinite recursion
- extmod: add test for '-' in character class in regex
- run-tests: close device under test using "finally"
- net_inet: update tls test to work with CPython and incl new site
unix port:
- rename modsocket.c to modusocket.c
- modusocket: remove #if MICROPY_SOCKET_EXTRA code blocks
- enable MICROPY_PY_REVERSE_SPECIAL_METHODS
stm32 port:
- modmachine: make machine.bootloader() work when MPU is enabled
- modmachine: improve support for sleep/deepsleep on F7 MCUs
- compute PLL freq table during build instead of at run time
- modmachine: for F7 MCU, save power by reducing internal volt reg
- boards/pllvalues.py: make script work with both Python 2 and 3
- Makefile: use lib/stm32lib instead of local cmsis and hal files
- remove cmsis and hal files, they are now a submodule
- Makefile: automatically fetch stm32lib submodule if needed
- update to new STM Cube HAL library
- fix clock initialisation of L4 MCUs
- rename stmhal port directory to stm32
- remove unused usbd_msc.c file
- boards: change remaining stm32f4xx_hal_conf.h to unix line ending
- boards: change linker scripts to use "K" instead of hex byte size
- boards: fix I2C1 pin mapping on NUCLEO_F401RE/F411RE boards
- i2c: when scanning for I2C devices only do 1 probe per address
- modnwwiznet5k: release the GIL on blocking network operations
- boards: add new board B_L475E_IOT01A based on STM32L475
- make-stmconst.py: make sure mpz const data lives in ROM
- timer: make pyb.Timer() instances persistent
- mpconfigport.h: add configuration for max periphs on L4 series
- usbdev: make the USBD callback struct const so it can go in ROM
- usbdev: change static function variable to non-static
- usbdev: put all CDC state in a struct
- usbdev: put all HID state in a struct
- usbdev: simplify CDC tx/rx buffer passing
- usbdev: simplify HID tx/rx buffer passing
- usbdev/core: add state parameter to all callback functions
- usbdev: put all state for the USB device driver in a struct
- usbdev: simplify pointers to MSC state and block dev operations
- usbdev: merge all global USB device state into a single struct
- usbdev: make device descriptor callbacks take a state pointer
- usbdev: move all the USB device descriptor state into its struct
- timer: enable ARPE so that timer freq can be changed smoothly
- modnwwiznet5k: get the IP address of an established socket
- boards: fix typos in stm32f767_af.csv table
- usbd_cdc_interface: don't reset CDC output buf on initialisation
- modnwwiznet5k: implement WIZNET5K.isconnected() method
- modusocket: make getaddrinfo() work when passed an IP address
- modusocket: return OSError(-2) if getaddrinfo fails
- mpconfigport.h: add MICROPY_THREAD_YIELD() macro
- modnwwiznet5k: add support for W5500 Ethernet chip
- modnwwiznet5k: increase SPI bus speed to 42MHz
- modnwwiznet5k: implement stream ioctl for the Wiznet driver
- mphalport: improve efficiency of mp_hal_stdout_tx_strn_cooked
- make uos.dupterm() conform to specs by using extmod version
cc3200 port:
- enable micropython.kbd_intr() method
- use standard implementation of keyboard interrupt
esp8266 port:
- rename axtls_helpers.c to posix_helpers.c
- posix_helpers: set ENOMEM on memory alloc failure
- set DEFPSIZE=1024, MINCACHE=3 for "btree" module
- esp_mphal: send data in chunks to mp_uos_dupterm_tx_strn
- modnetwork: add "bssid" keyword arg to WLAN.connect() method
- modules/webrepl_setup: add info about allowed password length
zephyr port:
- Makefile: revamp "test" target after ports were moved to ports/
- use CONFIG_NET_APP_SETTINGS to setup initial network addresses
- switch to interrupt-driven pull-style console
pic16bit port:
- add definition of SEEK_SET to unistd.h
docs:
- pyboard/tutorial: add "timeout=0" to UART in pass-through example
- more xrefs to "MicroPython port" in glossary
- library/network: fix ref to "socket" module (should be "usocket")
- machine.Signal: improve style/grammar and add usage example
- library: add description of "index" parameter to uos.dupterm()
- library/micropython: fix typo in RST formatting
- library/framebuf.rst: generalise constructor to all colour formats
- btree: describe page caching policy of the underlying implementation
- esp8266/tutorial: update neopixel with example of using 4 bbp
- library/network: clarify usage of "bssid" arg in connect() method
- pyboard/quickref: add info for Switch, RTC, CAN, Accel classes
- pyboard/tutorial: update now that yellow LED also supports PWM
- esp8266/quickref: add quickref info for RTC class
- library: add missing cross-ref links for classes in pyb module
- library/network: update docs to state that W5500 is supported
- uselect: document one-shot polling mode
- usocket: elaborate descriptions
- usocket: document inet_ntop(), inet_pton()
- library/network: add dhcp_hostname parameter
- reference/isr_rules: minor typo correction
- ussl: fix module name refs and use "MicroPython port" term
- esp8266/general: add section on TLS limitations
- usocket: document that settimeout() isn't supported by all ports
- ure: add "|" (alternative) to the list of supported operators
- reference/isr_rules.rst: add tutorial on use of micropython.schedule()
travis:
- use --upgrade when pip is installing cpp-coveralls
- update build command now that stm32 Wiznet config has changed
examples:
- hwconfig_console: add .on()/.off() methods
all:
- convert mp_uint_t to mp_unary_op_t/mp_binary_op_t where appropriate
- convert remaining "mp_uint_t n_args" to "size_t n_args"
- make new ports/ sub-directory and move all ports there
- update Makefiles and others to build with new ports/ dir layout
- remove inclusion of internal py header files
- use NULL instead of "" when calling mp_raise exception helpers
README:
- update "Dependencies" section
- add explicit section on contributing
- add gcc and arm-none-eabi-newlib to list of required components
.gitattributes:
- remove obsolete entries for stmhal/hal, stmhal/cmsis
- add entries for files that will move to ports/ dir
If a soft reset happens while the gamepad module is scanning for button
presses, there is a moment when the pins get de-initialized, but the
gamepad module is still trying to read them, which ends in a crash.
We can avoid it by disabling scanning on reset.
(cherry picked from commit 470a23d4c9)
Conflicts:
atmel-samd/main.c
also initializes in the same way where it takes from a true random
source when available through os.urandom(). After initializing, it
produces deterministic results until the seed is set.
This replaces urandom!
Fixes#139.
Fixes for stmhal USB mass storage, lwIP bindings and VFS regressions
This release provides an important fix for the USB mass storage device in
the stmhal port by implementing the SCSI SYNCHRONIZE_CACHE command, which
is now require by some Operating Systems. There are also fixes for the
lwIP bindings to improve non-blocking sockets and error codes. The VFS has
some regressions fixed including the ability to statvfs the root.
All changes are listed below.
py core:
- modbuiltins: add core-provided version of input() function
- objstr: catch case of negative "maxsplit" arg to str.rsplit()
- persistentcode: allow to compile with complex numbers disabled
- objstr: allow to compile with obj-repr D, and unicode disabled
- modsys: allow to compile with obj-repr D and PY_ATTRTUPLE disabled
- provide mp_decode_uint_skip() to help reduce stack usage
- makeqstrdefs.py: make script run correctly with Python 2.6
- objstringio: if created from immutable object, follow copy on write policy
extmod:
- modlwip: connect: for non-blocking mode, return EINPROGRESS
- modlwip: fix error codes for duplicate calls to connect()
- modlwip: accept: fix error code for non-blocking mode
- vfs: allow to statvfs the root directory
- vfs: allow "buffering" and "encoding" args to VFS's open()
- modframebuf: fix signed/unsigned comparison pendantic warning
lib:
- libm: use isfinite instead of finitef, for C99 compatibility
- utils/interrupt_char: remove support for KBD_EXCEPTION disabled
tests:
- basics/string_rsplit: add tests for negative "maxsplit" argument
- float: convert "sys.exit()" to "raise SystemExit"
- float/builtin_float_minmax: PEP8 fixes
- basics: convert "sys.exit()" to "raise SystemExit"
- convert remaining "sys.exit()" to "raise SystemExit"
unix port:
- convert to use core-provided version of built-in import()
- Makefile: replace references to make with $(MAKE)
windows port:
- convert to use core-provided version of built-in import()
qemu-arm port:
- Makefile: adjust object-file lists to get correct dependencies
- enable micropython.mem_*() functions to allow more tests
stmhal port:
- boards: enable DAC for NUCLEO_F767ZI board
- add support for NUCLEO_F446RE board
- pass USB handler as parameter to allow more than one USB handler
- usb: use local USB handler variable in Start-of-Frame handler
- usb: make state for USB device private to top-level USB driver
- usbdev: for MSC implement SCSI SYNCHRONIZE_CACHE command
- convert from using stmhal's input() to core provided version
cc3200 port:
- convert from using stmhal's input() to core provided version
teensy port:
- convert from using stmhal's input() to core provided version
esp8266 port:
- Makefile: replace references to make with $(MAKE)
- Makefile: add clean-modules target
- convert from using stmhal's input() to core provided version
zephyr port:
- modusocket: getaddrinfo: Fix mp_obj_len() usage
- define MICROPY_PY_SYS_PLATFORM (to "zephyr")
- machine_pin: use native Zephyr types for Zephyr API calls
docs:
- machine.Pin: remove out_value() method
- machine.Pin: add on() and off() methods
- esp8266: consistently replace Pin.high/low methods with .on/off
- esp8266/quickref: polish Pin.on()/off() examples
- network: move confusingly-named cc3200 Server class to its reference
- uos: deconditionalize, remove minor port-specific details
- uos: move cc3200 port legacy VFS mounting functions to its ref doc
- machine: sort machine classes in logical order, not alphabetically
- network: first step to describe standard network class interface
examples:
- embedding: use core-provided KeyboardInterrupt object
* Track status pin use by user code separately so it can take over the pins and then give them back.
* Switch to hardware SPI for APA102 on Gemma and Trinket.
* Merge microcontroller/types.h into microcontroller/Pin.h to better match approach going forwards.
This was done to allow greatly granularity when deciding what functionality
is built into each board's build. For example, this way pulseio can be
omitted to allow for something else such as touchio.
This class focuses on the timing sensitive parts of the protocol.
Everything else will be done by Python code.
This also establishes that its OK to back a nativeio class with a
bitbang implementation when no hardware acceleration exists. When
it does, then bitbangio should be used to explicitly bitbang a
protocol.
1) Bus error will be thrown on read/write errors with errno set. (Read didn't used to fail at all.)
2) try_lock correctly returns boolean whether lock was grabbed.
Fixes#87
The new sequence is as follows:
* Solid blue during the boot/settings script.
* Solid green during the main/code script.
* After main while waiting to enter repl or reset:
* Fading green once main is done successfully.
* On error produce a series of flashes:
* Long flash color of script.
* Long flash color of error:
* Green = IndentationError
* Cyan = SyntaxError
* White = NameError
* Orange = OSError
* Yellow = Other error
* Line number of the exception by digit. Number of flashes represents value.
* Thousands = White
* Hundreds = Blue
* Tens = Yellow
* Ones = Cyan
* Off for a period and then repeats.
At any point a write to the flash storage will flicker red.
Fixes#63
Docs are here: http://tannewt-micropython.readthedocs.io/en/microcontroller/
It differs from upstream's machine in the following ways:
* Python API is identical across ports due to code structure. (Lives in shared-bindings)
* Focuses on abstracting common functionality (AnalogIn) and not representing structure (ADC).
* Documentation lives with code making it easy to ensure they match.
* Pin is split into references (board.D13 and microcontroller.pin.PA17) and functionality (DigitalInOut).
* All nativeio classes claim underlying hardware resources when inited on construction, support Context Managers (aka with statements) and have deinit methods which release the claimed hardware.
* All constructors take pin references rather than peripheral ids. Its up to the implementation to find hardware or throw and exception.
