This simplifies allocating outside of the VM because the VM doesn't
take up all remaining memory by default.
On ESP we delegate to the IDF for allocations. For all other ports,
we use TLSF to manage an outer "port" heap. The IDF uses TLSF
internally and we use their fork for the other ports.
This also removes the dynamic C stack sizing. It wasn't often used
and is not possible with a fixed outer heap.
Fixes#8512. Fixes#7334.
This 2-in-1 PR started with the goal of support the Bangle.js 2
smartwatch with *no USB*.
* Adds "secure" DFU build support with a committed private key.
* Adds 3-bit color support with one dummy bit for the JDI memory display
* Allows nrf boards to have a board_background_task() run in RUN_BACKGROUND_TASK.
This is needed because the Bangle.js 2 uses the watchdog to reset.
* Renamed port_background_task() to port_background_tick() to indicate it
runs on tick, not RUN_BACKGROUND_TASK.
* Marks serial connected when the display terminal is inited. This means
that safe mode messages show up on the display.
ACep, 7-color epaper displays also pack 3 bits in 4. So, I added that
support as well.
* Adds 3-bit ACeP color support for 7-color e-paper displays. (Not
watch related but similar due to color depth.)
* Allows a refresh sequence instead of a single int command. The 7" ACeP
display requires a data byte for refresh.
* Adds optional delay after resetting the display. The ACeP displays
need this. (Probably to load LUTs from flash.)
* Adds a cleaning phase for ACeP displays before the real refresh.
For both:
* Add dither support to Palette.
* Palette no longer converts colors when set. Instead, it caches
converted colors at each index.
* ColorConverter now caches the last converted color. It should make
conversions faster for repeated colors (not dithering.)
This targets the 64-bit CPU Raspberry Pis. The BCM2711 on the Pi 4
and the BCM2837 on the Pi 3 and Zero 2W. There are 64-bit fixes
outside of the ports directory for it.
There are a couple other cleanups that were incidental:
* Use const mcu_pin_obj_t instead of omitting the const. The structs
themselves are const because they are in ROM.
* Use PTR <-> OBJ conversions in more places. They were found when
mp_obj_t was set to an integer type rather than pointer.
* Optimize submodule checkout because the Pi submodules are heavy
and unnecessary for the vast majority of builds.
Fixes#4314
Hybrid allocation is now part of the infrastructure. Moving memory contents would not be necessary because displayio can recreate them, but does not hurt.
This allows calls to `allocate_memory()` while the VM is running, it will then allocate from the GC heap (unless there is a suitable hole among the supervisor allocations), and when the VM exits and the GC heap is freed, the allocation will be moved to the bottom of the former GC heap and transformed into a proper supervisor allocation. Existing movable allocations will also be moved to defragment the supervisor heap and ensure that the next VM run gets as much memory as possible for the GC heap.
By itself this breaks terminalio because it violates the assumption that supervisor_display_move_memory() still has access to an undisturbed heap to copy the tilegrid from. It will work in many cases, but if you're unlucky you will get garbled terminal contents after exiting from the vm run that created the display. This will be fixed in the following commit, which is separate to simplify review.
This is enabled by #3482
I was unable to determine why previously I had added sizeof(void*)
to the GC heap allocation, so I removed that code as a mistake.
It was incorrect to NULL out the pointer to our heap allocated buffer in
`reset`, because subsequent to framebuffer_reset, but while
the heap was still active, we could call `get_bufinfo` again,
leading to a fresh allocation on the heap that is about to be destroyed.
Typical stack trace:
```
#1 0x0006c368 in sharpdisplay_framebuffer_get_bufinfo
#2 0x0006ad6e in _refresh_display
#3 0x0006b168 in framebufferio_framebufferdisplay_background
#4 0x00069d22 in displayio_background
#5 0x00045496 in supervisor_background_tasks
#6 0x000446e8 in background_callback_run_all
#7 0x00045546 in supervisor_run_background_tasks_if_tick
#8 0x0005b042 in common_hal_neopixel_write
#9 0x00044c4c in clear_temp_status
#10 0x000497de in spi_flash_flush_keep_cache
#11 0x00049a66 in supervisor_external_flash_flush
#12 0x00044b22 in supervisor_flash_flush
#13 0x0004490e in filesystem_flush
#14 0x00043e18 in cleanup_after_vm
#15 0x0004414c in run_repl
#16 0x000441ce in main
```
When this happened -- which was inconsistent -- the display would keep
some heap allocation across reset which is exactly what we need to avoid.
NULLing the pointer in reconstruct follows what RGBMatrix does, and that
code is a bit more battle-tested anyway.
If I had a motivation for structuring the SharpMemory code differently,
I can no longer recall it.
Testing performed: Ran my complicated calculator program over multiple
iterations without observing signs of heap corruption.
Closes: #3473
