Commit Graph

643 Commits

Author SHA1 Message Date
Dan Halbert ce6a5e33fb
Merge pull request #8567 from eightycc/tinyfs2
Shrink root dir size for tiny (<=128K) FAT12 fs
2023-11-09 19:38:01 -05:00
eightycc 4560f49279
Update lib/oofatfs/ff.c
Co-authored-by: Dan Halbert <halbert@halwitz.org>
2023-11-07 13:51:32 -08:00
eightycc 2e72ea99e4
Update lib/oofatfs/ff.c
Co-authored-by: Dan Halbert <halbert@halwitz.org>
2023-11-07 13:51:18 -08:00
Bob Abeles 966e6e4fe5 Shrink root dir size for tiny (<=128K) FAT12 fs 2023-11-07 10:24:36 -08:00
Scott Shawcroft 8137e2d6d2
Switch all ports to auto-growing split heap
This simplifies allocating outside of the VM because the VM doesn't
take up all remaining memory by default.

On ESP we delegate to the IDF for allocations. For all other ports,
we use TLSF to manage an outer "port" heap. The IDF uses TLSF
internally and we use their fork for the other ports.

This also removes the dynamic C stack sizing. It wasn't often used
and is not possible with a fixed outer heap.

Fixes #8512. Fixes #7334.
2023-11-01 15:24:16 -07:00
Scott Shawcroft 00457788e9
Fix mbedtls and stop removing it 2023-10-24 14:47:43 -07:00
Dan Halbert 557efbeb29 restore lib/mbedtls; remove gzip doc; restore zlib tests 2023-10-24 12:51:54 -04:00
Dan Halbert 4b42a6f4a0 restore old uzlib; remove remaining U and u prefixes 2023-10-19 21:29:57 -04:00
Dan Halbert 367e13c69f change CIRCUITPY change markers to CIRCUITPY-CHANGE 2023-10-19 16:42:36 -04:00
Dan Halbert f2ebe6839c Initial MicroPython v1.21.0 merge; not compiled yet 2023-10-18 17:49:14 -04:00
Scott Shawcroft 544553c542
Fix cyw43 (pico w) builds 2023-10-16 10:43:30 -07:00
Scott Shawcroft 9633c4e78f
Merge remote-tracking branch 'adafruit/main' into v1.20-merge 2023-10-11 11:21:57 -07:00
Scott Shawcroft 4c4eba88db
Add lib/mbedtls back 2023-10-11 10:05:02 -07:00
Scott Shawcroft 376cb87a25
Update protomatter to fix non-S3 and C6 2023-10-05 14:56:11 -07:00
Scott Shawcroft 4d2bc35d40
Re-enable rgbmatrix on IDF 5.1 2023-10-05 14:56:11 -07:00
Scott Shawcroft 85650bf1ab
Fix re unescaping 2023-10-05 13:35:07 -07:00
Scott Shawcroft 18c03a74dd
Fix a few tests
* Re-enable a couple FATFS configurations we added.
* Remove MICROPY_PY_IO_FILEIO.
* Remove uasyncio from standard unix build.
* Re-add our unicode printing improvements.
2023-10-05 10:59:08 -07:00
Damien George 342ebcb41d lib/micropython-lib: Update submodule to latest.
Signed-off-by: Damien George <damien@micropython.org>
2023-10-05 11:21:34 +11:00
Damien George 5d53783a46 lib/cyw43-driver: Update driver to latest version v1.0.2.
Includes more error checking, and a fix to handle buffer overflow when
getting STA MACs.

Signed-off-by: Damien George <damien@micropython.org>
2023-10-05 11:20:05 +11:00
Jeff Epler 031c163f84
make RE compile 2023-09-22 13:57:05 -05:00
Jeff Epler 018baf75d5
Add micropython-lib
this is needed during the unix coverage build
2023-09-22 10:38:21 -05:00
Dan Halbert 10b95a1998 add translate.h and linker.h includes back to runtime.h; remove linker.h refs; remove top-level lib/cyw43-driver 2023-09-20 22:00:09 -04:00
Dan Halbert 2c0fa0f7dc initial merge from v1.20.0; just satisifying conflicts 2023-09-19 11:10:12 -04:00
iabdalkader dc5ea0c77d lib/protobuf-c: Add protobuf-c library.
Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2023-09-14 23:51:30 +10:00
iabdalkader 86fd958212 lib/tinyusb: Update to the most recent master.
Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2023-09-14 23:35:34 +10:00
Dan Halbert 9f1d52601e merge from 8.2.x 2023-09-12 20:14:30 -04:00
Dan Halbert 722d64b0e7 Add root cert DST Root CA X3 for Let's Encrypt 2023-09-11 12:07:45 -04:00
Dan Halbert 4dd45b6dc6
Merge pull request #8374 from dhalbert/new-certificates-submodule
Update TLS certificates and use new certificates submodule
2023-09-07 10:19:05 -04:00
Dan Halbert c5bc59e692 update protomatter to 1.6.1 (used for RGBMatrix) 2023-09-07 08:32:21 -04:00
Dan Halbert 58a21326b2 Update to new certificates submodule. 2023-09-06 19:36:48 -04:00
Dan Halbert d871bc37c5 remove lib/certificates/nina-fw submodule 2023-09-06 15:56:26 -04:00
Jim Mussared 32db4c58f7 extmod/moddeflate: Change default window size.
The primary purpose of this commit is to make decompress default to
wbits=15 when the format is gzip (or auto format with gzip detected). The
idea is that someone decompressing a gzip stream should be able to use the
default `deflate.DeflateIO(f)` and it will "just work" for any input
stream, even though it uses a lot of memory.

This is done by making uzlib report gzip files as having wbits set to 15
in their header (where it previously only set the wbits out parameter for
zlib files), and then fixing up the logic in `deflateio_init_read`.

Updates the documentation to match.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-09-01 12:23:37 +10:00
Scott Shawcroft f4dba86f1c
Merge remote-tracking branch 'adafruit/main' into update_tinyusb_abort 2023-08-29 11:11:36 -07:00
Jeff Epler c029d6a165
Merge remote-tracking branch 'origin/8.2.x' into merge-82x 2023-08-22 08:49:16 -05:00
Scott Shawcroft df19aa1646
Update TinyUSB to improve abort handling 2023-08-17 11:42:16 -07:00
Dan Halbert 2171e67c1b merge latest adafruit/main 2023-08-13 19:43:54 -04:00
Dan Halbert 8260f816f8
Merge pull request #8220 from jepler/update-protomatter-in-8.2.x
bump protomatter to 1.5.10
2023-08-10 06:35:30 -04:00
Jeff Epler a34d9b70e2
update protomatter to 1.5.10 2023-08-04 20:54:11 -05:00
Dan Halbert 0d4bc8c163 initial v1.19.1 merge; not compiled yet 2023-08-01 13:50:05 -04:00
Jeff Epler 76d69283a4
update certificates to nina-fw 1.7.5 2023-07-28 10:44:38 -05:00
Scott Shawcroft e45a61f57e
Align fatfs window buffer for tinyusb
Also, update tinyusb.
2023-07-27 09:15:58 -07:00
Jeff Epler 197a7f31ab
bump protomatter to 1.5.9 2023-07-26 15:02:05 -05:00
Jim Mussared e6c290c3d1 lib/uzlib: Add a source_read_data var to pass to source_read_cb.
For better abstraction for users of this API.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 19:29:34 +10:00
Jim Mussared 7f16bfca9f lib/uzlib/defl_static: Optimize zlib_start/finish_block.
Collapsing the two adjacent calls to outbits saves 32 bytes.

Bringing defl_static.c into lz77.c allows better inlining, saves 24 bytes.

Merge the Outbuf/uzlib_lz77_state_t structs, a minor simplification that
doesn't change code size.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 19:29:34 +10:00
Jim Mussared ef5061fefd lib/uzlib/tinflate: Implement more compact lookup tables.
Saves 68 bytes on PYBV11.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 19:29:34 +10:00
Jim Mussared d75a3cd861 lib/uzlib: Combine zlib/gzip header parsing to allow auto-detect.
This supports `wbits` values between +40 to +47.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 19:29:34 +10:00
Jim Mussared c2b8e6e5d6 lib/uzlib: Clean up tinf -> uzlib rename.
This library used a mix of "tinf" and "uzlib" to refer to itself.  Remove
all use of "tinf" in the public API.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 19:29:24 +10:00
Jim Mussared 0900976384 lib/uzlib/defl_static: Implement some code size improvements.
This commit makes the following changes:
- Replace 256-byte reverse-bits-in-byte lookup table with computation.
- Replace length and distance code lookup tables with computation.
- Remove comp_disabled check (it's unused).
- Make the dest_write_cb take the data pointer directly, rather than the
  Outbuf.

Saves 500 bytes on PYBV11.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 18:58:33 +10:00
Jim Mussared 82db9926ed lib/uzlib/lz77: Always use separate history buffer.
Because we only use the streaming source, this is just extra code size.

Saves 64 bytes on PYBV11.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 18:57:49 +10:00
Damien George c4feb806e0 lib/uzlib: Add memory-efficient, streaming LZ77 compression support.
The compression algorithm implemented in this commit uses much less memory
compared to the standard way of implementing it using a hash table and
large look-back window.  In particular the algorithm here doesn't allocate
hash table to store indices into the history of the previously seen text.
Instead it simply does a brute-force-search of the history text to find a
match for the compressor.  This is slower (linear search vs hash table
lookup) but with a small enough history (eg 512 bytes) it's not that slow.
And a small history does not impact the compression too much.

To give some more concrete numbers comparing memory use between the
approaches:

- Standard approach: inplace compression, all text to compress must be in
  RAM (or at least memory addressable), and then an additional 16k bytes
  RAM of hash table pointers, pointing into the text

- The approach in this commit: streaming compression, only a limited amount
  of previous text must be in RAM (user selectable, defaults to 512 bytes).

To compress, say, 1k of data, the standard approach requires all that data
to be in RAM, plus an additional 16k of RAM for the hash table pointers.
With this commit, you only need the 1k of data in RAM.  Or if it's
streaming from a file (or elsewhere), you could get away with only 256
bytes of RAM for the sliding history and still get very decent compression.

In summary: because compression takes such a large amount of RAM (in the
standard algorithm) and it's not really suitable for microcontrollers, the
approach taken in this commit is to minimise RAM usage as much as possible,
and still have acceptable performance (speed and compression ratio).

Signed-off-by: Damien George <damien@micropython.org>
2023-07-21 18:54:22 +10:00