djsundog/circuitpython
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
circuitpython/py
History
Jeff Epler 40ab5c6b21 compression: Implement ciscorn's dictionary approach 
Massive savings.  Thanks so much @ciscorn for providing the initial
code for choosing the dictionary.

This adds a bit of time to the build, both to find the dictionary
but also because (for reasons I don't fully understand), the binary
search in the compress() function no longer worked and had to be
replaced with a linear search.

I think this is because the intended invariant is that for codebook
entries that encode to the same number of bits, the entries are ordered
in ascending value.  However, I mis-placed the transition from "words"
to "byte/char values" so the codebook entries for words are in word-order
rather than their code order.

Because this price is only paid at build time, I didn't care to determine
exactly where the correct fix was.

I also commented out a line to produce the "estimated total memory size"
-- at least on the unix build with TRANSLATION=ja, this led to a build
time KeyError trying to compute the codebook size for all the strings.
I think this occurs because some single unicode code point ('ァ') is
no longer present as itself in the compressed strings, due to always
being replaced by a word.

As promised, this seems to save hundreds of bytes in the German translation
on the trinket m0.

Testing performed:
 - built trinket_m0 in several languages
 - built and ran unix port in several languages (en, de_DE, ja) and ran
   simple error-producing codes like ./micropython -c '1/0'
2020-09-12 10:10:45 -05:00
..
argcheck.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
asmarm.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
asmarm.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
asmbase.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
asmbase.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
asmthumb.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
asmthumb.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
asmx64.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
asmx64.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
asmx86.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
asmx86.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
asmxtensa.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
asmxtensa.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
bc0.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
bc.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
bc.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
binary.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
binary.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
builtin.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
builtinevex.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
builtinhelp.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
builtinimport.c
builtinimport: Fix a crash with 'import ulab.linalg' on unix port only
2020-08-30 11:09:49 -05:00
circuitpy_defns.mk
Merge remote-tracking branch 'adafruit/main' into native_wifi
2020-09-03 16:34:56 -07:00
circuitpy_mpconfig.h
Merge pull request #3326 from tannewt/native_wifi
2020-09-10 11:20:44 -07:00
circuitpy_mpconfig.mk
Merge remote-tracking branch 'adafruit/main' into native_wifi
2020-09-03 16:34:56 -07:00
compile.c
Merge pull request #3222 from WarriorOfWire/pick_micropython
2020-07-29 10:54:37 -07:00
compile.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
emit.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
emitbc.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
emitcommon.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
emitglue.c
add coroutine behavior for generators
2020-07-23 20:40:16 -07:00
emitglue.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
emitinlinethumb.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
emitinlinextensa.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
emitnarm.c
py: Refactor how native emitter code is compiled with a file per arch.
2018-04-10 15:06:47 +10:00
emitnative.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
emitnthumb.c
py: Refactor how native emitter code is compiled with a file per arch.
2018-04-10 15:06:47 +10:00
emitnx64.c
py: Refactor how native emitter code is compiled with a file per arch.
2018-04-10 15:06:47 +10:00
emitnx86.c
py/emitnx86: Fix 32-bit x86 native emitter build by including header.
2018-05-04 20:39:16 +10:00
emitnxtensa.c
py: Refactor how native emitter code is compiled with a file per arch.
2018-04-10 15:06:47 +10:00
formatfloat.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
formatfloat.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
frozenmod.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
frozenmod.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
gc_long_lived.c
Remove debug extern
2020-03-03 10:55:50 -08:00
gc_long_lived.h
Introduce a long lived section of the heap.
2018-01-24 10:33:46 -08:00
gc.c
Get AllocationAlarm working
2020-07-17 17:15:03 -07:00
gc.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
grammar.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
ioctl.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
lexer.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
lexer.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
makemoduledefs.py
Add files via upload
2019-04-05 21:38:32 +02:00
makeqstrdata.py
compression: Implement ciscorn's dictionary approach
2020-09-12 10:10:45 -05:00
makeqstrdefs.py
Update filter and handle nested quotes
2018-08-09 14:16:28 -07:00
makeversionhdr.py
Support internationalisation.
2018-08-07 14:58:57 -07:00
malloc.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
map.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
misc.h
wip: compiles
2020-08-02 11:36:38 -04:00
mkenv.mk
Use python3 for mpy-tool
2018-10-18 10:37:42 -07:00
mkrules.mk
hci early wip; refactor supervisor bluetooth.c for nrf: tested
2020-06-25 20:57:17 -04:00
modarray.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
modbuiltins.c
Add basic error handling
2020-08-19 14:23:28 -07:00
modcmath.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
modcollections.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
modgc.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
modio.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
modmath.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
modmicropython.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
modstruct.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
modsys.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
modthread.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
moduerrno.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
mpconfig.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
mperrno.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
mphal.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
mpprint.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
mpprint.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
mpstate.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
mpstate.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
mpthread.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
mpz.c
Fix to pass mpy-cross build
2020-07-13 22:54:52 -05:00
mpz.h
core: add int.bit_length() when MICROPY_CYPTHON_COMPAT is enabled
2020-09-06 09:53:16 -05:00
nativeglue.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
nlr.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
nlr.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
nlrsetjmp.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
nlrthumb.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
nlrx64.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
nlrx86.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
nlrxtensa.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
obj.c
mp_obj_print_helper: Handle a ctrl-c that comes in during printing
2020-08-25 11:47:50 -05:00
obj.h
Merge remote-tracking branch 'adafruit/main' into native_wifi
2020-09-03 16:34:56 -07:00
objarray.c
Turn off find when CPYTHON_COMPAT is off
2020-07-21 15:40:51 -07:00
objarray.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objattrtuple.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objbool.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objboundmeth.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objcell.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objclosure.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objcomplex.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objdeque.c
More make_new fixes for unix build
2019-01-18 11:53:09 -08:00
objdict.c
"pop from empty %q"
2020-08-04 18:42:09 -05:00
objenumerate.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objexcept.c
Add basic error handling
2020-08-19 14:23:28 -07:00
objexcept.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objfilter.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objfloat.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objfun.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objfun.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objgenerator.c
remove new char*s because m0 is way oversubscribed
2020-07-23 20:41:10 -07:00
objgenerator.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objgetitemiter.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objint_longlong.c
core: add int.bit_length() when MICROPY_CYPTHON_COMPAT is enabled
2020-09-06 09:53:16 -05:00
objint_mpz.c
core: add int.bit_length() when MICROPY_CYPTHON_COMPAT is enabled
2020-09-06 09:53:16 -05:00
objint.c
core: add int.bit_length() when MICROPY_CYPTHON_COMPAT is enabled
2020-09-06 09:53:16 -05:00
objint.h
core: add int.bit_length() when MICROPY_CYPTHON_COMPAT is enabled
2020-09-06 09:53:16 -05:00
objlist.c
"pop from empty %q"
2020-08-04 18:42:09 -05:00
objlist.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objmap.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objmodule.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objmodule.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objnamedtuple.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objnamedtuple.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objnone.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objobject.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objpolyiter.c
all: Remove inclusion of internal py header files.
2017-10-04 12:37:50 +11:00
objproperty.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objproperty.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objrange.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objreversed.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objset.c
"pop from empty %q"
2020-08-04 18:42:09 -05:00
objsingleton.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objslice.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objstr.c
Combine 'index out of range' messages
2020-08-04 14:45:45 -05:00
objstr.h
Add externs. GCC10 complains about duplicate defines
2020-07-22 16:26:46 -07:00
objstringio.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objstringio.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objstrunicode.c
Combine 'index out of range' messages
2020-08-04 14:45:45 -05:00
objtuple.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objtuple.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objtype.c
various: Use mp_obj_get_type_qstr more widely
2020-08-04 14:45:45 -05:00
objtype.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
objzip.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
opmethods.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
parse.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
parse.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
parsenum.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
parsenum.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
parsenumbase.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
parsenumbase.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
persistentcode.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
persistentcode.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
proto.c
various: Use mp_obj_get_type_qstr more widely
2020-08-04 14:45:45 -05:00
proto.h
Fix up end of file and trailing whitespace.
2020-06-03 10:56:35 +01:00
py.mk
wip: compiles
2020-08-02 11:36:38 -04:00
pystack.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
pystack.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
qstr.c
sharpmemory: Implement support for Sharp Memory Displays in framebufferio
2020-08-12 07:32:18 -05:00
qstr.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
qstrdefs.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
reader.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
reader.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
reload.c
move reload exception to reload.c
2018-05-14 17:41:17 -04:00
reload.h
move reload exception to reload.c
2018-05-14 17:41:17 -04:00
repl.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
repl.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
ringbuf.c
ringbuf tested
2020-04-21 22:40:12 -04:00
ringbuf.h
ringbuf tested
2020-04-21 22:40:12 -04:00
runtime0.h
add coroutine behavior for generators
2020-07-23 20:40:16 -07:00
runtime_utils.c
WIP: after merge; before testing
2018-07-11 16:45:30 -04:00
runtime.c
Make socket reads interruptable
2020-08-21 11:00:02 -07:00
runtime.h
Make socket reads interruptable
2020-08-21 11:00:02 -07:00
scheduler.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
scope.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
scope.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
sequence.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
showbc.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
smallint.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
smallint.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
stackctrl.c
Fix up end of file and trailing whitespace.
2020-06-03 10:56:35 +01:00
stackctrl.h
Initial merge of micropython v1.9.2 into circuitpython 2.0.0 (in development) master.
2017-08-25 22:17:07 -04:00
stream.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
stream.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
unicode.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
unicode.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
vm.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
vmentrytable.h
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
vstr.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00
warning.c
Add license to some obvious files.
2020-07-06 19:16:25 +01:00