Commit Graph

46 Commits

Author SHA1 Message Date
Scott Shawcroft
de5a9d72dc
Compress all translated strings with Huffman coding.
This saves code space in builds which use link-time optimization.
The optimization drops the untranslated strings and replaces them
with a compressed_string_t struct. It can then be decompressed to
a c string.

Builds without LTO work as well but include both untranslated
strings and compressed strings.

This work could be expanded to include QSTRs and loaded strings if
a compress method is added to C. Its tracked in #531.
2018-08-16 17:40:57 -07:00
Scott Shawcroft
4513bd6ea3
Fix translation newlines
Escape table was incorrect
2018-08-10 16:17:03 -07:00
Scott Shawcroft
24e53ad591
Rework escaping and fix ESP build. 2018-08-09 15:58:45 -07:00
Scott Shawcroft
96ebf5bc3f
Two fixes and translate more strings.
* Fix finding translations with escaped characters.
* Add back \r to translations since its needed by screen.
2018-08-09 13:29:30 -07:00
Scott Shawcroft
933add6cd8
Support internationalisation. 2018-08-07 14:58:57 -07:00
Damien George
3678a6bdc6 py/modbuiltins: Make built-in dir support the __dir__ special method.
If MICROPY_PY_ALL_SPECIAL_METHODS is enabled then dir() will now delegate
to the special method __dir__ if the object it is listing has this method.
2018-05-10 23:14:23 +10:00
Paul Sokolovsky
9956fd0710 py/objtype: Fit qstrs for special methods in byte type.
Update makeqstrdata.py to sort strings starting with "__" to the beginning
of qstr list, so they get low qstr id's, guaranteedly fitting in 8 bits.
Then use this property to further compact op_id => qstr mapping arrays.
2017-10-21 11:06:32 +03:00
Damien George
f127bef3e4 py/makeqstrdata.py: Compute the qstr hash from bytes, not characters. 2016-09-02 14:32:47 +10:00
Damien George
202d5acd06 py/makeqstrdata.py: Allow to have double-quote characters in qstrs.
When rendering the qstr for a C header file, the double-quate character
must be escaped.
2016-05-23 15:18:55 +01:00
Damien George
a649d72606 py/makeqstrdata: Add special case to handle \n qstr. 2016-04-14 15:22:36 +01:00
Damien George
2243d68345 py/makeqstrdata: Reinstate Python2 compatibility. 2016-04-14 14:37:04 +01:00
Damien George
49bb04ee64 py/makeqstrdata: Fix rendering of qstrs that have non-printable ASCII.
The qstr data needs to be turned into a proper C string so non-ASCII
chars must be properly escaped according to C rules.
2016-04-14 14:20:25 +01:00
Damien George
f30b6f0af5 py/makeqstrdata: Add more names for escaped chars and esc non-printable.
Non-printable characters are escaped as 0xXX, where XX are the hex
digits of the character value.
2016-04-13 22:12:39 +01:00
Damien George
594fa73411 py/makeqstrdata: Factor out some code to functions that can be reused. 2016-04-13 16:05:43 +01:00
Paul Sokolovsky
53ca6ae1f3 py/makeqstrdata.py: Catch and report case of empty input file.
The usual cause would be that a cross-compiler for a port is not in PATH.
2015-10-11 11:09:57 +03:00
Tony Abboud
8d8fdcb4be stmhal: add option to query for the current usb mode
Fetch the current usb mode and return a string representation when
pyb.usb_mode() is called with no args. The possible string values are interned
as qstr's. None will be returned if an incorrect mode is set.
2015-09-03 23:30:43 +01:00
Paul Sokolovsky
3a2fb201a5 makeqstrdata.py: Typo fix in comment. 2015-07-31 14:58:14 +03:00
Damien George
c3bd9415cc py: Make qstr hash size configurable, defaults to 2 bytes.
This patch makes configurable, via MICROPY_QSTR_BYTES_IN_HASH, the
number of bytes used for a qstr hash.  It was originally fixed at 2
bytes, and now defaults to 2 bytes.  Setting it to 1 byte will save
ROM and RAM at a small expense of hash collisions.
2015-07-20 11:03:13 +00:00
Damien George
26b512ea1b py: Get makeqstrdata.py and makeversionhdr.py running under Python 2.6.
These scripts should run under as wide a range of Python versions as
possible.
2015-05-30 23:11:16 +01:00
Paul Sokolovsky
f88eec0de2 makeqstrdata.py: Add support for strings with backslash escapes. 2015-04-02 01:10:11 +03:00
Damien George
99ab64ffd4 py/makeqstrdata.py: Make it work again with both Python2 and Python3. 2015-01-11 22:40:38 +00:00
Damien George
95836f8439 py: Add MICROPY_QSTR_BYTES_IN_LEN config option, defaulting to 1.
This new config option sets how many fixed-number-of-bytes to use to
store the length of each qstr.  Previously this was hard coded to 2,
but, as per issue #1056, this is considered overkill since no-one
needs identifiers longer than 255 bytes.

With this patch the number of bytes for the length is configurable, and
defaults to 1 byte.  The configuration option filters through to the
makeqstrdata.py script.

Code size savings going from 2 to 1 byte:
- unix x64 down by 592 bytes
- stmhal down by 1148 bytes
- bare-arm down by 284 bytes

Also has RAM savings, and will be slightly more efficient in execution.
2015-01-11 22:27:30 +00:00
Damien George
6942f80a8f py: Add qstr cfg capability; generate QSTR_NULL and QSTR_ from script. 2015-01-11 22:06:53 +00:00
Damien George
56e1f99ca1 py/makeqstrdata.py: Add more allowed qstr characters; escape quot. 2015-01-11 14:16:24 +00:00
Damien George
e191d42188 py: Use % str formatting instead of {} in makeqstrdata.py.
Script is equivalent, but now also runs under ancient Python 2.6.
Goes part way to addressing issue #847.
2014-09-05 13:16:19 +01:00
Chris Angelico
de09caaa37 Bring the C and Python compute_hash functions into consistency 2014-06-07 07:06:18 +10:00
stijn
1dc7f0427b More relaxed parsing of preprocessed qstr header
The original parsing would error out on any C declarations that are not typedefs
or extern variables. This limits what can go in mpconfig.h and mpconfigport.h,
as they are included in qstr.h. For instance even a function declaration would be
rejected and including system headers is a complete no-go.
That seems too limiting for a global config header, so makeqstrdata now
ignores everything that does not match a qstr definition.
2014-05-03 10:26:31 +02:00
Damien George
708c073250 py: Add '*' qstr for 'import *'; use blank qstr for comprehension arg. 2014-04-27 19:23:46 +01:00
Damien George
897fe0c0d0 py: Add builtin functions bin and oct, and some tests for them. 2014-04-15 22:03:55 +01:00
Damien George
b013aea809 py: Fix builtin hex to print prefix.
I was too hasty.  Still a one-liner though.
2014-04-15 12:50:21 +01:00
Damien George
5805111732 py: Add hex builtin function.
A one-liner, added especially for @pfalcon :)
2014-04-15 12:42:52 +01:00
Damien George
3683789207 py: Clean up and add comments to makeqstrdata. 2014-04-14 23:38:37 +01:00
Damien George
5bb7d99175 py: Modify makeqstrdata to recognise better the output of CPP. 2014-04-13 13:16:51 +01:00
Paul Sokolovsky
73b7027b83 objstr: Add str.encode() and bytes.decode() methods.
These largely duplicate str() & bytes() constructors' functionality,
but can be used to achieve Python2 compatibility.
2014-04-13 06:45:02 +03:00
Paul Sokolovsky
a925cb54f1 py: Preprocess qstrdefs.h before feeding to makeqstrdata.py.
This is alternative implementation of supporting conditionals in qstrdefs.h,
hard to say if it's much cleaner than munging #ifdef's in Python code...
2014-04-12 00:39:55 +03:00
Paul Sokolovsky
6ea0e928d8 Revert "makeqstrdata.py: Add support for conditionally defined qstrs."
This reverts commit acb133d1b1.

Conditionals will be suported using C preprocessor.
2014-04-12 00:39:54 +03:00
Paul Sokolovsky
acb133d1b1 makeqstrdata.py: Add support for conditionally defined qstrs.
Syntax is usual C #if*/#endif, but each qstr must be wrapped individually.
2014-04-10 03:38:42 +03:00
Damien George
6e628c49ca py: Replace naive and teribble hash function with djb2. 2014-03-25 15:27:15 +00:00
Dave Hylands
0308f964a0 Fix makeqstrdata.py to work in Python 2.7 2014-03-10 00:07:35 -07:00
Damien George
fdf0da5436 makeqstrdata: print error to stderr. 2014-03-08 15:03:25 +00:00
Paul Sokolovsky
4aee119f94 stm: Initialize sys.path with ["0:/", "0:/src", "0:/lib"].
This is compatible with what search path was before sys.path refactor,
with addition of module library path ("0:/lib").
2014-02-18 00:06:37 +02:00
Damien George
a71c83a1d1 Change mp_obj_type_t.name from const char * to qstr.
Ultimately all static strings should be qstr.  This entry in the type
structure is only used for printing error messages (to tell the type of
the bad argument), and printing objects that don't supply a .print method.
2014-02-15 11:34:50 +00:00
Damien George
1976baeeb7 Retain file order of qstr definitions.
Want common qstrs to be first in the list so they have the lowest ids,
so that in the byte code they take up the least room.
2014-01-24 22:22:00 +00:00
Paul Sokolovsky
ab5d08280b Allow qstr's with non-ident chars, construct good identifier for them.
Also, add qstr's for string appearing in unix REPL loop, gross effect
being less allocations for each command run.
2014-01-24 02:34:22 +02:00
Dave Hylands
7a996b1f65 Fix malformed generated file when using python 2.7 2014-01-21 15:28:27 -08:00
Damien George
55baff4c9b Revamp qstrs: they now include length and hash.
Can now have null bytes in strings.  Can define ROM qstrs per port using
qstrdefsport.h
2014-01-21 21:40:13 +00:00