Commit Graph

22 Commits

Author SHA1 Message Date
stijn a9448c0a86 all: Fix MICROPY_OBJ_REPR_D compilation with msvc. 2022-01-23 09:25:01 +11:00
Jim Mussared b326edf68c all: Remove MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE.
This commit removes all parts of code associated with the existing
MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE optimisation option, including the
-mcache-lookup-bc option to mpy-cross.

This feature originally provided a significant performance boost for Unix,
but wasn't able to be enabled for MCU targets (due to frozen bytecode), and
added significant extra complexity to generating and distributing .mpy
files.

The equivalent performance gain is now provided by the combination of
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE (which has
been enabled on the unix port in the previous commit).

It's hard to provide precise performance numbers, but tests have been run
on a wide variety of architectures (x86-64, ARM Cortex, Aarch64, RISC-V,
xtensa) and they all generally agree on the qualitative improvements seen
by the combination of MICROPY_OPT_LOAD_ATTR_FAST_PATH and
MICROPY_OPT_MAP_LOOKUP_CACHE.

For example, on a "quiet" Linux x64 environment (i3-5010U @ 2.10GHz) the
change from CACHE_MAP_LOOKUP_IN_BYTECODE, to LOAD_ATTR_FAST_PATH combined
with MAP_LOOKUP_CACHE is:

diff of scores (higher is better)
N=2000 M=2000       bccache -> attrmapcache      diff      diff% (error%)
bm_chaos.py        13742.56 ->   13905.67 :   +163.11 =  +1.187% (+/-3.75%)
bm_fannkuch.py        60.13 ->      61.34 :     +1.21 =  +2.012% (+/-2.11%)
bm_fft.py         113083.20 ->  114793.68 :  +1710.48 =  +1.513% (+/-1.57%)
bm_float.py       256552.80 ->  243908.29 : -12644.51 =  -4.929% (+/-1.90%)
bm_hexiom.py         521.93 ->     625.41 :   +103.48 = +19.826% (+/-0.40%)
bm_nqueens.py     197544.25 ->  217713.12 : +20168.87 = +10.210% (+/-3.01%)
bm_pidigits.py      8072.98 ->    8198.75 :   +125.77 =  +1.558% (+/-3.22%)
misc_aes.py        17283.45 ->   16480.52 :   -802.93 =  -4.646% (+/-0.82%)
misc_mandel.py     99083.99 ->  128939.84 : +29855.85 = +30.132% (+/-5.88%)
misc_pystone.py    83860.10 ->   82592.56 :  -1267.54 =  -1.511% (+/-2.27%)
misc_raytrace.py   21490.40 ->   22227.23 :   +736.83 =  +3.429% (+/-1.88%)

This shows that the new optimisations are at least as good as the existing
inline-bytecode-caching, and are sometimes much better (because the new
ones apply caching to a wider variety of map lookups).

The new optimisations can also benefit code generated by the native
emitter, because they apply to the runtime rather than the generated code.
The improvement for the native emitter when LOAD_ATTR_FAST_PATH and
MAP_LOOKUP_CACHE are enabled is (same Linux environment as above):

diff of scores (higher is better)
N=2000 M=2000        native -> nat-attrmapcache  diff      diff% (error%)
bm_chaos.py        14130.62 ->   15464.68 :  +1334.06 =  +9.441% (+/-7.11%)
bm_fannkuch.py        74.96 ->      76.16 :     +1.20 =  +1.601% (+/-1.80%)
bm_fft.py         166682.99 ->  168221.86 :  +1538.87 =  +0.923% (+/-4.20%)
bm_float.py       233415.23 ->  265524.90 : +32109.67 = +13.756% (+/-2.57%)
bm_hexiom.py         628.59 ->     734.17 :   +105.58 = +16.796% (+/-1.39%)
bm_nqueens.py     225418.44 ->  232926.45 :  +7508.01 =  +3.331% (+/-3.10%)
bm_pidigits.py      6322.00 ->    6379.52 :    +57.52 =  +0.910% (+/-5.62%)
misc_aes.py        20670.10 ->   27223.18 :  +6553.08 = +31.703% (+/-1.56%)
misc_mandel.py    138221.11 ->  152014.01 : +13792.90 =  +9.979% (+/-2.46%)
misc_pystone.py    85032.14 ->  105681.44 : +20649.30 = +24.284% (+/-2.25%)
misc_raytrace.py   19800.01 ->   23350.73 :  +3550.72 = +17.933% (+/-2.79%)

In summary, compared to MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE, the new
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE options:
- are simpler;
- take less code size;
- are faster (generally);
- work with code generated by the native emitter;
- can be used on embedded targets with a small and constant RAM overhead;
- allow the same .mpy bytecode to run on all targets.

See #7680 for further discussion.  And see also #7653 for a discussion
about simplifying mpy-cross options.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-09-16 16:04:03 +10:00
Jim Mussared 692d36d779 py: Implement partial PEP-498 (f-string) support.
This implements (most of) the PEP-498 spec for f-strings and is based on
https://github.com/micropython/micropython/pull/4998 by @klardotsh.

It is implemented in the lexer as a syntax translation to `str.format`:
  f"{a}" --> "{}".format(a)

It also supports:
  f"{a=}" --> "a={}".format(a)

This is done by extracting the arguments into a temporary vstr buffer,
then after the string has been tokenized, the lexer input queue is saved
and the contents of the temporary vstr buffer are injected into the lexer
instead.

There are four main limitations:
- raw f-strings (`fr` or `rf` prefixes) are not supported and will raise
  `SyntaxError: raw f-strings are not supported`.

- literal concatenation of f-strings with adjacent strings will fail
    "{}" f"{a}" --> "{}{}".format(a)    (str.format will incorrectly use
                                         the braces from the non-f-string)
    f"{a}" f"{a}" --> "{}".format(a) "{}".format(a) (cannot concatenate)

- PEP-498 requires the full parser to understand the interpolated
  argument, however because this entirely runs in the lexer it cannot
  resolve nested braces in expressions like
    f"{'}'}"

- The !r, !s, and !a conversions are not supported.

Includes tests and cpydiffs.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-14 16:58:40 +10:00
David Lechner 115acadf92 mpy-cross: Disable stack check when building with Emscripten.
`mpy-cross` can be compiled to WASM using Emscripten, but it is not happy
unless the stack check is disabled.

Signed-off-by: David Lechner <david@pybricks.com>
2021-06-25 10:48:56 +10:00
David CARLIER cb30928ac8 py/persistentcode: Introduce MICROPY_PERSISTENT_CODE_SAVE_FILE option.
This should be enabled when the mp_raw_code_save_file function is needed.

It is enabled for mpy-cross, and a check for defined(__APPLE__) is added to
cover Mac M1 systems.
2021-01-30 15:13:24 +11:00
Damien George 5c8bf12acf all: Fix auto-enable of MICROPY_GCREGS_SETJMP to select GC behaviour.
Only enable it if MICROPY_GCREGS_SETJMP is not already defined, and no
supported architecture is defined.
2020-04-30 16:49:42 +10:00
Jim Mussared 710426024a all: Factor gchelper code to one place and use it for unix & ARM ports.
No functionality change is intended with this commit, it just consolidates
the separate implementations of GC helper code to the lib/utils/ directory
as a general set of helper functions useful for any port.  This reduces
duplication of code, and makes it easier for future ports or embedders to
get the GC implementation correct.

Ports should now link against gchelper_native.c and either gchelper_m0.s or
gchelper_m3.s (currently only Cortex-M is supported but other architectures
can follow), or use the fallback gchelper_generic.c which will work on
x86/x64/ARM.

The gc_helper_get_sp function from gchelper_m3.s is not really GC related
and was only used by cc3200, so it has been moved to that port and renamed
to cortex_m3_get_sp.
2020-04-29 23:45:19 +10:00
stijn 84fa3312cf all: Format code to add space after C++-style comment start.
Note: the uncrustify configuration is explicitly set to 'add' instead of
'force' in order not to alter the comments which use extra spaces after //
as a means of indenting text for clarity.
2020-04-23 11:24:25 +10:00
Damien George 1a3e386c67 all: Remove spaces inside and around parenthesis.
Using new options enabled in the uncrustify configuration.
2020-03-28 23:36:44 +11:00
Damien George 69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
Damien George 1d21b4e7d1 mpy-cross: Enable Xtensa-Windowed native emitter.
Selectable via the command line: -march=xtensawin.
2019-10-05 13:45:32 +10:00
stijn 5501092776 mpy-cross: Make mpconfigport.h compatible with msvc compiler.
Add a copy of the required options from windows/mpconfigport.h.
2019-09-18 22:16:19 +10:00
Damien George c066dadc5b mpy-cross/mpconfigport.h: Remove defn of MP_NOINLINE to use global one.
A global definition of MP_NOINLINE was added to py/mpconfig.h long ago in
0f5bf1aafe
2019-05-24 14:51:48 +10:00
Damien George 9c9bc65e74 mpy-cross: Add "-march=<arch>" option to select native emitter. 2019-03-14 12:22:25 +11:00
Damien George 31d2d83e79 mpy-cross: Enable building of x64 native .mpy files. 2019-03-08 16:51:04 +11:00
Damien George 03053f82db mpy-cross, unix, windows, stmhal: Enable return-if-else optimisation.
Prior to making this a config option it was previously available on these
(and all other) ports, and it makes sense to keep it enabled for mpy-cross
as well as ports that have a decent amount of space for the code.
2017-04-22 15:12:48 +10:00
Damien George 4c307bfba1 all: Move BYTES_PER_WORD definition from ports to py/mpconfig.h
It can still be overwritten by a port in mpconfigport.h but for almost
all cases one can use the provided default.
2017-04-01 11:39:38 +11:00
Damien George 85ae17c993 mpy-cross: Get compiling after recent persistent code refactors. 2016-11-16 20:25:36 +11:00
Damien George 2b882e9aca mpy-cross: Don't use the internal printf functions.
They require mp_hal_stdout_tx_strn_cooked, which requires extra work to
add to mpy-cross.
2016-09-05 17:30:24 +10:00
Damien George 675d1c9c60 ports: Remove typedef of machine_ptr_t, it's no longer needed.
This type was used only for the typedef of mp_obj_t, which is now defined
by the object representation.  So we can now remove this unused typedef,
to simplify the mpconfigport.h file.
2016-08-15 11:02:59 +10:00
stijn 37b143ce9e mpy-cross: Fix mingw and msys2 compilation
When compiling with msys2's gcc there's no need to apply the binary fmode
so adjust the Makefile to reflect that.
When compiling with mingw we need to include malloc.h since there is no
alloca.h, and the 64bit detection in mpconfigport.h needs some adjustment.
2016-07-30 21:15:45 +03:00
Damien George 56f76b873a mpy-cross: Add new component, a cross compiler for MicroPython bytecode.
This component allows to generate .mpy files (pre compiled bytecode)
which can be executed within any MicroPython runtime/VM.
2016-02-25 10:12:21 +00:00