djsundog/circuitpython
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
0c2cadf1c8
circuitpython/py
History
Jim Mussared 692d36d779 py: Implement partial PEP-498 (f-string) support. 
This implements (most of) the PEP-498 spec for f-strings and is based on
https://github.com/micropython/micropython/pull/4998 by @klardotsh.

It is implemented in the lexer as a syntax translation to `str.format`:
  f"{a}" --> "{}".format(a)

It also supports:
  f"{a=}" --> "a={}".format(a)

This is done by extracting the arguments into a temporary vstr buffer,
then after the string has been tokenized, the lexer input queue is saved
and the contents of the temporary vstr buffer are injected into the lexer
instead.

There are four main limitations:
- raw f-strings (`fr` or `rf` prefixes) are not supported and will raise
  `SyntaxError: raw f-strings are not supported`.

- literal concatenation of f-strings with adjacent strings will fail
    "{}" f"{a}" --> "{}{}".format(a)    (str.format will incorrectly use
                                         the braces from the non-f-string)
    f"{a}" f"{a}" --> "{}".format(a) "{}".format(a) (cannot concatenate)

- PEP-498 requires the full parser to understand the interpolated
  argument, however because this entirely runs in the lexer it cannot
  resolve nested braces in expressions like
    f"{'}'}"

- The !r, !s, and !a conversions are not supported.

Includes tests and cpydiffs.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-14 16:58:40 +10:00
..
argcheck.c py: Add option to compile without any error messages at all. 2021-04-27 23:51:52 +10:00
asmarm.c py/emitglue: Always flush caches when assigning native ARM code. 2021-06-05 11:03:04 +10:00
asmarm.h py/emitglue: Always flush caches when assigning native ARM code. 2021-06-05 11:03:04 +10:00
asmbase.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
asmbase.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
asmthumb.c py/emitglue: Always flush caches when assigning native ARM code. 2021-06-05 11:03:04 +10:00
asmthumb.h py/emitglue: Always flush caches when assigning native ARM code. 2021-06-05 11:03:04 +10:00
asmx64.c py/asmx64: Support use of top 8 regs in src_r64 argument. 2021-05-20 23:43:25 +10:00
asmx64.h py/asm: Add condition codes for signed comparisons. 2020-06-27 00:24:04 +10:00
asmx86.c py/asm: Add funcs/macros to emit machine code for logical-shift-right. 2020-06-27 00:24:04 +10:00
asmx86.h py/asm: Add condition codes for signed comparisons. 2020-06-27 00:24:04 +10:00
asmxtensa.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
asmxtensa.h py/asm: Add funcs/macros to emit machine code for logical-shift-right. 2020-06-27 00:24:04 +10:00
bc0.h py/bc0.h: Shift comment to start of line to improve format consistency. 2020-02-28 10:29:32 +11:00
bc.c py: Add option to compile without any error messages at all. 2021-04-27 23:51:52 +10:00
bc.h py: Mark unused arguments from bytecode decoding macros. 2021-06-25 10:58:22 +10:00
binary.c all: Fix signed shifts and NULL access errors from -fsanitize=undefined. 2021-06-24 23:01:04 +10:00
binary.h py/binary: Change mp_uint_t to size_t for index, size, align args. 2019-09-02 13:14:27 +10:00
builtin.h extmod/uasyncio: Add optional implementation of core uasyncio in C. 2020-03-26 01:25:45 +11:00
builtinevex.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
builtinhelp.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
builtinimport.c py/builtinimport: Fix condition for including do_execute_raw_code(). 2021-07-31 16:51:58 +10:00
compile.c py/compile: Raise an error on async with/for outside an async function. 2021-05-30 10:38:48 +10:00
compile.h py: Add global default_emit_opt variable to make emit kind persistent. 2019-08-28 12:47:58 +10:00
dynruntime.h py: Introduce and use mp_raise_type_arg helper. 2021-07-15 00:12:41 +10:00
dynruntime.mk py/dynruntime.mk: Set MICROPY_ENABLE_DYNRUNTIME instead of per module. 2020-04-05 14:13:02 +10:00
emit.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
emitbc.c all: Rename BYTES_PER_WORD to MP_BYTES_PER_OBJ_WORD. 2021-02-04 22:46:42 +11:00
emitcommon.c py/scope: Optimise scope_find_or_add_id to not need "added" arg. 2018-10-28 00:38:18 +11:00
emitglue.c py/emitglue: Always flush caches when assigning native ARM code. 2021-06-05 11:03:04 +10:00
emitglue.h py: Integrate sys.settrace feature into the VM and runtime. 2019-08-30 16:44:12 +10:00
emitinlinethumb.c py/emitinlinethumb: Exclude code using #if when ARMV7M disabled. 2021-01-29 23:57:10 +11:00
emitinlinextensa.c py: Use unsigned comparison of chars. 2020-10-22 11:47:36 +02:00
emitnarm.c py/emitnative: Add support for return/break/continue in try and with. 2018-09-04 14:31:28 +10:00
emitnative.c py/emitnative: Ensure stack settling is safe mid-branch. 2021-07-19 23:18:59 +10:00
emitnthumb.c py/emitnative: Add support for return/break/continue in try and with. 2018-09-04 14:31:28 +10:00
emitnx64.c py/emitnative: Add support for return/break/continue in try and with. 2018-09-04 14:31:28 +10:00
emitnx86.c py/nativeglue: Add new header file with native function table typedef. 2019-12-12 20:15:28 +11:00
emitnxtensa.c py/emitnative: Add support for return/break/continue in try and with. 2018-09-04 14:31:28 +10:00
emitnxtensawin.c py: Add new Xtensa-Windowed arch for native emitter. 2019-10-05 13:44:53 +10:00
formatfloat.c all: Fix implicit conversion from double to float. 2020-04-18 22:42:24 +10:00
formatfloat.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
frozenmod.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
frozenmod.h various: Update early copyright years to match actual edit history. 2019-05-17 18:06:11 +10:00
gc.c py/gc: Only use no_sanitize_address attribute for GCC 4.8 and above. 2021-06-18 14:15:37 +10:00
gc.h py/gc: Change include of stdint.h to stddef.h. 2021-02-05 15:46:56 +11:00
grammar.h py/compile: Implement PEP 526, syntax for variable annotations. 2020-06-16 23:18:01 +10:00
lexer.c py: Implement partial PEP-498 (f-string) support. 2021-08-14 16:58:40 +10:00
lexer.h py: Implement partial PEP-498 (f-string) support. 2021-08-14 16:58:40 +10:00
makecompresseddata.py py/makecompresseddata.py: Make compression deterministic. 2020-04-20 10:32:49 +10:00
makemoduledefs.py all: Update Python code to conform to latest black formatting. 2020-08-29 15:18:01 +10:00
makeqstrdata.py all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
makeqstrdefs.py py/makeqstrdefs.py: Don't include .h files explicitly in preprocessing. 2021-06-25 10:50:54 +10:00
makeversionhdr.py py/makeversionhdr: Add --tags arg to git describe. 2021-07-05 10:41:31 -05:00
malloc.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
map.c py/objdict: Add mp_const_empty_dict_obj, use it for mp_const_empty_map. 2020-10-10 00:16:26 +11:00
misc.h py: Add option to compile without any error messages at all. 2021-04-27 23:51:52 +10:00
mkenv.mk py/mkrules: Automatically build mpy-cross if it doesn't exist. 2021-08-07 20:25:32 +10:00
mkrules.cmake py/mkrules: Automatically build mpy-cross if it doesn't exist. 2021-08-07 20:25:32 +10:00
mkrules.mk py/mkrules: Automatically build mpy-cross if it doesn't exist. 2021-08-07 20:25:32 +10:00
modarray.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
modbuiltins.c py: Support single argument to optimised MP_OBJ_STOP_ITERATION. 2021-07-15 00:12:41 +10:00
modcmath.c all: Format code to add space after C++-style comment start. 2020-04-23 11:24:25 +10:00
modcollections.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
modgc.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
modio.c py: Rename remaining object types to be of the form mp_type_xxx. 2021-03-26 13:48:34 +11:00
modmath.c py/modmath: Simplify handling of positional args to reduce code size. 2020-12-14 13:30:56 +11:00
modmicropython.c py/gc: Make gc_lock_depth have a count per thread. 2021-05-10 13:07:16 +10:00
modstruct.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
modsys.c py/modsys: Optimise sys.exit for code size by using exception helpers. 2021-07-15 00:12:41 +10:00
modthread.c py/mpstate: Make exceptions thread-local. 2021-06-19 09:43:44 +10:00
moduerrno.c all: Format code to add space after C++-style comment start. 2020-04-23 11:24:25 +10:00
mpconfig.h py: Implement partial PEP-498 (f-string) support. 2021-08-14 16:58:40 +10:00
mperrno.h py/mperrno: Add MP_ECANCELED error code. 2021-06-24 23:14:01 +10:00
mphal.h all: Rename absolute time-based functions to include "epoch". 2020-09-18 17:20:34 +10:00
mpprint.c py/mpprint: Fix length calculation for strings with precision-modifier. 2020-12-07 23:32:06 +11:00
mpprint.h extmod/modujson: Add support for dump/dumps separators keyword-argument. 2021-08-07 13:52:16 +10:00
mpstate.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
mpstate.h py: Support single argument to optimised MP_OBJ_STOP_ITERATION. 2021-07-15 00:12:41 +10:00
mpthread.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
mpz.c all: Fix signed shifts and NULL access errors from -fsanitize=undefined. 2021-06-24 23:01:04 +10:00
mpz.h tools/codeformat.py: Eliminate need for sizeof fixup. 2020-03-11 14:34:40 +11:00
nativeglue.c all: Fix implicit floating point promotion. 2020-04-18 22:36:14 +10:00
nativeglue.h py/nativeglue.h: Rename "setjmp" entry to "setjmp_" to avoid any clash. 2020-05-14 21:48:05 +10:00
nlr.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
nlr.h py/nlr: Implement NLR for AArch64. 2021-03-11 12:51:10 +11:00
nlraarch64.c py/nlraarch64: Add underscore prefix to function symbols for Darwin ABI. 2021-05-18 11:46:30 +10:00
nlrpowerpc.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
nlrsetjmp.c py/nlr: Factor out common NLR code to macro and generic funcs in nlr.c. 2017-12-28 16:46:30 +11:00
nlrthumb.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
nlrx64.c py/nlrx64: Correct the detection of Darwin ABI. 2021-05-18 11:52:00 +10:00
nlrx86.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
nlrxtensa.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
obj.c py: Add option to compile without any error messages at all. 2021-04-27 23:51:52 +10:00
obj.h py/obj: Fix formatting of comment for mp_obj_is_integer. 2021-07-15 00:12:41 +10:00
objarray.c py/objarray: Fix constructing a memoryview from a memoryview. 2021-05-18 10:18:56 +10:00
objarray.h py/objarray.h: Add mp_obj_memoryview_init() helper function. 2020-09-25 12:23:11 +10:00
objattrtuple.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objbool.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objboundmeth.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objcell.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objclosure.c py: Rename remaining object types to be of the form mp_type_xxx. 2021-03-26 13:48:34 +11:00
objcomplex.c py/objcomplex: Add mp_obj_get_complex_maybe for use in complex bin-op. 2020-06-27 01:03:10 +10:00
objdeque.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
objdict.c extmod/modujson: Add support for dump/dumps separators keyword-argument. 2021-08-07 13:52:16 +10:00
objenumerate.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objexcept.c py/objexcept: Make mp_obj_new_exception_arg1 inline. 2021-07-15 00:12:41 +10:00
objexcept.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objfilter.c py: Downcase all MP_OBJ_IS_xxx macros to make a more consistent C API. 2019-02-12 14:54:51 +11:00
objfloat.c py: Rename BITS_PER_BYTE to MP_BITS_PER_BYTE. 2021-02-04 22:46:42 +11:00
objfun.c py: Mark unused arguments from bytecode decoding macros. 2021-06-25 10:58:22 +10:00
objfun.h py: Integrate sys.settrace feature into the VM and runtime. 2019-08-30 16:44:12 +10:00
objgenerator.c py: Support single argument to optimised MP_OBJ_STOP_ITERATION. 2021-07-15 00:12:41 +10:00
objgenerator.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
objgetitemiter.c py: Support single argument to optimised MP_OBJ_STOP_ITERATION. 2021-07-15 00:12:41 +10:00
objint_longlong.c py/modsys: Use consistent naming pattern for module-level const objects. 2020-05-28 10:02:14 +10:00
objint_mpz.c py/mpz: Do sign extension in mpz_as_bytes for negative values. 2020-11-11 22:18:24 +11:00
objint.c all: Fix signed shifts and NULL access errors from -fsanitize=undefined. 2021-06-24 23:01:04 +10:00
objint.h py/modsys: Use consistent naming pattern for module-level const objects. 2020-05-28 10:02:14 +10:00
objlist.c extmod/modujson: Add support for dump/dumps separators keyword-argument. 2021-08-07 13:52:16 +10:00
objlist.h py/obj.h: Move declaration of mp_obj_list_init to objlist.h. 2018-03-13 14:03:15 +11:00
objmap.c py: Downcase all MP_OBJ_IS_xxx macros to make a more consistent C API. 2019-02-12 14:54:51 +11:00
objmodule.c all: Rename "sys" module to "usys". 2020-09-04 00:10:24 +10:00
objmodule.h py/objmodule.h: Remove obsolete mp_builtin_module_weak_links_map decl. 2020-02-11 15:43:13 +11:00
objnamedtuple.c py: Add option to compile without any error messages at all. 2021-04-27 23:51:52 +10:00
objnamedtuple.h py/objnamedtuple: Allow to reuse namedtuple basic functionality. 2017-11-20 09:30:06 +02:00
objnone.c py/obj: Add MICROPY_OBJ_IMMEDIATE_OBJS option to reduce code size. 2020-01-13 01:01:45 +11:00
objobject.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
objpolyiter.c all: Remove inclusion of internal py header files. 2017-10-04 12:37:50 +11:00
objproperty.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objrange.c py: Rename remaining object types to be of the form mp_type_xxx. 2021-03-26 13:48:34 +11:00
objreversed.c py: Downcase all MP_OBJ_IS_xxx macros to make a more consistent C API. 2019-02-12 14:54:51 +11:00
objset.c py: Introduce and use mp_raise_type_arg helper. 2021-07-15 00:12:41 +10:00
objsingleton.c py/objsingleton: Use mp_generic_unary_op for singleton objects. 2019-12-27 12:53:36 +11:00
objslice.c all: Clean up error strings to use lowercase and change cannot to can't. 2020-04-13 22:19:37 +10:00
objstr.c py: Introduce and use mp_raise_type_arg helper. 2021-07-15 00:12:41 +10:00
objstr.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objstringio.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
objstringio.h py/objstringio: If created from immutable object, follow copy on write policy. 2017-06-09 17:33:01 +03:00
objstrunicode.c py/mpprint: Fix length calculation for strings with precision-modifier. 2020-12-07 23:32:06 +11:00
objtuple.c extmod/modujson: Add support for dump/dumps separators keyword-argument. 2021-08-07 13:52:16 +10:00
objtuple.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
objtype.c py: Add option to compile without any error messages at all. 2021-04-27 23:51:52 +10:00
objtype.h py/runtime: Don't allocate iter buf for user-defined types. 2019-12-27 12:34:22 +11:00
objzip.c py: Downcase all MP_OBJ_IS_xxx macros to make a more consistent C API. 2019-02-12 14:54:51 +11:00
opmethods.c py: Make mp_obj_get_type() return a const ptr to mp_obj_type_t. 2020-01-09 11:25:26 +11:00
pairheap.c py/pairheap: Properly unlink node on pop and delete. 2020-03-26 01:21:04 +11:00
pairheap.h py/pairheap: Add helper function to initialise a new node. 2020-03-26 01:21:04 +11:00
parse.c py: Implement partial PEP-498 (f-string) support. 2021-08-14 16:58:40 +10:00
parse.h py/parse: Pass in an mp_print_t to mp_parse_node_print. 2020-09-11 23:00:03 +10:00
parsenum.c py: Add option to compile without any error messages at all. 2021-04-27 23:51:52 +10:00
parsenum.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
parsenumbase.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
parsenumbase.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
persistentcode.c all: Rename BYTES_PER_WORD to MP_BYTES_PER_OBJ_WORD. 2021-02-04 22:46:42 +11:00
persistentcode.h py/persistentcode: Make ARM Thumb archs support multiple sub-archs. 2019-12-12 20:15:28 +11:00
profile.c py/mpstate: Make exceptions thread-local. 2021-06-19 09:43:44 +10:00
profile.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
py.cmake py/py.cmake: Introduce MICROPY_INC_CORE as a list with core includes. 2021-04-09 13:08:35 +10:00
py.mk all: Update to point to files in new shared/ directory. 2021-07-12 17:08:10 +10:00
pystack.c py: Introduce and use mp_raise_type_arg helper. 2021-07-15 00:12:41 +10:00
pystack.h all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
qstr.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
qstr.h py/qstr.h: Remove QSTR_FROM_STR_STATIC macro. 2021-01-30 13:40:48 +11:00
qstrdefs.h all: Add *FORMAT-OFF* in various places. 2020-02-28 10:31:07 +11:00
reader.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
reader.h py: Allow lexer to raise exceptions during construction. 2017-03-14 11:52:05 +11:00
repl.c py/repl: Don't read past the end of import_str. 2021-05-30 11:50:51 +10:00
repl.h all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
ringbuf.c py/ringbuf: Add peek16 method. 2019-11-21 12:04:53 +11:00
ringbuf.h extmod/modbluetooth: Make modbluetooth event not a bitfield. 2020-06-05 14:04:20 +10:00
runtime0.h py/nativeglue: Add new header file with native function table typedef. 2019-12-12 20:15:28 +11:00
runtime_utils.c py: mp_call_function_*_protected(): Pass-thru return value if possible. 2017-12-05 00:38:41 +02:00
runtime.c py/runtime: Fix bool unary op for subclasses of native types. 2021-07-23 12:40:00 +10:00
runtime.h py: Introduce and use mp_raise_type_arg helper. 2021-07-15 00:12:41 +10:00
scheduler.c py/mpstate: Schedule KeyboardInterrupt on main thread. 2021-06-19 09:49:00 +10:00
scope.c py/scope: Name and use id_kind_type_t. 2020-10-22 11:40:56 +02:00
scope.h py/scope: Name and use id_kind_type_t. 2020-10-22 11:40:56 +02:00
sequence.c all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
showbc.c all: Fix signed shifts and NULL access errors from -fsanitize=undefined. 2021-06-24 23:01:04 +10:00
smallint.c all: Use the name MicroPython consistently in comments 2017-07-31 18:35:40 +10:00
smallint.h all: Fix signed shifts and NULL access errors from -fsanitize=undefined. 2021-06-24 23:01:04 +10:00
stackctrl.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
stackctrl.h py/stackctrl: Prevent unused-var warning when stack checking disabled. 2021-06-05 11:03:09 +10:00
stream.c py/stream: Remove mp_stream_errno and use system errno instead. 2020-04-27 23:58:46 +10:00
stream.h py/stream.h: Include sys/types.h to get size_t and off_t for POSIX API. 2020-03-25 01:00:52 +11:00
unicode.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
unicode.h py/objstr: Add check for valid UTF-8 when making a str from bytes. 2017-09-06 16:43:09 +10:00
usermod.cmake rp2: Add support for USER_C_MODULES to CMake build system. 2021-03-31 00:26:01 +11:00
vm.c py/vm: Simplify handling of MP_OBJ_STOP_ITERATION in yield-from opcode. 2021-07-15 00:12:41 +10:00
vmentrytable.h py/vmentrytable: Ignore GCC -Woverride-init. 2020-10-22 11:47:36 +02:00
vstr.c all: Reformat C and Python source code with tools/codeformat.py. 2020-02-28 10:33:03 +11:00
warning.c py: Update my copyright info on some files. 2019-02-06 00:19:00 +11:00