4509 Commits

Author SHA1 Message Date
Jeff Epler
32647cd9b4 lexer: catch concatenation of f'' and '' strings
This turns the "edge case" into a parse-time error.
2020-03-09 09:03:25 -05:00
Josh Klar
40bc05ee1e Address dpgeorge feedback - largely simplifications 2020-03-09 08:16:07 -05:00
Josh Klar
3a7a5ba686 py: Implement partial PEP-498 (f-string) support
This implements (most of) the PEP-498 spec for f-strings, with two
exceptions:

- raw f-strings (`fr` or `rf` prefixes) raise `NotImplementedError`
- one special corner case does not function as specified in the PEP
(more on that in a moment)

This is implemented in the core as a syntax translation, brute-forcing
all f-strings to run through `String.format`. For example, the statement
`x='world'; print(f'hello {x}')` gets translated *at a syntax level*
(injected into the lexer) to `x='world'; print('hello {}'.format(x))`.
While this may lead to weird column results in tracebacks, it seemed
like the fastest, most efficient, and *likely* most RAM-friendly option,
despite being implemented under the hood with a completely separate
`vstr_t`.

Since [string concatenation of adjacent literals is implemented in the
lexer](534b7c368d),
two side effects emerge:

- All strings with at least one f-string portion are concatenated into a
single literal which *must* be run through `String.format()` wholesale,
and:
- Concatenation of a raw string with interpolation characters with an
f-string will cause `IndexError`/`KeyError`, which is both different
from CPython *and* different from the corner case mentioned in the PEP
(which gave an example of the following:)

```python
x = 10
y = 'hi'
assert ('a' 'b' f'{x}' '{c}' f'str<{y:^4}>' 'd' 'e') == 'ab10{c}str< hi >de'
```

The above-linked commit detailed a pretty solid case for leaving string
concatenation in the lexer rather than putting it in the parser, and
undoing that decision would likely be disproportionately costly on
resources for the sake of a probably-low-impact corner case. An
alternative to become complaint with this corner case of the PEP would
be to revert to string concatenation in the parser *only when an
f-string is part of concatenation*, though I've done no investigation on
the difficulty or costs of doing this.

A decent set of tests is included. I've manually tested this on the
`unix` port on Linux and on a Feather M4 Express (`atmel-samd`) and
things seem sane.
2020-03-09 08:16:07 -05:00
Jeff Epler
4fa90261a3
Merge pull request #2657 from tannewt/builtin_package
Support importing native modules in native packages.
2020-03-05 14:28:46 -06:00
Jeff Epler
da31acfcc4 Merge remote-tracking branch 'origin/master' into ulab 2020-03-03 20:13:53 -06:00
Scott Shawcroft
274cb597b0
Remove debug extern 2020-03-03 10:55:50 -08:00
Jeff Epler
862830da32 compile: Give a proper error on 'async with'/'async for' outside 'async def'
A simple reproducer is:
   async for x in():x
2020-03-01 09:40:43 -06:00
Jeff Epler
511c180869 parse: push_result_token: throw an exception on too-long names
Before this, such names would instead cause an assertion error inside
qstr_from_strn.

A simple reproducer is a python source file containing the letter "a"
repeated 256 times
2020-03-01 09:38:34 -06:00
Damien George
1993c8cf9a py/builtinevex: Support passing in a bytearray/buffer to eval/exec.
CPython allows this and it's a simple generalisation of the existing code
which just supported str/bytes.

Fixes issue #5704.
2020-02-28 12:45:36 +11:00
Damien George
69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
Damien George
3f39d18c2b all: Add *FORMAT-OFF* in various places.
This string is recognised by uncrustify, to disable formatting in the
region marked by these comments.  This is necessary in the qstrdef*.h files
to prevent modification of the strings within the Q(...).  In other places
it is used to prevent excessive reformatting that would make the code less
readable.
2020-02-28 10:31:07 +11:00
Damien George
b86075ef1f py/parse: Add parenthesis around calculated bit-width in struct.
To improve interaction with uncrustify formatter.
2020-02-28 10:30:52 +11:00
Damien George
17aeb43e18 py: Un-nest configuration #if/#endif's for selection of complex code.
Because un-nested #if's are simpler to handle with formatting tools.
2020-02-28 10:30:34 +11:00
Damien George
7768a8c38d py/malloc: Put { on separate line for funcs that have selective sigs.
To ensure there are balanced {}'s in the file, and to help with formatting.
2020-02-28 10:30:29 +11:00
Damien George
a642241a12 py/builtinimport: Adjust if-block order in find_file to clean up #if's. 2020-02-28 10:30:20 +11:00
Damien George
2b50afaee4 py/bc0.h: Shift comment to start of line to improve format consistency. 2020-02-28 10:29:32 +11:00
Damien George
a1b18b3ba7 py: Removing dangling "else" to improve code format consistency. 2020-02-28 10:29:27 +11:00
Jeff Epler
39cfe32c34 Update ulab from upstream again 2020-02-27 14:14:05 -06:00
Jeff Epler
fa3b9eba92 ulab: Incorporate it 2020-02-27 11:03:03 -06:00
Scott Shawcroft
6375d8699e
Validate builtin member is a module 2020-02-26 10:32:02 -08:00
Scott Shawcroft
86fd93bd03
Support importing native modules in native packages.
This only fixes the `import` portion. It doesn't actually change
reference behavior because modules within a package could already
be referenced through the parent package even though an error should
have been thrown.
2020-02-25 15:32:55 -08:00
Jim Mussared
3ccce89b83 py/objarray: Turn on MP_TYPE_FLAG_EQ_CHECKS_OTHER_TYPE for memoryview.
And add corresponding tests.

Fixes #5674 (comparison of memoryview against bytes).
2020-02-21 14:24:07 +11:00
Dan Halbert
e31ac710be Enable _bleio adapter when _bleio is imported 2020-02-20 21:55:04 -05:00
Damien George
f6d99bc795 py/dynruntime.h: Add implementation of mp_obj_cast_to_native_base. 2020-02-21 13:32:48 +11:00
Damien George
9344e876bb py/objtype: Allow mp_instance_cast_to_native_base to take native obj.
And rename it to mp_obj_cast_to_native_base() to indicate this.  This
allows users of this function to easily support native and native-subclass
objects in the same way (by just passing the object through this function).
2020-02-21 13:20:12 +11:00
Scott Shawcroft
e8f7141564
Disable BLE file service for now
Fixes #2633
2020-02-20 12:15:56 -08:00
Damien George
d3b2c6e44c py/objtuple: Remove code that handles tuple-subclass equality test.
Since commit 3aab54bf434e7f025a91ea05052f1bac439fad8c this piece of code is
no longer needed because the top-level function mp_obj_equal_not_equal()
now handles the case of user types, and will never call tuple's binary_op
function with MP_BINARY_OP_EQUAL and a non-tuple on the RHS.
2020-02-20 10:48:03 +11:00
Damien George
1fccda049f py/objexcept: Rename mp_obj_new_exception_msg_varg2 to ..._vlist.
Follow up to recent commit ad7213d3c31bccb26a3f54f7492ccf4b0cc920f3, the
name "varg2" is misleading, vlist describes better that the argument is a
va_list.  This name also matches CircuitPython, which already has such
helper functions.
2020-02-18 21:00:42 +11:00
Damien George
ce39c958ef py: Factor out definition of mp_float_union_t to one location. 2020-02-18 13:04:36 +11:00
Dan Halbert
c592bd612a Implement to_bytes(..., signed=True) 2020-02-14 15:12:20 -05:00
Scott Shawcroft
e97b0cfc61
Merge pull request #2581 from jamesbowman/master
First draft of eveL, the low-level module of the Gameduino bindings
2020-02-13 11:21:32 -08:00
David Lechner
f020eac6a8 py/obj.h: Remove TODO idea comment about truncated mp_map_t.
It was suggested to move this to a GitHub issue rather than keep it in the
code, which isn't really sustainable for all ideas.
2020-02-13 13:14:17 +11:00
Damien George
ad7213d3c3 py: Add mp_raise_msg_varg helper and use it where appropriate.
This commit adds mp_raise_msg_varg(type, fmt, ...) as a helper for
nlr_raise(mp_obj_new_exception_msg_varg(type, fmt, ...)).  It makes the
C-level API for raising exceptions more consistent, and reduces code size
on most ports:

   bare-arm:   +28 +0.042%
minimal x86:  +100 +0.067%
   unix x64:   -56 -0.011%
unix nanbox:  -300 -0.068%
      stm32:  -204 -0.054% PYBV10
     cc3200:    +0 +0.000%
    esp8266:   -64 -0.010% GENERIC
      esp32:  -104 -0.007% GENERIC
        nrf:  -136 -0.094% pca10040
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2020-02-13 11:52:40 +11:00
Damien George
97eca38c4f py: Add mp_raise_type helper macro and use it where appropriate.
This provides a more consistent C-level API to raise exceptions, ie moving
away from nlr_raise towards mp_raise_XXX.  It also reduces code size by a
small amount on some ports.
2020-02-13 11:03:37 +11:00
Scott Shawcroft
36e6cc8feb
Track first free atb for multiple block sizes.
This speeds up cases where no single block allocations happen while
Python code is allocating other block sizes.
2020-02-11 17:06:24 -08:00
Damien George
7679e3be96 py/objmodule.h: Remove obsolete mp_builtin_module_weak_links_map decl.
It was made obsolete in d2384efa809953152c57cbda4c339dfbaa64cf29
2020-02-11 15:43:13 +11:00
Damien George
9ec1caf42e py: Expand type equality flags to 3 separate ones, fix bool/namedtuple.
Both bool and namedtuple will check against other types for equality; int,
float and complex for bool, and tuple for namedtuple.  So to make them work
after the recent commit 3aab54bf434e7f025a91ea05052f1bac439fad8c they would
need MP_TYPE_FLAG_NEEDS_FULL_EQ_TEST set.  But that makes all bool and
namedtuple equality checks less efficient because mp_obj_equal_not_equal()
could no longer short-cut x==x, and would need to try __ne__.  To improve
this, this commit splits the MP_TYPE_FLAG_NEEDS_FULL_EQ_TEST flags into 3
separate flags to give types more fine-grained control over how their
equality behaves.  These new flags are then used to fix bool and namedtuple
equality.

Fixes issue #5615 and #5620.
2020-02-11 11:06:00 +11:00
James Bowman
5c6d94d3e5 Split into shared-module and shared-bindings 2020-02-07 10:30:49 -08:00
Damien George
abe2caf6df py/scheduler: Move clearing of kbd traceback to mp_keyboard_interrupt.
This is a more logical place to clear the KeyboardInterrupt traceback,
right before it is set as a pending exception.  The clearing is also
optimised from a function call to a simple store of NULL.
2020-02-07 16:08:31 +11:00
Damien George
8fb5c8fdd5 py/scheduler: Allow a port to specify attrs for mp_keyboard_interrupt.
Functions like mp_keyboard_interrupt() may need to be called from an IRQ
handler and may need to be in a special memory section, so provide a
generic wrapping macro for a port to do this.  The macro name is chosen to
be MICROPY_WRAP_<function name in uppercase> so that (in the future with
more wrappers) each function could potentially be handled separately.
2020-02-07 16:08:29 +11:00
Damien George
9efb36bfa6 py/scheduler: Move mp_keyboard_interrupt from lib/utils to py core.
This function is tightly coupled to the state and behaviour of the
scheduler, and is a core part of the runtime: to schedule a pending
exception.  So move it there.
2020-02-07 16:08:26 +11:00
Damien George
98a3911c43 py/scheduler: Add "raise_exc" argument to mp_handle_pending.
Previous behaviour is when this argument is set to "true", in which case
the function will raise any pending exception.  Setting it to "false" will
cancel any pending exception.
2020-02-07 16:08:20 +11:00
James Bowman
acef93a253 Rename eveL to _eve, EVEL to _EVE 2020-02-05 18:17:58 -08:00
Petr Viktorin
e6c9800645 py/compile: Allow 'return' outside function in minimal builds.
A 'return' statement on module/class level is not correct Python, but
nothing terribly bad happens when it's allowed.  So remove the check unless
MICROPY_CPYTHON_COMPAT is on.

This is similar to MicroPython's treatment of 'import *' in functions
(except 'return' has unsurprising behavior if it's allowed).
2020-02-06 00:41:55 +11:00
Dan Halbert
e13642351c fix printing single-arg exceptions 2020-02-04 16:55:56 -05:00
Dan Halbert
7fe959ab39 fix multi-arg exception printing 2020-02-04 16:41:25 -05:00
Dan Halbert
5164719e67 preserve errno in OSError 2020-02-04 16:19:40 -05:00
Dan Halbert
57b566e736 Include filename for 'No such file/directory', etc. 2020-02-04 13:32:39 -05:00
James Bowman
8347f2b5c3 Correct default state to off for eveL module.
Fix build break because of overflowing small ports
2020-02-03 19:23:42 -08:00
James Bowman
7fd30e7d20 First draft of eveL, the low-level module of the Gameduino (and BridgeTek EVE) bindings.
[adafruit/circuitpython#2578]
2020-02-03 16:46:14 -08:00