Commit Graph

390 Commits

Author SHA1 Message Date
Scott Shawcroft 3fda0c0a1b
Fix board builds and use MP_ERROR_TEXT in py and extmod 2021-05-05 17:51:52 -07:00
Jeff Epler 68d4682298 compile: Reserve labels for native await 2021-05-05 11:00:39 -05:00
Scott Shawcroft f0bb26d70f
Merge MicroPython 1.13 into CircuitPython 2021-05-04 18:06:33 -07:00
Scott Shawcroft b35fa44c8a
Merge MicroPython 1.12 into CircuitPython 2021-05-03 14:01:18 -07:00
Scott Shawcroft 76033d5115
Merge MicroPython v1.11 into CircuitPython 2021-04-26 15:47:41 -07:00
Scott Shawcroft 09f7b43c64
Merge MicroPython 1.10 into CircuitPython 2021-04-21 15:59:17 -07:00
Scott Shawcroft b057fb8a4b
codeformat 2021-04-19 22:22:44 -07:00
Hugo Dahl 36f3897fe3 Make error messages platform agnostic
Remove mentions of 'MicroPython' in error messages as they could lead to confusion in lesser-experienced users
2021-04-10 18:36:53 -05:00
microDev a52eb88031
run code formatting script 2021-03-15 19:27:36 +05:30
Jeff Epler b2b8520880 Always use preprocessor for MICROPY_ERROR_REPORTING
This ensures that only the translate("") alternative that will be used
is seen after preprocessing.  Improves the quality of the Huffman encoding
and reduces binary size slightly.

Also makes one "enhanced" error message only occur when ERROR_REPORTING_DETAILED:
Instead of the word-for-word python3 error message
"Type object has no attribute '%q'", the message will be
"'type' object has no attribute '%q'".  Also reduces binary size.
(that's rolled into this commit as it was right next to a change to
use the preprocessor for MICROPY_ERROR_REPORTING)

Note that the odd semicolon after "value_error:" in parsenum.c is necessary
due to a detail of the C grammar, in which a declaration cannot follow
a label directly.
2020-11-19 16:18:52 -06:00
Kenny 94beeabc51 remove unnecessary board configuration and address feedback 2020-10-11 22:42:59 -07:00
Kenny 98aa4b7943 update async tests with less upython workaround and more cpython compatibility 2020-10-10 23:39:32 -07:00
Kenny 5d96afc5c2 i do not know if this is needed but this is not the vm i use anymore 2020-10-10 15:45:08 -07:00
Kenny bf849ff674 async def syntax rigor and __await__ magic method
Some examples of improved compliance with CPython that currently
have divergent behavior in CircuitPython are listed below:

* yield from is not allowed in async methods
```
>>> async def f():
...     yield from 'abc'
...
Traceback (most recent call last):
  File "<stdin>", line 2, in f
SyntaxError: 'yield from' inside async function
```

* await only works on awaitable expressions
```
>>> async def f():
...     await 'not awaitable'
...
>>> f().send(None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in f
AttributeError: 'str' object has no attribute '__await__'
```

* only __await__()able expressions are awaitable
Okay this one actually does not work in circuitpython at all today.
This is how CPython works though and pretending __await__ does not
exist will only bite users who write both.
```
>>> class c:
...     pass
...
>>> def f(self):
...     yield
...     yield
...     return 'f to pay respects'
...
>>> c.__await__ = f  # could just as easily have put it on the class but this shows how it's wired
>>> async def g():
...     awaitable_thing = c()
...     partial = await awaitable_thing
...     return 'press ' + partial
...
>>> q = g()
>>> q.send(None)
>>> q.send(None)
>>> q.send(None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration: press f to pay respects
```
2020-10-10 15:45:08 -07:00
Scott Shawcroft 61d1148bb3
Merge pull request #3222 from WarriorOfWire/pick_micropython
py/compile: Don't await __aiter__ special method in async-for.
2020-07-29 10:54:37 -07:00
Jonathan Hogg 901f3dce6e py/compile: Don't await __aiter__ special method in async-for.
MicroPython's original implementation of __aiter__ was correct for an
earlier (provisional) version of PEP492 (CPython 3.5), where __aiter__ was
an async-def function.  But that changed in the final version of PEP492 (in
CPython 3.5.2) where the function was changed to a normal one.  See
https://www.python.org/dev/peps/pep-0492/#why-aiter-does-not-return-an-awaitable
See also the note at the end of this subsection in the docs:
https://docs.python.org/3.5/reference/datamodel.html#asynchronous-iterators
And for completeness the BPO: https://bugs.python.org/issue27243

To be consistent with the Python spec as it stands today (and now that
PEP492 is final) this commit changes MicroPython's behaviour to match
CPython:  __aiter__ should return an async-iterable object, but is not
itself awaitable.

The relevant tests are updated to match.

See #6267.
2020-07-24 22:55:37 -07:00
Kenny 9a1f1236cc require async for and async with to actually be in an async def method instead of just a generator 2020-07-24 19:47:34 -07:00
Jonathan Hogg 37e1b5c891 py/compile: Don't await __aiter__ special method in async-for.
MicroPython's original implementation of __aiter__ was correct for an
earlier (provisional) version of PEP492 (CPython 3.5), where __aiter__ was
an async-def function.  But that changed in the final version of PEP492 (in
CPython 3.5.2) where the function was changed to a normal one.  See
https://www.python.org/dev/peps/pep-0492/#why-aiter-does-not-return-an-awaitable
See also the note at the end of this subsection in the docs:
https://docs.python.org/3.5/reference/datamodel.html#asynchronous-iterators
And for completeness the BPO: https://bugs.python.org/issue27243

To be consistent with the Python spec as it stands today (and now that
PEP492 is final) this commit changes MicroPython's behaviour to match
CPython:  __aiter__ should return an async-iterable object, but is not
itself awaitable.

The relevant tests are updated to match.

See #6267.
2020-07-25 00:58:18 +10:00
Kenny 51a79b1af7 add coroutine behavior for generators
coroutines don't have __next__; they also call themselves coroutines.
This does not change the fact that `async def` methods are generators,
but it does make them behave more like CPython.
2020-07-23 20:40:16 -07:00
Diego Elio Pettenò 34b4993d63 Add license to some obvious files. 2020-07-06 19:16:25 +01:00
Damien George f2e267da68 py/compile: Implement PEP 526, syntax for variable annotations.
This addition to the grammar was introduced in Python 3.6.  It allows
annotating the type of a varilable, like:

    x: int = 123
    s: str

The implementation in this commit is quite simple and just ignores the
annotation (the int and str bits above).  The reason to implement this is
to allow Python 3.6+ code that uses this feature to compile under
MicroPython without change, and for users to use type checkers.

In the future viper could use this syntax as a way to give types to
variables, which is currently done in a bit of an ad-hoc way, eg
x = int(123).  And this syntax could potentially be used in the inline
assembler to define labels in an way that's easier to read.
2020-06-16 23:18:01 +10:00
Damien George 131b0de70a py/grammar.h: Consolidate duplicate sub-rules for :test and =test. 2020-06-16 23:18:01 +10:00
Damien George 1783950311 py/compile: Implement PEP 572, assignment expressions with := operator.
The syntax matches CPython and the semantics are equivalent except that,
unlike CPython, MicroPython allows using := to assign to comprehension
iteration variables, because disallowing this would take a lot of code to
check for it.

The new compile-time option MICROPY_PY_ASSIGN_EXPR selects this feature and
is enabled by default, following MICROPY_PY_ASYNC_AWAIT.
2020-06-16 22:02:24 +10:00
Damien George 0fd91e39b1 py/compile: Convert scope test to SCOPE_IS_COMP_LIKE macro.
This macro can be used elsewhere.
2020-06-16 21:42:37 +10:00
Damien George 172fc040aa py/parse: Make mp_parse_node_extract_list return size_t instead of int.
Because this function can only return non-negative values, and having the
correct return type gives more information to the caller.
2020-05-09 00:55:44 +10:00
stijn 84fa3312cf all: Format code to add space after C++-style comment start.
Note: the uncrustify configuration is explicitly set to 'add' instead of
'force' in order not to alter the comments which use extra spaces after //
as a means of indenting text for clarity.
2020-04-23 11:24:25 +10:00
Jim Mussared def76fe4d9 all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
Jim Mussared 85858e72df py/objexcept: Allow compression of exception message text.
The decompression of error-strings is only done if the string is accessed
via printing or via er.args.  Tests are added for this feature to ensure
the decompression works.
2020-04-05 15:02:06 +10:00
Jim Mussared a9a745e4b4 py: Use preprocessor to detect error reporting level (terse/detailed).
Instead of compiler-level if-logic.  This is necessary to know what error
strings are included in the build at the preprocessor stage, so that string
compression can be implemented.
2020-04-05 14:11:51 +10:00
Jeff Epler 862830da32 compile: Give a proper error on 'async with'/'async for' outside 'async def'
A simple reproducer is:
   async for x in():x
2020-03-01 09:40:43 -06:00
Damien George 69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
Petr Viktorin e6c9800645 py/compile: Allow 'return' outside function in minimal builds.
A 'return' statement on module/class level is not correct Python, but
nothing terribly bad happens when it's allowed.  So remove the check unless
MICROPY_CPYTHON_COMPAT is on.

This is similar to MicroPython's treatment of 'import *' in functions
(except 'return' has unsurprising behavior if it's allowed).
2020-02-06 00:41:55 +11:00
Thea Flowers 84e1d7f304
Make the @micropython.native decorator no-op if support isn't enabled
When adding the ability for boards to turn on the `@micropython.native`, `viper`, and `asm_thumb` decorators it was pointed out that it's somewhat awkward to write libraries and drivers that can take advantage of this since the decorators raise `SyntaxErrors` if they aren't enabled. In the case of `viper` and `asm_thumb` this behavior makes sense as they require writing non-normative code. Drivers could have a normal and viper/thumb implementation and implement them as such:

```python
try:
    import _viper_impl as _impl
except SyntaxError:
    import _python_impl as _impl

def do_thing():
    return _impl.do_thing()
```

For `native`, however, this behavior and the pattern to work around it is less than ideal. Since `native` code should also be valid Python code (although not necessarily the other way around) using the pattern above means *duplicating* the Python implementation and adding `@micropython.native` in the code. This is an unnecessary maintenance burden.

This commit *modifies* the behavior of the `@micropython.native` decorator. On boards with `CIRCUITPY_ENABLE_MPY_NATIVE` turned on it operates as usual. On boards with it turned off it does *nothing*- it doesn't raise a `SyntaxError` and doesn't apply optimizations. This means we can write our drivers/libraries once and take advantage of speedups on boards where they are enabled.
2019-11-26 13:09:30 -08:00
Petr Viktorin 57c18fdd38 py/compile: Coalesce error message for break/continue outside loop.
To reduce code size.
2019-11-21 12:13:11 +11:00
Thea Flowers c7195c4bc5
Allow boards to enable the `micropython.native` decorator
Adds the `CIRCUITPY_ENABLE_MPY_NATIVE` for `mpconfigboard.mk` that enables
the `micropython.native` decorator.
2019-11-05 14:27:53 -08:00
Damien George 9adedce42e py: Add new Xtensa-Windowed arch for native emitter.
Enabled via the configuration MICROPY_EMIT_XTENSAWIN.
2019-10-05 13:44:53 +10:00
Petr Viktorin 25a9bccdee py/compile: Disallow 'import *' outside module level.
This check follows CPython's behaviour, because 'import *' always populates
the globals with the imported names, not locals.

Since it's safe to do this (doesn't lead to a crash or undefined behaviour)
the check is only enabled for MICROPY_CPYTHON_COMPAT.

Fixes issue #5121.
2019-10-04 16:46:47 +10:00
Josh Lloyd 7d58a197cf py: Rename MP_QSTR_NULL to MP_QSTRnull to avoid intern collisions.
Fixes #5140.
2019-09-26 16:04:56 +10:00
Damien George 14e203282a py/compile: Use calculation instead of switch to convert token to op. 2019-09-26 14:37:26 +10:00
Milan Rossa 310b3d1b81 py: Integrate sys.settrace feature into the VM and runtime.
This commit adds support for sys.settrace, allowing to install Python
handlers to trace execution of Python code.  The interface follows CPython
as closely as possible.  The feature is disabled by default and can be
enabled via MICROPY_PY_SYS_SETTRACE.
2019-08-30 16:44:12 +10:00
Damien George c7c6703950 py/compile: Improve the line numbering precision for lambdas.
Prior to this patch the line number for a lambda would be "line 1" if the
body of the lambda contained only a simple expression (with no line number
stored in the parse node).  Now the line number is always reported
correctly.
2019-08-30 16:43:46 +10:00
Damien George af20c2ead3 py: Add global default_emit_opt variable to make emit kind persistent.
mp_compile no longer takes an emit_opt argument, rather this setting is now
provided by the global default_emit_opt variable.

Now, when -X emit=native is passed as a command-line option, the emitter
will be set for all compiled modules (included imports), not just the
top-level script.

In the future there could be a way to also set this variable from a script.

Fixes issue #4267.
2019-08-28 12:47:58 +10:00
Milan Rossa ae6fe8b43c py/compile: Improve the line numbering precision for comprehensions.
The line number for comprehensions is now always reported as the correct
global location in the script, instead of just "line 1".
2019-08-19 23:50:30 +10:00
Jun Wu 089c9b71d1 py: remove "if (0)" and "if (false)" branches.
Prior to this commit, building the unix port with `DEBUG=1` and
`-finstrument-functions` the compilation would fail with an error like
"control reaches end of non-void function".  This change fixes this by
removing the problematic "if (0)" branches.  Not all branches affect
compilation, but they are all removed for consistency.
2019-05-06 18:28:28 +10:00
Damien George 9ef784dcc6 py/asmthumb: Support asm_thumb code running on normal ARM processors.
With this change, @micropython.asm_thumb functions will work on standard
ARM processors (that are in ARM state by default), in scripts and
precompiled .mpy files.

Addresses issue #4675.
2019-05-01 15:24:21 +10:00
Damien George 5a6026c614 py/compile: Check that arch is set when compiling native, viper or asm. 2019-03-14 12:22:25 +11:00
Damien George 55fcb83a42 py/compile: Support multiple inline asm emitters. 2019-03-14 12:22:25 +11:00
Damien George d9d92f27d7 py/compile: Add support to select the native emitter at runtime. 2019-03-14 12:22:25 +11:00
Damien George 1396a026be py: Add support to save native, viper and asm code to .mpy files.
This commit adds support for saving and loading .mpy files that contain
native code (native, viper and inline-asm).  A lot of the ground work was
already done for this in the form of removing pointers from generated
native code.  The changes here are mainly to link in qstr values to the
native code, and change the format of .mpy files to contain native code
blocks (possibly mixed with bytecode).

A top-level summary:

- @micropython.native, @micropython.viper and @micropython.asm_thumb/
  asm_xtensa are now allowed in .py files when compiling to .mpy, and they
  work transparently to the user.

- Entire .py files can be compiled to native via mpy-cross -X emit=native
  and for the most part the generated .mpy files should work the same as
  their bytecode version.

- The .mpy file format is changed to 1) specify in the header if the file
  contains native code and if so the architecture (eg x86, ARMV7M, Xtensa);
  2) for each function block the kind of code is specified (bytecode,
  native, viper, asm).

- When native code is loaded from a .mpy file the native code must be
  modified (in place) to link qstr values in, just like bytecode (see
  py/persistentcode.c:arch_link_qstr() function).

In addition, this now defines a public, native ABI for dynamically loadable
native code generated by other languages, like C.
2019-03-08 15:53:05 +11:00
Damien George 01a1f31f67 py/emitnative: Consolidate where HASCONSTS is set to load-const-obj fun.
Simplifies the code and fixes handling of the Ellipsis const in native code
generation (which also needs the constant table so must set this flag).
2019-03-08 15:53:04 +11:00