Commit Graph

44 Commits

Author SHA1 Message Date
David Lechner
783b1a868f py/runtime: Allow multiple *args in a function call.
This is a partial implementation of PEP 448 to allow unpacking multiple
star args in a function or method call.

This is implemented by changing the emitted bytecodes so that both
positional args and star args are stored as positional args.  A bitmap is
added to indicate if an argument at a given position is a positional
argument or a star arg.

In the generated code, this new bitmap takes the place of the old star arg.
It is stored as a small int, so this means only the first N arguments can
be star args where N is the number of bits in a small int.

The runtime is modified to interpret this new bytecode format while still
trying to perform as few memory reallocations as possible.

Signed-off-by: David Lechner <david@pybricks.com>
2022-03-31 16:59:30 +11:00
Damien George
ab2923dfa1 all: Update Python formatting to latest Black version 22.1.0.
Signed-off-by: Damien George <damien@micropython.org>
2022-02-02 16:49:55 +11:00
Jim Mussared
e99f7b6d25 tests/cpydiff: Clarify f-string diffs regarding concatenation.
Concatenation of any literals (including f-strings) should be avoided.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-11-25 22:09:59 +11:00
Damien George
11ed94797d py/lexer: Support nested [] and {} characters within f-string params.
Signed-off-by: Damien George <damien@micropython.org>
2021-11-25 21:50:58 +11:00
Jim Mussared
692d36d779 py: Implement partial PEP-498 (f-string) support.
This implements (most of) the PEP-498 spec for f-strings and is based on
https://github.com/micropython/micropython/pull/4998 by @klardotsh.

It is implemented in the lexer as a syntax translation to `str.format`:
  f"{a}" --> "{}".format(a)

It also supports:
  f"{a=}" --> "a={}".format(a)

This is done by extracting the arguments into a temporary vstr buffer,
then after the string has been tokenized, the lexer input queue is saved
and the contents of the temporary vstr buffer are injected into the lexer
instead.

There are four main limitations:
- raw f-strings (`fr` or `rf` prefixes) are not supported and will raise
  `SyntaxError: raw f-strings are not supported`.

- literal concatenation of f-strings with adjacent strings will fail
    "{}" f"{a}" --> "{}{}".format(a)    (str.format will incorrectly use
                                         the braces from the non-f-string)
    f"{a}" f"{a}" --> "{}".format(a) "{}".format(a) (cannot concatenate)

- PEP-498 requires the full parser to understand the interpolated
  argument, however because this entirely runs in the lexer it cannot
  resolve nested braces in expressions like
    f"{'}'}"

- The !r, !s, and !a conversions are not supported.

Includes tests and cpydiffs.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-14 16:58:40 +10:00
David Lechner
cd506d6220 tests/cpydiff/modules_struct_whitespace_in_format: Run black.
This test snuck through without proper formatting and is causing CI for
other unrelated changes to fail.

Signed-off-by: David Lechner <david@pybricks.com>
2021-07-06 18:19:55 -05:00
Tom McDermott
c1f74b3005 docs/library: Warn that ustruct doesn't handle spaces in format strings.
And also add a test to capture the CPython difference.
2021-07-06 14:59:50 +10:00
Zoltán Vörös
c4ed17ff34 tests/cpydiff: Add test for array constructor with overflowing value. 2021-06-13 10:30:14 +10:00
Macarthur Inbody
34d4dab683 extmod/modurandom: Add error message when getrandbits has bad value.
The random module's getrandbits() method didn't give a proper error message
when calling it with a value that was outside of the range of 1-32, which
can lead to confusion using this function (which under CPython can accept
numbers larger than 32).  Now instead of simply giving a ValueError it
gives an error message that states that the number of bits is constrained.

Also, since the random module's functions getrandbits() and randint()
differ from CPython, tests have been added to describe these differences.
For getrandbits the relevant documentation is shown and added to the docs.
The same is given for randint method so that the information is more easily
found.

Finally, since the int object lacks the bit_length() method there is a test
for that method also to include within the docs, showing the difference to
CPython.
2021-05-30 16:41:30 +10:00
Damien George
47e6c52f0c tests/cpydiff: Add test and workaround for function.__module__ attr.
MicroPython does not store any reference from a function object to the
module it was defined in, but there is a way to use function.__globals__ to
indirectly get the module.

See issue #7259.

Signed-off-by: Damien George <damien@micropython.org>
2021-05-16 11:06:46 +10:00
stijn
57365d8557 py/objarray: Prohibit comparison of mismatching types.
Array equality is defined as each element being equal but to keep
code size down MicroPython implements a binary comparison.  This
can only be used correctly for elements with the same binary layout
though so turn it into an NotImplementedError when comparing types
for which the binary comparison yielded incorrect results: types
with different sizes, and floating point numbers because nan != nan.
2021-05-13 22:16:14 +10:00
Damien George
3c4bfd1dec py/objexcept: Support errno attribute on OSError exceptions.
This commit adds the errno attribute to exceptions, so code can retrieve
errno codes from an OSError using exc.errno.

The implementation here simply lets `errno` (and the existing `value`)
attributes work on any exception instance (they both alias args[0]).  This
is for efficiency and to keep code size down.  The pros and cons of this
are:

Pros:
- more compatible with CPython, less difference to document and learn
- OSError().errno will correctly return None, whereas the current way of
  doing it via OSError().args[0] will raise an IndexError
- it reduces code size on most bare-metal ports (because they already have
  the errno qstr)
- for Python code that uses exc.errno the generated bytecode is 2 bytes
  smaller and more efficient to execute (compared with exc.args[0]); so
  bytecode loaded to RAM saves 2 bytes RAM for each use of this attribute,
  and bytecode that is frozen saves 2 bytes flash/ROM for each use
- it's easier/shorter to type, and saves 2 bytes of space in .py files that
  use it (for each use)

Cons:
- increases code size by 4-8 bytes on minimal ports that don't already have
  the `errno` qstr
- all exceptions now have .errno and .value attributes (a cpydiff test is
  added to address this)

See also #2407.

Signed-off-by: Damien George <damien@micropython.org>
2021-04-23 22:03:46 +10:00
Damien George
a3c89cf907 tests/cpydiff: Add CPy diff test for assignment expression behaviour. 2020-06-16 22:06:47 +10:00
Jim Mussared
309c19d39b tests/cpydiff: Add cpydiff test for __all__ used in imported package. 2020-05-02 17:41:04 +10:00
David Lechner
3dc324d3f1 tests: Format all Python code with black, except tests in basics subdir.
This adds the Python files in the tests/ directory to be formatted with
./tools/codeformat.py.  The basics/ subdirectory is excluded for now so we
aren't changing too much at once.

In a few places `# fmt: off`/`# fmt: on` was used where the code had
special formatting for readability or where the test was actually testing
the specific formatting.
2020-03-30 13:21:58 +11:00
Damien George
69b415f745 tests: Move CPy diff test to real test now that subclass equality works.
Testing for equality of subclassed strings now works, thanks to commit
3aab54bf43
2020-02-04 18:25:34 +11:00
David Lechner
74106757ac tests/cpydiff: Add os module environ differences. 2020-02-04 17:54:31 +11:00
Damien George
4d528bbaa8 tests/cpydiff: Add CPy diff-test for using dict.keys() as a set.
See issue #5493.
2020-01-06 23:26:00 +11:00
clach04
ffd11486d4 tests/cpydiff: Fix typo in types_bytes_keywords.py doc comments. 2019-10-21 23:25:09 +11:00
Paul Sokolovsky
8181ec04a4 tests/cpydiff: Add case for difference in behaviour of bytes.format(). 2018-09-26 15:31:10 +10:00
Damien George
2ada1124d4 tests/cpydiff: Remove types_int_tobytesfloat now that it doesn't fail.
Commit e269cabe3e added a check that the
first argument to the to_bytes() method is an integer, and now uPy
follows CPython behaviour and raises a TypeError for this test.

Note: CPython checks the argument types before checking the number of
arguments, but uPy does it the other way around, so they give different
exception messages for this test, but still the same type, a TypeError.
2018-05-08 17:05:32 +10:00
Damien George
74ab341d3a tests/cpydiff: Remove working cases from types_float_rounding. 2018-05-04 22:30:50 +10:00
Damien George
cd9d71edc8 tests/cpydiff: Remove types_str_decodeerror now that it succeeds.
Commit 68c28174d0 implemented checking for
valid utf-8 data.
2018-05-04 22:27:14 +10:00
Damien George
4b5111f8e1 tests/cpydiff: Remove core_function_unpacking now that it succeeds.
Commit 1e70fda69f fixes this difference.
2018-05-04 22:19:50 +10:00
Damien George
c926e72750 tests/cpydiff: Indent workaround code snippet so it formats correctly. 2018-03-15 15:49:38 +11:00
Damien George
e4e3f0d727 tests/cpydiff: Update subclassing Exception case and give work-around. 2017-12-12 17:13:39 +11:00
Paul Sokolovsky
3c483842db tests/cpydiff: Fix markup where "" (xref) was used instead of "`" (code). 2017-12-03 15:32:09 +02:00
Damien George
63f47104fe tests/cpydiff: Add difference-test for second arg of builtin next(). 2017-11-28 10:50:53 +11:00
Paul Sokolovsky
75163325ae tests/cpydiff: Add cases for locals() discrepancies.
MicroPython doesn't maintain local symbolic environment, so any feature
depending on it won't work as expected.
2017-09-16 13:05:15 +03:00
Paul Sokolovsky
ad3abcd324 tests/cpydiff: Add case for str.ljust/rjust. 2017-07-09 15:04:26 +03:00
Paul Sokolovsky
0c5369a1f0 tests/cpydiff/: Improve wording, add more workarounds. 2017-07-09 14:33:55 +03:00
Paul Sokolovsky
5f65ad8c96 tests/cpydiff/core_class_supermultiple: Same cause as core_class_mro. 2017-07-09 13:47:23 +03:00
Paul Sokolovsky
c5efb8159f tests/cpydiff/core_arguments: Move under Functions subsection.
This is the last "orphan" case.
2017-07-09 13:36:28 +03:00
Paul Sokolovsky
b2979023ac tests/cpydiff/core_class_mro: Move under Classes, add workaround. 2017-07-09 13:32:17 +03:00
Paul Sokolovsky
ed52955c6b tests/cpydiff/modules_deque: Elaborate workaround. 2017-07-05 23:03:37 +03:00
Damien George
084824f866 tests: Move super-as-local test from cpydiff to basic tests.
It's now possible to use the name "super" as a local variable.
2017-05-06 11:01:57 +10:00
Paul Sokolovsky
12ea06567c tests/cpydiff/core_function_unpacking: Fill in workaround. 2017-05-04 00:48:48 +03:00
Paul Sokolovsky
92657c671f tests/cpydiff/core_import_split_ns_pkgs: Test for split namespace packages. 2017-05-03 17:16:19 +03:00
Paul Sokolovsky
3923f96dc6 tests/cpydiff/core_import_path: Test showing difference in package.__path__. 2017-05-03 12:22:53 +03:00
Paul Sokolovsky
c1b19115e5 tests/cpydiff/core_import_prereg: Fill in cause and workaround. 2017-05-02 14:12:52 +03:00
Paul Sokolovsky
8bb84cc627 tests/cpydiff/core_function_userattr: Clarify, fill in cause and workaround. 2017-05-02 03:43:21 +03:00
Paul Sokolovsky
58ecbc7752 tests/cpydiff/core_arguments: Fill in cause/workaround. 2017-05-02 03:36:47 +03:00
Damien George
465a604547 tests/cpydiff: Add a test for storing iterable to a list slice. 2017-02-20 17:22:12 +11:00
Rami Ali
86c7507233 tests/cpydiff: Add initial set of tests for uPy-CPython differences.
These tests are intended to fail, as they provide a programatic record of
differences between uPy and CPython.  They also contain a special comment
at the start of the file which has meta-data describing the difference,
including known causes and known workarounds.
2017-02-20 16:50:34 +11:00