Commit Graph

3752 Commits

Author SHA1 Message Date
Damien George 246b2e016a py/mkrules.mk: Remove stray vpath and unused -Itmp, add $(Q) for $(AR).
Signed-off-by: Damien George <damien@micropython.org>
2020-12-14 13:57:15 +11:00
Damien George 0091041f5a py/modmath: Simplify handling of positional args to reduce code size.
As a general pattern, required positional arguments that are not named do
not need to be parsed using mp_arg_parse_all().

Signed-off-by: Damien George <damien@micropython.org>
2020-12-14 13:30:56 +11:00
Joris Peeraer 5020b14d54 py/mpprint: Fix length calculation for strings with precision-modifier.
Two issues are tackled:

1. The calculation of the correct length to print is fixed to treat the
   precision as a maximum length instead as the exact length.
   This is done for both qstr (%q) and for regular str (%s).

2. Fix the incorrect use of mp_printf("%.*s") to mp_print_strn().

   Because of the fix of above issue, some testcases that would print
   an embedded null-byte (^@ in test-output) would now fail.
   The bug here is that "%s" was used to print null-bytes. Instead,
   mp_print_strn is used to make sure all bytes are outputted and the
   exact length is respected.

Test-cases are added for both %s and %q with a combination of precision
and padding specifiers.
2020-12-07 23:32:06 +11:00
Damien George be24e6a53f py/mpprint: Prevent case fall-through when assert is disabled.
Signed-off-by: Damien George <damien@micropython.org>
2020-11-29 17:31:24 +11:00
Jim Mussared 0e8af2b370 extmod/modbluetooth: Add API for L2CAP channels.
Also known as L2CAP "connection oriented channels". This provides a
socket-like data transfer mechanism for BLE.

Currently only implemented for NimBLE on STM32 / Unix.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-24 01:07:17 +11:00
Jim Mussared a7932ae4e6 tools/makeqstrdefs.py: Run qstr preprocessing in parallel.
This gives a substantial speedup of the preprocessing step, i.e. the
generation of qstr.i.last.  For example on a clean build, making
qstr.i.last:

    21s -> 4s on STM32 (WB55)
    8.9 -> 1.8s on Unix (dev).

Done in collaboration with @stinos.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-12 15:04:53 +11:00
Damien George d7e1526593 py/binary: Fix sign extension setting wide integer on 32-bit archs.
Signed-off-by: Damien George <damien@micropython.org>
2020-11-11 22:18:24 +11:00
Damien George 1fef5662ab py/mpz: Do sign extension in mpz_as_bytes for negative values.
Signed-off-by: Damien George <damien@micropython.org>
2020-11-11 22:18:24 +11:00
stijn 0153148fd2 py/py.mk: Support C++ code for user C modules.
Support C++ code in .cpp files by providing CXX counterparts of the
_USERMOD_ flags we have for C already.  This merely enables the Makefile of
user C modules to use variables specific to C++ compilation, it is still up
to each port's main Makefile to also include these in the build.
2020-10-29 15:29:20 +11:00
stijn e498a8bd13 py: Workaround clang error when building misc.h with C++ compiler. 2020-10-29 15:28:14 +11:00
stijn 8e94fa0d2e py/makeqstrdefs.py: Support preprocessing C++ files for QSTR generation.
When SCR_QSTR contains C++ files they should be preprocessed with the same
compiler flags (CXXFLAGS) as they will be compiled with, to make sure code
scanned for QSTR occurrences is effectively the code used in the rest of
the build.  The 'split SCR_QSTR in .c and .cpp files and process each with
different flags' logic isn't trivial to express in a Makefile and the
existing principle for deciding which files to preprocess was already
rather complicated, so the actual preprocessing is moved into
makeqstrdefs.py completely.
2020-10-29 15:27:30 +11:00
stijn f1666419a8 py/mkrules.mk: Add target for compiling C++ files.
Move the target from the ESP32 Makefile since that does what is needed
already, but also include files from user C modules as is done for the C
files.
2020-10-29 15:27:18 +11:00
stijn 2b9f0586e7 py/makeqstrdefs.py: Process C++ files as well.
Preprocessed C++ code isn't different from C code when it comes to QSTR
instances so process it as well.
2020-10-29 15:27:11 +11:00
stijn 1b723937e3 py/makeqstrdefs.py: Fix beaviour when scanning non-C preprocessed files.
When process_file() is passed a preprocessed C++ file for instance it won't
find any lines containing .c files and the last_fname variable remains
None, so handle that gracefully.
2020-10-29 15:26:35 +11:00
Emil Renner Berthing ccd92335a1 py, extmod: Introduce and use MP_FALLTHROUGH macro.
Newer GCC versions are able to warn about switch cases that fall
through.  This is usually a sign of a forgotten break statement, but in
the few cases where a fall through is intended we annotate it with this
macro to avoid the warning.
2020-10-22 11:53:16 +02:00
Emil Renner Berthing f1f6ef7b17 py/vmentrytable: Ignore GCC -Woverride-init.
Like Clang, GCC warns about this file, but only with -Woverride-init
which is enabled by -Wextra. Disable the warnings for this file just
like we do for Clang to make -Wextra happy.
2020-10-22 11:47:36 +02:00
Emil Renner Berthing 9aa58cf8ba py, extmod: Add explicit initializers for default values.
When compiling with -Wextra which includes -Wmissing-field-initializers
GCC will warn that the defval field of mp_arg_val_t is not initialized.
This is just a warning as it is defined to be zero initialized, but since
it is a union it makes sense to be explicit about which member we're
going to use, so add the explicit initializers and get rid of the
warning.
2020-10-22 11:47:36 +02:00
Emil Renner Berthing fdd6fa389e py: Use unsigned comparison of chars.
On x86 chars are signed, but we're comparing a char to '0' + unsigned int,
which is promoted to an unsigned int. Let's promote the char to unsigned
before doing the comparison to avoid weird corner cases.
2020-10-22 11:47:36 +02:00
Emil Renner Berthing 6d3aa16443 py/objexcept: Compare mp_emergency_exception_buf_size signed.
mp_emergency_exception_buf_size is signed, so let's make sure we compare
it as such.
2020-10-22 11:47:36 +02:00
Emil Renner Berthing 6324c3e054 py/scope: Name and use id_kind_type_t.
The function scope_find_or_add_id used to take a scope_kind_t enum and
save it in an uint8_t. Saving an enum in a uint8_t is fine, but
everywhere this function is called it is not actually given a
scope_kind_t but an anonymous enum instead. Let's give this enum a name
and use that as the argument type.

This doesn't change the generated code, but is a C type mismatch that
unfortunately doesn't show up unless you enable -Wenum-conversion.
2020-10-22 11:40:56 +02:00
Jim Mussared b137d064e9 py/objtype: Handle __dict__ attribute when type has no locals. 2020-10-10 00:16:32 +11:00
Jim Mussared 880875bea1 py/objdict: Add mp_const_empty_dict_obj, use it for mp_const_empty_map. 2020-10-10 00:16:26 +11:00
Damien George 843dcd4f85 py/parse: Expose rule-name printing as MICROPY_DEBUG_PARSE_RULE_NAME.
So it can be enabled without modifying the source.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-01 15:26:43 +10:00
Mike Wadsten c711c0049e py/makeversionhdr.py: Match only git tags which look like versions.
Some downstream projects may use tags in their repositories for more than
just designating MicroPython releases.  In those cases, the
makeversionhdr.py script would end up using a different tag than intended.
So tell `git describe` to only match tags that look like a MicroPython
version tag, such as `v1.12` or `v2.0`.
2020-10-01 11:01:43 +10:00
Damien George 50e34f979c py/objarray.h: Add mp_obj_memoryview_init() helper function.
Signed-off-by: Damien George <damien@micropython.org>
2020-09-25 12:23:11 +10:00
Iyassou Shimels ca017841d6 py/objstr: Make bytes(bytes_obj) return bytes_obj.
Calling the bytes constructor on a bytes object returns the original bytes
object.  This saves allocating a new instance, and matches CPython.

Signed-off-by: Iyassou Shimels <s.iyassou@gmail.com>
2020-09-24 11:04:58 +10:00
Jim Mussared 9d1983f078 py/dynruntime.h: Add mp_import_* and mp_load/store_*.
These functions already exist in the fun table, and this commit just adds
convenience macros for them.
2020-09-18 18:34:02 +10:00
Damien George 8f20cdc353 all: Rename absolute time-based functions to include "epoch".
For time-based functions that work with absolute time there is the need for
an Epoch, to set the zero-point at which the absolute time starts counting.
Such functions include time.time() and filesystem stat return values.  And
different ports may use a different Epoch.

To make it clearer what functions use the Epoch (whatever it may be), and
make the ports more consistent with their use of the Epoch, this commit
renames all Epoch related functions to include the word "epoch" in their
name (and remove references to "2000").

Along with this rename, the following things have changed:

- mp_hal_time_ns() is now specified to return the number of nanoseconds
  since the Epoch, rather than since 1970 (but since this is an internal
  function it doesn't change anything for the user).

- littlefs timestamps on the esp8266 have been fixed (they were previously
  off by 30 years in nanoseconds).

Otherwise, there is no functional change made by this commit.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-18 17:20:34 +10:00
Damien George acdb0608b7 py/parse: Pass in an mp_print_t to mp_parse_node_print.
So the output can be redirected if needed.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 23:00:03 +10:00
Damien George 85f2b239d8 py/showbc: Pass in an mp_print_t struct to all bytecode-print functions.
So the output can be redirected if needed.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 17:22:28 +10:00
stijn 2e54d9d146 py: Fix handling of NaN in certain pow implementations.
Adds a new compile-time option MICROPY_PY_MATH_POW_FIX_NAN for use with
toolchains that don't handle pow-of-NaN correctly.
2020-09-11 10:04:57 +10:00
Damien George 8d5a40c86e py/objfloat: Fix handling of negative float to power of nan.
Prior to this commit, pow(-2, float('nan')) would return (nan+nanj), or
raise an exception on targets that don't support complex numbers.  This is
fixed to return simply nan, as CPython does.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 10:03:57 +10:00
stijn 40ad8f1666 all: Rename "sys" module to "usys".
This is consistent with the other 'micro' modules and allows implementing
additional features in Python via e.g. micropython-lib's sys.

Note this is a breaking change (not backwards compatible) for ports which
do not enable weak links, as "import sys" must now be replaced with
"import usys".
2020-09-04 00:10:24 +10:00
Damien George b0932fcf2e all: Bump version to 1.13.
Signed-off-by: Damien George <damien@micropython.org>
2020-09-02 12:01:26 +10:00
Damien George 06659077a8 all: Update Python code to conform to latest black formatting.
Updating to Black v20.8b1 there are two changes that affect the code in
this repository:

- If there is a trailing comma in a list (eg [], () or function call) then
  that list is now written out with one line per element.  So remove such
  trailing commas where the list should stay on one line.

- Spaces at the start of """ doc strings are removed.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-29 15:18:01 +10:00
Damien George ee50a6effe py/mphal.h: Introduce mp_hal_time_ns and implement on various ports.
This should return a 64-bit value being the number of nanoseconds since
1970/1/1.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 16:13:44 +10:00
Damien George 5f9b105244 py/runtime: Fix builtin compile() in "single" mode so it prints exprs.
As per CPython behaviour, compile(stmt, "file", "single") should create
code which prints to stdout (via __repl_print__) the results of any
expressions in stmt.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 11:38:46 +10:00
Damien George 9883d8e818 py/persistentcode: Maintain root ptr list of imported native .mpy code.
On ports where normal heap memory can contain executable code (eg ARM-based
ports such as stm32), native code loaded from an .mpy file may be reclaimed
by the GC because there's no reference to the very start of the native
machine code block that is reachable from root pointers (only pointers to
internal parts of the machine code block are reachable, but that doesn't
help the GC find the memory).

This commit fixes this issue by maintaining an explicit list of root
pointers pointing to native code that is loaded from an .mpy file.  This
is not needed for all ports so is selectable by the new configuration
option MICROPY_PERSISTENT_CODE_TRACK_RELOC_CODE.  It's enabled by default
if a port does not specify any special functions to allocate or commit
executable memory.

A test is included to test that native code loaded from an .mpy file does
not get reclaimed by the GC.

Fixes #6045.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-02 22:34:09 +10:00
Jonathan Hogg 37e1b5c891 py/compile: Don't await __aiter__ special method in async-for.
MicroPython's original implementation of __aiter__ was correct for an
earlier (provisional) version of PEP492 (CPython 3.5), where __aiter__ was
an async-def function.  But that changed in the final version of PEP492 (in
CPython 3.5.2) where the function was changed to a normal one.  See
https://www.python.org/dev/peps/pep-0492/#why-aiter-does-not-return-an-awaitable
See also the note at the end of this subsection in the docs:
https://docs.python.org/3.5/reference/datamodel.html#asynchronous-iterators
And for completeness the BPO: https://bugs.python.org/issue27243

To be consistent with the Python spec as it stands today (and now that
PEP492 is final) this commit changes MicroPython's behaviour to match
CPython:  __aiter__ should return an async-iterable object, but is not
itself awaitable.

The relevant tests are updated to match.

See #6267.
2020-07-25 00:58:18 +10:00
Damien George a853fff838 py/obj.h: Fix mp_seq_replace_slice_no_grow to use memmove not memcpy.
Because the argument arrays may overlap, as show by the new tests in this
commit.

Also remove the debugging comments for these macros, add a new comment
about overlapping regions, and separate the macros by blank lines to make
them easier to read.

Fixes issue #6244.

Signed-off-by: Damien George <damien@micropython.org>
2020-07-21 01:05:59 +10:00
Damien George 332d83343f py: Rework mp_convert_member_lookup to properly handle built-ins.
This commit fixes lookups of class members to make it so that built-in
functions that are used as methods/functions of a class work correctly.

The mp_convert_member_lookup() function is pretty much completely changed
by this commit, but for the most part it's just reorganised and the
indenting changed.  The functional changes are:

- staticmethod and classmethod checks moved to later in the if-logic,
  because they are less common and so should be checked after the more
  common cases.

- The explicit mp_obj_is_type(member, &mp_type_type) check is removed
  because it's now subsumed by other, more general tests in this function.

- MP_TYPE_FLAG_BINDS_SELF and MP_TYPE_FLAG_BUILTIN_FUN type flags added to
  make the checks in this function much simpler (now they just test this
  bit in type->flags).

- An extra check is made for mp_obj_is_instance_type(type) to fix lookup of
  built-in functions.

Fixes #1326 and #6198.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-30 23:55:32 +10:00
Damien George d06ae1d2b1 py/obj.h: Make existing MP_TYPE_FLAG_xxx macros sequential.
There's no reason to have them non-sequential, this was likely a typo from
commit 9ec1caf42e.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-30 23:55:32 +10:00
Damien George 9f911d822e py/objcomplex: Add mp_obj_get_complex_maybe for use in complex bin-op.
This allows complex binary operations to fail gracefully with unsupported
operation rather than raising an exception, so that special methods work
correctly.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-27 01:03:10 +10:00
Damien George 41fa8b5482 py/emitnative: Implement binary operations for viper uint operands.
uint types in viper mode can now be used for all binary operators except
floor-divide and modulo.

Fixes issue #1847 and issue #6177.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-27 00:24:04 +10:00
Damien George b3b8706d27 py/asm: Add condition codes for signed comparisons.
Signed-off-by: Damien George <damien@micropython.org>
2020-06-27 00:24:04 +10:00
Damien George aa26fe62d8 py/asm: Add funcs/macros to emit machine code for logical-shift-right.
Signed-off-by: Damien George <damien@micropython.org>
2020-06-27 00:24:04 +10:00
Damien George 457fdf61c3 py/objtype: Support passing in an OrderedDict to type() as the locals.
An OrderedDict can now be used for the locals when creating a type
explicitly via type(name, bases, locals).

Signed-off-by: Damien George <damien@micropython.org>
2020-06-24 12:05:40 +10:00
Damien George 456a3abe8d py/obj.h: Add public mp_obj_is_dict_or_ordereddict() helper macro.
And use it in py/objdict.c instead of mp_obj_is_dict_type.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-24 12:05:00 +10:00
Damien George 6164c7e666 py/misc.h: Add missing semi-colon in mp_float_union_t for big-endian.
Fixes issue #6161.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-22 13:42:24 +10:00
Damien George f2e267da68 py/compile: Implement PEP 526, syntax for variable annotations.
This addition to the grammar was introduced in Python 3.6.  It allows
annotating the type of a varilable, like:

    x: int = 123
    s: str

The implementation in this commit is quite simple and just ignores the
annotation (the int and str bits above).  The reason to implement this is
to allow Python 3.6+ code that uses this feature to compile under
MicroPython without change, and for users to use type checkers.

In the future viper could use this syntax as a way to give types to
variables, which is currently done in a bit of an ad-hoc way, eg
x = int(123).  And this syntax could potentially be used in the inline
assembler to define labels in an way that's easier to read.
2020-06-16 23:18:01 +10:00