Commit Graph

4734 Commits

Author SHA1 Message Date
Dan Halbert
ef0830bfe2 merge from upstream + wip 2020-11-25 17:52:06 -05:00
Dan Halbert
9dbea36eac changed alarm.time API 2020-11-25 15:09:27 -05:00
Jeff Epler
e778fc1f87
Merge pull request #3741 from hathach/fix-cdc-connection-race
update tinyusb to fix cdc connection race
2020-11-24 19:11:41 -06:00
Jeff Epler
c451b22255 Disable 3-arg pow() function on m0 boards
`pow(a, b, c)` can compute `(a ** b) % c` efficiently (in time and memory).
This can be useful for extremely specific applications, like implementing
the RSA cryptosystem.  For typical uses of CircuitPython, this is not an
important feature.  A survey of the bundle and learn system didn't find
any uses.

Disable it on M0 builds so that we can fit in needed upgrades to the USB
stack.
2020-11-24 16:54:33 -06:00
Dan Halbert
7a45afc549 working, but need to avoid deep sleeping too fast before USB ready 2020-11-23 22:44:53 -05:00
Scott Shawcroft
f8dcb25170
Merge pull request #3694 from jepler/update-ulab2
ulab: Update to release tag 1.1.0
2020-11-23 15:17:46 -08:00
Jeff Epler
9d8be648ee ulab: Update to release tag 1.1.0
Disable certain classes of diagnostic when building ulab.  We should
submit patches upstream to (A) fix these errors and (B) upgrade their
CI so that the problems are caught before we want to integrate with
CircuitPython, but not right now.
2020-11-23 10:23:50 -06:00
Jim Mussared
0e8af2b370 extmod/modbluetooth: Add API for L2CAP channels.
Also known as L2CAP "connection oriented channels". This provides a
socket-like data transfer mechanism for BLE.

Currently only implemented for NimBLE on STM32 / Unix.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-24 01:07:17 +11:00
Dan Halbert
75559f35cc wip: ResetReason to microcontroller.cpu 2020-11-21 23:29:52 -05:00
Dan Halbert
e4c66990e2 compiles 2020-11-20 23:33:39 -05:00
Jeff Epler
982bce7259 py.mk: allow translation to be overriden in GNUmakefile
I like to use local makefile overrides, in the file GNUmakefile
(or, on case-sensitive systems, makefile) to set compilation choices.
However, writing
    TRANSLATION := de_DE
    include Makefile
did not work, because py.mk would override the TRANSLATION := specified
in an earlier part of the makefiles (but not from the commandline).

By using ?= instead of := the local makefile override works, but when
TRANSLATION is not specified it continues to work as before.
2020-11-19 16:23:35 -06:00
Jeff Epler
b2b8520880 Always use preprocessor for MICROPY_ERROR_REPORTING
This ensures that only the translate("") alternative that will be used
is seen after preprocessing.  Improves the quality of the Huffman encoding
and reduces binary size slightly.

Also makes one "enhanced" error message only occur when ERROR_REPORTING_DETAILED:
Instead of the word-for-word python3 error message
"Type object has no attribute '%q'", the message will be
"'type' object has no attribute '%q'".  Also reduces binary size.
(that's rolled into this commit as it was right next to a change to
use the preprocessor for MICROPY_ERROR_REPORTING)

Note that the odd semicolon after "value_error:" in parsenum.c is necessary
due to a detail of the C grammar, in which a declaration cannot follow
a label directly.
2020-11-19 16:18:52 -06:00
Jeff Epler
c06fc8e02d Introduce, use mp_raise_arg1
This raises an exception with a given object value.  Saves a bit of
code size.
2020-11-19 16:15:06 -06:00
Jeff Epler
d5f6748d1b Use mp_raise instead of nlr_raise(new_exception) where possible
This saves a bit of code space
2020-11-19 16:13:01 -06:00
Jeff Epler
0556f9f851 Revert "samd21: Enable terse error reporting on resource constrained chip family"
This reverts commit 9a642fc049.
2020-11-19 15:12:56 -06:00
Dan Halbert
649c930536 wip 2020-11-19 15:43:39 -05:00
Dan Halbert
5bb3c321e9 merge from main 2020-11-19 00:29:14 -05:00
Dan Halbert
682054a216 WIP: redo API; not compiled yet 2020-11-19 00:23:27 -05:00
Jeff Epler
9a642fc049 samd21: Enable terse error reporting on resource constrained chip family
This reclaims over 1kB of flash space by simplifying certain exception
messages.  e.g., it will no longer display the requested/actual length
when a fixed list/tuple of N items is needed:

        if (MICROPY_ERROR_REPORTING == MICROPY_ERROR_REPORTING_TERSE) {
            mp_raise_ValueError(translate("tuple/list has wrong length"));
        } else {
            mp_raise_ValueError_varg(translate("requested length %d but object has length %d"),
                (int)len, (int)seq_len);

Other chip families including samd51 keep their current error reporting
capabilities.
2020-11-18 20:37:36 -06:00
Dan Halbert
ffff02c053 Merge remote-tracking branch 'adafruit/main' into sleep 2020-11-16 12:06:11 -05:00
Dan Halbert
bb77f1d130 wip: initial code changes, starting from @tannewt's sleepio branch 2020-11-16 11:56:20 -05:00
root
0cd951fb73 Prevent exceptions from accumulating in REPL 2020-11-16 10:36:05 -06:00
Scott Shawcroft
bda3267432
Save flash space
* No weak link for modules. It only impacts _os and _time and is
  already disabled for non-full builds.
* Turn off PA00 and PA01 because they are the crystal on the Metro
  M0 Express.
* Change ejected default to false to move it to BSS. It is set on
  USB connection anyway.
* Set sinc_filter to const. Doesn't help flash but keeps it out of
  RAM.
2020-11-13 18:57:52 -08:00
Jim Mussared
a7932ae4e6 tools/makeqstrdefs.py: Run qstr preprocessing in parallel.
This gives a substantial speedup of the preprocessing step, i.e. the
generation of qstr.i.last.  For example on a clean build, making
qstr.i.last:

    21s -> 4s on STM32 (WB55)
    8.9 -> 1.8s on Unix (dev).

Done in collaboration with @stinos.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-12 15:04:53 +11:00
Damien George
d7e1526593 py/binary: Fix sign extension setting wide integer on 32-bit archs.
Signed-off-by: Damien George <damien@micropython.org>
2020-11-11 22:18:24 +11:00
Damien George
1fef5662ab py/mpz: Do sign extension in mpz_as_bytes for negative values.
Signed-off-by: Damien George <damien@micropython.org>
2020-11-11 22:18:24 +11:00
gamblor21
4c93db3595 Renamed to adafruit_bus_device 2020-11-03 18:35:20 -06:00
Dan Halbert
72b829dff0 add binascii to most builds 2020-11-01 14:52:03 -05:00
gamblor21
78477a374a Initial SPI commit 2020-10-31 12:17:29 -05:00
stijn
0153148fd2 py/py.mk: Support C++ code for user C modules.
Support C++ code in .cpp files by providing CXX counterparts of the
_USERMOD_ flags we have for C already.  This merely enables the Makefile of
user C modules to use variables specific to C++ compilation, it is still up
to each port's main Makefile to also include these in the build.
2020-10-29 15:29:20 +11:00
stijn
e498a8bd13 py: Workaround clang error when building misc.h with C++ compiler. 2020-10-29 15:28:14 +11:00
stijn
8e94fa0d2e py/makeqstrdefs.py: Support preprocessing C++ files for QSTR generation.
When SCR_QSTR contains C++ files they should be preprocessed with the same
compiler flags (CXXFLAGS) as they will be compiled with, to make sure code
scanned for QSTR occurrences is effectively the code used in the rest of
the build.  The 'split SCR_QSTR in .c and .cpp files and process each with
different flags' logic isn't trivial to express in a Makefile and the
existing principle for deciding which files to preprocess was already
rather complicated, so the actual preprocessing is moved into
makeqstrdefs.py completely.
2020-10-29 15:27:30 +11:00
stijn
f1666419a8 py/mkrules.mk: Add target for compiling C++ files.
Move the target from the ESP32 Makefile since that does what is needed
already, but also include files from user C modules as is done for the C
files.
2020-10-29 15:27:18 +11:00
stijn
2b9f0586e7 py/makeqstrdefs.py: Process C++ files as well.
Preprocessed C++ code isn't different from C code when it comes to QSTR
instances so process it as well.
2020-10-29 15:27:11 +11:00
stijn
1b723937e3 py/makeqstrdefs.py: Fix beaviour when scanning non-C preprocessed files.
When process_file() is passed a preprocessed C++ file for instance it won't
find any lines containing .c files and the last_fname variable remains
None, so handle that gracefully.
2020-10-29 15:26:35 +11:00
microDev
930cf14dce
Add check for invalid io, function to disable all alarms 2020-10-27 16:17:26 -07:00
microDev
59df1a11ad
Add alarm_touch module 2020-10-27 16:16:52 -07:00
microDev
da449723df
Fix build error 2020-10-27 16:16:15 -07:00
microDev
4d8ffdca8d
restructure alarm modules 2020-10-27 16:15:09 -07:00
microDev
e5ff55b15c
Renamed alarm modules 2020-10-27 16:13:25 -07:00
microDev
e310b871c8
Get io wake working 2020-10-27 16:13:25 -07:00
microDev
90b9ec6f2c
Initial Sleep Support 2020-10-27 16:13:22 -07:00
gamblor21
b637d3911e Initial commit 2020-10-24 20:48:35 -05:00
Emil Renner Berthing
ccd92335a1 py, extmod: Introduce and use MP_FALLTHROUGH macro.
Newer GCC versions are able to warn about switch cases that fall
through.  This is usually a sign of a forgotten break statement, but in
the few cases where a fall through is intended we annotate it with this
macro to avoid the warning.
2020-10-22 11:53:16 +02:00
Emil Renner Berthing
f1f6ef7b17 py/vmentrytable: Ignore GCC -Woverride-init.
Like Clang, GCC warns about this file, but only with -Woverride-init
which is enabled by -Wextra. Disable the warnings for this file just
like we do for Clang to make -Wextra happy.
2020-10-22 11:47:36 +02:00
Emil Renner Berthing
9aa58cf8ba py, extmod: Add explicit initializers for default values.
When compiling with -Wextra which includes -Wmissing-field-initializers
GCC will warn that the defval field of mp_arg_val_t is not initialized.
This is just a warning as it is defined to be zero initialized, but since
it is a union it makes sense to be explicit about which member we're
going to use, so add the explicit initializers and get rid of the
warning.
2020-10-22 11:47:36 +02:00
Emil Renner Berthing
fdd6fa389e py: Use unsigned comparison of chars.
On x86 chars are signed, but we're comparing a char to '0' + unsigned int,
which is promoted to an unsigned int. Let's promote the char to unsigned
before doing the comparison to avoid weird corner cases.
2020-10-22 11:47:36 +02:00
Emil Renner Berthing
6d3aa16443 py/objexcept: Compare mp_emergency_exception_buf_size signed.
mp_emergency_exception_buf_size is signed, so let's make sure we compare
it as such.
2020-10-22 11:47:36 +02:00
Emil Renner Berthing
6324c3e054 py/scope: Name and use id_kind_type_t.
The function scope_find_or_add_id used to take a scope_kind_t enum and
save it in an uint8_t. Saving an enum in a uint8_t is fine, but
everywhere this function is called it is not actually given a
scope_kind_t but an anonymous enum instead. Let's give this enum a name
and use that as the argument type.

This doesn't change the generated code, but is a C type mismatch that
unfortunately doesn't show up unless you enable -Wenum-conversion.
2020-10-22 11:40:56 +02:00
Christian Walther
1eab0692b5 Fix missing nproc on macOS.
396979a breaks building on macOS: `nproc` is a Linux thing, use a cross-platform alternative.
2020-10-20 16:39:32 +02:00
Dan Halbert
82b49afe43 enable CIRCUITPY_BLEIO_HCI on non-nRF boards where it will fit 2020-10-15 11:27:21 -04:00
Dan Halbert
f1e8f2b404
Merge pull request #3554 from gamblor21/move_ordereddict
Moved ORDEREDDICT define to central location
2020-10-14 22:39:04 -04:00
gamblor21
f6f89565d8 Remove ordered dict from SAMD21 2020-10-14 20:18:49 -05:00
gamblor21
e6d0b207ec Removed MICROPY_PY_COLLECTIONS_NAMEDTUPLE__ASDICT from unix coverage 2020-10-14 14:06:34 -05:00
gamblor21
4270061db4 Moved ORDEREDDICT define to central location 2020-10-13 18:52:27 -05:00
Scott Shawcroft
9de96786ad
Merge pull request #3538 from jepler/parallel-qstrlast
build: parallelize the qstr build steps
2020-10-12 15:52:43 -07:00
Scott Shawcroft
1eb1434fc9
Merge pull request #3537 from jepler/update-protomatter-2
rgbmatrix: update protomatter to 1.0.5 tag
2020-10-12 15:45:51 -07:00
Scott Shawcroft
179e13f103
Merge pull request #3539 from jepler/lto-type-mismatch
remove warning-disable flag that seems unneeded now
2020-10-12 15:44:25 -07:00
Kenny
94beeabc51 remove unnecessary board configuration and address feedback 2020-10-11 22:42:59 -07:00
Jeff Epler
479552ce56 build: Make genlast write the "split" files
This gets a further speedup of about 2s (12s -> 9.5s elapsed build time)
for stm32f405_feather

For what are probably historical reasons, the qstr process involves
preprocessing a large number of source files into a single "qstr.i.last"
file, then reading this and splitting it into one "qstr" file for each
original source ("*.c") file.

By eliminating the step of writing qstr.i.last as well as making the
regular-expression-matching part be parallelized, build speed is further
improved.

Because the step to build QSTR_DEFS_COLLECTED does not access
qstr.i.last, the path is replaced with "-" in the Makefile.
2020-10-11 21:18:03 -05:00
Jeff Epler
607e4a905a build: parallelize the creation of qstr.i.last
Rather than simply invoking gcc in preprocessor mode with a list of files, use
a Python script with the (python3) ThreadPoolExecutor to invoke the
preprocessor in parallel.

The amount of concurrency is the number of system CPUs, not the makefile "-j"
parallelism setting, because there is no simple and correct way for a Python
program to correctly work together with make's idea of parallelism.

This reduces the build time of stm32f405 feather (a non-LTO build) from 16s to
12s on my 16-thread Ryzen machine.
2020-10-11 20:19:59 -05:00
Jeff Epler
c139eccc92 remove warning that seems unneeded now 2020-10-11 16:23:02 -05:00
Kenny
98aa4b7943 update async tests with less upython workaround and more cpython compatibility 2020-10-10 23:39:32 -07:00
Kenny
5d96afc5c2 i do not know if this is needed but this is not the vm i use anymore 2020-10-10 15:45:08 -07:00
Kenny
bf849ff674 async def syntax rigor and __await__ magic method
Some examples of improved compliance with CPython that currently
have divergent behavior in CircuitPython are listed below:

* yield from is not allowed in async methods
```
>>> async def f():
...     yield from 'abc'
...
Traceback (most recent call last):
  File "<stdin>", line 2, in f
SyntaxError: 'yield from' inside async function
```

* await only works on awaitable expressions
```
>>> async def f():
...     await 'not awaitable'
...
>>> f().send(None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in f
AttributeError: 'str' object has no attribute '__await__'
```

* only __await__()able expressions are awaitable
Okay this one actually does not work in circuitpython at all today.
This is how CPython works though and pretending __await__ does not
exist will only bite users who write both.
```
>>> class c:
...     pass
...
>>> def f(self):
...     yield
...     yield
...     return 'f to pay respects'
...
>>> c.__await__ = f  # could just as easily have put it on the class but this shows how it's wired
>>> async def g():
...     awaitable_thing = c()
...     partial = await awaitable_thing
...     return 'press ' + partial
...
>>> q = g()
>>> q.send(None)
>>> q.send(None)
>>> q.send(None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration: press f to pay respects
```
2020-10-10 15:45:08 -07:00
warriorofwire
5cadf525bd fix missing cflag defeating the board gating 2020-10-10 15:45:08 -07:00
warriorofwire
d94d2d2975 Add async/await syntax to FULL_BUILD
This adds the `async def` and `await` verbs to valid CircuitPython syntax using the Micropython implementation.

Consider:
```
>>> class Awaitable:
...     def __iter__(self):
...         for i in range(3):
...             print('awaiting', i)
...             yield
...         return 42
...
>>> async def wait_for_it():
...     a = Awaitable()
...     result = await a
...     return result
...
>>> task = wait_for_it()
>>> next(task)
awaiting 0
>>> next(task)
awaiting 1
>>> next(task)
awaiting 2
>>> next(task)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  StopIteration: 42
>>>
```

and more excitingly:
```
>>> async def it_awaits_a_subtask():
...     value = await wait_for_it()
...     print('twice as good', value * 2)
...
>>> task = it_awaits_a_subtask()
>>> next(task)
awaiting 0
>>> next(task)
awaiting 1
>>> next(task)
awaiting 2
>>> next(task)
twice as good 84
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  StopIteration:
```

Note that this is just syntax plumbing, not an all-encompassing implementation of an asynchronous task scheduler or asynchronous hardware apis.
  uasyncio might be a good module to bring in, or something else - but the standard Python syntax does not _strictly require_ deeper hardware
  support.
Micropython implements the await verb via the __iter__ function rather than __await__.  It's okay.

The syntax being present will enable users to write clean and expressive multi-step state machines that are written serially and interleaved
  according to the rules provided by those users.

Given that this does not include an all-encompassing C scheduler, this is expected to be an advanced functionality until the community settles
  on the future of deep hardware support for async/await in CircuitPython.  Users will implement yield-based schedulers and tasks wrapping
  synchronous hardware APIs with polling to avoid blocking, while their application business logic gets simple `await` statements.
2020-10-10 15:38:40 -07:00
Jeff Epler
5e38bb98cb rgbmatrix: update protomatter to 1.0.5 tag
this is compile-tested on
 stm32f405 feather
 matrixportal
 nrf52840 feather

but not actually tested-tested.
2020-10-10 14:30:37 -05:00
Jim Mussared
b137d064e9 py/objtype: Handle __dict__ attribute when type has no locals. 2020-10-10 00:16:32 +11:00
Jim Mussared
880875bea1 py/objdict: Add mp_const_empty_dict_obj, use it for mp_const_empty_map. 2020-10-10 00:16:26 +11:00
Damien George
843dcd4f85 py/parse: Expose rule-name printing as MICROPY_DEBUG_PARSE_RULE_NAME.
So it can be enabled without modifying the source.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-01 15:26:43 +10:00
Mike Wadsten
c711c0049e py/makeversionhdr.py: Match only git tags which look like versions.
Some downstream projects may use tags in their repositories for more than
just designating MicroPython releases.  In those cases, the
makeversionhdr.py script would end up using a different tag than intended.
So tell `git describe` to only match tags that look like a MicroPython
version tag, such as `v1.12` or `v2.0`.
2020-10-01 11:01:43 +10:00
Jeff Epler
a4cc3ad6cb canio: RemoteTransmissionRequest: Split implementation, keep one structure
This already begins obscuring things, because now there are two sets of
shared-module functions for manipulating the same structure, e.g.,
common_hal_canio_remote_transmission_request_get_id and
common_hal_canio_message_get_id
2020-09-28 17:22:00 -05:00
Damien George
50e34f979c py/objarray.h: Add mp_obj_memoryview_init() helper function.
Signed-off-by: Damien George <damien@micropython.org>
2020-09-25 12:23:11 +10:00
Iyassou Shimels
ca017841d6 py/objstr: Make bytes(bytes_obj) return bytes_obj.
Calling the bytes constructor on a bytes object returns the original bytes
object.  This saves allocating a new instance, and matches CPython.

Signed-off-by: Iyassou Shimels <s.iyassou@gmail.com>
2020-09-24 11:04:58 +10:00
Scott Shawcroft
a8558a48ed
Merge pull request #3456 from jepler/qstr-and-or-demagic
makeqstrdefs: don't make _and_, _or_ poisoned substrings for QSTRs
2020-09-23 12:24:02 -07:00
Jeff Epler
28e80e47d7 makeqstrdefs: don't make _and_, _or_ poisoned substrings for QSTRs
New contributor @mdroberts1243 encountered an interesting problem in
which the argument they had named "column_underscore_and_page_addressing"
simply couldn't be used; I discovered that internally this had been
transformed into "column_underscore∧page_addressing", because QSTR
makes _ENTITY_ stand for the same thing as &ENTITY; does in HTML.

This might be nice for some things, but we don't want it here!
I was unable to find a sensible way to "escape" and prevent this entity
coding, so instead I ripped out support for the _and_ and _or_ escapes.
2020-09-22 17:39:44 -05:00
Jeff Epler
4869dbdc67 canio: rename from _canio
This reflects our belief that the API is stable enough to avoid incompatible changes during 6.x.
2020-09-21 16:44:26 -05:00
Jeff Epler
a2e1867f69 _canio: Minimal implementation for SAM E5x MCUs
Tested & working:

 * Send standard packets
 * Receive standard packets (1 FIFO, no filter)

Interoperation between SAM E54 Xplained running this tree and
MicroPython running on STM32F405 Feather with an external
transceiver was also tested.

Many other aspects of a full implementation are not yet present,
such as error detection and recovery.
2020-09-21 16:44:26 -05:00
Jeff Epler
e7a213a114 py: Add enum helper code
This makes it much easier to implement enums, and the printing code is
shared.  We might want to convert other enums to this in the future.
2020-09-21 16:44:26 -05:00
Jeff Epler
0318eb359f makeqstrdata: Work around python3.6 compatibility problem
Discord user Folknology encountered a problem building with Python 3.6.9,
`TypeError: ord() expected a character, but string of length 0 found`.

I was able to reproduce the problem using Python3.5*, and discovered that
the meaning of the regular expression `"|."` had changed in 3.7.  Before,
```
>>> [m.group(0) for m in re.finditer("|.", "hello")]
['', '', '', '', '', '']
```
After:
```
>>> [m.group(0) for m in re.finditer("|.", "hello")]
['', 'h', '', 'e', '', 'l', '', 'l', '', 'o', '']
```
Check if `words` is empty and if so use `"."` as the regular expression
instead.  This gives the same result on both versions:
```
['h', 'e', 'l', 'l', 'o']
```
and fixes the generation of the huffman dictionary.

Folknology verified that this fix worked for them.

 * I could easily install 3.5 but not 3.6.  3.5 reproduced the same problem
2020-09-21 10:03:07 -05:00
Jeff Epler
bfbbbd6c5c makeqstrdata: Work with older Python
This construct (which I added without sufficient testing,
apparently) is only supported in Python 3.7 and newer.  Make it
optional so that this script works on other Python versions.  This
means that if you have a system with non-UTF-8 encoding you will
need to use Python 3.7.

In particular, this affects a problem building circuitpython in
github's ubuntu-18.04 virtual environment when Python 3.7 is not
explicitly installed.  cookie-cuttered libraries call for Python
3.6:
```
    - name: Set up Python 3.6
      uses: actions/setup-python@v1
      with:
        python-version: 3.6
```
Since CircuitPython's own build calls for 3.8, this problem was not
detected.

This problem was also encountered by discord user mdroberts1243.

The failure I encountered was here:
https://github.com/jepler/Jepler_CircuitPython_udecimal/runs/1138045020?check_suite_focus=true
.. while my step of "clone and build circuitpython unix port" is
unusual, I think the same problem would have affected "build assets"
if that step had been reached.
2020-09-19 10:16:13 -05:00
Jim Mussared
9d1983f078 py/dynruntime.h: Add mp_import_* and mp_load/store_*.
These functions already exist in the fun table, and this commit just adds
convenience macros for them.
2020-09-18 18:34:02 +10:00
Damien George
8f20cdc353 all: Rename absolute time-based functions to include "epoch".
For time-based functions that work with absolute time there is the need for
an Epoch, to set the zero-point at which the absolute time starts counting.
Such functions include time.time() and filesystem stat return values.  And
different ports may use a different Epoch.

To make it clearer what functions use the Epoch (whatever it may be), and
make the ports more consistent with their use of the Epoch, this commit
renames all Epoch related functions to include the word "epoch" in their
name (and remove references to "2000").

Along with this rename, the following things have changed:

- mp_hal_time_ns() is now specified to return the number of nanoseconds
  since the Epoch, rather than since 1970 (but since this is an internal
  function it doesn't change anything for the user).

- littlefs timestamps on the esp8266 have been fixed (they were previously
  off by 30 years in nanoseconds).

Otherwise, there is no functional change made by this commit.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-18 17:20:34 +10:00
Scott Shawcroft
750bc1e04a
Merge pull request #3398 from jepler/better-dictionary-compression
compression: Implement @ciscorn's dictionary approach
2020-09-16 11:10:22 -07:00
Jeff Epler
a8e98cda83 makeqstrdata: comment my understanding of @ciscorn's code 2020-09-16 08:28:15 -05:00
Kamil Tomaszewski
c2fc592c2c camera: Change API 2020-09-14 13:11:15 +02:00
Kamil Tomaszewski
064c597b60 camera: Implement new library for camera 2020-09-14 13:11:15 +02:00
Jeff Epler
90f7340bfc move implicit-fallthrough warning enable to defns.mk 2020-09-13 13:13:09 -05:00
Taku Fukada
d18d79ac47 Small improvements to the dictionary compression 2020-09-14 01:50:01 +09:00
Jeff Epler
15964a4750 makeqstrdata: Avoid encoding problems
Most users and the CI system are running in configurations where Python
configures stdout and stderr in UTF-8 mode.  However, Windows is different,
setting values like CP1252.  This led to a build failure on Windows, because
makeqstrdata printed Unicode strings to its stdout, expecting them to be
encoded as UTF-8.

This script is writing (stdout) to a compiler input file and potentially
printing messages (stderr) to a log or console.  Explicitly configure stdout to
use utf-8 to get consistent behavior on all platforms, and configure stderr so
that if any log/diagnostic messages are printed that cannot be displayed
correctly, they are still displayed instead of creating an error while trying
to print the diagnostic information.

I considered setting the encodings both to ascii, but this would just be
occasionally inconvenient to developers like me who want to show diagnostic
info on stderr and in comments while working with the compression code.

Closes: #3408
2020-09-12 19:43:08 -05:00
Jeff Epler
12d826d941 Add FALLTHROUGH comments as needed
I investigated these cases and confirmed that the fallthrough behavior
was intentional.
2020-09-12 15:11:29 -05:00
Jeff Epler
54d97251fe modstruct: Improve compliance with python3
While checking whether we can enable -Wimplicit-fallthrough, I encountered
a diagnostic in mp_binary_set_val_array_from_int which led to discovering
the following bug:
```
>>> struct.pack("xb", 3)
b'\x03\x03'
```
That is, the next value (3) was used as the value of a padding byte, while
standard Python always fills "x" bytes with zeros.  I initially thought
this had to do with the unintentional fallthrough, but it doesn't.
Instead, this code would relate to an array.array with a typecode of
padding ('x'), which is ALSO not desktop Python compliant:
```
>>> array.array('x', (1, 2, 3))
array('x', [1, 0, 0])
```
Possibly this is dead code that used to be shared between struct-setting
and array-setting, but it no longer is.

I also discovered that the argument list length for struct.pack
and struct.pack_into were not checked, and that the length of binary data
passed to array.array was not checked to be a multiple of the element
size.

I have corrected all of these to conform more closely to standard Python
and revised some tests where necessary.  Some tests for micropython-specific
behavior that does not conform to standard Python and is not present
in CircuitPython was deleted outright.
2020-09-12 14:07:23 -05:00
Jeff Epler
40ab5c6b21 compression: Implement ciscorn's dictionary approach
Massive savings.  Thanks so much @ciscorn for providing the initial
code for choosing the dictionary.

This adds a bit of time to the build, both to find the dictionary
but also because (for reasons I don't fully understand), the binary
search in the compress() function no longer worked and had to be
replaced with a linear search.

I think this is because the intended invariant is that for codebook
entries that encode to the same number of bits, the entries are ordered
in ascending value.  However, I mis-placed the transition from "words"
to "byte/char values" so the codebook entries for words are in word-order
rather than their code order.

Because this price is only paid at build time, I didn't care to determine
exactly where the correct fix was.

I also commented out a line to produce the "estimated total memory size"
-- at least on the unix build with TRANSLATION=ja, this led to a build
time KeyError trying to compute the codebook size for all the strings.
I think this occurs because some single unicode code point ('ァ') is
no longer present as itself in the compressed strings, due to always
being replaced by a word.

As promised, this seems to save hundreds of bytes in the German translation
on the trinket m0.

Testing performed:
 - built trinket_m0 in several languages
 - built and ran unix port in several languages (en, de_DE, ja) and ran
   simple error-producing codes like ./micropython -c '1/0'
2020-09-12 10:10:45 -05:00
Damien George
acdb0608b7 py/parse: Pass in an mp_print_t to mp_parse_node_print.
So the output can be redirected if needed.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 23:00:03 +10:00
Damien George
85f2b239d8 py/showbc: Pass in an mp_print_t struct to all bytecode-print functions.
So the output can be redirected if needed.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 17:22:28 +10:00
stijn
2e54d9d146 py: Fix handling of NaN in certain pow implementations.
Adds a new compile-time option MICROPY_PY_MATH_POW_FIX_NAN for use with
toolchains that don't handle pow-of-NaN correctly.
2020-09-11 10:04:57 +10:00
Damien George
8d5a40c86e py/objfloat: Fix handling of negative float to power of nan.
Prior to this commit, pow(-2, float('nan')) would return (nan+nanj), or
raise an exception on targets that don't support complex numbers.  This is
fixed to return simply nan, as CPython does.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 10:03:57 +10:00
Scott Shawcroft
1ba28b3edc
Merge pull request #3370 from jepler/compression-bigrams
add bigram compression to makeqstrdata (save ~100 bytes on trinket m0 de_DE)
2020-09-10 11:44:56 -07:00
Scott Shawcroft
683462c1b1
Merge pull request #3326 from tannewt/native_wifi
Add native wifi API with ESP32S2 support
2020-09-10 11:20:44 -07:00
Jeff Epler
bdb07adfcc translations: Make decompression clearer
Now this gets filled in with values e.g., 128 (0x80) and 159 (0x9f).
2020-09-08 19:07:53 -05:00
Jeff Epler
73858ea682 circuitpy_mpconfig: enable 3-arg pow() with CIRCUITPY_FULL_BUILD
This is needed for a port of python3's decimal.py module.
2020-09-06 10:07:57 -05:00
Jeff Epler
20c2dd0c08 core: add int.bit_length() when MICROPY_CYPTHON_COMPAT is enabled
This method of integer objects is needed for a port of python3's
decimal.py module.

MICROPY_CPYTHON_COMPAT is enabled by CIRCUITPY_FULL_BUILD.
2020-09-06 09:53:16 -05:00
Scott Shawcroft
96cf60fbbd
Merge remote-tracking branch 'adafruit/main' into native_wifi 2020-09-03 16:34:56 -07:00
Scott Shawcroft
0b94638aeb
Changes based on Dan's feedback 2020-09-03 16:32:12 -07:00
stijn
40ad8f1666 all: Rename "sys" module to "usys".
This is consistent with the other 'micro' modules and allows implementing
additional features in Python via e.g. micropython-lib's sys.

Note this is a breaking change (not backwards compatible) for ports which
do not enable weak links, as "import sys" must now be replaced with
"import usys".
2020-09-04 00:10:24 +10:00
Jeff Epler
cbfd38d1ce Rename functions to encode_ngrams / decode_ngrams 2020-09-02 19:09:23 -05:00
Jeff Epler
c34cb82ecb makeqstrdata: correct range of low code points to 0x80..0x9f inclusive
The previous range was unintentionally big and overlaps some characters
we'd like to use (and also 0xa0, which we don't intentionally use)
2020-09-02 15:52:02 -05:00
Damien George
b0932fcf2e all: Bump version to 1.13.
Signed-off-by: Damien George <damien@micropython.org>
2020-09-02 12:01:26 +10:00
Jeff Epler
07740d19f3 add bigram compression to makeqstrdata
Compress common unicode bigrams by making code points in the range
0x80 - 0xbf (inclusive) represent them.  Then, they can be greedily
encoded and the substituted code points handled by the existing Huffman
compression.  Normally code points in the range 0x80-0xbf are not used
in Unicode, so we stake our own claim.  Using the more arguably correct
"Private Use Area" (PUA) would mean that for scripts that only use
code points under 256 we would use more memory for the "values" table.

bigram means "two letters", and is also sometimes called a "digram".
It's nothing to do with "big RAM".  For our purposes, a bigram represents
two successive unicode code points, so for instance in our build on
trinket m0 for english the most frequent are:
['t ', 'e ', 'in', 'd ', ...].

The bigrams are selected based on frequency in the corpus, but the
selection is not necessarily optimal, for these reasons I can think of:
 * Suppose the corpus was just "tea" repeated 100 times.  The
   top bigrams would be "te", and "ea".  However,
   overlap, "te" could never be used.  Thus, some bigrams might actually
   waste space
    * I _assume_ this has to be why e.g., bigram 0x86 "s " is more
      frequent than bigram 0x85 " a" in English for Trinket M0, because
      sequences like "can't add" would get the "t " digram and then
      be unable to use the " a" digram.

 * And generally, if a bigram is frequent then so are its constituents.
   Say that "i" and "n" both encode to just 5 or 6 bits, then the huffman
   code for "in" had better compress to 10 or fewer bits or it's a net
   loss!
    * I checked though!  "i" is 5 bits, "n" is 6 bits (lucky guess)
      but the bigram 0x83 also just 6 bits, so this one is a win of
      5 bits for every "it" minus overhead.  Yay, this round goes to team
      compression.
    * On the other hand, the least frequent bigram 0x9d " n" is 10 bits
      long and its constituent code points are 4+6 bits so there's no
      savings, but there is the cost of the table entry.
    * and somehow 0x9f 'an' is never used at all!

With or without accounting for overlaps, there is some optimum number
of bigrams.  Adding one more bigram uses at least 2 bytes (for the
entry in the bigram table; 4 bytes if code points >255 are in the
source text) and also needs a slot in the Huffman dictionary, so
adding bigrams beyond the optimim number makes compression worse again.

If it's an improvement, the fact that it's not guaranteed optimal
doesn't seem to matter too much.  It just leaves a little more fruit
for the next sweep to pick up.  Perhaps try adding the most frequent
bigram not yet present, until it doesn't improve compression overall.

Right now, de_DE is again the "fullest" build on trinket_m0.  (It's
reclaimed that spot from the ja translation somehow)  This change saves
104 bytes there, increasing free space about 6.8%.  In the larger
(but not critically full) pyportal build it saves 324 bytes.

The specific number of bigrams used (32) was chosen as it is the max
number that fit within the 0x80..0xbf range.  Larger tables would
require the use of 16 bit code points in the de_DE build, losing savings
overall.

(Side note: The most frequent letters in English have been said
to be: ETA OIN SHRDLU; but we have UAC EIL MOPRST in our corpus)
2020-09-01 17:12:22 -05:00
Scott Shawcroft
f0e60da51f
Merge pull request #3310 from dhalbert/ble_hci
_bleio HCI implementation
2020-09-01 11:28:05 -07:00
Dan Halbert
6dbd369272 merge from upstream 2020-08-30 14:39:03 -04:00
Dan Halbert
b27d511251 address review; use constructor for HCI Adapter 2020-08-30 14:06:48 -04:00
Jeff Epler
455226ffde builtinimport: Fix a crash with 'import ulab.linalg' on unix port only
A crash like the following occurs in the unix port:
```
Program received signal SIGSEGV, Segmentation fault.
0x00005555555a2d7a in mp_obj_module_set_globals (self_in=0x55555562c860 <ulab_user_cmodule>, globals=0x55555562c840 <mp_module_ulab_globals>) at ../../py/objmodule.c:145
145	    self->globals = globals;
(gdb) up
#1  0x00005555555b2781 in mp_builtin___import__ (n_args=5, args=0x7fffffffdbb0) at ../../py/builtinimport.c:496
496	                mp_obj_module_set_globals(outer_module_obj,
(gdb)
#2  0x00005555555940c9 in mp_import_name (name=824, fromlist=0x555555621f10 <mp_const_none_obj>, level=0x1) at ../../py/runtime.c:1392
1392	    return mp_builtin___import__(5, args);
```

I don't understand how it doesn't happen on the embedded ports, because
the module object should reside in ROM and the assignment of self->globals
should trigger a Hard Fault.

By checking VERIFY_PTR, we know that the pointed-to data is on the heap
so we can do things like mutate it.
2020-08-30 11:09:49 -05:00
Damien George
06659077a8 all: Update Python code to conform to latest black formatting.
Updating to Black v20.8b1 there are two changes that affect the code in
this repository:

- If there is a trailing comma in a list (eg [], () or function call) then
  that list is now written out with one line per element.  So remove such
  trailing commas where the list should stay on one line.

- Spaces at the start of """ doc strings are removed.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-29 15:18:01 +10:00
Scott Shawcroft
767ca5c3dc
Merge remote-tracking branch 'adafruit/main' into native_wifi 2020-08-27 11:42:31 -07:00
Jeff Epler
2e0a109331
Merge pull request #3318 from jepler/interrupt-serial-rx
supervisor: check for interrupt during rx_chr
2020-08-25 21:01:33 -05:00
Scott Shawcroft
8b71e26abd
Merge remote-tracking branch 'adafruit/main' into native_wifi 2020-08-25 16:39:23 -07:00
Jeff Epler
c0753c1afb mp_obj_print_helper: Handle a ctrl-c that comes in during printing
In #2689, hitting ctrl-c during the printing of an object with a lot of sub-objects could cause the screen to stop updating (without showing a KeyboardInterrupt).  This makes the printing of such objects acutally interruptable, and also correctly handles the KeyboardInterrupt:

```
>>> l = ["a" * 100] * 200
>>> l
['aaaaaaaaaaaaaaaaaaaaaa...aaaaaaaaaaa', Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyboardInterrupt:
>>>
```
2020-08-25 11:47:50 -05:00
Damien George
ee50a6effe py/mphal.h: Introduce mp_hal_time_ns and implement on various ports.
This should return a 64-bit value being the number of nanoseconds since
1970/1/1.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 16:13:44 +10:00
Damien George
5f9b105244 py/runtime: Fix builtin compile() in "single" mode so it prints exprs.
As per CPython behaviour, compile(stmt, "file", "single") should create
code which prints to stdout (via __repl_print__) the results of any
expressions in stmt.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 11:38:46 +10:00
Scott Shawcroft
701e80a025
Make socket reads interruptable 2020-08-21 11:00:02 -07:00
Dan Halbert
0e30dd8bcc merge from upstream; working; includes debug_out code for debugging via Saleae for posterity 2020-08-20 20:29:57 -04:00
Scott Shawcroft
eb8b42aff1
Add basic error handling 2020-08-19 14:23:28 -07:00
Scott Shawcroft
1034cc1217
Add espidf module. 2020-08-19 14:23:28 -07:00
Scott Shawcroft
430530c74b
SSL works until it runs out of memory 2020-08-19 14:23:28 -07:00
Scott Shawcroft
c9ece21c28
SocketPool stubbed out 2020-08-19 14:22:13 -07:00
Scott Shawcroft
3860991111
Ping work and start to add socketpool 2020-08-19 14:22:13 -07:00
Scott Shawcroft
c53a72d3f5
Fix ipaddress import and parse ipv4 strings 2020-08-19 14:22:13 -07:00
Scott Shawcroft
c62ab6e09a
Add ipaddress 2020-08-19 14:22:12 -07:00
Scott Shawcroft
1a6f4e0fe0
Scanning WIP. Need to sort out supervisor memory 2020-08-19 14:22:12 -07:00
Scott Shawcroft
c5b8401a15
First crack at native wifi API 2020-08-19 14:21:59 -07:00
Scott Shawcroft
6857f98426
Split pulseio.PWMOut into pwmio
This gives us better granularity when implementing new ports because
PWMOut is commonly implemented before PulseIn and PulseOut.

Fixes #3211
2020-08-18 13:08:33 -07:00
Scott Shawcroft
24ca5c0218
Merge pull request #3295 from tannewt/turn_off_terminalio
Turn off terminalio for ja and ko
2020-08-18 12:10:31 -07:00
Taku Fukada
79a3796b1c Calculate the Huffman codebook without MP_QSTRs 2020-08-18 23:21:14 +09:00
Scott Shawcroft
d01f5dc0bd
Turn off terminalio for ja and ko
The font is missing many characters and the build needs the space.
We can optimize font storage when we get a good font.

The serial output will work as usual.
2020-08-17 17:17:59 -07:00
Jeff Epler
08ed09acc6 makeqstrdata: don't print "compression incrased length" messages
This check as implemented is misleading, because it compares the
compressed size in bytes (including the length indication) with the source
string length in Unicode code points.  For English this is approximately
fair, but for Japanese this is quite unfair and produces an excess of
"increased length" messages.

This message might have existed for one of two reasons:
 * to alert to an improperly function huffman compression
 * to call attention to a need for a "string is stored uncompressed" case
We know by now that the huffman compression is functioning as designed and
effective in general.

Just to be on the safe side, I did some back-of-the-envelope estimates.
I considered these three replacements for "the true source string size, in bytes":
+    decompressed_len_utf8 = len(decompressed.encode('utf-8'))
+    decompressed_len_utf16 = len(decompressed.encode('utf-16be'))
+    decompressed_len_bitsize = ((1+len(decompressed)) * math.ceil(math.log(1+len(values), 2)) + 7) // 8

The third counts how many bits each character requires (fewer than 128
characters in the source character set = 7, fewer than 256 = 8, fewer than 512
= 9, etc, adding a string-terminating value) and is in some way representative
of the best way we would be able to store "uncompressed strings".  The Japanese
translation (largest as of writing) has just a few strings which increase by
this metric.  However, the amount of loss due to expansion in those cases is
outweighed by the cost of adding 1 bit per string to indicate whether it's
compressed or not.  For instance, in the BOARD=trinket_m0 TRANSLATION=ja build
the loss is 47 bytes over 300 strings.  Adding 1 bit to each of 300 strings will
cost about 37 bytes, leaving just 5 Thumb instructions to implement the code to
check and decode "uncompressed" strings in order to break even.
2020-08-16 20:50:48 -05:00
Jeff Epler
cff448205f Don't define SHARPDISPLAY when !DISPLAYIO
.. even if FULL_BUILD
2020-08-12 07:39:28 -05:00
Jeff Epler
c1400bae9b sharpmemory: Implement support for Sharp Memory Displays in framebufferio 2020-08-12 07:32:18 -05:00
Jeff Epler
93b373d617 "pop from empty %q"
Saves 12 bytes code on trinket m0
2020-08-04 18:42:09 -05:00
Jeff Epler
65e26f4a06 py: mp_obj_get_type_qstr as macro saves 24 bytes 2020-08-04 14:45:45 -05:00
Jeff Epler
024c8da578 Combine some "can't convert" messages 2020-08-04 14:45:45 -05:00
Jeff Epler
c849b781c0 Combine 'index out of range' messages 2020-08-04 14:45:45 -05:00
Jeff Epler
89797fd3f9 various: Use mp_obj_get_type_qstr more widely
This removes runtime allocations of the cstring version of the qstring.

It is not a size improvement
2020-08-04 14:45:45 -05:00
Jeff Epler
c37a25f0e5 Use qstrs to save an additional 4 bytes 2020-08-04 14:45:45 -05:00
Jeff Epler
92917b84f1 fix exception type for pop from empty set 2020-08-04 13:58:29 -05:00
Jeff Epler
67eb93fc98 py: introduce, use mp_raise_msg_vlist
This saves a very small amount of flash, 8 bytes on trinket_m0
2020-08-04 13:34:29 -05:00
Jeff Epler
dddd25a776 Combine similar strings to reduce size of translations
This is a slight trade-off with code size, in places where a "_varg"
mp_raise variant is now used.  The net savings on trinket_m0 is
just 32 bytes.

It also means that the translation will include the original English
text, and cannot be translated.  These are usually names of Python
types such as int, set, or dict or special values such as "inf" or
"Nan".
2020-08-04 13:34:29 -05:00
Dan Halbert
0a60aee3e4 wip: compiles 2020-08-02 11:36:38 -04:00
Damien George
9883d8e818 py/persistentcode: Maintain root ptr list of imported native .mpy code.
On ports where normal heap memory can contain executable code (eg ARM-based
ports such as stm32), native code loaded from an .mpy file may be reclaimed
by the GC because there's no reference to the very start of the native
machine code block that is reachable from root pointers (only pointers to
internal parts of the machine code block are reachable, but that doesn't
help the GC find the memory).

This commit fixes this issue by maintaining an explicit list of root
pointers pointing to native code that is loaded from an .mpy file.  This
is not needed for all ports so is selectable by the new configuration
option MICROPY_PERSISTENT_CODE_TRACK_RELOC_CODE.  It's enabled by default
if a port does not specify any special functions to allocate or commit
executable memory.

A test is included to test that native code loaded from an .mpy file does
not get reclaimed by the GC.

Fixes #6045.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-02 22:34:09 +10:00
Jeff Epler
d69f081c04 Merge remote-tracking branch 'origin/main' into blm_badge 2020-07-30 07:24:48 -05:00
Scott Shawcroft
61d1148bb3
Merge pull request #3222 from WarriorOfWire/pick_micropython
py/compile: Don't await __aiter__ special method in async-for.
2020-07-29 10:54:37 -07:00
Jeff Epler
9b8df7f635 Upgrade ulab
This version
 * moves source files to reflect module structure
 * adds inline documentation suitable for extract_pyi
 * incompatibly moves spectrogram to fft
 * incompatibly removes "extras"

There are some remaining markup errors in the specific revision of
extmod/ulab but they do not prevent the doc building process from
completing.
2020-07-28 16:57:48 -05:00
Dan Halbert
aa97ea2501 Merge remote-tracking branch 'adafruit/main' into blm_badge 2020-07-28 14:15:02 -04:00
Dan Halbert
e5e132a364 add blm_badge; add CIRCUITPY_AUDIOBUSIOIO_I2SOUT 2020-07-28 11:49:54 -04:00
Jerry Needell
2bdd62220e adjust stack for SAMD21 to accomodate larger pystack -- update frozen module adafruit_busdevice 2020-07-27 21:50:36 -04:00
Jonathan Hogg
901f3dce6e py/compile: Don't await __aiter__ special method in async-for.
MicroPython's original implementation of __aiter__ was correct for an
earlier (provisional) version of PEP492 (CPython 3.5), where __aiter__ was
an async-def function.  But that changed in the final version of PEP492 (in
CPython 3.5.2) where the function was changed to a normal one.  See
https://www.python.org/dev/peps/pep-0492/#why-aiter-does-not-return-an-awaitable
See also the note at the end of this subsection in the docs:
https://docs.python.org/3.5/reference/datamodel.html#asynchronous-iterators
And for completeness the BPO: https://bugs.python.org/issue27243

To be consistent with the Python spec as it stands today (and now that
PEP492 is final) this commit changes MicroPython's behaviour to match
CPython:  __aiter__ should return an async-iterable object, but is not
itself awaitable.

The relevant tests are updated to match.

See #6267.
2020-07-24 22:55:37 -07:00
Kenny
9a1f1236cc require async for and async with to actually be in an async def method instead of just a generator 2020-07-24 19:47:34 -07:00
Jonathan Hogg
37e1b5c891 py/compile: Don't await __aiter__ special method in async-for.
MicroPython's original implementation of __aiter__ was correct for an
earlier (provisional) version of PEP492 (CPython 3.5), where __aiter__ was
an async-def function.  But that changed in the final version of PEP492 (in
CPython 3.5.2) where the function was changed to a normal one.  See
https://www.python.org/dev/peps/pep-0492/#why-aiter-does-not-return-an-awaitable
See also the note at the end of this subsection in the docs:
https://docs.python.org/3.5/reference/datamodel.html#asynchronous-iterators
And for completeness the BPO: https://bugs.python.org/issue27243

To be consistent with the Python spec as it stands today (and now that
PEP492 is final) this commit changes MicroPython's behaviour to match
CPython:  __aiter__ should return an async-iterable object, but is not
itself awaitable.

The relevant tests are updated to match.

See #6267.
2020-07-25 00:58:18 +10:00
Kenny
e9b4e0bd35 remove new char*s because m0 is way oversubscribed 2020-07-23 20:41:10 -07:00
Kenny
51a79b1af7 add coroutine behavior for generators
coroutines don't have __next__; they also call themselves coroutines.
This does not change the fact that `async def` methods are generators,
but it does make them behave more like CPython.
2020-07-23 20:40:16 -07:00
Dan Halbert
f6f45c82a1 wip: ATT protocol 2020-07-23 18:54:26 -04:00
Damien George
f43834aba2
py/py.mk: Use additional CFLAGS to compile string0.c.
Otherwise functions like memset might get optimised to call themselves (eg
with gcc 10).  And provide CFLAGS_BUILTIN so these options can be changed
by a port if needed.

Fixes issue #6053.
2020-07-22 16:26:47 -07:00
Scott Shawcroft
2bd6d05663
Add externs. GCC10 complains about duplicate defines 2020-07-22 16:26:46 -07:00
Scott Shawcroft
1ec3580946
Merge branch 'main' into memmonitor 2020-07-22 12:24:19 -07:00
Scott Shawcroft
6db10f9c7d
Turn off find when CPYTHON_COMPAT is off 2020-07-21 15:40:51 -07:00
Scott Shawcroft
1b3bc98068
Merge remote-tracking branch 'adafruit/main' into bytearray_find 2020-07-21 15:12:04 -07:00
Scott Shawcroft
cb7df2e1ed
Merge pull request #3160 from tannewt/enable_pystack
Enable PYSTACK to keep function state out of the heap
2020-07-21 13:38:52 -07:00
Scott Shawcroft
900edb2b8e
Only add .find without CPYTHON_COMPAT 2020-07-21 13:08:42 -07:00
Damien George
a853fff838 py/obj.h: Fix mp_seq_replace_slice_no_grow to use memmove not memcpy.
Because the argument arrays may overlap, as show by the new tests in this
commit.

Also remove the debugging comments for these macros, add a new comment
about overlapping regions, and separate the macros by blank lines to make
them easier to read.

Fixes issue #6244.

Signed-off-by: Damien George <damien@micropython.org>
2020-07-21 01:05:59 +10:00
Scott Shawcroft
f38a9c8d29
Add cast for mpy-cross warning 2020-07-17 17:55:06 -07:00
Scott Shawcroft
a1e4814a27
Get AllocationAlarm working 2020-07-17 17:15:03 -07:00
Scott Shawcroft
518d909b2c
Add memorymonitor module 2020-07-17 17:15:03 -07:00
Scott Shawcroft
4b157aa6b8
Add find varients to bytearray 2020-07-17 17:07:37 -07:00
Scott Shawcroft
1160635608
Enable PYSTACK to keep function state out of the heap 2020-07-17 17:03:42 -07:00
DavePutz
e5f7adcf5d
Fix to pass mpy-cross build 2020-07-13 22:54:52 -05:00
DavePutz
14799f9ee6
more formatting fix 2020-07-13 22:43:46 -05:00
DavePutz
30361f6f2a
Fix formatting 2020-07-13 22:39:30 -05:00
DavePutz
e963cff72d
Issue #2949 Run background checks during long multiplications 2020-07-13 22:27:06 -05:00
Scott Shawcroft
51c888d4be
Merge pull request #3003 from Flameeyes/master
License tagging according to REUSE specifications.
2020-07-13 16:28:49 -07:00
Jeff Epler
6d97f6fccc audioio: Remove compatibility code
These items were aliased from audiocore to audioio for compatibility
with 4.x, but according to our deprecation schedule can be removed
in 6.0.
2020-07-08 20:31:35 -05:00
Dan Halbert
a5ab2829eb use zephyr include files; wip: compiles 2020-07-08 09:33:41 -04:00
Diego Elio Pettenò
34b4993d63 Add license to some obvious files. 2020-07-06 19:16:25 +01:00
Dan Halbert
11cb3e3b4b hci skeleton done; not working yet 2020-06-30 23:19:40 -04:00
Damien George
332d83343f py: Rework mp_convert_member_lookup to properly handle built-ins.
This commit fixes lookups of class members to make it so that built-in
functions that are used as methods/functions of a class work correctly.

The mp_convert_member_lookup() function is pretty much completely changed
by this commit, but for the most part it's just reorganised and the
indenting changed.  The functional changes are:

- staticmethod and classmethod checks moved to later in the if-logic,
  because they are less common and so should be checked after the more
  common cases.

- The explicit mp_obj_is_type(member, &mp_type_type) check is removed
  because it's now subsumed by other, more general tests in this function.

- MP_TYPE_FLAG_BINDS_SELF and MP_TYPE_FLAG_BUILTIN_FUN type flags added to
  make the checks in this function much simpler (now they just test this
  bit in type->flags).

- An extra check is made for mp_obj_is_instance_type(type) to fix lookup of
  built-in functions.

Fixes #1326 and #6198.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-30 23:55:32 +10:00
Damien George
d06ae1d2b1 py/obj.h: Make existing MP_TYPE_FLAG_xxx macros sequential.
There's no reason to have them non-sequential, this was likely a typo from
commit 9ec1caf42e.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-30 23:55:32 +10:00
Dan Halbert
1bc2e979eb wip; compiles; much commented out 2020-06-26 17:23:20 -04:00
Jeff Epler
05837b2841 sdioio: Add shared-bindings
There is no implementation yet.
2020-06-26 11:50:25 -05:00
Jeff Epler
57fde2e07b sdcardio: implement new library for SD card I/O
Testing performed: That a card is successfully mounted on Pygamer with
the built in SD card slot

This module is enabled for most FULL_BUILD boards, but is disabled for
samd21 ("M0"), litex, and pca10100 for various reasons.
2020-06-26 11:50:23 -05:00
Damien George
9f911d822e py/objcomplex: Add mp_obj_get_complex_maybe for use in complex bin-op.
This allows complex binary operations to fail gracefully with unsupported
operation rather than raising an exception, so that special methods work
correctly.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-27 01:03:10 +10:00
Damien George
41fa8b5482 py/emitnative: Implement binary operations for viper uint operands.
uint types in viper mode can now be used for all binary operators except
floor-divide and modulo.

Fixes issue #1847 and issue #6177.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-27 00:24:04 +10:00
Damien George
b3b8706d27 py/asm: Add condition codes for signed comparisons.
Signed-off-by: Damien George <damien@micropython.org>
2020-06-27 00:24:04 +10:00
Damien George
aa26fe62d8 py/asm: Add funcs/macros to emit machine code for logical-shift-right.
Signed-off-by: Damien George <damien@micropython.org>
2020-06-27 00:24:04 +10:00
Dan Halbert
759929c24a hci early wip; refactor supervisor bluetooth.c for nrf: tested 2020-06-25 20:57:17 -04:00
Jeff Epler
1d2cc0b968 I2CPeripheral: Rename class and its module
This is an incompatible change.
2020-06-25 11:44:19 -05:00
Kamil Tomaszewski
84f424f631 gnss: Implement new library for GNSS 2020-06-24 11:14:44 +02:00
Damien George
457fdf61c3 py/objtype: Support passing in an OrderedDict to type() as the locals.
An OrderedDict can now be used for the locals when creating a type
explicitly via type(name, bases, locals).

Signed-off-by: Damien George <damien@micropython.org>
2020-06-24 12:05:40 +10:00
Damien George
456a3abe8d py/obj.h: Add public mp_obj_is_dict_or_ordereddict() helper macro.
And use it in py/objdict.c instead of mp_obj_is_dict_type.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-24 12:05:00 +10:00
Jeff Epler
a7b36f724f circuitpy_defns.mk: Sort several lists of source files
and regularize whitespace in one spot
2020-06-22 10:45:27 -05:00
Jeff Epler
a580f0f1c4 _pew: move to common-hal
I noticed that this code was referring to samd-specific functionality,
and isn't enabled except in one samd board (pewpew10).  Move it.

There is incomplte support for _pew in mimxrt10xx which then caused build
errors; adding a #if guard to check for _pew being enabled fixes it.
The _pew module is not likely to be important on mimxrt but I'll leave the
choice to remove it to someone else.
2020-06-22 10:45:27 -05:00
Damien George
6164c7e666 py/misc.h: Add missing semi-colon in mp_float_union_t for big-endian.
Fixes issue #6161.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-22 13:42:24 +10:00
Damien George
f2e267da68 py/compile: Implement PEP 526, syntax for variable annotations.
This addition to the grammar was introduced in Python 3.6.  It allows
annotating the type of a varilable, like:

    x: int = 123
    s: str

The implementation in this commit is quite simple and just ignores the
annotation (the int and str bits above).  The reason to implement this is
to allow Python 3.6+ code that uses this feature to compile under
MicroPython without change, and for users to use type checkers.

In the future viper could use this syntax as a way to give types to
variables, which is currently done in a bit of an ad-hoc way, eg
x = int(123).  And this syntax could potentially be used in the inline
assembler to define labels in an way that's easier to read.
2020-06-16 23:18:01 +10:00
Damien George
131b0de70a py/grammar.h: Consolidate duplicate sub-rules for :test and =test. 2020-06-16 23:18:01 +10:00
Damien George
1783950311 py/compile: Implement PEP 572, assignment expressions with := operator.
The syntax matches CPython and the semantics are equivalent except that,
unlike CPython, MicroPython allows using := to assign to comprehension
iteration variables, because disallowing this would take a lot of code to
check for it.

The new compile-time option MICROPY_PY_ASSIGN_EXPR selects this feature and
is enabled by default, following MICROPY_PY_ASYNC_AWAIT.
2020-06-16 22:02:24 +10:00
Damien George
0fd91e39b1 py/compile: Convert scope test to SCOPE_IS_COMP_LIKE macro.
This macro can be used elsewhere.
2020-06-16 21:42:37 +10:00
David Lechner
ecd7826316 tools/codeformat.py: Remove sizeof fixup.
Formatting for `* sizeof` was fixed in uncrustify v0.71, so we no longer
need the fixups for it.  Also, there was one file where the updated
uncrustify caught a problem that the regex didn't pick up, which is updated
in this commit.

Signed-off-by: David Lechner <david@pybricks.com>
2020-06-14 20:24:18 +10:00
Damien George
f3062b5cbd py/obj.h: Clarify comments about mp_map_t is_fixed and is_ordered.
Long ago, prior to 0ef01d0a75, fixed and
ordered maps were the same setting with the "table_is_fixed_array" member
of mp_map_t.  But these settings are actually independent, and it is
possible to have is_fixed=1, is_ordered=0 (although this can currently
only be done by tools/cc1).  So update the comments to reflect this.
2020-06-10 22:27:10 +10:00
Andrew Leech
95cbe6b65e py/objtype: Use mp_obj_dict_copy() for creating obj.__dict__ attribute.
The resulting dict is now marked as read-only (is_fixed=1) to enforce the
fact that changes to this dict will not be reflected in the class instance.

This commit reduces code size by about 20 bytes, and should be more
efficient because it creates a direct copy of the dict rather than
reinserting all elements.
2020-06-10 22:03:39 +10:00
Andrew Leech
28370c0450 py/objtype: Add __dict__ attribute for class objects.
The behavior mirrors the instance object dict attribute where a copy of the
local attributes are provided (unless the dict is read-only, then that dict
itself is returned, as an optimisation).  MicroPython does not support
modifying this dict because the changes will not be reflected in the class.

The feature is only enabled if MICROPY_CPYTHON_COMPAT is set, the same as
the instance version.
2020-06-10 21:58:13 +10:00
Damien George
bd06c698f0 py/dynruntime.h: Make mp_obj_str_get_str raise if arg not a str/bytes. 2020-06-08 09:05:03 +02:00
Jim Mussared
e6881f0829 extmod/modbluetooth: Make modbluetooth event not a bitfield.
There doesn't appear to be any use for only triggering on specific events,
so it's just easier to number them sequentially.  This makes them smaller
values so they take up only 1 byte in the ringbuf, only 1 byte for the
opcode in the bytecode, and makes room for more events.

Also add a couple of new event types that need to be implemented (to avoid
re-numbering later).

And rename _COMPLETE and _STATUS to _DONE for consistency.

In the future the "trigger" keyword argument can be reinstated by requiring
the user to compute the bitmask, eg:

    ble.irq(handler, 1 << _IRQ_SCAN_RESULT | 1 << _IRQ_SCAN_DONE)
2020-06-05 14:04:20 +10:00
Diego Elio Pettenò
dd5d7c86d2 Fix up end of file and trailing whitespace.
This can be enforced by pre-commit, but correct it separately to make it easier to review.
2020-06-03 10:56:35 +01:00
Damien George
203b10703e py/modbuiltins: Fix getattr to work with class raising AttributeError.
Fixes issue #6089.
2020-06-02 15:42:20 +10:00
Scott Shawcroft
959f7297da
Merge remote-tracking branch 'adafruit/master' into wdt-nrf 2020-06-01 17:01:16 -07:00
Jeff Epler
f211a090e2 py.mk: Assume we want all C source files in ulab 2020-06-01 08:26:23 -05:00
Jeff Epler
1cc281b6a4 py.mk: Assume we want all C files from ulab 2020-06-01 08:20:23 -05:00
Scott Shawcroft
2fd7a43f2f
Merge remote-tracking branch 'adafruit/master' into wdt-nrf 2020-05-29 10:16:24 -07:00
Dan Halbert
6ace4ee7e8
Merge pull request #2968 from jepler/more-efficient-translation
More efficient translation
2020-05-28 21:37:45 -04:00
Scott Shawcroft
796373b8be
A number of small ESP32S2 fixes:
* Fix flash writes that don't end on a sector boundary. Fixes #2944
* Fix enum incompatibility with IDF.
* Fix printf output so it goes out debug UART.
* Increase stack size to 8k.
* Fix sleep of less than a tick so it doesn't crash.
2020-05-28 15:43:55 -07:00
Jeff Epler
d0f9b5901e translations: document the compressed format 2020-05-28 11:30:46 -05:00
Jeff Epler
fe3e8d1589 string compression: save a few bits per string
Length was stored as a 16-bit number always.  Most translations have
a max length far less.  For example, US English translation lengths
always fit in just 8 bits.  probably all languages fit in 9 bits.

This also has the side effect of reducing the alignment of
compressed_string_t from 2 bytes to 1.

testing performed: ran in german and english on pyruler, printed messages
looked right.

Firmware size, en_US
Before: 3044 bytes free in flash
After: 3408 bytes free in flash

Firmware size, de_DE (with #2967 merged to restore translations)
Before: 1236 bytes free in flash
After: 1600 bytes free in flash
2020-05-28 08:36:08 -05:00
David Lechner
093fd80760 py/modsys: Use consistent naming pattern for module-level const objects.
This renames a few identifiers to follow the usual naming convention of
mp_<module>_<name>.  This makes them easier to find, e.g. when grep'ing.
2020-05-28 10:02:14 +10:00
stijn
97ccde0c43 py/ringbuf: Fix compilation with msvc.
Older versions do not have "inline" so fetch the definition from
mpconfigport.h.
2020-05-28 09:56:18 +10:00
stijn
81db22f693 py/modmath: Work around msvc float bugs in atan2, fmod and modf.
Older implementations deal with infinity/negative zero incorrectly.  This
commit adds generic fixes that can be enabled by any port that needs them,
along with new tests cases.
2020-05-28 09:54:54 +10:00
Damien George
a902b69dd5 py/py.mk: Use additional CFLAGS to compile string0.c.
Otherwise functions like memset might get optimised to call themselves (eg
with gcc 10).  And provide CFLAGS_BUILTIN so these options can be changed
by a port if needed.

Fixes issue #6053.
2020-05-27 23:10:23 +10:00
Sean Cross
bd086a102e Revert "add WatchDogTimeout exception"
This reverts commit 561e7e619095869f58fc728d428f3ff20e8bfc40.
2020-05-27 11:28:49 +08:00
Sean Cross
08362c9cab watchdogtimer: refactor to new api
This refactors the WatchDogTimer API to use the format proposed in
https://github.com/adafruit/circuitpython/pull/2933#issuecomment-632268227

Signed-off-by: Sean Cross <sean@xobs.io>
2020-05-27 11:28:49 +08:00
Sean Cross
ae950bc050 add WatchDogTimeout exception
This adds an exception to be raised when the WatchDogTimer times out.

Note that this currently causes a HardFault, and it's not clear why it's
not behaving properly.

Signed-off-by: Sean Cross <sean@xobs.io>
2020-05-27 11:28:49 +08:00
Sean Cross
595f6387c2 watchdog: rename module from wdt and move to microcontroller
This also places it under the `microcontroller` object.

Signed-off-by: Sean Cross <sean@xobs.io>
2020-05-27 11:28:49 +08:00
Sean Cross
f4719609f7 wdt: add watchdog support
This adds shared bindings for a watchdog timer, based on the API
provided by micropython.

Signed-off-by: Sean Cross <sean@xobs.io>
2020-05-27 11:28:48 +08:00
Scott Shawcroft
acf4b1bede
Remove reverse methods from per-board defs 2020-05-19 11:43:57 -07:00
Scott Shawcroft
cf690bd390
Merge remote-tracking branch 'adafruit/master' into esp32s2 2020-05-18 16:46:41 -07:00
Scott Shawcroft
9811c1fe4b
Merge pull request #2896 from theacodes/add-bytearray-decode
Add bytearray.decode() for CPython compatibility
2020-05-18 14:38:45 -07:00
Scott Shawcroft
6aaab005c5
Initial ESP32S2 port.
Basic blinky works but doesn't check pins.
2020-05-15 15:36:16 -07:00
Thea Flowers
0876034adb Make decode only apply to bytearray and not array as well 2020-05-15 14:32:44 -07:00
Thea Flowers
fe3e8ee02c Add bytearray.decode() for CPython compatibility
CPython has a `decode()` method on `bytearray`. This adds that method using the code from `bytes.decode`.

Test program:

```python

byte_boi = bytes([0x6D, 0x65, 0x65, 0x70])
print(byte_boi)  # b'meep'
byte_boi_str = byte_boi.decode("utf-8")
print(byte_boi_str)  # meep

byte_array_boi = bytearray(byte_boi)
print(byte_array_boi)  # bytearray(b'meep')
byte_array_boi_str = byte_array_boi.decode("utf-8")
print(byte_array_boi_str)  # meep

print(byte_array_boi_str == byte_boi_str)  # True

```
2020-05-14 21:52:58 -07:00
Damien George
801f7dca78 py/nativeglue.h: Rename "setjmp" entry to "setjmp_" to avoid any clash.
Because some compilers may define setjmp to something.

Fixes issue #6032.
2020-05-14 21:48:05 +10:00
Scott Shawcroft
7546d47f77
Merge remote-tracking branch 'adafruit/master' into improve_verification 2020-05-13 08:23:55 -07:00
Scott Shawcroft
2efe17bcb7
Don't build in empty help.c 2020-05-12 19:01:17 -07:00
Scott Shawcroft
4612270f98
Only enable COUNTIO in full builds 2020-05-12 19:00:51 -07:00
Scott Shawcroft
277e8d528b
Merge branch 'master' into Optical-Encoder-Module 2020-05-12 18:22:57 -07:00
warriorofwire
5af59cbabe Enable vectorio by default where displayio is enabled 2020-05-12 11:46:04 -07:00
Daniel Pollard
ee2cb703c8 merged master 2020-05-12 14:41:28 +10:00
warriorofwire
cfd0de9c11 set vectorio to FULL_BUILD and see what works 2020-05-11 21:08:58 -07:00
warriorofwire
1c6e646f56 vectorio: fix mpconfig definition for disabled boards 2020-05-09 17:58:01 -07:00
warriorofwire
206d0e598a Add vectorio: for drawing shapes
vectorio builds on m4 express feather

Concrete shapes are composed into a VectorShape which is put into a displayio Group for display.

VectorShape provides transpose and x/y positioning for shape implementations.

Included Shapes:

* Circle
  - A radius; Circle is positioned at its axis in the VectorShape.
  - You can freely modify the radius to grow and shrink the circle in-place.

* Polygon
  - An ordered list of points.
  - Beteween each successive point an edge is inferred.  A final edge closing the shape is inferred between the last
    point and the first point.
  - You can modify the points in a Polygon.  The points' coordinate system is relative to (0, 0) so if you'd like a
      top-center justified 10x20 rectangle you can do points [(-5, 0), (5, 0), (5, 20), (0, 20)] and your VectorShape
      x and y properties will position the rectangle relative to its top center point

* Rectangle
  A width and a height.
2020-05-09 15:38:22 -07:00
Daniel Pollard
e947e4eb58 removed duplicate build param 2020-05-09 17:56:38 +10:00
Damien George
172fc040aa py/parse: Make mp_parse_node_extract_list return size_t instead of int.
Because this function can only return non-negative values, and having the
correct return type gives more information to the caller.
2020-05-09 00:55:44 +10:00
Damien George
f792e6c283 py/scheduler: Convert mp_sched_full and mp_sched_num_pending to macros.
So they are guaranteed to be inlined within functions like
mp_sched_schedule which may be located in a special memory region.
2020-05-08 23:20:45 +10:00
Daniel Pollard
94ca233d97
Update py/circuitpy_mpconfig.mk
Co-authored-by: Scott Shawcroft <scott@tannewt.org>
2020-05-08 07:55:11 +10:00
Daniel Pollard
84c806a4be updated descriptions and build variable 2020-05-07 12:42:46 +10:00
Sean Cross
b168784fa0 aesio: add basic AES encryption and decryption
This adds initial support for an AES module named aesio.  This
implementation supports only a subset of AES modes, namely
ECB, CBC, and CTR modes.

Example usage:

```
>>> import aesio
>>>
>>> key = b'Sixteen byte key'
>>> cipher = aesio.AES(key, aesio.MODE_ECB)
>>> output = bytearray(16)
>>> cipher.encrypt_into(b'Circuit Python!!', output)
>>> output
bytearray(b'E\x14\x85\x18\x9a\x9c\r\x95>\xa7kV\xa2`\x8b\n')
>>>
```

This key is 16-bytes, so it uses AES128.  If your key is 24- or 32-
bytes long, it will switch to AES192 or AES256 respectively.

This has been tested with many of the official NIST test vectors,
such as those used in `pycryptodome` at
39626a5b01/lib/Crypto/SelfTest/Cipher/test_vectors/AES

CTR has not been tested as NIST does not provide test vectors for it.

Signed-off-by: Sean Cross <sean@xobs.io>
2020-05-06 17:40:06 +08:00
Daniel Pollard
8961dd9fe7 changed build variables as per advice 2020-05-06 09:05:14 +10:00
Daniel Pollard
bfa5cd9c13 refactor countio based on feedback 2020-05-05 15:23:38 +10:00
Dan Halbert
bae7a5e433 make translate again 2020-05-04 17:26:59 -04:00
Damien George
4ede703687 py/parse: Support constant folding of power operator for integers.
Constant expression like "2 ** 3" will now be folded, and the special form
"X = const(2 ** 3)" will now compile because the argument to the const is
now a constant.

Fixes issue #5865.
2020-05-03 16:23:19 +10:00
Damien George
544c308c18 py/scheduler: Add option to wrap mp_sched_schedule in arbitrary attr.
So ports can put it in a special memory section if needed.
2020-04-30 23:47:11 +10:00
Dan Halbert
241d7e2ae6 change many ifndefs to ?= 2020-04-29 23:31:34 -04:00
Dan Halbert
180f5c6a94 Merge remote-tracking branch 'adafruit/master' into ringbuf-fixes 2020-04-29 22:11:22 -04:00
Scott Shawcroft
de42e4af02
Merge remote-tracking branch 'adafruit/master' into lower_power 2020-04-27 17:43:32 -07:00
Scott Shawcroft
40a1f3edbc
Merge pull request #2811 from jepler/update-ulab
ulab: actually update the submodule
2020-04-27 17:41:43 -07:00
Scott Shawcroft
755d404edf
Merge remote-tracking branch 'adafruit/master' into lower_power 2020-04-27 16:45:10 -07:00
Damien George
bd6ca15444 py/modio: Allow uio.IOBase streams to return errno for read/write error.
This allows user code that inherits from uio.IOBase to return an errno
error code from the user readinto/write function, by returning a negative
value.  Eg returning -123 means an errno of 123.  This is already how the
custom ioctl works.
2020-04-27 23:58:46 +10:00
Damien George
e08ca78f40 py/stream: Remove mp_stream_errno and use system errno instead.
This change is made for two reasons:

1. A 3rd-party library (eg berkeley-db-1.xx, axtls) may use the system
   provided errno for certain errors, and yet MicroPython stream objects
   that it calls will be using the internal mp_stream_errno.  So if the
   library returns an error it is not known whether the corresponding errno
   code is stored in the system errno or mp_stream_errno.  Using the system
   errno in all cases (eg in the mp_stream_posix_XXX wrappers) fixes this
   ambiguity.

2. For systems that have threading the system-provided errno should always
   be used because the errno value is thread-local.

For systems that do not have an errno, the new lib/embed/__errno.c file is
provided.
2020-04-27 23:58:46 +10:00
Jim Mussared
57fce3bdb2 py/objdict: Fix popitem for ordered dicts.
The popitem method wasn't implemented for ordered dicts and would result in
an invalid state.

Fixes issue #5956.
2020-04-27 23:53:17 +10:00
Jeff Epler
c016ea6f0a ulab: actually update the submodule
PR#2802 missed the submodule update itself.
2020-04-26 10:12:56 -05:00
Lucian Copeland
bd0df9e3bc Minor redundancy fix 2020-04-23 17:43:35 -04:00
Lucian Copeland
c6c77726e7 Merge remote-tracking branch 'upstream/master' into stm32-docfix 2020-04-23 13:39:48 -04:00
Lucian Copeland
8791ca6af3 implement requested changes 2020-04-23 13:33:41 -04:00
stijn
84fa3312cf all: Format code to add space after C++-style comment start.
Note: the uncrustify configuration is explicitly set to 'add' instead of
'force' in order not to alter the comments which use extra spaces after //
as a means of indenting text for clarity.
2020-04-23 11:24:25 +10:00
Lucian Copeland
d0a2106547 Remove old build flags, add fixes for shared_matrix 2020-04-22 16:06:08 -04:00
Dan Halbert
fbc8719fad ringbuf tested 2020-04-21 22:40:12 -04:00
Dan Halbert
77cd93ac2d merge from adafruit 2020-04-21 17:47:51 -04:00
Dan Halbert
38ec3bc574 further ringbuf cleanup 2020-04-21 17:38:20 -04:00
Scott Shawcroft
bebf27e733
Merge remote-tracking branch 'adafruit/master' into lower_power
This isn't perfect and needs a bit more testing.
2020-04-20 18:25:13 -07:00
Damien George
388d419ba3 py/makecompresseddata.py: Make compression deterministic.
Error string compression is not deterministic in certain cases: it depends
on the Python version (whether dicts are ordered by default or not) and
probably also the order files are passed to this script, leading to a
difference in which words are included in the top 128 most common.

The changes in this commit use OrderedDict to keep parsed lines in a known
order, and, when computing how many bytes are saved by a given word, it
uses the word itself to break ties (which would otherwise be "random").
2020-04-20 10:32:49 +10:00
stijn
f31f9a8b70 py/objint: Do not use fpclassify.
For combinations of certain versions of glibc and gcc the definition of
fpclassify always takes float as argument instead of adapting itself to
float/double/long double as required by the C99 standard.  At the time of
writing this happens for instance for glibc 2.27 with gcc 7.5.0 when
compiled with -Os and glibc 3.0.7 with gcc 9.3.0.  When calling fpclassify
with double as argument, as in objint.c, this results in an implicit
narrowing conversion which is not really correct plus results in a warning
when compiled with -Wfloat-conversion.  So fix this by spelling out the
logic manually.
2020-04-18 22:42:24 +10:00
stijn
70affd9ba2 all: Fix implicit floating point to integer conversions.
These are found when building with -Wfloat-conversion.
2020-04-18 22:42:24 +10:00
stijn
bcf01d1686 all: Fix implicit conversion from double to float.
These are found when building with -Wfloat-conversion.
2020-04-18 22:42:24 +10:00
stijn
dc4d119d3d py/objarray: Fix sign mismatch in comparison.
Found when compiling with clang and -Wsign-compare.
2020-04-18 22:42:19 +10:00
stijn
0ba68f8a1d all: Fix implicit floating point promotion.
Initially some of these were found building the unix coverage variant on
MacOS because that build uses clang and has -Wdouble-promotion enabled, and
clang performs more vigorous promotion checks than gcc.  Additionally the
codebase has been compiled with clang and msvc (the latter with warning
level 3), and with MICROPY_FLOAT_IMPL_FLOAT to find the rest of the
conversions.

Fixes are implemented either as explicit casts, or by using the correct
type, or by using one of the utility functions to handle floating point
casting; these have been moved from nativeglue.c to the public API.
2020-04-18 22:36:14 +10:00
stijn
b909e8b2dd Revert "all: Fix implicit casts of float/double, and signed comparison."
This reverts commit a2110bd3fc.  There's
nothing inherently wrong with it, but upcoming commits will apply similar
fixes in a slightly different way.
2020-04-18 22:36:06 +10:00
Jeff Epler
5fcc6d6286 RGBMatrix: finish renaming from Protomatter
This gets all the purely internal references.  Some uses of
protomatter/Protomatter/PROTOMATTER remain, as they are references
to symbols in the Protomatter C library itself.
2020-04-17 18:44:07 -05:00
Jeff Epler
3d6258f63d Rename Protomatter -> RGBMatrix
This is a quick rename, it changes the user-facing names but not the
internal names of things.
2020-04-17 18:43:57 -05:00
Jeff Epler
a32337718d Rename _protomatter -> protomatter
I originally believed that there would be a wrapper library around it,
like with _pixelbuf; but this proves not to be the case, as there's
too little for the library to do.
2020-04-17 18:43:57 -05:00
Scott Shawcroft
b580b34cbf
Merge remote-tracking branch 'adafruit/master' into lower_power 2020-04-14 17:14:44 -07:00
Jeff Epler
b1fab1cdac Make stripping circuitpython optional, not the default 2020-04-14 18:24:58 -05:00
Jeff Epler
09dc46a984 Add Protomatter and FramebufferDisplay 2020-04-14 18:24:54 -05:00
Jeff Epler
135fb5b887 py.mk: update warning flags needed for ulab 2020-04-14 15:37:36 -05:00
Damien George
2725a79192 py: Always give noop defines when MICROPY_ROM_TEXT_COMPRESSION disabled.
This commit provides a typedef for mp_rom_error_text_t, and a macro define
for MP_COMPRESSED_ROM_TEXT, when MICROPY_ROM_TEXT_COMPRESSION is disabled.
This simplifies the configuration (it no longer has a special case for
MICROPY_ENABLE_DYNRUNTIME) and makes it work for other cases that don't use
compression (eg examples/embedding).  This commit also ensures
MICROPY_ROM_TEXT_COMPRESSION is defined during qstr processing.
2020-04-14 22:13:11 +10:00
Romain Goyet
bd63c26dd5 py/scope: Add assert to check that low numbered qstrs do fit in uint8_t. 2020-04-13 22:27:27 +10:00
Damien George
7654907e1e py/makecompresseddata.py: Don't prefix str with mark if not compressed. 2020-04-13 22:21:57 +10:00
Damien George
8e048d2548 all: Clean up error strings to use lowercase and change cannot to can't.
Now that error string compression is supported it's more important to have
consistent error string formatting (eg all lowercase English words,
consistent contractions).  This commit cleans up some of the strings to
make them more consistent.
2020-04-13 22:19:37 +10:00
Jim Mussared
8470cd0be9 py/scheduler: Add assert that scheduler is locked when unlocking.
And add a test that shows how this can happen when multiple threads are
accessing the scheduler, which fails if atomic sections are not used.
2020-04-13 21:55:47 +10:00
Jim Mussared
243805d776 py/scheduler: Fix race in checking scheduler pending state.
Because the atomic section starts after checking whether the scheduler
state is pending, it's possible it can become a different state by the time
the atomic section starts.

This is especially likely on ports where MICROPY_BEGIN_ATOMIC_SECTION is
implemented with a mutex (i.e. it might block), but the race exists
regardless, i.e. if a context switch occurs between those two lines.
2020-04-13 21:55:47 +10:00
Damien George
e292296d52 py/objexcept: Remove optional TimeoutError exception.
TimeoutError was added back in 077812b2ab for
the cc3200 port. In f522849a4d the cc3200
port enabled use of it in the socket module aliased to socket.timeout.  So
it was never added to the builtins.  Then it was replaced by
OSError(ETIMEDOUT) in 047af9b10b.

The esp32 port enables this exception, since the very beginning of that
port, but it could never be accessed because it's not in builtins.

It's being removed: 1) to not encourage its use; 2) because there are a lot
of other OSError subclasses which are not defined at all, and having
TimeoutError is a bit inconsistent.

Note that ports can add anything to the builtins via MICROPY_PORT_BUILTINS.
And they can also define their own exceptions using the
MP_DEFINE_EXCEPTION() macro.
2020-04-09 16:09:38 +10:00
Damien George
4914731e58 py/parse: Remove unnecessary check in const folding for ** operator.
In this part of the code there is no way to get the ** operator, so no need
to check for it.

This commit also adds tests for this, and other related, invalid const
operations.
2020-04-09 16:02:39 +10:00
Jim Mussared
def76fe4d9 all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
Jim Mussared
85858e72df py/objexcept: Allow compression of exception message text.
The decompression of error-strings is only done if the string is accessed
via printing or via er.args.  Tests are added for this feature to ensure
the decompression works.
2020-04-05 15:02:06 +10:00
Jim Mussared
154b4eb354 py: Implement "common word" compression scheme for error messages.
The idea here is that there's a moderate amount of ROM used up by exception
text.  Obviously we try to keep the messages short, and the code can enable
terse errors, but it still adds up.  Listed below is the total string data
size for various ports:

    bare-arm 2860
    minimal 2876
    stm32 8926  (PYBV11)
    cc3200 3751
    esp32 5721

This commit implements compression of these strings.  It takes advantage of
the fact that these strings are all 7-bit ascii and extracts the top 128
frequently used words from the messages and stores them packed (dropping
their null-terminator), then uses (0x80 | index) inside strings to refer to
these common words.  Spaces are automatically added around words, saving
more bytes.  This happens transparently in the build process, mirroring the
steps that are used to generate the QSTR data.  The MP_COMPRESSED_ROM_TEXT
macro wraps any literal string that should compressed, and it's
automatically decompressed in mp_decompress_rom_string.

There are many schemes that could be used for the compression, and some are
included in py/makecompresseddata.py for reference (space, Huffman, ngram,
common word).  Results showed that the common-word compression gets better
results.  This is before counting the increased cost of the Huffman
decoder.  This might be slightly counter-intuitive, but this data is
extremely repetitive at a word-level, and the byte-level entropy coder
can't quite exploit that as efficiently.  Ideally one would combine both
approaches, but for now the common-word approach is the one that is used.

For additional comparison, the size of the raw data compressed with gzip
and zlib is calculated, as a sort of proxy for a lower entropy bound.  With
this scheme we come within 15% on stm32, and 30% on bare-arm (i.e. we use
x% more bytes than the data compressed with gzip -- not counting the code
overhead of a decoder, and how this would be hypothetically implemented).

The feature is disabled by default and can be enabled by setting
MICROPY_ROM_TEXT_COMPRESSION at the Makefile-level.
2020-04-05 14:20:57 +10:00
Jim Mussared
c34e7b9d4c py/dynruntime.mk: Set MICROPY_ENABLE_DYNRUNTIME instead of per module.
So this setting could be used by other source files if needed.
2020-04-05 14:13:02 +10:00
Jim Mussared
a9a745e4b4 py: Use preprocessor to detect error reporting level (terse/detailed).
Instead of compiler-level if-logic.  This is necessary to know what error
strings are included in the build at the preprocessor stage, so that string
compression can be implemented.
2020-04-05 14:11:51 +10:00
Scott Shawcroft
c248730bd1
Clean up 2020-04-02 17:36:09 -07:00
Scott Shawcroft
8fe512c7e9
Merge remote-tracking branch 'adafruit/master' into lower_power 2020-03-31 15:13:58 -07:00
David Lechner
a2110bd3fc all: Fix implicit casts of float/double, and signed comparison.
These were found by buiding the unix coverage variant on macOS (so clang
compiler).  Mostly, these are fixing implicit cast of float/double to
mp_float_t which is one of those two and one mp_int_t to size_t fix for
good measure.
2020-03-30 12:04:21 +11:00
Damien George
1a3e386c67 all: Remove spaces inside and around parenthesis.
Using new options enabled in the uncrustify configuration.
2020-03-28 23:36:44 +11:00
Damien George
bc009fdd62 extmod/uasyncio: Add optional implementation of core uasyncio in C.
Implements Task and TaskQueue classes in C, using a pairing-heap data
structure.  Using this reduces RAM use of each Task, and improves overall
performance of the uasyncio scheduler.
2020-03-26 01:25:45 +11:00
Damien George
6c7e78de72 py/pairheap: Add helper function to initialise a new node. 2020-03-26 01:21:04 +11:00
Damien George
c47a3ddf4a py/pairheap: Properly unlink node on pop and delete.
This fixes a bug in the pairing-heap implementation when nodes are deleted
with mp_pairheap_delete and then reinserted later on.
2020-03-26 01:21:04 +11:00
Damien George
f8fc78691d py/mpconfig.h: Enable MICROPY_MODULE_GETATTR by default.
To enable lazy loading of submodules (among other things), which is very
useful for MicroPython libraries that want to have optional subcomponents.

Disabled explicitly on minimal ports.
2020-03-26 01:21:04 +11:00
David Lechner
2f7d2bb3e2 py/stream.h: Include sys/types.h to get size_t and off_t for POSIX API. 2020-03-25 01:00:52 +11:00
Damien George
feb2577585 all: Remove spaces between nested paren and inside function arg paren.
Using new options enabled in the uncrustify configuration.
2020-03-25 00:39:46 +11:00
Jeff Epler
9d2270a979 runtime: Improve error message when setting read-only property
Formerly, if you wrote
   SPI.frequency = 0
you would get the sightly erroneous error message
   AttributeError: 'SPI' object has no attribute 'frequency'
In this case, a better message would read
   AttributeError: 'SPI' object cannot assign attribute 'frequency'

This new message will both be used in the case where the attribute doesn't
exist at all (and the object has no dynamic attributes; most instances of
built in types behave this way), or if the attribute exists but is
read-only.
2020-03-20 17:57:28 -05:00
Damien George
ad9a0ec8ab all: Convert exceptions to use mp_raise_XXX helpers in remaining places. 2020-03-18 17:26:19 +11:00
Jeff Epler
96f2288b84 ulab: include new 'extras' source file 2020-03-17 09:33:17 -05:00
Jeff Epler
d6342af980 ulab: rename enable macro so it appears in the support matrix 2020-03-17 09:33:03 -05:00
Jeff Epler
69aa010cfc objslice: take mp_obj_slice_indices from micropython 2020-03-17 08:32:52 -05:00
Scott Shawcroft
48b5f2a384
Initial work on SAMD 2020-03-13 11:16:41 -07:00
Andrew Leech
86bfabec11 py/modmicropython: Add heap_locked function to test state of heap.
This commit adds micropython.heap_locked() which returns the current
lock-depth of the heap, and can be used by Python code to check if the heap
is locked or not.  This new function is configured via
MICROPY_PY_MICROPYTHON_HEAP_LOCKED and is disabled by default.

This commit also changes the return value of micropython.heap_unlock() so
it returns the current lock-depth as well.
2020-03-11 16:54:16 +11:00
Andrew Leech
ed93778e00 py/objstringio: Expose tell() on StringIO and BytesIO objects.
To match file objects.

Fixes issue #5581.
2020-03-11 14:43:03 +11:00
David Lechner
8a4ce6b79a tools/codeformat.py: Eliminate need for sizeof fixup.
This eliminates the need for the sizeof regex fixup by rearranging things a
bit.  All other bitfields already use the parentheses around expressions
with sizeof, so one case is fixed by following this convention.

VM_MAX_STATE_ON_STACK is the only remaining problem and it can be worked
around by changing the order of the operands.
2020-03-11 14:34:40 +11:00
Tom Collins
fccf17521a py/objstr: Remove duplicate % in error string.
The double-% was added in 11de8399fe (Jun
2014) when such errors were formatted with printf.  But then
55830dd9bf (Dec 2018) changed
mp_obj_new_exception_msg() to not format the message, as discussed
in #3004.  So such error strings are no longer formatted and a % is just
that.
2020-03-11 14:31:29 +11:00
Scott Shawcroft
d988af02d1
Merge pull request #2690 from jepler/topic-pep-498-fstrings
Implement partial PEP-498 (f-string) support
2020-03-10 10:53:04 -07:00
Jeff Epler
491a31a731 circuitpy_mpconfig.mk: Enable ULAB for "full builds"
This enables it on the imxrt and cds56 ports where it was disabled before
2020-03-09 15:54:40 -05:00
Jeff Epler
473e9c5ffb f-strings: Make optional, defaulting to !CIRCUITPY_MINIMAL_BUILD
This should reclaim *most* code space added to handle f-strings.
However, there may be some small code growth as parse_string_literal
takes a new parameter (which will always be 0, so hopefully the optimizer
eliminates it)
2020-03-09 09:03:25 -05:00
Jeff Epler
32647cd9b4 lexer: catch concatenation of f'' and '' strings
This turns the "edge case" into a parse-time error.
2020-03-09 09:03:25 -05:00
Josh Klar
40bc05ee1e Address dpgeorge feedback - largely simplifications 2020-03-09 08:16:07 -05:00
Josh Klar
3a7a5ba686 py: Implement partial PEP-498 (f-string) support
This implements (most of) the PEP-498 spec for f-strings, with two
exceptions:

- raw f-strings (`fr` or `rf` prefixes) raise `NotImplementedError`
- one special corner case does not function as specified in the PEP
(more on that in a moment)

This is implemented in the core as a syntax translation, brute-forcing
all f-strings to run through `String.format`. For example, the statement
`x='world'; print(f'hello {x}')` gets translated *at a syntax level*
(injected into the lexer) to `x='world'; print('hello {}'.format(x))`.
While this may lead to weird column results in tracebacks, it seemed
like the fastest, most efficient, and *likely* most RAM-friendly option,
despite being implemented under the hood with a completely separate
`vstr_t`.

Since [string concatenation of adjacent literals is implemented in the
lexer](534b7c368d),
two side effects emerge:

- All strings with at least one f-string portion are concatenated into a
single literal which *must* be run through `String.format()` wholesale,
and:
- Concatenation of a raw string with interpolation characters with an
f-string will cause `IndexError`/`KeyError`, which is both different
from CPython *and* different from the corner case mentioned in the PEP
(which gave an example of the following:)

```python
x = 10
y = 'hi'
assert ('a' 'b' f'{x}' '{c}' f'str<{y:^4}>' 'd' 'e') == 'ab10{c}str< hi >de'
```

The above-linked commit detailed a pretty solid case for leaving string
concatenation in the lexer rather than putting it in the parser, and
undoing that decision would likely be disproportionately costly on
resources for the sake of a probably-low-impact corner case. An
alternative to become complaint with this corner case of the PEP would
be to revert to string concatenation in the parser *only when an
f-string is part of concatenation*, though I've done no investigation on
the difficulty or costs of doing this.

A decent set of tests is included. I've manually tested this on the
`unix` port on Linux and on a Feather M4 Express (`atmel-samd`) and
things seem sane.
2020-03-09 08:16:07 -05:00
Jeff Epler
4fa90261a3
Merge pull request #2657 from tannewt/builtin_package
Support importing native modules in native packages.
2020-03-05 14:28:46 -06:00
Jeff Epler
da31acfcc4 Merge remote-tracking branch 'origin/master' into ulab 2020-03-03 20:13:53 -06:00
Scott Shawcroft
274cb597b0
Remove debug extern 2020-03-03 10:55:50 -08:00
Jeff Epler
862830da32 compile: Give a proper error on 'async with'/'async for' outside 'async def'
A simple reproducer is:
   async for x in():x
2020-03-01 09:40:43 -06:00
Jeff Epler
511c180869 parse: push_result_token: throw an exception on too-long names
Before this, such names would instead cause an assertion error inside
qstr_from_strn.

A simple reproducer is a python source file containing the letter "a"
repeated 256 times
2020-03-01 09:38:34 -06:00
Damien George
1993c8cf9a py/builtinevex: Support passing in a bytearray/buffer to eval/exec.
CPython allows this and it's a simple generalisation of the existing code
which just supported str/bytes.

Fixes issue #5704.
2020-02-28 12:45:36 +11:00
Damien George
69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
Damien George
3f39d18c2b all: Add *FORMAT-OFF* in various places.
This string is recognised by uncrustify, to disable formatting in the
region marked by these comments.  This is necessary in the qstrdef*.h files
to prevent modification of the strings within the Q(...).  In other places
it is used to prevent excessive reformatting that would make the code less
readable.
2020-02-28 10:31:07 +11:00
Damien George
b86075ef1f py/parse: Add parenthesis around calculated bit-width in struct.
To improve interaction with uncrustify formatter.
2020-02-28 10:30:52 +11:00
Damien George
17aeb43e18 py: Un-nest configuration #if/#endif's for selection of complex code.
Because un-nested #if's are simpler to handle with formatting tools.
2020-02-28 10:30:34 +11:00
Damien George
7768a8c38d py/malloc: Put { on separate line for funcs that have selective sigs.
To ensure there are balanced {}'s in the file, and to help with formatting.
2020-02-28 10:30:29 +11:00
Damien George
a642241a12 py/builtinimport: Adjust if-block order in find_file to clean up #if's. 2020-02-28 10:30:20 +11:00
Damien George
2b50afaee4 py/bc0.h: Shift comment to start of line to improve format consistency. 2020-02-28 10:29:32 +11:00
Damien George
a1b18b3ba7 py: Removing dangling "else" to improve code format consistency. 2020-02-28 10:29:27 +11:00
Jeff Epler
39cfe32c34 Update ulab from upstream again 2020-02-27 14:14:05 -06:00
Jeff Epler
fa3b9eba92 ulab: Incorporate it 2020-02-27 11:03:03 -06:00
Scott Shawcroft
6375d8699e
Validate builtin member is a module 2020-02-26 10:32:02 -08:00
Scott Shawcroft
86fd93bd03
Support importing native modules in native packages.
This only fixes the `import` portion. It doesn't actually change
reference behavior because modules within a package could already
be referenced through the parent package even though an error should
have been thrown.
2020-02-25 15:32:55 -08:00
Jim Mussared
3ccce89b83 py/objarray: Turn on MP_TYPE_FLAG_EQ_CHECKS_OTHER_TYPE for memoryview.
And add corresponding tests.

Fixes #5674 (comparison of memoryview against bytes).
2020-02-21 14:24:07 +11:00
Dan Halbert
e31ac710be Enable _bleio adapter when _bleio is imported 2020-02-20 21:55:04 -05:00
Damien George
f6d99bc795 py/dynruntime.h: Add implementation of mp_obj_cast_to_native_base. 2020-02-21 13:32:48 +11:00
Damien George
9344e876bb py/objtype: Allow mp_instance_cast_to_native_base to take native obj.
And rename it to mp_obj_cast_to_native_base() to indicate this.  This
allows users of this function to easily support native and native-subclass
objects in the same way (by just passing the object through this function).
2020-02-21 13:20:12 +11:00
Scott Shawcroft
e8f7141564
Disable BLE file service for now
Fixes #2633
2020-02-20 12:15:56 -08:00
Damien George
d3b2c6e44c py/objtuple: Remove code that handles tuple-subclass equality test.
Since commit 3aab54bf43 this piece of code is
no longer needed because the top-level function mp_obj_equal_not_equal()
now handles the case of user types, and will never call tuple's binary_op
function with MP_BINARY_OP_EQUAL and a non-tuple on the RHS.
2020-02-20 10:48:03 +11:00
Damien George
1fccda049f py/objexcept: Rename mp_obj_new_exception_msg_varg2 to ..._vlist.
Follow up to recent commit ad7213d3c3, the
name "varg2" is misleading, vlist describes better that the argument is a
va_list.  This name also matches CircuitPython, which already has such
helper functions.
2020-02-18 21:00:42 +11:00
Damien George
ce39c958ef py: Factor out definition of mp_float_union_t to one location. 2020-02-18 13:04:36 +11:00
Dan Halbert
c592bd612a Implement to_bytes(..., signed=True) 2020-02-14 15:12:20 -05:00
Scott Shawcroft
e97b0cfc61
Merge pull request #2581 from jamesbowman/master
First draft of eveL, the low-level module of the Gameduino bindings
2020-02-13 11:21:32 -08:00
David Lechner
f020eac6a8 py/obj.h: Remove TODO idea comment about truncated mp_map_t.
It was suggested to move this to a GitHub issue rather than keep it in the
code, which isn't really sustainable for all ideas.
2020-02-13 13:14:17 +11:00
Damien George
ad7213d3c3 py: Add mp_raise_msg_varg helper and use it where appropriate.
This commit adds mp_raise_msg_varg(type, fmt, ...) as a helper for
nlr_raise(mp_obj_new_exception_msg_varg(type, fmt, ...)).  It makes the
C-level API for raising exceptions more consistent, and reduces code size
on most ports:

   bare-arm:   +28 +0.042%
minimal x86:  +100 +0.067%
   unix x64:   -56 -0.011%
unix nanbox:  -300 -0.068%
      stm32:  -204 -0.054% PYBV10
     cc3200:    +0 +0.000%
    esp8266:   -64 -0.010% GENERIC
      esp32:  -104 -0.007% GENERIC
        nrf:  -136 -0.094% pca10040
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2020-02-13 11:52:40 +11:00
Damien George
97eca38c4f py: Add mp_raise_type helper macro and use it where appropriate.
This provides a more consistent C-level API to raise exceptions, ie moving
away from nlr_raise towards mp_raise_XXX.  It also reduces code size by a
small amount on some ports.
2020-02-13 11:03:37 +11:00
Scott Shawcroft
36e6cc8feb
Track first free atb for multiple block sizes.
This speeds up cases where no single block allocations happen while
Python code is allocating other block sizes.
2020-02-11 17:06:24 -08:00
Damien George
7679e3be96 py/objmodule.h: Remove obsolete mp_builtin_module_weak_links_map decl.
It was made obsolete in d2384efa80
2020-02-11 15:43:13 +11:00
Damien George
9ec1caf42e py: Expand type equality flags to 3 separate ones, fix bool/namedtuple.
Both bool and namedtuple will check against other types for equality; int,
float and complex for bool, and tuple for namedtuple.  So to make them work
after the recent commit 3aab54bf43 they would
need MP_TYPE_FLAG_NEEDS_FULL_EQ_TEST set.  But that makes all bool and
namedtuple equality checks less efficient because mp_obj_equal_not_equal()
could no longer short-cut x==x, and would need to try __ne__.  To improve
this, this commit splits the MP_TYPE_FLAG_NEEDS_FULL_EQ_TEST flags into 3
separate flags to give types more fine-grained control over how their
equality behaves.  These new flags are then used to fix bool and namedtuple
equality.

Fixes issue #5615 and #5620.
2020-02-11 11:06:00 +11:00
James Bowman
5c6d94d3e5 Split into shared-module and shared-bindings 2020-02-07 10:30:49 -08:00
Damien George
abe2caf6df py/scheduler: Move clearing of kbd traceback to mp_keyboard_interrupt.
This is a more logical place to clear the KeyboardInterrupt traceback,
right before it is set as a pending exception.  The clearing is also
optimised from a function call to a simple store of NULL.
2020-02-07 16:08:31 +11:00
Damien George
8fb5c8fdd5 py/scheduler: Allow a port to specify attrs for mp_keyboard_interrupt.
Functions like mp_keyboard_interrupt() may need to be called from an IRQ
handler and may need to be in a special memory section, so provide a
generic wrapping macro for a port to do this.  The macro name is chosen to
be MICROPY_WRAP_<function name in uppercase> so that (in the future with
more wrappers) each function could potentially be handled separately.
2020-02-07 16:08:29 +11:00
Damien George
9efb36bfa6 py/scheduler: Move mp_keyboard_interrupt from lib/utils to py core.
This function is tightly coupled to the state and behaviour of the
scheduler, and is a core part of the runtime: to schedule a pending
exception.  So move it there.
2020-02-07 16:08:26 +11:00
Damien George
98a3911c43 py/scheduler: Add "raise_exc" argument to mp_handle_pending.
Previous behaviour is when this argument is set to "true", in which case
the function will raise any pending exception.  Setting it to "false" will
cancel any pending exception.
2020-02-07 16:08:20 +11:00
James Bowman
acef93a253 Rename eveL to _eve, EVEL to _EVE 2020-02-05 18:17:58 -08:00
Petr Viktorin
e6c9800645 py/compile: Allow 'return' outside function in minimal builds.
A 'return' statement on module/class level is not correct Python, but
nothing terribly bad happens when it's allowed.  So remove the check unless
MICROPY_CPYTHON_COMPAT is on.

This is similar to MicroPython's treatment of 'import *' in functions
(except 'return' has unsurprising behavior if it's allowed).
2020-02-06 00:41:55 +11:00
Dan Halbert
e13642351c fix printing single-arg exceptions 2020-02-04 16:55:56 -05:00
Dan Halbert
7fe959ab39 fix multi-arg exception printing 2020-02-04 16:41:25 -05:00
Dan Halbert
5164719e67 preserve errno in OSError 2020-02-04 16:19:40 -05:00
Dan Halbert
57b566e736 Include filename for 'No such file/directory', etc. 2020-02-04 13:32:39 -05:00
James Bowman
8347f2b5c3 Correct default state to off for eveL module.
Fix build break because of overflowing small ports
2020-02-03 19:23:42 -08:00
James Bowman
7fd30e7d20 First draft of eveL, the low-level module of the Gameduino (and BridgeTek EVE) bindings.
[adafruit/circuitpython#2578]
2020-02-03 16:46:14 -08:00
David Lechner
3e1bbeabaf py/modthread: Fix spelling error in comment. 2020-01-31 23:57:25 +11:00
Scott Shawcroft
55eb1730b8
Merge remote-tracking branch 'adafruit/master' into tweak_pixelbuf 2020-01-30 10:59:21 -08:00
Scott Shawcroft
2cc20e8816
Remove unneeded native cast. 2020-01-30 10:59:16 -08:00
Nicko van Someren
3aab54bf43 py: Support non-boolean results for equality and inequality tests.
This commit implements a more complete replication of CPython's behaviour
for equality and inequality testing of objects.  This addresses the issues
discussed in #5382 and a few other inconsistencies.  Improvements over the
old code include:

- Support for returning non-boolean results from comparisons (as used by
  numpy and others).
- Support for non-reflexive equality tests.
- Preferential use of __ne__ methods and MP_BINARY_OP_NOT_EQUAL binary
  operators for inequality tests, when available.
- Fallback to op2 == op1 or op2 != op1 when op1 does not implement the
  (in)equality operators.

The scheme here makes use of a new flag, MP_TYPE_FLAG_NEEDS_FULL_EQ_TEST,
in the flags word of mp_obj_type_t to indicate if various shortcuts can or
cannot be used when performing equality and inequality tests.  Currently
four built-in classes have the flag set: float and complex are
non-reflexive (since nan != nan) while bytearray and frozenszet instances
can equal other builtin class instances (bytes and set respectively).  The
flag is also set for any new class defined by the user.

This commit also includes a more comprehensive set of tests for the
behaviour of (in)equality operators implemented in special methods.
2020-01-30 14:53:07 +11:00
Damien George
c3450effd4 py/objtype: Make mp_obj_type_t.flags constants public, moved to obj.h. 2020-01-30 14:53:07 +11:00
Scott Shawcroft
5d24ade5c9
Tweak error messages to reduce code size. 2020-01-29 17:32:07 -08:00
Scott Shawcroft
d655c785b6
Merge commit 'b36b24' into tweak_pixelbuf 2020-01-29 13:35:22 -08:00
Scott Shawcroft
5b6b4eb326
Merge pull request #2551 from jepler/build-mpy-cross-static-linux-win64
Build static binaries of mpy-cross for desktop linux, desktop windows, mac, and raspbian
2020-01-29 11:45:08 -08:00
David Lechner
b72cb0ca1b py/mpthread.h: Use strong type for mp_thread_set_state() argument.
This modifies the signature of mp_thread_set_state() to use
mp_state_thread_t* instead of void*.  This matches the return type of
mp_thread_get_state(), which returns the same value.

`struct _mp_state_thread_t;` had to be moved before
`#include <mpthreadport.h>` since the stm32 port uses it in its
mpthreadport.h file.
2020-01-29 17:10:32 +11:00
Scott Shawcroft
f6a635b102
Fix subclassing of objects that are tested. Others may still be broken. 2020-01-27 14:52:42 -08:00
Scott Shawcroft
b36b2493bc
Merge pull request #2532 from tannewt/teensy4-dev
Refine iMX RT memory layout and add three boards
2020-01-27 14:11:08 -08:00
Jim Mussared
c3095b37e9 py/nativeglue: Fix typo about where the native fun table enum is. 2020-01-27 13:22:19 +11:00
Jim Mussared
888ddb81dd py/emitnative: Stop after finding an unwind target.
The loop searches backwards for a target, but doesn't stop after finding
the first result, meaning that it'll always end up at the outermost
exception handler.
2020-01-27 13:22:03 +11:00
Jim Mussared
0de304e7da py/emitnative: Use NULL for pending exception (not None).
This previously made the native emitter incompatible with the bytecode
emitter, and mp_resume (and subsequently mp_obj_generator_resume) expects
the bytecode emitter behavior (i.e. throw==NULL).
2020-01-27 13:21:49 +11:00
David Lechner
62537a18e3 py: Release GIL during syscalls in reader and writer code.
This releases the GIL during POSIX system calls that could block.
2020-01-26 23:26:31 +11:00
Jeff Epler
d9e0742a07 accomodate excessively old gcc versions for raspbian mpy-cross cross-build 2020-01-25 15:32:52 -06:00
Scott Shawcroft
81c3bc411f
Don't assume native methods want the native object as self. 2020-01-24 18:22:28 -08:00
Yonatan Goldschmidt
d9433d3e94 py/obj.h: Add and use mp_obj_is_bool() helper.
Commit d96cfd13e3 introduced a regression in
testing for bool objects, that such objects were in some cases no longer
recognised and bools, eg when using mp_obj_is_type(o, &mp_type_bool), or
mp_obj_is_integer(o).

This commit fixes that problem by adding mp_obj_is_bool(o).  Builds with
MICROPY_OBJ_IMMEDIATE_OBJS enabled check if the object is any of the const
True or False objects.  Builds without it use the old method of ->type
checking, which compiles to smaller code (compared with the former
mentioned method).

Fixes #5538.
2020-01-24 10:53:45 +11:00
Damien George
e2c1226da4 py/objexcept: Optimise mp_obj_new_exception[_arg1/_args] functions.
This reduces code size by 10-70 bytes on all ports (except cc3200 which has
no change).
2020-01-23 13:37:25 +11:00
Damien George
edc7a8bf1d py/objgenerator: Use mp_obj_new_exception_arg1 to make StopIteration. 2020-01-23 13:37:25 +11:00
David Lechner
edbb73a411 py/qstr: Don't include or init qstr_mutex when GIL is enabled.
When threads and the GIL are enabled, then the qstr mutex is not needed.
The qstr_mutex field is never used in this case because of:

    #if MICROPY_PY_THREAD && !MICROPY_PY_THREAD_GIL
    #define QSTR_ENTER() mp_thread_mutex_lock(&MP_STATE_VM(qstr_mutex), 1)
    #define QSTR_EXIT() mp_thread_mutex_unlock(&MP_STATE_VM(qstr_mutex))
    #else
    #define QSTR_ENTER()
    #define QSTR_EXIT()
    #endif

So, we can completely remove qstr_mutex everywhere when MICROPY_PY_THREAD
&& !MICROPY_PY_THREAD_GIL.
2020-01-23 13:29:11 +11:00
David Lechner
ccc18f047d py/gc: Don't include or init gc_mutex when GIL is enabled.
When threads and the GIL are enabled, then the GC mutex is not needed.  The
gc_mutex field is never used in this case because of:

    #if MICROPY_PY_THREAD && !MICROPY_PY_THREAD_GIL
    #define GC_ENTER() mp_thread_mutex_lock(&MP_STATE_MEM(gc_mutex), 1)
    #define GC_EXIT() mp_thread_mutex_unlock(&MP_STATE_MEM(gc_mutex))
    #else
    #define GC_ENTER()
    #define GC_EXIT()
    #endif

So, we can completely remove gc_mutex everywhere when MICROPY_PY_THREAD
&& !MICROPY_PY_THREAD_GIL.
2020-01-23 13:28:42 +11:00
Damien George
fe203bb3e2 py/pairheap: Add generic implementation of pairing heap data structure. 2020-01-22 17:31:18 +11:00
tsupplis
b66abd47b8
Update fix (missing pragma gcc diagnostic push)
Update fix (missing pragma gcc diagnostic push)
2020-01-21 19:45:20 +00:00
Scott Shawcroft
7d8dac9211
Refine iMX RT memory layout and add three boards
Introduces a way to place CircuitPython code and data into
tightly coupled memory (TCM) which is accessible by the CPU in a
single cycle. It also frees up room in the corresponding cache for
intermittent data. Loading from external flash is slow!

The data cache is also now enabled.

Adds support for the iMX RT 1021 chip. Adds three new boards:
* iMX RT 1020 EVK
* iMX RT 1060 EVK
* Teensy 4.0

Related to #2492, #2472 and #2477. Fixes #2475.
2020-01-17 17:36:08 -08:00