Commit Graph

4811 Commits

Author SHA1 Message Date
Jeff Epler 3ce0b512f8 rasberrypi: IncrementalEncoder: factor out state machine 2021-04-08 16:25:47 -05:00
Artyom Skrobov c3e40d50ab [qstr] Separate hash and len from string data
This allows the compiler to merge strings: e.g. "update",
"difference_update" and "symmetric_difference_update"
will all point to the same memory.

Shaves ~1KB off the image size, and potentially allows
bigger savings if qstr attrs are initialized in qstr_init(),
and not stored in the image.
2021-04-06 12:58:42 -04:00
Artyom Skrobov 59fa9b01ad [repl] Autocomplete builtin modules 2021-04-03 16:56:02 -04:00
Artyom Skrobov 1d55dee15b [repl] Refactor autocomplete, extracting reusable parts 2021-04-03 16:54:34 -04:00
Artyom Skrobov 6e17db3286 [repl] Refactor autocomplete to reduce nesting 2021-04-03 15:54:36 -04:00
Jeff Epler fb7b968b10 py.mk: locate ulab sources with shell-find. 2021-04-02 13:09:23 -05:00
Jeff Epler dc9daba906 update ulab to 2.1.5 2021-04-02 13:09:23 -05:00
Phil Howard 0cf12dd59c rp2: Add support for USER_C_MODULES to CMake build system.
The parts that are generic are added to py/ so they can be used by other
ports that use CMake.

py/usermod.cmake:

* Creates a usermod target to hang user C/CXX modules from.
* Gathers sources from user C/CXX modules and libs for QSTR scan.

ports/rp2/CMakeLists.txt:

* Includes py/usermod.cmake.
* Links the resulting usermod library to the MicroPython target.

py/mkrules.cmake:

Add cxxflags to qstr.i.last custom command for CXX modules:

* MICROPY_CPP_FLAGS so CXX modules will find includes.
* -DNO_QSTR to fix fatal error missing "genhdr/qstrdefs.generated.h".

Usage:

The rp2 port can be linked against user C modules by running:

make USER_C_MODULES=/path/to/module/micropython.cmake

CMake will print a list of included modules.

Co-authored-by: Graham Sanderson <graham.sanderson@raspberrypi.org>
Co-authored-by: Michael O'Cleirigh <michael.ocleirigh@rivulet.ca>
Signed-off-by: Phil Howard <phil@pimoroni.com>
2021-03-31 00:26:01 +11:00
Artyom Skrobov 8056af8648 [synthio] add a simple MidiTrack implementation 2021-03-26 16:38:10 -04:00
Artyom Skrobov d7dc3801ab [ure] to save space, disable debug dumps by default
Has to stay enabled in unix port for the sake of tests/extmod/ure_debug.py
2021-03-26 13:24:20 -04:00
Damien George 9fef1c0bde py: Rename remaining object types to be of the form mp_type_xxx.
For consistency with all other object types in the core.

Signed-off-by: Damien George <damien@micropython.org>
2021-03-26 13:48:34 +11:00
Damien George 42cf77f48b py/vm: For tracing use mp_printf, and print state when thread enabled.
mp_printf should be used to print the prefix because it's also used in
mp_bytecode_print2 (otherwise, depending on the system, different output
streams may be used).

Also print the current thread state when threading is enabled to easily see
which thread executes what opcode.

Signed-off-by: Damien George <damien@micropython.org>
2021-03-17 12:13:53 +11:00
microDev a52eb88031
run code formatting script 2021-03-15 19:27:36 +05:30
Dan Halbert 5ace7af933 disable default usb_cdc 2021-03-11 11:20:29 -05:00
Yonatan Goldschmidt e196cb762e py/nlrx64: Fix typo in comment. 2021-03-11 12:51:10 +11:00
Yonatan Goldschmidt 2d5cece5ac py/nlr: Implement NLR for AArch64. 2021-03-11 12:51:10 +11:00
Jeff Epler 1e5ffe1a53 Revert "circuitpy_mpconfig: Disable flash multi-partition"
This reverts commit 156ee4833a.
2021-03-09 11:08:36 -06:00
Scott Shawcroft b5a955a168
Merge pull request #4354 from jepler/disable-fat-partition
circuitpy_mpconfig: Disable flash multi-partition
2021-03-08 15:25:58 -08:00
Jeff Epler 156ee4833a circuitpy_mpconfig: Disable flash multi-partition
This adds some additional code in mkfs which doesn't seem necessary, and
Disabling it saves 172 bytes flash.

Testing performed: Using a Feather M0 Adalogger, checked that
 * an sdcard could still be mounted (using adafruit_sdcard)
 * os.listdir() of "/" and "/sd" worked
 * CIRCUITPY still mounted
2021-03-07 09:37:18 -06:00
Jeff Epler 4f040af481 vm: Make the speed-size trade-off compile time settable
.. and enable for all samd21 boards
2021-03-05 18:29:27 -06:00
Jeff Epler 7b359d7a8a vm: Consolodate all dispatch instructions
Flash savings: 1268 bytes
Performance: 10,000 iteration loop .665 -> .676s (+1.7%)
2021-03-05 16:52:35 -06:00
Jeff Epler 28638df589 vm: cut vmentrytable size
This also adds a bit of code everywhere we DISPATCH(), but the net is
+232 bytes free on Feather M0 Adalogger.

Key assumption: All of the offsets in mp_execute_bytecode fit in 16 bits;
it is not clear whether the compiler will verify this assumption (e.g.,
by warning that a constant will be truncated)
2021-03-05 16:52:35 -06:00
Scott Shawcroft e4f0e47d9f
Merge pull request #4233 from pewpew-game/displayio-group-list
displayio: make Group use a python list internally
2021-03-02 09:59:58 -08:00
Scott Shawcroft d0eab5c561
Merge pull request #4256 from kmatch98/bt_cleanup
Add `bitmaptools` module
2021-03-01 18:24:12 -08:00
Radomir Dopieralski 24473b7983 Separate out mp_obj_list_insert for use in display.Group
Note that for some reason this makes the binary 500 bytes larger!
2021-02-27 21:13:55 +01:00
Radomir Dopieralski e505c59ed8 Separate mp_obj_list_pop so it can be used outside of objlist.c 2021-02-27 20:52:38 +01:00
Scott Shawcroft 3f08cb47b8
Merge remote-tracking branch 'adafruit/main' into busio-uart-rp 2021-02-25 16:59:15 -08:00
Scott Shawcroft 52bc935fa7
A few minor fixes for corner cases
* Always clear the peripheral interrupt so we don't hang when full
* Store the ringbuf in the object so it gets collected when we're alive
* Make UART objects have a finaliser so they are deinit when their
  memory is freed
* Copy bytes into the ringbuf from the FIFO after we read to ensure
  the interrupt is enabled ASAP
* Copy bytes into the ringbuf from the FIFO before measuring our
  rx available because the interrupt is based on a threshold (not
  > 0). For example, a single byte won't trigger an interrupt.
2021-02-25 16:50:57 -08:00
Kevin Matocha a7a4f93ff8 Updated build configs to turn off BITMAPTOOLS for smaller builds 2021-02-25 15:44:32 -06:00
Kevin Matocha b720028642 Add bitmaptools module 2021-02-23 23:23:14 -06:00
Dan Halbert 7a4198885e Fix typos for CIRCUITPY_USB_HID_SYS_CONTROL 2021-02-23 09:22:20 -05:00
Dan Halbert 93bf269c0d Avoid pulling in extra float-uint64 routines 2021-02-21 12:05:03 -05:00
Damien George 2adf20c5f2 py/mkrules.cmake: Add MICROPY_QSTRDEFS_PORT to qstr build process.
This allows a port to specify a custom qstrdefsport.h file, the same as the
QSTR_DEFS variable in a Makefile.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-21 15:56:54 +11:00
Damien George d867d20d9a py/mkrules.cmake: Rename QSTR_DEFS variables to QSTRDEFS.
And also MICROPY_PY_QSTRDEFS to MICROPY_QSTRDEFS_PY.  These variables are
all related.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-21 15:56:54 +11:00
Jeff Epler a04369e0f5 add ifdef-guard 2021-02-20 17:41:08 -06:00
Jeff Epler 646eb9a7db py: memoryview: implement memoryview.cast if CPYTHON_COMPAT 2021-02-20 10:35:36 -06:00
Dan Halbert 076a7b7161 CIRCUITPY_BITOPS merge error 2021-02-19 15:30:27 -05:00
Dan Halbert 67406488d1 merge from upstream; re-alphabetize 2021-02-19 14:22:50 -05:00
Jeff Epler 72e8fb085c remove debug print during build 2021-02-18 16:01:23 -06:00
Jeff Epler 7fd4567893 bitops: rename from _bit_transpose, describe the algorithm 2021-02-18 15:41:23 -06:00
Jeff Epler 9cf7d73c6c core: add bit_transpose function
.. this version can only handle exactly 8 bits "across".  The restriction
may be relaxed in a future revision.
2021-02-18 11:32:47 -06:00
Dan Halbert ed49c02feb add timeout; finish up for PR 2021-02-17 23:24:11 -05:00
Maureen Helm dff6fc64d2 py: Expand lists in core cmake custom commands.
The core cmake rules use custom commands to invoke qstr processing
scripts. For the zephyr port, it's possible that list arguments to these
commands may contain generator expressions, therefore we need to expand
them properly.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2021-02-16 18:38:13 +11:00
Dan Halbert 93d788543c Merge remote-tracking branch 'adafruit/main' into secondary-cdc 2021-02-15 20:03:53 -05:00
Bernhard Boser f8f59598c7 memoryview is valid argument to exec 2021-02-15 10:05:57 -08:00
Damien George 66098c0985 py,extmod: Add core cmake rule files.
These allow a port to use cmake natively instead of make.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-15 12:48:18 +11:00
Dan Halbert d54b5861a3 wip 2021-02-12 19:01:14 -05:00
Dan Halbert f0564b4986 merge from upstream; complicated webusb merge 2021-02-11 18:50:02 -05:00
Damien George 0a59938574 py/mpz: Fix overflow of borrow in mpn_div.
For certain operands to mpn_div, the existing code path for
`DIG_SIZE == MPZ_DBL_DIG_SIZE / 2` had a bug in it where borrow could still
overflow in the `(x >= *n || *n - x <= borrow)` branch, ie
`borrow + x - (mpz_dbl_dig_t)*n` overflows the borrow variable.  In such
cases the subsequent right-shift of borrow would not bring in the overflow
bit, leading to an error in the result.  An example division that had
overflow when MPZ_DIG_SIZE = 16 is `(2 ** 48 - 1) ** 2 // (2 ** 48 - 1)`.

This is fixed in this commit by simplifying the code and handling the low
digits of borrow first, and then the upper bits (to shift down) separately.
There is no longer a distinction between `DIG_SIZE < MPZ_DBL_DIG_SIZE / 2`
and `DIG_SIZE == MPZ_DBL_DIG_SIZE / 2`.

This commit also simplifies the second part of the calculation so that
borrow does not need to be negated (instead the code just works knowing
that borrow is negative and using + instead of - in calculations involving
borrow).

Fixes #6777.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-08 11:50:05 +11:00
Damien George 9dedcf122d py/gc: Change include of stdint.h to stddef.h.
No std-int types are used in gc.h, but size_t is which needs stddef.h.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-05 15:46:56 +11:00
Dan Halbert 0802b22ed6 usb descriptors all set; rework of enabling for USB devices 2021-02-04 19:23:40 -05:00
Damien George c891190c69 py: Rename WORD_MSBIT_HIGH to MP_OBJ_WORD_MSBIT_HIGH.
To make it clear it is for mp_obj_t/mp_uint_t "word" types, and to prefix
this macro with MP_.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-04 22:46:42 +11:00
Damien George ad4656b861 all: Rename BYTES_PER_WORD to MP_BYTES_PER_OBJ_WORD.
The "word" referred to by BYTES_PER_WORD is actually the size of mp_obj_t
which is not always the same as the size of a pointer on the target
architecture.  So rename this config value to better reflect what it
measures, and also prefix it with MP_.

For uses of BYTES_PER_WORD in setting the stack limit this has been
changed to sizeof(void *), because the stack usually grows with
machine-word sized values (eg an nlr_buf_t has many machine words in it).

Signed-off-by: Damien George <damien@micropython.org>
2021-02-04 22:46:42 +11:00
Damien George 7e956fae28 py: Rename BITS_PER_BYTE to MP_BITS_PER_BYTE.
To give this macro a standard MP_ prefix.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-04 22:46:42 +11:00
Damien George 8a41ee19c2 py: Remove BITS_PER_WORD definition.
It's only used in one location, to test if << or >> will overflow when
shifting mp_uint_t.  For such a test it's clearer to use sizeof(lhs_val),
which will be valid even if the type of lhs_val changes.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-04 22:46:42 +11:00
Xiang Xiao 5fdf351178 py/gc: Don't include mpconfig.h and misc.h in gc.h.
Because gc.h doesn't reference any symbol from these header files.

Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
2021-02-04 22:37:26 +11:00
Kevin Banks 215a56c920 Merge branch 'main' into webusb (trying to get CI back online) 2021-02-03 16:48:28 -06:00
Damien George 78b23c3a1f all: Bump version to 1.14.
Signed-off-by: Damien George <damien@micropython.org>
2021-02-03 00:59:07 +11:00
Lucian Copeland 66639b237f Merge remote-tracking branch 'upstream/main' into sslsocket 2021-02-01 11:43:01 -05:00
iTitou 4fb5f012c3 py/makeversionhdr: Honor SOURCE_DATE_EPOCH if present.
This environment variable, if defined during the build process,
indicates a fixed time that should be used in place of "now" when
such a time is explicitely referenced.

This allows for reproducible builds of micropython.
See https://reproducible-builds.org/specs/source-date-epoch/

Signed-off-by: iTitou <moiandme@gmail.com>
2021-01-31 17:48:59 +01:00
David CARLIER cb30928ac8 py/persistentcode: Introduce MICROPY_PERSISTENT_CODE_SAVE_FILE option.
This should be enabled when the mp_raw_code_save_file function is needed.

It is enabled for mpy-cross, and a check for defined(__APPLE__) is added to
cover Mac M1 systems.
2021-01-30 15:13:24 +11:00
stijn cb8e2f02ab py/gc: Fix debug printing of pointer.
When DEBUG_printf is the standard printf, compilers require the value for
%p to be an actual pointer instead of an integer.
2021-01-30 14:41:29 +11:00
stijn b9a35bebf7 py/qstr.h: Remove QSTR_FROM_STR_STATIC macro.
It practically does the same as qstr_from_str and was only used in one
place, which should actually use the compile-time MP_QSTR_XXX form for
consistency; qstr_from_str is for runtime strings only.
2021-01-30 13:40:48 +11:00
graham sanderson 794df0f1d5 py/emitnative: Support binary ops on ARMv6M without use of ite instr. 2021-01-29 23:57:10 +11:00
Damien George 75fea330bf py/emitinlinethumb: Exclude code using #if when ARMV7M disabled.
So there are no references to undeclared asm_thumb_mov_reg_i16().

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 23:57:10 +11:00
Damien George c9f4c5acd6 py/emitnative: Ensure encoding to load prelude_offset doesn't change sz.
Based on change made by Graham Sanderson.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 23:57:10 +11:00
graham sanderson 40d2010882 py/asmthumb: Add support for ARMv6M in native emitter.
Adds a new compile-time option MICROPY_EMIT_THUMB_ARMV7M which is enabled
by default (to get existing behaviour) and which should be disabled (set to
0) when building native emitter support (@micropython.native) on ARMv6M
targets.
2021-01-29 23:57:10 +11:00
Damien George 925bd67cfb py/objfun: Support fun.__globals__ attribute.
This returns a reference to the globals dict associated with the function,
ie the global scope that the function was defined in.  This attribute is
read-only but the dict itself is modifiable, per CPython behaviour.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 23:57:10 +11:00
Kevin Banks b152c658e2 Making WebUSB default to off (build unbreakage, caught by CI) 2021-01-26 18:18:39 -06:00
Dan Halbert 13812a788f
Merge pull request #4060 from dhalbert/regularize-extmod-modules
CIRCUITPY_* switches for JSON, RE, etc. Doc cleanup
2021-01-26 12:54:21 -05:00
Dan Halbert da6869dbd5 Set MICROPY_PY_UERRNO_ERRORCODE correctly 2021-01-26 08:37:05 -05:00
Dan Halbert a9f339b461 typo in circuitpy_mpconfig.h; forgot cxd56 port 2021-01-25 22:40:31 -05:00
Kevin Banks fbfb7b68cc Most of the code we need has been pulled in from the tinyusb webusb_serial demo. Still LOTS to do regarding descriptors. 2021-01-25 20:37:58 -06:00
Scott Shawcroft a2ac2da7cc
Merge pull request #3936 from gamblor21/busdevice_fixes
Changing adafruit_bus_device to duck typing
2021-01-25 14:41:53 -08:00
Lucian Copeland 9f34ec78c4 Separate Socket and SSLSocket, add LWIP connect 2021-01-25 17:07:25 -05:00
Dan Halbert 69869e1439 CIRCUITPY_* switches for JSON, RE, etc. Doc cleanup 2021-01-24 23:10:20 -05:00
Dan Halbert e3275be8b1
Merge pull request #3911 from hugodahl/add-translation-for-builtin-object-help
Update built-in help output for localization (Issue #3907)
2021-01-22 03:49:19 -05:00
Scott Shawcroft b7a63dc498
Alphabetical 2021-01-21 10:18:04 -08:00
Scott Shawcroft 733094aead
Add initial RP2040 support
The RP2040 is new microcontroller from Raspberry Pi that features
two Cortex M0s and eight PIO state machines that are good for
crunching lots of data. It has 264k RAM and a built in UF2
bootloader too.

Datasheet: https://pico.raspberrypi.org/files/rp2040_datasheet.pdf
2021-01-20 19:16:56 -08:00
gamblor21 f50c9f4145 Reenabling busdevice in core 2021-01-16 14:21:57 -06:00
Hugo Dahl 8094435193
Merge branch 'main' of https://github.com/adafruit/circuitpython into add-translation-for-builtin-object-help 2021-01-14 22:12:30 -06:00
Jeff Epler 1ca29ec47c Merge remote-tracking branch 'origin/main' into audioout-esp32 2021-01-12 09:23:07 -06:00
Hugo Dahl 2d8404c599
Merge branch 'main' into add-translation-for-builtin-object-help 2021-01-11 18:32:07 -06:00
Scott Shawcroft ba507a8193
Merge pull request #3957 from dhalbert/gcc10-debug-builds
new CIRCUITPY_DEBUG flag passes DEBUG into C preprocessor
2021-01-11 15:36:12 -08:00
Dan Halbert 255ffa979c avoid inline compile errors 2021-01-08 23:07:21 -05:00
Jeff Epler 30c6dd3f76 Merge remote-tracking branch 'origin/main' into audioout-esp32 2021-01-07 14:11:14 -06:00
iot49 1a82555803
Merge branch 'main' into msgpack 2021-01-05 11:19:11 -08:00
Hugo Dahl ad7f4d8ae9
Merge branch 'main' of https://github.com/adafruit/circuitpython into add-translation-for-builtin-object-help
* 'main' of https://github.com/adafruit/circuitpython:
  update wake-alarm implementation
  enable light-sleep functionality
  update frozen libs for 6.1.0-beta.3
  Revert "Removing frozen libs"
  add pretend-to-sleep functionality
  expose wake pin parameter and more tweaks
  add touch alarm support for esp32s2
  implement touch alarm
2020-12-30 23:37:55 -06:00
Hugo Dahl 1407af7291
Add localization for built-in help
Support localizing the output of a call to  to match the
firmware's language.
2020-12-30 18:20:43 -06:00
microDev 28ce5e8988
Merge branch 'main' into touch-s2 2020-12-30 22:44:22 +05:30
Jeff Epler 2cd377f1a7 audiobusio: Make PDMIn optional 2020-12-29 14:06:32 -06:00
Dan Halbert 8061a2574d
Merge branch 'main' into pin_alarm 2020-12-23 22:05:02 -05:00
Jeff Epler 42a229c08b circuitpy_mpconfig.mk: Unconditionally disable CIRCUITPY_BUSDEVICE
Several issues have been found in the implementation.  While they're
unresolved, it may be better to disable the built-in module.  (This
means that to work on fixing the module, it'll be necessary to
revert this commit)
2020-12-23 10:45:07 -06:00
Scott Shawcroft 1fca297a2d
A couple sleep fixes
* Better messaging when code is stopped by an auto-reload.
* Auto-reload works during sleeps on ESP32-S2. Ticks wake up the
  main task each time.
* Made internal naming consistent. CamelCase Python names are NOT
  separated by an underscore.
2020-12-22 16:13:02 -08:00
microDev 8eaf2b0c19
implement touch alarm 2020-12-18 12:54:36 +05:30
microDev 4512290ba0
Merge branch 'main' into ota-s2 2020-12-18 00:44:00 +05:30
microDev 4863413bc9
rename ota to dualbank 2020-12-18 00:34:56 +05:30
Dan Halbert 8f9cd7075e
Merge pull request #3752 from jepler/gcc10
build: Update to gcc10
2020-12-17 11:03:40 -05:00
Scott Shawcroft 1ad49d9a18
Add alarm.pin that wakes on pin level
Fixes #3787
2020-12-15 18:12:59 -08:00
Jeff Epler afcc00fed4
Merge pull request #3771 from cwalther/exceptionprint
#3702 breaks printing an exception twice
2020-12-15 08:11:28 -06:00
Scott Shawcroft d076296659
Merge pull request #3816 from dhalbert/sleepmemory
alarm.sleep_memory + alarm.wake_alarm
2020-12-14 17:40:02 -08:00
Dan Halbert 6abe3cd0ef -Os for SAMD51; fix CSUPEROPT typo 2020-12-14 18:57:31 -05:00
Damien George 246b2e016a py/mkrules.mk: Remove stray vpath and unused -Itmp, add $(Q) for $(AR).
Signed-off-by: Damien George <damien@micropython.org>
2020-12-14 13:57:15 +11:00
Damien George 0091041f5a py/modmath: Simplify handling of positional args to reduce code size.
As a general pattern, required positional arguments that are not named do
not need to be parsed using mp_arg_parse_all().

Signed-off-by: Damien George <damien@micropython.org>
2020-12-14 13:30:56 +11:00
microDev 37ee5e683d
Merge branch 'main' into ota-s2 2020-12-12 05:25:46 +05:30
Scott Shawcroft 344d3c59cb
Merge branch 'main' into msgpack 2020-12-11 11:10:30 -08:00
Dan Halbert d83d46a52d Invoke scripts with 2020-12-10 14:28:22 -05:00
Dan Halbert 5964163649 Initial SleepMemory code 2020-12-10 13:03:40 -05:00
Scott Shawcroft 40118bcf57
Add `board_deinit` for use with sleep
This changes lots of files to unify `board.h` across ports. It adds
`board_deinit` when CIRCUITPY_ALARM is set. `main.c` uses it to
deinit the board before deep sleeping (even when pretending.)

Deep sleep is now a two step process for the port. First, the
port should prepare to deep sleep based on the given alarms. It
should set alarms for both deep and pretend sleep. In particular,
the pretend versions should be set immediately so that we don't
miss an alarm as we shutdown. These alarms should also wake from
`port_idle_until_interrupt` which is used when pretending to deep
sleep.

Second, when real deep sleeping, `alarm_enter_deep_sleep` is called.
The port should set any alarms it didn't during prepare based on
data it saved internally during prepare.

ESP32-S2 sleep is a bit reorganized to locate more logic with
TimeAlarm. This will help it scale to more alarm types.

Fixes #3786
2020-12-08 10:52:25 -08:00
microDev fc23a0cc8a
implement ota module 2020-12-08 11:30:00 +05:30
Bernhard Boser 534b48fcfe remove a ~ from doc that causes an error; add ExtType.c to circuitpy_defns.mk 2020-12-07 16:08:16 -08:00
Joris Peeraer 5020b14d54 py/mpprint: Fix length calculation for strings with precision-modifier.
Two issues are tackled:

1. The calculation of the correct length to print is fixed to treat the
   precision as a maximum length instead as the exact length.
   This is done for both qstr (%q) and for regular str (%s).

2. Fix the incorrect use of mp_printf("%.*s") to mp_print_strn().

   Because of the fix of above issue, some testcases that would print
   an embedded null-byte (^@ in test-output) would now fail.
   The bug here is that "%s" was used to print null-bytes. Instead,
   mp_print_strn is used to make sure all bytes are outputted and the
   exact length is respected.

Test-cases are added for both %s and %q with a combination of precision
and padding specifiers.
2020-12-07 23:32:06 +11:00
Scott Shawcroft 1130b80e2a
Merge pull request #3612 from gamblor21/bus_device
Moving Adafruit_CircuitPython_BusDevice to core
2020-12-02 13:23:02 -08:00
Scott Shawcroft 608c98501b
Merge remote-tracking branch 'adafruit/main' into msgpack 2020-12-02 13:10:39 -08:00
Scott Shawcroft d7ba641ff6
Merge pull request #3767 from dhalbert/sleep
Initial alarm and sleep PR: time alarms with light and deep sleep; PinAlarms not yet implemented
2020-12-02 12:51:43 -08:00
Bernhard Boser 513253bc3f
moved logic to shared-module and added documentation 2020-12-01 18:38:14 -08:00
Bernhard Boser 748472de7a
removed empty line at end of py/circuitpy_mpconfig.mk 2020-12-01 18:38:14 -08:00
Bernhard Boser 90c203a3dd
add module msgpack 2020-12-01 18:38:14 -08:00
Dan Halbert 8b7c23c1ee address review comments 2020-12-01 20:01:14 -05:00
Mark 237385798c
Merge branch 'main' into bus_device 2020-12-01 15:47:16 -06:00
Scott Shawcroft a975ef4971
Merge pull request #3695 from cwalther/movable
Add movable supervisor allocations
2020-11-30 16:00:55 -08:00
Dan Halbert 9768951a2a Disable complex arithmetic on SAMD21 builds to make space 2020-11-29 15:34:38 -05:00
Damien George be24e6a53f py/mpprint: Prevent case fall-through when assert is disabled.
Signed-off-by: Damien George <damien@micropython.org>
2020-11-29 17:31:24 +11:00
Christian Walther bde1c4166d Revert "Prevent exceptions from accumulating in REPL"
This reverts commit 0cd951fb73.

It is not a correct solution because it prevents printing the same exception twice.
2020-11-28 23:10:17 +01:00
Christian Walther 2ba9805f84 Use movable allocation system for terminal tilegrid.
Moving memory is now done by the infrastructure and neither necessary nor correct here anymore.
2020-11-28 17:54:34 +01:00
Christian Walther c7404a3ff8 Add movable allocation system.
This allows calls to `allocate_memory()` while the VM is running, it will then allocate from the GC heap (unless there is a suitable hole among the supervisor allocations), and when the VM exits and the GC heap is freed, the allocation will be moved to the bottom of the former GC heap and transformed into a proper supervisor allocation. Existing movable allocations will also be moved to defragment the supervisor heap and ensure that the next VM run gets as much memory as possible for the GC heap.

By itself this breaks terminalio because it violates the assumption that supervisor_display_move_memory() still has access to an undisturbed heap to copy the tilegrid from. It will work in many cases, but if you're unlucky you will get garbled terminal contents after exiting from the vm run that created the display. This will be fixed in the following commit, which is separate to simplify review.
2020-11-28 17:50:23 +01:00
Dan Halbert 28d9e9186e Disable complex arithmetic on SAMD21 builds to make space 2020-11-28 10:12:46 -05:00
Dan Halbert ef0830bfe2 merge from upstream + wip 2020-11-25 17:52:06 -05:00
Dan Halbert 9dbea36eac changed alarm.time API 2020-11-25 15:09:27 -05:00
Jeff Epler e778fc1f87
Merge pull request #3741 from hathach/fix-cdc-connection-race
update tinyusb to fix cdc connection race
2020-11-24 19:11:41 -06:00
Jeff Epler c451b22255 Disable 3-arg pow() function on m0 boards
`pow(a, b, c)` can compute `(a ** b) % c` efficiently (in time and memory).
This can be useful for extremely specific applications, like implementing
the RSA cryptosystem.  For typical uses of CircuitPython, this is not an
important feature.  A survey of the bundle and learn system didn't find
any uses.

Disable it on M0 builds so that we can fit in needed upgrades to the USB
stack.
2020-11-24 16:54:33 -06:00
Dan Halbert 7a45afc549 working, but need to avoid deep sleeping too fast before USB ready 2020-11-23 22:44:53 -05:00
Scott Shawcroft f8dcb25170
Merge pull request #3694 from jepler/update-ulab2
ulab: Update to release tag 1.1.0
2020-11-23 15:17:46 -08:00
Jeff Epler 9d8be648ee ulab: Update to release tag 1.1.0
Disable certain classes of diagnostic when building ulab.  We should
submit patches upstream to (A) fix these errors and (B) upgrade their
CI so that the problems are caught before we want to integrate with
CircuitPython, but not right now.
2020-11-23 10:23:50 -06:00
Jim Mussared 0e8af2b370 extmod/modbluetooth: Add API for L2CAP channels.
Also known as L2CAP "connection oriented channels". This provides a
socket-like data transfer mechanism for BLE.

Currently only implemented for NimBLE on STM32 / Unix.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-24 01:07:17 +11:00
Dan Halbert 75559f35cc wip: ResetReason to microcontroller.cpu 2020-11-21 23:29:52 -05:00
Dan Halbert e4c66990e2 compiles 2020-11-20 23:33:39 -05:00
Jeff Epler 982bce7259 py.mk: allow translation to be overriden in GNUmakefile
I like to use local makefile overrides, in the file GNUmakefile
(or, on case-sensitive systems, makefile) to set compilation choices.
However, writing
    TRANSLATION := de_DE
    include Makefile
did not work, because py.mk would override the TRANSLATION := specified
in an earlier part of the makefiles (but not from the commandline).

By using ?= instead of := the local makefile override works, but when
TRANSLATION is not specified it continues to work as before.
2020-11-19 16:23:35 -06:00
Jeff Epler b2b8520880 Always use preprocessor for MICROPY_ERROR_REPORTING
This ensures that only the translate("") alternative that will be used
is seen after preprocessing.  Improves the quality of the Huffman encoding
and reduces binary size slightly.

Also makes one "enhanced" error message only occur when ERROR_REPORTING_DETAILED:
Instead of the word-for-word python3 error message
"Type object has no attribute '%q'", the message will be
"'type' object has no attribute '%q'".  Also reduces binary size.
(that's rolled into this commit as it was right next to a change to
use the preprocessor for MICROPY_ERROR_REPORTING)

Note that the odd semicolon after "value_error:" in parsenum.c is necessary
due to a detail of the C grammar, in which a declaration cannot follow
a label directly.
2020-11-19 16:18:52 -06:00
Jeff Epler c06fc8e02d Introduce, use mp_raise_arg1
This raises an exception with a given object value.  Saves a bit of
code size.
2020-11-19 16:15:06 -06:00
Jeff Epler d5f6748d1b Use mp_raise instead of nlr_raise(new_exception) where possible
This saves a bit of code space
2020-11-19 16:13:01 -06:00
Jeff Epler 0556f9f851 Revert "samd21: Enable terse error reporting on resource constrained chip family"
This reverts commit 9a642fc049.
2020-11-19 15:12:56 -06:00
Dan Halbert 649c930536 wip 2020-11-19 15:43:39 -05:00
Dan Halbert 5bb3c321e9 merge from main 2020-11-19 00:29:14 -05:00
Dan Halbert 682054a216 WIP: redo API; not compiled yet 2020-11-19 00:23:27 -05:00
Jeff Epler 9a642fc049 samd21: Enable terse error reporting on resource constrained chip family
This reclaims over 1kB of flash space by simplifying certain exception
messages.  e.g., it will no longer display the requested/actual length
when a fixed list/tuple of N items is needed:

        if (MICROPY_ERROR_REPORTING == MICROPY_ERROR_REPORTING_TERSE) {
            mp_raise_ValueError(translate("tuple/list has wrong length"));
        } else {
            mp_raise_ValueError_varg(translate("requested length %d but object has length %d"),
                (int)len, (int)seq_len);

Other chip families including samd51 keep their current error reporting
capabilities.
2020-11-18 20:37:36 -06:00
Dan Halbert ffff02c053 Merge remote-tracking branch 'adafruit/main' into sleep 2020-11-16 12:06:11 -05:00
Dan Halbert bb77f1d130 wip: initial code changes, starting from @tannewt's sleepio branch 2020-11-16 11:56:20 -05:00
root 0cd951fb73 Prevent exceptions from accumulating in REPL 2020-11-16 10:36:05 -06:00
Scott Shawcroft bda3267432
Save flash space
* No weak link for modules. It only impacts _os and _time and is
  already disabled for non-full builds.
* Turn off PA00 and PA01 because they are the crystal on the Metro
  M0 Express.
* Change ejected default to false to move it to BSS. It is set on
  USB connection anyway.
* Set sinc_filter to const. Doesn't help flash but keeps it out of
  RAM.
2020-11-13 18:57:52 -08:00
Jim Mussared a7932ae4e6 tools/makeqstrdefs.py: Run qstr preprocessing in parallel.
This gives a substantial speedup of the preprocessing step, i.e. the
generation of qstr.i.last.  For example on a clean build, making
qstr.i.last:

    21s -> 4s on STM32 (WB55)
    8.9 -> 1.8s on Unix (dev).

Done in collaboration with @stinos.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2020-11-12 15:04:53 +11:00
Damien George d7e1526593 py/binary: Fix sign extension setting wide integer on 32-bit archs.
Signed-off-by: Damien George <damien@micropython.org>
2020-11-11 22:18:24 +11:00
Damien George 1fef5662ab py/mpz: Do sign extension in mpz_as_bytes for negative values.
Signed-off-by: Damien George <damien@micropython.org>
2020-11-11 22:18:24 +11:00
gamblor21 4c93db3595 Renamed to adafruit_bus_device 2020-11-03 18:35:20 -06:00
Dan Halbert 72b829dff0 add binascii to most builds 2020-11-01 14:52:03 -05:00
gamblor21 78477a374a Initial SPI commit 2020-10-31 12:17:29 -05:00
stijn 0153148fd2 py/py.mk: Support C++ code for user C modules.
Support C++ code in .cpp files by providing CXX counterparts of the
_USERMOD_ flags we have for C already.  This merely enables the Makefile of
user C modules to use variables specific to C++ compilation, it is still up
to each port's main Makefile to also include these in the build.
2020-10-29 15:29:20 +11:00
stijn e498a8bd13 py: Workaround clang error when building misc.h with C++ compiler. 2020-10-29 15:28:14 +11:00
stijn 8e94fa0d2e py/makeqstrdefs.py: Support preprocessing C++ files for QSTR generation.
When SCR_QSTR contains C++ files they should be preprocessed with the same
compiler flags (CXXFLAGS) as they will be compiled with, to make sure code
scanned for QSTR occurrences is effectively the code used in the rest of
the build.  The 'split SCR_QSTR in .c and .cpp files and process each with
different flags' logic isn't trivial to express in a Makefile and the
existing principle for deciding which files to preprocess was already
rather complicated, so the actual preprocessing is moved into
makeqstrdefs.py completely.
2020-10-29 15:27:30 +11:00
stijn f1666419a8 py/mkrules.mk: Add target for compiling C++ files.
Move the target from the ESP32 Makefile since that does what is needed
already, but also include files from user C modules as is done for the C
files.
2020-10-29 15:27:18 +11:00
stijn 2b9f0586e7 py/makeqstrdefs.py: Process C++ files as well.
Preprocessed C++ code isn't different from C code when it comes to QSTR
instances so process it as well.
2020-10-29 15:27:11 +11:00
stijn 1b723937e3 py/makeqstrdefs.py: Fix beaviour when scanning non-C preprocessed files.
When process_file() is passed a preprocessed C++ file for instance it won't
find any lines containing .c files and the last_fname variable remains
None, so handle that gracefully.
2020-10-29 15:26:35 +11:00
microDev 930cf14dce
Add check for invalid io, function to disable all alarms 2020-10-27 16:17:26 -07:00
microDev 59df1a11ad
Add alarm_touch module 2020-10-27 16:16:52 -07:00
microDev da449723df
Fix build error 2020-10-27 16:16:15 -07:00
microDev 4d8ffdca8d
restructure alarm modules 2020-10-27 16:15:09 -07:00
microDev e5ff55b15c
Renamed alarm modules 2020-10-27 16:13:25 -07:00
microDev e310b871c8
Get io wake working 2020-10-27 16:13:25 -07:00
microDev 90b9ec6f2c
Initial Sleep Support 2020-10-27 16:13:22 -07:00
gamblor21 b637d3911e Initial commit 2020-10-24 20:48:35 -05:00
Emil Renner Berthing ccd92335a1 py, extmod: Introduce and use MP_FALLTHROUGH macro.
Newer GCC versions are able to warn about switch cases that fall
through.  This is usually a sign of a forgotten break statement, but in
the few cases where a fall through is intended we annotate it with this
macro to avoid the warning.
2020-10-22 11:53:16 +02:00
Emil Renner Berthing f1f6ef7b17 py/vmentrytable: Ignore GCC -Woverride-init.
Like Clang, GCC warns about this file, but only with -Woverride-init
which is enabled by -Wextra. Disable the warnings for this file just
like we do for Clang to make -Wextra happy.
2020-10-22 11:47:36 +02:00
Emil Renner Berthing 9aa58cf8ba py, extmod: Add explicit initializers for default values.
When compiling with -Wextra which includes -Wmissing-field-initializers
GCC will warn that the defval field of mp_arg_val_t is not initialized.
This is just a warning as it is defined to be zero initialized, but since
it is a union it makes sense to be explicit about which member we're
going to use, so add the explicit initializers and get rid of the
warning.
2020-10-22 11:47:36 +02:00
Emil Renner Berthing fdd6fa389e py: Use unsigned comparison of chars.
On x86 chars are signed, but we're comparing a char to '0' + unsigned int,
which is promoted to an unsigned int. Let's promote the char to unsigned
before doing the comparison to avoid weird corner cases.
2020-10-22 11:47:36 +02:00
Emil Renner Berthing 6d3aa16443 py/objexcept: Compare mp_emergency_exception_buf_size signed.
mp_emergency_exception_buf_size is signed, so let's make sure we compare
it as such.
2020-10-22 11:47:36 +02:00
Emil Renner Berthing 6324c3e054 py/scope: Name and use id_kind_type_t.
The function scope_find_or_add_id used to take a scope_kind_t enum and
save it in an uint8_t. Saving an enum in a uint8_t is fine, but
everywhere this function is called it is not actually given a
scope_kind_t but an anonymous enum instead. Let's give this enum a name
and use that as the argument type.

This doesn't change the generated code, but is a C type mismatch that
unfortunately doesn't show up unless you enable -Wenum-conversion.
2020-10-22 11:40:56 +02:00
Christian Walther 1eab0692b5 Fix missing `nproc` on macOS.
396979a breaks building on macOS: `nproc` is a Linux thing, use a cross-platform alternative.
2020-10-20 16:39:32 +02:00
Dan Halbert 82b49afe43 enable CIRCUITPY_BLEIO_HCI on non-nRF boards where it will fit 2020-10-15 11:27:21 -04:00
Dan Halbert f1e8f2b404
Merge pull request #3554 from gamblor21/move_ordereddict
Moved ORDEREDDICT define to central location
2020-10-14 22:39:04 -04:00
gamblor21 f6f89565d8 Remove ordered dict from SAMD21 2020-10-14 20:18:49 -05:00
gamblor21 e6d0b207ec Removed MICROPY_PY_COLLECTIONS_NAMEDTUPLE__ASDICT from unix coverage 2020-10-14 14:06:34 -05:00
gamblor21 4270061db4 Moved ORDEREDDICT define to central location 2020-10-13 18:52:27 -05:00
Scott Shawcroft 9de96786ad
Merge pull request #3538 from jepler/parallel-qstrlast
build: parallelize the qstr build steps
2020-10-12 15:52:43 -07:00
Scott Shawcroft 1eb1434fc9
Merge pull request #3537 from jepler/update-protomatter-2
rgbmatrix: update protomatter to 1.0.5 tag
2020-10-12 15:45:51 -07:00
Scott Shawcroft 179e13f103
Merge pull request #3539 from jepler/lto-type-mismatch
remove warning-disable flag that seems unneeded now
2020-10-12 15:44:25 -07:00
Kenny 94beeabc51 remove unnecessary board configuration and address feedback 2020-10-11 22:42:59 -07:00
Jeff Epler 479552ce56 build: Make genlast write the "split" files
This gets a further speedup of about 2s (12s -> 9.5s elapsed build time)
for stm32f405_feather

For what are probably historical reasons, the qstr process involves
preprocessing a large number of source files into a single "qstr.i.last"
file, then reading this and splitting it into one "qstr" file for each
original source ("*.c") file.

By eliminating the step of writing qstr.i.last as well as making the
regular-expression-matching part be parallelized, build speed is further
improved.

Because the step to build QSTR_DEFS_COLLECTED does not access
qstr.i.last, the path is replaced with "-" in the Makefile.
2020-10-11 21:18:03 -05:00
Jeff Epler 607e4a905a build: parallelize the creation of qstr.i.last
Rather than simply invoking gcc in preprocessor mode with a list of files, use
a Python script with the (python3) ThreadPoolExecutor to invoke the
preprocessor in parallel.

The amount of concurrency is the number of system CPUs, not the makefile "-j"
parallelism setting, because there is no simple and correct way for a Python
program to correctly work together with make's idea of parallelism.

This reduces the build time of stm32f405 feather (a non-LTO build) from 16s to
12s on my 16-thread Ryzen machine.
2020-10-11 20:19:59 -05:00
Jeff Epler c139eccc92 remove warning that seems unneeded now 2020-10-11 16:23:02 -05:00
Kenny 98aa4b7943 update async tests with less upython workaround and more cpython compatibility 2020-10-10 23:39:32 -07:00
Kenny 5d96afc5c2 i do not know if this is needed but this is not the vm i use anymore 2020-10-10 15:45:08 -07:00
Kenny bf849ff674 async def syntax rigor and __await__ magic method
Some examples of improved compliance with CPython that currently
have divergent behavior in CircuitPython are listed below:

* yield from is not allowed in async methods
```
>>> async def f():
...     yield from 'abc'
...
Traceback (most recent call last):
  File "<stdin>", line 2, in f
SyntaxError: 'yield from' inside async function
```

* await only works on awaitable expressions
```
>>> async def f():
...     await 'not awaitable'
...
>>> f().send(None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in f
AttributeError: 'str' object has no attribute '__await__'
```

* only __await__()able expressions are awaitable
Okay this one actually does not work in circuitpython at all today.
This is how CPython works though and pretending __await__ does not
exist will only bite users who write both.
```
>>> class c:
...     pass
...
>>> def f(self):
...     yield
...     yield
...     return 'f to pay respects'
...
>>> c.__await__ = f  # could just as easily have put it on the class but this shows how it's wired
>>> async def g():
...     awaitable_thing = c()
...     partial = await awaitable_thing
...     return 'press ' + partial
...
>>> q = g()
>>> q.send(None)
>>> q.send(None)
>>> q.send(None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration: press f to pay respects
```
2020-10-10 15:45:08 -07:00
warriorofwire 5cadf525bd fix missing cflag defeating the board gating 2020-10-10 15:45:08 -07:00
warriorofwire d94d2d2975 Add async/await syntax to FULL_BUILD
This adds the `async def` and `await` verbs to valid CircuitPython syntax using the Micropython implementation.

Consider:
```
>>> class Awaitable:
...     def __iter__(self):
...         for i in range(3):
...             print('awaiting', i)
...             yield
...         return 42
...
>>> async def wait_for_it():
...     a = Awaitable()
...     result = await a
...     return result
...
>>> task = wait_for_it()
>>> next(task)
awaiting 0
>>> next(task)
awaiting 1
>>> next(task)
awaiting 2
>>> next(task)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  StopIteration: 42
>>>
```

and more excitingly:
```
>>> async def it_awaits_a_subtask():
...     value = await wait_for_it()
...     print('twice as good', value * 2)
...
>>> task = it_awaits_a_subtask()
>>> next(task)
awaiting 0
>>> next(task)
awaiting 1
>>> next(task)
awaiting 2
>>> next(task)
twice as good 84
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  StopIteration:
```

Note that this is just syntax plumbing, not an all-encompassing implementation of an asynchronous task scheduler or asynchronous hardware apis.
  uasyncio might be a good module to bring in, or something else - but the standard Python syntax does not _strictly require_ deeper hardware
  support.
Micropython implements the await verb via the __iter__ function rather than __await__.  It's okay.

The syntax being present will enable users to write clean and expressive multi-step state machines that are written serially and interleaved
  according to the rules provided by those users.

Given that this does not include an all-encompassing C scheduler, this is expected to be an advanced functionality until the community settles
  on the future of deep hardware support for async/await in CircuitPython.  Users will implement yield-based schedulers and tasks wrapping
  synchronous hardware APIs with polling to avoid blocking, while their application business logic gets simple `await` statements.
2020-10-10 15:38:40 -07:00
Jeff Epler 5e38bb98cb rgbmatrix: update protomatter to 1.0.5 tag
this is compile-tested on
 stm32f405 feather
 matrixportal
 nrf52840 feather

but not actually tested-tested.
2020-10-10 14:30:37 -05:00
Jim Mussared b137d064e9 py/objtype: Handle __dict__ attribute when type has no locals. 2020-10-10 00:16:32 +11:00
Jim Mussared 880875bea1 py/objdict: Add mp_const_empty_dict_obj, use it for mp_const_empty_map. 2020-10-10 00:16:26 +11:00
Damien George 843dcd4f85 py/parse: Expose rule-name printing as MICROPY_DEBUG_PARSE_RULE_NAME.
So it can be enabled without modifying the source.

Signed-off-by: Damien George <damien@micropython.org>
2020-10-01 15:26:43 +10:00
Mike Wadsten c711c0049e py/makeversionhdr.py: Match only git tags which look like versions.
Some downstream projects may use tags in their repositories for more than
just designating MicroPython releases.  In those cases, the
makeversionhdr.py script would end up using a different tag than intended.
So tell `git describe` to only match tags that look like a MicroPython
version tag, such as `v1.12` or `v2.0`.
2020-10-01 11:01:43 +10:00
Jeff Epler a4cc3ad6cb canio: RemoteTransmissionRequest: Split implementation, keep one structure
This already begins obscuring things, because now there are two sets of
shared-module functions for manipulating the same structure, e.g.,
common_hal_canio_remote_transmission_request_get_id and
common_hal_canio_message_get_id
2020-09-28 17:22:00 -05:00
Damien George 50e34f979c py/objarray.h: Add mp_obj_memoryview_init() helper function.
Signed-off-by: Damien George <damien@micropython.org>
2020-09-25 12:23:11 +10:00
Iyassou Shimels ca017841d6 py/objstr: Make bytes(bytes_obj) return bytes_obj.
Calling the bytes constructor on a bytes object returns the original bytes
object.  This saves allocating a new instance, and matches CPython.

Signed-off-by: Iyassou Shimels <s.iyassou@gmail.com>
2020-09-24 11:04:58 +10:00
Scott Shawcroft a8558a48ed
Merge pull request #3456 from jepler/qstr-and-or-demagic
makeqstrdefs: don't make _and_, _or_ poisoned substrings for QSTRs
2020-09-23 12:24:02 -07:00
Jeff Epler 28e80e47d7 makeqstrdefs: don't make _and_, _or_ poisoned substrings for QSTRs
New contributor @mdroberts1243 encountered an interesting problem in
which the argument they had named "column_underscore_and_page_addressing"
simply couldn't be used; I discovered that internally this had been
transformed into "column_underscore∧page_addressing", because QSTR
makes _ENTITY_ stand for the same thing as &ENTITY; does in HTML.

This might be nice for some things, but we don't want it here!
I was unable to find a sensible way to "escape" and prevent this entity
coding, so instead I ripped out support for the _and_ and _or_ escapes.
2020-09-22 17:39:44 -05:00
Jeff Epler 4869dbdc67 canio: rename from _canio
This reflects our belief that the API is stable enough to avoid incompatible changes during 6.x.
2020-09-21 16:44:26 -05:00
Jeff Epler a2e1867f69 _canio: Minimal implementation for SAM E5x MCUs
Tested & working:

 * Send standard packets
 * Receive standard packets (1 FIFO, no filter)

Interoperation between SAM E54 Xplained running this tree and
MicroPython running on STM32F405 Feather with an external
transceiver was also tested.

Many other aspects of a full implementation are not yet present,
such as error detection and recovery.
2020-09-21 16:44:26 -05:00
Jeff Epler e7a213a114 py: Add enum helper code
This makes it much easier to implement enums, and the printing code is
shared.  We might want to convert other enums to this in the future.
2020-09-21 16:44:26 -05:00
Jeff Epler 0318eb359f makeqstrdata: Work around python3.6 compatibility problem
Discord user Folknology encountered a problem building with Python 3.6.9,
`TypeError: ord() expected a character, but string of length 0 found`.

I was able to reproduce the problem using Python3.5*, and discovered that
the meaning of the regular expression `"|."` had changed in 3.7.  Before,
```
>>> [m.group(0) for m in re.finditer("|.", "hello")]
['', '', '', '', '', '']
```
After:
```
>>> [m.group(0) for m in re.finditer("|.", "hello")]
['', 'h', '', 'e', '', 'l', '', 'l', '', 'o', '']
```
Check if `words` is empty and if so use `"."` as the regular expression
instead.  This gives the same result on both versions:
```
['h', 'e', 'l', 'l', 'o']
```
and fixes the generation of the huffman dictionary.

Folknology verified that this fix worked for them.

 * I could easily install 3.5 but not 3.6.  3.5 reproduced the same problem
2020-09-21 10:03:07 -05:00
Jeff Epler bfbbbd6c5c makeqstrdata: Work with older Python
This construct (which I added without sufficient testing,
apparently) is only supported in Python 3.7 and newer.  Make it
optional so that this script works on other Python versions.  This
means that if you have a system with non-UTF-8 encoding you will
need to use Python 3.7.

In particular, this affects a problem building circuitpython in
github's ubuntu-18.04 virtual environment when Python 3.7 is not
explicitly installed.  cookie-cuttered libraries call for Python
3.6:
```
    - name: Set up Python 3.6
      uses: actions/setup-python@v1
      with:
        python-version: 3.6
```
Since CircuitPython's own build calls for 3.8, this problem was not
detected.

This problem was also encountered by discord user mdroberts1243.

The failure I encountered was here:
https://github.com/jepler/Jepler_CircuitPython_udecimal/runs/1138045020?check_suite_focus=true
.. while my step of "clone and build circuitpython unix port" is
unusual, I think the same problem would have affected "build assets"
if that step had been reached.
2020-09-19 10:16:13 -05:00
Jim Mussared 9d1983f078 py/dynruntime.h: Add mp_import_* and mp_load/store_*.
These functions already exist in the fun table, and this commit just adds
convenience macros for them.
2020-09-18 18:34:02 +10:00
Damien George 8f20cdc353 all: Rename absolute time-based functions to include "epoch".
For time-based functions that work with absolute time there is the need for
an Epoch, to set the zero-point at which the absolute time starts counting.
Such functions include time.time() and filesystem stat return values.  And
different ports may use a different Epoch.

To make it clearer what functions use the Epoch (whatever it may be), and
make the ports more consistent with their use of the Epoch, this commit
renames all Epoch related functions to include the word "epoch" in their
name (and remove references to "2000").

Along with this rename, the following things have changed:

- mp_hal_time_ns() is now specified to return the number of nanoseconds
  since the Epoch, rather than since 1970 (but since this is an internal
  function it doesn't change anything for the user).

- littlefs timestamps on the esp8266 have been fixed (they were previously
  off by 30 years in nanoseconds).

Otherwise, there is no functional change made by this commit.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-18 17:20:34 +10:00
Scott Shawcroft 750bc1e04a
Merge pull request #3398 from jepler/better-dictionary-compression
compression: Implement @ciscorn's dictionary approach
2020-09-16 11:10:22 -07:00
Jeff Epler a8e98cda83 makeqstrdata: comment my understanding of @ciscorn's code 2020-09-16 08:28:15 -05:00
Kamil Tomaszewski c2fc592c2c camera: Change API 2020-09-14 13:11:15 +02:00
Kamil Tomaszewski 064c597b60 camera: Implement new library for camera 2020-09-14 13:11:15 +02:00
Jeff Epler 90f7340bfc move implicit-fallthrough warning enable to defns.mk 2020-09-13 13:13:09 -05:00
Taku Fukada d18d79ac47 Small improvements to the dictionary compression 2020-09-14 01:50:01 +09:00
Jeff Epler 15964a4750 makeqstrdata: Avoid encoding problems
Most users and the CI system are running in configurations where Python
configures stdout and stderr in UTF-8 mode.  However, Windows is different,
setting values like CP1252.  This led to a build failure on Windows, because
makeqstrdata printed Unicode strings to its stdout, expecting them to be
encoded as UTF-8.

This script is writing (stdout) to a compiler input file and potentially
printing messages (stderr) to a log or console.  Explicitly configure stdout to
use utf-8 to get consistent behavior on all platforms, and configure stderr so
that if any log/diagnostic messages are printed that cannot be displayed
correctly, they are still displayed instead of creating an error while trying
to print the diagnostic information.

I considered setting the encodings both to ascii, but this would just be
occasionally inconvenient to developers like me who want to show diagnostic
info on stderr and in comments while working with the compression code.

Closes: #3408
2020-09-12 19:43:08 -05:00
Jeff Epler 12d826d941 Add FALLTHROUGH comments as needed
I investigated these cases and confirmed that the fallthrough behavior
was intentional.
2020-09-12 15:11:29 -05:00
Jeff Epler 54d97251fe modstruct: Improve compliance with python3
While checking whether we can enable -Wimplicit-fallthrough, I encountered
a diagnostic in mp_binary_set_val_array_from_int which led to discovering
the following bug:
```
>>> struct.pack("xb", 3)
b'\x03\x03'
```
That is, the next value (3) was used as the value of a padding byte, while
standard Python always fills "x" bytes with zeros.  I initially thought
this had to do with the unintentional fallthrough, but it doesn't.
Instead, this code would relate to an array.array with a typecode of
padding ('x'), which is ALSO not desktop Python compliant:
```
>>> array.array('x', (1, 2, 3))
array('x', [1, 0, 0])
```
Possibly this is dead code that used to be shared between struct-setting
and array-setting, but it no longer is.

I also discovered that the argument list length for struct.pack
and struct.pack_into were not checked, and that the length of binary data
passed to array.array was not checked to be a multiple of the element
size.

I have corrected all of these to conform more closely to standard Python
and revised some tests where necessary.  Some tests for micropython-specific
behavior that does not conform to standard Python and is not present
in CircuitPython was deleted outright.
2020-09-12 14:07:23 -05:00
Jeff Epler 40ab5c6b21 compression: Implement ciscorn's dictionary approach
Massive savings.  Thanks so much @ciscorn for providing the initial
code for choosing the dictionary.

This adds a bit of time to the build, both to find the dictionary
but also because (for reasons I don't fully understand), the binary
search in the compress() function no longer worked and had to be
replaced with a linear search.

I think this is because the intended invariant is that for codebook
entries that encode to the same number of bits, the entries are ordered
in ascending value.  However, I mis-placed the transition from "words"
to "byte/char values" so the codebook entries for words are in word-order
rather than their code order.

Because this price is only paid at build time, I didn't care to determine
exactly where the correct fix was.

I also commented out a line to produce the "estimated total memory size"
-- at least on the unix build with TRANSLATION=ja, this led to a build
time KeyError trying to compute the codebook size for all the strings.
I think this occurs because some single unicode code point ('ァ') is
no longer present as itself in the compressed strings, due to always
being replaced by a word.

As promised, this seems to save hundreds of bytes in the German translation
on the trinket m0.

Testing performed:
 - built trinket_m0 in several languages
 - built and ran unix port in several languages (en, de_DE, ja) and ran
   simple error-producing codes like ./micropython -c '1/0'
2020-09-12 10:10:45 -05:00
Damien George acdb0608b7 py/parse: Pass in an mp_print_t to mp_parse_node_print.
So the output can be redirected if needed.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 23:00:03 +10:00
Damien George 85f2b239d8 py/showbc: Pass in an mp_print_t struct to all bytecode-print functions.
So the output can be redirected if needed.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 17:22:28 +10:00
stijn 2e54d9d146 py: Fix handling of NaN in certain pow implementations.
Adds a new compile-time option MICROPY_PY_MATH_POW_FIX_NAN for use with
toolchains that don't handle pow-of-NaN correctly.
2020-09-11 10:04:57 +10:00
Damien George 8d5a40c86e py/objfloat: Fix handling of negative float to power of nan.
Prior to this commit, pow(-2, float('nan')) would return (nan+nanj), or
raise an exception on targets that don't support complex numbers.  This is
fixed to return simply nan, as CPython does.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-11 10:03:57 +10:00
Scott Shawcroft 1ba28b3edc
Merge pull request #3370 from jepler/compression-bigrams
add bigram compression to makeqstrdata (save ~100 bytes on trinket m0 de_DE)
2020-09-10 11:44:56 -07:00
Scott Shawcroft 683462c1b1
Merge pull request #3326 from tannewt/native_wifi
Add native wifi API with ESP32S2 support
2020-09-10 11:20:44 -07:00
Jeff Epler bdb07adfcc translations: Make decompression clearer
Now this gets filled in with values e.g., 128 (0x80) and 159 (0x9f).
2020-09-08 19:07:53 -05:00
Jeff Epler 73858ea682 circuitpy_mpconfig: enable 3-arg pow() with CIRCUITPY_FULL_BUILD
This is needed for a port of python3's decimal.py module.
2020-09-06 10:07:57 -05:00
Jeff Epler 20c2dd0c08 core: add int.bit_length() when MICROPY_CYPTHON_COMPAT is enabled
This method of integer objects is needed for a port of python3's
decimal.py module.

MICROPY_CPYTHON_COMPAT is enabled by CIRCUITPY_FULL_BUILD.
2020-09-06 09:53:16 -05:00
Scott Shawcroft 96cf60fbbd
Merge remote-tracking branch 'adafruit/main' into native_wifi 2020-09-03 16:34:56 -07:00
Scott Shawcroft 0b94638aeb
Changes based on Dan's feedback 2020-09-03 16:32:12 -07:00
stijn 40ad8f1666 all: Rename "sys" module to "usys".
This is consistent with the other 'micro' modules and allows implementing
additional features in Python via e.g. micropython-lib's sys.

Note this is a breaking change (not backwards compatible) for ports which
do not enable weak links, as "import sys" must now be replaced with
"import usys".
2020-09-04 00:10:24 +10:00
Jeff Epler cbfd38d1ce Rename functions to encode_ngrams / decode_ngrams 2020-09-02 19:09:23 -05:00
Jeff Epler c34cb82ecb makeqstrdata: correct range of low code points to 0x80..0x9f inclusive
The previous range was unintentionally big and overlaps some characters
we'd like to use (and also 0xa0, which we don't intentionally use)
2020-09-02 15:52:02 -05:00
Damien George b0932fcf2e all: Bump version to 1.13.
Signed-off-by: Damien George <damien@micropython.org>
2020-09-02 12:01:26 +10:00
Jeff Epler 07740d19f3 add bigram compression to makeqstrdata
Compress common unicode bigrams by making code points in the range
0x80 - 0xbf (inclusive) represent them.  Then, they can be greedily
encoded and the substituted code points handled by the existing Huffman
compression.  Normally code points in the range 0x80-0xbf are not used
in Unicode, so we stake our own claim.  Using the more arguably correct
"Private Use Area" (PUA) would mean that for scripts that only use
code points under 256 we would use more memory for the "values" table.

bigram means "two letters", and is also sometimes called a "digram".
It's nothing to do with "big RAM".  For our purposes, a bigram represents
two successive unicode code points, so for instance in our build on
trinket m0 for english the most frequent are:
['t ', 'e ', 'in', 'd ', ...].

The bigrams are selected based on frequency in the corpus, but the
selection is not necessarily optimal, for these reasons I can think of:
 * Suppose the corpus was just "tea" repeated 100 times.  The
   top bigrams would be "te", and "ea".  However,
   overlap, "te" could never be used.  Thus, some bigrams might actually
   waste space
    * I _assume_ this has to be why e.g., bigram 0x86 "s " is more
      frequent than bigram 0x85 " a" in English for Trinket M0, because
      sequences like "can't add" would get the "t " digram and then
      be unable to use the " a" digram.

 * And generally, if a bigram is frequent then so are its constituents.
   Say that "i" and "n" both encode to just 5 or 6 bits, then the huffman
   code for "in" had better compress to 10 or fewer bits or it's a net
   loss!
    * I checked though!  "i" is 5 bits, "n" is 6 bits (lucky guess)
      but the bigram 0x83 also just 6 bits, so this one is a win of
      5 bits for every "it" minus overhead.  Yay, this round goes to team
      compression.
    * On the other hand, the least frequent bigram 0x9d " n" is 10 bits
      long and its constituent code points are 4+6 bits so there's no
      savings, but there is the cost of the table entry.
    * and somehow 0x9f 'an' is never used at all!

With or without accounting for overlaps, there is some optimum number
of bigrams.  Adding one more bigram uses at least 2 bytes (for the
entry in the bigram table; 4 bytes if code points >255 are in the
source text) and also needs a slot in the Huffman dictionary, so
adding bigrams beyond the optimim number makes compression worse again.

If it's an improvement, the fact that it's not guaranteed optimal
doesn't seem to matter too much.  It just leaves a little more fruit
for the next sweep to pick up.  Perhaps try adding the most frequent
bigram not yet present, until it doesn't improve compression overall.

Right now, de_DE is again the "fullest" build on trinket_m0.  (It's
reclaimed that spot from the ja translation somehow)  This change saves
104 bytes there, increasing free space about 6.8%.  In the larger
(but not critically full) pyportal build it saves 324 bytes.

The specific number of bigrams used (32) was chosen as it is the max
number that fit within the 0x80..0xbf range.  Larger tables would
require the use of 16 bit code points in the de_DE build, losing savings
overall.

(Side note: The most frequent letters in English have been said
to be: ETA OIN SHRDLU; but we have UAC EIL MOPRST in our corpus)
2020-09-01 17:12:22 -05:00
Scott Shawcroft f0e60da51f
Merge pull request #3310 from dhalbert/ble_hci
_bleio HCI implementation
2020-09-01 11:28:05 -07:00
Dan Halbert 6dbd369272 merge from upstream 2020-08-30 14:39:03 -04:00
Dan Halbert b27d511251 address review; use constructor for HCI Adapter 2020-08-30 14:06:48 -04:00
Jeff Epler 455226ffde builtinimport: Fix a crash with 'import ulab.linalg' on unix port only
A crash like the following occurs in the unix port:
```
Program received signal SIGSEGV, Segmentation fault.
0x00005555555a2d7a in mp_obj_module_set_globals (self_in=0x55555562c860 <ulab_user_cmodule>, globals=0x55555562c840 <mp_module_ulab_globals>) at ../../py/objmodule.c:145
145	    self->globals = globals;
(gdb) up
#1  0x00005555555b2781 in mp_builtin___import__ (n_args=5, args=0x7fffffffdbb0) at ../../py/builtinimport.c:496
496	                mp_obj_module_set_globals(outer_module_obj,
(gdb)
#2  0x00005555555940c9 in mp_import_name (name=824, fromlist=0x555555621f10 <mp_const_none_obj>, level=0x1) at ../../py/runtime.c:1392
1392	    return mp_builtin___import__(5, args);
```

I don't understand how it doesn't happen on the embedded ports, because
the module object should reside in ROM and the assignment of self->globals
should trigger a Hard Fault.

By checking VERIFY_PTR, we know that the pointed-to data is on the heap
so we can do things like mutate it.
2020-08-30 11:09:49 -05:00
Damien George 06659077a8 all: Update Python code to conform to latest black formatting.
Updating to Black v20.8b1 there are two changes that affect the code in
this repository:

- If there is a trailing comma in a list (eg [], () or function call) then
  that list is now written out with one line per element.  So remove such
  trailing commas where the list should stay on one line.

- Spaces at the start of """ doc strings are removed.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-29 15:18:01 +10:00
Scott Shawcroft 767ca5c3dc
Merge remote-tracking branch 'adafruit/main' into native_wifi 2020-08-27 11:42:31 -07:00
Jeff Epler 2e0a109331
Merge pull request #3318 from jepler/interrupt-serial-rx
supervisor: check for interrupt during rx_chr
2020-08-25 21:01:33 -05:00
Scott Shawcroft 8b71e26abd
Merge remote-tracking branch 'adafruit/main' into native_wifi 2020-08-25 16:39:23 -07:00
Jeff Epler c0753c1afb mp_obj_print_helper: Handle a ctrl-c that comes in during printing
In #2689, hitting ctrl-c during the printing of an object with a lot of sub-objects could cause the screen to stop updating (without showing a KeyboardInterrupt).  This makes the printing of such objects acutally interruptable, and also correctly handles the KeyboardInterrupt:

```
>>> l = ["a" * 100] * 200
>>> l
['aaaaaaaaaaaaaaaaaaaaaa...aaaaaaaaaaa', Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyboardInterrupt:
>>>
```
2020-08-25 11:47:50 -05:00
Damien George ee50a6effe py/mphal.h: Introduce mp_hal_time_ns and implement on various ports.
This should return a 64-bit value being the number of nanoseconds since
1970/1/1.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 16:13:44 +10:00
Damien George 5f9b105244 py/runtime: Fix builtin compile() in "single" mode so it prints exprs.
As per CPython behaviour, compile(stmt, "file", "single") should create
code which prints to stdout (via __repl_print__) the results of any
expressions in stmt.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 11:38:46 +10:00
Scott Shawcroft 701e80a025
Make socket reads interruptable 2020-08-21 11:00:02 -07:00
Dan Halbert 0e30dd8bcc merge from upstream; working; includes debug_out code for debugging via Saleae for posterity 2020-08-20 20:29:57 -04:00
Scott Shawcroft eb8b42aff1
Add basic error handling 2020-08-19 14:23:28 -07:00
Scott Shawcroft 1034cc1217
Add espidf module. 2020-08-19 14:23:28 -07:00
Scott Shawcroft 430530c74b
SSL works until it runs out of memory 2020-08-19 14:23:28 -07:00
Scott Shawcroft c9ece21c28
SocketPool stubbed out 2020-08-19 14:22:13 -07:00
Scott Shawcroft 3860991111
Ping work and start to add socketpool 2020-08-19 14:22:13 -07:00
Scott Shawcroft c53a72d3f5
Fix ipaddress import and parse ipv4 strings 2020-08-19 14:22:13 -07:00
Scott Shawcroft c62ab6e09a
Add ipaddress 2020-08-19 14:22:12 -07:00
Scott Shawcroft 1a6f4e0fe0
Scanning WIP. Need to sort out supervisor memory 2020-08-19 14:22:12 -07:00
Scott Shawcroft c5b8401a15
First crack at native wifi API 2020-08-19 14:21:59 -07:00
Scott Shawcroft 6857f98426
Split pulseio.PWMOut into pwmio
This gives us better granularity when implementing new ports because
PWMOut is commonly implemented before PulseIn and PulseOut.

Fixes #3211
2020-08-18 13:08:33 -07:00
Scott Shawcroft 24ca5c0218
Merge pull request #3295 from tannewt/turn_off_terminalio
Turn off terminalio for ja and ko
2020-08-18 12:10:31 -07:00
Taku Fukada 79a3796b1c Calculate the Huffman codebook without MP_QSTRs 2020-08-18 23:21:14 +09:00
Scott Shawcroft d01f5dc0bd
Turn off terminalio for ja and ko
The font is missing many characters and the build needs the space.
We can optimize font storage when we get a good font.

The serial output will work as usual.
2020-08-17 17:17:59 -07:00
Jeff Epler 08ed09acc6 makeqstrdata: don't print "compression incrased length" messages
This check as implemented is misleading, because it compares the
compressed size in bytes (including the length indication) with the source
string length in Unicode code points.  For English this is approximately
fair, but for Japanese this is quite unfair and produces an excess of
"increased length" messages.

This message might have existed for one of two reasons:
 * to alert to an improperly function huffman compression
 * to call attention to a need for a "string is stored uncompressed" case
We know by now that the huffman compression is functioning as designed and
effective in general.

Just to be on the safe side, I did some back-of-the-envelope estimates.
I considered these three replacements for "the true source string size, in bytes":
+    decompressed_len_utf8 = len(decompressed.encode('utf-8'))
+    decompressed_len_utf16 = len(decompressed.encode('utf-16be'))
+    decompressed_len_bitsize = ((1+len(decompressed)) * math.ceil(math.log(1+len(values), 2)) + 7) // 8

The third counts how many bits each character requires (fewer than 128
characters in the source character set = 7, fewer than 256 = 8, fewer than 512
= 9, etc, adding a string-terminating value) and is in some way representative
of the best way we would be able to store "uncompressed strings".  The Japanese
translation (largest as of writing) has just a few strings which increase by
this metric.  However, the amount of loss due to expansion in those cases is
outweighed by the cost of adding 1 bit per string to indicate whether it's
compressed or not.  For instance, in the BOARD=trinket_m0 TRANSLATION=ja build
the loss is 47 bytes over 300 strings.  Adding 1 bit to each of 300 strings will
cost about 37 bytes, leaving just 5 Thumb instructions to implement the code to
check and decode "uncompressed" strings in order to break even.
2020-08-16 20:50:48 -05:00
Jeff Epler cff448205f Don't define SHARPDISPLAY when !DISPLAYIO
.. even if FULL_BUILD
2020-08-12 07:39:28 -05:00
Jeff Epler c1400bae9b sharpmemory: Implement support for Sharp Memory Displays in framebufferio 2020-08-12 07:32:18 -05:00
Jeff Epler 93b373d617 "pop from empty %q"
Saves 12 bytes code on trinket m0
2020-08-04 18:42:09 -05:00
Jeff Epler 65e26f4a06 py: mp_obj_get_type_qstr as macro saves 24 bytes 2020-08-04 14:45:45 -05:00
Jeff Epler 024c8da578 Combine some "can't convert" messages 2020-08-04 14:45:45 -05:00
Jeff Epler c849b781c0 Combine 'index out of range' messages 2020-08-04 14:45:45 -05:00
Jeff Epler 89797fd3f9 various: Use mp_obj_get_type_qstr more widely
This removes runtime allocations of the cstring version of the qstring.

It is not a size improvement
2020-08-04 14:45:45 -05:00
Jeff Epler c37a25f0e5 Use qstrs to save an additional 4 bytes 2020-08-04 14:45:45 -05:00
Jeff Epler 92917b84f1 fix exception type for pop from empty set 2020-08-04 13:58:29 -05:00
Jeff Epler 67eb93fc98 py: introduce, use mp_raise_msg_vlist
This saves a very small amount of flash, 8 bytes on trinket_m0
2020-08-04 13:34:29 -05:00
Jeff Epler dddd25a776 Combine similar strings to reduce size of translations
This is a slight trade-off with code size, in places where a "_varg"
mp_raise variant is now used.  The net savings on trinket_m0 is
just 32 bytes.

It also means that the translation will include the original English
text, and cannot be translated.  These are usually names of Python
types such as int, set, or dict or special values such as "inf" or
"Nan".
2020-08-04 13:34:29 -05:00
Dan Halbert 0a60aee3e4 wip: compiles 2020-08-02 11:36:38 -04:00
Damien George 9883d8e818 py/persistentcode: Maintain root ptr list of imported native .mpy code.
On ports where normal heap memory can contain executable code (eg ARM-based
ports such as stm32), native code loaded from an .mpy file may be reclaimed
by the GC because there's no reference to the very start of the native
machine code block that is reachable from root pointers (only pointers to
internal parts of the machine code block are reachable, but that doesn't
help the GC find the memory).

This commit fixes this issue by maintaining an explicit list of root
pointers pointing to native code that is loaded from an .mpy file.  This
is not needed for all ports so is selectable by the new configuration
option MICROPY_PERSISTENT_CODE_TRACK_RELOC_CODE.  It's enabled by default
if a port does not specify any special functions to allocate or commit
executable memory.

A test is included to test that native code loaded from an .mpy file does
not get reclaimed by the GC.

Fixes #6045.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-02 22:34:09 +10:00
Jeff Epler d69f081c04 Merge remote-tracking branch 'origin/main' into blm_badge 2020-07-30 07:24:48 -05:00
Scott Shawcroft 61d1148bb3
Merge pull request #3222 from WarriorOfWire/pick_micropython
py/compile: Don't await __aiter__ special method in async-for.
2020-07-29 10:54:37 -07:00
Jeff Epler 9b8df7f635 Upgrade ulab
This version
 * moves source files to reflect module structure
 * adds inline documentation suitable for extract_pyi
 * incompatibly moves spectrogram to fft
 * incompatibly removes "extras"

There are some remaining markup errors in the specific revision of
extmod/ulab but they do not prevent the doc building process from
completing.
2020-07-28 16:57:48 -05:00
Dan Halbert aa97ea2501 Merge remote-tracking branch 'adafruit/main' into blm_badge 2020-07-28 14:15:02 -04:00
Dan Halbert e5e132a364 add blm_badge; add CIRCUITPY_AUDIOBUSIOIO_I2SOUT 2020-07-28 11:49:54 -04:00
Jerry Needell 2bdd62220e adjust stack for SAMD21 to accomodate larger pystack -- update frozen module adafruit_busdevice 2020-07-27 21:50:36 -04:00
Jonathan Hogg 901f3dce6e py/compile: Don't await __aiter__ special method in async-for.
MicroPython's original implementation of __aiter__ was correct for an
earlier (provisional) version of PEP492 (CPython 3.5), where __aiter__ was
an async-def function.  But that changed in the final version of PEP492 (in
CPython 3.5.2) where the function was changed to a normal one.  See
https://www.python.org/dev/peps/pep-0492/#why-aiter-does-not-return-an-awaitable
See also the note at the end of this subsection in the docs:
https://docs.python.org/3.5/reference/datamodel.html#asynchronous-iterators
And for completeness the BPO: https://bugs.python.org/issue27243

To be consistent with the Python spec as it stands today (and now that
PEP492 is final) this commit changes MicroPython's behaviour to match
CPython:  __aiter__ should return an async-iterable object, but is not
itself awaitable.

The relevant tests are updated to match.

See #6267.
2020-07-24 22:55:37 -07:00
Kenny 9a1f1236cc require async for and async with to actually be in an async def method instead of just a generator 2020-07-24 19:47:34 -07:00
Jonathan Hogg 37e1b5c891 py/compile: Don't await __aiter__ special method in async-for.
MicroPython's original implementation of __aiter__ was correct for an
earlier (provisional) version of PEP492 (CPython 3.5), where __aiter__ was
an async-def function.  But that changed in the final version of PEP492 (in
CPython 3.5.2) where the function was changed to a normal one.  See
https://www.python.org/dev/peps/pep-0492/#why-aiter-does-not-return-an-awaitable
See also the note at the end of this subsection in the docs:
https://docs.python.org/3.5/reference/datamodel.html#asynchronous-iterators
And for completeness the BPO: https://bugs.python.org/issue27243

To be consistent with the Python spec as it stands today (and now that
PEP492 is final) this commit changes MicroPython's behaviour to match
CPython:  __aiter__ should return an async-iterable object, but is not
itself awaitable.

The relevant tests are updated to match.

See #6267.
2020-07-25 00:58:18 +10:00
Kenny e9b4e0bd35 remove new char*s because m0 is way oversubscribed 2020-07-23 20:41:10 -07:00
Kenny 51a79b1af7 add coroutine behavior for generators
coroutines don't have __next__; they also call themselves coroutines.
This does not change the fact that `async def` methods are generators,
but it does make them behave more like CPython.
2020-07-23 20:40:16 -07:00
Dan Halbert f6f45c82a1 wip: ATT protocol 2020-07-23 18:54:26 -04:00
Damien George f43834aba2
py/py.mk: Use additional CFLAGS to compile string0.c.
Otherwise functions like memset might get optimised to call themselves (eg
with gcc 10).  And provide CFLAGS_BUILTIN so these options can be changed
by a port if needed.

Fixes issue #6053.
2020-07-22 16:26:47 -07:00
Scott Shawcroft 2bd6d05663
Add externs. GCC10 complains about duplicate defines 2020-07-22 16:26:46 -07:00
Scott Shawcroft 1ec3580946
Merge branch 'main' into memmonitor 2020-07-22 12:24:19 -07:00
Scott Shawcroft 6db10f9c7d
Turn off find when CPYTHON_COMPAT is off 2020-07-21 15:40:51 -07:00
Scott Shawcroft 1b3bc98068
Merge remote-tracking branch 'adafruit/main' into bytearray_find 2020-07-21 15:12:04 -07:00
Scott Shawcroft cb7df2e1ed
Merge pull request #3160 from tannewt/enable_pystack
Enable PYSTACK to keep function state out of the heap
2020-07-21 13:38:52 -07:00
Scott Shawcroft 900edb2b8e
Only add .find without CPYTHON_COMPAT 2020-07-21 13:08:42 -07:00
Damien George a853fff838 py/obj.h: Fix mp_seq_replace_slice_no_grow to use memmove not memcpy.
Because the argument arrays may overlap, as show by the new tests in this
commit.

Also remove the debugging comments for these macros, add a new comment
about overlapping regions, and separate the macros by blank lines to make
them easier to read.

Fixes issue #6244.

Signed-off-by: Damien George <damien@micropython.org>
2020-07-21 01:05:59 +10:00
Scott Shawcroft f38a9c8d29
Add cast for mpy-cross warning 2020-07-17 17:55:06 -07:00
Scott Shawcroft a1e4814a27
Get AllocationAlarm working 2020-07-17 17:15:03 -07:00
Scott Shawcroft 518d909b2c
Add memorymonitor module 2020-07-17 17:15:03 -07:00
Scott Shawcroft 4b157aa6b8
Add find varients to bytearray 2020-07-17 17:07:37 -07:00
Scott Shawcroft 1160635608
Enable PYSTACK to keep function state out of the heap 2020-07-17 17:03:42 -07:00
DavePutz e5f7adcf5d
Fix to pass mpy-cross build 2020-07-13 22:54:52 -05:00
DavePutz 14799f9ee6
more formatting fix 2020-07-13 22:43:46 -05:00
DavePutz 30361f6f2a
Fix formatting 2020-07-13 22:39:30 -05:00
DavePutz e963cff72d
Issue #2949 Run background checks during long multiplications 2020-07-13 22:27:06 -05:00
Scott Shawcroft 51c888d4be
Merge pull request #3003 from Flameeyes/master
License tagging according to REUSE specifications.
2020-07-13 16:28:49 -07:00
Jeff Epler 6d97f6fccc audioio: Remove compatibility code
These items were aliased from audiocore to audioio for compatibility
with 4.x, but according to our deprecation schedule can be removed
in 6.0.
2020-07-08 20:31:35 -05:00
Dan Halbert a5ab2829eb use zephyr include files; wip: compiles 2020-07-08 09:33:41 -04:00
Diego Elio Pettenò 34b4993d63 Add license to some obvious files. 2020-07-06 19:16:25 +01:00
Dan Halbert 11cb3e3b4b hci skeleton done; not working yet 2020-06-30 23:19:40 -04:00
Damien George 332d83343f py: Rework mp_convert_member_lookup to properly handle built-ins.
This commit fixes lookups of class members to make it so that built-in
functions that are used as methods/functions of a class work correctly.

The mp_convert_member_lookup() function is pretty much completely changed
by this commit, but for the most part it's just reorganised and the
indenting changed.  The functional changes are:

- staticmethod and classmethod checks moved to later in the if-logic,
  because they are less common and so should be checked after the more
  common cases.

- The explicit mp_obj_is_type(member, &mp_type_type) check is removed
  because it's now subsumed by other, more general tests in this function.

- MP_TYPE_FLAG_BINDS_SELF and MP_TYPE_FLAG_BUILTIN_FUN type flags added to
  make the checks in this function much simpler (now they just test this
  bit in type->flags).

- An extra check is made for mp_obj_is_instance_type(type) to fix lookup of
  built-in functions.

Fixes #1326 and #6198.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-30 23:55:32 +10:00
Damien George d06ae1d2b1 py/obj.h: Make existing MP_TYPE_FLAG_xxx macros sequential.
There's no reason to have them non-sequential, this was likely a typo from
commit 9ec1caf42e.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-30 23:55:32 +10:00
Dan Halbert 1bc2e979eb wip; compiles; much commented out 2020-06-26 17:23:20 -04:00
Jeff Epler 05837b2841 sdioio: Add shared-bindings
There is no implementation yet.
2020-06-26 11:50:25 -05:00
Jeff Epler 57fde2e07b sdcardio: implement new library for SD card I/O
Testing performed: That a card is successfully mounted on Pygamer with
the built in SD card slot

This module is enabled for most FULL_BUILD boards, but is disabled for
samd21 ("M0"), litex, and pca10100 for various reasons.
2020-06-26 11:50:23 -05:00
Damien George 9f911d822e py/objcomplex: Add mp_obj_get_complex_maybe for use in complex bin-op.
This allows complex binary operations to fail gracefully with unsupported
operation rather than raising an exception, so that special methods work
correctly.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-27 01:03:10 +10:00
Damien George 41fa8b5482 py/emitnative: Implement binary operations for viper uint operands.
uint types in viper mode can now be used for all binary operators except
floor-divide and modulo.

Fixes issue #1847 and issue #6177.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-27 00:24:04 +10:00
Damien George b3b8706d27 py/asm: Add condition codes for signed comparisons.
Signed-off-by: Damien George <damien@micropython.org>
2020-06-27 00:24:04 +10:00
Damien George aa26fe62d8 py/asm: Add funcs/macros to emit machine code for logical-shift-right.
Signed-off-by: Damien George <damien@micropython.org>
2020-06-27 00:24:04 +10:00
Dan Halbert 759929c24a hci early wip; refactor supervisor bluetooth.c for nrf: tested 2020-06-25 20:57:17 -04:00
Jeff Epler 1d2cc0b968 I2CPeripheral: Rename class and its module
This is an incompatible change.
2020-06-25 11:44:19 -05:00
Kamil Tomaszewski 84f424f631 gnss: Implement new library for GNSS 2020-06-24 11:14:44 +02:00
Damien George 457fdf61c3 py/objtype: Support passing in an OrderedDict to type() as the locals.
An OrderedDict can now be used for the locals when creating a type
explicitly via type(name, bases, locals).

Signed-off-by: Damien George <damien@micropython.org>
2020-06-24 12:05:40 +10:00
Damien George 456a3abe8d py/obj.h: Add public mp_obj_is_dict_or_ordereddict() helper macro.
And use it in py/objdict.c instead of mp_obj_is_dict_type.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-24 12:05:00 +10:00
Jeff Epler a7b36f724f circuitpy_defns.mk: Sort several lists of source files
and regularize whitespace in one spot
2020-06-22 10:45:27 -05:00
Jeff Epler a580f0f1c4 _pew: move to common-hal
I noticed that this code was referring to samd-specific functionality,
and isn't enabled except in one samd board (pewpew10).  Move it.

There is incomplte support for _pew in mimxrt10xx which then caused build
errors; adding a #if guard to check for _pew being enabled fixes it.
The _pew module is not likely to be important on mimxrt but I'll leave the
choice to remove it to someone else.
2020-06-22 10:45:27 -05:00
Damien George 6164c7e666 py/misc.h: Add missing semi-colon in mp_float_union_t for big-endian.
Fixes issue #6161.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-22 13:42:24 +10:00
Damien George f2e267da68 py/compile: Implement PEP 526, syntax for variable annotations.
This addition to the grammar was introduced in Python 3.6.  It allows
annotating the type of a varilable, like:

    x: int = 123
    s: str

The implementation in this commit is quite simple and just ignores the
annotation (the int and str bits above).  The reason to implement this is
to allow Python 3.6+ code that uses this feature to compile under
MicroPython without change, and for users to use type checkers.

In the future viper could use this syntax as a way to give types to
variables, which is currently done in a bit of an ad-hoc way, eg
x = int(123).  And this syntax could potentially be used in the inline
assembler to define labels in an way that's easier to read.
2020-06-16 23:18:01 +10:00
Damien George 131b0de70a py/grammar.h: Consolidate duplicate sub-rules for :test and =test. 2020-06-16 23:18:01 +10:00
Damien George 1783950311 py/compile: Implement PEP 572, assignment expressions with := operator.
The syntax matches CPython and the semantics are equivalent except that,
unlike CPython, MicroPython allows using := to assign to comprehension
iteration variables, because disallowing this would take a lot of code to
check for it.

The new compile-time option MICROPY_PY_ASSIGN_EXPR selects this feature and
is enabled by default, following MICROPY_PY_ASYNC_AWAIT.
2020-06-16 22:02:24 +10:00
Damien George 0fd91e39b1 py/compile: Convert scope test to SCOPE_IS_COMP_LIKE macro.
This macro can be used elsewhere.
2020-06-16 21:42:37 +10:00
David Lechner ecd7826316 tools/codeformat.py: Remove sizeof fixup.
Formatting for `* sizeof` was fixed in uncrustify v0.71, so we no longer
need the fixups for it.  Also, there was one file where the updated
uncrustify caught a problem that the regex didn't pick up, which is updated
in this commit.

Signed-off-by: David Lechner <david@pybricks.com>
2020-06-14 20:24:18 +10:00
Damien George f3062b5cbd py/obj.h: Clarify comments about mp_map_t is_fixed and is_ordered.
Long ago, prior to 0ef01d0a75, fixed and
ordered maps were the same setting with the "table_is_fixed_array" member
of mp_map_t.  But these settings are actually independent, and it is
possible to have is_fixed=1, is_ordered=0 (although this can currently
only be done by tools/cc1).  So update the comments to reflect this.
2020-06-10 22:27:10 +10:00
Andrew Leech 95cbe6b65e py/objtype: Use mp_obj_dict_copy() for creating obj.__dict__ attribute.
The resulting dict is now marked as read-only (is_fixed=1) to enforce the
fact that changes to this dict will not be reflected in the class instance.

This commit reduces code size by about 20 bytes, and should be more
efficient because it creates a direct copy of the dict rather than
reinserting all elements.
2020-06-10 22:03:39 +10:00
Andrew Leech 28370c0450 py/objtype: Add __dict__ attribute for class objects.
The behavior mirrors the instance object dict attribute where a copy of the
local attributes are provided (unless the dict is read-only, then that dict
itself is returned, as an optimisation).  MicroPython does not support
modifying this dict because the changes will not be reflected in the class.

The feature is only enabled if MICROPY_CPYTHON_COMPAT is set, the same as
the instance version.
2020-06-10 21:58:13 +10:00
Damien George bd06c698f0 py/dynruntime.h: Make mp_obj_str_get_str raise if arg not a str/bytes. 2020-06-08 09:05:03 +02:00
Jim Mussared e6881f0829 extmod/modbluetooth: Make modbluetooth event not a bitfield.
There doesn't appear to be any use for only triggering on specific events,
so it's just easier to number them sequentially.  This makes them smaller
values so they take up only 1 byte in the ringbuf, only 1 byte for the
opcode in the bytecode, and makes room for more events.

Also add a couple of new event types that need to be implemented (to avoid
re-numbering later).

And rename _COMPLETE and _STATUS to _DONE for consistency.

In the future the "trigger" keyword argument can be reinstated by requiring
the user to compute the bitmask, eg:

    ble.irq(handler, 1 << _IRQ_SCAN_RESULT | 1 << _IRQ_SCAN_DONE)
2020-06-05 14:04:20 +10:00
Diego Elio Pettenò dd5d7c86d2 Fix up end of file and trailing whitespace.
This can be enforced by pre-commit, but correct it separately to make it easier to review.
2020-06-03 10:56:35 +01:00
Damien George 203b10703e py/modbuiltins: Fix getattr to work with class raising AttributeError.
Fixes issue #6089.
2020-06-02 15:42:20 +10:00
Scott Shawcroft 959f7297da
Merge remote-tracking branch 'adafruit/master' into wdt-nrf 2020-06-01 17:01:16 -07:00
Jeff Epler f211a090e2 py.mk: Assume we want all C source files in ulab 2020-06-01 08:26:23 -05:00
Jeff Epler 1cc281b6a4 py.mk: Assume we want all C files from ulab 2020-06-01 08:20:23 -05:00
Scott Shawcroft 2fd7a43f2f
Merge remote-tracking branch 'adafruit/master' into wdt-nrf 2020-05-29 10:16:24 -07:00
Dan Halbert 6ace4ee7e8
Merge pull request #2968 from jepler/more-efficient-translation
More efficient translation
2020-05-28 21:37:45 -04:00
Scott Shawcroft 796373b8be
A number of small ESP32S2 fixes:
* Fix flash writes that don't end on a sector boundary. Fixes #2944
* Fix enum incompatibility with IDF.
* Fix printf output so it goes out debug UART.
* Increase stack size to 8k.
* Fix sleep of less than a tick so it doesn't crash.
2020-05-28 15:43:55 -07:00
Jeff Epler d0f9b5901e translations: document the compressed format 2020-05-28 11:30:46 -05:00
Jeff Epler fe3e8d1589 string compression: save a few bits per string
Length was stored as a 16-bit number always.  Most translations have
a max length far less.  For example, US English translation lengths
always fit in just 8 bits.  probably all languages fit in 9 bits.

This also has the side effect of reducing the alignment of
compressed_string_t from 2 bytes to 1.

testing performed: ran in german and english on pyruler, printed messages
looked right.

Firmware size, en_US
Before: 3044 bytes free in flash
After: 3408 bytes free in flash

Firmware size, de_DE (with #2967 merged to restore translations)
Before: 1236 bytes free in flash
After: 1600 bytes free in flash
2020-05-28 08:36:08 -05:00
David Lechner 093fd80760 py/modsys: Use consistent naming pattern for module-level const objects.
This renames a few identifiers to follow the usual naming convention of
mp_<module>_<name>.  This makes them easier to find, e.g. when grep'ing.
2020-05-28 10:02:14 +10:00
stijn 97ccde0c43 py/ringbuf: Fix compilation with msvc.
Older versions do not have "inline" so fetch the definition from
mpconfigport.h.
2020-05-28 09:56:18 +10:00