Commit Graph

4990 Commits

Author SHA1 Message Date
microDev
a46aa48e23
Convert more modules to use MP_REGISTER_MODULE
Convert espidf, os, rp2pio, samd and time to use MP_REGISTER_MODULE.

Related to #5183.
2021-10-05 16:58:24 +05:30
stijn
ea880d5674 py/builtinimport: Forward all debug printing to MICROPY_DEBUG_PRINTER. 2021-09-24 13:17:19 +10:00
Jeff Epler
10fdc80b9c
Merge pull request #5378 from jepler/esp32s2-paralleldisplay-i2s
Espressif: Use i2s peripheral for parallel LCD displays
2021-09-23 20:25:43 -05:00
Jeff Epler
a0cfceec28 Stub out the non-sequential parallel bus constructor
.. most boards can't support non-sequential pins, so share the
implementation of the error routine so we get an identical message.
2021-09-21 15:00:43 -05:00
iabdalkader
2c5e9bbdfa extmod: Add platform module.
It contains the compiler version, and underlying system HAL/SDK version.
2021-09-19 23:35:10 +10:00
Max Holliday
f16a4754ba changed boot counter #ifdef to safer #if 2021-09-17 14:40:16 -06:00
Max Holliday
818020d4c9
Merge branch 'adafruit:main' into pycubed_v05c 2021-09-17 14:35:17 -06:00
Jeff Epler
2916167bf4 paralleldisplay: make it actually importable 2021-09-16 14:11:16 -05:00
Jim Mussared
b326edf68c all: Remove MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE.
This commit removes all parts of code associated with the existing
MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE optimisation option, including the
-mcache-lookup-bc option to mpy-cross.

This feature originally provided a significant performance boost for Unix,
but wasn't able to be enabled for MCU targets (due to frozen bytecode), and
added significant extra complexity to generating and distributing .mpy
files.

The equivalent performance gain is now provided by the combination of
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE (which has
been enabled on the unix port in the previous commit).

It's hard to provide precise performance numbers, but tests have been run
on a wide variety of architectures (x86-64, ARM Cortex, Aarch64, RISC-V,
xtensa) and they all generally agree on the qualitative improvements seen
by the combination of MICROPY_OPT_LOAD_ATTR_FAST_PATH and
MICROPY_OPT_MAP_LOOKUP_CACHE.

For example, on a "quiet" Linux x64 environment (i3-5010U @ 2.10GHz) the
change from CACHE_MAP_LOOKUP_IN_BYTECODE, to LOAD_ATTR_FAST_PATH combined
with MAP_LOOKUP_CACHE is:

diff of scores (higher is better)
N=2000 M=2000       bccache -> attrmapcache      diff      diff% (error%)
bm_chaos.py        13742.56 ->   13905.67 :   +163.11 =  +1.187% (+/-3.75%)
bm_fannkuch.py        60.13 ->      61.34 :     +1.21 =  +2.012% (+/-2.11%)
bm_fft.py         113083.20 ->  114793.68 :  +1710.48 =  +1.513% (+/-1.57%)
bm_float.py       256552.80 ->  243908.29 : -12644.51 =  -4.929% (+/-1.90%)
bm_hexiom.py         521.93 ->     625.41 :   +103.48 = +19.826% (+/-0.40%)
bm_nqueens.py     197544.25 ->  217713.12 : +20168.87 = +10.210% (+/-3.01%)
bm_pidigits.py      8072.98 ->    8198.75 :   +125.77 =  +1.558% (+/-3.22%)
misc_aes.py        17283.45 ->   16480.52 :   -802.93 =  -4.646% (+/-0.82%)
misc_mandel.py     99083.99 ->  128939.84 : +29855.85 = +30.132% (+/-5.88%)
misc_pystone.py    83860.10 ->   82592.56 :  -1267.54 =  -1.511% (+/-2.27%)
misc_raytrace.py   21490.40 ->   22227.23 :   +736.83 =  +3.429% (+/-1.88%)

This shows that the new optimisations are at least as good as the existing
inline-bytecode-caching, and are sometimes much better (because the new
ones apply caching to a wider variety of map lookups).

The new optimisations can also benefit code generated by the native
emitter, because they apply to the runtime rather than the generated code.
The improvement for the native emitter when LOAD_ATTR_FAST_PATH and
MAP_LOOKUP_CACHE are enabled is (same Linux environment as above):

diff of scores (higher is better)
N=2000 M=2000        native -> nat-attrmapcache  diff      diff% (error%)
bm_chaos.py        14130.62 ->   15464.68 :  +1334.06 =  +9.441% (+/-7.11%)
bm_fannkuch.py        74.96 ->      76.16 :     +1.20 =  +1.601% (+/-1.80%)
bm_fft.py         166682.99 ->  168221.86 :  +1538.87 =  +0.923% (+/-4.20%)
bm_float.py       233415.23 ->  265524.90 : +32109.67 = +13.756% (+/-2.57%)
bm_hexiom.py         628.59 ->     734.17 :   +105.58 = +16.796% (+/-1.39%)
bm_nqueens.py     225418.44 ->  232926.45 :  +7508.01 =  +3.331% (+/-3.10%)
bm_pidigits.py      6322.00 ->    6379.52 :    +57.52 =  +0.910% (+/-5.62%)
misc_aes.py        20670.10 ->   27223.18 :  +6553.08 = +31.703% (+/-1.56%)
misc_mandel.py    138221.11 ->  152014.01 : +13792.90 =  +9.979% (+/-2.46%)
misc_pystone.py    85032.14 ->  105681.44 : +20649.30 = +24.284% (+/-2.25%)
misc_raytrace.py   19800.01 ->   23350.73 :  +3550.72 = +17.933% (+/-2.79%)

In summary, compared to MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE, the new
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE options:
- are simpler;
- take less code size;
- are faster (generally);
- work with code generated by the native emitter;
- can be used on embedded targets with a small and constant RAM overhead;
- allow the same .mpy bytecode to run on all targets.

See #7680 for further discussion.  And see also #7653 for a discussion
about simplifying mpy-cross options.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-09-16 16:04:03 +10:00
Jim Mussared
11ef8f22fe py/map: Add an optional cache of (map+index) to speed up map lookups.
The existing inline bytecode caching optimisation, selected by
MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE, reserves an extra byte in the
bytecode after certain opcodes, which at runtime stores a map index of the
likely location of this field when looking up the qstr.  This scheme is
incompatible with bytecode-in-ROM, and doesn't work with native generated
code.  It also stores bytecode in .mpy files which is of a different format
to when the feature is disabled, making generation of .mpy files more
complex.

This commit provides an alternative optimisation via an approach that adds
a global cache for map offsets, then all mp_map_lookup operations use it.
It's less precise than bytecode caching, but allows the cache to be
independent and external to the bytecode that is executing.  It also works
for the native emitter and adds a similar performance boost on top of the
gain already provided by the native emitter.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-09-16 16:02:19 +10:00
Jim Mussared
7b89ad8dbf py/vm: Add a fast path for LOAD_ATTR on instance types.
When the LOAD_ATTR opcode is executed there are quite a few different cases
that have to be handled, but the common case is accessing a member on an
instance type.  Typically, built-in types provide methods which is why this
is common.

Fortunately, for this specific case, if the member is found in the member
map then there's no further processing.

This optimisation does a relatively cheap check (type is instance) and then
forwards directly to the member map lookup, falling back to the regular
path if necessary.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-09-16 16:02:15 +10:00
Jim Mussared
01374d941f py/mpconfig.h: Define initial templates for "feature levels".
This is the beginning of a set of changes to simplify enabling/disabling
features.  The goals are:
- Remove redundancy from mpconfigport.h (never set a value to the default
  -- make it clear exactly what's being enabled).
- Improve consistency between ports.  All "similar" ports (i.e. approx same
  flash size) should get the same features.
- Simplify mpconfigport.h -- just get default/sensible options for the size
  of the port.
- Make it easy for defining constrained boards (e.g. STM32F0/L0), they can
  just set a lower level.

This commit makes a step towards this and defines the "core" level as the
current default feature set, and a "minimal" level to turn off everything.
And a few placeholder levels are added for where the other ports will
roughly land.

This is a no-op change for all ports.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-09-16 13:19:11 +10:00
Max Holliday
5b69aa0a58
Merge branch 'adafruit:main' into pycubed_v05c 2021-09-15 14:23:56 -06:00
Scott Shawcroft
e544909b5b
Clear the buffer byte between ATB and FTB
This ensures it has a known start state. An unknown state risks it
looking like a tail block similar to the problem fixed by #5245.

Fixes #5305
2021-09-14 16:24:37 -07:00
Scott Shawcroft
973a90f2aa
Rename esp32s2 port to espressif
This is in preparation for ESP32-S3 support and potentially others.

Related to #4363
2021-09-13 16:44:55 -07:00
Damien George
426785a19e py/emitnative: Ensure load_subscr does not clobber existing REG_RET.
Fixes issue #7782, and part of issue #6314.

Signed-off-by: Damien George <damien@micropython.org>
2021-09-13 22:30:24 +10:00
Damien George
e6850838cd py/parse: Simplify parse nodes representing a list.
This commit simplifies and optimises the parse tree in-memory
representation of lists of expressions, for tuples and lists, and when
tuples are used on the left-hand-side of assignments and within del
statements.  This reduces memory usage of the parse tree when such code is
compiled, and also reduces the size of the compiler.

For example, (1,) was previously the following parse tree:

    expr_stmt(5) (n=2)
      atom_paren(45) (n=1)
        testlist_comp(146) (n=2)
          int(1)
          testlist_comp_3b(149) (n=1)
            NULL
      NULL

and with this commit is now:

    expr_stmt(5) (n=2)
      atom_paren(45) (n=1)
        testlist_comp(146) (n=1)
          int(1)
      NULL

Similarly, (1, 2, 3) was previously:

    expr_stmt(5) (n=2)
      atom_paren(45) (n=1)
        testlist_comp(146) (n=2)
          int(1)
          testlist_comp_3c(150) (n=2)
            int(2)
            int(3)
      NULL

and is now:

    expr_stmt(5) (n=2)
      atom_paren(45) (n=1)
        testlist_comp(146) (n=3)
          int(1)
          int(2)
          int(3)
      NULL

Signed-off-by: Damien George <damien@micropython.org>
2021-09-10 14:09:44 +10:00
Max Holliday
fa83c1c2f9
adding boot counter to main.c
The boot counter is a uint8_t single-byte counter stored in the first NVM byte position (`micrcontroller.nvm[0]`). The counter increments by 1 each time the board boots, regardless if it's a hard or soft reset. 

Enable the boot counter by adding `#define CIRCUITPY_BOOT_COUNTER 1` to your board's mpconfigboard.h file. Note that an error will be thrown during the build if `CIRCUITPY_INTERNAL_NVM_SIZE` is not also set within mpconfigboard.h.
2021-09-08 19:56:23 -06:00
microDev
86221b9895
revert localization of repl messages 2021-09-07 16:28:00 +05:30
Damien George
af64c2ddbd extmod/machine_pwm: Factor out machine.PWM bindings to common code.
This commit refactors machine.PWM and creates extmod/machine_pwm.c.  The
esp8266, esp32 and rp2 ports all use this and provide implementations of
the required PWM functionality.  This helps to reduce code duplication and
keep the same Python API across ports.

This commit does not make any functional changes.

Signed-off-by: Damien George <damien@micropython.org>
2021-09-04 16:31:17 +10:00
microDev
d526925d49
make aesio module full build dependent 2021-09-02 19:02:01 +05:30
Damien George
d41f6dde56 extmod/modonewire: Make _onewire module configurable via macro option.
Signed-off-by: Damien George <damien@micropython.org>
2021-09-02 13:11:23 +10:00
Damien George
afe0634c98 extmod/machine_spi: Make SoftSPI configurable via macro option.
Signed-off-by: Damien George <damien@micropython.org>
2021-09-02 13:11:23 +10:00
Damien George
122d901ef1 extmod/machine_i2c: Make SoftI2C configurable via macro option.
The zephyr port doesn't support SoftI2C so it's not enabled, and the legacy
I2C constructor check can be removed.

Signed-off-by: Damien George <damien@micropython.org>
2021-09-02 13:11:23 +10:00
Damien George
7c54b64280 all: Bump version to 1.17.
Signed-off-by: Damien George <damien@micropython.org>
2021-09-02 00:07:13 +10:00
Rob Capellini
9002f351b4 Convert more modules to use MP_REGISTER_MODULE
Convert _eve, _pew, aesio, alarm, audiopwmio, bitops, camera, canio, dualbank, gnss, i2cperipheral, imagecapture, ipaddress, memorymonitor, sdioio, socketpool, ssl, uheap, ustack, watchdog, and wifi modules to use MP_REGISTER_MODULE.

Related to #5183.
2021-09-01 00:19:05 -04:00
Scott Shawcroft
92a43192f8
Fix mpy-cross by providing default 2021-08-31 13:38:37 -07:00
Scott Shawcroft
771b4c7464
Add two space saving knobs
* Reduce the number of supported HID reports of IDs per descriptor.
  This saves ~200 bytes in the default HID objects.
* (Not enabled) Compute QSTR attrs on init. This trades 1k RAM for
  flash. Flash is the default (1).
2021-08-31 13:02:34 -07:00
Rob Capellini
1c8828223f Convert more modules to use MP_REGISTER_MODULE
Convert neopixel_write, onewireio, ps2io, pulseio, pwmio, rainbowio, random, rgbmatrix, rotaryio, rtc, sdcardio, sharpdisplay, _stage, storage, struct, supervisor, synthio, touchio, traceback, usb_cdc, usb_hid, usb_midi, and vectorio modules to use MP_REGISTER_MODULE.

Related to #5183.
2021-08-30 22:29:51 -04:00
Scott Shawcroft
5fb4fa6f12
Merge pull request #5260 from jepler/paralleldisplay
Split out paralleldisplay to its own module
2021-08-30 18:35:15 -07:00
Jeff Epler
db3945edfe Make %p include the 0x prefix
.. and modify some messages where 0x was specified "manually".

This involves updating some tests to expect the new 0x to appear.
2021-08-29 11:36:29 -05:00
Jeff Epler
266b6a1dd9 mkrules: Fix warning preprocessing C++ files
Messages like 'command-line option is [not valid] for C++' can result
from the way the preprocessor is invoked by `genlast`.  Instead, cause
the files to be preprocessed as though their content is "C".  This
should generally be OK, as they'll eventually be _compiled_ as C++.
When preprocessed as C, the file simply needs to generate all the same
QSTRS and TRANSLATEs.
2021-08-29 11:36:17 -05:00
Jeff Epler
7520feed1c Move ParallelBus to its own module 2021-08-29 07:33:54 -05:00
Scott Shawcroft
31b9dd408d
Merge pull request #5245 from jepler/pool-fix-circuitpython
gc.c: Ensure a gap of one byte before the finaliser table.
2021-08-27 09:33:05 -07:00
Jeff Epler
955c0276c2 Merge remote-tracking branch 'origin/main' into pool-fix-circuitpython 2021-08-27 09:26:29 -05:00
Jeff Epler
5a162eb168 gc.c: Ensure a gap of one byte before the finaliser table.
.. or, for !MICROPY_ENABLE_FINALISER, before the first block of the pool.

Closes: adafruit/circuitpython#5021
Closes: micropython/micropython#7116
Signed-off-by: Jeff Epler <jepler@gmail.com>
2021-08-27 09:23:08 -05:00
Jeff Epler
15133cbf59 gc.c: Avoid valgrind false positives.
When you wish to use the valgrind memory analysis tool on micropython,
you can arrange to define MICROPY_DEBUG_VALGRIND to enable use of
special valgrind macros. For now, this only fixes `gc_get_ptr`
so that it never emits the diagnostic "Conditional jump or move depends
on uninitialised value(s)".

Signed-off-by: Jeff Epler <jepler@gmail.com>
2021-08-27 09:23:08 -05:00
Rob Capellini
b8f2799609 Convert more modules to use MP_REGISTER_MODULE
Convert bitbangio, bitmaptools, _bleio, board, busio, countio, digitalio, framebufferio, frequencyio, gamepadshift, getpass, keypad, math, microcontroller, and msgpack modules to use MP_REGISTER_MODULE.

Related to #5183.
2021-08-26 21:35:02 -04:00
Dan Halbert
a608934f31 restrict WaveFile buffer size to 8-1024 2021-08-26 09:34:02 -04:00
microDev
5425454341
Merge pull request #5208 from jepler/small-memory-savings
Small memory savings
2021-08-26 09:22:14 +05:30
Rob Capellini
ad103ac6f1 Convert more modules to use MP_REGISTER_MODULE
Convert adafruit_bus_device, adafruit_pixelbuf, analogio, atexit, audiobusio, audiocore, audioio, audiomixer, and audiomp3 modules to use MP_REGISTER_MODULE.

Related to #5183.
2021-08-25 20:48:55 -04:00
Rob Capellini
deb8e2a50a Remove comment that no longer applies 2021-08-24 20:55:39 -04:00
Rob Capellini
17af5cd492 Use MP_REGISTER_MODULE with displayio, terminalio, and fontio
Convert from using MICROPY_PORT_BUILTIN_MODULES_STRONG_LINKS to using MP_REGISTER_MODULE for displayio, terminalio, and fontio modules.

Related to #5183.
2021-08-24 20:07:10 -04:00
Jeff Epler
6d26225b60 objgenerator: Unify two messages 2021-08-23 20:27:20 -05:00
Jeff Epler
c310a618e8 objtraceback: merge with an existing message in objtype 2021-08-23 20:23:23 -05:00
microDev
57b4d0176f
Merge pull request #5077 from microDev1/atexit
Add atexit module
2021-08-23 19:49:46 +05:30
Dan Halbert
57841dc92b
Merge pull request #5171 from tannewt/unicode_filenames
Turn on unicode for FATFS
2021-08-21 16:31:54 -04:00
Jim Mussared
b51e7e9d01 stm32: Disable computed goto on constrained boards.
Saves ~1kiB.  Add comment to this effect to mpconfig.h.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-20 20:18:52 +10:00
microDev
25437dcb33
Merge branch 'main' into atexit 2021-08-20 09:45:54 +05:30
microDev
dd5b711dc7
Merge pull request #5166 from jepler/issue5154
Don't double-list modules that MP_REGISTER_MODULE
2021-08-20 09:25:03 +05:30
Scott Shawcroft
5c6e80a2bb
Shrink builds by defining advanced micropython API
Also, ignore more pins on SAMD boards and disable EXFAT on others.
2021-08-19 16:49:33 -07:00
Jeff Epler
f1bbf406c4 remove a straggling QRIO_MODULE mention 2021-08-19 14:01:14 -05:00
Jim Mussared
85adc25558 py/mkrules.mk: Do submodule sync in "make submodules".
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-19 22:53:44 +10:00
Jim Mussared
e64cda5295 stm32: Add implementation of machine.bitstream.
Hand-written version for M0, and cycle-counter version for everything else.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-19 22:50:32 +10:00
Jim Mussared
870000f35b extmod: Add machine.bitstream.
This is a generic API for synchronously bit-banging data on a pin.

Initially this adds a single supported encoding, which supports controlling
WS2812 LEDs.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-19 22:50:11 +10:00
Jim Mussared
5555f147df py/lexer: Clear fstring_args vstr on lexer free.
This was missed in 692d36d779.  It's not
strictly necessary as the GC will clean it anyway, but it's good to
pre-emptively gc_free() all the blocks used in lexing/parsing.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-19 17:31:02 +10:00
Scott Shawcroft
e0ce5ff045
Fix some builds by disabling exfat 2021-08-18 08:22:29 -07:00
Scott Shawcroft
11f1c42bb5
Turn on unicode for FATFS
This also tweaks the repr for unicode strings to only escape a few
utf-8 code points. This makes emoji show in os.listdir() for
example.

Also, enable exfat support on full builds.

Fixes #5146
2021-08-17 17:41:59 -07:00
Dan Halbert
5b0009cbc4
Merge pull request #5151 from dhalbert/usb_hid-changes
Support multiple reports per device in usb_hid
2021-08-17 17:14:28 -04:00
microDev
6c763762d4
restore sys.atexit() to prevent merge conflict 2021-08-17 10:13:01 +05:30
Scott Shawcroft
58b3b48939
Merge pull request #5157 from DavePutz/issue_5155
Make MICROPY_CPYTHON_COMPAT settable
2021-08-16 12:15:35 -07:00
microDev
1c4a6c3667
atexit module refinements
- add test for atexit module
- add callback to gc collection
- fix callback memory allocation
- execute callback on both code and repl exit
2021-08-16 21:37:32 +05:30
Jeff Epler
ccc371ed53 Don't double-list modules that MP_REGISTER_MODULE
Closes: #5154
2021-08-16 08:50:57 -05:00
root
6c0e983658 Make MICROPY_CPYTHON_COMPAT settable 2021-08-15 20:49:24 -05:00
microDev
27563936ed
fix crash on raise of reload exception 2021-08-15 11:11:11 +05:30
Dan Halbert
4f8ff12afa wip 2021-08-14 12:36:40 -04:00
Jim Mussared
692d36d779 py: Implement partial PEP-498 (f-string) support.
This implements (most of) the PEP-498 spec for f-strings and is based on
https://github.com/micropython/micropython/pull/4998 by @klardotsh.

It is implemented in the lexer as a syntax translation to `str.format`:
  f"{a}" --> "{}".format(a)

It also supports:
  f"{a=}" --> "a={}".format(a)

This is done by extracting the arguments into a temporary vstr buffer,
then after the string has been tokenized, the lexer input queue is saved
and the contents of the temporary vstr buffer are injected into the lexer
instead.

There are four main limitations:
- raw f-strings (`fr` or `rf` prefixes) are not supported and will raise
  `SyntaxError: raw f-strings are not supported`.

- literal concatenation of f-strings with adjacent strings will fail
    "{}" f"{a}" --> "{}{}".format(a)    (str.format will incorrectly use
                                         the braces from the non-f-string)
    f"{a}" f"{a}" --> "{}".format(a) "{}".format(a) (cannot concatenate)

- PEP-498 requires the full parser to understand the interpolated
  argument, however because this entirely runs in the lexer it cannot
  resolve nested braces in expressions like
    f"{'}'}"

- The !r, !s, and !a conversions are not supported.

Includes tests and cpydiffs.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-14 16:58:40 +10:00
Scott Shawcroft
49dcdef512
Fix copy pasta 2021-08-13 11:32:26 -07:00
Scott Shawcroft
de796e2304
Move OneWire to onewireio from busio
This will allow finer grained inclusion in 8.0.0

Fixes #5135
2021-08-12 10:47:14 -07:00
Scott Shawcroft
fb6b438580
Make getpass and traceback full build only
This leaves much more space on SAMD21 builds that aren't "full builds".
These are new APIs that we don't need to add to old boards.

Also, tweak two Arduino boards to save space on them.
2021-08-11 14:47:35 -07:00
Scott Shawcroft
0c1d6cef75
Merge remote-tracking branch 'adafruit/main' into fix_cp_irremote 2021-08-11 13:43:59 -07:00
Scott Shawcroft
083960ce90
Fix SAMD51 builds and Prox Trinkey
Adds CIRCUITPY_BUSIO_UART to disable UART by raising ValueError
that no pins work.
2021-08-11 11:53:26 -07:00
microDev
93c9d59960
Merge branch 'main' into getpass 2021-08-10 01:00:10 +05:30
Jeff Epler
c1ffab7476 Reduce code duplication in traceback module 2021-08-09 08:27:28 -05:00
Jeff Epler
bfea6947e5 Improve mp_printf with support for compressed strings
* The new nonstandard '%S' format takes a pointer to compressed_string_t
   and prints it

 * The new mp_cprintf and mp_vcprintf take a format string that is a
   compressed_string_t
2021-08-09 08:27:28 -05:00
microDev
51f8603783
add getpass module 2021-08-09 17:50:28 +05:30
Jeff Epler
d59a28db97 Compress word offset table
By storing "count of words by length", the long `wends` table can be
replaced with a short `wlencount` table.  This saves flash storage space.

Extend the range of string lengths that can be in the dictionary.
Originally it was to 2 to 9; at one point it was changed to 3 to 9.
Putting the lower bound back at 2 has a positive impact on the French
translation (a bunch of them, such as "ch", "\r\n", "%q", are used).
Increasing the maximum length gets 'mpossible', ' doit être ',
and 'CircuitPyth' at the long end.  This adds a bit of processing time
to makeqstrdata. The specific 2/11 values are again empirical based on
the French translation on the adafruit_proxlight_trinkey_m0.
2021-08-07 09:23:35 -05:00
Damien George
78718fffb1 py/mkrules: Automatically build mpy-cross if it doesn't exist.
Commit 4173950658 removed automatic building
of mpy-cross, which rebuilt it whenever any of its dependent source files
changed.

But needing to build mpy-cross, and not knowing how, is a frequent issue.
This commit aims to help by automatically building mpy-cross only if it
doesn't exist.  For Makefiles it uses an order-only prerequisite, while
for CMake it uses a custom command.

If MICROPY_MPYCROSS (which is what makemanifest.py uses to locate the
mpy-cross executable) is defined in the environment then automatic build
will not be attempted, allowing a way to prevent this auto-build if needed.

Thanks to Trammell Hudson aka @osresearch for the original idea; see #5760.

Signed-off-by: Damien George <damien@micropython.org>
2021-08-07 20:25:32 +10:00
microDev
063e3946d6
Merge pull request #5094 from jepler/quirc
Add qrio: Decode QR codes with quirc lib
2021-08-07 09:30:20 +05:30
Peter Züger
ffc854f17f extmod/modujson: Add support for dump/dumps separators keyword-argument.
Optionally enabled via MICROPY_PY_UJSON_SEPARATORS.  Enabled by default.

For dump, make sure mp_get_stream_raise is called after
mod_ujson_separators since CPython does it in this order (if both
separators and stream are invalid, separators will raise an exception
first).

Add separators argument in the docs as well.

Signed-off-by: Peter Züger <zueger.peter@icloud.com>
Signed-off-by: Damien George <damien@micropython.org>
2021-08-07 13:52:16 +10:00
Jeff Epler
67551c1ac0 qrio: Split QRInfo & PixelPolicy to their own .c/.h files 2021-08-06 09:53:17 -05:00
Jeff Epler
bd903f2f03 Add qrio: Decode QR codes with quirc lib 2021-08-04 09:16:00 -05:00
microDev
193a8d2e67
add traceback object 2021-08-04 12:10:57 +05:30
David Lechner
afcc77cebc py/builtinimport: Fix condition for including do_execute_raw_code().
Commit e33bc597 ("py: Remove calls to file reader functions when these
are disabled.") changed the condition for one caller of
do_execute_raw_code() from

    MICROPY_PERSISTENT_CODE_LOAD

to

    MICROPY_HAS_FILE_READER && MICROPY_PERSISTENT_CODE_LOAD

The condition that enables compiling the function itself needs to be
changed to match.

Signed-off-by: David Lechner <david@pybricks.com>
2021-07-31 16:51:58 +10:00
microDev
4938851122
remove legacy sys.atexit() implementation 2021-07-30 10:00:00 +05:30
microDev
a3998d0626
add atexit module 2021-07-30 09:32:28 +05:30
microDev
f371c0a609
add traceback module 2021-07-28 18:06:23 +05:30
Dan Halbert
b0fb709d83 more gamepad removal 2021-07-26 23:34:25 -04:00
Dan Halbert
901a6c27c0 remove gamepad; deprecate gamepadshift 2021-07-26 22:15:09 -04:00
microDev
79aaa3cc3a
make pre-commit happy
fix formatting
2021-07-23 09:27:54 +05:30
Jim Mussared
4e39ff221a py/runtime: Fix bool unary op for subclasses of native types.
Previously a subclass of a type that didn't implement unary_op, or didn't
handle MP_UNARY_OP_BOOL, would raise TypeError on bool conversion.

Fixes #5677.
2021-07-23 12:40:00 +10:00
Scott Shawcroft
f84cb94819
Remove OSError(0) and old network modules
The newer modules are `socketpool` and `ssl`.

Fixes #3924, related to #2021, closes #1880, closes #2273, closes #2274
2021-07-21 17:33:40 -07:00
Jeff Epler
9a373c4fb9
Merge pull request #4961 from dhalbert/restore-1970-epoch
Restore default epoch to 1970
2021-07-19 14:51:08 -05:00
Jim Mussared
0e3752e82a py/emitnative: Ensure stack settling is safe mid-branch.
And add a test for the case where REG_RET could be in use.

Fixes #7523.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-07-19 23:18:59 +10:00
Damien George
d0227d5862 py/emitnative: Reuse need_reg_all func in need_stack_settled.
To reduce code size and code duplication.

Signed-off-by: Damien George <damien@micropython.org>
2021-07-19 23:18:13 +10:00
David Lechner
8758504f0f extmod/moduselect: Conditionally compile select().
This adds #if MICROPY_PY_USELECT_SELECT around the uselect.select()
function. According to the docs, this function is only for CPython
compatibility and should not normally be used. So we can disable it
and save a few bytes of flash space where possible.

Signed-off-by: David Lechner <david@pybricks.com>
2021-07-17 23:32:39 +10:00
Kattni Rembor
dc2907ac1b Rename _pixelbuf to adafruit_pixelbuf, alias. 2021-07-16 13:31:40 -04:00
Scott Shawcroft
0fadf028ef
Create first BLE-only board, Micro:Bit v2
This fixes build issues with USB off, tweaks the README to allow
for BLE-only boards and adds the Micro:Bit v2 definition.

Fixes #4546
2021-07-14 09:55:51 -07:00
Damien George
70b8e1d1f5 py/obj: Fix formatting of comment for mp_obj_is_integer.
Signed-off-by: Damien George <damien@micropython.org>
2021-07-15 00:12:41 +10:00
Damien George
022b8a7fea py/objexcept: Make mp_obj_new_exception_arg1 inline.
This function is rarely used so making it inline reduces code size.

Signed-off-by: Damien George <damien@micropython.org>
2021-07-15 00:12:41 +10:00
Damien George
74085f167e py/modsys: Optimise sys.exit for code size by using exception helpers.
Signed-off-by: Damien George <damien@micropython.org>
2021-07-15 00:12:41 +10:00
Damien George
38a204ed96 py: Introduce and use mp_raise_type_arg helper.
To reduce code size.

Signed-off-by: Damien George <damien@micropython.org>
2021-07-15 00:12:41 +10:00
Damien George
bb00125aaa py: Support single argument to optimised MP_OBJ_STOP_ITERATION.
The MP_OBJ_STOP_ITERATION optimisation is a shortcut for creating a
StopIteration() exception object, and means that heap memory does not need
to be allocated for the exception (in cases where it can be used).  This
commit allows this optimised object to take an optional argument (before,
it could only have no argument).

The commit also adds some new tests to cover corner cases with
StopIteration and generators that previously did not work.

Signed-off-by: Damien George <damien@micropython.org>
2021-07-15 00:12:41 +10:00
Damien George
e3825e28e6 py/objexcept: Make mp_obj_exception_get_value support subclassed excs.
Signed-off-by: Damien George <damien@micropython.org>
2021-07-15 00:12:41 +10:00
Damien George
b8255dd2e0 py/vm: Simplify handling of MP_OBJ_STOP_ITERATION in yield-from opcode.
Signed-off-by: Damien George <damien@micropython.org>
2021-07-15 00:12:41 +10:00
Scott Shawcroft
9fdecacec7
Merge pull request #4903 from jepler/split-type-objects
Split type objects
2021-07-12 16:37:55 -07:00
Kattni Rembor
4579bf9b12 Rename rainbow to rainbowio 2021-07-12 14:40:01 -04:00
Jeff Epler
93c6d16d3c Rename mp_type_attr -> mp_type_get_attr_slot 2021-07-12 07:30:29 -05:00
Jeff Epler
88434c53c7 Rename mp_type_parent -> mp_type_get_parent_slot 2021-07-12 07:30:16 -05:00
Jeff Epler
75e995f372 Rename mp_type_protocol -> mp_type_get_protocol_slot 2021-07-12 07:04:04 -05:00
Jeff Epler
1b2da0cda3 Rename mp_type_getbuffer -> mp_type_get_getbuffer_slot 2021-07-12 07:04:02 -05:00
Jeff Epler
cdd879a3cc Rename mp_type_iternext -> mp_type_get_iternext_slot 2021-07-12 07:03:59 -05:00
Jeff Epler
9f6132b2ba Rename mp_type_getiter -> mp_type_get_getiter_slot 2021-07-12 07:03:57 -05:00
Jeff Epler
0efdf4a549 Rename mp_type_subscr -> mp_type_get_subscr_slot 2021-07-12 07:03:55 -05:00
Jeff Epler
6b8190b408 Rename mp_type_binary_op -> mp_type_get_binary_op_slot 2021-07-12 07:03:52 -05:00
Jeff Epler
46b5ed33ed Rename mp_type_unary_op -> mp_type_get_unary_op_slot 2021-07-12 07:03:50 -05:00
Jeff Epler
ec53a674cf Rename mp_type_call -> mp_type_get_call_slot 2021-07-12 07:03:25 -05:00
Jeff Epler
9c1434014c remove redundant docs of struct _mp_obj_full_type_t 2021-07-12 07:01:14 -05:00
Jeff Epler
52540a9830 Rename EXTENDED_FIELDS -> MP_TYPE_EXTENDED_FIELDS 2021-07-12 06:57:59 -05:00
Damien George
136369d72f all: Update to point to files in new shared/ directory.
Signed-off-by: Damien George <damien@micropython.org>
2021-07-12 17:08:10 +10:00
Jeff Epler
0b8b16f6ac increase comment on accuracy of the net savings estimate function
Thanks to tyomitch for suggesting the comment could be more accurate.
2021-07-11 08:57:27 -05:00
Jeff Epler
7302bc09a7 rename the type flag to EXTENDED for consistency 2021-07-09 14:59:37 -05:00
Jeff Epler
52e75c645d makeqstrdata: Don't include strings that are a net loss! 2021-07-09 14:26:43 -05:00
Jeff Epler
8836198ff1 TextSplitter: don't mutate 'words'
I was puzzled by why the dictionary words were sorted by length.
It was because TextSplitter sorted its parameter, instead of a copy.

This doesn't affect encoding size, but does affect the encoding NUMBER
of the found words.  We'll deliberately restore sorting by length next,
for other reasons, but not by spooky action.
2021-07-09 14:02:31 -05:00
Jeff Epler
99abd03b7a makeqstrdata: use an extremely accurate dictionary heuristic
Try to accurately measure the costs of including a word in the dictionary
vs the gains from using it in messages.

This saves about 160 bytes on trinket_m0 ja, the fullest translation
for that board.  Other translations on the same board all have savings,
ranging from 24 to 228 bytes.

```
Translation     Before  After   Savings
ja              1164    1324    160
de_DE           1260    1396    136
fr              1424    1652    228
zh_Latn_pinyin  1448    1520    72
pt_BR           1584    1736    152
pl              1592    1640    48
es              1724    1816    92
ko              1724    1816    92
fil             1764    1800    36
it_IT           1896    2040    144
nl              1956    2136    180
ID              2072    2180    108
cs              2124    2148    24
sv              2340    2448    108
en_x_pirate     2644    2740    96
en_GB           2652    2752    100
el              2656    2768    112
en_US           2656    2768    112
hi              2656    2768    112
```
2021-07-09 12:45:49 -05:00
Kattni Rembor
289575a811 Adding rainbow module. 2021-07-08 15:55:13 -04:00
Bryan Tong Minh
3d9af87721 windows/Makefile: Add .exe extension to executables name.
Uses the same logic applied in 5b57ae985f
to determine when to add .exe.

See related: #3310, #3361, #3370, #4143, #5727.
2021-07-08 12:35:08 +10:00
Jeff Epler
26ae1c7705 Move & comment the field access macros 2021-07-07 08:32:59 -05:00
Jeff Epler
44a3da55e6 Fix EXTENDED_FIELDS macro to work with clang
This appears to work with clang versions at least since 3.0.
2021-07-07 08:32:33 -05:00
Jeff Epler
2997113e73 obj.h: Make mp_obj_is_type work between full & non-full types
By comparing the address of the initial 'name' field instead of the
addresses of the objects themselves, a small amount of type safety is
added back, vs just casting to void.

In the event that some other kind of object is passed in as 't',
which happens to have a 'name' field of the right type, the construct
would be (undesirably) accepted but it would almost certainly evaluate
to false at runtime.
2021-07-07 08:31:54 -05:00
Jeff Epler
0d756066e8 milestone: coverage make test_full passes 2021-07-06 10:37:32 -05:00
Jeff Epler
df56ba207f WIP 2021-07-06 09:25:56 -05:00
Jeff Epler
cd119ca3ec Introduce, use getters for fields which will be optional in types 2021-07-06 09:15:01 -05:00
David Lechner
d934f8c8a8 py/makeversionhdr: Add --tags arg to git describe.
This adds the --tags argument to the git describe command that is used
to define the MICROPY_GIT_TAG macro. This makes it match non-annotated
tags. This is useful for MicroPython derivatives that don't use
annotated tags.

Signed-off-by: David Lechner <david@pybricks.com>
2021-07-05 10:41:31 -05:00
Dan Halbert
0d9c4cb746 Merge remote-tracking branch 'adafruit/main' into restore-1970-epoch 2021-07-02 09:27:16 -04:00
Dan Halbert
af13f9579c Restore default epoch to 1970 2021-07-02 09:04:42 -04:00
Dan Halbert
c7ffc2c10b
Merge pull request #4957 from tannewt/ble_serial
Add serial over BLE
2021-07-01 23:05:18 -04:00
David Lechner
58e4d72338 py/objexcept: Pretty print OSError also when it has 2 arguments.
This extends pretty-printing of OSError's to handle two arguments when the
exception name is known.

Signed-off-by: David Lechner <david@pybricks.com>
2021-07-01 13:23:54 +10:00
Dan Halbert
7f39779f45
Merge pull request #4931 from jepler/struct-time-construct
Allow construction of struct_time from another struct_time
2021-06-30 22:59:46 -04:00
microDev
e0c626af16
fix bytes() and bytearray() input validation 2021-06-29 19:59:44 +05:30
Yonatan Goldschmidt
4ada56d4cb tools/makemanifest.py: Allow passing flags to mpy-tool.py. 2021-06-28 01:50:00 +03:00
Jeff Epler
33482e0831 mp_obj_get_array: Work with namedtuple, attrtuple.
Signed-off-by: Jeff Epler <jepler@gmail.com>
2021-06-26 20:25:24 -05:00
Jeff Epler
f380a91e7c objtuple: Move mp_obj_is_tuple_compatible to obj.h.
Signed-off-by: Jeff Epler <jepler@gmail.com>
2021-06-26 20:25:23 -05:00
Scott Shawcroft
3177973843
Add BLE serial service 2021-06-25 15:00:10 -07:00
Damien George
cfd08448a1 py: Mark unused arguments from bytecode decoding macros.
Signed-off-by: Damien George <damien@micropython.org>
2021-06-25 10:58:22 +10:00
Damien George
08e0e065f4 py/makeqstrdefs.py: Don't include .h files explicitly in preprocessing.
Only include .c and .cpp files explicitly in the list of files passed to
the preprocessor for QSTR extraction.  All relevant .h files will be
included in this process by "#include" from the .c(pp) files.  In
particular for moduledefs.h, this is included by py/objmodule.c (and
doesn't actually contain any extractable MP_QSTR_xxx, but rather defines
macros with MP_QSTR_xxx's in them which are then part of py/objmodule.c).

The main reason for this change is to simplify the preprocessing step on
the javascript port, which tries to compile .h files as C++ precompiled
headers if they are passed with -E to clang.

Signed-off-by: Damien George <damien@micropython.org>
2021-06-25 10:50:54 +10:00
Scott Shawcroft
b81573d439
Merge pull request #4891 from dhalbert/keypad-scanning-events
keypad: support for vector and matrix key scanning
2021-06-24 10:25:21 -07:00
David Lechner
b51ae20c07 py/mperrno: Add MP_ECANCELED error code.
This is useful when binding asynchronous functions in C.

Signed-off-by: David Lechner <david@pybricks.com>
2021-06-24 23:14:01 +10:00
Jeff Epler
413f34cd8f all: Fix signed shifts and NULL access errors from -fsanitize=undefined.
Fixes the following (the line numbers match commit 0e87459e2b):

../../extmod/crypto-algorithms/sha256.c:49:19: runtime error: left shif...
../../extmod/moduasyncio.c:106:35: runtime error: member access within ...
../../py/binary.c:210:13: runtime error: left shift of negative value -...
../../py/mpz.c:744:16: runtime error: negation of -9223372036854775808 ...
../../py/objint.c:109:22: runtime error: left shift of 1 by 31 places c...
../../py/objint_mpz.c:374:9: runtime error: left shift of 4611686018427...
../../py/objint_mpz.c:374:9: runtime error: left shift of negative valu...
../../py/parsenum.c:106:14: runtime error: left shift of 46116860184273...
../../py/runtime.c:395:33: runtime error: left shift of negative value ...
../../py/showbc.c:177:28: runtime error: left shift of negative value -...
../../py/vm.c:321:36: runtime error: left shift of negative value -1```

Testing was done on an amd64 Debian Buster system using gcc-8.3 and these
settings:

    CFLAGS += -g3 -Og -fsanitize=undefined
    LDFLAGS += -fsanitize=undefined

The introduced TASK_PAIRHEAP macro's conditional (x ? &x->i : NULL)
assembles (under amd64 gcc 8.3 -Os) to the same as &x->i, since i is the
initial field of the struct.  However, for the purposes of undefined
behavior analysis the conditional is needed.

Signed-off-by: Jeff Epler <jepler@gmail.com>
2021-06-24 23:01:04 +10:00
Dan Halbert
8c74b4a5f2 fix some typos 2021-06-21 17:47:12 -04:00
Dan Halbert
51c547a5b9 add generic arg validation routines; add interval args to keypad 2021-06-21 12:13:39 -04:00
David Lechner
259d9b69fe py/mpstate: Schedule KeyboardInterrupt on main thread.
This introduces a new macro to get the main thread and uses it to ensure
that asynchronous exceptions such as KeyboardInterrupt (CTRL+C) are only
scheduled on the main thread. This is more deterministic than being
scheduled on a random thread and is more in line with CPython that only
allow signal handlers to run on the main thread.

Fixes issue #7026.

Signed-off-by: David Lechner <david@pybricks.com>
2021-06-19 09:49:00 +10:00
David Lechner
ca920f7218 py/mpstate: Make exceptions thread-local.
This moves mp_pending_exception from mp_state_vm_t to mp_state_thread_t.
This allows exceptions to be scheduled on a specific thread.

Signed-off-by: David Lechner <david@pybricks.com>
2021-06-19 09:43:44 +10:00
Jeff Epler
a5876f4dea fix compile error on boards with inline asm 2021-06-18 16:59:48 -05:00
Jeff Epler
fa29be5aea restore a redundant NULL-comparison, it silences compiler diagnostics 2021-06-18 11:24:39 -05:00
Jeff Epler
87d3740c64 Merge tag 'v1.16' 2021-06-18 10:54:19 -05:00
Damien George
7c51cb2307 all: Bump version to 1.16.
Signed-off-by: Damien George <damien@micropython.org>
2021-06-18 16:38:06 +10:00
Damien George
bc89cdeb45 py/gc: Only use no_sanitize_address attribute for GCC 4.8 and above.
It's not supported on older GCC versions.

Signed-off-by: Damien George <damien@micropython.org>
2021-06-18 14:15:37 +10:00
Dan Halbert
af66931f71 Added keypad.ShiftRegisterKeys 2021-06-17 20:51:45 -04:00
Dan Halbert
04b69cde9b turn off gamepad on most builds;turn off keypad where appropriate 2021-06-17 13:46:45 -04:00
Dan Halbert
cd31136c30 factor out keypad.EventQueue 2021-06-16 15:55:54 -04:00
Dan Halbert
3d18c5c327 Use a single list of keypad scanners 2021-06-16 13:29:38 -04:00
Dan Halbert
32eec85230 compiles 2021-06-14 20:54:43 -04:00
Dan Halbert
627c426259 wip 2021-06-14 16:00:15 -04:00
Dan Halbert
7d23206018 Inital keypad work: Keys working: one pin per key
keypad.Buttons and keypad.State

Buttons -> Keys; further work

wip

wip

wip: compiles

about to try

keypad.Keys working
2021-06-08 20:27:31 -04:00
Damien George
5e1d3c8b5d py/stackctrl: Prevent unused-var warning when stack checking disabled.
Signed-off-by: Damien George <damien@micropython.org>
2021-06-05 11:03:09 +10:00
Damien George
a70a4e6688 py/emitglue: Always flush caches when assigning native ARM code.
Prior to this commit, cache flushing for ARM native code was done only in
the assembler code asm_thumb_end_pass()/asm_arm_end_pass(), at the last
pass of the assembler.  But this misses flushing the cache when loading
native code from an .mpy file, ie in persistentcode.c.

The change here makes sure the cache is always flushed/cleaned/invalidated
when assigning native code on ARM architectures.

This problem was found running tests/micropython/import_mpy_native_gc.py on
the mimxrt port.

Signed-off-by: Damien George <damien@micropython.org>
2021-06-05 11:03:04 +10:00
George Waters
c5066efacb
Enable slice indices 2021-06-03 22:55:32 -04:00
Tsutomu IKEGAMI
9490010d7d Fix format (expand tab) 2021-06-02 12:10:41 +09:00
Tsutomu IKEGAMI
6278c2bb9f Fix bit_length() method to work with zero-valued mpz integer. 2021-06-02 11:57:55 +09:00
Jonathan Giles
064e6881ed Fix mpy-cross build on Apple M1 machines 2021-05-31 14:14:20 -04:00
Damien George
53519e322a py/builtinimport: Change relative import's ValueError to ImportError.
Following CPython change, see https://bugs.python.org/issue37444.

Signed-off-by: Damien George <damien@micropython.org>
2021-05-30 19:35:03 +10:00
Jeff Epler
d67f4115b4 py/repl: Don't read past the end of import_str.
asan considers that memcmp(p, q, N) is permitted to access N bytes at each
of p and q, even for values of p and q that have a difference earlier.
Accessing additional values is frequently done in practice, reading 4 or
more bytes from each input at a time for efficiency, so when completing
"non_exist<TAB>" in the repl, this causes a diagnostic:

    ==16938==ERROR: AddressSanitizer: global-buffer-overflow on
    address 0x555555cd8dc8 at pc 0x7ffff726457b bp 0x7fffffffda20 sp 0x7fff
    READ of size 9 at 0x555555cd8dc8 thread T0
        #0 0x7ffff726457a  (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xb857a)
        #1 0x555555b0e82a in mp_repl_autocomplete ../../py/repl.c:301
        #2 0x555555c89585 in readline_process_char ../../lib/mp-readline/re
        #3 0x555555c8ac6e in readline ../../lib/mp-readline/readline.c:513
        #4 0x555555b8dcbd in do_repl /home/jepler/src/micropython/ports/uni
        #5 0x555555b90859 in main_ /home/jepler/src/micropython/ports/unix/
        #6 0x555555b90a3a in main /home/jepler/src/micropython/ports/unix/m
        #7 0x7ffff619a09a in __libc_start_main ../csu/libc-start.c:308
        #8 0x55555595fd69 in _start (/home/jepler/src/micropython/ports/uni

    0x555555cd8dc8 is located 0 bytes to the right of global variable
    'import_str' defined in '../../py/repl.c:285:23' (0x555555cd8dc0) of
    size 8
      'import_str' is ascii string 'import '

Signed-off-by: Jeff Epler <jepler@gmail.com>
2021-05-30 11:50:51 +10:00
Jeff Epler
9a74546f8d py/gc: Access the list of root pointers in an asan-compatible way.
Signed-off-by: Jeff Epler <jepler@gmail.com>
2021-05-30 11:50:51 +10:00
Jeff Epler
f2dbc91022 py/compile: Raise an error on async with/for outside an async function.
A simple reproducer is:

   async for x in (): x

Before this change, it would cause an assertion error in mpy-cross and
micropython-coverage.
2021-05-30 10:38:48 +10:00
Damien George
4ee8ec6931 py/asmarm: Use builtin func to flush I- and D-cache on ARM 7 archs.
The inline assembler code does not work for __ARM_ARCH == 7.

Signed-off-by: Damien George <damien@micropython.org>
2021-05-26 16:24:00 +10:00
James Carr
61a7212b28
Update comment in persistentcode.c
Update comment in mp_raw_code_save so that it matches the code.
2021-05-23 08:57:41 +01:00
Damien George
e61ac453dc py/mkrules.cmake: Add MPY_LIB_DIR and BOARD_DIR to makemanifest call.
So that the FROZEN_MANIFEST option in cmake works the same as make.

Signed-off-by: Damien George <damien@micropython.org>
2021-05-23 00:00:39 +10:00
Scott Shawcroft
5643355e90
Merge remote-tracking branch 'adafruit/main' into simplify_status_led 2021-05-20 08:35:15 -07:00
Damien George
5176a2d732 py/emitnative: Fix x86-64 emitter to generate correct 8/16-bit stores.
Fixes issue #6643.

Signed-off-by: Damien George <damien@micropython.org>
2021-05-20 23:43:25 +10:00
Damien George
f49d47c167 py/asmx64: Support use of top 8 regs in src_r64 argument.
Signed-off-by: Damien George <damien@micropython.org>
2021-05-20 23:43:25 +10:00
Scott Shawcroft
499a4388cf
Handle inverted neopixel power 2021-05-19 17:29:02 -07:00
Bob Abeles
7ceccad4e2 py/nlrx64: Correct the detection of Darwin ABI.
__APPLE__ tests for an Apple OS and __MACH__ tests that it is based on CMU
Mach.  Using both tests ensures that just Darwin is recognized.
2021-05-18 11:52:00 +10:00
Bob Abeles
126b1c7271 py/nlraarch64: Add underscore prefix to function symbols for Darwin ABI.
The proper way to do this is to test for __APPLE__ and __MACH__, where
__APPLE__ tests for an Apple OS and __MACH__ tests that it is based on CMU
Mach.  Using both tests ensures that just Darwin (Apple's open source base
for MacOS, iOS, etc.) is recognized. __APPLE__ by itself will test for any
Apple OS, which can include older OS 7-9 and any future Apple OS. __MACH__
tests for any OS based on CMU Mach, including Darwin and GNU Hurd.

Fixes #7232.
2021-05-18 11:46:30 +10:00
Damien George
6d2680fa36 py/objarray: Fix constructing a memoryview from a memoryview.
Fixes issue #7261.

Signed-off-by: Damien George <damien@micropython.org>
2021-05-18 10:18:56 +10:00
Damien George
1446107b4d py/objarray: Use mp_obj_memoryview_init helper in mp_obj_new_memoryview.
Signed-off-by: Damien George <damien@micropython.org>
2021-05-18 00:22:53 +10:00
Limor "Ladyada" Fried
9752162508
Merge pull request #4765 from dhalbert/rotary-trinkey-rev-c-pins
Update various Trinkey pins and add _pixelbuf
2021-05-15 11:52:33 -04:00
Dan Halbert
1e2a945782 Make busio.SPI be NotImplementedError on proxlight to save space 2021-05-14 21:52:54 -04:00
Scott Shawcroft
689ec86225
Merge pull request #4754 from dhalbert/fix-endpoint-checking
Fix endpoint checking; Clean up safe mode printing
2021-05-14 11:25:01 -07:00
Dan Halbert
fa6c06fb38 count in/out endpoints; allow more usb modules on low-endpoint boards 2021-05-13 21:59:02 -04:00
stijn
09be0c083c py/objarray: Implement more/less comparisons for array. 2021-05-13 22:16:14 +10:00
stijn
57365d8557 py/objarray: Prohibit comparison of mismatching types.
Array equality is defined as each element being equal but to keep
code size down MicroPython implements a binary comparison.  This
can only be used correctly for elements with the same binary layout
though so turn it into an NotImplementedError when comparing types
for which the binary comparison yielded incorrect results: types
with different sizes, and floating point numbers because nan != nan.
2021-05-13 22:16:14 +10:00
Jeff Epler
47947dc23c
Merge pull request #4749 from tannewt/merge_1.15
Merge in MicroPython 1.15
2021-05-12 20:52:11 -05:00
Scott Shawcroft
966d25c6a5
Merge MicroPython v1.15 into CircuitPython 2021-05-12 17:51:42 -07:00
Dan Halbert
0651ef17dd Fix endpoint checking; Clean up safe mode printing. 2021-05-12 15:53:52 -04:00
Damien George
300fc842ce py/mkenv.mk: Don't emit info about BUILD_VERBOSE if it's set.
If the user sets V or BUILD_VERBOSE then they don't need to see this
message.

Signed-off-by: Damien George <damien@micropython.org>
2021-05-12 13:22:13 +10:00
Jeff Epler
45dc0953a5 makeqstrdata.py: Remove a problematic print
.. it contained non-ASCII characters, even when building the standard
English translation.

This may help resolve the build problems reported at #4750.
2021-05-11 21:48:21 -05:00
Scott Shawcroft
e02a26453c
Merge MicroPython 1.14 into CircuitPython 2021-05-11 15:07:40 -07:00
Scott Shawcroft
80f9f51297
Merge pull request #4732 from jepler/fix-natmod-1.13
Fix all "natmod" (native module) examples
2021-05-10 12:37:44 -07:00
Jeff Epler
ca74f282fd modmath: Remove stray "pragma GCC diagnostic pop"
This caused an error when building on macos:

```
../../py/modmath.c:247:36: error: pragma diagnostic pop could not pop, no matching push [-Werror,-Wunknown-pragmas]
            #pragma GCC diagnostic pop
                                   ^
1 error generated.
```

The pragma was half-removed during the merge of micropython.
2021-05-10 11:05:10 -05:00
Damien George
b6b39bff47 py/gc: Make gc_lock_depth have a count per thread.
This commit makes gc_lock_depth have one counter per thread, instead of one
global counter.  This makes threads properly independent with respect to
the GC, in particular threads can now independently lock the GC for
themselves without locking it for other threads.  It also means a given
thread can run a hard IRQ without temporarily locking the GC for all other
threads and potentially making them have MemoryError exceptions at random
locations (this really only occurs on MCUs with multiple cores and no GIL,
eg on the rp2 port).

The commit also removes protection of the GC lock/unlock functions, which
is no longer needed when the counter is per thread (and this also fixes the
cas where a hard IRQ calling gc_lock() may stall waiting for the mutex).

It also puts the check for `gc_lock_depth > 0` outside the GC mutex in
gc_alloc, gc_realloc and gc_free, to potentially prevent a hard IRQ from
waiting on a mutex if it does attempt to allocate heap memory (and putting
the check outside the GC mutex is now safe now that there is a
gc_lock_depth per thread).

Signed-off-by: Damien George <damien@micropython.org>
2021-05-10 13:07:16 +10:00
Jeff Epler
aca9d5bc40 Fix up all natmod examples
* modframebuf: _mp_framebuf_p_t is not "really" a protocol, but the
   QSTR assignment caused problems when building as a dynamic module
 * modure: str_index_to_ptr is not in the natmod API, disable URE match
   spans when dynamic.  mp_obj_len() is a bugfix, we should throw here
   if the object is not string-like
 * moduzlib: Correct paths to uzlib headers & sources.  this relative
   path (from moduzlib.c to the referenced file) works in all cases,
   the other only worked from ports/PORTNAME.
 * dynruntime: Handle 2-arg m_malloc, assert_native_inited, add a
   micropythonish mp_arg_check_num_mp, fix mp_raise_msg to use dumb
   strings, add mp_raise_arg1
 * nativeglue: ad assert_native_inited
 * translate: MP_ERROR_TEXT evaluates to its argument for DYNRUNTIME
 * mpy-tool: A straggling magic number change
 * mpy_ld: Have to renumber manually after dynruntime change
 * import_mpy_native_gc.py: Update copy of features0 baked into this test
2021-05-08 20:17:16 -05:00
Scott Shawcroft
cebb10ddae
Fixes from review 2021-05-07 09:31:59 -07:00
Scott Shawcroft
42f4065c8a
Merge remote-tracking branch 'adafruit/main' into merge_1.13 2021-05-06 11:17:53 -07:00
Dan Halbert
ebf9dcb47a
Merge pull request #4689 from dhalbert/dynamic-usb-descriptors
Dynamic USB descriptors
2021-05-06 13:51:38 -04:00
Scott Shawcroft
71d2536725
Build fixes thanks to jepler 2021-05-06 09:04:56 -07:00
Scott Shawcroft
3fda0c0a1b
Fix board builds and use MP_ERROR_TEXT in py and extmod 2021-05-05 17:51:52 -07:00
Jeff Epler
ef3ec93c8b Change the first byte of CircuitPython 'mpy' files to "C"
.. and also distinguish CircuitPython better in `mpy-cross --version`
2021-05-05 18:06:06 -05:00
Dan Halbert
579cdf30f1 fix more build errors 2021-05-05 16:49:29 -04:00
Jeff Epler
716a82ba67 set the coroutine generator for native generators 2021-05-05 11:36:40 -05:00
Dan Halbert
fc8e1c4c2e address review comments 2021-05-05 12:35:12 -04:00
Jeff Epler
68d4682298 compile: Reserve labels for native await 2021-05-05 11:00:39 -05:00
Jeff Epler
8fae7d2e30 runtime.c: Fix reading properties
.. this fixes the vfs_fat_ramdisk failure.
2021-05-05 11:00:39 -05:00
Jeff Epler
c3ce2bacde runtime: Remove unneeded includes
objtype.h got included twice, and vfs.h is unneeded.
2021-05-05 11:00:39 -05:00
Jeff Epler
14e0bdbae3 Move mp_get_stream non-inline
This caused a build error with external native code, as
MP_QSTR_protocol_stream was not available.
2021-05-05 11:00:35 -05:00
Jeff Epler
2c0664ea3f slice: restore ability to construct a slice 2021-05-05 08:47:15 -05:00
Artyom Skrobov
269d5bc80a qstr: One more cast could be avoided. 2021-05-05 09:01:47 +03:00
Scott Shawcroft
f0bb26d70f
Merge MicroPython 1.13 into CircuitPython 2021-05-04 18:06:33 -07:00
Dan Halbert
8bb3c6fd79 handle := defs in shared_bindings_matrix.py; update adafruit_hid 2021-05-04 18:07:01 -04:00
Artyom Skrobov
2d73ba2186 qstr: Use const consistently to avoid a cast. 2021-05-04 12:53:35 -04:00
Dan Halbert
adc3d7d55e update Python API according to review comments 2021-05-03 22:29:02 -04:00
Dan Halbert
d2b558993e merge from main, including MicroPython 1.12 merge 2021-05-03 20:56:04 -04:00
Scott Shawcroft
b35fa44c8a
Merge MicroPython 1.12 into CircuitPython 2021-05-03 14:01:18 -07:00
Dan Halbert
94ed5e0048
Merge pull request #4696 from dhalbert/no-pulseio-on-small-boards
Include pulseio only on CIRCUITPY_FULL_BUILD
2021-05-03 09:35:29 -04:00
Artyom Skrobov
ca35c0059c py/repl: Autocomplete builtin modules.
Doing "import <tab>" will now complete/list built-in modules.

Originally at adafruit#4548 and adafruit#4608

Signed-off-by: Artyom Skrobov <tyomitch@gmail.com>
2021-05-02 23:11:14 +10:00
Artyom Skrobov
7556e01f14 py/repl: Refactor autocomplete, extracting reusable parts.
Originally at adafruit#4548

Signed-off-by: Artyom Skrobov <tyomitch@gmail.com>
2021-05-02 23:11:12 +10:00
Artyom Skrobov
f85ea8d4fe py/repl: Refactor autocomplete to reduce nesting.
Originally at adafruit#4548

Signed-off-by: Artyom Skrobov <tyomitch@gmail.com>
2021-05-02 23:11:10 +10:00
scottbelden
befbff31b7 py/repl: Enter four spaces when there are no matches.
Originally at adafruit#1859

Signed-off-by: scottbelden <scottabelden@gmail.com>
2021-05-02 23:11:07 +10:00
Kathryn Lingel
1f1a54d0b1 py/repl: Filter private methods from tab completion.
Anything beginning with "_" will now only be tab-completed if there is
already a partial match for such an entry.  In other words, entering
foo.<tab> will no longer complete/list anything beginning with "_".

Originally at adafruit#1850

Signed-off-by: Kathryn Lingel <kathryn@lingel.net>
2021-05-02 23:11:03 +10:00
Damien George
aa061ae391 py/scheduler: Add missing MICROPY_WRAP_MP_SCHED_EXCEPTION usage.
This was missed in commit 7cbf826a95.

Signed-off-by: Damien George <damien@micropython.org>
2021-05-01 23:10:26 +10:00
Dan Halbert
96101016ec Include pulseio only on CIRCUITPY_FULL_BUILD 2021-04-30 23:13:32 -04:00
Jeff Epler
da19e8e2d7 Merge remote-tracking branch 'origin/main' into fix-codeformat-subdirs 2021-04-30 15:31:08 -05:00
Jeff Epler
dfa7c3d32d codeformat: Fix handling of **
After discussing with danh, I noticed that `a/**/b` would not match `a/b`.

After correcting this and re-running "pre-commit run --all", additional
files were reindented, including the codeformat script itself.
2021-04-30 15:30:13 -05:00
Scott Shawcroft
0573c17d70
Merge pull request #4690 from jepler/fix-vm-dispatch
py/vm.c: Restore lost bits of MICROPY_OPT_COMPUTED_GOTO_SAVE_SPACE
2021-04-30 10:40:07 -07:00
Dan Halbert
353e55d710
Merge pull request #4688 from jepler/fix-pixelbuf-subscr
objtype: Restore our customized behavior of instance_subscr
2021-04-30 10:45:44 -04:00
Jeff Epler
41b273e0c1 py/vm.c: Restore lost bits of MICROPY_OPT_COMPUTED_GOTO_SAVE_SPACE
This fixes a problem where boards that enabled this (most SAM D21s)
would crash on the first bytecode instruction.

Closes: #4686
2021-04-30 09:17:40 -05:00
Damien George
6b7c8d3e72 py/runtime: Remove commented-out code from mp_deinit().
These commented-out lines of code have been unused for a long time, so
remove them to avoid confusion as to why they are there.

mp_obj_dict_free() never existed, this line was converted from
mp_map_deinit() and commented out as soon as it was added.  The call to
mp_map_deinit(mp_loaded_modules_map) was commented in
1a1d11fa32.

Fixes issue #3507.

Signed-off-by: Damien George <damien@micropython.org>
2021-04-30 16:35:14 +10:00
Damien George
916c3fd23f py/scheduler: Add optional port hook for when something is scheduled.
So that a port can "wake up" when there is work to do.

Signed-off-by: Damien George <damien@micropython.org>
2021-04-30 15:32:16 +10:00
Damien George
e9e9c76ddf all: Rename mp_keyboard_interrupt to mp_sched_keyboard_interrupt.
To match mp_sched_exception() and mp_sched_schedule().

Signed-off-by: Damien George <damien@micropython.org>
2021-04-30 15:13:43 +10:00
Damien George
7cbf826a95 py/scheduler: Add mp_sched_exception() to schedule a pending exception.
This helper is added to properly set a pending exception, to mirror
mp_sched_schedule(), which schedules a function.

Signed-off-by: Damien George <damien@micropython.org>
2021-04-30 15:13:43 +10:00
Damien George
7e549b6718 py/profile: Use mp_handle_pending() to raise pending exception.
If MICROPY_ENABLE_SCHEDULER is enabled then MP_STATE_VM(sched_state) must
be updated after handling the pending exception, which is done by the
mp_handle_pending() function.

Signed-off-by: Damien George <damien@micropython.org>
2021-04-30 15:13:43 +10:00
Jeff Epler
d1e17fdccc objtype: Restore our customized behavior of instance_subscr
For pixelbuf's [] to be able to call _transmit on the Python subclass,
we need to do the subscripting operation specially.
2021-04-29 11:59:11 -05:00
microDev
1b972c51c3
add authmode class 2021-04-29 18:42:36 +05:30
Dan Halbert
f06d54524d merge from adafruit 2021-04-28 23:48:26 -04:00
Dan Halbert
8500e846c6 partially working 2021-04-27 23:53:23 -04:00
Dan Halbert
f98a54628b wip: compiles 2021-04-27 14:37:36 -04:00
Scott Shawcroft
17fcd499b2
Merge pull request #4649 from tannewt/merge_1.11_2
Merge MicroPython v1.11 into CircuitPython
2021-04-27 11:02:02 -07:00
Damien George
c5cbfd545a py/dynruntime.h: Add mp_obj_get_array() function.
Signed-off-by: Damien George <damien@micropython.org>
2021-04-28 00:06:46 +10:00
Damien George
d4b706c4d0 py: Add option to compile without any error messages at all.
This introduces a new option, MICROPY_ERROR_REPORTING_NONE, which
completely disables all error messages.  To be used in cases where
MicroPython needs to fit in very limited systems.

Signed-off-by: Damien George <damien@micropython.org>
2021-04-27 23:51:52 +10:00
Dan Halbert
7a40b4daec very much WIP 2021-04-26 23:54:01 -04:00