djsundog/circuitpython
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge tag 'v1.18'

Browse Source 
Boosted performance, board.json metadata, more mimxrt, rp2, samd features

This release of MicroPython sees a boost to the overall performance of the
VM and runtime.  This is achieved by the addition of an optional cache to
speed up general hash table lookups, as well as a fast path in the VM for
the LOAD_ATTR opcode on instance types.  The new configuration options are
MICROPY_OPT_MAP_LOOKUP_CACHE and MICROPY_OPT_LOAD_ATTR_FAST_PATH.  As part
of this improvement the MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE option has
been removed, which provided a similar map caching mechanism but with the
cache stored in the bytecode, which made it not useful on bare metal ports.
The new mechanism is measured to be at least as good as the old one,
applies to more map lookups, has a constant RAM overhead, and applies to
native code as well as bytecode.

These performance options are enabled on the esp32, mimxrt, rp2, stm32 and
unix ports.  For esp32 and mimxrt some code is also moved to RAM to further
boost performance.  On stm32, performance increases by about 20% for
benchmarks that are heavy on name lookups, like misc_pystone.py and
misc_raytrace.py.  On esp32 performance can increase by 2-3x, and on mimxrt
it is up to 6x.

All boards in all ports now have a board.json metadata file, which is used
to automatically build firmware and generate a webpage for that board
(among other possibilities).  Auto-build scripts have been added for this
purpose and they build all esp32, mimxrt, rp2, samd and stm32 boards.  The
generated output is available at https://micropython.org/download.

Support for FROZEN_DIR and FROZEN_MPY_DIR has been deprecated for some time
and was finally removed in this release.  Instead of these, FROZEN_MANIFEST
can be used.  The io.resource_stream() function is also removed, replaced
by the pure Python version in micropython-lib.

The search order for importing frozen Python modules is now controlled by
the ".frozen" entry in sys.path.  This string is added by default in the
second position in sys.path.  User code should adjust sys.path depending on
the desired behaviour.  Putting ".frozen" first in sys.path will speed up
importing frozen modules.

A bug in multiple precision integers with bitwise of -0 was fixed in commit
2c139bbf4e.

The platform module has been added to allow querying the compiler and
underlying SDK/HAL/libc version.  This is enabled on esp32, mimxrt and
stm32 ports.

The mpremote tool now supports seek, flush, mkdir and rmdir on PC-mounted
filesystems.  And a help command has been added.

The documentation has seen many additions and improvements thanks (for a
second time) to the Google Season of Docs project.  The rp2 documentation
now includes a reference for PIO assembly instructions, a PIO quick
reference and a PIO tutorial.  The random and stm modules have been
documented, along with sys.settrace, manifest.py files and mpremote.  There
is also now more detail about the differences between MicroPython and
standard Python 3.5 and above.

The esp32 port sees support for ESP32-S3 SoCs, and new boards GENERIC_S3,
ESP32_S2_WROVER, LOLIN_S2_MINI, LOLIN_S2_PICO and UM_FEATHERS2NEO.  The PWM
driver has been improved and now supports all PWM timers and channels, and
the duty_u16() and duty_ns() methods, and it keeps the duty constant when
changing frequency.  The machine.bitstream() function has been improved to
use RMT, with an option to select the original bit-banging implementation.

The mimxrt port gained new hardware features: SDRAM and SD card support, as
well as network integration with a LAN driver.  The machine.WDT class was
added along with the machine.reset_cause(), machine.soft_reset(),
machine.unique_id() add machine.bitstream() functions.  DHT sensor support
was added, and f-strings were enabled.

The rp2 port now has support for networking, and bluetooth using NimBLE.
The Nina-W10 WiFi/BT driver is fully integrated and supported by the new
Arduino Nano RP2040 connect board.  I2S protocol support is added along
with a machine.bitstream() driver and DHT sensor support.  The PWM driver
had a bug fix with the accuracy of setting/getting the frequency, and the
duty value is now retained when changing the frequency.

On the samd port there is now support for the internal flash being a block
device, and for filesystems and the os module.  Pin and LED classes have
been implemented.  There are more time functions, more Python features
enabled, and the help() function is added.  SEEED_WIO_TERMINAL and
SEEED_XIAO board definitions are now available.

The stm32 port now has support for F427, F479 and H7A3(Q)/H7B3(Q) MCUs, and
new board definitions for VCC_GND_H743VI, OLIMEX_H407, MIKROE_QUAIL,
GARATRONIC_PYBSTICK26_F411, STM32H73B3I_DK.  A bug was fixed in the SPI
driver where a SPI transfer could fail if the CYW43 WiFi driver was also
active at the same time.

On the windows port the help() function has been enabled, and support for
build variants added, to match the unix port.

The zephyr port upgraded Zephyr to v2.7.0.

The change in code size since the previous release for various ports is
(absolute and percentage change in the text section):

       bare-arm:  -1520  -2.605%
    minimal x86:  -2256  -1.531%
       unix x64:   -457  -0.089%
    unix nanbox:   -925  -0.204%
          stm32:   +312  +0.079% PYBV10
         cc3200:   -176  -0.096%
        esp8266:   +532  +0.076% GENERIC
          esp32: +27096  +1.820% GENERIC
            nrf:   -212  -0.121% pca10040
            rp2:  +9904  +2.051% PICO
           samd: +35332 +33.969% ADAFRUIT_ITSYBITSY_M4_EXPRESS

The changes that dominate these numbers are:
- bare-arm, minimal: use of new MICROPY_CONFIG_ROM_LEVEL_MINIMUM option and
  subsequent disabling of remaining optional features
- unix, cc3200, nrf: general code size reductions of the core
- stm32: performance improvements, addition of platform module
- esp8266: enabling f-strings
- esp32: use of -O2 instead of -Os
- rp2: machine.I2S and other new hardware features
- samd: filesystem support and other new hardware features

Thanks to everyone who contributed to this release: Alan Dragomirecký,
Alexey Shvetsov, Andrew Leech, Andrew Scheller, Antoine Aubert, Boris
Vinogradov, Chris Boudacoff, Chris Fiege, Christian Decker, Damien George,
Daniel Gorny, Dave Hylands, David Michieli, Emilie Feral, Frédéric Pierson,
gibbonsc, Henk Vergonet, iabdalkader, Ihor Nehrutsa, Jan Hrudka, Jan Staal,
jc_.kim, Jim Mussared, Jonathan Hogg, Laurens Valk, leo chung, Lorenzo
Cappelletti, Magnus von Wachenfeldt, Matt Trentini, Matt van de Werken,
Maureen Helm, Michael Bentley, Michael Buesch, Mike Causer, Mike Teachman,
Mike Wadsten, Ned Konz, NitiKaur, oli, patrick, Patrick Van Oosterwijck,
Peter Boin, Peter Hinch, Peter van der Burg, Philipp Ebensberger, Pooya
Moradi, retsyo, robert-hh, roland van straten, Scott Armitage, Sebastian
Wicki, Seon Rozenblum, Sergei Silnov, Simon Baatz, Stewart Bonnick, stijn,
Tobias Thyrrestrup, Tomas Vanek, YoungJoon Chun.

What follows is a detailed list of changes, generated from the git commit
history, and organised into sections.

Main components
===============

all:
- remove MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE
- update Python formatting to latest Black version 21.12b0
- remove support for FROZEN_DIR and FROZEN_MPY_DIR

py core:
- parse: simplify parse nodes representing a list
- emitnative: ensure load_subscr does not clobber existing REG_RET
- mpconfig.h: define initial templates for "feature levels"
- vm: add a fast path for LOAD_ATTR on instance types
- map: add an optional cache of (map+index) to speed up map lookups
- builtinimport: forward all debug printing to MICROPY_DEBUG_PRINTER
- add wrapper macros so hot VM functions can go in fast code location
- runtime: fix crash when exc __new__ doesn't return an exc instance
- mpconfig.h: define the "extra" feature level
- mpconfig.h: revert MICROPY_REPL_INFO to disabled at all levels
- gc: add hook to run code during time consuming GC operations
- showbc: print unary-op string when dumping bytecode
- modsys: replace non-ASCII quote char with ASCII char
- runtime: allow types to use both .attr and .locals_dict
- lexer: support nested [] and {} characters within f-string params
- objfun.h: remove obsolete comments about entries in extra_args
- builtinimport: refactor module importing
- showbc: fix printing of raw bytecode header on nanbox builds
- modio: remove io.resource_stream function
- only search frozen modules when '.frozen' is found in sys.path
- mkrules.cmake: set frozen preprocessor defs early
- runtime: allow initialising sys.path/argv with defaults
- mpstate.h: only include sys.path/argv objects in state when enabled
- mpz: fix bugs with bitwise of -0 by ensuring all 0's are positive
- qstr: reset mpstate.qstr_last_chunk before raising an error
- modbuiltins: add additional macro for extending builtins
- mpconfig.h: define MICROPY_PY_USSL_FINALISER only if not defined

extmod:
- machine_i2c: make SoftI2C configurable via macro option
- machine_spi: make SoftSPI configurable via macro option
- modonewire: make _onewire module configurable via macro option
- machine_pwm: factor out machine.PWM bindings to common code
- move modnetwork and modusocket from stm32 to extmod
- modnetwork: add STA_IF and AP_IF constants
- modnetwork: add extended socket state
- modusocket: add read/write stream methods to socket object
- modnetwork: define network interfaces in port config files
- network_cyw43: make consistent use of STA and AP constants
- modnetwork: remove STM32 references
- modnetwork: remove modnetwork socket u_state member
- mpbthci.h: add mp_bluetooth_hci_uart_any prototype
- nimble: add nimble CMake fragment file
- add platform module
- moduplatform: improve implementation for PC ports
- vfs_posix_file: support MP_STREAM_POLL in vfs_posix_file_ioctl
- modbluetooth: add connection interval to gap_connect
- nimble: update to NimBLE v1.4
- nimble: remove workaround for OS_ENOMEM
- uasyncio: fix gather returning exceptions from a cancelled task
- uplatform: remove unused definitions
- uplatform: use generic custom platform string
- network_ninaw10: fix scan list order to match other NICs
- modbluetooth: support gap_connect(None) to cancel a connection
- modure: redirect regex debug printing to mp_printf
- network_ninaw10: fix config of AP mode
- network_ninaw10: disable active connections before connecting
- network_ninaw10: make NIC state persistent
- network_ninaw10: return -1 on timeout from recv/send
- network_ninaw10: make recv/recvfrom interchangeable
- moduplatform: detect xtensa arch
- modusocket: allow setting timeout on unbound sockets
- modusocket: initialise accepted socket state
- network_ninaw10: use socket timeout preset in modusocket
- modbluetooth: fix conditional compilation of ringbuf_put_uuid
- modbluetooth: put declaration of connect_cancel in correct place

shared:
- libc/string0: don't include string.h, and provide __memcpy_chk
- runtime/pyexec: cleanup EXEC_FLAG flag constants

drivers:
- ninaw10: add ublox Nina-W10 WiFi/BT module driver
- lsm6dsox: add LSM6DSOX driver and examples
- neopixel: avoid heap alloc in fill()
- ninaw10: fix BSSID byte order, and add null byte to ESSID
- ninaw10/nina_wifi_drv: fix DNS resolution

mpy-cross: no changes specific to this component/port

lib:
- mynewt-nimble: switch to the MicroPython fork of NimBLE
- asf4: point submodule to latest commit on circuitpython branch
- update pico-sdk to 1.3.0 and tinyusb to 0.12.0
- stm32lib: update library for L4 v1.17.0, new G4, WL, and MMC fixes
- stm32lib: update library for fix to F7 USB HS

Support components
==================

docs:
- library/os.rst: clarify littlefs requirements for block erase
- library/bluetooth.rst: update incorrect link to gatts_write
- make.bat: change Windows output dir from '_build' to 'build'
- library/machine.I2S.rst: specify that I2S.shift args are kw-only
- esp32: explain ESP32 PWM modes, timers, and channels
- rp2: add reference for PIO assembly instructions, and PIO tutorial
- library/random.rst: document the random module
- reference/mpremote.rst: add docs for mpremote
- reference/manifest.rst: add docs for manifest.py files
- library/stm.rst: document the stm module
- esp32/tutorial: add an example of peripheral control via regs
- rp2/general.rst: fix typo with missing spaces
- library/framebuf.rst: adjust dimensions in example
- library/rp2.rst: update function asm_pio_encode to add sideset_opt
- reference/filesystem.rst: add detail on how to use littlefs fuse
- rp2/quickref.rst: add section on PIO
- library/sys.rst: add docs for sys.settrace
- esp8266/tutorial: fix comments of FrameBuffer examples
- library/uasyncio.rst: detail exception behaviour in cancel/timeout
- library/machine.Timer.rst: document 'id' as positional-only arg
- library/machine.SPI.rst: add example SPI usage
- library/machine.Timer.rst: document `period` and `callback` args
- library/machine.Pin.rst: add Pin.ANALOG mode constant
- remove trailing spaces and convert tabs to spaces
- library/sys.rst: add note about '.frozen' as an entry in sys.path
- differences: document details of new PEPs/features in Python 3.5+
- update copyright year range to include 2022
- esp32: update RMT quickref example to match latest code

examples: no changes specific to this component/port

tests:
- perf_bench: use math.log instead of math.log2
- basics: add tests for type-checking subclassed exc instances
- micropython/const.py: add comment about required config for test
- cpydiff: clarify f-string diffs regarding concatenation
- basics/int_big_cmp.py: add more tests for big-int comparison
- extmod: skip uselect_poll_udp when poll() is not available

tools:
- autobuild: add auto build for GENERIC_C3_USB
- ci.sh: use IDF v4.4 as part of esp32 CI and build GENERIC_S3
- autobuild: add the MIMXRT1010_EVK board to autobuild
- ci.sh: use a specific ESP IDF v4.4 commit
- autobuild: add script to generate website board metadata
- dfu.py: make tool work with python3 when parsing DFU files
- autobuild: automatically build all mimxrt, rp2 and samd boards
- autobuild: automatically build all stm32 boards
- mpremote: implement seek and flush in ioctl method
- autobuild: automatically build all esp32 boards
- upip.py: support == to specify exact package version
- makemanifest.py: make str conversion compatible with Python 2
- makemanifest.py: merge make-frozen.py
- mpremote: add mkdir and rmdir to RemoteFS
- mpremote: add help command
- mpremote: add link to mpremote docs URL in help message
- upip.py: skip '.frozen' entry in sys.path for install path
- autobuild: build esp8266 OTA image with GENERIC_1M board
- ci.sh: upgrade Zephyr docker image to v0.21.0
- ci.sh: build zephyr nucleo_wb55rg to test zephyr bluetooth build

CI:
- workflows: use Python 3.8 for macos workflow
- workflows: add new workflow to build ports download metadata

The ports
=========

all ports:
- add board.json for all boards
- add images, features and urls to board.json
- add '.frozen' as the first entry in sys.path
- move '.frozen' to second entry in sys.path

bare-arm port:
- mpconfigport.h: use MICROPY_CONFIG_ROM_LEVEL_MINIMUM
- mpconfigport.h: disable remaining optional features

cc3200 port: no changes specific to this component/port

esp8266 port:
- boards/GENERIC: enable f-strings
- extract qstr from object when comparing keys in config()
- etshal.h: remove unneeded function declarations
- allow building a board to any dest directory

esp32 port:
- boards: add new FeatherS2-Neo board definition
- machine_timer: use tx_update member for IDF 4.4 and above
- add support for ESP32-S3 SoCs
- boards: add new GENERIC_S3 board definition
- machine_hw_spi: fix hardware SPI DMA channels for S2/S3
- boards: add board definition for ESP32-S2-WROVER module
- boards: add LOLIN_S2_MINI ESP32-S2 board
- machine_pwm: add support for all PWM timers and channels
- README: updated readme with req IDF vers for ESP32-S2, C3 and S3
- usb: add USB host connection detection for CDC serial output
- machine_pin: block out IO16 and IO17 when using SPIRAM on ESP32
- mpthreadport: fix TCB cleanup function so thread_mutex is ready
- main: add option for a board to hook code into startup sequence
- split out WLAN code from modnetwork.c to network_wlan.c
- enable optimisations and move code to iRAM to boost performance
- usb: improve speed of USB CDC output
- add specific deploy_s2.md instructions for esp32-s2
- boards/LOLIN_S2_MINI: add image to board.json
- boards: update board and deploy metadata for UM_xxx boards
- usb: further improve speed of USB CDC output
- boards/LOLIN_S2_PICO: add LOLIN_S2_PICO board definition files
- boards/ESP32_S2_WROVER: link to specific deploy_s2 instructions
- support building with latest IDF v5
- in machine_i2s, send null samples in underflow situations
- in machine_i2s, make object reference arrays root pointers
- add SDCard support for S3, and a GENERIC_S3_SPIRAM board
- boards/GENERIC_S3: enable BLE on ESP32 S3
- machine_pwm: implement duty_u16() and duty_ns() PWM methods
- extract qstr from object when comparing keys in config()
- machine_pin: make GPIO 26 usable for S2,S3 if SPIRAM not config'd
- machine_hw_spi: fix SPI default pins reordering on ESP32-S2/S3
- machine_hw_spi: set proper default SPI(id=1) pins on S2,S3 and C3
- machine_hw_spi: set proper default SPI(id=2) pins on S2 and S3
- boards: remove SPI pin defaults from GENERIC S2/S3 boards
- modnetwork: synchronize WiFi AUTH_xxx constants with IDF values
- machine_pwm: keep duty constant when changing frequency
- machine_bitstream: replace bit-bang code with RMT-based driver
- machine_i2s: add support for ESP-IDF 4.4
- machine_bitstream: fix signal duplication on output pins
- esp32: enable platform module with IDF version
- boards/GENERIC_D2WD: build with -Os optimisation
- esp32_rmt: install RMT driver on core 1
- machine_bitstream: reinstate bitstream bit-bang implementation

javascript port: no changes specific to this component/port

mimxrt port:
- sdcard: implement SDCard driver
- machine_bitstream: add bitstream function to machine module
- rework flash configuration
- sdram: add SDRAM support
- eth: add LAN support and integrate the network module
- modmachine: implement machine.WDT() and machine.reset_cause()
- boards: fix the D14/D15 pin assignment of MIMXRT1050/60/64_EVK
- hal: remove duplicate definitions from flexspi_hyper_flash.h
- dma_channel: fix the DMA channel management
- fix cycle counter for time.ticks_cpu() and machine.bitstream()
- add dht_readinto() to the mimxrt module, and freeze dht.py
- extend the help() message and README.md
- mpconfigport.h: enable f-strings
- modmachine: implement soft_reset() and unique_id() functions
- boards/make-pins.py: allow empty lines and comments in pins.csv
- optimize the runtime speed
- enable the platform module
- boards: add the Seeed ARCH MIX board
- boards: update the board.json files and add deploy_xx.md files
- fix mp_hal_quiet_timing_enter()/exit() so timer still runs
- support PWM using the FLEXPWM and QTMR modules
- define UART 0 on MIMXRT boards
- support selection of PHY type and address
- re-enable eth checksum creation by HW
- fix a tiny unnoticed bug in sdcard.c
- add a driver for the DP83848 PHY device
- refactor the reading of the machine id
- enable ticks_cpu at boot time for NDEBUG builds only
- use -Og instead of -O0 for DEBUG builds
- tidy up the board flash related files
- hal: allow readSampleClkSrc to be configured by a board
- enable MICROPY_PY_USSL_FINALISER

minimal port:
- mpconfigport.h: use MICROPY_CONFIG_ROM_LEVEL_MINIMUM
- Makefile: don't force a 32-bit build
- mpconfigport.h: disable features that are not needed

nrf port:
- Makefile: improve Black Magic Probe commands
- main: use VFS helper function to mount fs and chdir

pic16bit port: no changes specific to this component/port

powerpc port: no changes specific to this component/port

qemu-arm port: no changes specific to this component/port

rp2 port:
- mpconfigport.h: enable heapq module
- add support for bluetooth module using NimBLE
- add framework for networking
- mpconfigport.h: use the "extra" feature level
- enable optimisations (comp goto, map cache, fast attr)
- machine_i2s: add I2S protocol support
- add support for Nina-W10 WiFi/BT module
- boards: add support for Arduino Nano RP2040
- machine_bitstream: implement the machine.bitstream driver
- boards: add neopixel.py to manifest.py
- rp2_pio: support exec with sideset
- boards/PIMORONI_PICOLIPO_16MB: fix 16MB flash size
- boards: add PYBSTICK26 RP2040 board definition
- machine_uart: handle and clear UART RX timeout IRQ
- boards/ARDUINO_NANO_RP2040_CONNECT: set default I2C pins
- machine_pwm: fix PWM frequency setting
- machine_pwm: keep duty value when changing the frequency
- add support for DHT11 and DHT22 sensors
- CMakeLists.txt: allow a board to override PICO_BOARD
- boards/GARATRONIC_PYBSTICK26_RP2040: use correct pico-sdk board cfg

samd port:
- integrate latest asf4, add help, more time funcs and uPy features
- samd_soc: allow a board to configure the low-level MCU config
- add internal flash block device, filesystem and uos support
- add Pin and LED classes, and machine.unique_id
- boards/ADAFRUIT_FEATHER_M0_EXPRESS: update for flash and pins
- boards/ADAFRUIT_ITSYBITSY_M4_EXPRESS: update for flash and pins
- boards/MINISAM_M4: update for flash and pins
- boards/ADAFRUIT_TRINKET_M0: update for flash and pins
- boards/SAMD21_XPLAINED_PRO: update for flash and pins
- boards/SEEED_WIO_TERMINAL: add new board definition
- boards/SEEED_XIAO: add new board definition
- README.md: update README to reflect new features and boards

stm32 port:
- pin: enable GPIO clock of pin if it's constructed without init
- main: don't unconditionally enable GPIO A,B,C,D clocks
- boards/VCC_GND_H743VI: add board definition for VCC_GND_H743VI
- boards/OLIMEX_E407: add Ethernet RMII support
- boards/LEGO_HUB_NO6: remove user paths from cc2564 init file
- boards: remove trailing spaces, and add newline at end of file
- add basic support for STM32H750
- add support for H7A3(Q)/H7B3(Q), and STM32H73B3I_DK board defn
- suggest putting code in main.py not boot.py
- boards/make-pins.py: allow a CPU pin to be hidden
- boards/make-pins.py: allow empty lines and comments in pins.csv
- dma: add functions for external users of DMA to enable clock
- enable LOAD_ATTR fast path, and map lookup caching on >M0
- boards: add OLIMEX H407 board definition
- enable platform module
- extended flash filesystem space to 512K on H743 boards
- boards/NUCLEO_H743ZI: enable VfsLfs2 on NUCLEO_H743ZI(2) boards
- boards: add PF11-BOOT0 to stm32f091_af.csv
- machine_i2c: use hardware I2C for STM32H7
- sdram: enforce gcc opt, and use volatile and DSB in sdram_test
- usbd_cdc_interface: allow a board to hook into USBD CDC RX events
- mpbthciport: allow a board to hook BT HCI poll functions
- pendsv: allow a board to add entries for pendsv_schedule_dispatch
- boards: add images to board.json for Adafruit and VCC_GND boards
- uart: fix race conditions and clearing status in IRQ handler
- mpconfigport.h: use the "extra" feature level
- in machine_i2s, send null samples in underflow situations
- in machine_i2s, make object reference arrays root pointers
- led: support an extra 2 LEDs in board configuration
- boards/MIKROE_CLICKER2_STM32: add more detail to board.json
- boards: add new board MikroElektronika Quail, and F427 support
- main: run optional frozen module at boot
- sdio: don't explicitly disable DMA2 on deinit of SDIO
- dma: make DMA2_Stream3 exclusive to SDIO when CYW43 enabled
- boards: build NUCLEO_WB55 and STM32F769DISC without mboot enabled
- boards: add PYBSTICK26 F411 board definition
- boards/NADHAT_PYBF405: rename board to GARATRONIC_NADHAT_F405
- usb: use a table of allowed values to simplify usb_mode get/set
- boards/NUCLEO_WB55: update rfcore_firmwre for new WS
- flashbdev: support generic flash storage config via link symbols
- boards: convert F413,F439,H743,L4xx,WB55 to new flash FS config
- add support for F479 MCUs
- include HAL MMC code in F4 builds
- boards/make-pins.py: use cpu pins to define static alt-fun macros
- boards/NUCLEO_WB55: fix LED ordering
- boards/LEGO_HUB_NO6: set filesystem label as HUB_NO6
- boards: remove stray '+' characters at start of lines in ld files
- boards: remove unused MICROPY_HW_ENABLE_TIMER config
- boards: enable MICROPY_HW_ENABLE_SERVO on various boards
- update L4 code to build with latest stm32lib and L4 HAL 1.17.0
- main: call sdcard_init when only MICROPY_HW_ENABLE_MMCARD enabled
- sdcard: support 8-bit wide SDIO bus
- sdcard: add config option to force MM card capacity
- factoryreset: init vfs flags before calling pyb_flash_init_vfs
- qspi: fix typo in address comment
- boards/make-pins.py: generate empty ADC table if needed
- boards/OLIMEX_H407: fix typo in OLIMEX H407 board.json
- network_wiznet5k: fix build error with wiznet5k and lwip enabled
- enable MICROPY_PY_USSL_FINALISER

teensy port:
- switch to use manifest.py instead of FROZEN_DIR

unix port:
- enable LOAD_ATTR fast path, and map lookup caching
- modusocket: support MP_STREAM_POLL in unix socket_ioctl
- modos: add support for uos.urandom(n)
- coverage: change remaining printf to mp_printf
- Makefile: use -Og instead of -O0 for debug builds

windows port:
- README: remove unsupported Python instructions for Cygwin
- mpconfigport.h: enable help and help("modules")
- add support for build variants to windows port
- run tests via Makefile
- appveyor: build both standard and dev variants
- appveyor: build mpy-cross only once for mingw-w64
- msvc: run qstr preprocessing phase in parallel

zephyr port:
- mphalport.h: remove unused and unimplemented C-level pin API
- increase minimum CMake version to 3.20.0
- update include path to reboot.h
- get UART console device from devicetree instead of Kconfig
- use CONFIG_USB_DEVICE_STACK for conditional USB device support
- upgrade to Zephyr v2.7.0
- modbluetooth_zephyr: provide dummy connect_cancel function
This commit is contained in:
Jeff Epler 2022-02-15 12:36:26 -06:00
commit 01cabb0324
117 changed files with 2755 additions and 1534 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.gitmodules vendored
View File

@ -92,9 +92,6 @@
[submodule "frozen/circuitpython-stage"] [submodule "frozen/circuitpython-stage"]
path = frozen/circuitpython-stage path = frozen/circuitpython-stage
url = https://github.com/python-ugame/circuitpython-stage.git url = https://github.com/python-ugame/circuitpython-stage.git
[submodule "ports/cxd56/spresense-exported-sdk"]
path = ports/cxd56/spresense-exported-sdk
url = https://github.com/sonydevworld/spresense-exported-sdk.git
[submodule "frozen/Adafruit_CircuitPython_SD"] [submodule "frozen/Adafruit_CircuitPython_SD"]
path = frozen/Adafruit_CircuitPython_SD path = frozen/Adafruit_CircuitPython_SD
url = https://github.com/adafruit/Adafruit_CircuitPython_SD.git url = https://github.com/adafruit/Adafruit_CircuitPython_SD.git

2
LICENSE
View File

@ -1,6 +1,6 @@
The MIT License (MIT) The MIT License (MIT)
Copyright (c) 2013-2021 Damien P. George Copyright (c) 2013-2022 Damien P. George
Permission is hereby granted, free of charge, to any person obtaining a copy Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal of this software and associated documentation files (the "Software"), to deal

181
docs/differences/python_35.rst Normal file
View File

@ -0,0 +1,181 @@
.. _python_35:
Python 3.5
==========
Below is a list of finalised/accepted PEPs for Python 3.5 grouped into their impact to MicroPython.
+----------------------------------------------------------------------------------------------------------+---------------+
| **Extensions to the syntax:** | **Status** |
+--------------------------------------------------------+-------------------------------------------------+---------------+
| `PEP 448 <https://www.python.org/dev/peps/pep-0448/>`_ | additional unpacking generalizations | |
+--------------------------------------------------------+-------------------------------------------------+---------------+
| `PEP 465 <https://www.python.org/dev/peps/pep-0465/>`_ | a new matrix multiplication operator | Completed |
+--------------------------------------------------------+-------------------------------------------------+---------------+
| `PEP 492 <https://www.python.org/dev/peps/pep-0492/>`_ | coroutines with async and await syntax | Completed |
+--------------------------------------------------------+-------------------------------------------------+---------------+
| **Extensions and changes to runtime:** |
+--------------------------------------------------------+-------------------------------------------------+---------------+
| `PEP 461 <https://www.python.org/dev/peps/pep-0461/>`_ | % formatting for binary strings | Completed |
+--------------------------------------------------------+-------------------------------------------------+---------------+
| `PEP 475 <https://www.python.org/dev/peps/pep-0475/>`_ | retrying system calls that fail with EINTR | Completed |
+--------------------------------------------------------+-------------------------------------------------+---------------+
| `PEP 479 <https://www.python.org/dev/peps/pep-0479/>`_ | change StopIteration handling inside generators | Completed |
+--------------------------------------------------------+-------------------------------------------------+---------------+
| **Standard library changes:** |
+--------------------------------------------------------+-------------------------------------------------+---------------+
| `PEP 471 <https://www.python.org/dev/peps/pep-0471/>`_ | os.scandir() | |
+--------------------------------------------------------+-------------------------------------------------+---------------+
| `PEP 485 <https://www.python.org/dev/peps/pep-0485/>`_ | math.isclose(), a function for testing | Completed |
| | approximate equality | |
+--------------------------------------------------------+-------------------------------------------------+---------------+
| **Miscellaneous changes:** |
+--------------------------------------------------------+-------------------------------------------------+---------------+
| `PEP 441 <https://www.python.org/dev/peps/pep-0441/>`_ | improved Python zip application support | |
+--------------------------------------------------------+-------------------------------------------------+---------------+
| `PEP 486 <https://www.python.org/dev/peps/pep-0486/>`_ | make the Python Laucher aware of virtual | |
| | environments | |
+--------------------------------------------------------+-------------------------------------------------+---------------+
| `PEP 484 <https://www.python.org/dev/peps/pep-0484/>`_ | type hints (advisory only) | In Progress |
+--------------------------------------------------------+-------------------------------------------------+---------------+
| `PEP 488 <https://www.python.org/dev/peps/pep-0488/>`_ | elimination of PYO files | Not relevant |
+--------------------------------------------------------+-------------------------------------------------+---------------+
| `PEP 489 <https://www.python.org/dev/peps/pep-0489/>`_ | redesigning extension module loading | |
+--------------------------------------------------------+-------------------------------------------------+---------------+
Other Language Changes:
+-----------------------------------------------------------------------------------------------------------+---------------+
| Added the *namereplace* error handlers. The *backslashreplace* error handlers now work with decoding and | |
| translating. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| Property docstrings are now writable. This is especially useful for collections.namedtuple() docstrings | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| Circular imports involving relative imports are now supported. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
New Modules:
* `typing <https://docs.python.org/3/whatsnew/3.5.html#typing>`_
* `zipzap <https://docs.python.org/3/whatsnew/3.5.html#zipapp>`_
Changes to built-in modules:
+-----------------------------------------------------------------------------------------------------------+---------------+
| `collections <https://docs.python.org/3/whatsnew/3.5.html#collections>`_ |
+-----------------------------------------------------------------------------------------------------------+---------------+
| The *OrderedDict* class is now implemented in C, which makes it 4 to 100 times faster. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| *OrderedDict.items()* , *OrderedDict.keys()* , *OrderedDict.values()* views now support reversed() | |
| iteration. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| The deque class now defines *index()*, *insert()*, and *copy()*, and supports the + and * operators. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| Docstrings produced by namedtuple() can now be updated. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| The UserString class now implements the *__getnewargs__()*, *__rmod__()*, *casefold()*, *format_map()*, | |
| *isprintable()*, and *maketrans()* methods to match the corresponding methods of str. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| `heapq <https://docs.python.org/3/whatsnew/3.5.html#heapq>`_ |
+-----------------------------------------------------------------------------------------------------------+---------------+
| Element comparison in *merge()* can now be customized by passing a key function in a new optional key | |
| keyword argument, and a new optional *reverse* keyword argument can be used to reverse element comparison | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| `io <https://docs.python.org/3/whatsnew/3.5.html#io>`_ |
+-----------------------------------------------------------------------------------------------------------+---------------+
| A new *BufferedIOBase.readinto1()* method, that uses at most one call to the underlying raw stream's | |
| *RawIOBase.read()* or *RawIOBase.readinto()* methods | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| `json <https://docs.python.org/3/whatsnew/3.5.html#json>`_ | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| JSON decoder now raises JSONDecodeError instead of ValueError to provide better context information about | |
| the error. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| `math <https://docs.python.org/3/whatsnew/3.5.html#math>`_ |
+-----------------------------------------------------------------------------------------------------------+---------------+
| Two new constants have been added to the math module: *inf* and *nan*. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| A new function *isclose()* provides a way to test for approximate equality. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| A new *gcd()* function has been added. The *fractions.gcd()* function is now deprecated. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| `os <https://docs.python.org/3/whatsnew/3.5.html#os>`_ |
+-----------------------------------------------------------------------------------------------------------+---------------+
| The new *scandir()* function returning an iterator of DirEntry objects has been added. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| The *urandom()* function now uses the *getrandom()* syscall on Linux 3.17 or newer, and *getentropy()* on | |
| OpenBSD 5.6 and newer, removing the need to use /dev/urandom and avoiding failures due to potential file | |
| descriptor exhaustion. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| New *get_blocking()* and *set_blocking()* functions allow getting and setting a file descriptor's blocking| |
| mode (O_NONBLOCK.) | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| There is a new *os.path.commonpath()* function returning the longest common sub-path of each passed | |
| pathname | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| `re <https://docs.python.org/3/whatsnew/3.5.html#re>`_ | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| References and conditional references to groups with fixed length are now allowed in lookbehind assertions| |
+-----------------------------------------------------------------------------------------------------------+---------------+
| The number of capturing groups in regular expressions is no longer limited to 100. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| The *sub()* and *subn()* functions now replace unmatched groups with empty strings instead of raising an | |
| exception. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| The *re.error* exceptions have new attributes, msg, pattern, pos, lineno, and colno, that provide better | |
| context information about the error | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| `socket <https://docs.python.org/3/whatsnew/3.5.html#socket>`_ |
+-----------------------------------------------------------------------------------------------------------+---------------+
| Functions with timeouts now use a monotonic clock, instead of a system clock. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| A new *socket.sendfile()* method allows sending a file over a socket by using the high-performance | |
| *os.sendfile()* function on UNIX, resulting in uploads being from 2 to 3 times faster than when using | |
| plain *socket.send()* | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| The *socket.sendall()* method no longer resets the socket timeout every time bytes are received or sent. | |
| The socket timeout is now the maximum total duration to send all data. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| The backlog argument of the *socket.listen()* method is now optional. By default it is set to SOMAXCONN or| |
| to 128, whichever is less. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| `ssl <https://docs.python.org/3/whatsnew/3.5.html#ssl>`_ |
+-----------------------------------------------------------------------------------------------------------+---------------+
| Memory BIO Support | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| Application-Layer Protocol Negotiation Support | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| There is a new *SSLSocket.version()* method to query the actual protocol version in use. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| The SSLSocket class now implements a *SSLSocket.sendfile()* method. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| The *SSLSocket.send()* method now raises either the *ssl.SSLWantReadError* or *ssl.SSLWantWriteError* | |
| exception on a non-blocking socket if the operation would block. Previously, it would return 0. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| The *cert_time_to_seconds()* function now interprets the input time as UTC and not as local time, per RFC | |
| 5280. Additionally, the return value is always an int. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| New *SSLObject.shared_ciphers()* and *SSLSocket.shared_ciphers()* methods return the list of ciphers sent | |
| by the client during the handshake. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| The *SSLSocket.do_handshake()*, *SSLSocket.read()*, *SSLSocket.shutdown()*, and *SSLSocket.write()* | |
| methods of the SSLSocket class no longer reset the socket timeout every time bytes are received or sent. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| The *match_hostname()* function now supports matching of IP addresses. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| `sys <https://docs.python.org/3/whatsnew/3.5.html#sys>`_ |
+-----------------------------------------------------------------------------------------------------------+---------------+
| A new *set_coroutine_wrapper()* function allows setting a global hook that will be called whenever a | |
| coroutine object is created by an async def function. A corresponding *get_coroutine_wrapper()* can be | |
| used to obtain a currently set wrapper. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| A new *is_finalizing()* function can be used to check if the Python interpreter is shutting down. | |
+-----------------------------------------------------------------------------------------------------------+---------------+
| `time <https://docs.python.org/3/whatsnew/3.5.html#time>`_ |
+-----------------------------------------------------------------------------------------------------------+---------------+
| The *monotonic()* function is now always available | |
+-----------------------------------------------------------------------------------------------------------+---------------+

191
docs/differences/python_36.rst Normal file
View File

@ -0,0 +1,191 @@
.. _python_36:
Python 3.6
==========
Python 3.6 beta 1 was released on 12 Sep 2016, and a summary of the new features can be found here:
+-----------------------------------------------------------------------------------------------------------+--------------+
| **New Syntax Features:** | **Status** |
+--------------------------------------------------------+--------------------------------------------------+--------------+
| `PEP 498 <https://www.python.org/dev/peps/pep-0498/>`_ | Literal String Formatting | |
+--------------------------------------------------------+--------------------------------------------------+--------------+
| `PEP 515 <https://www.python.org/dev/peps/pep-0515/>`_ | Underscores in Numeric Literals | |
+--------------------------------------------------------+--------------------------------------------------+--------------+
| `PEP 525 <https://www.python.org/dev/peps/pep-0525/>`_ | Asynchronous Generators | |
+--------------------------------------------------------+--------------------------------------------------+--------------+
| `PEP 526 <https://www.python.org/dev/peps/pep-0526/>`_ | Syntax for Variable Annotations (provisional) | |
+--------------------------------------------------------+--------------------------------------------------+--------------+
| `PEP 530 <https://www.python.org/dev/peps/pep-0530/>`_ | Asynchronous Comprehensions | |
+--------------------------------------------------------+--------------------------------------------------+--------------+
| **New Built-in Features:** |
+--------------------------------------------------------+--------------------------------------------------+--------------+
| `PEP 468 <https://www.python.org/dev/peps/pep-0468/>`_ | Preserving the order of *kwargs* in a function | |
+--------------------------------------------------------+--------------------------------------------------+--------------+
| `PEP 487 <https://www.python.org/dev/peps/pep-0487/>`_ | Simpler customization of class creation | |
+--------------------------------------------------------+--------------------------------------------------+--------------+
| `PEP 520 <https://www.python.org/dev/peps/pep-0520/>`_ | Preserving Class Attribute Definition Order | |
+--------------------------------------------------------+--------------------------------------------------+--------------+
| **Standard Library Changes:** |
+--------------------------------------------------------+--------------------------------------------------+--------------+
| `PEP 495 <https://www.python.org/dev/peps/pep-0495/>`_ | Local Time Disambiguation | |
+--------------------------------------------------------+--------------------------------------------------+--------------+
| `PEP 506 <https://www.python.org/dev/peps/pep-0506/>`_ | Adding A Secrets Module To The Standard Library | |
+--------------------------------------------------------+--------------------------------------------------+--------------+
| `PEP 519 <https://www.python.org/dev/peps/pep-0519/>`_ | Adding a file system path protocol | |
+--------------------------------------------------------+--------------------------------------------------+--------------+
| **CPython internals:** |
+--------------------------------------------------------+--------------------------------------------------+--------------+
| `PEP 509 <https://www.python.org/dev/peps/pep-0509/>`_ | Add a private version to dict | |
+--------------------------------------------------------+--------------------------------------------------+--------------+
| `PEP 523 <https://www.python.org/dev/peps/pep-0523/>`_ | Adding a frame evaluation API to CPython | |
+--------------------------------------------------------+--------------------------------------------------+--------------+
| **Linux/Window Changes** |
+--------------------------------------------------------+--------------------------------------------------+--------------+
| `PEP 524 <https://www.python.org/dev/peps/pep-0524/>`_ | Make os.urandom() blocking on Linux | |
| | (during system startup) | |
+--------------------------------------------------------+--------------------------------------------------+--------------+
| `PEP 528 <https://www.python.org/dev/peps/pep-0528/>`_ | Change Windows console encoding to UTF-8 | |
+--------------------------------------------------------+--------------------------------------------------+--------------+
| `PEP 529 <https://www.python.org/dev/peps/pep-0529/>`_ | Change Windows filesystem encoding to UTF-8 | |
+--------------------------------------------------------+--------------------------------------------------+--------------+
Other Language Changes:
+-------------------------------------------------------------------------------------------------------------+---------------+
| A *global* or *nonlocal* statement must now textually appear before the first use of the affected name in | |
| the same scope. Previously this was a SyntaxWarning. | |
+-------------------------------------------------------------------------------------------------------------+---------------+
| It is now possible to set a special method to None to indicate that the corresponding operation is not | |
| available. For example, if a class sets *__iter__()* to *None* , the class is not iterable. | |
+-------------------------------------------------------------------------------------------------------------+---------------+
| Long sequences of repeated traceback lines are now abbreviated as *[Previous line repeated {count} more | |
| times]* | |
+-------------------------------------------------------------------------------------------------------------+---------------+
| Import now raises the new exception *ModuleNotFoundError* when it cannot find a module. Code that currently | |
| checks for ImportError (in try-except) will still work. | |
+-------------------------------------------------------------------------------------------------------------+---------------+
| Class methods relying on zero-argument *super()* will now work correctly when called from metaclass methods | |
| during class creation. | |
+-------------------------------------------------------------------------------------------------------------+---------------+
Changes to built-in modules:
+--------------------------------------------------------------------------------------------------------------+----------------+
| `array <https://docs.python.org/3.6/whatsnew/3.6.html#array>`_ | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| Exhausted iterators of *array.array* will now stay exhausted even if the iterated array is extended. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| `binascii <https://docs.python.org/3.6/whatsnew/3.6.html#binascii>`_ | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| The b2a_base64() function now accepts an optional newline keyword argument to control whether the newline | |
| character is appended to the return value | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| `cmath <https://docs.python.org/3.6/whatsnew/3.6.html#cmath>`_ | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| The new cmath.tau (τ) constant has been added | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| New constants: *cmath.inf* and *cmath.nan* to match *math.inf* and *math.nan* , and also *cmath.infj* and | |
| *cmath.nanj* to match the format used by complex repr | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| `collections <https://docs.python.org/3.6/whatsnew/3.6.html#collections>`_ |
+--------------------------------------------------------------------------------------------------------------+----------------+
| The new Collection abstract base class has been added to represent sized iterable container classes | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| The new *Reversible* abstract base class represents iterable classes that also provide the *__reversed__()* | |
| method. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| The new *AsyncGenerator* abstract base class represents asynchronous generators. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| The *namedtuple()* function now accepts an optional keyword argument module, which, when specified, is used | |
| for the *__module__* attribute of the returned named tuple class. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| The verbose and rename arguments for *namedtuple()* are now keyword-only. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| Recursive *collections.deque* instances can now be pickled. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| `hashlib <https://docs.python.org/3.6/whatsnew/3.6.html#hashlib>`_ |
+--------------------------------------------------------------------------------------------------------------+----------------+
| BLAKE2 hash functions were added to the module. *blake2b()* and *blake2s()* are always available and support | |
| the full feature set of BLAKE2. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| The SHA-3 hash functions *sha3_224()*, *sha3_256()*, *sha3_384()*, *sha3_512()*, and *SHAKE* hash functions | |
| *shake_128()* and *shake_256()* were added. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| The password-based key derivation function *scrypt()* is now available with OpenSSL 1.1.0 and newer. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| `json <https://docs.python.org/3.6/whatsnew/3.6.html#json>`_ |
+--------------------------------------------------------------------------------------------------------------+----------------+
| *json.load()* and *json.loads()* now support binary input. Encoded JSON should be represented using either | |
| UTF-8, UTF-16, or UTF-32. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| `math <https://docs.python.org/3.6/whatsnew/3.6.html#math>`_ |
+--------------------------------------------------------------------------------------------------------------+----------------+
| The new math.tau (τ) constant has been added | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| `os <https://docs.python.org/3.6/whatsnew/3.6.html#os>`_ |
+--------------------------------------------------------------------------------------------------------------+----------------+
| A new *close()* method allows explicitly closing a *scandir()* iterator. The *scandir()* iterator now | |
| supports the context manager protocol. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| On Linux, *os.urandom()* now blocks until the system urandom entropy pool is initialized to increase the | |
| security. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| The Linux *getrandom()* syscall (get random bytes) is now exposed as the new *os.getrandom()* function. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| `re <https://docs.python.org/3.6/whatsnew/3.6.html#re>`_ |
+--------------------------------------------------------------------------------------------------------------+----------------+
| Added support of modifier spans in regular expressions. Examples: *'(?i:p)ython'* matches 'python' and | |
| 'Python', but not 'PYTHON'; *'(?i)g(?-i:v)r'* matches *'GvR'* and *'gvr'*, but not *'GVR'* . | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| Match object groups can be accessed by *__getitem__*, which is equivalent to *group()*. So *mo['name']* is | |
| now equivalent to *mo.group('name')*. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| Match objects now support index-like objects as group indices. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| `socket <https://docs.python.org/3.6/whatsnew/3.6.html#socket>`_ |
+--------------------------------------------------------------------------------------------------------------+----------------+
| The *ioctl()* function now supports the *SIO_LOOPBACK_FAST_PATH* control code. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| The *getsockopt()* constants *SO_DOMAIN* , *SO_PROTOCOL*, *SO_PEERSEC* , and *SO_PASSSEC* are now supported. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| The *setsockopt()* now supports the *setsockopt(level, optname, None, optlen: int)* form. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| The socket module now supports the address family *AF_ALG* to interface with Linux Kernel crypto API. | |
| *ALG_*, *SOL_ALG* and *sendmsg_afalg()* were added. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| New Linux constants *TCP_USER_TIMEOUT* and *TCP_CONGESTION* were added. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| `ssl <https://docs.python.org/3.6/whatsnew/3.6.html#ssl>`_ |
+--------------------------------------------------------------------------------------------------------------+----------------+
| ssl supports OpenSSL 1.1.0. The minimum recommend version is 1.0.2. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| 3DES has been removed from the default cipher suites and ChaCha20 Poly1305 cipher suites have been added. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| *SSLContext* has better default configuration for options and ciphers. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| SSL session can be copied from one client-side connection to another with the new *SSLSession* class. TLS | |
| session resumption can speed up the initial handshake, reduce latency and improve performance. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| The new *get_ciphers()* method can be used to get a list of enabled ciphers in order of cipher priority. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| All constants and flags have been converted to *IntEnum* and *IntFlags*. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| Server and client-side specific TLS protocols for *SSLContext* were added. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| Added *SSLContext.post_handshake_auth* to enable and *ssl.SSLSocket.verify_client_post_handshake()* to | |
| initiate TLS 1.3 post-handshake authentication. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| `struct <https://docs.python.org/3.6/whatsnew/3.6.html#struct>`_ | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| now supports IEEE 754 half-precision floats via the 'e' format specifier. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| `sys <https://docs.python.org/3.6/whatsnew/3.6.html#sys>`_ | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| The new *getfilesystemencodeerrors()* function returns the name of the error mode used to convert between | |
| Unicode filenames and bytes filenames. | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| `zlib <https://docs.python.org/3.6/whatsnew/3.6.html#zlib>`_ | |
+--------------------------------------------------------------------------------------------------------------+----------------+
| The *compress()* and *decompress()* functions now accept keyword arguments | |
+--------------------------------------------------------------------------------------------------------------+----------------+

95
docs/differences/python_37.rst Normal file
View File

@ -0,0 +1,95 @@
.. _python_37:
Python 3.7
==========
New Features:
+--------------------------------------------------------+--------------------------------------------------+----------------+
| **Features:** | **Status** |
+--------------------------------------------------------+--------------------------------------------------+----------------+
| `PEP 538 <https://www.python.org/dev/peps/pep-0538/>`_ | Coercing the legacy C locale to a UTF-8 based | |
| | locale | |
+--------------------------------------------------------+--------------------------------------------------+----------------+
| `PEP 539 <https://www.python.org/dev/peps/pep-0539/>`_ | A New C-API for Thread-Local Storage in CPython | |
+--------------------------------------------------------+--------------------------------------------------+----------------+
| `PEP 540 <https://www.python.org/dev/peps/pep-0540/>`_ | UTF-8 mode | |
+--------------------------------------------------------+--------------------------------------------------+----------------+
| `PEP 552 <https://www.python.org/dev/peps/pep-0552/>`_ | Deterministic pyc | |
+--------------------------------------------------------+--------------------------------------------------+----------------+
| `PEP 553 <https://www.python.org/dev/peps/pep-0553/>`_ | Built-in breakpoint() | |
+--------------------------------------------------------+--------------------------------------------------+----------------+
| `PEP 557 <https://www.python.org/dev/peps/pep-0557/>`_ | Data Classes | |
+--------------------------------------------------------+--------------------------------------------------+----------------+
| `PEP 560 <https://www.python.org/dev/peps/pep-0560/>`_ | Core support for typing module and generic types | |
+--------------------------------------------------------+--------------------------------------------------+----------------+
| `PEP 562 <https://www.python.org/dev/peps/pep-0562/>`_ | Module __getattr__ and __dir__ | Partially done |
+--------------------------------------------------------+--------------------------------------------------+----------------+
| `PEP 563 <https://www.python.org/dev/peps/pep-0563/>`_ | Postponed Evaluation of Annotations | |
+--------------------------------------------------------+--------------------------------------------------+----------------+
| `PEP 564 <https://www.python.org/dev/peps/pep-0564/>`_ | Time functions with nanosecond resolution | |
+--------------------------------------------------------+--------------------------------------------------+----------------+
| `PEP 565 <https://www.python.org/dev/peps/pep-0565/>`_ | Show DeprecationWarning in __main__ | |
+--------------------------------------------------------+--------------------------------------------------+----------------+
| `PEP 567 <https://www.python.org/dev/peps/pep-0567/>`_ | Context Variables | |
+--------------------------------------------------------+--------------------------------------------------+----------------+
Other Language Changes:
+----------------------------------------------------------------------------------------------------------+----------------+
| async and await are now reserved keywords | Completed |
+----------------------------------------------------------------------------------------------------------+----------------+
| dict objects must preserve insertion-order | |
+----------------------------------------------------------------------------------------------------------+----------------+
| More than 255 arguments can now be passed to a function; a function can now have more than 255 parameters| |
+----------------------------------------------------------------------------------------------------------+----------------+
| bytes.fromhex() and bytearray.fromhex() now ignore all ASCII whitespace, not only spaces | |
+----------------------------------------------------------------------------------------------------------+----------------+
| str, bytes, and bytearray gained support for the new isascii() method, which can be used to test if a | |
| string or bytes contain only the ASCII characters | |
+----------------------------------------------------------------------------------------------------------+----------------+
| ImportError now displays module name and module __file__ path whenfrom ... import ... fails | |
+----------------------------------------------------------------------------------------------------------+----------------+
| Circular imports involving absolute imports with binding a submodule to a name are now supported | |
+----------------------------------------------------------------------------------------------------------+----------------+
| object.__format__(x, '') is now equivalent to str(x) rather than format(str(self), '') | |
+----------------------------------------------------------------------------------------------------------+----------------+
| In order to better support dynamic creation of stack traces, types.TracebackType can now be instantiated | |
| from Python code, and the tb_next attribute on tracebacks is now writable | |
+----------------------------------------------------------------------------------------------------------+----------------+
| When using the -m switch, sys.path[0] is now eagerly expanded to the full starting directory path, rather| |
| than being left as the empty directory (which allows imports from the current working directory at the | |
| time when an import occurs) | |
+----------------------------------------------------------------------------------------------------------+----------------+
| The new -X importtime option or the PYTHONPROFILEIMPORTTIME environment variable can be used to show the | |
| timing of each module import | |
+----------------------------------------------------------------------------------------------------------+----------------+
Changes to built-in modules:
+------------------------------------------------------------------------------------------------------------+----------------+
| `asyncio <https://docs.python.org/3/whatsnew/3.7.html#asyncio>`_ | |
+------------------------------------------------------------------------------------------------------------+----------------+
| asyncio (many, may need a separate ticket) | |
+------------------------------------------------------------------------------------------------------------+----------------+
| `gc <https://docs.python.org/3/whatsnew/3.7.html#gc>`_ | |
+------------------------------------------------------------------------------------------------------------+----------------+
| New features include *gc.freeze()*, *gc.unfreeze()*, *gc-get_freeze_count* | |
+------------------------------------------------------------------------------------------------------------+----------------+
| `math <https://docs.python.org/3/whatsnew/3.7.html#math>`_ | |
+------------------------------------------------------------------------------------------------------------+----------------+
| math.remainder() added to implement IEEE 754-style remainder | |
+------------------------------------------------------------------------------------------------------------+----------------+
| `re <https://docs.python.org/3/whatsnew/3.7.html#re>`_ | |
+------------------------------------------------------------------------------------------------------------+----------------+
| A number of tidy up features including better support for splitting on empty strings and copy support for | |
| compiled expressions and match objects | |
+------------------------------------------------------------------------------------------------------------+----------------+
| `sys <https://docs.python.org/3/whatsnew/3.7.html#sys>`_ | |
+------------------------------------------------------------------------------------------------------------+----------------+
| sys.breakpointhook() added. sys.get(/set)_coroutine_origin_tracking_depth() added | |
+------------------------------------------------------------------------------------------------------------+----------------+
| `time <https://docs.python.org/3/whatsnew/3.7.html#time>`_ | |
+------------------------------------------------------------------------------------------------------------+----------------+
| Mostly updates to support nanosecond resolution in PEP564, see above | |
+------------------------------------------------------------------------------------------------------------+----------------+

118
docs/differences/python_38.rst Normal file
View File

@ -0,0 +1,118 @@
.. _python_38:
Python 3.8
==========
Python 3.8.0 (final) was released on the 14 October 2019. The Features for 3.8
are defined in `PEP 569 <https://www.python.org/dev/peps/pep-0569/#id9>`_ and
a detailed description of the changes can be found in What's New in `Python
3.8. <https://docs.python.org/3/whatsnew/3.8.html>`_
+--------------------------------------------------------+---------------------------------------------------+---------------+
| **Features:** | Status |
+--------------------------------------------------------+---------------------------------------------------+---------------+
| `PEP 570 <https://www.python.org/dev/peps/pep-0570/>`_ | Positional-only arguments | |
+--------------------------------------------------------+---------------------------------------------------+---------------+
| `PEP 572 <https://www.python.org/dev/peps/pep-0572/>`_ | Assignment Expressions | |
+--------------------------------------------------------+---------------------------------------------------+---------------+
| `PEP 574 <https://www.python.org/dev/peps/pep-0574/>`_ | Pickle protocol 5 with out-of-band data | |
+--------------------------------------------------------+---------------------------------------------------+---------------+
| `PEP 578 <https://www.python.org/dev/peps/pep-0578/>`_ | Runtime audit hooks | |
+--------------------------------------------------------+---------------------------------------------------+---------------+
| `PEP 587 <https://www.python.org/dev/peps/pep-0587/>`_ | Python Initialization Configuration | |
+--------------------------------------------------------+---------------------------------------------------+---------------+
| `PEP 590 <https://www.python.org/dev/peps/pep-0590/>`_ | Vectorcall: a fast calling protocol for CPython | |
+--------------------------------------------------------+---------------------------------------------------+---------------+
| **Miscellaneous** |
+------------------------------------------------------------------------------------------------------------+---------------+
| f-strings support = for self-documenting expressions and debugging | Completed |
+------------------------------------------------------------------------------------------------------------+---------------+
Other Language Changes:
+------------------------------------------------------------------------------------------------------------+-------------+
| A *continue* statement was illegal in the *finally* clause due to a problem with the implementation. In | Completed |
| Python 3.8 this restriction was lifted | |
+------------------------------------------------------------------------------------------------------------+-------------+
| The *bool*, *int* , and *fractions.Fraction* types now have an *as_integer_ratio()* method like that found | |
| in *float* and *decimal.Decimal* | |
+------------------------------------------------------------------------------------------------------------+-------------+
| Constructors of *int*, *float* and *complex* will now use the *__index__()* special method, if available | |
| and the corresponding method *__int__()*, *__float__()* or *__complex__()* is not available | |
+------------------------------------------------------------------------------------------------------------+-------------+
| Added support of *\N{name}* escapes in regular expressions | |
+------------------------------------------------------------------------------------------------------------+-------------+
| Dict and dictviews are now iterable in reversed insertion order using *reversed()* | |
+------------------------------------------------------------------------------------------------------------+-------------+
| The syntax allowed for keyword names in function calls was further restricted. In particular, | |
| f((keyword)=arg) is no longer allowed | |
+------------------------------------------------------------------------------------------------------------+-------------+
| Generalized iterable unpacking in yield and return statements no longer requires enclosing parentheses | |
+------------------------------------------------------------------------------------------------------------+-------------+
| When a comma is missed in code such as [(10, 20) (30, 40)], the compiler displays a SyntaxWarning with a | |
| helpful suggestion | |
+------------------------------------------------------------------------------------------------------------+-------------+
| Arithmetic operations between subclasses of *datetime.date* or *datetime.datetime* and *datetime.timedelta*| |
| objects now return an instance of the subclass, rather than the base class | |
+------------------------------------------------------------------------------------------------------------+-------------+
| When the Python interpreter is interrupted by *Ctrl-C (SIGINT)* and the resulting *KeyboardInterrupt* | |
| exception is not caught, the Python process now exits via a SIGINT signal or with the correct exit code | |
| such that the calling process can detect that it died due to a *Ctrl-C* | |
+------------------------------------------------------------------------------------------------------------+-------------+
| Some advanced styles of programming require updating the *types.CodeType* object for an existing function | |
+------------------------------------------------------------------------------------------------------------+-------------+
| For integers, the three-argument form of the pow() function now permits the exponent to be negative in the | |
| case where the base is relatively prime to the modulus | |
+------------------------------------------------------------------------------------------------------------+-------------+
| Dict comprehensions have been synced-up with dict literals so that the key is computed first and the value | |
| second | |
+------------------------------------------------------------------------------------------------------------+-------------+
| The *object.__reduce__()* method can now return a tuple from two to six elements long | |
+------------------------------------------------------------------------------------------------------------+-------------+
Changes to built-in modules:
+------------------------------------------------------------------------------------------------------------+-------------+
| `asyncio` |
+------------------------------------------------------------------------------------------------------------+-------------+
| *asyncio.run()* has graduated from the provisional to stable API | Completed |
+------------------------------------------------------------------------------------------------------------+-------------+
| Running *python -m asyncio* launches a natively async REPL | |
+------------------------------------------------------------------------------------------------------------+-------------+
| The exception *asyncio.CancelledError* now inherits from *BaseException* rather than *Exception* and no | Completed |
| longer inherits from *concurrent.futures.CancelledError* | |
+------------------------------------------------------------------------------------------------------------+-------------+
| Added *asyncio.Task.get_coro()* for getting the wrapped coroutine within an *asyncio.Task* | |
+------------------------------------------------------------------------------------------------------------+-------------+
| Asyncio tasks can now be named, either by passing the name keyword argument to *asyncio.create_task()* or | |
| the *create_task()* event loop method, or by calling the *set_name()* method on the task object | |
+------------------------------------------------------------------------------------------------------------+-------------+
| Added support for Happy Eyeballs to *asyncio.loop.create_connection()*. To specify the behavior, two new | |
| parameters have been added: *happy_eyeballs_delay* and interleave. | |
+------------------------------------------------------------------------------------------------------------+-------------+
| `gc` |
+------------------------------------------------------------------------------------------------------------+-------------+
| *get_objects()* can now receive an optional generation parameter indicating a generation to get objects | |
| from. (Note, though, that while *gc* is a built-in, *get_objects()* is not implemented for MicroPython) | |
+------------------------------------------------------------------------------------------------------------+-------------+
| `math` |
+------------------------------------------------------------------------------------------------------------+-------------+
| Added new function *math.dist()* for computing Euclidean distance between two points | |
+------------------------------------------------------------------------------------------------------------+-------------+
| Expanded the *math.hypot()* function to handle multiple dimensions | |
+------------------------------------------------------------------------------------------------------------+-------------+
| Added new function, *math.prod()*, as analogous function to *sum()* that returns the product of a "start" | |
| value (default: 1) times an iterable of numbers | |
+------------------------------------------------------------------------------------------------------------+-------------+
| Added two new combinatoric functions *math.perm()* and *math.comb()* | |
+------------------------------------------------------------------------------------------------------------+-------------+
| Added a new function *math.isqrt()* for computing accurate integer square roots without conversion to | |
| floating point | |
+------------------------------------------------------------------------------------------------------------+-------------+
| The function *math.factorial()* no longer accepts arguments that are not int-like | Completed |
+------------------------------------------------------------------------------------------------------------+-------------+
| `sys` |
+------------------------------------------------------------------------------------------------------------+-------------+
| Add new *sys.unraisablehook()* function which can be overridden to control how "unraisable exceptions" | |
| are handled | |
+------------------------------------------------------------------------------------------------------------+-------------+

121
docs/differences/python_39.rst Normal file
View File

@ -0,0 +1,121 @@
.. _python_39:
Python 3.9
==========
Python 3.9.0 (final) was released on the 5th October 2020. The Features for 3.9 are
defined in `PEP 596 <https://www.python.org/dev/peps/pep-0596/#features-for-3-9>`_
and a detailed description of the changes can be found in
`What's New in Python 3.9 <https://docs.python.org/3/whatsnew/3.9.html>`_
+--------------------------------------------------------+----------------------------------------------------+--------------+
| **Features:** | | **Status** |
+--------------------------------------------------------+----------------------------------------------------+--------------+
| `PEP 573 <https://www.python.org/dev/peps/pep-0573/>`_ | fast access to module state from methods of C | |
| | extension types | |
+--------------------------------------------------------+----------------------------------------------------+--------------+
| `PEP 584 <https://www.python.org/dev/peps/pep-0584/>`_ | union operators added to dict | |
+--------------------------------------------------------+----------------------------------------------------+--------------+
| `PEP 585 <https://www.python.org/dev/peps/pep-0584/>`_ | type hinting generics in standard collections | |
+--------------------------------------------------------+----------------------------------------------------+--------------+
| `PEP 593 <https://www.python.org/dev/peps/pep-0593/>`_ | flexible function and variable annotations | |
+--------------------------------------------------------+----------------------------------------------------+--------------+
| `PEP 602 <https://www.python.org/dev/peps/pep-0602/>`_ | CPython adopts an annual release cycle. Instead of | |
| | annual, aiming for two month release cycle | |
+--------------------------------------------------------+----------------------------------------------------+--------------+
| `PEP 614 <https://www.python.org/dev/peps/pep-0614/>`_ | relaxed grammar restrictions on decorators | |
+--------------------------------------------------------+----------------------------------------------------+--------------+
| `PEP 615 <https://www.python.org/dev/peps/pep-0615/>`_ | the IANA Time Zone Database is now present in the | |
| | standard library in the zoneinfo module | |
+--------------------------------------------------------+----------------------------------------------------+--------------+
| `PEP 616 <https://www.python.org/dev/peps/pep-0616/>`_ | string methods to remove prefixes and suffixes | |
+--------------------------------------------------------+----------------------------------------------------+--------------+
| `PEP 617 <https://www.python.org/dev/peps/pep-0617/>`_ | CPython now uses a new parser based on PEG | |
+--------------------------------------------------------+----------------------------------------------------+--------------+
Other Language Changes:
+-------------------------------------------------------------------------------------------------------------+---------------+
| *__import__()* now raises *ImportError* instead of *ValueError* | Completed |
+-------------------------------------------------------------------------------------------------------------+---------------+
| Python now gets the absolute path of the script filename specified on the command line (ex: *python3* | |
| *script.py*): the *__file__* attribute of the *__main__* module became an absolute path, rather than a | |
| relative path | |
+-------------------------------------------------------------------------------------------------------------+---------------+
| By default, for best performance, the errors argument is only checked at the first encoding/decoding error | |
| and the encoding argument is sometimes ignored for empty strings | |
+-------------------------------------------------------------------------------------------------------------+---------------+
| *"".replace("", s, n)* now returns *s* instead of an empty string for all non-zero n. It is now consistent | |
| with *"".replace("", s)* | |
+-------------------------------------------------------------------------------------------------------------+---------------+
| Any valid expression can now be used as a decorator. Previously, the grammar was much more restrictive | |
+-------------------------------------------------------------------------------------------------------------+---------------+
| Parallel running of *aclose()* / *asend()* / *athrow()* is now prohibited, and *ag_running* now reflects | |
| the actual running status of the async generator | |
+-------------------------------------------------------------------------------------------------------------+---------------+
| Unexpected errors in calling the *__iter__* method are no longer masked by TypeError in the in operator and | |
| functions contains(), indexOf() and countOf() of the operator module | |
+-------------------------------------------------------------------------------------------------------------+---------------+
| Unparenthesized lambda expressions can no longer be the expression part in an if clause in comprehensions | |
| and generator expressions | |
+-------------------------------------------------------------------------------------------------------------+---------------+
Changes to built-in modules:
+---------------------------------------------------------------------------------------------------------------+---------------+
| `asyncio` |
+---------------------------------------------------------------------------------------------------------------+---------------+
| Due to significant security concerns, the reuse_address parameter of *asyncio.loop.create_datagram_endpoint()*| |
| is no longer supported | |
+---------------------------------------------------------------------------------------------------------------+---------------+
| Added a new coroutine *shutdown_default_executor()* that schedules a shutdown for the default executor that | |
| waits on the *ThreadPoolExecutor* to finish closing. Also, *asyncio.run()* has been updated to use the new | |
| coroutine. | |
+---------------------------------------------------------------------------------------------------------------+---------------+
| Added *asyncio.PidfdChildWatcher*, a Linux-specific child watcher implementation that polls process file | |
| descriptors | |
+---------------------------------------------------------------------------------------------------------------+---------------+
| added a new *coroutine asyncio.to_thread()* | |
+---------------------------------------------------------------------------------------------------------------+---------------+
| When cancelling the task due to a timeout, *asyncio.wait_for()* will now wait until the cancellation is | |
| complete also in the case when timeout is <= 0, like it does with positive timeouts | |
+---------------------------------------------------------------------------------------------------------------+---------------+
| *asyncio* now raises *TyperError* when calling incompatible methods with an *ssl.SSLSocket* socket | |
+---------------------------------------------------------------------------------------------------------------+---------------+
| `gc` |
+---------------------------------------------------------------------------------------------------------------+---------------+
| Garbage collection does not block on resurrected objects | |
+---------------------------------------------------------------------------------------------------------------+---------------+
| Added a new function *gc.is_finalized()* to check if an object has been finalized by the garbage collector | |
+---------------------------------------------------------------------------------------------------------------+---------------+
| `math` |
+---------------------------------------------------------------------------------------------------------------+---------------+
| Expanded the *math.gcd()* function to handle multiple arguments. Formerly, it only supported two arguments | |
+---------------------------------------------------------------------------------------------------------------+---------------+
| Added *math.lcm()*: return the least common multiple of specified arguments | |
+---------------------------------------------------------------------------------------------------------------+---------------+
| Added *math.nextafter()*: return the next floating-point value after x towards y | |
+---------------------------------------------------------------------------------------------------------------+---------------+
| Added *math.ulp()*: return the value of the least significant bit of a float | |
+---------------------------------------------------------------------------------------------------------------+---------------+
| `os` |
+---------------------------------------------------------------------------------------------------------------+---------------+
| Exposed the Linux-specific *os.pidfd_open()* and *os.P_PIDFD* | |
+---------------------------------------------------------------------------------------------------------------+---------------+
| The *os.unsetenv()* function is now also available on Windows | Completed |
+---------------------------------------------------------------------------------------------------------------+---------------+
| The *os.putenv()* and *os.unsetenv()* functions are now always available | Completed |
+---------------------------------------------------------------------------------------------------------------+---------------+
| Added *os.waitstatus_to_exitcode()* function: convert a wait status to an exit code | |
+---------------------------------------------------------------------------------------------------------------+---------------+
| `random` |
+---------------------------------------------------------------------------------------------------------------+---------------+
| Added a new *random.Random.randbytes* method: generate random bytes | |
+---------------------------------------------------------------------------------------------------------------+---------------+
| `sys` |
+---------------------------------------------------------------------------------------------------------------+---------------+
| Added a new *sys.platlibdir* attribute: name of the platform-specific library directory | |
+---------------------------------------------------------------------------------------------------------------+---------------+
| Previously, *sys.stderr* was block-buffered when non-interactive. Now stderr defaults to always being | |
| line-buffered | |
+---------------------------------------------------------------------------------------------------------------+---------------+

4
docs/library/framebuf.rst
View File

@ -21,11 +21,11 @@ For example::
import framebuf import framebuf
# FrameBuffer needs 2 bytes for every RGB565 pixel # FrameBuffer needs 2 bytes for every RGB565 pixel
fbuf = framebuf.FrameBuffer(bytearray(10 * 100 * 2), 10, 100, framebuf.RGB565) fbuf = framebuf.FrameBuffer(bytearray(100 * 10 * 2), 100, 10, framebuf.RGB565)
fbuf.fill(0) fbuf.fill(0)
fbuf.text('MicroPython!', 0, 0, 0xffff) fbuf.text('MicroPython!', 0, 0, 0xffff)
fbuf.hline(0, 10, 96, 0xffff) fbuf.hline(0, 9, 96, 0xffff)
Constructors Constructors
------------ ------------

82
docs/library/random.rst Normal file
View File

@ -0,0 +1,82 @@
:mod:`random` -- generate random numbers
========================================
.. module:: random
:synopsis: random numbers
This module implements a pseudo-random number generator (PRNG).
|see_cpython_module| :mod:`python:random` .
.. note::
The following notation is used for intervals:
- () are open interval brackets and do not include their endpoints.
For example, (0, 1) means greater than 0 and less than 1.
In set notation: (0, 1) = {x | 0 < x < 1}.
- [] are closed interval brackets which include all their limit points.
For example, [0, 1] means greater than or equal to 0 and less than
or equal to 1.
In set notation: [0, 1] = {x | 0 <= x <= 1}.
.. note::
The :func:`randrange`, :func:`randint` and :func:`choice` functions are only
available if the ``MICROPY_PY_URANDOM_EXTRA_FUNCS`` configuration option is
enabled.
Functions for integers
----------------------
.. function:: getrandbits(n)
Return an integer with *n* random bits (0 <= n <= 32).
.. function:: randint(a, b)
Return a random integer in the range [*a*, *b*].
.. function:: randrange(stop)
randrange(start, stop)
randrange(start, stop[, step])
The first form returns a random integer from the range [0, *stop*).
The second form returns a random integer from the range [*start*, *stop*).
The third form returns a random integer from the range [*start*, *stop*) in
steps of *step*. For instance, calling ``randrange(1, 10, 2)`` will
return odd numbers between 1 and 9 inclusive.
Functions for floats
--------------------
.. function:: random()
Return a random floating point number in the range [0.0, 1.0).
.. function:: uniform(a, b)
Return a random floating point number N such that *a* <= N <= *b* for *a* <= *b*,
and *b* <= N <= *a* for *b* < *a*.
Other Functions
---------------
.. function:: seed(n=None, /)
Initialise the random number generator module with the seed *n* which should
be an integer. When no argument (or ``None``) is passed in it will (if
supported by the port) initialise the PRNG with a true random number
(usually a hardware generated random number).
The ``None`` case only works if ``MICROPY_PY_URANDOM_SEED_INIT_FUNC`` is
enabled by the port, otherwise it raises ``ValueError``.
.. function:: choice(sequence)
Chooses and returns one item at random from *sequence* (tuple, list or
any object that supports the subscript operation).

104
docs/library/stm.rst Normal file
View File

@ -0,0 +1,104 @@
.. currentmodule:: stm
:mod:`stm` --- functionality specific to STM32 MCUs
===================================================
.. module:: stm
:synopsis: functionality specific to STM32 MCUs
This module provides functionality specific to STM32 microcontrollers, including
direct access to peripheral registers.
Memory access
-------------
The module exposes three objects used for raw memory access.
.. data:: mem8
Read/write 8 bits of memory.
.. data:: mem16
Read/write 16 bits of memory.
.. data:: mem32
Read/write 32 bits of memory.
Use subscript notation ``[...]`` to index these objects with the address of
interest.
These memory objects can be used in combination with the peripheral register
constants to read and write registers of the MCU hardware peripherals, as well
as all other areas of address space.
Peripheral register constants
-----------------------------
The module defines constants for registers which are generated from CMSIS header
files, and the constants available depend on the microcontroller series that is
being compiled for. Examples of some constants include:
.. data:: GPIOA
Base address of the GPIOA peripheral.
.. data:: GPIOB
Base address of the GPIOB peripheral.
.. data:: GPIO_BSRR
Offset of the GPIO bit set/reset register.
.. data:: GPIO_IDR
Offset of the GPIO input data register.
.. data:: GPIO_ODR
Offset of the GPIO output data register.
Constants that are named after a peripheral, like ``GPIOA``, are the absolute
address of that peripheral. Constants that have a prefix which is the name of a
peripheral, like ``GPIO_BSRR``, are relative offsets of the register. Accessing
peripheral registers requires adding the absolute base address of the peripheral
and the relative register offset. For example ``GPIOA + GPIO_BSRR`` is the
full, absolute address of the ``GPIOA->BSRR`` register.
Example use:
.. code-block:: python3
# set PA2 high
stm.mem32[stm.GPIOA + stm.GPIO_BSRR] = 1 << 2
# read PA3
value = (stm.mem32[stm.GPIOA + stm.GPIO_IDR] >> 3) & 1
Functions specific to STM32WBxx MCUs
------------------------------------
These functions are available on STM32WBxx microcontrollers, and interact with
the second CPU, the RF core.
.. function:: rfcore_status()
Returns the status of the second CPU as an integer (the first word of device
info table).
.. function:: rfcore_fw_version(id)
Get the version of the firmware running on the second CPU. Pass in 0 for
*id* to get the FUS version, and 1 to get the WS version.
Returns a 5-tuple with the full version number.
.. function:: rfcore_sys_hci(ogf, ocf, data, timeout_ms=0)
Execute a HCI command on the SYS channel. The execution is synchronous.
Returns a bytes object with the result of the SYS command.

8
docs/library/sys.rst
View File

@ -80,6 +80,14 @@ Constants
A mutable list of directories to search for imported modules. A mutable list of directories to search for imported modules.
.. admonition:: Difference to CPython
:class: attention
On MicroPython, an entry with the value ``".frozen"`` will indicate that import
should search :term:`frozen modules <frozen module>` at that point in the search.
If no frozen module is found then search will *not* look for a directory called
``.frozen``, instead it will continue with the next entry in ``sys.path``.
.. data:: platform .. data:: platform
The platform that CircuitPython is running on. For OS/RTOS ports, this is The platform that CircuitPython is running on. For OS/RTOS ports, this is

175
docs/reference/glossary.rst
View File

@ -1,175 +0,0 @@
Glossary
========
.. glossary::
baremetal
A system without a (full-fledged) operating system, for example an
:term:`MCU`-based system. When running on a baremetal system,
MicroPython effectively functions like a small operating system,
running user programs and providing a command interpreter
(:term:`REPL`).
buffer protocol
Any Python object that can be automatically converted into bytes, such
as ``bytes``, ``bytearray``, ``memoryview`` and ``str`` objects, which
all implement the "buffer protocol".
board
Typically this refers to a printed circuit board (PCB) containing a
:term:`microcontroller <MCU>` and supporting components.
MicroPython firmware is typically provided per-board, as the firmware
contains both MCU-specific functionality but also board-level
functionality such as drivers or pin names.
bytecode
A compact representation of a Python program that generated by
compiling the Python source code. This is what the VM actually
executes. Bytecode is typically generated automatically at runtime and
is invisible to the user. Note that while :term:`CPython` and
MicroPython both use bytecode, the format is different. You can also
pre-compile source code offline using the :term:`cross-compiler`.
callee-owned tuple
This is a MicroPython-specific construct where, for efficiency
reasons, some built-in functions or methods may re-use the same
underlying tuple object to return data. This avoids having to allocate
a new tuple for every call, and reduces heap fragmentation. Programs
should not hold references to callee-owned tuples and instead only
extract data from them (or make a copy).
CircuitPython
A variant of MicroPython developed by `Adafruit Industries
<https://circuitpython.org>`_.
CPython
CPython is the reference implementation of the Python programming
language, and the most well-known one. It is, however, one of many
implementations (including Jython, IronPython, PyPy, and MicroPython).
While MicroPython's implementation differs substantially from CPython,
it aims to maintain as much compatibility as possible.
cross-compiler
Also known as ``mpy-cross``. This tool runs on your PC and converts a
:term:`.py file` containing MicroPython code into a :term:`.mpy file`
containing MicroPython bytecode. This means it loads faster (the board
doesn't have to compile the code), and uses less space on flash (the
bytecode is more space efficient).
driver
A MicroPython library that implements support for a particular
component, such as a sensor or display.
FFI
Acronym for Foreign Function Interface. A mechanism used by the
:term:`MicroPython Unix port` to access operating system functionality.
This is not available on :term:`baremetal` ports.
filesystem
Most MicroPython ports and boards provide a filesystem stored in flash
that is available to user code via the standard Python file APIs such
as ``open()``. Some boards also make this internal filesystem
accessible to the host via USB mass-storage.
frozen module
A Python module that has been cross compiled and bundled into the
firmware image. This reduces RAM requirements as the code is executed
directly from flash.
Garbage Collector
A background process that runs in Python (and MicroPython) to reclaim
unused memory in the :term:`heap`.
GPIO
General-purpose input/output. The simplest means to control electrical
signals (commonly referred to as "pins") on a microcontroller. GPIO
typically allows pins to be either input or output, and to set or get
their digital value (logical "0" or "1"). MicroPython abstracts GPIO
access using the :class:`machine.Pin` and :class:`machine.Signal`
classes.
GPIO port
A group of :term:`GPIO` pins, usually based on hardware properties of
these pins (e.g. controllable by the same register).
heap
A region of RAM where MicroPython stores dynamic data. It is managed
automatically by the :term:`Garbage Collector`. Different MCUs and
boards have vastly different amounts of RAM available for the heap, so
this will affect how complex your program can be.
interned string
An optimisation used by MicroPython to improve the efficiency of
working with strings. An interned string is referenced by its (unique)
identity rather than its address and can therefore be quickly compared
just by its identifier. It also means that identical strings can be
de-duplicated in memory. String interning is almost always invisible to
the user.
MCU
Microcontroller. Microcontrollers usually have much less resources
than a desktop, laptop, or phone, but are smaller, cheaper and
require much less power. MicroPython is designed to be small and
optimized enough to run on an average modern microcontroller.
MicroPython port
MicroPython supports different :term:`boards <board>`, RTOSes, and
OSes, and can be relatively easily adapted to new systems. MicroPython
with support for a particular system is called a "port" to that
system. Different ports may have widely different functionality. This
documentation is intended to be a reference of the generic APIs
available across different ports ("MicroPython core"). Note that some
ports may still omit some APIs described here (e.g. due to resource
constraints). Any such differences, and port-specific extensions
beyond the MicroPython core functionality, would be described in the
separate port-specific documentation.
MicroPython Unix port
The unix port is one of the major :term:`MicroPython ports
<MicroPython port>`. It is intended to run on POSIX-compatible
operating systems, like Linux, MacOS, FreeBSD, Solaris, etc. It also
serves as the basis of Windows port. The Unix port is very useful for
quick development and testing of the MicroPython language and
machine-independent features. It can also function in a similar way to
:term:`CPython`'s ``python`` executable.
.mpy file
The output of the :term:`cross-compiler`. A compiled form of a
:term:`.py file` that contains MicroPython bytecode instead of Python
source code.
native
Usually refers to "native code", i.e. machine code for the target
microcontroller (such as ARM Thumb, Xtensa, x86/x64). The ``@native``
decorator can be applied to a MicroPython function to generate native
code instead of bytecode for that function, which will likely be
faster but use more RAM.
port
Usually short for :term:`MicroPython port`, but could also refer to
:term:`GPIO port`.
.py file
A file containing Python source code.
REPL
An acronym for "Read, Eval, Print, Loop". This is the interactive
Python prompt, useful for debugging or testing short snippets of code.
Most MicroPython boards make a REPL available over a UART, and this is
typically accessible on a host PC via USB.
stream
Also known as a "file-like object". An Python object which provides
sequential read-write access to the underlying data. A stream object
implements a corresponding interface, which consists of methods like
``read()``, ``write()``, ``readinto()``, ``seek()``, ``flush()``,
``close()``, etc. A stream is an important concept in MicroPython;
many I/O objects implement the stream interface, and thus can be used
consistently and interchangeably in different contexts. For more
information on streams in MicroPython, see the `io` module.
UART
Acronym for "Universal Asynchronous Receiver/Transmitter". This is a
peripheral that sends data over a pair of pins (TX & RX). Many boards
include a way to make at least one of the UARTs available to a host PC
as a serial port over USB.

123
docs/rp2/tutorial/pio.rst Normal file
View File

@ -0,0 +1,123 @@
Programmable IO
===============
The RP2040 has hardware support for standard communication protocols like I2C,
SPI and UART. For protocols where there is no hardware support, or where there
is a requirement of custom I/O behaviour, Programmable Input Output (PIO) comes
into play. Also, some MicroPython applications make use of a technique called
bit banging in which pins are rapidly turned on and off to transmit data. This
can make the entire process slow as the processor concentrates on bit banging
rather than executing other logic. However, PIO allows bit banging to happen
in the background while the CPU is executing the main work.
Along with the two central Cortex-M0+ processing cores, the RP2040 has two PIO
blocks each of which has four independent state machines. These state machines
can transfer data to/from other entities using First-In-First-Out (FIFO) buffers,
which allow the state machine and main processor to work independently yet also
synchronise their data. Each FIFO has four words (each of 32 bits) which can be
linked to the DMA to transfer larger amounts of data.
All PIO instructions follow a common pattern::
<instruction> .side(<side_set_value>) [<delay_value>]
The side-set ``.side(...)`` and delay ``[...]`` parts are both optional, and if
specified allow the instruction to perform more than one operation. This keeps
PIO programs small and efficient.
There are nine instructions which perform the following tasks:
- ``jmp()`` transfers control to a different part of the code
- ``wait()`` pauses until a particular action happens
- ``in_()`` shifts the bits from a source (scratch register or set of pins) to the
input shift register
- ``out()`` shifts the bits from the output shift register to a destination
- ``push()`` sends data to the RX FIFO
- ``pull()`` receives data from the TX FIFO
- ``mov()`` moves data from a source to a destination
- ``irq()`` sets or clears an IRQ flag
- ``set()`` writes a literal value to a destination
The instruction modifiers are:
- ``.side()`` sets the side-set pins at the start of the instruction
- ``[]`` delays for a certain number of cycles after execution of the instruction
There are also directives:
- ``wrap_target()`` specifies where the program execution will get continued from
- ``wrap()`` specifies the instruction where the control flow of the program will
get wrapped from
- ``label()`` sets a label for use with ``jmp()`` instructions
- ``word()`` emits a raw 16-bit value which acts as an instruction in the program
An example
----------
Take the ``pio_1hz.py`` example for a simple understanding of how to use the PIO
and state machines. Below is the code for reference.
.. code-block:: python3
# Example using PIO to blink an LED and raise an IRQ at 1Hz.
import time
from machine import Pin
import rp2
@rp2.asm_pio(set_init=rp2.PIO.OUT_LOW)
def blink_1hz():
# Cycles: 1 + 1 + 6 + 32 * (30 + 1) = 1000
irq(rel(0))
set(pins, 1)
set(x, 31) [5]
label("delay_high")
nop() [29]
jmp(x_dec, "delay_high")
# Cycles: 1 + 7 + 32 * (30 + 1) = 1000
set(pins, 0)
set(x, 31) [6]
label("delay_low")
nop() [29]
jmp(x_dec, "delay_low")
# Create the StateMachine with the blink_1hz program, outputting on Pin(25).
sm = rp2.StateMachine(0, blink_1hz, freq=2000, set_base=Pin(25))
# Set the IRQ handler to print the millisecond timestamp.
sm.irq(lambda p: print(time.ticks_ms()))
# Start the StateMachine.
sm.active(1)
This creates an instance of class :class:`rp2.StateMachine` which runs the
``blink_1hz`` program at 2000Hz, and connects to pin 25. The ``blink_1hz``
program uses the PIO to blink an LED connected to this pin at 1Hz, and also
raises an IRQ as the LED turns on. This IRQ then calls the ``lambda`` function
which prints out a millisecond timestamp.
The ``blink_1hz`` program is a PIO assembler routine. It connects to a single
pin which is configured as an output and starts out low. The instructions do
the following:
- ``irq(rel(0))`` raises the IRQ associated with the state machine.
- The LED is turned on via the ``set(pins, 1)`` instruction.
- The value 31 is put into register X, and then there is a delay for 5 more
cycles, specified by the ``[5]``.
- The ``nop() [29]`` instruction waits for 30 cycles.
- The ``jmp(x_dec, "delay_high")`` will keep looping to the ``delay_high`` label
as long as the register X is non-zero, and will also post-decrement X. Since
X starts with the value 31 this jump will happen 31 times, so the ``nop() [29]``
runs 32 times in total (note there is also one instruction cycle taken by the
``jmp`` for each of these 32 loops).
- ``set(pins, 0)`` will turn the LED off by setting pin 25 low.
- Another 32 loops of ``nop() [29]`` and ``jmp(...)`` will execute.
- Because ``wrap_target()`` and ``wrap()`` are not specified, their default will
be used and execution of the program will wrap around from the bottom to the
top. This wrapping does not cost any execution cycles.
The entire routine takes exactly 2000 cycles of the state machine. Setting the
frequency of the state machine to 2000Hz makes the LED blink at 1Hz.

4
extmod/extmod.cmake
View File

@ -10,11 +10,13 @@ set(MICROPY_SOURCE_EXTMOD
${MICROPY_EXTMOD_DIR}/machine_i2c.c ${MICROPY_EXTMOD_DIR}/machine_i2c.c
${MICROPY_EXTMOD_DIR}/machine_mem.c ${MICROPY_EXTMOD_DIR}/machine_mem.c
${MICROPY_EXTMOD_DIR}/machine_pulse.c ${MICROPY_EXTMOD_DIR}/machine_pulse.c
${MICROPY_EXTMOD_DIR}/machine_pwm.c
${MICROPY_EXTMOD_DIR}/machine_signal.c ${MICROPY_EXTMOD_DIR}/machine_signal.c
${MICROPY_EXTMOD_DIR}/machine_spi.c ${MICROPY_EXTMOD_DIR}/machine_spi.c
${MICROPY_EXTMOD_DIR}/modbluetooth.c ${MICROPY_EXTMOD_DIR}/modbluetooth.c
${MICROPY_EXTMOD_DIR}/modbtree.c ${MICROPY_EXTMOD_DIR}/modbtree.c
${MICROPY_EXTMOD_DIR}/modframebuf.c ${MICROPY_EXTMOD_DIR}/modframebuf.c
${MICROPY_EXTMOD_DIR}/modnetwork.c
${MICROPY_EXTMOD_DIR}/modonewire.c ${MICROPY_EXTMOD_DIR}/modonewire.c
${MICROPY_EXTMOD_DIR}/moduasyncio.c ${MICROPY_EXTMOD_DIR}/moduasyncio.c
${MICROPY_EXTMOD_DIR}/modubinascii.c ${MICROPY_EXTMOD_DIR}/modubinascii.c
@ -23,9 +25,11 @@ set(MICROPY_SOURCE_EXTMOD
${MICROPY_EXTMOD_DIR}/moduhashlib.c ${MICROPY_EXTMOD_DIR}/moduhashlib.c
${MICROPY_EXTMOD_DIR}/moduheapq.c ${MICROPY_EXTMOD_DIR}/moduheapq.c
${MICROPY_EXTMOD_DIR}/modujson.c ${MICROPY_EXTMOD_DIR}/modujson.c
${MICROPY_EXTMOD_DIR}/moduplatform.c
${MICROPY_EXTMOD_DIR}/modurandom.c ${MICROPY_EXTMOD_DIR}/modurandom.c
${MICROPY_EXTMOD_DIR}/modure.c ${MICROPY_EXTMOD_DIR}/modure.c
${MICROPY_EXTMOD_DIR}/moduselect.c ${MICROPY_EXTMOD_DIR}/moduselect.c
${MICROPY_EXTMOD_DIR}/modusocket.c
${MICROPY_EXTMOD_DIR}/modussl_axtls.c ${MICROPY_EXTMOD_DIR}/modussl_axtls.c
${MICROPY_EXTMOD_DIR}/modussl_mbedtls.c ${MICROPY_EXTMOD_DIR}/modussl_mbedtls.c
${MICROPY_EXTMOD_DIR}/modutimeq.c ${MICROPY_EXTMOD_DIR}/modutimeq.c

4
extmod/modonewire.c
View File

@ -9,6 +9,8 @@
#include "py/obj.h" #include "py/obj.h"
#include "py/mphal.h" #include "py/mphal.h"
#if MICROPY_PY_ONEWIRE
/******************************************************************************/ /******************************************************************************/
// Low-level 1-Wire routines // Low-level 1-Wire routines
@ -139,3 +141,5 @@ const mp_obj_module_t mp_module_onewire = {
.base = { &mp_type_module }, .base = { &mp_type_module },
.globals = (mp_obj_dict_t *)&onewire_module_globals, .globals = (mp_obj_dict_t *)&onewire_module_globals,
}; };
#endif // MICROPY_PY_ONEWIRE

146
extmod/moduplatform.c Normal file
View File

@ -0,0 +1,146 @@
/*
* This file is part of the MicroPython project, http://micropython.org/
*
* The MIT License (MIT)
*
* Copyright (c) 2013-2021 Ibrahim Abdelkader <iabdalkader@openmv.io>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*
*/
#include "py/runtime.h"
#include "py/objtuple.h"
#include "py/objstr.h"
#include "py/mphal.h"
#include "genhdr/mpversion.h"
#if MICROPY_PY_UPLATFORM
// platform - Access to underlying platform's identifying data
// TODO: Add more architectures, compilers and libraries.
// See: https://sourceforge.net/p/predef/wiki/Home/
#if defined(__ARM_ARCH)
#define PLATFORM_ARCH "arm"
#elif defined(__x86_64__) || defined(_WIN64)
#define PLATFORM_ARCH "x86_64"
#elif defined(__i386__) || defined(_M_IX86)
#define PLATFORM_ARCH "x86"
#elif defined(__xtensa__) || defined(_M_IX86)
#define PLATFORM_ARCH "xtensa"
#else
#define PLATFORM_ARCH ""
#endif
#if defined(__GNUC__)
#define PLATFORM_COMPILER \
"GCC " \
MP_STRINGIFY(__GNUC__) "." \
MP_STRINGIFY(__GNUC_MINOR__) "." \
MP_STRINGIFY(__GNUC_PATCHLEVEL__)
#elif defined(__ARMCC_VERSION)
#define PLATFORM_COMPILER \
"ARMCC " \
MP_STRINGIFY((__ARMCC_VERSION / 1000000)) "." \
MP_STRINGIFY((__ARMCC_VERSION / 10000 % 100)) "." \
MP_STRINGIFY((__ARMCC_VERSION % 10000))
#elif defined(_MSC_VER)
#if defined(_WIN64)
#define COMPILER_BITS "64 bit"
#elif defined(_M_IX86)
#define COMPILER_BITS "32 bit"
#else
#define COMPILER_BITS ""
#endif
#define PLATFORM_COMPILER \
"MSC v." MP_STRINGIFY(_MSC_VER) " " COMPILER_BITS
#else
#define PLATFORM_COMPILER ""
#endif
#if defined(__GLIBC__)
#define PLATFORM_LIBC_LIB "glibc"
#define PLATFORM_LIBC_VER \
MP_STRINGIFY(__GLIBC__) "." \
MP_STRINGIFY(__GLIBC_MINOR__)
#elif defined(__NEWLIB__)
#define PLATFORM_LIBC_LIB "newlib"
#define PLATFORM_LIBC_VER _NEWLIB_VERSION
#else
#define PLATFORM_LIBC_LIB ""
#define PLATFORM_LIBC_VER ""
#endif
#if defined(__linux)
#define PLATFORM_SYSTEM "Linux"
#elif defined(__unix__)
#define PLATFORM_SYSTEM "Unix"
#elif defined(__CYGWIN__)
#define PLATFORM_SYSTEM "Cygwin"
#elif defined(_WIN32)
#define PLATFORM_SYSTEM "Windows"
#else
#define PLATFORM_SYSTEM "MicroPython"
#endif
#ifndef MICROPY_PLATFORM_VERSION
#define MICROPY_PLATFORM_VERSION ""
#endif
STATIC const MP_DEFINE_STR_OBJ(info_platform_obj, PLATFORM_SYSTEM "-" MICROPY_VERSION_STRING "-" \
PLATFORM_ARCH "-" MICROPY_PLATFORM_VERSION "-with-" PLATFORM_LIBC_LIB "" PLATFORM_LIBC_VER);
STATIC const MP_DEFINE_STR_OBJ(info_python_compiler_obj, PLATFORM_COMPILER);
STATIC const MP_DEFINE_STR_OBJ(info_libc_lib_obj, PLATFORM_LIBC_LIB);
STATIC const MP_DEFINE_STR_OBJ(info_libc_ver_obj, PLATFORM_LIBC_VER);
STATIC const mp_rom_obj_tuple_t info_libc_tuple_obj = {
{&mp_type_tuple}, 2, {MP_ROM_PTR(&info_libc_lib_obj), MP_ROM_PTR(&info_libc_ver_obj)}
};
STATIC mp_obj_t platform_platform(size_t n_args, const mp_obj_t *pos_args, mp_map_t *kw_args) {
return MP_OBJ_FROM_PTR(&info_platform_obj);
}
STATIC MP_DEFINE_CONST_FUN_OBJ_KW(platform_platform_obj, 0, platform_platform);
STATIC mp_obj_t platform_python_compiler(void) {
return MP_OBJ_FROM_PTR(&info_python_compiler_obj);
}
STATIC MP_DEFINE_CONST_FUN_OBJ_0(platform_python_compiler_obj, platform_python_compiler);
STATIC mp_obj_t platform_libc_ver(size_t n_args, const mp_obj_t *pos_args, mp_map_t *kw_args) {
return MP_OBJ_FROM_PTR(&info_libc_tuple_obj);
}
STATIC MP_DEFINE_CONST_FUN_OBJ_KW(platform_libc_ver_obj, 0, platform_libc_ver);
STATIC const mp_rom_map_elem_t modplatform_globals_table[] = {
{ MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_uplatform) },
{ MP_ROM_QSTR(MP_QSTR_platform), MP_ROM_PTR(&platform_platform_obj) },
{ MP_ROM_QSTR(MP_QSTR_python_compiler), MP_ROM_PTR(&platform_python_compiler_obj) },
{ MP_ROM_QSTR(MP_QSTR_libc_ver), MP_ROM_PTR(&platform_libc_ver_obj) },
};
STATIC MP_DEFINE_CONST_DICT(modplatform_globals, modplatform_globals_table);
const mp_obj_module_t mp_module_uplatform = {
.base = { &mp_type_module },
.globals = (mp_obj_dict_t *)&modplatform_globals,
};
#endif // MICROPY_PY_UPLATFORM

11
extmod/modure.c
View File

@ -477,11 +477,16 @@ MP_REGISTER_MODULE(MP_QSTR_re, mp_module_ure, MICROPY_PY_URE);
// only if module is enabled by config setting. // only if module is enabled by config setting.
#define re1_5_fatal(x) assert(!x) #define re1_5_fatal(x) assert(!x)
#include "lib/re1.5/compilecode.c" #include "lib/re1.5/compilecode.c"
#if MICROPY_PY_URE_DEBUG
#include "lib/re1.5/dumpcode.c"
#endif
#include "lib/re1.5/recursiveloop.c" #include "lib/re1.5/recursiveloop.c"
#include "lib/re1.5/charclass.c" #include "lib/re1.5/charclass.c"
#if MICROPY_PY_URE_DEBUG
// Make sure the output print statements go to the same output as other Python output.
#define printf(...) mp_printf(&mp_plat_print, __VA_ARGS__)
#include "lib/re1.5/dumpcode.c"
#undef printf
#endif
#endif // MICROPY_PY_URE #endif // MICROPY_PY_URE

2
extmod/uasyncio/core.py
View File

@ -36,7 +36,7 @@ class SingletonGenerator:
self.state = None self.state = None
self.exc = StopIteration() self.exc = StopIteration()
def __iter__(self): def __await__(self):
return self return self
def __next__(self): def __next__(self):

2
extmod/uasyncio/funcs.py
View File

@ -66,7 +66,7 @@ async def gather(*aws, return_exceptions=False):
# # cancel all waiting tasks # # cancel all waiting tasks
# raise er # raise er
ts[i] = await ts[i] ts[i] = await ts[i]
except Exception as er: except (core.CancelledError, Exception) as er:
if return_exceptions: if return_exceptions:
ts[i] = er ts[i] = er
else: else:

2
extmod/uasyncio/task.py
View File

@ -130,7 +130,7 @@ class Task:
self.ph_next = None # Paring heap self.ph_next = None # Paring heap
self.ph_rightmost_parent = None # Paring heap self.ph_rightmost_parent = None # Paring heap
def __iter__(self): def __await__(self):
if not self.state: if not self.state:
# Task finished, signal that is has been await'ed on. # Task finished, signal that is has been await'ed on.
self.state = False self.state = False

28
extmod/vfs_posix_file.c
View File

@ -17,6 +17,8 @@
#ifdef _WIN32 #ifdef _WIN32
#define fsync _commit #define fsync _commit
#else
#include <poll.h>
#endif #endif
typedef struct _mp_obj_vfs_posix_file_t { typedef struct _mp_obj_vfs_posix_file_t {
@ -180,6 +182,32 @@ STATIC mp_uint_t vfs_posix_file_ioctl(mp_obj_t o_in, mp_uint_t request, uintptr_
return 0; return 0;
case MP_STREAM_GET_FILENO: case MP_STREAM_GET_FILENO:
return o->fd; return o->fd;
#if MICROPY_PY_USELECT
case MP_STREAM_POLL: {
#ifdef _WIN32
mp_raise_NotImplementedError(MP_ERROR_TEXT("poll on file not available on win32"));
#else
mp_uint_t ret = 0;
uint8_t pollevents = 0;
if (arg & MP_STREAM_POLL_RD) {
pollevents |= POLLIN;
}
if (arg & MP_STREAM_POLL_WR) {
pollevents |= POLLOUT;
}
struct pollfd pfd = { .fd = o->fd, .events = pollevents };
if (poll(&pfd, 1, 0) > 0) {
if (pfd.revents & POLLIN) {
ret |= MP_STREAM_POLL_RD;
}
if (pfd.revents & POLLOUT) {
ret |= MP_STREAM_POLL_WR;
}
}
return ret;
#endif
}
#endif
default: default:
*errcode = EINVAL; *errcode = EINVAL;
return MP_STREAM_ERROR; return MP_STREAM_ERROR;

1
lib/pico-sdk Submodule

@ -0,0 +1 @@
Subproject commit fc10a97c386f65c1a44c68684fe52a56aaf50df0

1
lib/stm32lib Submodule

@ -0,0 +1 @@
Subproject commit 302c52794d2f579903f4e49cbad1f5d3a7f401ad

15
locale/circuitpython.pot
View File

@ -2874,6 +2874,10 @@ msgstr ""
msgid "can't load with '%q' index" msgid "can't load with '%q' index"
msgstr "" msgstr ""
#: py/builtinimport.c
msgid "can't perform relative import"
msgstr ""
#: py/objgenerator.c #: py/objgenerator.c
msgid "can't send non-None value to a just-started generator" msgid "can't send non-None value to a just-started generator"
msgstr "" msgstr ""
@ -2936,10 +2940,6 @@ msgstr ""
msgid "cannot import name %q" msgid "cannot import name %q"
msgstr "" msgstr ""
#: py/builtinimport.c
msgid "cannot perform relative import"
msgstr ""
#: extmod/moductypes.c #: extmod/moductypes.c
msgid "cannot unambiguously get sizeof scalar" msgid "cannot unambiguously get sizeof scalar"
msgstr "" msgstr ""
@ -3630,7 +3630,7 @@ msgstr ""
msgid "matrix is not positive definite" msgid "matrix is not positive definite"
msgstr "" msgstr ""
#: shared-bindings/wifi/Radio.c #: ports/espressif/common-hal/wifi/Radio.c
msgid "max_connections must be between 0 and 10" msgid "max_connections must be between 0 and 10"
msgstr "" msgstr ""
@ -4010,6 +4010,10 @@ msgstr ""
msgid "pixel_shader must be displayio.Palette or displayio.ColorConverter" msgid "pixel_shader must be displayio.Palette or displayio.ColorConverter"
msgstr "" msgstr ""
#: extmod/vfs_posix_file.c
msgid "poll on file not available on win32"
msgstr ""
#: shared-module/vectorio/Polygon.c #: shared-module/vectorio/Polygon.c
msgid "polygon can only be registered in one parent" msgid "polygon can only be registered in one parent"
msgstr "" msgstr ""
@ -4055,6 +4059,7 @@ msgstr ""
#: ports/espressif/boards/crumpspace_crumps2/mpconfigboard.h #: ports/espressif/boards/crumpspace_crumps2/mpconfigboard.h
#: ports/espressif/boards/electroniccats_bastwifi/mpconfigboard.h #: ports/espressif/boards/electroniccats_bastwifi/mpconfigboard.h
#: ports/espressif/boards/espressif_esp32c3_devkitm_1_n4/mpconfigboard.h #: ports/espressif/boards/espressif_esp32c3_devkitm_1_n4/mpconfigboard.h
#: ports/espressif/boards/espressif_esp32s2_devkitc_1_n4r2/mpconfigboard.h
#: ports/espressif/boards/espressif_esp32s3_box/mpconfigboard.h #: ports/espressif/boards/espressif_esp32s3_box/mpconfigboard.h
#: ports/espressif/boards/espressif_esp32s3_devkitc_1_n8/mpconfigboard.h #: ports/espressif/boards/espressif_esp32s3_devkitc_1_n8/mpconfigboard.h
#: ports/espressif/boards/espressif_esp32s3_devkitc_1_n8r2/mpconfigboard.h #: ports/espressif/boards/espressif_esp32s3_devkitc_1_n8r2/mpconfigboard.h

3
main.c
View File

@ -162,9 +162,6 @@ STATIC void start_mp(supervisor_allocation *heap, bool first_run) {
mp_obj_list_init((mp_obj_list_t *)mp_sys_path, 0); mp_obj_list_init((mp_obj_list_t *)mp_sys_path, 0);
mp_obj_list_append(mp_sys_path, MP_OBJ_NEW_QSTR(MP_QSTR_)); // current dir (or base dir of the script) mp_obj_list_append(mp_sys_path, MP_OBJ_NEW_QSTR(MP_QSTR_)); // current dir (or base dir of the script)
mp_obj_list_append(mp_sys_path, MP_OBJ_NEW_QSTR(MP_QSTR__slash_)); mp_obj_list_append(mp_sys_path, MP_OBJ_NEW_QSTR(MP_QSTR__slash_));
// Frozen modules are in their own pseudo-dir, e.g., ".frozen".
// Prioritize .frozen over /lib.
mp_obj_list_append(mp_sys_path, MP_OBJ_NEW_QSTR(MP_FROZEN_FAKE_DIR_QSTR));
mp_obj_list_append(mp_sys_path, MP_OBJ_NEW_QSTR(MP_QSTR__slash_lib)); mp_obj_list_append(mp_sys_path, MP_OBJ_NEW_QSTR(MP_QSTR__slash_lib));
mp_obj_list_init((mp_obj_list_t *)mp_sys_argv, 0); mp_obj_list_init((mp_obj_list_t *)mp_sys_argv, 0);

5
mpy-cross/README.md
View File

@ -23,10 +23,7 @@ by the target MicroPython runtime (eg onto a pyboard's filesystem), and then
imported like any other Python module using `import foo`. imported like any other Python module using `import foo`.
Different target runtimes may require a different format of the compiled Different target runtimes may require a different format of the compiled
bytecode, and such options can be passed to the cross compiler. For example, bytecode, and such options can be passed to the cross compiler.
the unix port of MicroPython requires the following:
$ ./mpy-cross -mcache-lookup-bc foo.py
If the Python code contains `@native` or `@viper` annotations, then you must If the Python code contains `@native` or `@viper` annotations, then you must
specify `-march` to match the target architecture. specify `-march` to match the target architecture.

8
mpy-cross/main.c
View File

@ -87,7 +87,6 @@ STATIC int usage(char **argv) {
"Target specific options:\n" "Target specific options:\n"
"-msmall-int-bits=number : set the maximum bits used to encode a small-int\n" "-msmall-int-bits=number : set the maximum bits used to encode a small-int\n"
"-mno-unicode : don't support unicode in compiled strings\n" "-mno-unicode : don't support unicode in compiled strings\n"
"-mcache-lookup-bc : cache map lookups in the bytecode\n"
"-march=<arch> : set architecture for native emitter; x86, x64, armv6, armv7m, armv7em, armv7emsp, armv7emdp, xtensa, xtensawin\n" "-march=<arch> : set architecture for native emitter; x86, x64, armv6, armv7m, armv7em, armv7emsp, armv7emdp, xtensa, xtensawin\n"
"\n" "\n"
"Implementation specific options:\n", argv[0] "Implementation specific options:\n", argv[0]
@ -172,8 +171,6 @@ MP_NOINLINE int main_(int argc, char **argv) {
#ifdef _WIN32 #ifdef _WIN32
set_fmode_binary(); set_fmode_binary();
#endif #endif
mp_obj_list_init(mp_sys_path, 0);
mp_obj_list_init(mp_sys_argv, 0);
#if MICROPY_EMIT_NATIVE #if MICROPY_EMIT_NATIVE
// Set default emitter options // Set default emitter options
@ -184,7 +181,6 @@ MP_NOINLINE int main_(int argc, char **argv) {
// set default compiler configuration // set default compiler configuration
mp_dynamic_compiler.small_int_bits = 31; mp_dynamic_compiler.small_int_bits = 31;
mp_dynamic_compiler.opt_cache_map_lookup_in_bytecode = 0;
mp_dynamic_compiler.py_builtins_str_unicode = 1; mp_dynamic_compiler.py_builtins_str_unicode = 1;
#if defined(__i386__) #if defined(__i386__)
mp_dynamic_compiler.native_arch = MP_NATIVE_ARCH_X86; mp_dynamic_compiler.native_arch = MP_NATIVE_ARCH_X86;
@ -243,10 +239,6 @@ MP_NOINLINE int main_(int argc, char **argv) {
return usage(argv); return usage(argv);
} }
// TODO check that small_int_bits is within range of host's capabilities // TODO check that small_int_bits is within range of host's capabilities
} else if (strcmp(argv[a], "-mno-cache-lookup-bc") == 0) {
mp_dynamic_compiler.opt_cache_map_lookup_in_bytecode = 0;
} else if (strcmp(argv[a], "-mcache-lookup-bc") == 0) {
mp_dynamic_compiler.opt_cache_map_lookup_in_bytecode = 1;
} else if (strcmp(argv[a], "-mno-unicode") == 0) { } else if (strcmp(argv[a], "-mno-unicode") == 0) {
mp_dynamic_compiler.py_builtins_str_unicode = 0; mp_dynamic_compiler.py_builtins_str_unicode = 0;
} else if (strcmp(argv[a], "-municode") == 0) { } else if (strcmp(argv[a], "-municode") == 0) {

2
mpy-cross/mpconfigport.h
View File

@ -36,8 +36,6 @@
#define MICROPY_COMP_TRIPLE_TUPLE_ASSIGN (1) #define MICROPY_COMP_TRIPLE_TUPLE_ASSIGN (1)
#define MICROPY_COMP_RETURN_IF_EXPR (1) #define MICROPY_COMP_RETURN_IF_EXPR (1)
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE (0)
#define MICROPY_READER_POSIX (1) #define MICROPY_READER_POSIX (1)
#define MICROPY_ENABLE_RUNTIME (0) #define MICROPY_ENABLE_RUNTIME (0)
#define MICROPY_ENABLE_GC (1) #define MICROPY_ENABLE_GC (1)

15
ports/unix/Makefile
View File

@ -13,6 +13,9 @@ include ../../py/mkenv.mk
-include mpconfigport.mk -include mpconfigport.mk
include $(VARIANT_DIR)/mpconfigvariant.mk include $(VARIANT_DIR)/mpconfigvariant.mk
# Use the default frozen manifest, variants may override this.
FROZEN_MANIFEST ?= variants/manifest.py
# This should be configured by the mpconfigvariant.mk # This should be configured by the mpconfigvariant.mk
PROG ?= micropython PROG ?= micropython
@ -39,7 +42,7 @@ CFLAGS += $(INC) $(CWARN) -std=gnu99 -DUNIX $(CFLAGS_MOD) $(COPT) -I$(VARIANT_DI
# Debugging/Optimization # Debugging/Optimization
ifdef DEBUG ifdef DEBUG
COPT ?= -O0 COPT ?= -Og
else else
COPT ?= -Os COPT ?= -Os
COPT += -DNDEBUG COPT += -DNDEBUG
@ -256,12 +259,14 @@ SRC_QSTR += $(SRC_C) $(SRC_CXX) $(SHARED_SRC_C) $(EXTMOD_SRC_C)
# SRC_QSTR # SRC_QSTR
SRC_QSTR_AUTO_DEPS += SRC_QSTR_AUTO_DEPS +=
ifneq ($(FROZEN_MPY_DIR),) ifneq ($(FROZEN_MANIFEST),)
# To use frozen code create a manifest.py file with a description of files to
# freeze, then invoke make with FROZEN_MANIFEST=manifest.py (be sure to build from scratch).
CFLAGS += -DMICROPY_QSTR_EXTRA_POOL=mp_qstr_frozen_const_pool CFLAGS += -DMICROPY_QSTR_EXTRA_POOL=mp_qstr_frozen_const_pool
CFLAGS += -DMICROPY_MODULE_FROZEN_MPY CFLAGS += -DMICROPY_MODULE_FROZEN_MPY
CFLAGS += -DMICROPY_MODULE_FROZEN_STR CFLAGS += -DMICROPY_MODULE_FROZEN_STR
CFLAGS += -DMPZ_DIG_SIZE=16 # force 16 bits to work on both 32 and 64 bit archs CFLAGS += -DMPZ_DIG_SIZE=16 # force 16 bits to work on both 32 and 64 bit archs
MPY_CROSS_FLAGS += -mcache-lookup-bc CFLAGS += -DMICROPY_MODULE_FROZEN_STR
endif endif
HASCPP17 = $(shell expr `$(CC) -dumpversion | cut -f1 -d.` \>= 7) HASCPP17 = $(shell expr `$(CC) -dumpversion | cut -f1 -d.` \>= 7)
@ -273,9 +278,7 @@ endif
CXXFLAGS += $(filter-out -Wmissing-prototypes -Wold-style-definition -std=gnu99 -std=gnu11,$(CFLAGS) $(CXXFLAGS_MOD)) CXXFLAGS += $(filter-out -Wmissing-prototypes -Wold-style-definition -std=gnu99 -std=gnu11,$(CFLAGS) $(CXXFLAGS_MOD))
ifeq ($(MICROPY_FORCE_32BIT),1) ifeq ($(MICROPY_FORCE_32BIT),1)
RUN_TESTS_MPY_CROSS_FLAGS = --mpy-cross-flags='-mcache-lookup-bc -march=x86' RUN_TESTS_MPY_CROSS_FLAGS = --mpy-cross-flags='-march=x86'
else
RUN_TESTS_MPY_CROSS_FLAGS = --mpy-cross-flags='-mcache-lookup-bc'
endif endif
ifeq ($(CROSS_COMPILE),arm-linux-gnueabi-) ifeq ($(CROSS_COMPILE),arm-linux-gnueabi-)

6
ports/unix/coverage.c
View File

@ -161,7 +161,7 @@ STATIC void pairheap_test(size_t nops, int *ops) {
mp_pairheap_init_node(pairheap_lt, &node[i]); mp_pairheap_init_node(pairheap_lt, &node[i]);
} }
mp_pairheap_t *heap = mp_pairheap_new(pairheap_lt); mp_pairheap_t *heap = mp_pairheap_new(pairheap_lt);
printf("create:"); mp_printf(&mp_plat_print, "create:");
for (size_t i = 0; i < nops; ++i) { for (size_t i = 0; i < nops; ++i) {
if (ops[i] >= 0) { if (ops[i] >= 0) {
heap = mp_pairheap_push(pairheap_lt, heap, &node[ops[i]]); heap = mp_pairheap_push(pairheap_lt, heap, &node[ops[i]]);
@ -175,13 +175,13 @@ STATIC void pairheap_test(size_t nops, int *ops) {
; ;
} }
} }
printf("\npop all:"); mp_printf(&mp_plat_print, "\npop all:");
while (!mp_pairheap_is_empty(pairheap_lt, heap)) { while (!mp_pairheap_is_empty(pairheap_lt, heap)) {
mp_printf(&mp_plat_print, " %d", mp_pairheap_peek(pairheap_lt, heap) - &node[0]); mp_printf(&mp_plat_print, " %d", mp_pairheap_peek(pairheap_lt, heap) - &node[0]);
; ;
heap = mp_pairheap_pop(pairheap_lt, heap); heap = mp_pairheap_pop(pairheap_lt, heap);
} }
printf("\n"); mp_printf(&mp_plat_print, "\n");
} }
// function to run extra tests for things that can't be checked by scripts // function to run extra tests for things that can't be checked by scripts

16
ports/unix/main.c
View File

@ -494,16 +494,10 @@ MP_NOINLINE int main_(int argc, char **argv) {
char *home = getenv("HOME"); char *home = getenv("HOME");
char *path = getenv("MICROPYPATH"); char *path = getenv("MICROPYPATH");
if (path == NULL) { if (path == NULL) {
#ifdef MICROPY_PY_SYS_PATH_DEFAULT
path = MICROPY_PY_SYS_PATH_DEFAULT; path = MICROPY_PY_SYS_PATH_DEFAULT;
#else
path = "~/.micropython/lib:/usr/lib/micropython";
#endif
} }
size_t path_num = 2; // [0] is for current dir (or base dir of the script) size_t path_num = 1; // [0] is for current dir (or base dir of the script)
// [1] is for frozen files. if (*path == PATHLIST_SEP_CHAR) {
size_t builtin_path_count = path_num;
if (*path == ':') {
path_num++; path_num++;
} }
for (char *p = path; p != NULL; p = strchr(p, PATHLIST_SEP_CHAR)) { for (char *p = path; p != NULL; p = strchr(p, PATHLIST_SEP_CHAR)) {
@ -516,11 +510,9 @@ MP_NOINLINE int main_(int argc, char **argv) {
mp_obj_t *path_items; mp_obj_t *path_items;
mp_obj_list_get(mp_sys_path, &path_num, &path_items); mp_obj_list_get(mp_sys_path, &path_num, &path_items);
path_items[0] = MP_OBJ_NEW_QSTR(MP_QSTR_); path_items[0] = MP_OBJ_NEW_QSTR(MP_QSTR_);
// Frozen modules are in their own pseudo-dir, e.g., ".frozen".
path_items[1] = MP_OBJ_NEW_QSTR(MP_FROZEN_FAKE_DIR_QSTR);
{ {
char *p = path; char *p = path;
for (mp_uint_t i = builtin_path_count; i < path_num; i++) { for (mp_uint_t i = 1; i < path_num; i++) {
char *p1 = strchr(p, PATHLIST_SEP_CHAR); char *p1 = strchr(p, PATHLIST_SEP_CHAR);
if (p1 == NULL) { if (p1 == NULL) {
p1 = p + strlen(p); p1 = p + strlen(p);
@ -661,7 +653,7 @@ MP_NOINLINE int main_(int argc, char **argv) {
break; break;
} }
// Set base dir of the script as first entry in sys.path // Set base dir of the script as first entry in sys.path.
char *p = strrchr(basedir, '/'); char *p = strrchr(basedir, '/');
path_items[0] = mp_obj_new_str_via_qstr(basedir, p - basedir); path_items[0] = mp_obj_new_str_via_qstr(basedir, p - basedir);
free(pathbuf); free(pathbuf);

25
ports/unix/modos.c
View File

@ -27,6 +27,7 @@
#include <sys/types.h> #include <sys/types.h>
#include <sys/stat.h> #include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h> #include <unistd.h>
#include <errno.h> #include <errno.h>
#include <stdio.h> #include <stdio.h>
@ -48,6 +49,29 @@
#define USE_STATFS 1 #define USE_STATFS 1
#endif #endif
#if defined(__GLIBC__) && defined(__GLIBC_PREREQ)
#if __GLIBC_PREREQ(2, 25)
#include <sys/random.h>
#define _HAVE_GETRANDOM
#endif
#endif
STATIC mp_obj_t mod_os_urandom(mp_obj_t num) {
mp_int_t n = mp_obj_get_int(num);
vstr_t vstr;
vstr_init_len(&vstr, n);
#ifdef _HAVE_GETRANDOM
RAISE_ERRNO(getrandom(vstr.buf, n, 0), errno);
#else
int fd = open("/dev/urandom", O_RDONLY);
RAISE_ERRNO(fd, errno);
RAISE_ERRNO(read(fd, vstr.buf, n), errno);
close(fd);
#endif
return mp_obj_new_str_from_vstr(&mp_type_bytes, &vstr);
}
STATIC MP_DEFINE_CONST_FUN_OBJ_1(mod_os_urandom_obj, mod_os_urandom);
STATIC mp_obj_t mod_os_stat(mp_obj_t path_in) { STATIC mp_obj_t mod_os_stat(mp_obj_t path_in) {
struct stat sb; struct stat sb;
const char *path = mp_obj_str_get_str(path_in); const char *path = mp_obj_str_get_str(path_in);
@ -308,6 +332,7 @@ STATIC const mp_rom_map_elem_t mp_module_os_globals_table[] = {
{ MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_uos) }, { MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_uos) },
{ MP_ROM_QSTR(MP_QSTR_errno), MP_ROM_PTR(&mod_os_errno_obj) }, { MP_ROM_QSTR(MP_QSTR_errno), MP_ROM_PTR(&mod_os_errno_obj) },
{ MP_ROM_QSTR(MP_QSTR_stat), MP_ROM_PTR(&mod_os_stat_obj) }, { MP_ROM_QSTR(MP_QSTR_stat), MP_ROM_PTR(&mod_os_stat_obj) },
{ MP_ROM_QSTR(MP_QSTR_urandom), MP_ROM_PTR(&mod_os_urandom_obj) },
#if MICROPY_PY_OS_STATVFS #if MICROPY_PY_OS_STATVFS
{ MP_ROM_QSTR(MP_QSTR_statvfs), MP_ROM_PTR(&mod_os_statvfs_obj) }, { MP_ROM_QSTR(MP_QSTR_statvfs), MP_ROM_PTR(&mod_os_statvfs_obj) },
#endif #endif

13
ports/unix/mpconfigport.h
View File

@ -80,10 +80,14 @@
#endif #endif
#define MICROPY_STREAMS_POSIX_API (1) #define MICROPY_STREAMS_POSIX_API (1)
#define MICROPY_OPT_COMPUTED_GOTO (1) #define MICROPY_OPT_COMPUTED_GOTO (1)
#ifndef MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE #ifndef MICROPY_OPT_LOAD_ATTR_FAST_PATH
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE (1) #define MICROPY_OPT_LOAD_ATTR_FAST_PATH (1)
#endif
#ifndef MICROPY_OPT_MAP_LOOKUP_CACHE
#define MICROPY_OPT_MAP_LOOKUP_CACHE (1)
#endif #endif
#define MICROPY_MODULE_WEAK_LINKS (1) #define MICROPY_MODULE_WEAK_LINKS (1)
#define MICROPY_MODULE_OVERRIDE_MAIN_IMPORT (1)
#define MICROPY_CAN_OVERRIDE_BUILTINS (1) #define MICROPY_CAN_OVERRIDE_BUILTINS (1)
#define MICROPY_VFS_POSIX_FILE (1) #define MICROPY_VFS_POSIX_FILE (1)
#define MICROPY_PY_FUNCTION_ATTRS (1) #define MICROPY_PY_FUNCTION_ATTRS (1)
@ -107,6 +111,8 @@
#define MICROPY_PY_ARRAY_SLICE_ASSIGN (1) #define MICROPY_PY_ARRAY_SLICE_ASSIGN (1)
#define MICROPY_PY_BUILTINS_SLICE_ATTRS (1) #define MICROPY_PY_BUILTINS_SLICE_ATTRS (1)
#define MICROPY_PY_BUILTINS_SLICE_INDICES (1) #define MICROPY_PY_BUILTINS_SLICE_INDICES (1)
#define MICROPY_PY_SYS_ATEXIT (1)
#define MICROPY_PY_SYS_PATH_ARGV_DEFAULTS (0)
#define MICROPY_PY_SYS_EXIT (1) #define MICROPY_PY_SYS_EXIT (1)
#if MICROPY_PY_SYS_SETTRACE #if MICROPY_PY_SYS_SETTRACE
#define MICROPY_PERSISTENT_CODE_SAVE (1) #define MICROPY_PERSISTENT_CODE_SAVE (1)
@ -119,6 +125,9 @@
#define MICROPY_PY_SYS_PLATFORM "linux" #define MICROPY_PY_SYS_PLATFORM "linux"
#endif #endif
#endif #endif
#ifndef MICROPY_PY_SYS_PATH_DEFAULT
#define MICROPY_PY_SYS_PATH_DEFAULT ".frozen:~/.micropython/lib:/usr/lib/micropython"
#endif
#define MICROPY_PY_SYS_MAXSIZE (1) #define MICROPY_PY_SYS_MAXSIZE (1)
#define MICROPY_PY_SYS_STDFILES (1) #define MICROPY_PY_SYS_STDFILES (1)
#define MICROPY_PY_SYS_EXC_INFO (1) #define MICROPY_PY_SYS_EXC_INFO (1)

1
ports/unix/variants/coverage/mpconfigvariant.h
View File

@ -51,7 +51,6 @@
#define MICROPY_PY_MATH_FACTORIAL (1) #define MICROPY_PY_MATH_FACTORIAL (1)
#define MICROPY_PY_URANDOM_EXTRA_FUNCS (1) #define MICROPY_PY_URANDOM_EXTRA_FUNCS (1)
#define MICROPY_PY_IO_BUFFEREDWRITER (1) #define MICROPY_PY_IO_BUFFEREDWRITER (1)
#define MICROPY_PY_IO_RESOURCE_STREAM (1)
#define MICROPY_PY_UASYNCIO (1) #define MICROPY_PY_UASYNCIO (1)
#define MICROPY_PY_URE_DEBUG (1) #define MICROPY_PY_URE_DEBUG (1)
#define MICROPY_PY_URE_MATCH_GROUPS (1) #define MICROPY_PY_URE_MATCH_GROUPS (1)

4
ports/unix/variants/coverage/mpconfigvariant.mk
View File

@ -12,15 +12,13 @@ CFLAGS += \
LDFLAGS += -fprofile-arcs -ftest-coverage LDFLAGS += -fprofile-arcs -ftest-coverage
FROZEN_MANIFEST ?= $(VARIANT_DIR)/manifest.py
USER_C_MODULES = $(TOP)/examples/usercmodule USER_C_MODULES = $(TOP)/examples/usercmodule
MICROPY_VFS_FAT = 1 MICROPY_VFS_FAT = 1
MICROPY_VFS_LFS1 = 1 MICROPY_VFS_LFS1 = 1
MICROPY_VFS_LFS2 = 1 MICROPY_VFS_LFS2 = 1
FROZEN_DIR=variants/coverage/frzstr
FROZEN_MPY_DIR=variants/coverage/frzmpy
SRC_QRIO := $(patsubst ../../%,%,$(wildcard ../../shared-bindings/qrio/*.c ../../shared-module/qrio/*.c ../../lib/quirc/lib/*.c)) SRC_QRIO := $(patsubst ../../%,%,$(wildcard ../../shared-bindings/qrio/*.c ../../shared-module/qrio/*.c ../../lib/quirc/lib/*.c))
SRC_C += $(SRC_QRIO) SRC_C += $(SRC_QRIO)

6
ports/unix/variants/minimal/mpconfigvariant.h
View File

@ -55,7 +55,8 @@
#define MICROPY_LONGINT_IMPL (MICROPY_LONGINT_IMPL_NONE) #define MICROPY_LONGINT_IMPL (MICROPY_LONGINT_IMPL_NONE)
#define MICROPY_STREAMS_NON_BLOCK (0) #define MICROPY_STREAMS_NON_BLOCK (0)
#define MICROPY_OPT_COMPUTED_GOTO (0) #define MICROPY_OPT_COMPUTED_GOTO (0)
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE (0) #define MICROPY_OPT_LOAD_ATTR_FAST_PATH (0)
#define MICROPY_OPT_MAP_LOOKUP_CACHE (0)
#define MICROPY_CAN_OVERRIDE_BUILTINS (0) #define MICROPY_CAN_OVERRIDE_BUILTINS (0)
#define MICROPY_BUILTIN_METHOD_CHECK_SELF_ARG (0) #define MICROPY_BUILTIN_METHOD_CHECK_SELF_ARG (0)
#define MICROPY_CPYTHON_COMPAT (0) #define MICROPY_CPYTHON_COMPAT (0)
@ -87,6 +88,9 @@
#define MICROPY_PY_SYS (1) #define MICROPY_PY_SYS (1)
#define MICROPY_PY_SYS_EXIT (0) #define MICROPY_PY_SYS_EXIT (0)
#define MICROPY_PY_SYS_PLATFORM "linux" #define MICROPY_PY_SYS_PLATFORM "linux"
#ifndef MICROPY_PY_SYS_PATH_DEFAULT
#define MICROPY_PY_SYS_PATH_DEFAULT ".frozen:~/.micropython/lib:/usr/lib/micropython"
#endif
#define MICROPY_PY_SYS_MAXSIZE (0) #define MICROPY_PY_SYS_MAXSIZE (0)
#define MICROPY_PY_SYS_STDFILES (0) #define MICROPY_PY_SYS_STDFILES (0)
#define MICROPY_PY_CMATH (0) #define MICROPY_PY_CMATH (0)

14
py/bc.c
View File

@ -314,24 +314,10 @@ void mp_setup_code_state(mp_code_state_t *code_state, size_t n_args, size_t n_kw
// The following table encodes the number of bytes that a specific opcode // The following table encodes the number of bytes that a specific opcode
// takes up. Some opcodes have an extra byte, defined by MP_BC_MASK_EXTRA_BYTE. // takes up. Some opcodes have an extra byte, defined by MP_BC_MASK_EXTRA_BYTE.
// There are 4 special opcodes that have an extra byte only when
// MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE is enabled (and they take a qstr):
// MP_BC_LOAD_NAME
// MP_BC_LOAD_GLOBAL
// MP_BC_LOAD_ATTR
// MP_BC_STORE_ATTR
uint mp_opcode_format(const byte *ip, size_t *opcode_size, bool count_var_uint) { uint mp_opcode_format(const byte *ip, size_t *opcode_size, bool count_var_uint) {
uint f = MP_BC_FORMAT(*ip); uint f = MP_BC_FORMAT(*ip);
const byte *ip_start = ip; const byte *ip_start = ip;
if (f == MP_BC_FORMAT_QSTR) { if (f == MP_BC_FORMAT_QSTR) {
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE_DYNAMIC) {
if (*ip == MP_BC_LOAD_NAME
|| *ip == MP_BC_LOAD_GLOBAL
|| *ip == MP_BC_LOAD_ATTR
|| *ip == MP_BC_STORE_ATTR) {
ip += 1;
}
}
ip += 3; ip += 3;
} else { } else {
int extra_byte = (*ip & MP_BC_MASK_EXTRA_BYTE) == 0; int extra_byte = (*ip & MP_BC_MASK_EXTRA_BYTE) == 0;

9
py/builtin.h
View File

@ -119,13 +119,8 @@ extern const mp_obj_module_t mp_module_utimeq;
extern const mp_obj_module_t mp_module_machine; extern const mp_obj_module_t mp_module_machine;
extern const mp_obj_module_t mp_module_framebuf; extern const mp_obj_module_t mp_module_framebuf;
extern const mp_obj_module_t mp_module_btree; extern const mp_obj_module_t mp_module_btree;
extern const mp_obj_module_t ulab_user_cmodule; extern const mp_obj_module_t mp_module_ubluetooth;
extern mp_obj_module_t ulab_fft_module; extern const mp_obj_module_t mp_module_uplatform;
extern mp_obj_module_t ulab_filter_module;
extern mp_obj_module_t ulab_linalg_module;
extern mp_obj_module_t ulab_numerical_module;
extern mp_obj_module_t ulab_poly_module;
extern const char MICROPY_PY_BUILTINS_HELP_TEXT[]; extern const char MICROPY_PY_BUILTINS_HELP_TEXT[];

15
py/builtinhelp.c
View File

@ -69,10 +69,10 @@ STATIC void mp_help_add_from_map(mp_obj_t list, const mp_map_t *map) {
#if MICROPY_MODULE_FROZEN #if MICROPY_MODULE_FROZEN
STATIC void mp_help_add_from_names(mp_obj_t list, const char *name) { STATIC void mp_help_add_from_names(mp_obj_t list, const char *name) {
while (*name) { while (*name) {
size_t l = strlen(name); size_t len = strlen(name);
// name should end in '.py' and we strip it off // name should end in '.py' and we strip it off
mp_obj_list_append(list, mp_obj_new_str(name, l - 3)); mp_obj_list_append(list, mp_obj_new_str(name, len - 3));
name += l + 1; name += len + 1;
} }
} }
#endif #endif
@ -92,12 +92,9 @@ STATIC void mp_help_print_modules(void) {
mp_help_add_from_map(list, &mp_builtin_module_map); mp_help_add_from_map(list, &mp_builtin_module_map);
#if MICROPY_MODULE_FROZEN_STR #if MICROPY_MODULE_FROZEN
mp_help_add_from_names(list, mp_frozen_str_names); extern const char mp_frozen_names[];
#endif mp_help_add_from_names(list, mp_frozen_names);
#if MICROPY_MODULE_FROZEN_MPY
mp_help_add_from_names(list, mp_frozen_mpy_names);
#endif #endif
// sort the list so it's printed in alphabetical order // sort the list so it's printed in alphabetical order

538
py/builtinimport.c
View File

@ -5,6 +5,7 @@
* *
* SPDX-FileCopyrightText: Copyright (c) 2013-2019 Damien P. George * SPDX-FileCopyrightText: Copyright (c) 2013-2019 Damien P. George
* Copyright (c) 2014 Paul Sokolovsky * Copyright (c) 2014 Paul Sokolovsky
* Copyright (c) 2021 Jim Mussared
* *
* Permission is hereby granted, free of charge, to any person obtaining a copy * Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal * of this software and associated documentation files (the "Software"), to deal
@ -50,7 +51,11 @@
#if MICROPY_ENABLE_EXTERNAL_IMPORT #if MICROPY_ENABLE_EXTERNAL_IMPORT
#define PATH_SEP_CHAR '/' // Must be a string of one byte.
#define PATH_SEP_CHAR "/"
// Virtual sys.path entry that maps to the frozen modules.
#define MP_FROZEN_PATH_PREFIX ".frozen/"
bool mp_obj_is_package(mp_obj_t module) { bool mp_obj_is_package(mp_obj_t module) {
mp_obj_t dest[2]; mp_obj_t dest[2];
@ -58,31 +63,33 @@ bool mp_obj_is_package(mp_obj_t module) {
return dest[0] != MP_OBJ_NULL; return dest[0] != MP_OBJ_NULL;
} }
// Stat either frozen or normal module by a given path // Wrapper for mp_import_stat (which is provided by the port, and typically
// (whatever is available, if at all). // uses mp_vfs_import_stat) to also search frozen modules. Given an exact
STATIC mp_import_stat_t mp_import_stat_any(const char *path) { // path to a file or directory (e.g. "foo/bar", foo/bar.py" or "foo/bar.mpy"),
// will return whether the path is a file, directory, or doesn't exist.
STATIC mp_import_stat_t stat_path_or_frozen(const char *path) {
#if MICROPY_MODULE_FROZEN #if MICROPY_MODULE_FROZEN
if (strncmp(MP_FROZEN_FAKE_DIR_SLASH, // Only try and load as a frozen module if it starts with .frozen/.
path, const int frozen_path_prefix_len = strlen(MP_FROZEN_PATH_PREFIX);
MP_FROZEN_FAKE_DIR_SLASH_LENGTH) == 0) { if (strncmp(path, MP_FROZEN_PATH_PREFIX, frozen_path_prefix_len) == 0) {
mp_import_stat_t st = mp_frozen_stat(path + MP_FROZEN_FAKE_DIR_SLASH_LENGTH); return mp_find_frozen_module(path + frozen_path_prefix_len, NULL, NULL);
if (st != MP_IMPORT_STAT_NO_EXIST) {
return st;
}
} }
#endif #endif
return mp_import_stat(path); return mp_import_stat(path);
} }
// Given a path to a .py file, try and find this path as either a .py or .mpy
// in either the filesystem or frozen modules.
STATIC mp_import_stat_t stat_file_py_or_mpy(vstr_t *path) { STATIC mp_import_stat_t stat_file_py_or_mpy(vstr_t *path) {
mp_import_stat_t stat = mp_import_stat_any(vstr_null_terminated_str(path)); mp_import_stat_t stat = stat_path_or_frozen(vstr_null_terminated_str(path));
if (stat == MP_IMPORT_STAT_FILE) { if (stat == MP_IMPORT_STAT_FILE) {
return stat; return stat;
} }
#if MICROPY_PERSISTENT_CODE_LOAD #if MICROPY_PERSISTENT_CODE_LOAD
// Didn't find .py -- try the .mpy instead by inserting an 'm' into the '.py'.
vstr_ins_byte(path, path->len - 2, 'm'); vstr_ins_byte(path, path->len - 2, 'm');
stat = mp_import_stat_any(vstr_null_terminated_str(path)); stat = stat_path_or_frozen(vstr_null_terminated_str(path));
if (stat == MP_IMPORT_STAT_FILE) { if (stat == MP_IMPORT_STAT_FILE) {
return stat; return stat;
} }
@ -91,8 +98,10 @@ STATIC mp_import_stat_t stat_file_py_or_mpy(vstr_t *path) {
return MP_IMPORT_STAT_NO_EXIST; return MP_IMPORT_STAT_NO_EXIST;
} }
// Given an import path (e.g. "foo/bar"), try and find "foo/bar" (a directory)
// or "foo/bar.(m)py" in either the filesystem or frozen modules.
STATIC mp_import_stat_t stat_dir_or_file(vstr_t *path) { STATIC mp_import_stat_t stat_dir_or_file(vstr_t *path) {
mp_import_stat_t stat = mp_import_stat_any(vstr_null_terminated_str(path)); mp_import_stat_t stat = stat_path_or_frozen(vstr_null_terminated_str(path));
DEBUG_printf("stat %s: %d\n", vstr_str(path), stat); DEBUG_printf("stat %s: %d\n", vstr_str(path), stat);
if (stat == MP_IMPORT_STAT_DIR) { if (stat == MP_IMPORT_STAT_DIR) {
return stat; return stat;
@ -103,20 +112,16 @@ STATIC mp_import_stat_t stat_dir_or_file(vstr_t *path) {
return stat_file_py_or_mpy(path); return stat_file_py_or_mpy(path);
} }
STATIC mp_import_stat_t find_file(const char *file_str, uint file_len, vstr_t *dest) { // Given a top-level module, try and find it in each of the sys.path entries
// via stat_dir_or_file.
STATIC mp_import_stat_t stat_top_level_dir_or_file(qstr mod_name, vstr_t *dest) {
DEBUG_printf("stat_top_level_dir_or_file: '%s'\n", qstr_str(mod_name));
#if MICROPY_PY_SYS #if MICROPY_PY_SYS
// extract the list of paths
size_t path_num; size_t path_num;
mp_obj_t *path_items; mp_obj_t *path_items;
mp_obj_list_get(mp_sys_path, &path_num, &path_items); mp_obj_list_get(mp_sys_path, &path_num, &path_items);
if (path_num == 0) { if (path_num > 0) {
#endif
// mp_sys_path is empty, so just use the given file name
vstr_add_strn(dest, file_str, file_len);
return stat_dir_or_file(dest);
#if MICROPY_PY_SYS
} else {
// go through each path looking for a directory or file // go through each path looking for a directory or file
for (size_t i = 0; i < path_num; i++) { for (size_t i = 0; i < path_num; i++) {
vstr_reset(dest); vstr_reset(dest);
@ -124,9 +129,9 @@ STATIC mp_import_stat_t find_file(const char *file_str, uint file_len, vstr_t *d
const char *p = mp_obj_str_get_data(path_items[i], &p_len); const char *p = mp_obj_str_get_data(path_items[i], &p_len);
if (p_len > 0) { if (p_len > 0) {
vstr_add_strn(dest, p, p_len); vstr_add_strn(dest, p, p_len);
vstr_add_char(dest, PATH_SEP_CHAR); vstr_add_char(dest, PATH_SEP_CHAR[0]);
} }
vstr_add_strn(dest, file_str, file_len); vstr_add_str(dest, qstr_str(mod_name));
mp_import_stat_t stat = stat_dir_or_file(dest); mp_import_stat_t stat = stat_dir_or_file(dest);
if (stat != MP_IMPORT_STAT_NO_EXIST) { if (stat != MP_IMPORT_STAT_NO_EXIST) {
return stat; return stat;
@ -135,8 +140,13 @@ STATIC mp_import_stat_t find_file(const char *file_str, uint file_len, vstr_t *d
// could not find a directory or file // could not find a directory or file
return MP_IMPORT_STAT_NO_EXIST; return MP_IMPORT_STAT_NO_EXIST;
} }
#endif #endif
// mp_sys_path is empty (or not enabled), so just stat the given path
// directly.
vstr_add_str(dest, qstr_str(mod_name));
return stat_dir_or_file(dest);
} }
#if MICROPY_MODULE_FROZEN_STR || MICROPY_ENABLE_COMPILER #if MICROPY_MODULE_FROZEN_STR || MICROPY_ENABLE_COMPILER
@ -193,19 +203,17 @@ STATIC void do_execute_raw_code(mp_obj_t module_obj, mp_raw_code_t *raw_code, co
STATIC void do_load(mp_obj_t module_obj, vstr_t *file) { STATIC void do_load(mp_obj_t module_obj, vstr_t *file) {
#if MICROPY_MODULE_FROZEN || MICROPY_ENABLE_COMPILER || (MICROPY_PERSISTENT_CODE_LOAD && MICROPY_HAS_FILE_READER) #if MICROPY_MODULE_FROZEN || MICROPY_ENABLE_COMPILER || (MICROPY_PERSISTENT_CODE_LOAD && MICROPY_HAS_FILE_READER)
char *file_str = vstr_null_terminated_str(file); const char *file_str = vstr_null_terminated_str(file);
#endif #endif
#if MICROPY_MODULE_FROZEN || MICROPY_MODULE_FROZEN_MPY
if (strncmp(MP_FROZEN_FAKE_DIR_SLASH,
file_str,
MP_FROZEN_FAKE_DIR_SLASH_LENGTH) == 0) {
// If we support frozen modules (either as str or mpy) then try to find the // If we support frozen modules (either as str or mpy) then try to find the
// requested filename in the list of frozen module filenames. // requested filename in the list of frozen module filenames.
#if MICROPY_MODULE_FROZEN #if MICROPY_MODULE_FROZEN
void *modref; void *modref;
int frozen_type = mp_find_frozen_module(file_str + MP_FROZEN_FAKE_DIR_SLASH_LENGTH, file->len - MP_FROZEN_FAKE_DIR_SLASH_LENGTH, &modref); int frozen_type;
#endif const int frozen_path_prefix_len = strlen(MP_FROZEN_PATH_PREFIX);
if (strncmp(file_str, MP_FROZEN_PATH_PREFIX, frozen_path_prefix_len) == 0) {
mp_find_frozen_module(file_str + frozen_path_prefix_len, &frozen_type, &modref);
// If we support frozen str modules and the compiler is enabled, and we // If we support frozen str modules and the compiler is enabled, and we
// found the filename in the list of frozen files, then load and execute it. // found the filename in the list of frozen files, then load and execute it.
@ -220,13 +228,12 @@ STATIC void do_load(mp_obj_t module_obj, vstr_t *file) {
// its data) in the list of frozen files, execute it. // its data) in the list of frozen files, execute it.
#if MICROPY_MODULE_FROZEN_MPY #if MICROPY_MODULE_FROZEN_MPY
if (frozen_type == MP_FROZEN_MPY) { if (frozen_type == MP_FROZEN_MPY) {
do_execute_raw_code(module_obj, modref, file_str); do_execute_raw_code(module_obj, modref, file_str + frozen_path_prefix_len);
return; return;
} }
#endif #endif
} }
#endif // MICROPY_MODULE_FROZEN || MICROPY_MODULE_FROZEN_MPY #endif // MICROPY_MODULE_FROZEN
// If we support loading .mpy files then check if the file extension is of // If we support loading .mpy files then check if the file extension is of
// the correct format and, if so, load and execute the file. // the correct format and, if so, load and execute the file.
@ -251,15 +258,212 @@ STATIC void do_load(mp_obj_t module_obj, vstr_t *file) {
#endif #endif
} }
STATIC void chop_component(const char *start, const char **end) { // Convert a relative (to the current module) import, going up "level" levels,
const char *p = *end; // into an absolute import.
while (p > start) { STATIC void evaluate_relative_import(mp_int_t level, const char **module_name, size_t *module_name_len) {
// What we want to do here is to take the name of the current module,
// remove <level> trailing components, and concatenate the passed-in
// module name.
// For example, level=3, module_name="foo.bar", __name__="a.b.c.d" --> "a.foo.bar"
// "Relative imports use a module's __name__ attribute to determine that
// module's position in the package hierarchy."
// http://legacy.python.org/dev/peps/pep-0328/#relative-imports-and-name
mp_obj_t current_module_name_obj = mp_obj_dict_get(MP_OBJ_FROM_PTR(mp_globals_get()), MP_OBJ_NEW_QSTR(MP_QSTR___name__));
assert(current_module_name_obj != MP_OBJ_NULL);
#if MICROPY_MODULE_OVERRIDE_MAIN_IMPORT && MICROPY_CPYTHON_COMPAT
if (MP_OBJ_QSTR_VALUE(current_module_name_obj) == MP_QSTR___main__) {
// This is a module loaded by -m command-line switch (e.g. unix port),
// and so its __name__ has been set to "__main__". Get its real name
// that we stored during import in the __main__ attribute.
current_module_name_obj = mp_obj_dict_get(MP_OBJ_FROM_PTR(mp_globals_get()), MP_OBJ_NEW_QSTR(MP_QSTR___main__));
}
#endif
// If we have a __path__ in the globals dict, then we're a package.
bool is_pkg = mp_map_lookup(&mp_globals_get()->map, MP_OBJ_NEW_QSTR(MP_QSTR___path__), MP_MAP_LOOKUP);
#if DEBUG_PRINT
DEBUG_printf("Current module/package: ");
mp_obj_print_helper(MICROPY_DEBUG_PRINTER, current_module_name_obj, PRINT_REPR);
DEBUG_printf(", is_package: %d", is_pkg);
DEBUG_printf("\n");
#endif
size_t current_module_name_len;
const char *current_module_name = mp_obj_str_get_data(current_module_name_obj, &current_module_name_len);
const char *p = current_module_name + current_module_name_len;
if (is_pkg) {
// If we're evaluating relative to a package, then take off one fewer
// level (i.e. the relative search starts inside the package, rather
// than as a sibling of the package).
--level;
}
// Walk back 'level' dots (or run out of path).
while (level && p > current_module_name) {
if (*--p == '.') { if (*--p == '.') {
*end = p; --level;
return;
} }
} }
*end = p;
// We must have some component left over to import from.
if (p == current_module_name) {
mp_raise_msg(&mp_type_ImportError, MP_ERROR_TEXT("can't perform relative import"));
}
// New length is len("<chopped path>.<module_name>"). Note: might be one byte
// more than we need if module_name is empty (for the extra . we will
// append).
uint new_module_name_len = (size_t)(p - current_module_name) + 1 + *module_name_len;
char *new_mod = mp_local_alloc(new_module_name_len);
memcpy(new_mod, current_module_name, p - current_module_name);
// Only append ".<module_name>" if there was one).
if (*module_name_len != 0) {
new_mod[p - current_module_name] = '.';
memcpy(new_mod + (p - current_module_name) + 1, *module_name, *module_name_len);
} else {
--new_module_name_len;
}
// Copy into a QSTR.
qstr new_mod_q = qstr_from_strn(new_mod, new_module_name_len);
mp_local_free(new_mod);
DEBUG_printf("Resolved base name for relative import: '%s'\n", qstr_str(new_mod_q));
*module_name = qstr_str(new_mod_q);
*module_name_len = new_module_name_len;
}
// Load a module at the specified absolute path, possibly as a submodule of the given outer module.
// full_mod_name: The full absolute path to this module (e.g. "foo.bar.baz").
// level_mod_name: The final component of the path (e.g. "baz").
// outer_module_obj: The parent module (we need to store this module as an
// attribute on it) (or MP_OBJ_NULL for top-level).
// path: The filesystem path where we found the parent module
// (or empty for a top level module).
// override_main: Whether to set the __name__ to "__main__" (and use __main__
// for the actual path).
STATIC mp_obj_t process_import_at_level(qstr full_mod_name, qstr level_mod_name, mp_obj_t outer_module_obj, vstr_t *path, bool override_main) {
mp_import_stat_t stat = MP_IMPORT_STAT_NO_EXIST;
// Exact-match of built-in (or already-loaded) takes priority.
mp_obj_t module_obj = mp_module_get_loaded_or_builtin(full_mod_name);
// Even if we find the module, go through the motions of searching for it
// because we may actually be in the process of importing a sub-module.
// So we need to (re-)find the correct path to be finding the sub-module
// on the next iteration of process_import_at_level.
if (outer_module_obj == MP_OBJ_NULL) {
DEBUG_printf("Searching for top-level module\n");
// First module in the dotted-name; search for a directory or file
// relative to all the locations in sys.path.
stat = stat_top_level_dir_or_file(full_mod_name, path);
// If the module "foo" doesn't exist on the filesystem, and it's not a
// builtin, try and find "ufoo" as a built-in. (This feature was
// formerly known as "weak links").
#if MICROPY_MODULE_WEAK_LINKS
if (stat == MP_IMPORT_STAT_NO_EXIST && module_obj == MP_OBJ_NULL) {
char *umodule_buf = vstr_str(path);
umodule_buf[0] = 'u';
strcpy(umodule_buf + 1, qstr_str(level_mod_name));
qstr umodule_name = qstr_from_str(umodule_buf);
module_obj = mp_module_get_builtin(umodule_name);
}
#endif
} else {
DEBUG_printf("Searching for sub-module\n");
// Add the current part of the module name to the path.
vstr_add_char(path, PATH_SEP_CHAR[0]);
vstr_add_str(path, qstr_str(level_mod_name));
// Because it's not top level, we already know which path the parent was found in.
stat = stat_dir_or_file(path);
}
DEBUG_printf("Current path: %.*s\n", (int)vstr_len(path), vstr_str(path));
if (module_obj == MP_OBJ_NULL) {
// Not a built-in and not already-loaded.
if (stat == MP_IMPORT_STAT_NO_EXIST) {
// And the file wasn't found -- fail.
#if MICROPY_ERROR_REPORTING <= MICROPY_ERROR_REPORTING_TERSE
mp_raise_msg(&mp_type_ImportError, MP_ERROR_TEXT("module not found"));
#else
mp_raise_msg_varg(&mp_type_ImportError, MP_ERROR_TEXT("no module named '%q'"), full_mod_name);
#endif
}
// Not a built-in but found on the filesystem, try and load it.
DEBUG_printf("Found path: %.*s\n", (int)vstr_len(path), vstr_str(path));
// Prepare for loading from the filesystem. Create a new shell module.
module_obj = mp_obj_new_module(full_mod_name);
#if MICROPY_MODULE_OVERRIDE_MAIN_IMPORT
// If this module is being loaded via -m on unix, then
// override __name__ to "__main__". Do this only for *modules*
// however - packages never have their names replaced, instead
// they're -m'ed using a special __main__ submodule in them. (This all
// apparently is done to not touch the package name itself, which is
// important for future imports).
if (override_main && stat != MP_IMPORT_STAT_DIR) {
mp_obj_module_t *o = MP_OBJ_TO_PTR(module_obj);
mp_obj_dict_store(MP_OBJ_FROM_PTR(o->globals), MP_OBJ_NEW_QSTR(MP_QSTR___name__), MP_OBJ_NEW_QSTR(MP_QSTR___main__));
#if MICROPY_CPYTHON_COMPAT
// Store module as "__main__" in the dictionary of loaded modules (returned by sys.modules).
mp_obj_dict_store(MP_OBJ_FROM_PTR(&MP_STATE_VM(mp_loaded_modules_dict)), MP_OBJ_NEW_QSTR(MP_QSTR___main__), module_obj);
// Store real name in "__main__" attribute. Need this for
// resolving relative imports later. "__main__ was chosen
// semi-randonly, to reuse existing qstr's.
mp_obj_dict_store(MP_OBJ_FROM_PTR(o->globals), MP_OBJ_NEW_QSTR(MP_QSTR___main__), MP_OBJ_NEW_QSTR(full_mod_name));
#endif
}
#endif // MICROPY_MODULE_OVERRIDE_MAIN_IMPORT
if (stat == MP_IMPORT_STAT_DIR) {
// Directory -- execute "path/__init__.py".
DEBUG_printf("%.*s is dir\n", (int)vstr_len(path), vstr_str(path));
// Store the __path__ attribute onto this module.
// https://docs.python.org/3/reference/import.html
// "Specifically, any module that contains a __path__ attribute is considered a package."
mp_store_attr(module_obj, MP_QSTR___path__, mp_obj_new_str(vstr_str(path), vstr_len(path)));
size_t orig_path_len = path->len;
vstr_add_str(path, PATH_SEP_CHAR "__init__.py");
if (stat_file_py_or_mpy(path) == MP_IMPORT_STAT_FILE) {
do_load(module_obj, path);
} else {
// No-op. Nothing to load.
// mp_warning("%s is imported as namespace package", vstr_str(&path));
}
// Remove /__init__.py suffix.
path->len = orig_path_len;
} else { // MP_IMPORT_STAT_FILE
// File -- execute "path.(m)py".
do_load(module_obj, path);
// Note: This should be the last component in the import path. If
// there are remaining components then it's an ImportError
// because the current path(the module that was just loaded) is
// not a package. This will be caught on the next iteration
// because the file will not exist.
}
}
if (outer_module_obj != MP_OBJ_NULL && VERIFY_PTR(MP_OBJ_TO_PTR(outer_module_obj))) {
// If it's a sub-module (not a built-in one), then make it available on
// the parent module.
mp_store_attr(outer_module_obj, level_mod_name, module_obj);
}
return module_obj;
} }
mp_obj_t mp_builtin___import__(size_t n_args, const mp_obj_t *args) { mp_obj_t mp_builtin___import__(size_t n_args, const mp_obj_t *args) {
@ -267,14 +471,28 @@ mp_obj_t mp_builtin___import__(size_t n_args, const mp_obj_t *args) {
DEBUG_printf("__import__:\n"); DEBUG_printf("__import__:\n");
for (size_t i = 0; i < n_args; i++) { for (size_t i = 0; i < n_args; i++) {
DEBUG_printf(" "); DEBUG_printf(" ");
mp_obj_print(args[i], PRINT_REPR); mp_obj_print_helper(MICROPY_DEBUG_PRINTER, args[i], PRINT_REPR);
DEBUG_printf("\n"); DEBUG_printf("\n");
} }
#endif #endif
mp_obj_t module_name = args[0]; // This is the import path, with any leading dots stripped.
// "import foo.bar" --> module_name="foo.bar"
// "from foo.bar import baz" --> module_name="foo.bar"
// "from . import foo" --> module_name=""
// "from ...foo.bar import baz" --> module_name="foo.bar"
mp_obj_t module_name_obj = args[0];
// These are the imported names.
// i.e. "from foo.bar import baz, zap" --> fromtuple=("baz", "zap",)
// Note: There's a special case on the Unix port, where this is set to mp_const_false which means that it's __main__.
mp_obj_t fromtuple = mp_const_none; mp_obj_t fromtuple = mp_const_none;
// Level is the number of leading dots in a relative import.
// i.e. "from . import foo" --> level=1
// i.e. "from ...foo.bar import baz" --> level=3
mp_int_t level = 0; mp_int_t level = 0;
if (n_args >= 4) { if (n_args >= 4) {
fromtuple = args[3]; fromtuple = args[3];
if (n_args >= 5) { if (n_args >= 5) {
@ -285,226 +503,64 @@ mp_obj_t mp_builtin___import__(size_t n_args, const mp_obj_t *args) {
} }
} }
size_t mod_len; size_t module_name_len;
const char *mod_str = mp_obj_str_get_data(module_name, &mod_len); const char *module_name = mp_obj_str_get_data(module_name_obj, &module_name_len);
if (level != 0) { if (level != 0) {
// What we want to do here is to take name of current module, // Turn "foo.bar" into "<current module minus 3 components>.foo.bar".
// chop <level> trailing components, and concatenate with passed-in evaluate_relative_import(level, &module_name, &module_name_len);
// module name, thus resolving relative import name into absolute.
// This even appears to be correct per
// http://legacy.python.org/dev/peps/pep-0328/#relative-imports-and-name
// "Relative imports use a module's __name__ attribute to determine that
// module's position in the package hierarchy."
level--;
mp_obj_t this_name_q = mp_obj_dict_get(MP_OBJ_FROM_PTR(mp_globals_get()), MP_OBJ_NEW_QSTR(MP_QSTR___name__));
assert(this_name_q != MP_OBJ_NULL);
#if MICROPY_CPYTHON_COMPAT
if (MP_OBJ_QSTR_VALUE(this_name_q) == MP_QSTR___main__) {
// This is a module run by -m command-line switch, get its real name from backup attribute
this_name_q = mp_obj_dict_get(MP_OBJ_FROM_PTR(mp_globals_get()), MP_OBJ_NEW_QSTR(MP_QSTR___main__));
}
#endif
mp_map_t *globals_map = &mp_globals_get()->map;
mp_map_elem_t *elem = mp_map_lookup(globals_map, MP_OBJ_NEW_QSTR(MP_QSTR___path__), MP_MAP_LOOKUP);
bool is_pkg = (elem != NULL);
#if DEBUG_PRINT
DEBUG_printf("Current module/package: ");
mp_obj_print(this_name_q, PRINT_REPR);
DEBUG_printf(", is_package: %d", is_pkg);
DEBUG_printf("\n");
#endif
size_t this_name_l;
const char *this_name = mp_obj_str_get_data(this_name_q, &this_name_l);
const char *p = this_name + this_name_l;
if (!is_pkg) {
// We have module, but relative imports are anchored at package, so
// go there.
chop_component(this_name, &p);
} }
while (level--) { if (module_name_len == 0) {
chop_component(this_name, &p);
}
// We must have some component left over to import from
if (p == this_name) {
mp_raise_ImportError(MP_ERROR_TEXT("cannot perform relative import"));
}
uint new_mod_l = (mod_len == 0 ? (size_t)(p - this_name) : (size_t)(p - this_name) + 1 + mod_len);
char *new_mod = mp_local_alloc(new_mod_l);
memcpy(new_mod, this_name, p - this_name);
if (mod_len != 0) {
new_mod[p - this_name] = '.';
memcpy(new_mod + (p - this_name) + 1, mod_str, mod_len);
}
qstr new_mod_q = qstr_from_strn(new_mod, new_mod_l);
mp_local_free(new_mod);
DEBUG_printf("Resolved base name for relative import: '%s'\n", qstr_str(new_mod_q));
module_name = MP_OBJ_NEW_QSTR(new_mod_q);
mod_str = qstr_str(new_mod_q);
mod_len = new_mod_l;
}
if (mod_len == 0) {
mp_raise_ValueError(NULL); mp_raise_ValueError(NULL);
} }
// check if module already exists DEBUG_printf("Starting module search for '%s'\n", module_name);
qstr module_name_qstr = mp_obj_str_get_qstr(module_name);
mp_obj_t module_obj = mp_module_get(module_name_qstr);
if (module_obj != MP_OBJ_NULL) {
DEBUG_printf("Module already loaded\n");
// If it's not a package, return module right away
char *p = strchr(mod_str, '.');
if (p == NULL) {
return module_obj;
}
// If fromlist is not empty, return leaf module
if (fromtuple != mp_const_none) {
return module_obj;
}
// Otherwise, we need to return top-level package
qstr pkg_name = qstr_from_strn(mod_str, p - mod_str);
return mp_module_get(pkg_name);
}
DEBUG_printf("Module not yet loaded\n");
uint last = 0;
VSTR_FIXED(path, MICROPY_ALLOC_PATH_MAX) VSTR_FIXED(path, MICROPY_ALLOC_PATH_MAX)
module_obj = MP_OBJ_NULL;
mp_obj_t top_module_obj = MP_OBJ_NULL; mp_obj_t top_module_obj = MP_OBJ_NULL;
mp_obj_t outer_module_obj = MP_OBJ_NULL; mp_obj_t outer_module_obj = MP_OBJ_NULL;
uint i;
for (i = 1; i <= mod_len; i++) {
if (i == mod_len || mod_str[i] == '.') {
// create a qstr for the module name up to this depth
qstr mod_name = qstr_from_strn(mod_str, i);
DEBUG_printf("Processing module: %s\n", qstr_str(mod_name));
DEBUG_printf("Previous path: =%.*s=\n", vstr_len(&path), vstr_str(&path));
// find the file corresponding to the module name // Search for the end of each component.
mp_import_stat_t stat; size_t current_component_start = 0;
if (vstr_len(&path) == 0) { for (size_t i = 1; i <= module_name_len; i++) {
// first module in the dotted-name; search for a directory or file if (i == module_name_len || module_name[i] == '.') {
DEBUG_printf("Find file =%.*s=\n", vstr_len(&path), vstr_str(&path)); // The module name up to this depth (e.g. foo.bar.baz).
stat = find_file(mod_str, i, &path); qstr full_mod_name = qstr_from_strn(module_name, i);
} else { // The current level name (e.g. baz).
// latter module in the dotted-name; append to path qstr level_mod_name = qstr_from_strn(module_name + current_component_start, i - current_component_start);
vstr_add_char(&path, PATH_SEP_CHAR);
vstr_add_strn(&path, mod_str + last, i - last);
stat = stat_dir_or_file(&path);
}
DEBUG_printf("Current path: %.*s\n", vstr_len(&path), vstr_str(&path));
if (stat == MP_IMPORT_STAT_NO_EXIST) { DEBUG_printf("Processing module: '%s' at level '%s'\n", qstr_str(full_mod_name), qstr_str(level_mod_name));
// This is just the module name after the previous . DEBUG_printf("Previous path: =%.*s=\n", (int)vstr_len(&path), vstr_str(&path));
qstr current_module_name = qstr_from_strn(mod_str + last, i - last);
mp_map_elem_t *el = NULL;
if (outer_module_obj == MP_OBJ_NULL) {
el = mp_map_lookup((mp_map_t *)&mp_builtin_module_map,
MP_OBJ_NEW_QSTR(current_module_name),
MP_MAP_LOOKUP);
} else {
el = mp_map_lookup(&((mp_obj_module_t *)outer_module_obj)->globals->map,
MP_OBJ_NEW_QSTR(current_module_name),
MP_MAP_LOOKUP);
}
if (el != NULL && mp_obj_is_type(el->value, &mp_type_module)) { #if MICROPY_MODULE_OVERRIDE_MAIN_IMPORT
module_obj = el->value; // On unix, if this is being loaded via -m (magic mp_const_false),
mp_module_call_init(mod_name, module_obj); // then handle that if it's the final component.
} else { bool override_main = (i == module_name_len && fromtuple == mp_const_false);
// couldn't find the file, so fail
#if MICROPY_ERROR_REPORTING <= MICROPY_ERROR_REPORTING_TERSE
mp_raise_ImportError(MP_ERROR_TEXT("module not found"));
#else #else
mp_raise_msg_varg(&mp_type_ImportError, bool override_main = false;
MP_ERROR_TEXT("no module named '%q'"), mod_name);
#endif #endif
}
} else {
// found the file, so get the module
module_obj = mp_module_get(mod_name);
}
if (module_obj == MP_OBJ_NULL) { // Import this module.
// module not already loaded, so load it! mp_obj_t module_obj = process_import_at_level(full_mod_name, level_mod_name, outer_module_obj, &path, override_main);
module_obj = mp_obj_new_module(mod_name); // Set this as the parent module, and remember the top-level module if it's the first.
// if args[3] (fromtuple) has magic value False, set up
// this module for command-line "-m" option (set module's
// name to __main__ instead of real name). Do this only
// for *modules* however - packages never have their names
// replaced, instead they're -m'ed using a special __main__
// submodule in them. (This all apparently is done to not
// touch package name itself, which is important for future
// imports).
if (i == mod_len && fromtuple == mp_const_false && stat != MP_IMPORT_STAT_DIR) {
mp_obj_module_t *o = MP_OBJ_TO_PTR(module_obj);
mp_obj_dict_store(MP_OBJ_FROM_PTR(o->globals), MP_OBJ_NEW_QSTR(MP_QSTR___name__), MP_OBJ_NEW_QSTR(MP_QSTR___main__));
#if MICROPY_CPYTHON_COMPAT
// Store module as "__main__" in the dictionary of loaded modules (returned by sys.modules).
mp_obj_dict_store(MP_OBJ_FROM_PTR(&MP_STATE_VM(mp_loaded_modules_dict)), MP_OBJ_NEW_QSTR(MP_QSTR___main__), module_obj);
// Store real name in "__main__" attribute. Chosen semi-randonly, to reuse existing qstr's.
mp_obj_dict_store(MP_OBJ_FROM_PTR(o->globals), MP_OBJ_NEW_QSTR(MP_QSTR___main__), MP_OBJ_NEW_QSTR(mod_name));
#endif
}
if (stat == MP_IMPORT_STAT_DIR) {
DEBUG_printf("%.*s is dir\n", vstr_len(&path), vstr_str(&path));
// https://docs.python.org/3/reference/import.html
// "Specifically, any module that contains a __path__ attribute is considered a package."
mp_store_attr(module_obj, MP_QSTR___path__, mp_obj_new_str(vstr_str(&path), vstr_len(&path)));
size_t orig_path_len = path.len;
vstr_add_char(&path, PATH_SEP_CHAR);
vstr_add_str(&path, "__init__.py");
if (stat_file_py_or_mpy(&path) != MP_IMPORT_STAT_FILE) {
// mp_warning("%s is imported as namespace package", vstr_str(&path));
} else {
do_load(module_obj, &path);
}
path.len = orig_path_len;
} else { // MP_IMPORT_STAT_FILE
do_load(module_obj, &path);
// This should be the last component in the import path. If there are
// remaining components then it's an ImportError because the current path
// (the module that was just loaded) is not a package. This will be caught
// on the next iteration because the file will not exist.
}
// Loading a module thrashes the heap significantly so we explicitly clean up
// afterwards.
gc_collect();
}
if (outer_module_obj != MP_OBJ_NULL && VERIFY_PTR(MP_OBJ_TO_PTR(outer_module_obj))) {
qstr s = qstr_from_strn(mod_str + last, i - last);
mp_store_attr(outer_module_obj, s, module_obj);
// The above store can cause a dictionary rehash and new allocation. So,
// lets make sure the globals dictionary is still long lived.
mp_obj_module_set_globals(outer_module_obj,
make_dict_long_lived(mp_obj_module_get_globals(outer_module_obj), 10));
}
outer_module_obj = module_obj; outer_module_obj = module_obj;
if (top_module_obj == MP_OBJ_NULL) { if (top_module_obj == MP_OBJ_NULL) {
top_module_obj = module_obj; top_module_obj = module_obj;
} }
last = i + 1;
current_component_start = i + 1;
} }
} }
// If fromlist is not empty, return leaf module
if (fromtuple != mp_const_none) { if (fromtuple != mp_const_none) {
return module_obj; // If fromtuple is not empty, return leaf module
} return outer_module_obj;
} else {
// Otherwise, we need to return top-level package // Otherwise, we need to return top-level package
return top_module_obj; return top_module_obj;
}
} }
#else // MICROPY_ENABLE_EXTERNAL_IMPORT #else // MICROPY_ENABLE_EXTERNAL_IMPORT
@ -517,17 +573,19 @@ mp_obj_t mp_builtin___import__(size_t n_args, const mp_obj_t *args) {
// Check if module already exists, and return it if it does // Check if module already exists, and return it if it does
qstr module_name_qstr = mp_obj_str_get_qstr(args[0]); qstr module_name_qstr = mp_obj_str_get_qstr(args[0]);
mp_obj_t module_obj = mp_module_get(module_name_qstr); mp_obj_t module_obj = mp_module_get_loaded_or_builtin(module_name_qstr);
if (module_obj != MP_OBJ_NULL) { if (module_obj != MP_OBJ_NULL) {
return module_obj; return module_obj;
} }
#if MICROPY_MODULE_WEAK_LINKS #if MICROPY_MODULE_WEAK_LINKS
// Check if there is a weak link to this module // Check if there is a weak link to this module
module_obj = mp_module_search_umodule(qstr_str(module_name_qstr)); char umodule_buf[MICROPY_ALLOC_PATH_MAX];
umodule_buf[0] = 'u';
strcpy(umodule_buf + 1, args[0]);
qstr umodule_name_qstr = qstr_from_str(umodule_buf);
module_obj = mp_module_get_loaded_or_builtin(umodule_name_qstr);
if (module_obj != MP_OBJ_NULL) { if (module_obj != MP_OBJ_NULL) {
// Found weak-linked module
mp_module_call_init(module_name_qstr, module_obj);
return module_obj; return module_obj;
} }
#endif #endif

11
py/circuitpy_mpconfig.h
View File

@ -541,5 +541,16 @@ void supervisor_run_background_tasks_if_tick(void);
#define USB_MIDI_EP_NUM_IN (0) #define USB_MIDI_EP_NUM_IN (0)
#endif #endif
#ifndef MICROPY_WRAP_MP_MAP_LOOKUP
#define MICROPY_WRAP_MP_MAP_LOOKUP PLACE_IN_ITCM
#endif
#ifndef MICROPY_WRAP_MP_BINARY_OP
#define MICROPY_WRAP_MP_BINARY_OP PLACE_IN_ITCM
#endif
#ifndef MICROPY_WRAP_MP_EXECUTE_BYTECODE
#define MICROPY_WRAP_MP_EXECUTE_BYTECODE PLACE_IN_ITCM
#endif
#endif // __INCLUDED_MPCONFIG_CIRCUITPY_H #endif // __INCLUDED_MPCONFIG_CIRCUITPY_H

155
py/compile.c
View File

@ -61,6 +61,12 @@ typedef enum {
#undef DEF_RULE_NC #undef DEF_RULE_NC
} pn_kind_t; } pn_kind_t;
// Whether a mp_parse_node_struct_t that has pns->kind == PN_testlist_comp
// corresponds to a list comprehension or generator.
#define MP_PARSE_NODE_TESTLIST_COMP_HAS_COMP_FOR(pns) \
(MP_PARSE_NODE_STRUCT_NUM_NODES(pns) == 2 && \
MP_PARSE_NODE_IS_STRUCT_KIND(pns->nodes[1], PN_comp_for))
#define NEED_METHOD_TABLE MICROPY_EMIT_NATIVE #define NEED_METHOD_TABLE MICROPY_EMIT_NATIVE
#if NEED_METHOD_TABLE #if NEED_METHOD_TABLE
@ -319,25 +325,13 @@ STATIC void compile_delete_id(compiler_t *comp, qstr qst) {
} }
} }
STATIC void c_tuple(compiler_t *comp, mp_parse_node_t pn, mp_parse_node_struct_t *pns_list) {
int total = 0;
if (!MP_PARSE_NODE_IS_NULL(pn)) {
compile_node(comp, pn);
total += 1;
}
if (pns_list != NULL) {
int n = MP_PARSE_NODE_STRUCT_NUM_NODES(pns_list);
for (int i = 0; i < n; i++) {
compile_node(comp, pns_list->nodes[i]);
}
total += n;
}
EMIT_ARG(build, total, MP_EMIT_BUILD_TUPLE);
}
STATIC void compile_generic_tuple(compiler_t *comp, mp_parse_node_struct_t *pns) { STATIC void compile_generic_tuple(compiler_t *comp, mp_parse_node_struct_t *pns) {
// a simple tuple expression // a simple tuple expression
c_tuple(comp, MP_PARSE_NODE_NULL, pns); size_t num_nodes = MP_PARSE_NODE_STRUCT_NUM_NODES(pns);
for (size_t i = 0; i < num_nodes; i++) {
compile_node(comp, pns->nodes[i]);
}
EMIT_ARG(build, num_nodes, MP_EMIT_BUILD_TUPLE);
} }
STATIC void c_if_cond(compiler_t *comp, mp_parse_node_t pn, bool jump_if, int label) { STATIC void c_if_cond(compiler_t *comp, mp_parse_node_t pn, bool jump_if, int label) {
@ -454,21 +448,14 @@ STATIC void c_assign_atom_expr(compiler_t *comp, mp_parse_node_struct_t *pns, as
compile_syntax_error(comp, (mp_parse_node_t)pns, MP_ERROR_TEXT("can't assign to expression")); compile_syntax_error(comp, (mp_parse_node_t)pns, MP_ERROR_TEXT("can't assign to expression"));
} }
// we need to allow for a caller passing in 1 initial node (node_head) followed by an array of nodes (nodes_tail) STATIC void c_assign_tuple(compiler_t *comp, uint num_tail, mp_parse_node_t *nodes_tail) {
STATIC void c_assign_tuple(compiler_t *comp, mp_parse_node_t node_head, uint num_tail, mp_parse_node_t *nodes_tail) {
uint num_head = (node_head == MP_PARSE_NODE_NULL) ? 0 : 1;
// look for star expression // look for star expression
uint have_star_index = -1; uint have_star_index = -1;
if (num_head != 0 && MP_PARSE_NODE_IS_STRUCT_KIND(node_head, PN_star_expr)) {
EMIT_ARG(unpack_ex, 0, num_tail);
have_star_index = 0;
}
for (uint i = 0; i < num_tail; i++) { for (uint i = 0; i < num_tail; i++) {
if (MP_PARSE_NODE_IS_STRUCT_KIND(nodes_tail[i], PN_star_expr)) { if (MP_PARSE_NODE_IS_STRUCT_KIND(nodes_tail[i], PN_star_expr)) {
if (have_star_index == (uint)-1) { if (have_star_index == (uint)-1) {
EMIT_ARG(unpack_ex, num_head + i, num_tail - i - 1); EMIT_ARG(unpack_ex, i, num_tail - i - 1);
have_star_index = num_head + i; have_star_index = i;
} else { } else {
compile_syntax_error(comp, nodes_tail[i], MP_ERROR_TEXT("multiple *x in assignment")); compile_syntax_error(comp, nodes_tail[i], MP_ERROR_TEXT("multiple *x in assignment"));
return; return;
@ -476,17 +463,10 @@ STATIC void c_assign_tuple(compiler_t *comp, mp_parse_node_t node_head, uint num
} }
} }
if (have_star_index == (uint)-1) { if (have_star_index == (uint)-1) {
EMIT_ARG(unpack_sequence, num_head + num_tail); EMIT_ARG(unpack_sequence, num_tail);
}
if (num_head != 0) {
if (0 == have_star_index) {
c_assign(comp, ((mp_parse_node_struct_t *)node_head)->nodes[0], ASSIGN_STORE);
} else {
c_assign(comp, node_head, ASSIGN_STORE);
}
} }
for (uint i = 0; i < num_tail; i++) { for (uint i = 0; i < num_tail; i++) {
if (num_head + i == have_star_index) { if (i == have_star_index) {
c_assign(comp, ((mp_parse_node_struct_t *)nodes_tail[i])->nodes[0], ASSIGN_STORE); c_assign(comp, ((mp_parse_node_struct_t *)nodes_tail[i])->nodes[0], ASSIGN_STORE);
} else { } else {
c_assign(comp, nodes_tail[i], ASSIGN_STORE); c_assign(comp, nodes_tail[i], ASSIGN_STORE);
@ -528,7 +508,7 @@ STATIC void c_assign(compiler_t *comp, mp_parse_node_t pn, assign_kind_t assign_
if (assign_kind != ASSIGN_STORE) { if (assign_kind != ASSIGN_STORE) {
goto cannot_assign; goto cannot_assign;
} }
c_assign_tuple(comp, MP_PARSE_NODE_NULL, MP_PARSE_NODE_STRUCT_NUM_NODES(pns), pns->nodes); c_assign_tuple(comp, MP_PARSE_NODE_STRUCT_NUM_NODES(pns), pns->nodes);
break; break;
case PN_atom_paren: case PN_atom_paren:
@ -553,13 +533,13 @@ STATIC void c_assign(compiler_t *comp, mp_parse_node_t pn, assign_kind_t assign_
} }
if (MP_PARSE_NODE_IS_NULL(pns->nodes[0])) { if (MP_PARSE_NODE_IS_NULL(pns->nodes[0])) {
// empty list, assignment allowed // empty list, assignment allowed
c_assign_tuple(comp, MP_PARSE_NODE_NULL, 0, NULL); c_assign_tuple(comp, 0, NULL);
} else if (MP_PARSE_NODE_IS_STRUCT_KIND(pns->nodes[0], PN_testlist_comp)) { } else if (MP_PARSE_NODE_IS_STRUCT_KIND(pns->nodes[0], PN_testlist_comp)) {
pns = (mp_parse_node_struct_t *)pns->nodes[0]; pns = (mp_parse_node_struct_t *)pns->nodes[0];
goto testlist_comp; goto testlist_comp;
} else { } else {
// brackets around 1 item // brackets around 1 item
c_assign_tuple(comp, pns->nodes[0], 0, NULL); c_assign_tuple(comp, 1, pns->nodes);
} }
break; break;
@ -570,27 +550,10 @@ STATIC void c_assign(compiler_t *comp, mp_parse_node_t pn, assign_kind_t assign_
testlist_comp: testlist_comp:
// lhs is a sequence // lhs is a sequence
if (MP_PARSE_NODE_IS_STRUCT(pns->nodes[1])) { if (MP_PARSE_NODE_TESTLIST_COMP_HAS_COMP_FOR(pns)) {
mp_parse_node_struct_t *pns2 = (mp_parse_node_struct_t *)pns->nodes[1];
if (MP_PARSE_NODE_STRUCT_KIND(pns2) == PN_testlist_comp_3b) {
// sequence of one item, with trailing comma
assert(MP_PARSE_NODE_IS_NULL(pns2->nodes[0]));
c_assign_tuple(comp, pns->nodes[0], 0, NULL);
} else if (MP_PARSE_NODE_STRUCT_KIND(pns2) == PN_testlist_comp_3c) {
// sequence of many items
uint n = MP_PARSE_NODE_STRUCT_NUM_NODES(pns2);
c_assign_tuple(comp, pns->nodes[0], n, pns2->nodes);
} else if (MP_PARSE_NODE_STRUCT_KIND(pns2) == PN_comp_for) {
goto cannot_assign; goto cannot_assign;
} else {
// sequence with 2 items
goto sequence_with_2_items;
}
} else {
// sequence with 2 items
sequence_with_2_items:
c_assign_tuple(comp, MP_PARSE_NODE_NULL, 2, pns->nodes);
} }
c_assign_tuple(comp, MP_PARSE_NODE_STRUCT_NUM_NODES(pns), pns->nodes);
return; return;
} }
return; return;
@ -993,32 +956,11 @@ STATIC void c_del_stmt(compiler_t *comp, mp_parse_node_t pn) {
} else { } else {
assert(MP_PARSE_NODE_IS_STRUCT_KIND(pn, PN_testlist_comp)); assert(MP_PARSE_NODE_IS_STRUCT_KIND(pn, PN_testlist_comp));
mp_parse_node_struct_t *pns = (mp_parse_node_struct_t *)pn; mp_parse_node_struct_t *pns = (mp_parse_node_struct_t *)pn;
// TODO perhaps factorise testlist_comp code with other uses of PN_testlist_comp if (MP_PARSE_NODE_TESTLIST_COMP_HAS_COMP_FOR(pns)) {
if (MP_PARSE_NODE_IS_STRUCT(pns->nodes[1])) {
mp_parse_node_struct_t *pns1 = (mp_parse_node_struct_t *)pns->nodes[1];
if (MP_PARSE_NODE_STRUCT_KIND(pns1) == PN_testlist_comp_3b) {
// sequence of one item, with trailing comma
assert(MP_PARSE_NODE_IS_NULL(pns1->nodes[0]));
c_del_stmt(comp, pns->nodes[0]);
} else if (MP_PARSE_NODE_STRUCT_KIND(pns1) == PN_testlist_comp_3c) {
// sequence of many items
int n = MP_PARSE_NODE_STRUCT_NUM_NODES(pns1);
c_del_stmt(comp, pns->nodes[0]);
for (int i = 0; i < n; i++) {
c_del_stmt(comp, pns1->nodes[i]);
}
} else if (MP_PARSE_NODE_STRUCT_KIND(pns1) == PN_comp_for) {
goto cannot_delete; goto cannot_delete;
} else {
// sequence with 2 items
goto sequence_with_2_items;
} }
} else { for (size_t i = 0; i < MP_PARSE_NODE_STRUCT_NUM_NODES(pns); ++i) {
// sequence with 2 items c_del_stmt(comp, pns->nodes[i]);
sequence_with_2_items:
c_del_stmt(comp, pns->nodes[0]);
c_del_stmt(comp, pns->nodes[1]);
} }
} }
} else { } else {
@ -2502,31 +2444,16 @@ STATIC void compile_comprehension(compiler_t *comp, mp_parse_node_struct_t *pns,
STATIC void compile_atom_paren(compiler_t *comp, mp_parse_node_struct_t *pns) { STATIC void compile_atom_paren(compiler_t *comp, mp_parse_node_struct_t *pns) {
if (MP_PARSE_NODE_IS_NULL(pns->nodes[0])) { if (MP_PARSE_NODE_IS_NULL(pns->nodes[0])) {
// an empty tuple // an empty tuple
c_tuple(comp, MP_PARSE_NODE_NULL, NULL); EMIT_ARG(build, 0, MP_EMIT_BUILD_TUPLE);
} else { } else {
assert(MP_PARSE_NODE_IS_STRUCT_KIND(pns->nodes[0], PN_testlist_comp)); assert(MP_PARSE_NODE_IS_STRUCT_KIND(pns->nodes[0], PN_testlist_comp));
pns = (mp_parse_node_struct_t *)pns->nodes[0]; pns = (mp_parse_node_struct_t *)pns->nodes[0];
assert(!MP_PARSE_NODE_IS_NULL(pns->nodes[1])); if (MP_PARSE_NODE_TESTLIST_COMP_HAS_COMP_FOR(pns)) {
if (MP_PARSE_NODE_IS_STRUCT(pns->nodes[1])) {
mp_parse_node_struct_t *pns2 = (mp_parse_node_struct_t *)pns->nodes[1];
if (MP_PARSE_NODE_STRUCT_KIND(pns2) == PN_testlist_comp_3b) {
// tuple of one item, with trailing comma
assert(MP_PARSE_NODE_IS_NULL(pns2->nodes[0]));
c_tuple(comp, pns->nodes[0], NULL);
} else if (MP_PARSE_NODE_STRUCT_KIND(pns2) == PN_testlist_comp_3c) {
// tuple of many items
c_tuple(comp, pns->nodes[0], pns2);
} else if (MP_PARSE_NODE_STRUCT_KIND(pns2) == PN_comp_for) {
// generator expression // generator expression
compile_comprehension(comp, pns, SCOPE_GEN_EXPR); compile_comprehension(comp, pns, SCOPE_GEN_EXPR);
} else { } else {
// tuple with 2 items // tuple with N items
goto tuple_with_2_items; compile_generic_tuple(comp, pns);
}
} else {
// tuple with 2 items
tuple_with_2_items:
c_tuple(comp, MP_PARSE_NODE_NULL, pns);
} }
} }
} }
@ -2537,31 +2464,13 @@ STATIC void compile_atom_bracket(compiler_t *comp, mp_parse_node_struct_t *pns)
EMIT_ARG(build, 0, MP_EMIT_BUILD_LIST); EMIT_ARG(build, 0, MP_EMIT_BUILD_LIST);
} else if (MP_PARSE_NODE_IS_STRUCT_KIND(pns->nodes[0], PN_testlist_comp)) { } else if (MP_PARSE_NODE_IS_STRUCT_KIND(pns->nodes[0], PN_testlist_comp)) {
mp_parse_node_struct_t *pns2 = (mp_parse_node_struct_t *)pns->nodes[0]; mp_parse_node_struct_t *pns2 = (mp_parse_node_struct_t *)pns->nodes[0];
if (MP_PARSE_NODE_IS_STRUCT(pns2->nodes[1])) { if (MP_PARSE_NODE_TESTLIST_COMP_HAS_COMP_FOR(pns2)) {
mp_parse_node_struct_t *pns3 = (mp_parse_node_struct_t *)pns2->nodes[1];
if (MP_PARSE_NODE_STRUCT_KIND(pns3) == PN_testlist_comp_3b) {
// list of one item, with trailing comma
assert(MP_PARSE_NODE_IS_NULL(pns3->nodes[0]));
compile_node(comp, pns2->nodes[0]);
EMIT_ARG(build, 1, MP_EMIT_BUILD_LIST);
} else if (MP_PARSE_NODE_STRUCT_KIND(pns3) == PN_testlist_comp_3c) {
// list of many items
compile_node(comp, pns2->nodes[0]);
compile_generic_all_nodes(comp, pns3);
EMIT_ARG(build, 1 + MP_PARSE_NODE_STRUCT_NUM_NODES(pns3), MP_EMIT_BUILD_LIST);
} else if (MP_PARSE_NODE_STRUCT_KIND(pns3) == PN_comp_for) {
// list comprehension // list comprehension
compile_comprehension(comp, pns2, SCOPE_LIST_COMP); compile_comprehension(comp, pns2, SCOPE_LIST_COMP);
} else { } else {
// list with 2 items // list with N items
goto list_with_2_items; compile_generic_all_nodes(comp, pns2);
} EMIT_ARG(build, MP_PARSE_NODE_STRUCT_NUM_NODES(pns2), MP_EMIT_BUILD_LIST);
} else {
// list with 2 items
list_with_2_items:
compile_node(comp, pns2->nodes[0]);
compile_node(comp, pns2->nodes[1]);
EMIT_ARG(build, 2, MP_EMIT_BUILD_LIST);
} }
} else { } else {
// list with 1 item // list with 1 item

2
py/dynruntime.h
View File

@ -141,7 +141,7 @@ static inline mp_obj_t mp_obj_cast_to_native_base_dyn(mp_obj_t self_in, mp_const
if (MP_OBJ_FROM_PTR(self_type) == native_type) { if (MP_OBJ_FROM_PTR(self_type) == native_type) {
return self_in; return self_in;
} }
mp_parent_t parent = mp_type_get_parent_slot(self_type); const void *parent = mp_type_get_parent_slot(self_type);
if (parent != native_type) { if (parent != native_type) {
// The self_in object is not a direct descendant of native_type, so fail the cast. // The self_in object is not a direct descendant of native_type, so fail the cast.
// This is a very simple version of mp_obj_is_subclass_fast that could be improved. // This is a very simple version of mp_obj_is_subclass_fast that could be improved.

2
py/dynruntime.mk
View File

@ -46,7 +46,6 @@ ifeq ($(ARCH),x86)
# x86 # x86
CROSS = CROSS =
CFLAGS += -m32 -fno-stack-protector CFLAGS += -m32 -fno-stack-protector
MPY_CROSS_FLAGS += -mcache-lookup-bc
MICROPY_FLOAT_IMPL ?= double MICROPY_FLOAT_IMPL ?= double
else ifeq ($(ARCH),x64) else ifeq ($(ARCH),x64)
@ -54,7 +53,6 @@ else ifeq ($(ARCH),x64)
# x64 # x64
CROSS = CROSS =
CFLAGS += -fno-stack-protector CFLAGS += -fno-stack-protector
MPY_CROSS_FLAGS += -mcache-lookup-bc
MICROPY_FLOAT_IMPL ?= double MICROPY_FLOAT_IMPL ?= double
else ifeq ($(ARCH),armv7m) else ifeq ($(ARCH),armv7m)

6
py/emitbc.c
View File

@ -560,9 +560,6 @@ void mp_emit_bc_load_global(emit_t *emit, qstr qst, int kind) {
MP_STATIC_ASSERT(MP_BC_LOAD_NAME + MP_EMIT_IDOP_GLOBAL_GLOBAL == MP_BC_LOAD_GLOBAL); MP_STATIC_ASSERT(MP_BC_LOAD_NAME + MP_EMIT_IDOP_GLOBAL_GLOBAL == MP_BC_LOAD_GLOBAL);
(void)qst; (void)qst;
emit_write_bytecode_byte_qstr(emit, 1, MP_BC_LOAD_NAME + kind, qst); emit_write_bytecode_byte_qstr(emit, 1, MP_BC_LOAD_NAME + kind, qst);
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE_DYNAMIC) {
emit_write_bytecode_raw_byte(emit, 0);
}
} }
void mp_emit_bc_load_method(emit_t *emit, qstr qst, bool is_super) { void mp_emit_bc_load_method(emit_t *emit, qstr qst, bool is_super) {
@ -596,9 +593,6 @@ void mp_emit_bc_attr(emit_t *emit, qstr qst, int kind) {
} }
emit_write_bytecode_byte_qstr(emit, -2, MP_BC_STORE_ATTR, qst); emit_write_bytecode_byte_qstr(emit, -2, MP_BC_STORE_ATTR, qst);
} }
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE_DYNAMIC) {
emit_write_bytecode_raw_byte(emit, 0);
}
} }
void mp_emit_bc_store_local(emit_t *emit, qstr qst, mp_uint_t local_num, int kind) { void mp_emit_bc_store_local(emit_t *emit, qstr qst, mp_uint_t local_num, int kind) {

2
py/emitnative.c
View File

@ -1588,6 +1588,7 @@ STATIC void emit_native_load_subscr(emit_t *emit) {
int reg_base = REG_ARG_1; int reg_base = REG_ARG_1;
int reg_index = REG_ARG_2; int reg_index = REG_ARG_2;
emit_pre_pop_reg_flexible(emit, &vtype_base, &reg_base, reg_index, reg_index); emit_pre_pop_reg_flexible(emit, &vtype_base, &reg_base, reg_index, reg_index);
need_reg_single(emit, REG_RET, 0);
switch (vtype_base) { switch (vtype_base) {
case VTYPE_PTR8: { case VTYPE_PTR8: {
// pointer to 8-bit memory // pointer to 8-bit memory
@ -1651,6 +1652,7 @@ STATIC void emit_native_load_subscr(emit_t *emit) {
int reg_index = REG_ARG_2; int reg_index = REG_ARG_2;
emit_pre_pop_reg_flexible(emit, &vtype_index, &reg_index, REG_ARG_1, REG_ARG_1); emit_pre_pop_reg_flexible(emit, &vtype_index, &reg_index, REG_ARG_1, REG_ARG_1);
emit_pre_pop_reg(emit, &vtype_base, REG_ARG_1); emit_pre_pop_reg(emit, &vtype_base, REG_ARG_1);
need_reg_single(emit, REG_RET, 0);
if (vtype_index != VTYPE_INT && vtype_index != VTYPE_UINT) { if (vtype_index != VTYPE_INT && vtype_index != VTYPE_UINT) {
EMIT_NATIVE_VIPER_TYPE_ERROR(emit, EMIT_NATIVE_VIPER_TYPE_ERROR(emit,
MP_ERROR_TEXT("can't load with '%q' index"), vtype_to_qstr(vtype_index)); MP_ERROR_TEXT("can't load with '%q' index"), vtype_to_qstr(vtype_index));

176
py/frozenmod.c
View File

@ -5,6 +5,7 @@
* *
* Copyright (c) 2015 Paul Sokolovsky * Copyright (c) 2015 Paul Sokolovsky
* SPDX-FileCopyrightText: Copyright (c) 2016 Damien P. George * SPDX-FileCopyrightText: Copyright (c) 2016 Damien P. George
* SPDX-FileCopyrightText: Copyright (c) 2021 Jim Mussared
* *
* Permission is hereby granted, free of charge, to any person obtaining a copy * Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal * of this software and associated documentation files (the "Software"), to deal
@ -31,6 +32,13 @@
#include "py/lexer.h" #include "py/lexer.h"
#include "py/frozenmod.h" #include "py/frozenmod.h"
#if MICROPY_MODULE_FROZEN
// Null-separated frozen file names. All string-type entries are listed first,
// followed by mpy-type entries. Use mp_frozen_str_sizes to determine how
// many string entries.
extern const char mp_frozen_names[];
#if MICROPY_MODULE_FROZEN_STR #if MICROPY_MODULE_FROZEN_STR
#ifndef MICROPY_MODULE_FROZEN_LEXER #ifndef MICROPY_MODULE_FROZEN_LEXER
@ -39,129 +47,89 @@
mp_lexer_t *MICROPY_MODULE_FROZEN_LEXER(qstr src_name, const char *str, mp_uint_t len, mp_uint_t free_len); mp_lexer_t *MICROPY_MODULE_FROZEN_LEXER(qstr src_name, const char *str, mp_uint_t len, mp_uint_t free_len);
#endif #endif
extern const char mp_frozen_str_names[]; // Size in bytes of each string entry, followed by a zero (terminator).
extern const uint32_t mp_frozen_str_sizes[]; extern const uint32_t mp_frozen_str_sizes[];
// Null-separated string content.
extern const char mp_frozen_str_content[]; extern const char mp_frozen_str_content[];
#endif // MICROPY_MODULE_FROZEN_STR
// str_len is length of str. *len is set on on output to size of content
const char *mp_find_frozen_str(const char *str, size_t str_len, size_t *len) {
// If the frozen module pseudo dir (e.g., ".frozen/") is a prefix of str, remove it.
if (strncmp(str, MP_FROZEN_FAKE_DIR_SLASH, MP_FROZEN_FAKE_DIR_SLASH_LENGTH) == 0) {
str = str + MP_FROZEN_FAKE_DIR_SLASH_LENGTH;
str_len = str_len - MP_FROZEN_FAKE_DIR_SLASH_LENGTH;
}
const char *name = mp_frozen_str_names;
size_t offset = 0;
for (int i = 0; *name != 0; i++) {
size_t l = strlen(name);
if (l == str_len && !memcmp(str, name, l)) {
*len = mp_frozen_str_sizes[i];
return mp_frozen_str_content + offset;
}
name += l + 1;
offset += mp_frozen_str_sizes[i] + 1;
}
return NULL;
}
STATIC mp_lexer_t *mp_lexer_frozen_str(const char *str, size_t str_len) {
size_t file_len;
const char *content = mp_find_frozen_str(str, str_len, &file_len);
if (content == NULL) {
return NULL;
}
qstr source = qstr_from_strn(str, str_len);
mp_lexer_t *lex = MICROPY_MODULE_FROZEN_LEXER(source, content, file_len, 0);
return lex;
}
#endif
#if MICROPY_MODULE_FROZEN_MPY #if MICROPY_MODULE_FROZEN_MPY
#include "py/emitglue.h" #include "py/emitglue.h"
extern const char mp_frozen_mpy_names[];
extern const mp_raw_code_t *const mp_frozen_mpy_content[]; extern const mp_raw_code_t *const mp_frozen_mpy_content[];
STATIC const mp_raw_code_t *mp_find_frozen_mpy(const char *str, size_t str_len) { #endif // MICROPY_MODULE_FROZEN_MPY
const char *name = mp_frozen_mpy_names;
for (size_t i = 0; *name != 0; i++) {
size_t l = strlen(name);
if (l == str_len && !memcmp(str, name, l)) {
return mp_frozen_mpy_content[i];
}
name += l + 1;
}
return NULL;
}
#endif // Search for "str" as a frozen entry, returning the stat result
// (no-exist/file/dir), as well as the type (none/str/mpy) and data.
#if MICROPY_MODULE_FROZEN // frozen_type can be NULL if its value isn't needed (and then data is assumed to be NULL).
mp_import_stat_t mp_find_frozen_module(const char *str, int *frozen_type, void **data) {
STATIC mp_import_stat_t mp_frozen_stat_helper(const char *name, const char *str) {
size_t len = strlen(str); size_t len = strlen(str);
const char *name = mp_frozen_names;
if (frozen_type != NULL) {
*frozen_type = MP_FROZEN_NONE;
}
// Count the number of str lengths we have to find how many str entries.
size_t num_str = 0;
#if MICROPY_MODULE_FROZEN_STR && MICROPY_MODULE_FROZEN_MPY
for (const uint32_t *s = mp_frozen_str_sizes; *s != 0; ++s) {
++num_str;
}
#endif
for (size_t i = 0; *name != 0; i++) {
size_t entry_len = strlen(name);
if (entry_len >= len && memcmp(str, name, len) == 0) {
// Query is a prefix of the current entry.
if (entry_len == len) {
// Exact match --> file.
if (frozen_type != NULL) {
#if MICROPY_MODULE_FROZEN_STR
if (i < num_str) {
*frozen_type = MP_FROZEN_STR;
// Use the size table to figure out where this index starts.
size_t offset = 0;
for (size_t j = 0; j < i; ++j) {
offset += mp_frozen_str_sizes[j] + 1;
}
size_t content_len = mp_frozen_str_sizes[i];
const char *content = &mp_frozen_str_content[offset];
// Note: str & len have been updated by find_frozen_entry to strip
// the ".frozen/" prefix (to avoid this being a distinct qstr to
// the original path QSTR in frozen_content.c).
qstr source = qstr_from_strn(str, len);
mp_lexer_t *lex = MICROPY_MODULE_FROZEN_LEXER(source, content, content_len, 0);
*data = lex;
}
#endif
#if MICROPY_MODULE_FROZEN_MPY
if (i >= num_str) {
*frozen_type = MP_FROZEN_MPY;
// Load the corresponding index as a raw_code, taking
// into account any string entries to offset by.
*data = (void *)mp_frozen_mpy_content[i - num_str];
}
#endif
}
for (int i = 0; *name != 0; i++) {
size_t l = strlen(name);
if (l >= len && !memcmp(str, name, len)) {
if (name[len] == 0) {
return MP_IMPORT_STAT_FILE; return MP_IMPORT_STAT_FILE;
} else if (name[len] == '/') { } else if (name[len] == '/') {
// Matches up to directory separator, this is a valid
// directory path.
return MP_IMPORT_STAT_DIR; return MP_IMPORT_STAT_DIR;
} }
} }
name += l + 1; // Skip null separator.
name += entry_len + 1;
} }
return MP_IMPORT_STAT_NO_EXIST;
}
mp_import_stat_t mp_frozen_stat(const char *str) {
mp_import_stat_t stat;
#if MICROPY_MODULE_FROZEN_STR
stat = mp_frozen_stat_helper(mp_frozen_str_names, str);
if (stat != MP_IMPORT_STAT_NO_EXIST) {
return stat;
}
#endif
#if MICROPY_MODULE_FROZEN_MPY
stat = mp_frozen_stat_helper(mp_frozen_mpy_names, str);
if (stat != MP_IMPORT_STAT_NO_EXIST) {
return stat;
}
#endif
return MP_IMPORT_STAT_NO_EXIST; return MP_IMPORT_STAT_NO_EXIST;
} }
int mp_find_frozen_module(const char *str, size_t len, void **data) { #endif // MICROPY_MODULE_FROZEN
// If the frozen module pseudo dir (e.g., ".frozen/") is a prefix of str, remove it.
if (strncmp(str, MP_FROZEN_FAKE_DIR_SLASH, MP_FROZEN_FAKE_DIR_SLASH_LENGTH) == 0) {
str = str + MP_FROZEN_FAKE_DIR_SLASH_LENGTH;
len = len - MP_FROZEN_FAKE_DIR_SLASH_LENGTH;
}
#if MICROPY_MODULE_FROZEN_STR
mp_lexer_t *lex = mp_lexer_frozen_str(str, len);
if (lex != NULL) {
*data = lex;
return MP_FROZEN_STR;
}
#endif
#if MICROPY_MODULE_FROZEN_MPY
const mp_raw_code_t *rc = mp_find_frozen_mpy(str, len);
if (rc != NULL) {
*data = (void *)rc;
return MP_FROZEN_MPY;
}
#endif
return MP_FROZEN_NONE;
}
#endif

14
py/frozenmod.h
View File

@ -35,18 +35,6 @@ enum {
MP_FROZEN_MPY, MP_FROZEN_MPY,
}; };
// Frozen modules are in a pseudo-directory, so sys.path can control how they're found. mp_import_stat_t mp_find_frozen_module(const char *str, int *frozen_type, void **data);
#define MP_FROZEN_FAKE_DIR ".frozen"
#define MP_FROZEN_FAKE_DIR_LENGTH (sizeof(MP_FROZEN_FAKE_DIR) - 1)
#define MP_FROZEN_FAKE_DIR_SLASH (MP_FROZEN_FAKE_DIR "/")
#define MP_FROZEN_FAKE_DIR_SLASH_LENGTH (sizeof(MP_FROZEN_FAKE_DIR_SLASH) - 1)
// This should match MP_FROZEN_FAKE_DIR.
#define MP_FROZEN_FAKE_DIR_QSTR MP_QSTR__dot_frozen
int mp_find_frozen_module(const char *str, size_t len, void **data);
const char *mp_find_frozen_str(const char *str, size_t str_len, size_t *len);
mp_import_stat_t mp_frozen_stat(const char *str);
#endif // MICROPY_INCLUDED_PY_FROZENMOD_H #endif // MICROPY_INCLUDED_PY_FROZENMOD_H

5
py/gc.c
View File

@ -238,6 +238,7 @@ STATIC void gc_mark_subtree(size_t block) {
// Start with the block passed in the argument. // Start with the block passed in the argument.
size_t sp = 0; size_t sp = 0;
for (;;) { for (;;) {
MICROPY_GC_HOOK_LOOP
// work out number of consecutive blocks in the chain starting with this one // work out number of consecutive blocks in the chain starting with this one
size_t n_blocks = 0; size_t n_blocks = 0;
do { do {
@ -247,6 +248,7 @@ STATIC void gc_mark_subtree(size_t block) {
// check this block's children // check this block's children
void **ptrs = (void **)PTR_FROM_BLOCK(block); void **ptrs = (void **)PTR_FROM_BLOCK(block);
for (size_t i = n_blocks * BYTES_PER_BLOCK / sizeof(void *); i > 0; i--, ptrs++) { for (size_t i = n_blocks * BYTES_PER_BLOCK / sizeof(void *); i > 0; i--, ptrs++) {
MICROPY_GC_HOOK_LOOP
void *ptr = *ptrs; void *ptr = *ptrs;
if (VERIFY_PTR(ptr)) { if (VERIFY_PTR(ptr)) {
// Mark and push this pointer // Mark and push this pointer
@ -280,6 +282,7 @@ STATIC void gc_deal_with_stack_overflow(void) {
// scan entire memory looking for blocks which have been marked but not their children // scan entire memory looking for blocks which have been marked but not their children
for (size_t block = 0; block < MP_STATE_MEM(gc_alloc_table_byte_len) * BLOCKS_PER_ATB; block++) { for (size_t block = 0; block < MP_STATE_MEM(gc_alloc_table_byte_len) * BLOCKS_PER_ATB; block++) {
MICROPY_GC_HOOK_LOOP
// trace (again) if mark bit set // trace (again) if mark bit set
if (ATB_GET_KIND(block) == AT_MARK) { if (ATB_GET_KIND(block) == AT_MARK) {
gc_mark_subtree(block); gc_mark_subtree(block);
@ -295,6 +298,7 @@ STATIC void gc_sweep(void) {
// free unmarked heads and their tails // free unmarked heads and their tails
int free_tail = 0; int free_tail = 0;
for (size_t block = 0; block < MP_STATE_MEM(gc_alloc_table_byte_len) * BLOCKS_PER_ATB; block++) { for (size_t block = 0; block < MP_STATE_MEM(gc_alloc_table_byte_len) * BLOCKS_PER_ATB; block++) {
MICROPY_GC_HOOK_LOOP
switch (ATB_GET_KIND(block)) { switch (ATB_GET_KIND(block)) {
case AT_HEAD: case AT_HEAD:
#if MICROPY_ENABLE_FINALISER #if MICROPY_ENABLE_FINALISER
@ -407,6 +411,7 @@ static void *gc_get_ptr(void **ptrs, int i) {
void gc_collect_root(void **ptrs, size_t len) { void gc_collect_root(void **ptrs, size_t len) {
for (size_t i = 0; i < len; i++) { for (size_t i = 0; i < len; i++) {
MICROPY_GC_HOOK_LOOP
void *ptr = gc_get_ptr(ptrs, i); void *ptr = gc_get_ptr(ptrs, i);
gc_mark(ptr); gc_mark(ptr);
} }

11
py/lexer.c
View File

@ -365,9 +365,16 @@ STATIC void parse_string_literal(mp_lexer_t *lex, bool is_raw, bool is_fstring)
// (MicroPython limitation) note: this is completely unaware of // (MicroPython limitation) note: this is completely unaware of
// Python syntax and will not handle any expression containing '}' or ':'. // Python syntax and will not handle any expression containing '}' or ':'.
// e.g. f'{"}"}' or f'{foo({})}'. // e.g. f'{"}"}' or f'{foo({})}'.
while (!is_end(lex) && !is_char_or(lex, ':', '}')) { unsigned int nested_bracket_level = 0;
while (!is_end(lex) && (nested_bracket_level != 0 || !is_char_or(lex, ':', '}'))) {
unichar c = CUR_CHAR(lex);
if (c == '[' || c == '{') {
nested_bracket_level += 1;
} else if (c == ']' || c == '}') {
nested_bracket_level -= 1;
}
// like the default case at the end of this function, stay 8-bit clean // like the default case at the end of this function, stay 8-bit clean
vstr_add_byte(&lex->fstring_args, CUR_CHAR(lex)); vstr_add_byte(&lex->fstring_args, c);
next_char(lex); next_char(lex);
} }
if (lex->fstring_args.buf[lex->fstring_args.len - 1] == '=') { if (lex->fstring_args.buf[lex->fstring_args.len - 1] == '=') {

37
py/map.c
View File

@ -42,6 +42,27 @@
#define DEBUG_printf(...) (void)0 #define DEBUG_printf(...) (void)0
#endif #endif
#if MICROPY_OPT_MAP_LOOKUP_CACHE
// MP_STATE_VM(map_lookup_cache) provides a cache of index to the last known
// position of that index in any map. On a cache hit, this allows
// short-circuiting the full linear search in the case of an ordered map
// (i.e. all builtin modules and objects' locals dicts), and computation of
// the hash (and potentially some linear probing) in the case of a regular
// map. Note the same cache is shared across all maps.
// Gets the index into the cache for this index. Shift down by two to remove
// mp_obj_t tag bits.
#define MAP_CACHE_OFFSET(index) ((((uintptr_t)(index)) >> 2) % MICROPY_OPT_MAP_LOOKUP_CACHE_SIZE)
// Gets the map cache entry for the corresponding index.
#define MAP_CACHE_ENTRY(index) (MP_STATE_VM(map_lookup_cache)[MAP_CACHE_OFFSET(index)])
// Retrieve the mp_obj_t at the location suggested by the cache.
#define MAP_CACHE_GET(map, index) (&(map)->table[MAP_CACHE_ENTRY(index) % (map)->alloc])
// Update the cache for this index.
#define MAP_CACHE_SET(index, pos) MAP_CACHE_ENTRY(index) = (pos) & 0xff;
#else
#define MAP_CACHE_SET(index, pos)
#endif
// This table of sizes is used to control the growth of hash tables. // This table of sizes is used to control the growth of hash tables.
// The first set of sizes are chosen so the allocation fits exactly in a // The first set of sizes are chosen so the allocation fits exactly in a
// 4-word GC block, and it's not so important for these small values to be // 4-word GC block, and it's not so important for these small values to be
@ -134,10 +155,22 @@ STATIC void mp_map_rehash(mp_map_t *map) {
// - returns slot, with key non-null and value=MP_OBJ_NULL if it was added // - returns slot, with key non-null and value=MP_OBJ_NULL if it was added
// MP_MAP_LOOKUP_REMOVE_IF_FOUND behaviour: // MP_MAP_LOOKUP_REMOVE_IF_FOUND behaviour:
// - returns NULL if not found, else the slot if was found in with key null and value non-null // - returns NULL if not found, else the slot if was found in with key null and value non-null
mp_map_elem_t *PLACE_IN_ITCM(mp_map_lookup)(mp_map_t * map, mp_obj_t index, mp_map_lookup_kind_t lookup_kind) { mp_map_elem_t *MICROPY_WRAP_MP_MAP_LOOKUP(mp_map_lookup)(mp_map_t * map, mp_obj_t index, mp_map_lookup_kind_t lookup_kind) {
// If the map is a fixed array then we must only be called for a lookup // If the map is a fixed array then we must only be called for a lookup
assert(!map->is_fixed || lookup_kind == MP_MAP_LOOKUP); assert(!map->is_fixed || lookup_kind == MP_MAP_LOOKUP);
#if MICROPY_OPT_MAP_LOOKUP_CACHE
// Try the cache for lookup or add-if-not-found.
if (lookup_kind != MP_MAP_LOOKUP_REMOVE_IF_FOUND && map->alloc) {
mp_map_elem_t *slot = MAP_CACHE_GET(map, index);
// Note: Just comparing key for value equality will have false negatives, but
// these will be handled by the regular path below.
if (slot->key == index) {
return slot;
}
}
#endif
// Work out if we can compare just pointers // Work out if we can compare just pointers
bool compare_only_ptrs = map->all_keys_are_qstrs; bool compare_only_ptrs = map->all_keys_are_qstrs;
if (compare_only_ptrs) { if (compare_only_ptrs) {
@ -174,6 +207,7 @@ mp_map_elem_t *PLACE_IN_ITCM(mp_map_lookup)(mp_map_t * map, mp_obj_t index, mp_m
elem->value = value; elem->value = value;
} }
#endif #endif
MAP_CACHE_SET(index, elem - map->table);
return elem; return elem;
} }
} }
@ -256,6 +290,7 @@ mp_map_elem_t *PLACE_IN_ITCM(mp_map_lookup)(mp_map_t * map, mp_obj_t index, mp_m
} }
// keep slot->value so that caller can access it if needed // keep slot->value so that caller can access it if needed
} }
MAP_CACHE_SET(index, pos);
return slot; return slot;
} }

14
py/mkrules.cmake
View File

@ -10,6 +10,15 @@ set(MICROPY_QSTRDEFS_COLLECTED "${MICROPY_GENHDR_DIR}/qstrdefs.collected.h")
set(MICROPY_QSTRDEFS_PREPROCESSED "${MICROPY_GENHDR_DIR}/qstrdefs.preprocessed.h") set(MICROPY_QSTRDEFS_PREPROCESSED "${MICROPY_GENHDR_DIR}/qstrdefs.preprocessed.h")
set(MICROPY_QSTRDEFS_GENERATED "${MICROPY_GENHDR_DIR}/qstrdefs.generated.h") set(MICROPY_QSTRDEFS_GENERATED "${MICROPY_GENHDR_DIR}/qstrdefs.generated.h")
# Need to do this before extracting MICROPY_CPP_DEF below. Rest of frozen
# manifest handling is at the end of this file.
if(MICROPY_FROZEN_MANIFEST)
target_compile_definitions(${MICROPY_TARGET} PUBLIC
MICROPY_QSTR_EXTRA_POOL=mp_qstr_frozen_const_pool
MICROPY_MODULE_FROZEN_MPY=\(1\)
)
endif()
# Provide defaults for preprocessor flags if not already defined # Provide defaults for preprocessor flags if not already defined
if(NOT MICROPY_CPP_FLAGS) if(NOT MICROPY_CPP_FLAGS)
get_target_property(MICROPY_CPP_INC ${MICROPY_TARGET} INCLUDE_DIRECTORIES) get_target_property(MICROPY_CPP_INC ${MICROPY_TARGET} INCLUDE_DIRECTORIES)
@ -120,10 +129,7 @@ if(MICROPY_FROZEN_MANIFEST)
${MICROPY_FROZEN_CONTENT} ${MICROPY_FROZEN_CONTENT}
) )
target_compile_definitions(${MICROPY_TARGET} PUBLIC # Note: target_compile_definitions already added earlier.
MICROPY_QSTR_EXTRA_POOL=mp_qstr_frozen_const_pool
MICROPY_MODULE_FROZEN_MPY=\(1\)
)
if(NOT MICROPY_LIB_DIR) if(NOT MICROPY_LIB_DIR)
set(MICROPY_LIB_DIR ${MICROPY_DIR}/../micropython-lib) set(MICROPY_LIB_DIR ${MICROPY_DIR}/../micropython-lib)

65
py/mkrules.mk
View File

@ -125,46 +125,18 @@ $(MICROPY_MPYCROSS_DEPENDENCY):
$(MAKE) -C $(dir $@) $(MAKE) -C $(dir $@)
endif endif
ifneq ($(FROZEN_DIR),)
$(error Support for FROZEN_DIR was removed. Please use manifest.py instead, see https://docs.micropython.org/en/latest/reference/manifest.html)
endif
ifneq ($(FROZEN_MPY_DIR),)
$(error Support for FROZEN_MPY_DIR was removed. Please use manifest.py instead, see https://docs.micropython.org/en/latest/reference/manifest.html)
endif
ifneq ($(FROZEN_MANIFEST),) ifneq ($(FROZEN_MANIFEST),)
# to build frozen_content.c from a manifest # to build frozen_content.c from a manifest
$(BUILD)/frozen_content.c: FORCE $(BUILD)/genhdr/qstrdefs.generated.h | $(MICROPY_MPYCROSS_DEPENDENCY) $(BUILD)/frozen_content.c: FORCE $(BUILD)/genhdr/qstrdefs.generated.h | $(MICROPY_MPYCROSS_DEPENDENCY)
$(Q)$(MAKE_MANIFEST) -o $@ -v "MPY_DIR=$(TOP)" -v "MPY_LIB_DIR=$(MPY_LIB_DIR)" -v "PORT_DIR=$(shell pwd)" -v "BOARD_DIR=$(BOARD_DIR)" -b "$(BUILD)" $(if $(MPY_CROSS_FLAGS),-f"$(MPY_CROSS_FLAGS)",) --mpy-tool-flags="$(MPY_TOOL_FLAGS)" $(FROZEN_MANIFEST) $(Q)$(MAKE_MANIFEST) -o $@ -v "MPY_DIR=$(TOP)" -v "MPY_LIB_DIR=$(MPY_LIB_DIR)" -v "PORT_DIR=$(shell pwd)" -v "BOARD_DIR=$(BOARD_DIR)" -b "$(BUILD)" $(if $(MPY_CROSS_FLAGS),-f"$(MPY_CROSS_FLAGS)",) --mpy-tool-flags="$(MPY_TOOL_FLAGS)" $(FROZEN_MANIFEST)
ifneq ($(FROZEN_DIR),)
$(error FROZEN_DIR cannot be used in conjunction with FROZEN_MANIFEST)
endif
ifneq ($(FROZEN_MPY_DIR),)
$(error FROZEN_MPY_DIR cannot be used in conjunction with FROZEN_MANIFEST)
endif
endif
ifneq ($(FROZEN_DIR),)
$(info Warning: FROZEN_DIR is deprecated in favour of FROZEN_MANIFEST)
$(BUILD)/frozen.c: $(wildcard $(FROZEN_DIR)/*) $(HEADER_BUILD) $(FROZEN_EXTRA_DEPS)
$(STEPECHO) "Generating $@"
$(Q)$(MAKE_FROZEN) $(FROZEN_DIR) > $@
endif
ifneq ($(FROZEN_MPY_DIRS),)
# Copy all the modules and single python files to freeze to a common area, omitting top-level dirs (the repo names).
# Do any preprocessing necessary: currently, this adds version information, removes examples, and
# non-library .py files in the modules (setup.py and conf.py)
# Then compile .mpy files from all the .py files, placing them in the same directories as the .py files.
$(BUILD)/frozen_mpy: $(FROZEN_MPY_DIRS)
$(ECHO) FREEZE $(FROZEN_MPY_DIRS)
$(Q)$(MKDIR) -p $@
$(Q)$(PREPROCESS_FROZEN_MODULES) -o $@ $(FROZEN_MPY_DIRS)
$(Q)$(CD) $@ && \
$(FIND) -L . -type f -name '*.py' | sed 's=^\./==' | \
xargs -n1 "$(abspath $(MICROPY_MPYCROSS_DEPENDENCY))" $(MPY_CROSS_FLAGS)
# to build frozen_mpy.c from all .mpy files
# You need to define MPY_TOOL_LONGINT_IMPL in mpconfigport.mk
# if the default will not work (mpz is the default).
$(BUILD)/frozen_mpy.c: $(BUILD)/frozen_mpy $(BUILD)/genhdr/qstrdefs.generated.h $(TOP)/tools/mpy-tool.py
$(STEPECHO) "Creating $@"
$(Q)$(MPY_TOOL) $(MPY_TOOL_LONGINT_IMPL) -f -q $(BUILD)/genhdr/qstrdefs.preprocessed.h $(shell $(FIND) -L $(BUILD)/frozen_mpy -type f -name '*.mpy') > $@
endif endif
ifneq ($(PROG),) ifneq ($(PROG),)
@ -220,27 +192,6 @@ clean:
$(RM) -rf $(BUILD) $(CLEAN_EXTRA) $(RM) -rf $(BUILD) $(CLEAN_EXTRA)
.PHONY: clean .PHONY: clean
# Clean every non-git file from FROZEN_DIR/FROZEN_MPY_DIR, but making a backup.
# We run rmdir below to avoid empty backup dir (it will silently fail if backup
# is non-empty).
clean-frozen:
if [ -n "$(FROZEN_MPY_DIR)" ]; then \
backup_dir=$(FROZEN_MPY_DIR).$$(date +%Y%m%dT%H%M%S); mkdir $$backup_dir; \
cd $(FROZEN_MPY_DIR); git status --ignored -u all -s . | awk ' {print $$2}' \
| xargs --no-run-if-empty cp --parents -t ../$$backup_dir; \
rmdir ../$$backup_dir 2>/dev/null || true; \
git clean -d -f .; \
fi
if [ -n "$(FROZEN_DIR)" ]; then \
backup_dir=$(FROZEN_DIR).$$(date +%Y%m%dT%H%M%S); mkdir $$backup_dir; \
cd $(FROZEN_DIR); git status --ignored -u all -s . | awk ' {print $$2}' \
| xargs --no-run-if-empty cp --parents -t ../$$backup_dir; \
rmdir ../$$backup_dir 2>/dev/null || true; \
git clean -d -f .; \
fi
.PHONY: clean-frozen
print-cfg: print-cfg:
$(ECHO) "PY_SRC = $(PY_SRC)" $(ECHO) "PY_SRC = $(PY_SRC)"
$(ECHO) "BUILD = $(BUILD)" $(ECHO) "BUILD = $(BUILD)"

1
py/modbuiltins.c
View File

@ -786,6 +786,7 @@ STATIC const mp_rom_map_elem_t mp_module_builtins_globals_table[] = {
// Extra builtins as defined by a port // Extra builtins as defined by a port
MICROPY_PORT_BUILTINS MICROPY_PORT_BUILTINS
MICROPY_PORT_EXTRA_BUILTINS
}; };
MP_DEFINE_CONST_DICT(mp_module_builtins_globals, mp_module_builtins_globals_table); MP_DEFINE_CONST_DICT(mp_module_builtins_globals, mp_module_builtins_globals_table);

47
py/modio.c
View File

@ -212,50 +212,6 @@ STATIC const mp_obj_type_t mp_type_bufwriter = {
}; };
#endif // MICROPY_PY_IO_BUFFEREDWRITER #endif // MICROPY_PY_IO_BUFFEREDWRITER
#if MICROPY_PY_IO_RESOURCE_STREAM
STATIC mp_obj_t resource_stream(mp_obj_t package_in, mp_obj_t path_in) {
VSTR_FIXED(path_buf, MICROPY_ALLOC_PATH_MAX);
size_t len;
// As an extension to pkg_resources.resource_stream(), we support
// package parameter being None, the path_in is interpreted as a
// raw path.
if (package_in != mp_const_none) {
// Pass "True" as sentinel value in fromlist to force returning of leaf module
mp_obj_t pkg = mp_import_name(mp_obj_str_get_qstr(package_in), mp_const_true, MP_OBJ_NEW_SMALL_INT(0));
mp_obj_t dest[2];
mp_load_method_maybe(pkg, MP_QSTR___path__, dest);
if (dest[0] == MP_OBJ_NULL) {
mp_raise_TypeError(NULL);
}
const char *path = mp_obj_str_get_data(dest[0], &len);
vstr_add_strn(&path_buf, path, len);
vstr_add_byte(&path_buf, '/');
}
const char *path = mp_obj_str_get_data(path_in, &len);
vstr_add_strn(&path_buf, path, len);
size_t file_len;
const char *data = mp_find_frozen_str(path_buf.buf, path_buf.len, &file_len);
if (data != NULL) {
mp_obj_stringio_t *o = m_new_obj(mp_obj_stringio_t);
o->base.type = &mp_type_bytesio;
o->vstr = m_new_obj(vstr_t);
vstr_init_fixed_buf(o->vstr, file_len + 1, (char *)data);
o->vstr->len = file_len;
o->pos = 0;
return MP_OBJ_FROM_PTR(o);
}
mp_obj_t path_out = mp_obj_new_str(path_buf.buf, path_buf.len);
return mp_builtin_open(1, &path_out, (mp_map_t *)&mp_const_empty_map);
}
STATIC MP_DEFINE_CONST_FUN_OBJ_2(resource_stream_obj, resource_stream);
#endif
STATIC const mp_rom_map_elem_t mp_module_io_globals_table[] = { STATIC const mp_rom_map_elem_t mp_module_io_globals_table[] = {
#if CIRCUITPY #if CIRCUITPY
{ MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_io) }, { MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_io) },
@ -268,9 +224,6 @@ STATIC const mp_rom_map_elem_t mp_module_io_globals_table[] = {
#if MICROPY_PY_IO_IOBASE #if MICROPY_PY_IO_IOBASE
{ MP_ROM_QSTR(MP_QSTR_IOBase), MP_ROM_PTR(&mp_type_iobase) }, { MP_ROM_QSTR(MP_QSTR_IOBase), MP_ROM_PTR(&mp_type_iobase) },
#endif #endif
#if MICROPY_PY_IO_RESOURCE_STREAM
{ MP_ROM_QSTR(MP_QSTR_resource_stream), MP_ROM_PTR(&resource_stream_obj) },
#endif
#if MICROPY_PY_IO_FILEIO #if MICROPY_PY_IO_FILEIO
{ MP_ROM_QSTR(MP_QSTR_FileIO), MP_ROM_PTR(&mp_type_fileio) }, { MP_ROM_QSTR(MP_QSTR_FileIO), MP_ROM_PTR(&mp_type_fileio) },
#if MICROPY_CPYTHON_COMPAT #if MICROPY_CPYTHON_COMPAT

2
py/modsys.c
View File

@ -158,7 +158,7 @@ STATIC MP_DEFINE_CONST_FUN_OBJ_1(mp_sys_atexit_obj, mp_sys_atexit);
#endif #endif
#if MICROPY_PY_SYS_SETTRACE #if MICROPY_PY_SYS_SETTRACE
// settrace(tracefunc): Set the system’s trace function. // settrace(tracefunc): Set the system's trace function.
STATIC mp_obj_t mp_sys_settrace(mp_obj_t obj) { STATIC mp_obj_t mp_sys_settrace(mp_obj_t obj) {
return mp_prof_settrace(obj); return mp_prof_settrace(obj);
} }

376
py/mpconfig.h
View File

@ -50,6 +50,31 @@
#define CIRCUITPY 0 #define CIRCUITPY 0
#endif #endif
// Disable all optional features (i.e. minimal port).
#define MICROPY_CONFIG_ROM_LEVEL_MINIMUM (0)
// Only enable core features (constrained flash, e.g. STM32L072)
#define MICROPY_CONFIG_ROM_LEVEL_CORE_FEATURES (10)
// Enable most common features (small on-device flash, e.g. STM32F411)
#define MICROPY_CONFIG_ROM_LEVEL_BASIC_FEATURES (20)
// Enable convenience features (medium on-device flash, e.g. STM32F405)
#define MICROPY_CONFIG_ROM_LEVEL_EXTRA_FEATURES (30)
// Enable all common features (large/external flash, rp2, unix)
#define MICROPY_CONFIG_ROM_LEVEL_FULL_FEATURES (40)
// Enable everything (e.g. coverage)
#define MICROPY_CONFIG_ROM_LEVEL_EVERYTHING (50)
// Ports/boards should set this, but default to level=core.
#ifndef MICROPY_CONFIG_ROM_LEVEL
#define MICROPY_CONFIG_ROM_LEVEL (MICROPY_CONFIG_ROM_LEVEL_CORE_FEATURES)
#endif
// Helper macros for "have at least this level".
#define MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES (MICROPY_CONFIG_ROM_LEVEL >= MICROPY_CONFIG_ROM_LEVEL_CORE_FEATURES)
#define MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_BASIC_FEATURES (MICROPY_CONFIG_ROM_LEVEL >= MICROPY_CONFIG_ROM_LEVEL_BASIC_FEATURES)
#define MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES (MICROPY_CONFIG_ROM_LEVEL >= MICROPY_CONFIG_ROM_LEVEL_EXTRA_FEATURES)
#define MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_FULL_FEATURES (MICROPY_CONFIG_ROM_LEVEL >= MICROPY_CONFIG_ROM_LEVEL_FULL_FEATURES)
#define MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EVERYTHING (MICROPY_CONFIG_ROM_LEVEL >= MICROPY_CONFIG_ROM_LEVEL_EVERYTHING)
// Any options not explicitly set in mpconfigport.h will get default // Any options not explicitly set in mpconfigport.h will get default
// values below. // values below.
@ -144,7 +169,7 @@
// Support automatic GC when reaching allocation threshold, // Support automatic GC when reaching allocation threshold,
// configurable by gc.threshold(). // configurable by gc.threshold().
#ifndef MICROPY_GC_ALLOC_THRESHOLD #ifndef MICROPY_GC_ALLOC_THRESHOLD
#define MICROPY_GC_ALLOC_THRESHOLD (1) #define MICROPY_GC_ALLOC_THRESHOLD (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Number of bytes to allocate initially when creating new chunks to store // Number of bytes to allocate initially when creating new chunks to store
@ -243,7 +268,11 @@
// Number of bytes used to store qstr hash // Number of bytes used to store qstr hash
#ifndef MICROPY_QSTR_BYTES_IN_HASH #ifndef MICROPY_QSTR_BYTES_IN_HASH
#if MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES
#define MICROPY_QSTR_BYTES_IN_HASH (2) #define MICROPY_QSTR_BYTES_IN_HASH (2)
#else
#define MICROPY_QSTR_BYTES_IN_HASH (1)
#endif
#endif #endif
// Avoid using C stack when making Python function calls. C stack still // Avoid using C stack when making Python function calls. C stack still
@ -385,7 +414,7 @@
// Whether to include the compiler // Whether to include the compiler
#ifndef MICROPY_ENABLE_COMPILER #ifndef MICROPY_ENABLE_COMPILER
#define MICROPY_ENABLE_COMPILER (1) #define MICROPY_ENABLE_COMPILER (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether the compiler is dynamically configurable (ie at runtime) // Whether the compiler is dynamically configurable (ie at runtime)
@ -396,49 +425,47 @@
// Configure dynamic compiler macros // Configure dynamic compiler macros
#if MICROPY_DYNAMIC_COMPILER #if MICROPY_DYNAMIC_COMPILER
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE_DYNAMIC (mp_dynamic_compiler.opt_cache_map_lookup_in_bytecode)
#define MICROPY_PY_BUILTINS_STR_UNICODE_DYNAMIC (mp_dynamic_compiler.py_builtins_str_unicode) #define MICROPY_PY_BUILTINS_STR_UNICODE_DYNAMIC (mp_dynamic_compiler.py_builtins_str_unicode)
#else #else
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE_DYNAMIC MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE
#define MICROPY_PY_BUILTINS_STR_UNICODE_DYNAMIC MICROPY_PY_BUILTINS_STR_UNICODE #define MICROPY_PY_BUILTINS_STR_UNICODE_DYNAMIC MICROPY_PY_BUILTINS_STR_UNICODE
#endif #endif
// Whether to enable constant folding; eg 1+2 rewritten as 3 // Whether to enable constant folding; eg 1+2 rewritten as 3
#ifndef MICROPY_COMP_CONST_FOLDING #ifndef MICROPY_COMP_CONST_FOLDING
#define MICROPY_COMP_CONST_FOLDING (1) #define MICROPY_COMP_CONST_FOLDING (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to enable optimisations for constant literals, eg OrderedDict // Whether to enable optimisations for constant literals, eg OrderedDict
#ifndef MICROPY_COMP_CONST_LITERAL #ifndef MICROPY_COMP_CONST_LITERAL
#define MICROPY_COMP_CONST_LITERAL (1) #define MICROPY_COMP_CONST_LITERAL (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to enable lookup of constants in modules; eg module.CONST // Whether to enable lookup of constants in modules; eg module.CONST
#ifndef MICROPY_COMP_MODULE_CONST #ifndef MICROPY_COMP_MODULE_CONST
#define MICROPY_COMP_MODULE_CONST (0) #define MICROPY_COMP_MODULE_CONST (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to enable constant optimisation; id = const(value) // Whether to enable constant optimisation; id = const(value)
#ifndef MICROPY_COMP_CONST #ifndef MICROPY_COMP_CONST
#define MICROPY_COMP_CONST (1) #define MICROPY_COMP_CONST (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to enable optimisation of: a, b = c, d // Whether to enable optimisation of: a, b = c, d
// Costs 124 bytes (Thumb2) // Costs 124 bytes (Thumb2)
#ifndef MICROPY_COMP_DOUBLE_TUPLE_ASSIGN #ifndef MICROPY_COMP_DOUBLE_TUPLE_ASSIGN
#define MICROPY_COMP_DOUBLE_TUPLE_ASSIGN (1) #define MICROPY_COMP_DOUBLE_TUPLE_ASSIGN (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to enable optimisation of: a, b, c = d, e, f // Whether to enable optimisation of: a, b, c = d, e, f
// Requires MICROPY_COMP_DOUBLE_TUPLE_ASSIGN and costs 68 bytes (Thumb2) // Requires MICROPY_COMP_DOUBLE_TUPLE_ASSIGN and costs 68 bytes (Thumb2)
#ifndef MICROPY_COMP_TRIPLE_TUPLE_ASSIGN #ifndef MICROPY_COMP_TRIPLE_TUPLE_ASSIGN
#define MICROPY_COMP_TRIPLE_TUPLE_ASSIGN (0) #define MICROPY_COMP_TRIPLE_TUPLE_ASSIGN (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to enable optimisation of: return a if b else c // Whether to enable optimisation of: return a if b else c
// Costs about 80 bytes (Thumb2) and saves 2 bytes of bytecode for each use // Costs about 80 bytes (Thumb2) and saves 2 bytes of bytecode for each use
#ifndef MICROPY_COMP_RETURN_IF_EXPR #ifndef MICROPY_COMP_RETURN_IF_EXPR
#define MICROPY_COMP_RETURN_IF_EXPR (0) #define MICROPY_COMP_RETURN_IF_EXPR (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to include parsing of f-string literals // Whether to include parsing of f-string literals
@ -511,23 +538,36 @@
#define MICROPY_OPT_COMPUTED_GOTO_SAVE_SPACE (0) #define MICROPY_OPT_COMPUTED_GOTO_SAVE_SPACE (0)
#endif #endif
// Whether to cache result of map lookups in LOAD_NAME, LOAD_GLOBAL, LOAD_ATTR, // Optimise the fast path for loading attributes from instance types. Increases
// STORE_ATTR bytecodes. Uses 1 byte extra RAM for each of these opcodes and // Thumb2 code size by about 48 bytes.
// uses a bit of extra code ROM, but greatly improves lookup speed. #ifndef MICROPY_OPT_LOAD_ATTR_FAST_PATH
#ifndef MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE #define MICROPY_OPT_LOAD_ATTR_FAST_PATH (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE (0) #endif
// Use extra RAM to cache map lookups by remembering the likely location of
// the index. Avoids the hash computation on unordered maps, and avoids the
// linear search on ordered (especially in-ROM) maps. Can provide a +10-15%
// performance improvement on benchmarks involving lots of attribute access
// or dictionary lookup.
#ifndef MICROPY_OPT_MAP_LOOKUP_CACHE
#define MICROPY_OPT_MAP_LOOKUP_CACHE (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif
// How much RAM (in bytes) to use for the map lookup cache.
#ifndef MICROPY_OPT_MAP_LOOKUP_CACHE_SIZE
#define MICROPY_OPT_MAP_LOOKUP_CACHE_SIZE (128)
#endif #endif
// Whether to use fast versions of bitwise operations (and, or, xor) when the // Whether to use fast versions of bitwise operations (and, or, xor) when the
// arguments are both positive. Increases Thumb2 code size by about 250 bytes. // arguments are both positive. Increases Thumb2 code size by about 250 bytes.
#ifndef MICROPY_OPT_MPZ_BITWISE #ifndef MICROPY_OPT_MPZ_BITWISE
#define MICROPY_OPT_MPZ_BITWISE (0) #define MICROPY_OPT_MPZ_BITWISE (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether math.factorial is large, fast and recursive (1) or small and slow (0). // Whether math.factorial is large, fast and recursive (1) or small and slow (0).
#ifndef MICROPY_OPT_MATH_FACTORIAL #ifndef MICROPY_OPT_MATH_FACTORIAL
#define MICROPY_OPT_MATH_FACTORIAL (0) #define MICROPY_OPT_MATH_FACTORIAL (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
/*****************************************************************************/ /*****************************************************************************/
@ -537,7 +577,7 @@
// When disabled, only importing of built-in modules is supported // When disabled, only importing of built-in modules is supported
// When enabled, a port must implement mp_import_stat (among other things) // When enabled, a port must implement mp_import_stat (among other things)
#ifndef MICROPY_ENABLE_EXTERNAL_IMPORT #ifndef MICROPY_ENABLE_EXTERNAL_IMPORT
#define MICROPY_ENABLE_EXTERNAL_IMPORT (1) #define MICROPY_ENABLE_EXTERNAL_IMPORT (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to use the POSIX reader for importing files // Whether to use the POSIX reader for importing files
@ -587,9 +627,14 @@
#define MICROPY_ENABLE_GC (0) #define MICROPY_ENABLE_GC (0)
#endif #endif
// Hook to run code during time consuming garbage collector operations
#ifndef MICROPY_GC_HOOK_LOOP
#define MICROPY_GC_HOOK_LOOP
#endif
// Whether to enable finalisers in the garbage collector (ie call __del__) // Whether to enable finalisers in the garbage collector (ie call __del__)
#ifndef MICROPY_ENABLE_FINALISER #ifndef MICROPY_ENABLE_FINALISER
#define MICROPY_ENABLE_FINALISER (0) #define MICROPY_ENABLE_FINALISER (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to enable a separate allocator for the Python stack. // Whether to enable a separate allocator for the Python stack.
@ -606,7 +651,7 @@
// Whether to check C stack usage. C stack used for calling Python functions, // Whether to check C stack usage. C stack used for calling Python functions,
// etc. Not checking means segfault on overflow. // etc. Not checking means segfault on overflow.
#ifndef MICROPY_STACK_CHECK #ifndef MICROPY_STACK_CHECK
#define MICROPY_STACK_CHECK (0) #define MICROPY_STACK_CHECK (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to measure maximum stack excursion // Whether to measure maximum stack excursion
@ -626,7 +671,7 @@
// Whether to provide the mp_kbd_exception object, and micropython.kbd_intr function // Whether to provide the mp_kbd_exception object, and micropython.kbd_intr function
#ifndef MICROPY_KBD_EXCEPTION #ifndef MICROPY_KBD_EXCEPTION
#define MICROPY_KBD_EXCEPTION (0) #define MICROPY_KBD_EXCEPTION (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Prefer to raise KeyboardInterrupt asynchronously (from signal or interrupt // Prefer to raise KeyboardInterrupt asynchronously (from signal or interrupt
@ -637,7 +682,7 @@
// Whether to include REPL helper function // Whether to include REPL helper function
#ifndef MICROPY_HELPER_REPL #ifndef MICROPY_HELPER_REPL
#define MICROPY_HELPER_REPL (0) #define MICROPY_HELPER_REPL (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Allow enabling debug prints after each REPL line // Allow enabling debug prints after each REPL line
@ -647,7 +692,7 @@
// Whether to include emacs-style readline behavior in REPL // Whether to include emacs-style readline behavior in REPL
#ifndef MICROPY_REPL_EMACS_KEYS #ifndef MICROPY_REPL_EMACS_KEYS
#define MICROPY_REPL_EMACS_KEYS (0) #define MICROPY_REPL_EMACS_KEYS (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to include emacs-style word movement/kill readline behavior in REPL. // Whether to include emacs-style word movement/kill readline behavior in REPL.
@ -667,7 +712,7 @@
// Whether to implement auto-indent in REPL // Whether to implement auto-indent in REPL
#ifndef MICROPY_REPL_AUTO_INDENT #ifndef MICROPY_REPL_AUTO_INDENT
#define MICROPY_REPL_AUTO_INDENT (0) #define MICROPY_REPL_AUTO_INDENT (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether port requires event-driven REPL functions // Whether port requires event-driven REPL functions
@ -696,7 +741,7 @@ typedef long long mp_longint_impl_t;
// Whether to include information in the byte code to determine source // Whether to include information in the byte code to determine source
// line number (increases RAM usage, but doesn't slow byte code execution) // line number (increases RAM usage, but doesn't slow byte code execution)
#ifndef MICROPY_ENABLE_SOURCE_LINE #ifndef MICROPY_ENABLE_SOURCE_LINE
#define MICROPY_ENABLE_SOURCE_LINE (0) #define MICROPY_ENABLE_SOURCE_LINE (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to include doc strings (increases RAM usage) // Whether to include doc strings (increases RAM usage)
@ -714,7 +759,13 @@ typedef long long mp_longint_impl_t;
#define MICROPY_ERROR_REPORTING_DETAILED (3) #define MICROPY_ERROR_REPORTING_DETAILED (3)
#ifndef MICROPY_ERROR_REPORTING #ifndef MICROPY_ERROR_REPORTING
#if MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_FULL_FEATURES
#define MICROPY_ERROR_REPORTING (MICROPY_ERROR_REPORTING_DETAILED)
#elif MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES
#define MICROPY_ERROR_REPORTING (MICROPY_ERROR_REPORTING_NORMAL) #define MICROPY_ERROR_REPORTING (MICROPY_ERROR_REPORTING_NORMAL)
#else
#define MICROPY_ERROR_REPORTING (MICROPY_ERROR_REPORTING_TERSE)
#endif
#endif #endif
// Whether issue warnings during compiling/execution // Whether issue warnings during compiling/execution
@ -770,7 +821,7 @@ typedef double mp_float_t;
// TODO: Originally intended as generic category to not // TODO: Originally intended as generic category to not
// add bunch of once-off options. May need refactoring later // add bunch of once-off options. May need refactoring later
#ifndef MICROPY_CPYTHON_COMPAT #ifndef MICROPY_CPYTHON_COMPAT
#define MICROPY_CPYTHON_COMPAT (1) #define MICROPY_CPYTHON_COMPAT (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Perform full checks as done by CPython. Disabling this // Perform full checks as done by CPython. Disabling this
@ -779,12 +830,12 @@ typedef double mp_float_t;
// grave issues (in other words, only user app should be, // grave issues (in other words, only user app should be,
// affected, not system). // affected, not system).
#ifndef MICROPY_FULL_CHECKS #ifndef MICROPY_FULL_CHECKS
#define MICROPY_FULL_CHECKS (1) #define MICROPY_FULL_CHECKS (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether POSIX-semantics non-blocking streams are supported // Whether POSIX-semantics non-blocking streams are supported
#ifndef MICROPY_STREAMS_NON_BLOCK #ifndef MICROPY_STREAMS_NON_BLOCK
#define MICROPY_STREAMS_NON_BLOCK (0) #define MICROPY_STREAMS_NON_BLOCK (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to provide stream functions with POSIX-like signatures // Whether to provide stream functions with POSIX-like signatures
@ -795,17 +846,23 @@ typedef double mp_float_t;
// Whether to call __init__ when importing builtin modules for the first time // Whether to call __init__ when importing builtin modules for the first time
#ifndef MICROPY_MODULE_BUILTIN_INIT #ifndef MICROPY_MODULE_BUILTIN_INIT
#define MICROPY_MODULE_BUILTIN_INIT (0) #define MICROPY_MODULE_BUILTIN_INIT (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to support module-level __getattr__ (see PEP 562) // Whether to support module-level __getattr__ (see PEP 562)
#ifndef MICROPY_MODULE_GETATTR #ifndef MICROPY_MODULE_GETATTR
#define MICROPY_MODULE_GETATTR (1) #define MICROPY_MODULE_GETATTR (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether module weak links are supported // Whether module weak links are supported
#ifndef MICROPY_MODULE_WEAK_LINKS #ifndef MICROPY_MODULE_WEAK_LINKS
#define MICROPY_MODULE_WEAK_LINKS (0) #define MICROPY_MODULE_WEAK_LINKS (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif
// Whether to enable importing foo.py with __name__ set to '__main__'
// Used by the unix port for the -m flag.
#ifndef MICROPY_MODULE_OVERRIDE_MAIN_IMPORT
#define MICROPY_MODULE_OVERRIDE_MAIN_IMPORT (0)
#endif #endif
// Whether frozen modules are supported in the form of strings // Whether frozen modules are supported in the form of strings
@ -825,7 +882,7 @@ typedef double mp_float_t;
// Whether you can override builtins in the builtins module // Whether you can override builtins in the builtins module
#ifndef MICROPY_CAN_OVERRIDE_BUILTINS #ifndef MICROPY_CAN_OVERRIDE_BUILTINS
#define MICROPY_CAN_OVERRIDE_BUILTINS (0) #define MICROPY_CAN_OVERRIDE_BUILTINS (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to check that the "self" argument of a builtin method has the // Whether to check that the "self" argument of a builtin method has the
@ -834,7 +891,7 @@ typedef double mp_float_t;
// list.append([], 1). Without this check such calls will have undefined // list.append([], 1). Without this check such calls will have undefined
// behaviour (usually segfault) if the first argument is the wrong type. // behaviour (usually segfault) if the first argument is the wrong type.
#ifndef MICROPY_BUILTIN_METHOD_CHECK_SELF_ARG #ifndef MICROPY_BUILTIN_METHOD_CHECK_SELF_ARG
#define MICROPY_BUILTIN_METHOD_CHECK_SELF_ARG (1) #define MICROPY_BUILTIN_METHOD_CHECK_SELF_ARG (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to use internally defined errno's (otherwise system provided ones) // Whether to use internally defined errno's (otherwise system provided ones)
@ -849,7 +906,7 @@ typedef double mp_float_t;
// Support for internal scheduler // Support for internal scheduler
#ifndef MICROPY_ENABLE_SCHEDULER #ifndef MICROPY_ENABLE_SCHEDULER
#define MICROPY_ENABLE_SCHEDULER (0) #define MICROPY_ENABLE_SCHEDULER (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Maximum number of entries in the scheduler // Maximum number of entries in the scheduler
@ -884,41 +941,41 @@ typedef double mp_float_t;
// inheritance makes some C functions inherently recursive, and adds a bit of // inheritance makes some C functions inherently recursive, and adds a bit of
// code overhead. // code overhead.
#ifndef MICROPY_MULTIPLE_INHERITANCE #ifndef MICROPY_MULTIPLE_INHERITANCE
#define MICROPY_MULTIPLE_INHERITANCE (1) #define MICROPY_MULTIPLE_INHERITANCE (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to implement attributes on functions // Whether to implement attributes on functions
#ifndef MICROPY_PY_FUNCTION_ATTRS #ifndef MICROPY_PY_FUNCTION_ATTRS
#define MICROPY_PY_FUNCTION_ATTRS (0) #define MICROPY_PY_FUNCTION_ATTRS (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to support the descriptors __get__, __set__, __delete__ // Whether to support the descriptors __get__, __set__, __delete__
// This costs some code size and makes load/store/delete of instance // This costs some code size and makes load/store/delete of instance
// attributes slower for the classes that use this feature // attributes slower for the classes that use this feature
#ifndef MICROPY_PY_DESCRIPTORS #ifndef MICROPY_PY_DESCRIPTORS
#define MICROPY_PY_DESCRIPTORS (0) #define MICROPY_PY_DESCRIPTORS (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to support class __delattr__ and __setattr__ methods // Whether to support class __delattr__ and __setattr__ methods
// This costs some code size and makes store/delete of instance // This costs some code size and makes store/delete of instance
// attributes slower for the classes that use this feature // attributes slower for the classes that use this feature
#ifndef MICROPY_PY_DELATTR_SETATTR #ifndef MICROPY_PY_DELATTR_SETATTR
#define MICROPY_PY_DELATTR_SETATTR (0) #define MICROPY_PY_DELATTR_SETATTR (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Support for async/await/async for/async with // Support for async/await/async for/async with
#ifndef MICROPY_PY_ASYNC_AWAIT #ifndef MICROPY_PY_ASYNC_AWAIT
#define MICROPY_PY_ASYNC_AWAIT (1) #define MICROPY_PY_ASYNC_AWAIT (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Support for literal string interpolation, f-strings (see PEP 498, Python 3.6+) // Support for literal string interpolation, f-strings (see PEP 498, Python 3.6+)
#ifndef MICROPY_PY_FSTRINGS #ifndef MICROPY_PY_FSTRINGS
#define MICROPY_PY_FSTRINGS (0) #define MICROPY_PY_FSTRINGS (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Support for assignment expressions with := (see PEP 572, Python 3.8+) // Support for assignment expressions with := (see PEP 572, Python 3.8+)
#ifndef MICROPY_PY_ASSIGN_EXPR #ifndef MICROPY_PY_ASSIGN_EXPR
#define MICROPY_PY_ASSIGN_EXPR (1) #define MICROPY_PY_ASSIGN_EXPR (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Non-standard .pend_throw() method for generators, allowing for // Non-standard .pend_throw() method for generators, allowing for
@ -927,7 +984,7 @@ typedef double mp_float_t;
// to generator's .send() or .__next__(). (This is useful to implement // to generator's .send() or .__next__(). (This is useful to implement
// async schedulers.) // async schedulers.)
#ifndef MICROPY_PY_GENERATOR_PEND_THROW #ifndef MICROPY_PY_GENERATOR_PEND_THROW
#define MICROPY_PY_GENERATOR_PEND_THROW (1) #define MICROPY_PY_GENERATOR_PEND_THROW (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Issue a warning when comparing str and bytes objects // Issue a warning when comparing str and bytes objects
@ -937,7 +994,7 @@ typedef double mp_float_t;
// Whether str object is proper unicode // Whether str object is proper unicode
#ifndef MICROPY_PY_BUILTINS_STR_UNICODE #ifndef MICROPY_PY_BUILTINS_STR_UNICODE
#define MICROPY_PY_BUILTINS_STR_UNICODE (0) #define MICROPY_PY_BUILTINS_STR_UNICODE (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to check for valid UTF-8 when converting bytes to str // Whether to check for valid UTF-8 when converting bytes to str
@ -947,42 +1004,42 @@ typedef double mp_float_t;
// Whether str.center() method provided // Whether str.center() method provided
#ifndef MICROPY_PY_BUILTINS_STR_CENTER #ifndef MICROPY_PY_BUILTINS_STR_CENTER
#define MICROPY_PY_BUILTINS_STR_CENTER (0) #define MICROPY_PY_BUILTINS_STR_CENTER (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether str.count() method provided // Whether str.count() method provided
#ifndef MICROPY_PY_BUILTINS_STR_COUNT #ifndef MICROPY_PY_BUILTINS_STR_COUNT
#define MICROPY_PY_BUILTINS_STR_COUNT (1) #define MICROPY_PY_BUILTINS_STR_COUNT (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether str % (...) formatting operator provided // Whether str % (...) formatting operator provided
#ifndef MICROPY_PY_BUILTINS_STR_OP_MODULO #ifndef MICROPY_PY_BUILTINS_STR_OP_MODULO
#define MICROPY_PY_BUILTINS_STR_OP_MODULO (1) #define MICROPY_PY_BUILTINS_STR_OP_MODULO (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether str.partition()/str.rpartition() method provided // Whether str.partition()/str.rpartition() method provided
#ifndef MICROPY_PY_BUILTINS_STR_PARTITION #ifndef MICROPY_PY_BUILTINS_STR_PARTITION
#define MICROPY_PY_BUILTINS_STR_PARTITION (0) #define MICROPY_PY_BUILTINS_STR_PARTITION (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether str.splitlines() method provided // Whether str.splitlines() method provided
#ifndef MICROPY_PY_BUILTINS_STR_SPLITLINES #ifndef MICROPY_PY_BUILTINS_STR_SPLITLINES
#define MICROPY_PY_BUILTINS_STR_SPLITLINES (0) #define MICROPY_PY_BUILTINS_STR_SPLITLINES (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to support bytearray object // Whether to support bytearray object
#ifndef MICROPY_PY_BUILTINS_BYTEARRAY #ifndef MICROPY_PY_BUILTINS_BYTEARRAY
#define MICROPY_PY_BUILTINS_BYTEARRAY (1) #define MICROPY_PY_BUILTINS_BYTEARRAY (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to support dict.fromkeys() class method // Whether to support dict.fromkeys() class method
#ifndef MICROPY_PY_BUILTINS_DICT_FROMKEYS #ifndef MICROPY_PY_BUILTINS_DICT_FROMKEYS
#define MICROPY_PY_BUILTINS_DICT_FROMKEYS (1) #define MICROPY_PY_BUILTINS_DICT_FROMKEYS (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to support memoryview object // Whether to support memoryview object
#ifndef MICROPY_PY_BUILTINS_MEMORYVIEW #ifndef MICROPY_PY_BUILTINS_MEMORYVIEW
#define MICROPY_PY_BUILTINS_MEMORYVIEW (0) #define MICROPY_PY_BUILTINS_MEMORYVIEW (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to support memoryview.itemsize attribute // Whether to support memoryview.itemsize attribute
@ -992,39 +1049,39 @@ typedef double mp_float_t;
// Whether to support set object // Whether to support set object
#ifndef MICROPY_PY_BUILTINS_SET #ifndef MICROPY_PY_BUILTINS_SET
#define MICROPY_PY_BUILTINS_SET (1) #define MICROPY_PY_BUILTINS_SET (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to support slice subscript operators and slice object // Whether to support slice subscript operators and slice object
#ifndef MICROPY_PY_BUILTINS_SLICE #ifndef MICROPY_PY_BUILTINS_SLICE
#define MICROPY_PY_BUILTINS_SLICE (1) #define MICROPY_PY_BUILTINS_SLICE (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to support slice attribute read access, // Whether to support slice attribute read access,
// i.e. slice.start, slice.stop, slice.step // i.e. slice.start, slice.stop, slice.step
#ifndef MICROPY_PY_BUILTINS_SLICE_ATTRS #ifndef MICROPY_PY_BUILTINS_SLICE_ATTRS
#define MICROPY_PY_BUILTINS_SLICE_ATTRS (0) #define MICROPY_PY_BUILTINS_SLICE_ATTRS (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to support the .indices(len) method on slice objects // Whether to support the .indices(len) method on slice objects
#ifndef MICROPY_PY_BUILTINS_SLICE_INDICES #ifndef MICROPY_PY_BUILTINS_SLICE_INDICES
#define MICROPY_PY_BUILTINS_SLICE_INDICES (0) #define MICROPY_PY_BUILTINS_SLICE_INDICES (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to support frozenset object // Whether to support frozenset object
#ifndef MICROPY_PY_BUILTINS_FROZENSET #ifndef MICROPY_PY_BUILTINS_FROZENSET
#define MICROPY_PY_BUILTINS_FROZENSET (0) #define MICROPY_PY_BUILTINS_FROZENSET (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to support property object // Whether to support property object
#ifndef MICROPY_PY_BUILTINS_PROPERTY #ifndef MICROPY_PY_BUILTINS_PROPERTY
#define MICROPY_PY_BUILTINS_PROPERTY (1) #define MICROPY_PY_BUILTINS_PROPERTY (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to implement the start/stop/step attributes (readback) on // Whether to implement the start/stop/step attributes (readback) on
// the "range" builtin type. Rarely used, and costs ~60 bytes (x86). // the "range" builtin type. Rarely used, and costs ~60 bytes (x86).
#ifndef MICROPY_PY_BUILTINS_RANGE_ATTRS #ifndef MICROPY_PY_BUILTINS_RANGE_ATTRS
#define MICROPY_PY_BUILTINS_RANGE_ATTRS (1) #define MICROPY_PY_BUILTINS_RANGE_ATTRS (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to support binary ops [only (in)equality is defined] between range // Whether to support binary ops [only (in)equality is defined] between range
@ -1042,7 +1099,7 @@ typedef double mp_float_t;
// Whether to support rounding of integers (incl bignum); eg round(123,-1)=120 // Whether to support rounding of integers (incl bignum); eg round(123,-1)=120
#ifndef MICROPY_PY_BUILTINS_ROUND_INT #ifndef MICROPY_PY_BUILTINS_ROUND_INT
#define MICROPY_PY_BUILTINS_ROUND_INT (0) #define MICROPY_PY_BUILTINS_ROUND_INT (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to support complete set of special methods for user // Whether to support complete set of special methods for user
@ -1051,7 +1108,7 @@ typedef double mp_float_t;
// "Reverse" methods are controlled by // "Reverse" methods are controlled by
// MICROPY_PY_REVERSE_SPECIAL_METHODS below. // MICROPY_PY_REVERSE_SPECIAL_METHODS below.
#ifndef MICROPY_PY_ALL_SPECIAL_METHODS #ifndef MICROPY_PY_ALL_SPECIAL_METHODS
#define MICROPY_PY_ALL_SPECIAL_METHODS (0) #define MICROPY_PY_ALL_SPECIAL_METHODS (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to support all inplace arithmetic operarion methods // Whether to support all inplace arithmetic operarion methods
@ -1064,17 +1121,17 @@ typedef double mp_float_t;
// (__radd__, etc.). Additionally gated by // (__radd__, etc.). Additionally gated by
// MICROPY_PY_ALL_SPECIAL_METHODS. // MICROPY_PY_ALL_SPECIAL_METHODS.
#ifndef MICROPY_PY_REVERSE_SPECIAL_METHODS #ifndef MICROPY_PY_REVERSE_SPECIAL_METHODS
#define MICROPY_PY_REVERSE_SPECIAL_METHODS (0) #define MICROPY_PY_REVERSE_SPECIAL_METHODS (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to support compile function // Whether to support compile function
#ifndef MICROPY_PY_BUILTINS_COMPILE #ifndef MICROPY_PY_BUILTINS_COMPILE
#define MICROPY_PY_BUILTINS_COMPILE (0) #define MICROPY_PY_BUILTINS_COMPILE (MICROPY_ENABLE_COMPILER && MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to support enumerate function(type) // Whether to support enumerate function(type)
#ifndef MICROPY_PY_BUILTINS_ENUMERATE #ifndef MICROPY_PY_BUILTINS_ENUMERATE
#define MICROPY_PY_BUILTINS_ENUMERATE (1) #define MICROPY_PY_BUILTINS_ENUMERATE (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to support eval and exec functions // Whether to support eval and exec functions
@ -1085,43 +1142,43 @@ typedef double mp_float_t;
// Whether to support the Python 2 execfile function // Whether to support the Python 2 execfile function
#ifndef MICROPY_PY_BUILTINS_EXECFILE #ifndef MICROPY_PY_BUILTINS_EXECFILE
#define MICROPY_PY_BUILTINS_EXECFILE (0) #define MICROPY_PY_BUILTINS_EXECFILE (MICROPY_ENABLE_COMPILER && MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to support filter function(type) // Whether to support filter function(type)
#ifndef MICROPY_PY_BUILTINS_FILTER #ifndef MICROPY_PY_BUILTINS_FILTER
#define MICROPY_PY_BUILTINS_FILTER (1) #define MICROPY_PY_BUILTINS_FILTER (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to support reversed function(type) // Whether to support reversed function(type)
#ifndef MICROPY_PY_BUILTINS_REVERSED #ifndef MICROPY_PY_BUILTINS_REVERSED
#define MICROPY_PY_BUILTINS_REVERSED (1) #define MICROPY_PY_BUILTINS_REVERSED (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to define "NotImplemented" special constant // Whether to define "NotImplemented" special constant
#ifndef MICROPY_PY_BUILTINS_NOTIMPLEMENTED #ifndef MICROPY_PY_BUILTINS_NOTIMPLEMENTED
#define MICROPY_PY_BUILTINS_NOTIMPLEMENTED (0) #define MICROPY_PY_BUILTINS_NOTIMPLEMENTED (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to provide the built-in input() function. The implementation of this // Whether to provide the built-in input() function. The implementation of this
// uses shared/readline, so can only be enabled if the port uses this readline. // uses shared/readline, so can only be enabled if the port uses this readline.
#ifndef MICROPY_PY_BUILTINS_INPUT #ifndef MICROPY_PY_BUILTINS_INPUT
#define MICROPY_PY_BUILTINS_INPUT (0) #define MICROPY_PY_BUILTINS_INPUT (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to support min/max functions // Whether to support min/max functions
#ifndef MICROPY_PY_BUILTINS_MIN_MAX #ifndef MICROPY_PY_BUILTINS_MIN_MAX
#define MICROPY_PY_BUILTINS_MIN_MAX (1) #define MICROPY_PY_BUILTINS_MIN_MAX (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Support for calls to pow() with 3 integer arguments // Support for calls to pow() with 3 integer arguments
#ifndef MICROPY_PY_BUILTINS_POW3 #ifndef MICROPY_PY_BUILTINS_POW3
#define MICROPY_PY_BUILTINS_POW3 (0) #define MICROPY_PY_BUILTINS_POW3 (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to provide the help function // Whether to provide the help function
#ifndef MICROPY_PY_BUILTINS_HELP #ifndef MICROPY_PY_BUILTINS_HELP
#define MICROPY_PY_BUILTINS_HELP (0) #define MICROPY_PY_BUILTINS_HELP (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Use this to configure the help text shown for help(). It should be a // Use this to configure the help text shown for help(). It should be a
@ -1132,17 +1189,17 @@ typedef double mp_float_t;
// Add the ability to list the available modules when executing help('modules') // Add the ability to list the available modules when executing help('modules')
#ifndef MICROPY_PY_BUILTINS_HELP_MODULES #ifndef MICROPY_PY_BUILTINS_HELP_MODULES
#define MICROPY_PY_BUILTINS_HELP_MODULES (0) #define MICROPY_PY_BUILTINS_HELP_MODULES (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to set __file__ for imported modules // Whether to set __file__ for imported modules
#ifndef MICROPY_PY___FILE__ #ifndef MICROPY_PY___FILE__
#define MICROPY_PY___FILE__ (1) #define MICROPY_PY___FILE__ (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to provide mem-info related functions in micropython module // Whether to provide mem-info related functions in micropython module
#ifndef MICROPY_PY_MICROPYTHON_MEM_INFO #ifndef MICROPY_PY_MICROPYTHON_MEM_INFO
#define MICROPY_PY_MICROPYTHON_MEM_INFO (0) #define MICROPY_PY_MICROPYTHON_MEM_INFO (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to provide "micropython.stack_use" function // Whether to provide "micropython.stack_use" function
@ -1159,13 +1216,13 @@ typedef double mp_float_t;
// underlying code is shared with "bytearray" builtin type, so to // underlying code is shared with "bytearray" builtin type, so to
// get real savings, it should be disabled too. // get real savings, it should be disabled too.
#ifndef MICROPY_PY_ARRAY #ifndef MICROPY_PY_ARRAY
#define MICROPY_PY_ARRAY (1) #define MICROPY_PY_ARRAY (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to support slice assignments for array (and bytearray). // Whether to support slice assignments for array (and bytearray).
// This is rarely used, but adds ~0.5K of code. // This is rarely used, but adds ~0.5K of code.
#ifndef MICROPY_PY_ARRAY_SLICE_ASSIGN #ifndef MICROPY_PY_ARRAY_SLICE_ASSIGN
#define MICROPY_PY_ARRAY_SLICE_ASSIGN (0) #define MICROPY_PY_ARRAY_SLICE_ASSIGN (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to support nonstandard typecodes "O", "P" and "S" // Whether to support nonstandard typecodes "O", "P" and "S"
@ -1177,22 +1234,22 @@ typedef double mp_float_t;
// Whether to support attrtuple type (MicroPython extension) // Whether to support attrtuple type (MicroPython extension)
// It provides space-efficient tuples with attribute access // It provides space-efficient tuples with attribute access
#ifndef MICROPY_PY_ATTRTUPLE #ifndef MICROPY_PY_ATTRTUPLE
#define MICROPY_PY_ATTRTUPLE (1) #define MICROPY_PY_ATTRTUPLE (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to provide "collections" module // Whether to provide "collections" module
#ifndef MICROPY_PY_COLLECTIONS #ifndef MICROPY_PY_COLLECTIONS
#define MICROPY_PY_COLLECTIONS (1) #define MICROPY_PY_COLLECTIONS (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to provide "ucollections.deque" type // Whether to provide "ucollections.deque" type
#ifndef MICROPY_PY_COLLECTIONS_DEQUE #ifndef MICROPY_PY_COLLECTIONS_DEQUE
#define MICROPY_PY_COLLECTIONS_DEQUE (0) #define MICROPY_PY_COLLECTIONS_DEQUE (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to provide "collections.OrderedDict" type // Whether to provide "collections.OrderedDict" type
#ifndef MICROPY_PY_COLLECTIONS_ORDEREDDICT #ifndef MICROPY_PY_COLLECTIONS_ORDEREDDICT
#define MICROPY_PY_COLLECTIONS_ORDEREDDICT (0) #define MICROPY_PY_COLLECTIONS_ORDEREDDICT (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to provide the _asdict function for namedtuple // Whether to provide the _asdict function for namedtuple
@ -1202,22 +1259,22 @@ typedef double mp_float_t;
// Whether to provide "math" module // Whether to provide "math" module
#ifndef MICROPY_PY_MATH #ifndef MICROPY_PY_MATH
#define MICROPY_PY_MATH (1) #define MICROPY_PY_MATH (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to provide special math functions: math.{erf,erfc,gamma,lgamma} // Whether to provide special math functions: math.{erf,erfc,gamma,lgamma}
#ifndef MICROPY_PY_MATH_SPECIAL_FUNCTIONS #ifndef MICROPY_PY_MATH_SPECIAL_FUNCTIONS
#define MICROPY_PY_MATH_SPECIAL_FUNCTIONS (0) #define MICROPY_PY_MATH_SPECIAL_FUNCTIONS (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to provide math.factorial function // Whether to provide math.factorial function
#ifndef MICROPY_PY_MATH_FACTORIAL #ifndef MICROPY_PY_MATH_FACTORIAL
#define MICROPY_PY_MATH_FACTORIAL (0) #define MICROPY_PY_MATH_FACTORIAL (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to provide math.isclose function // Whether to provide math.isclose function
#ifndef MICROPY_PY_MATH_ISCLOSE #ifndef MICROPY_PY_MATH_ISCLOSE
#define MICROPY_PY_MATH_ISCLOSE (0) #define MICROPY_PY_MATH_ISCLOSE (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to provide fix for atan2 Inf handling. // Whether to provide fix for atan2 Inf handling.
@ -1242,12 +1299,12 @@ typedef double mp_float_t;
// Whether to provide "cmath" module // Whether to provide "cmath" module
#ifndef MICROPY_PY_CMATH #ifndef MICROPY_PY_CMATH
#define MICROPY_PY_CMATH (0) #define MICROPY_PY_CMATH (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to provide "gc" module // Whether to provide "gc" module
#ifndef MICROPY_PY_GC #ifndef MICROPY_PY_GC
#define MICROPY_PY_GC (1) #define MICROPY_PY_GC (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to return number of collected objects from gc.collect() // Whether to return number of collected objects from gc.collect()
@ -1257,28 +1314,17 @@ typedef double mp_float_t;
// Whether to provide "io" module // Whether to provide "io" module
#ifndef MICROPY_PY_IO #ifndef MICROPY_PY_IO
#define MICROPY_PY_IO (1) #define MICROPY_PY_IO (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to provide "io.IOBase" class to support user streams // Whether to provide "io.IOBase" class to support user streams
#ifndef MICROPY_PY_IO_IOBASE #ifndef MICROPY_PY_IO_IOBASE
#define MICROPY_PY_IO_IOBASE (0) #define MICROPY_PY_IO_IOBASE (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif
// Whether to provide "uio.resource_stream()" function with
// the semantics of CPython's pkg_resources.resource_stream()
// (allows to access binary resources in frozen source packages).
// Note that the same functionality can be achieved in "pure
// Python" by prepocessing binary resources into Python source
// and bytecode-freezing it (with a simple helper module available
// e.g. in micropython-lib).
#ifndef MICROPY_PY_IO_RESOURCE_STREAM
#define MICROPY_PY_IO_RESOURCE_STREAM (0)
#endif #endif
// Whether to provide "io.FileIO" class // Whether to provide "io.FileIO" class
#ifndef MICROPY_PY_IO_FILEIO #ifndef MICROPY_PY_IO_FILEIO
#define MICROPY_PY_IO_FILEIO (0) #define MICROPY_PY_IO_FILEIO (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to provide "io.BytesIO" class // Whether to provide "io.BytesIO" class
@ -1293,17 +1339,22 @@ typedef double mp_float_t;
// Whether to provide "struct" module // Whether to provide "struct" module
#ifndef MICROPY_PY_STRUCT #ifndef MICROPY_PY_STRUCT
#define MICROPY_PY_STRUCT (1) #define MICROPY_PY_STRUCT (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif #endif
// Whether to provide "sys" module // Whether to provide "sys" module
#ifndef MICROPY_PY_SYS #ifndef MICROPY_PY_SYS
#define MICROPY_PY_SYS (1) #define MICROPY_PY_SYS (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES)
#endif
// Whether to initialise "sys.path" and "sys.argv" to their defaults in mp_init()
#ifndef MICROPY_PY_SYS_PATH_ARGV_DEFAULTS
#define MICROPY_PY_SYS_PATH_ARGV_DEFAULTS (MICROPY_PY_SYS)
#endif #endif
// Whether to provide "sys.maxsize" constant // Whether to provide "sys.maxsize" constant
#ifndef MICROPY_PY_SYS_MAXSIZE #ifndef MICROPY_PY_SYS_MAXSIZE
#define MICROPY_PY_SYS_MAXSIZE (0) #define MICROPY_PY_SYS_MAXSIZE (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to provide "sys.modules" dictionary // Whether to provide "sys.modules" dictionary
@ -1339,18 +1390,18 @@ typedef double mp_float_t;
// Whether to provide sys.{stdin,stdout,stderr} objects // Whether to provide sys.{stdin,stdout,stderr} objects
#ifndef MICROPY_PY_SYS_STDFILES #ifndef MICROPY_PY_SYS_STDFILES
#define MICROPY_PY_SYS_STDFILES (0) #define MICROPY_PY_SYS_STDFILES (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to provide sys.{stdin,stdout,stderr}.buffer object // Whether to provide sys.{stdin,stdout,stderr}.buffer object
// This is implemented per-port // This is implemented per-port
#ifndef MICROPY_PY_SYS_STDIO_BUFFER #ifndef MICROPY_PY_SYS_STDIO_BUFFER
#define MICROPY_PY_SYS_STDIO_BUFFER (0) #define MICROPY_PY_SYS_STDIO_BUFFER (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to provide "uerrno" module // Whether to provide "uerrno" module
#ifndef MICROPY_PY_UERRNO #ifndef MICROPY_PY_UERRNO
#define MICROPY_PY_UERRNO (0) #define MICROPY_PY_UERRNO (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to provide the uerrno.errorcode dict // Whether to provide the uerrno.errorcode dict
@ -1360,7 +1411,7 @@ typedef double mp_float_t;
// Whether to provide "uselect" module (baremetal implementation) // Whether to provide "uselect" module (baremetal implementation)
#ifndef MICROPY_PY_USELECT #ifndef MICROPY_PY_USELECT
#define MICROPY_PY_USELECT (0) #define MICROPY_PY_USELECT (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to enable the select() function in the "uselect" module (baremetal // Whether to enable the select() function in the "uselect" module (baremetal
@ -1406,11 +1457,11 @@ typedef double mp_float_t;
// Extended modules // Extended modules
#ifndef MICROPY_PY_UASYNCIO #ifndef MICROPY_PY_UASYNCIO
#define MICROPY_PY_UASYNCIO (0) #define MICROPY_PY_UASYNCIO (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
#ifndef MICROPY_PY_UCTYPES #ifndef MICROPY_PY_UCTYPES
#define MICROPY_PY_UCTYPES (0) #define MICROPY_PY_UCTYPES (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to provide SHORT, INT, LONG, etc. types in addition to // Whether to provide SHORT, INT, LONG, etc. types in addition to
@ -1420,11 +1471,11 @@ typedef double mp_float_t;
#endif #endif
#ifndef MICROPY_PY_UZLIB #ifndef MICROPY_PY_UZLIB
#define MICROPY_PY_UZLIB (0) #define MICROPY_PY_UZLIB (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
#ifndef MICROPY_PY_UJSON #ifndef MICROPY_PY_UJSON
#define MICROPY_PY_UJSON (0) #define MICROPY_PY_UJSON (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to support the "separators" argument to dump, dumps // Whether to support the "separators" argument to dump, dumps
@ -1437,7 +1488,7 @@ typedef double mp_float_t;
#endif #endif
#ifndef MICROPY_PY_URE #ifndef MICROPY_PY_URE
#define MICROPY_PY_URE (0) #define MICROPY_PY_URE (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
#ifndef MICROPY_PY_URE_DEBUG #ifndef MICROPY_PY_URE_DEBUG
@ -1453,20 +1504,20 @@ typedef double mp_float_t;
#endif #endif
#ifndef MICROPY_PY_URE_SUB #ifndef MICROPY_PY_URE_SUB
#define MICROPY_PY_URE_SUB (0) #define MICROPY_PY_URE_SUB (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
#ifndef MICROPY_PY_UHEAPQ #ifndef MICROPY_PY_UHEAPQ
#define MICROPY_PY_UHEAPQ (0) #define MICROPY_PY_UHEAPQ (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Optimized heap queue for relative timestamps // Optimized heap queue for relative timestamps (only used by uasyncio v2)
#ifndef MICROPY_PY_UTIMEQ #ifndef MICROPY_PY_UTIMEQ
#define MICROPY_PY_UTIMEQ (0) #define MICROPY_PY_UTIMEQ (0)
#endif #endif
#ifndef MICROPY_PY_UHASHLIB #ifndef MICROPY_PY_UHASHLIB
#define MICROPY_PY_UHASHLIB (0) #define MICROPY_PY_UHASHLIB (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
#ifndef MICROPY_PY_UHASHLIB_MD5 #ifndef MICROPY_PY_UHASHLIB_MD5
@ -1495,25 +1546,70 @@ typedef double mp_float_t;
#endif #endif
#ifndef MICROPY_PY_UBINASCII #ifndef MICROPY_PY_UBINASCII
#define MICROPY_PY_UBINASCII (0) #define MICROPY_PY_UBINASCII (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Depends on MICROPY_PY_UZLIB // Depends on MICROPY_PY_UZLIB
#ifndef MICROPY_PY_UBINASCII_CRC32 #ifndef MICROPY_PY_UBINASCII_CRC32
#define MICROPY_PY_UBINASCII_CRC32 (0) #define MICROPY_PY_UBINASCII_CRC32 (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
#ifndef MICROPY_PY_URANDOM #ifndef MICROPY_PY_URANDOM
#define MICROPY_PY_URANDOM (0) #define MICROPY_PY_URANDOM (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
// Whether to include: randrange, randint, choice, random, uniform // Whether to include: randrange, randint, choice, random, uniform
#ifndef MICROPY_PY_URANDOM_EXTRA_FUNCS #ifndef MICROPY_PY_URANDOM_EXTRA_FUNCS
#define MICROPY_PY_URANDOM_EXTRA_FUNCS (0) #define MICROPY_PY_URANDOM_EXTRA_FUNCS (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif
#ifndef MICROPY_PY_MACHINE
#define MICROPY_PY_MACHINE (0)
#endif
// Whether to include: bitstream
#ifndef MICROPY_PY_MACHINE_BITSTREAM
#define MICROPY_PY_MACHINE_BITSTREAM (0)
#endif
// Whether to include: time_pulse_us
#ifndef MICROPY_PY_MACHINE_PULSE
#define MICROPY_PY_MACHINE_PULSE (0)
#endif
#ifndef MICROPY_PY_MACHINE_I2C
#define MICROPY_PY_MACHINE_I2C (0)
#endif
// Whether to provide the "machine.SoftI2C" class
#ifndef MICROPY_PY_MACHINE_SOFTI2C
#define MICROPY_PY_MACHINE_SOFTI2C (0)
#endif
#ifndef MICROPY_PY_MACHINE_SPI
#define MICROPY_PY_MACHINE_SPI (0)
#endif
// Whether to provide the "machine.SoftSPI" class
#ifndef MICROPY_PY_MACHINE_SOFTSPI
#define MICROPY_PY_MACHINE_SOFTSPI (0)
#endif
#ifndef MICROPY_PY_USSL
#define MICROPY_PY_USSL (0)
#endif
// Whether to add finaliser code to ussl objects
#ifndef MICROPY_PY_USSL_FINALISER
#define MICROPY_PY_USSL_FINALISER (0)
#endif
#ifndef MICROPY_PY_UWEBSOCKET
#define MICROPY_PY_UWEBSOCKET (0)
#endif #endif
#ifndef MICROPY_PY_FRAMEBUF #ifndef MICROPY_PY_FRAMEBUF
#define MICROPY_PY_FRAMEBUF (0) #define MICROPY_PY_FRAMEBUF (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif #endif
#ifndef MICROPY_PY_BTREE #ifndef MICROPY_PY_BTREE
@ -1523,6 +1619,12 @@ typedef double mp_float_t;
#ifndef MICROPY_HW_ENABLE_USB #ifndef MICROPY_HW_ENABLE_USB
#define MICROPY_HW_ENABLE_USB (0) #define MICROPY_HW_ENABLE_USB (0)
#endif #endif
// Whether to provide the low-level "_onewire" module
#ifndef MICROPY_PY_ONEWIRE
#define MICROPY_PY_ONEWIRE (0)
#endif
/*****************************************************************************/ /*****************************************************************************/
/* Hooks for a port to add builtins */ /* Hooks for a port to add builtins */
@ -1531,6 +1633,12 @@ typedef double mp_float_t;
#define MICROPY_PORT_BUILTINS #define MICROPY_PORT_BUILTINS
#endif #endif
// Additional builtin function definitions for extension by command-line, boards or variants.
// See modbuiltins.c:mp_module_builtins_globals_table for format.
#ifndef MICROPY_PORT_EXTRA_BUILTINS
#define MICROPY_PORT_EXTRA_BUILTINS
#endif
// Additional builtin module definitions - see objmodule.c:mp_builtin_module_table for format. // Additional builtin module definitions - see objmodule.c:mp_builtin_module_table for format.
#ifndef MICROPY_PORT_BUILTIN_MODULES #ifndef MICROPY_PORT_BUILTIN_MODULES
#define MICROPY_PORT_BUILTIN_MODULES #define MICROPY_PORT_BUILTIN_MODULES
@ -1549,6 +1657,30 @@ typedef double mp_float_t;
/*****************************************************************************/ /*****************************************************************************/
/* Hooks for a port to wrap functions with attributes */ /* Hooks for a port to wrap functions with attributes */
#ifndef MICROPY_WRAP_MP_BINARY_OP
#define MICROPY_WRAP_MP_BINARY_OP(f) f
#endif
#ifndef MICROPY_WRAP_MP_EXECUTE_BYTECODE
#define MICROPY_WRAP_MP_EXECUTE_BYTECODE(f) f
#endif
#ifndef MICROPY_WRAP_MP_LOAD_GLOBAL
#define MICROPY_WRAP_MP_LOAD_GLOBAL(f) f
#endif
#ifndef MICROPY_WRAP_MP_LOAD_NAME
#define MICROPY_WRAP_MP_LOAD_NAME(f) f
#endif
#ifndef MICROPY_WRAP_MP_MAP_LOOKUP
#define MICROPY_WRAP_MP_MAP_LOOKUP(f) f
#endif
#ifndef MICROPY_WRAP_MP_OBJ_GET_TYPE
#define MICROPY_WRAP_MP_OBJ_GET_TYPE(f) f
#endif
#ifndef MICROPY_WRAP_MP_SCHED_EXCEPTION #ifndef MICROPY_WRAP_MP_SCHED_EXCEPTION
#define MICROPY_WRAP_MP_SCHED_EXCEPTION(f) f #define MICROPY_WRAP_MP_SCHED_EXCEPTION(f) f
#endif #endif

11
py/mpstate.h
View File

@ -44,7 +44,6 @@
#if MICROPY_DYNAMIC_COMPILER #if MICROPY_DYNAMIC_COMPILER
typedef struct mp_dynamic_compiler_t { typedef struct mp_dynamic_compiler_t {
uint8_t small_int_bits; // must be <= host small_int_bits uint8_t small_int_bits; // must be <= host small_int_bits
bool opt_cache_map_lookup_in_bytecode;
bool py_builtins_str_unicode; bool py_builtins_str_unicode;
uint8_t native_arch; uint8_t native_arch;
uint8_t nlr_buf_num_regs; uint8_t nlr_buf_num_regs;
@ -165,9 +164,12 @@ typedef struct _mp_state_vm_t {
// dictionary for the __main__ module // dictionary for the __main__ module
mp_obj_dict_t dict_main; mp_obj_dict_t dict_main;
// these two lists must be initialised per port, after the call to mp_init #if MICROPY_PY_SYS
// If MICROPY_PY_SYS_PATH_ARGV_DEFAULTS is not enabled then these two lists
// must be initialised after the call to mp_init.
mp_obj_list_t mp_sys_path_obj; mp_obj_list_t mp_sys_path_obj;
mp_obj_list_t mp_sys_argv_obj; mp_obj_list_t mp_sys_argv_obj;
#endif
// dictionary for overridden builtins // dictionary for overridden builtins
#if MICROPY_CAN_OVERRIDE_BUILTINS #if MICROPY_CAN_OVERRIDE_BUILTINS
@ -230,6 +232,11 @@ typedef struct _mp_state_vm_t {
// This is a global mutex used to make the VM/runtime thread-safe. // This is a global mutex used to make the VM/runtime thread-safe.
mp_thread_mutex_t gil_mutex; mp_thread_mutex_t gil_mutex;
#endif #endif
#if MICROPY_OPT_MAP_LOOKUP_CACHE
// See mp_map_lookup.
uint8_t map_lookup_cache[MICROPY_OPT_MAP_LOOKUP_CACHE_SIZE];
#endif
} mp_state_vm_t; } mp_state_vm_t;
// This structure holds state that is specific to a given thread. // This structure holds state that is specific to a given thread.

17
py/mpz.c
View File

@ -717,6 +717,7 @@ void mpz_set(mpz_t *dest, const mpz_t *src) {
void mpz_set_from_int(mpz_t *z, mp_int_t val) { void mpz_set_from_int(mpz_t *z, mp_int_t val) {
if (val == 0) { if (val == 0) {
z->neg = 0;
z->len = 0; z->len = 0;
return; return;
} }
@ -903,10 +904,6 @@ bool mpz_is_even(const mpz_t *z) {
#endif #endif
int mpz_cmp(const mpz_t *z1, const mpz_t *z2) { int mpz_cmp(const mpz_t *z1, const mpz_t *z2) {
// to catch comparison of -0 with +0
if (z1->len == 0 && z2->len == 0) {
return 0;
}
int cmp = (int)z2->neg - (int)z1->neg; int cmp = (int)z2->neg - (int)z1->neg;
if (cmp != 0) { if (cmp != 0) {
return cmp; return cmp;
@ -1056,7 +1053,9 @@ void mpz_neg_inpl(mpz_t *dest, const mpz_t *z) {
if (dest != z) { if (dest != z) {
mpz_set(dest, z); mpz_set(dest, z);
} }
if (dest->len) {
dest->neg = 1 - dest->neg; dest->neg = 1 - dest->neg;
}
} }
/* computes dest = ~z (= -z - 1) /* computes dest = ~z (= -z - 1)
@ -1152,7 +1151,7 @@ void mpz_add_inpl(mpz_t *dest, const mpz_t *lhs, const mpz_t *rhs) {
dest->len = mpn_sub(dest->dig, lhs->dig, lhs->len, rhs->dig, rhs->len); dest->len = mpn_sub(dest->dig, lhs->dig, lhs->len, rhs->dig, rhs->len);
} }
dest->neg = lhs->neg; dest->neg = lhs->neg & !!dest->len;
} }
/* computes dest = lhs - rhs /* computes dest = lhs - rhs
@ -1176,7 +1175,9 @@ void mpz_sub_inpl(mpz_t *dest, const mpz_t *lhs, const mpz_t *rhs) {
dest->len = mpn_sub(dest->dig, lhs->dig, lhs->len, rhs->dig, rhs->len); dest->len = mpn_sub(dest->dig, lhs->dig, lhs->len, rhs->dig, rhs->len);
} }
if (neg) { if (dest->len == 0) {
dest->neg = 0;
} else if (neg) {
dest->neg = 1 - lhs->neg; dest->neg = 1 - lhs->neg;
} else { } else {
dest->neg = lhs->neg; dest->neg = lhs->neg;
@ -1488,14 +1489,16 @@ void mpz_divmod_inpl(mpz_t *dest_quo, mpz_t *dest_rem, const mpz_t *lhs, const m
mpz_need_dig(dest_quo, lhs->len + 1); // +1 necessary? mpz_need_dig(dest_quo, lhs->len + 1); // +1 necessary?
memset(dest_quo->dig, 0, (lhs->len + 1) * sizeof(mpz_dig_t)); memset(dest_quo->dig, 0, (lhs->len + 1) * sizeof(mpz_dig_t));
dest_quo->neg = 0;
dest_quo->len = 0; dest_quo->len = 0;
mpz_need_dig(dest_rem, lhs->len + 1); // +1 necessary? mpz_need_dig(dest_rem, lhs->len + 1); // +1 necessary?
mpz_set(dest_rem, lhs); mpz_set(dest_rem, lhs);
mpn_div(dest_rem->dig, &dest_rem->len, rhs->dig, rhs->len, dest_quo->dig, &dest_quo->len); mpn_div(dest_rem->dig, &dest_rem->len, rhs->dig, rhs->len, dest_quo->dig, &dest_quo->len);
dest_rem->neg &= !!dest_rem->len;
// check signs and do Python style modulo // check signs and do Python style modulo
if (lhs->neg != rhs->neg) { if (lhs->neg != rhs->neg) {
dest_quo->neg = 1; dest_quo->neg = !!dest_quo->len;
if (!mpz_is_zero(dest_rem)) { if (!mpz_is_zero(dest_rem)) {
mpz_t mpzone; mpz_t mpzone;
mpz_init_from_int(&mpzone, -1); mpz_init_from_int(&mpzone, -1);

3
py/mpz.h
View File

@ -91,6 +91,7 @@ typedef int8_t mpz_dbl_dig_signed_t;
#define MPZ_NUM_DIG_FOR_LL ((sizeof(long long) * 8 + MPZ_DIG_SIZE - 1) / MPZ_DIG_SIZE) #define MPZ_NUM_DIG_FOR_LL ((sizeof(long long) * 8 + MPZ_DIG_SIZE - 1) / MPZ_DIG_SIZE)
typedef struct _mpz_t { typedef struct _mpz_t {
// Zero has neg=0, len=0. Negative zero is not allowed.
size_t neg : 1; size_t neg : 1;
size_t fixed_dig : 1; size_t fixed_dig : 1;
size_t alloc : (8 * sizeof(size_t) - 2); size_t alloc : (8 * sizeof(size_t) - 2);
@ -119,7 +120,7 @@ static inline bool mpz_is_zero(const mpz_t *z) {
return z->len == 0; return z->len == 0;
} }
static inline bool mpz_is_neg(const mpz_t *z) { static inline bool mpz_is_neg(const mpz_t *z) {
return z->len != 0 && z->neg != 0; return z->neg != 0;
} }
int mpz_cmp(const mpz_t *lhs, const mpz_t *rhs); int mpz_cmp(const mpz_t *lhs, const mpz_t *rhs);

2
py/obj.c
View File

@ -43,7 +43,7 @@
#include "supervisor/shared/stack.h" #include "supervisor/shared/stack.h"
#include "supervisor/shared/translate.h" #include "supervisor/shared/translate.h"
const mp_obj_type_t *mp_obj_get_type(mp_const_obj_t o_in) { const mp_obj_type_t *MICROPY_WRAP_MP_OBJ_GET_TYPE(mp_obj_get_type)(mp_const_obj_t o_in) {
#if MICROPY_OBJ_IMMEDIATE_OBJS && MICROPY_OBJ_REPR == MICROPY_OBJ_REPR_A #if MICROPY_OBJ_IMMEDIATE_OBJS && MICROPY_OBJ_REPR == MICROPY_OBJ_REPR_A
if (mp_obj_is_obj(o_in)) { if (mp_obj_is_obj(o_in)) {

1
py/obj.h
View File

@ -617,6 +617,7 @@ struct _mp_obj_type_t {
// //
// dest[0] = MP_OBJ_NULL means load // dest[0] = MP_OBJ_NULL means load
// return: for fail, do nothing // return: for fail, do nothing
// for fail but continue lookup in locals_dict, dest[1] = MP_OBJ_SENTINEL
// for attr, dest[0] = value // for attr, dest[0] = value
// for method, dest[0] = method, dest[1] = self // for method, dest[0] = method, dest[1] = self
// //

2
py/objfun.h
View File

@ -39,8 +39,6 @@ typedef struct _mp_obj_fun_bc_t {
// the following extra_args array is allocated space to take (in order): // the following extra_args array is allocated space to take (in order):
// - values of positional default args (if any) // - values of positional default args (if any)
// - a single slot for default kw args dict (if it has them) // - a single slot for default kw args dict (if it has them)
// - a single slot for var args tuple (if it takes them)
// - a single slot for kw args dict (if it takes them)
mp_obj_t extra_args[]; mp_obj_t extra_args[];
} mp_obj_fun_bc_t; } mp_obj_fun_bc_t;

67
py/objmodule.c
View File

@ -36,6 +36,10 @@
#include "genhdr/moduledefs.h" #include "genhdr/moduledefs.h"
#if MICROPY_MODULE_BUILTIN_INIT
STATIC void mp_module_call_init(mp_obj_t module_name, mp_obj_t module_obj);
#endif
STATIC void module_print(const mp_print_t *print, mp_obj_t self_in, mp_print_kind_t kind) { STATIC void module_print(const mp_print_t *print, mp_obj_t self_in, mp_print_kind_t kind) {
(void)kind; (void)kind;
mp_obj_module_t *self = MP_OBJ_TO_PTR(self_in); mp_obj_module_t *self = MP_OBJ_TO_PTR(self_in);
@ -279,47 +283,56 @@ STATIC const mp_rom_map_elem_t mp_builtin_module_table[] = {
MP_DEFINE_CONST_MAP(mp_builtin_module_map, mp_builtin_module_table); MP_DEFINE_CONST_MAP(mp_builtin_module_map, mp_builtin_module_table);
// returns MP_OBJ_NULL if not found // Tries to find a loaded module, otherwise attempts to load a builtin, otherwise MP_OBJ_NULL.
mp_obj_t mp_module_get(qstr module_name) { mp_obj_t mp_module_get_loaded_or_builtin(qstr module_name) {
mp_map_t *mp_loaded_modules_map = &MP_STATE_VM(mp_loaded_modules_dict).map; // First try loaded modules.
// lookup module mp_map_elem_t *elem = mp_map_lookup(&MP_STATE_VM(mp_loaded_modules_dict).map, MP_OBJ_NEW_QSTR(module_name), MP_MAP_LOOKUP);
mp_map_elem_t *el = mp_map_lookup(mp_loaded_modules_map, MP_OBJ_NEW_QSTR(module_name), MP_MAP_LOOKUP);
if (el == NULL) { if (!elem) {
// module not found, look for builtin module names #if MICROPY_MODULE_WEAK_LINKS
el = mp_map_lookup((mp_map_t *)&mp_builtin_module_map, MP_OBJ_NEW_QSTR(module_name), MP_MAP_LOOKUP); return mp_module_get_builtin(module_name);
if (el == NULL) { #else
// Otherwise try builtin.
elem = mp_map_lookup((mp_map_t *)&mp_builtin_module_map, MP_OBJ_NEW_QSTR(module_name), MP_MAP_LOOKUP);
if (!elem) {
return MP_OBJ_NULL; return MP_OBJ_NULL;
} }
mp_module_call_init(module_name, el->value);
#if MICROPY_MODULE_BUILTIN_INIT
// If found, it's a newly loaded built-in, so init it.
mp_module_call_init(MP_OBJ_NEW_QSTR(module_name), elem->value);
#endif
#endif
} }
// module found, return it return elem->value;
return el->value;
}
void mp_module_register(qstr qst, mp_obj_t module) {
mp_map_t *mp_loaded_modules_map = &MP_STATE_VM(mp_loaded_modules_dict).map;
mp_map_lookup(mp_loaded_modules_map, MP_OBJ_NEW_QSTR(qst), MP_MAP_LOOKUP_ADD_IF_NOT_FOUND)->value = module;
} }
#if MICROPY_MODULE_WEAK_LINKS #if MICROPY_MODULE_WEAK_LINKS
// Search for u"foo" in built-in modules, return MP_OBJ_NULL if not found // Tries to find a loaded module, otherwise attempts to load a builtin, otherwise MP_OBJ_NULL.
mp_obj_t mp_module_search_umodule(const char *module_str) { mp_obj_t mp_module_get_builtin(qstr module_name) {
for (size_t i = 0; i < MP_ARRAY_SIZE(mp_builtin_module_table); ++i) { // Try builtin.
const mp_map_elem_t *entry = (const mp_map_elem_t *)&mp_builtin_module_table[i]; mp_map_elem_t *elem = mp_map_lookup((mp_map_t *)&mp_builtin_module_map, MP_OBJ_NEW_QSTR(module_name), MP_MAP_LOOKUP);
const char *key = qstr_str(MP_OBJ_QSTR_VALUE(entry->key)); if (!elem) {
if (key[0] == 'u' && strcmp(&key[1], module_str) == 0) { return MP_OBJ_NULL;
return (mp_obj_t)entry->value;
} }
} #if MICROPY_MODULE_BUILTIN_INIT
return MP_OBJ_NULL; // If found, it's a newly loaded built-in, so init it.
mp_module_call_init(MP_OBJ_NEW_QSTR(module_name), elem->value);
#endif
return elem->value;
} }
#endif #endif
#if MICROPY_MODULE_BUILTIN_INIT #if MICROPY_MODULE_BUILTIN_INIT
void mp_module_call_init(qstr module_name, mp_obj_t module_obj) { STATIC void mp_module_register(mp_obj_t module_name, mp_obj_t module) {
mp_map_t *mp_loaded_modules_map = &MP_STATE_VM(mp_loaded_modules_dict).map;
mp_map_lookup(mp_loaded_modules_map, module_name, MP_MAP_LOOKUP_ADD_IF_NOT_FOUND)->value = module;
}
STATIC void mp_module_call_init(mp_obj_t module_name, mp_obj_t module_obj) {
// Look for __init__ and call it if it exists // Look for __init__ and call it if it exists
mp_obj_t dest[2]; mp_obj_t dest[2];
mp_load_method_maybe(module_obj, MP_QSTR___init__, dest); mp_load_method_maybe(module_obj, MP_QSTR___init__, dest);

15
py/objmodule.h
View File

@ -30,18 +30,9 @@
extern const mp_map_t mp_builtin_module_map; extern const mp_map_t mp_builtin_module_map;
mp_obj_t mp_module_get(qstr module_name); mp_obj_t mp_module_get_loaded_or_builtin(qstr module_name);
void mp_module_register(qstr qstr, mp_obj_t module); #if MICROPY_MODULE_WEAK_LINKS
mp_obj_t mp_module_get_builtin(qstr module_name);
mp_obj_t mp_module_search_umodule(const char *module_str);
#if MICROPY_MODULE_BUILTIN_INIT
void mp_module_call_init(qstr module_name, mp_obj_t module_obj);
#else
static inline void mp_module_call_init(qstr module_name, mp_obj_t module_obj) {
(void)module_name;
(void)module_obj;
}
#endif #endif
#endif // MICROPY_INCLUDED_PY_OBJMODULE_H #endif // MICROPY_INCLUDED_PY_OBJMODULE_H

1
py/objtype.c
View File

@ -616,6 +616,7 @@ STATIC void mp_obj_instance_load_attr(mp_obj_t self_in, qstr attr, mp_obj_t *des
assert(mp_obj_is_instance_type(mp_obj_get_type(self_in))); assert(mp_obj_is_instance_type(mp_obj_get_type(self_in)));
mp_obj_instance_t *self = MP_OBJ_TO_PTR(self_in); mp_obj_instance_t *self = MP_OBJ_TO_PTR(self_in);
// Note: This is fast-path'ed in the VM for the MP_BC_LOAD_ATTR operation.
mp_map_elem_t *elem = mp_map_lookup(&self->members, MP_OBJ_NEW_QSTR(attr), MP_MAP_LOOKUP); mp_map_elem_t *elem = mp_map_lookup(&self->members, MP_OBJ_NEW_QSTR(attr), MP_MAP_LOOKUP);
if (elem != NULL) { if (elem != NULL) {
// object member, always treated as a value // object member, always treated as a value

38
py/parse.c
View File

@ -802,9 +802,11 @@ STATIC bool fold_constants(parser_t *parser, uint8_t rule_id, size_t num_args) {
#endif #endif
STATIC void push_result_rule(parser_t *parser, size_t src_line, uint8_t rule_id, size_t num_args) { STATIC void push_result_rule(parser_t *parser, size_t src_line, uint8_t rule_id, size_t num_args) {
// optimise away parenthesis around an expression if possible // Simplify and optimise certain rules, to reduce memory usage and simplify the compiler.
if (rule_id == RULE_atom_paren) { if (rule_id == RULE_atom_paren) {
// there should be just 1 arg for this rule // Remove parenthesis around a single expression if possible.
// This atom_paren rule always has a single argument, and after this
// optimisation that argument is either NULL or testlist_comp.
mp_parse_node_t pn = peek_result(parser, 0); mp_parse_node_t pn = peek_result(parser, 0);
if (MP_PARSE_NODE_IS_NULL(pn)) { if (MP_PARSE_NODE_IS_NULL(pn)) {
// need to keep parenthesis for () // need to keep parenthesis for ()
@ -814,6 +816,34 @@ STATIC void push_result_rule(parser_t *parser, size_t src_line, uint8_t rule_id,
// parenthesis around a single expression, so it's just the expression // parenthesis around a single expression, so it's just the expression
return; return;
} }
} else if (rule_id == RULE_testlist_comp) {
// The testlist_comp rule can be the sole argument to either atom_parent
// or atom_bracket, for (...) and [...] respectively.
assert(num_args == 2);
mp_parse_node_t pn = peek_result(parser, 0);
if (MP_PARSE_NODE_IS_STRUCT(pn)) {
mp_parse_node_struct_t *pns = (mp_parse_node_struct_t *)pn;
if (MP_PARSE_NODE_STRUCT_KIND(pns) == RULE_testlist_comp_3b) {
// tuple of one item, with trailing comma
pop_result(parser);
--num_args;
} else if (MP_PARSE_NODE_STRUCT_KIND(pns) == RULE_testlist_comp_3c) {
// tuple of many items, convert testlist_comp_3c to testlist_comp
pop_result(parser);
assert(pn == peek_result(parser, 0));
pns->kind_num_nodes = rule_id | MP_PARSE_NODE_STRUCT_NUM_NODES(pns) << 8;
return;
} else if (MP_PARSE_NODE_STRUCT_KIND(pns) == RULE_comp_for) {
// generator expression
} else {
// tuple with 2 items
}
} else {
// tuple with 2 items
}
} else if (rule_id == RULE_testlist_comp_3c) {
// steal first arg of outer testlist_comp rule
++num_args;
} }
#if MICROPY_COMP_CONST_FOLDING #if MICROPY_COMP_CONST_FOLDING
@ -833,6 +863,10 @@ STATIC void push_result_rule(parser_t *parser, size_t src_line, uint8_t rule_id,
for (size_t i = num_args; i > 0; i--) { for (size_t i = num_args; i > 0; i--) {
pn->nodes[i - 1] = pop_result(parser); pn->nodes[i - 1] = pop_result(parser);
} }
if (rule_id == RULE_testlist_comp_3c) {
// need to push something non-null to replace stolen first arg of testlist_comp
push_result_node(parser, (mp_parse_node_t)pn);
}
push_result_node(parser, (mp_parse_node_t)pn); push_result_node(parser, (mp_parse_node_t)pn);
} }

11
py/persistentcode.h
View File

@ -41,16 +41,15 @@
#define MPY_FEATURE_ENCODE_ARCH(arch) ((arch) << 2) #define MPY_FEATURE_ENCODE_ARCH(arch) ((arch) << 2)
#define MPY_FEATURE_DECODE_ARCH(feat) ((feat) >> 2) #define MPY_FEATURE_DECODE_ARCH(feat) ((feat) >> 2)
// The feature flag bits encode the compile-time config options that // The feature flag bits encode the compile-time config options that affect
// affect the generate bytecode. // the generate bytecode. Note: position 0 is now unused
// (formerly MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE).
#define MPY_FEATURE_FLAGS ( \ #define MPY_FEATURE_FLAGS ( \
((MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE) << 0) \ ((MICROPY_PY_BUILTINS_STR_UNICODE) << 1) \
| ((MICROPY_PY_BUILTINS_STR_UNICODE) << 1) \
) )
// This is a version of the flags that can be configured at runtime. // This is a version of the flags that can be configured at runtime.
#define MPY_FEATURE_FLAGS_DYNAMIC ( \ #define MPY_FEATURE_FLAGS_DYNAMIC ( \
((MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE_DYNAMIC) << 0) \ ((MICROPY_PY_BUILTINS_STR_UNICODE_DYNAMIC) << 1) \
| ((MICROPY_PY_BUILTINS_STR_UNICODE_DYNAMIC) << 1) \
) )
// Define the host architecture // Define the host architecture

12
py/profile.c
View File

@ -540,9 +540,6 @@ STATIC const byte *mp_prof_opcode_decode(const byte *ip, const mp_uint_t *const_
instruction->qstr_opname = MP_QSTR_LOAD_NAME; instruction->qstr_opname = MP_QSTR_LOAD_NAME;
instruction->arg = qst; instruction->arg = qst;
instruction->argobj = MP_OBJ_NEW_QSTR(qst); instruction->argobj = MP_OBJ_NEW_QSTR(qst);
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE) {
instruction->argobjex_cache = MP_OBJ_NEW_SMALL_INT(*ip++);
}
break; break;
case MP_BC_LOAD_GLOBAL: case MP_BC_LOAD_GLOBAL:
@ -550,9 +547,6 @@ STATIC const byte *mp_prof_opcode_decode(const byte *ip, const mp_uint_t *const_
instruction->qstr_opname = MP_QSTR_LOAD_GLOBAL; instruction->qstr_opname = MP_QSTR_LOAD_GLOBAL;
instruction->arg = qst; instruction->arg = qst;
instruction->argobj = MP_OBJ_NEW_QSTR(qst); instruction->argobj = MP_OBJ_NEW_QSTR(qst);
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE) {
instruction->argobjex_cache = MP_OBJ_NEW_SMALL_INT(*ip++);
}
break; break;
case MP_BC_LOAD_ATTR: case MP_BC_LOAD_ATTR:
@ -560,9 +554,6 @@ STATIC const byte *mp_prof_opcode_decode(const byte *ip, const mp_uint_t *const_
instruction->qstr_opname = MP_QSTR_LOAD_ATTR; instruction->qstr_opname = MP_QSTR_LOAD_ATTR;
instruction->arg = qst; instruction->arg = qst;
instruction->argobj = MP_OBJ_NEW_QSTR(qst); instruction->argobj = MP_OBJ_NEW_QSTR(qst);
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE) {
instruction->argobjex_cache = MP_OBJ_NEW_SMALL_INT(*ip++);
}
break; break;
case MP_BC_LOAD_METHOD: case MP_BC_LOAD_METHOD:
@ -618,9 +609,6 @@ STATIC const byte *mp_prof_opcode_decode(const byte *ip, const mp_uint_t *const_
instruction->qstr_opname = MP_QSTR_STORE_ATTR; instruction->qstr_opname = MP_QSTR_STORE_ATTR;
instruction->arg = qst; instruction->arg = qst;
instruction->argobj = MP_OBJ_NEW_QSTR(qst); instruction->argobj = MP_OBJ_NEW_QSTR(qst);
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE) {
instruction->argobjex_cache = MP_OBJ_NEW_SMALL_INT(*ip++);
}
break; break;
case MP_BC_STORE_SUBSCR: case MP_BC_STORE_SUBSCR:

13
py/py.mk
View File

@ -224,19 +224,6 @@ ifneq ($(FROZEN_MANIFEST),)
PY_O += $(BUILD)/$(BUILD)/frozen_content.o PY_O += $(BUILD)/$(BUILD)/frozen_content.o
endif endif
# object file for frozen files
ifneq ($(FROZEN_DIR),)
PY_O += $(BUILD)/frozen.o
endif
# Combine old singular FROZEN_MPY_DIR with new multiple value form.
FROZEN_MPY_DIRS += $(FROZEN_MPY_DIR)
# object file for frozen bytecode (frozen .mpy files)
ifneq ($(FROZEN_MPY_DIRS),)
PY_O += $(BUILD)/frozen_mpy.o
endif
# Sources that may contain qstrings # Sources that may contain qstrings
SRC_QSTR_IGNORE = py/nlr% SRC_QSTR_IGNORE = py/nlr%
SRC_QSTR_EMITNATIVE = py/emitn% SRC_QSTR_EMITNATIVE = py/emitn%

6
py/qstr.c
View File

@ -169,6 +169,12 @@ STATIC qstr qstr_add(mp_uint_t hash, mp_uint_t len, const char *q_ptr) {
+ (sizeof(const char *) + sizeof(qstr_attr_t)) * new_pool_length; + (sizeof(const char *) + sizeof(qstr_attr_t)) * new_pool_length;
qstr_pool_t *pool = (qstr_pool_t *)m_malloc_maybe(pool_size, true); qstr_pool_t *pool = (qstr_pool_t *)m_malloc_maybe(pool_size, true);
if (pool == NULL) { if (pool == NULL) {
// Keep qstr_last_chunk consistent with qstr_pool_t: qstr_last_chunk is not scanned
// at garbage collection since it's reachable from a qstr_pool_t. And the caller of
// this function expects q_ptr to be stored in a qstr_pool_t so it can be reached
// by the collector. If qstr_pool_t allocation failed, qstr_last_chunk needs to be
// NULL'd. Otherwise it may become a dangling pointer at the next garbage collection.
MP_STATE_VM(qstr_last_chunk) = NULL;
QSTR_EXIT(); QSTR_EXIT();
m_malloc_fail(new_pool_length); m_malloc_fail(new_pool_length);
} }

4
py/qstrdefs.h
View File

@ -60,6 +60,10 @@ Q(<string>)
Q(<stdin>) Q(<stdin>)
Q(utf-8) Q(utf-8)
#if MICROPY_MODULE_FROZEN
Q(.frozen)
#endif
#if MICROPY_ENABLE_PYSTACK #if MICROPY_ENABLE_PYSTACK
Q(pystack exhausted) Q(pystack exhausted)
#endif #endif

33
py/runtime.c
View File

@ -126,6 +126,15 @@ void mp_init(void) {
sizeof(MP_STATE_VM(fs_user_mount)) - MICROPY_FATFS_NUM_PERSISTENT); sizeof(MP_STATE_VM(fs_user_mount)) - MICROPY_FATFS_NUM_PERSISTENT);
#endif #endif
#if MICROPY_PY_SYS_PATH_ARGV_DEFAULTS
mp_obj_list_init(MP_OBJ_TO_PTR(mp_sys_path), 0);
mp_obj_list_append(mp_sys_path, MP_OBJ_NEW_QSTR(MP_QSTR_)); // current dir (or base dir of the script)
#if MICROPY_MODULE_FROZEN
mp_obj_list_append(mp_sys_path, MP_OBJ_NEW_QSTR(MP_QSTR__dot_frozen));
#endif
mp_obj_list_init(MP_OBJ_TO_PTR(mp_sys_argv), 0);
#endif
#if MICROPY_PY_SYS_ATEXIT #if MICROPY_PY_SYS_ATEXIT
MP_STATE_VM(sys_exitfunc) = mp_const_none; MP_STATE_VM(sys_exitfunc) = mp_const_none;
#endif #endif
@ -157,7 +166,7 @@ void mp_deinit(void) {
#endif #endif
} }
mp_obj_t mp_load_name(qstr qst) { mp_obj_t MICROPY_WRAP_MP_LOAD_NAME(mp_load_name)(qstr qst) {
// logic: search locals, globals, builtins // logic: search locals, globals, builtins
DEBUG_OP_printf("load name %s\n", qstr_str(qst)); DEBUG_OP_printf("load name %s\n", qstr_str(qst));
// If we're at the outer scope (locals == globals), dispatch to load_global right away // If we're at the outer scope (locals == globals), dispatch to load_global right away
@ -170,7 +179,7 @@ mp_obj_t mp_load_name(qstr qst) {
return mp_load_global(qst); return mp_load_global(qst);
} }
mp_obj_t mp_load_global(qstr qst) { mp_obj_t MICROPY_WRAP_MP_LOAD_GLOBAL(mp_load_global)(qstr qst) {
// logic: search globals, builtins // logic: search globals, builtins
DEBUG_OP_printf("load global %s\n", qstr_str(qst)); DEBUG_OP_printf("load global %s\n", qstr_str(qst));
mp_map_elem_t *elem = mp_map_lookup(&mp_globals_get()->map, MP_OBJ_NEW_QSTR(qst), MP_MAP_LOOKUP); mp_map_elem_t *elem = mp_map_lookup(&mp_globals_get()->map, MP_OBJ_NEW_QSTR(qst), MP_MAP_LOOKUP);
@ -310,7 +319,7 @@ mp_obj_t mp_unary_op(mp_unary_op_t op, mp_obj_t arg) {
} }
} }
mp_obj_t PLACE_IN_ITCM(mp_binary_op)(mp_binary_op_t op, mp_obj_t lhs, mp_obj_t rhs) { mp_obj_t MICROPY_WRAP_MP_BINARY_OP(mp_binary_op)(mp_binary_op_t op, mp_obj_t lhs, mp_obj_t rhs) {
DEBUG_OP_printf("binary " UINT_FMT " %q %p %p\n", op, mp_binary_op_method_name[op], lhs, rhs); DEBUG_OP_printf("binary " UINT_FMT " %q %p %p\n", op, mp_binary_op_method_name[op], lhs, rhs);
// TODO correctly distinguish inplace operators for mutable objects // TODO correctly distinguish inplace operators for mutable objects
@ -1103,6 +1112,10 @@ void mp_load_method_maybe(mp_obj_t obj, qstr attr, mp_obj_t *dest) {
dest[0] = MP_OBJ_NULL; dest[0] = MP_OBJ_NULL;
dest[1] = MP_OBJ_NULL; dest[1] = MP_OBJ_NULL;
// Note: the specific case of obj being an instance type is fast-path'ed in the VM
// for the MP_BC_LOAD_ATTR opcode. Instance types handle type->attr and look up directly
// in their member's map.
// get the type // get the type
const mp_obj_type_t *type = mp_obj_get_type(obj); const mp_obj_type_t *type = mp_obj_get_type(obj);
@ -1124,8 +1137,15 @@ void mp_load_method_maybe(mp_obj_t obj, qstr attr, mp_obj_t *dest) {
if (attr_fun != NULL) { if (attr_fun != NULL) {
// this type can do its own load, so call it // this type can do its own load, so call it
attr_fun(obj, attr, dest); attr_fun(obj, attr, dest);
// If type->attr has set dest[1] = MP_OBJ_SENTINEL, we should proceed
// with lookups below (i.e. in locals_dict). If not, return right away.
if (dest[1] != MP_OBJ_SENTINEL) {
return; return;
} }
// Clear the fail flag set by type->attr so it's like it never ran.
dest[1] = MP_OBJ_NULL;
}
if (type->locals_dict != NULL) { if (type->locals_dict != NULL) {
// generic method lookup // generic method lookup
// this is a lookup in the object (ie not class or type) // this is a lookup in the object (ie not class or type)
@ -1135,6 +1155,7 @@ void mp_load_method_maybe(mp_obj_t obj, qstr attr, mp_obj_t *dest) {
if (elem != NULL) { if (elem != NULL) {
mp_convert_member_lookup(obj, type, elem->value, dest); mp_convert_member_lookup(obj, type, elem->value, dest);
} }
return;
} }
} }
@ -1426,8 +1447,10 @@ mp_obj_t mp_make_raise_obj(mp_obj_t o) {
// create and return a new exception instance by calling o // create and return a new exception instance by calling o
// TODO could have an option to disable traceback, then builtin exceptions (eg TypeError) // TODO could have an option to disable traceback, then builtin exceptions (eg TypeError)
// could have const instances in ROM which we return here instead // could have const instances in ROM which we return here instead
return mp_call_function_n_kw(o, 0, 0, NULL); o = mp_call_function_n_kw(o, 0, 0, NULL);
} else if (mp_obj_is_exception_instance(o)) { }
if (mp_obj_is_exception_instance(o)) {
// o is an instance of an exception, so use it as the exception // o is an instance of an exception, so use it as the exception
return o; return o;
} else { } else {

19
py/showbc.c
View File

@ -98,8 +98,8 @@ void mp_bytecode_print(const mp_print_t *print, const void *descr, const byte *i
// raw bytecode dump // raw bytecode dump
size_t prelude_size = ip - mp_showbc_code_start + n_info + n_cell; size_t prelude_size = ip - mp_showbc_code_start + n_info + n_cell;
mp_printf(print, "Raw bytecode (code_info_size=" UINT_FMT ", bytecode_size=" UINT_FMT "):\n", mp_printf(print, "Raw bytecode (code_info_size=%u, bytecode_size=%u):\n",
prelude_size, len - prelude_size); (unsigned)prelude_size, (unsigned)(len - prelude_size));
for (mp_uint_t i = 0; i < len; i++) { for (mp_uint_t i = 0; i < len; i++) {
if (i > 0 && i % 16 == 0) { if (i > 0 && i % 16 == 0) {
mp_printf(print, "\n"); mp_printf(print, "\n");
@ -208,25 +208,16 @@ const byte *mp_bytecode_print_str(const mp_print_t *print, const byte *ip) {
case MP_BC_LOAD_NAME: case MP_BC_LOAD_NAME:
DECODE_QSTR; DECODE_QSTR;
mp_printf(print, "LOAD_NAME %s", qstr_str(qst)); mp_printf(print, "LOAD_NAME %s", qstr_str(qst));
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE) {
mp_printf(print, " (cache=%u)", *ip++);
}
break; break;
case MP_BC_LOAD_GLOBAL: case MP_BC_LOAD_GLOBAL:
DECODE_QSTR; DECODE_QSTR;
mp_printf(print, "LOAD_GLOBAL %s", qstr_str(qst)); mp_printf(print, "LOAD_GLOBAL %s", qstr_str(qst));
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE) {
mp_printf(print, " (cache=%u)", *ip++);
}
break; break;
case MP_BC_LOAD_ATTR: case MP_BC_LOAD_ATTR:
DECODE_QSTR; DECODE_QSTR;
mp_printf(print, "LOAD_ATTR %s", qstr_str(qst)); mp_printf(print, "LOAD_ATTR %s", qstr_str(qst));
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE) {
mp_printf(print, " (cache=%u)", *ip++);
}
break; break;
case MP_BC_LOAD_METHOD: case MP_BC_LOAD_METHOD:
@ -270,9 +261,6 @@ const byte *mp_bytecode_print_str(const mp_print_t *print, const byte *ip) {
case MP_BC_STORE_ATTR: case MP_BC_STORE_ATTR:
DECODE_QSTR; DECODE_QSTR;
mp_printf(print, "STORE_ATTR %s", qstr_str(qst)); mp_printf(print, "STORE_ATTR %s", qstr_str(qst));
if (MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE) {
mp_printf(print, " (cache=%u)", *ip++);
}
break; break;
case MP_BC_STORE_SUBSCR: case MP_BC_STORE_SUBSCR:
@ -531,7 +519,8 @@ const byte *mp_bytecode_print_str(const mp_print_t *print, const byte *ip) {
} else if (ip[-1] < MP_BC_STORE_FAST_MULTI + 16) { } else if (ip[-1] < MP_BC_STORE_FAST_MULTI + 16) {
mp_printf(print, "STORE_FAST " UINT_FMT, (mp_uint_t)ip[-1] - MP_BC_STORE_FAST_MULTI); mp_printf(print, "STORE_FAST " UINT_FMT, (mp_uint_t)ip[-1] - MP_BC_STORE_FAST_MULTI);
} else if (ip[-1] < MP_BC_UNARY_OP_MULTI + MP_UNARY_OP_NUM_BYTECODE) { } else if (ip[-1] < MP_BC_UNARY_OP_MULTI + MP_UNARY_OP_NUM_BYTECODE) {
mp_printf(print, "UNARY_OP " UINT_FMT, (mp_uint_t)ip[-1] - MP_BC_UNARY_OP_MULTI); mp_uint_t op = ip[-1] - MP_BC_UNARY_OP_MULTI;
mp_printf(print, "UNARY_OP " UINT_FMT " %s", op, qstr_str(mp_unary_op_method_name[op]));
} else if (ip[-1] < MP_BC_BINARY_OP_MULTI + MP_BINARY_OP_NUM_BYTECODE) { } else if (ip[-1] < MP_BC_BINARY_OP_MULTI + MP_BINARY_OP_NUM_BYTECODE) {
mp_uint_t op = ip[-1] - MP_BC_BINARY_OP_MULTI; mp_uint_t op = ip[-1] - MP_BC_BINARY_OP_MULTI;
mp_printf(print, "BINARY_OP " UINT_FMT " %s", op, qstr_str(mp_binary_op_method_name[op])); mp_printf(print, "BINARY_OP " UINT_FMT " %s", op, qstr_str(mp_binary_op_method_name[op]));

106
py/vm.c
View File

@ -182,30 +182,13 @@
#define TRACE_TICK(current_ip, current_sp, is_exception) #define TRACE_TICK(current_ip, current_sp, is_exception)
#endif // MICROPY_PY_SYS_SETTRACE #endif // MICROPY_PY_SYS_SETTRACE
#if MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE
static inline mp_map_elem_t *mp_map_cached_lookup(mp_map_t *map, qstr qst, uint8_t *idx_cache) {
size_t idx = *idx_cache;
mp_obj_t key = MP_OBJ_NEW_QSTR(qst);
mp_map_elem_t *elem = NULL;
if (idx < map->alloc && map->table[idx].key == key) {
elem = &map->table[idx];
} else {
elem = mp_map_lookup(map, key, MP_MAP_LOOKUP);
if (elem != NULL) {
*idx_cache = (elem - &map->table[0]) & 0xff;
}
}
return elem;
}
#endif
// fastn has items in reverse order (fastn[0] is local[0], fastn[-1] is local[1], etc) // fastn has items in reverse order (fastn[0] is local[0], fastn[-1] is local[1], etc)
// sp points to bottom of stack which grows up // sp points to bottom of stack which grows up
// returns: // returns:
// MP_VM_RETURN_NORMAL, sp valid, return value in *sp // MP_VM_RETURN_NORMAL, sp valid, return value in *sp
// MP_VM_RETURN_YIELD, ip, sp valid, yielded value in *sp // MP_VM_RETURN_YIELD, ip, sp valid, yielded value in *sp
// MP_VM_RETURN_EXCEPTION, exception in state[0] // MP_VM_RETURN_EXCEPTION, exception in state[0]
mp_vm_return_kind_t PLACE_IN_ITCM(mp_execute_bytecode)(mp_code_state_t *code_state, volatile mp_obj_t inject_exc) { mp_vm_return_kind_t MICROPY_WRAP_MP_EXECUTE_BYTECODE(mp_execute_bytecode)(mp_code_state_t *code_state, volatile mp_obj_t inject_exc) {
#define SELECTIVE_EXC_IP (0) #define SELECTIVE_EXC_IP (0)
#if SELECTIVE_EXC_IP #if SELECTIVE_EXC_IP
#define MARK_EXC_IP_SELECTIVE() { code_state->ip = ip; } /* stores ip 1 byte past last opcode */ #define MARK_EXC_IP_SELECTIVE() { code_state->ip = ip; } /* stores ip 1 byte past last opcode */
@ -373,84 +356,46 @@ dispatch_loop:
goto load_check; goto load_check;
} }
#if !MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE
ENTRY(MP_BC_LOAD_NAME): { ENTRY(MP_BC_LOAD_NAME): {
MARK_EXC_IP_SELECTIVE(); MARK_EXC_IP_SELECTIVE();
DECODE_QSTR; DECODE_QSTR;
PUSH(mp_load_name(qst)); PUSH(mp_load_name(qst));
DISPATCH(); DISPATCH();
} }
#else
ENTRY(MP_BC_LOAD_NAME): {
MARK_EXC_IP_SELECTIVE();
DECODE_QSTR;
mp_map_elem_t *elem = mp_map_cached_lookup(&mp_locals_get()->map, qst, (uint8_t*)ip);
mp_obj_t obj;
if (elem != NULL) {
obj = elem->value;
} else {
obj = mp_load_name(qst);
}
PUSH(obj);
ip++;
DISPATCH();
}
#endif
#if !MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE
ENTRY(MP_BC_LOAD_GLOBAL): { ENTRY(MP_BC_LOAD_GLOBAL): {
MARK_EXC_IP_SELECTIVE(); MARK_EXC_IP_SELECTIVE();
DECODE_QSTR; DECODE_QSTR;
PUSH(mp_load_global(qst)); PUSH(mp_load_global(qst));
DISPATCH(); DISPATCH();
} }
#else
ENTRY(MP_BC_LOAD_GLOBAL): {
MARK_EXC_IP_SELECTIVE();
DECODE_QSTR;
mp_map_elem_t *elem = mp_map_cached_lookup(&mp_globals_get()->map, qst, (uint8_t*)ip);
mp_obj_t obj;
if (elem != NULL) {
obj = elem->value;
} else {
obj = mp_load_global(qst);
}
PUSH(obj);
ip++;
DISPATCH();
}
#endif
#if !MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE
ENTRY(MP_BC_LOAD_ATTR): {
FRAME_UPDATE();
MARK_EXC_IP_SELECTIVE();
DECODE_QSTR;
SET_TOP(mp_load_attr(TOP(), qst));
DISPATCH();
}
#else
ENTRY(MP_BC_LOAD_ATTR): { ENTRY(MP_BC_LOAD_ATTR): {
FRAME_UPDATE(); FRAME_UPDATE();
MARK_EXC_IP_SELECTIVE(); MARK_EXC_IP_SELECTIVE();
DECODE_QSTR; DECODE_QSTR;
mp_obj_t top = TOP(); mp_obj_t top = TOP();
mp_obj_t obj;
#if MICROPY_OPT_LOAD_ATTR_FAST_PATH
// For the specific case of an instance type, it implements .attr
// and forwards to its members map. Attribute lookups on instance
// types are extremely common, so avoid all the other checks and
// calls that normally happen first.
mp_map_elem_t *elem = NULL; mp_map_elem_t *elem = NULL;
if (mp_obj_is_instance_type(mp_obj_get_type(top))) { if (mp_obj_is_instance_type(mp_obj_get_type(top))) {
mp_obj_instance_t *self = MP_OBJ_TO_PTR(top); mp_obj_instance_t *self = MP_OBJ_TO_PTR(top);
elem = mp_map_cached_lookup(&self->members, qst, (uint8_t*)ip); elem = mp_map_lookup(&self->members, MP_OBJ_NEW_QSTR(qst), MP_MAP_LOOKUP);
} }
mp_obj_t obj; if (elem) {
if (elem != NULL) {
obj = elem->value; obj = elem->value;
} else { } else
#endif
{
obj = mp_load_attr(top, qst); obj = mp_load_attr(top, qst);
} }
SET_TOP(obj); SET_TOP(obj);
ip++;
DISPATCH(); DISPATCH();
} }
#endif
ENTRY(MP_BC_LOAD_METHOD): { ENTRY(MP_BC_LOAD_METHOD): {
MARK_EXC_IP_SELECTIVE(); MARK_EXC_IP_SELECTIVE();
@ -506,7 +451,6 @@ dispatch_loop:
DISPATCH(); DISPATCH();
} }
#if !MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE
ENTRY(MP_BC_STORE_ATTR): { ENTRY(MP_BC_STORE_ATTR): {
FRAME_UPDATE(); FRAME_UPDATE();
MARK_EXC_IP_SELECTIVE(); MARK_EXC_IP_SELECTIVE();
@ -515,32 +459,6 @@ dispatch_loop:
sp -= 2; sp -= 2;
DISPATCH(); DISPATCH();
} }
#else
// This caching code works with MICROPY_PY_BUILTINS_PROPERTY and/or
// MICROPY_PY_DESCRIPTORS enabled because if the attr exists in
// self->members then it can't be a property or have descriptors. A
// consequence of this is that we can't use MP_MAP_LOOKUP_ADD_IF_NOT_FOUND
// in the fast-path below, because that store could override a property.
ENTRY(MP_BC_STORE_ATTR): {
FRAME_UPDATE();
MARK_EXC_IP_SELECTIVE();
DECODE_QSTR;
mp_map_elem_t *elem = NULL;
mp_obj_t top = TOP();
if (mp_obj_is_instance_type(mp_obj_get_type(top)) && sp[-1] != MP_OBJ_NULL) {
mp_obj_instance_t *self = MP_OBJ_TO_PTR(top);
elem = mp_map_cached_lookup(&self->members, qst, (uint8_t*)ip);
}
if (elem != NULL) {
elem->value = sp[-1];
} else {
mp_store_attr(sp[0], qst, sp[-1]);
}
sp -= 2;
ip++;
DISPATCH();
}
#endif
ENTRY(MP_BC_STORE_SUBSCR): ENTRY(MP_BC_STORE_SUBSCR):
MARK_EXC_IP_SELECTIVE(); MARK_EXC_IP_SELECTIVE();

9
shared/libc/string0.c
View File

@ -25,7 +25,7 @@
*/ */
#include <stdint.h> #include <stdint.h>
#include <string.h> #include <stddef.h>
#ifndef likely #ifndef likely
#define likely(x) __builtin_expect((x), 1) #define likely(x) __builtin_expect((x), 1)
@ -68,6 +68,13 @@ void *memcpy(void *dst, const void *src, size_t n) {
return dst; return dst;
} }
void *__memcpy_chk(void *dest, const void *src, size_t len, size_t slen) {
if (len > slen) {
return NULL;
}
return memcpy(dest, src, len);
}
void *memmove(void *dest, const void *src, size_t n) { void *memmove(void *dest, const void *src, size_t n) {
if (src < dest && (uint8_t*)dest < (const uint8_t*)src + n) { if (src < dest && (uint8_t*)dest < (const uint8_t*)src + n) {
// need to copy backwards // need to copy backwards

24
shared/runtime/pyexec.c
View File

@ -55,21 +55,21 @@ int pyexec_system_exit = 0;
STATIC bool repl_display_debugging_info = 0; STATIC bool repl_display_debugging_info = 0;
#endif #endif
#define EXEC_FLAG_PRINT_EOF (1) #define EXEC_FLAG_PRINT_EOF (1 << 0)
#define EXEC_FLAG_ALLOW_DEBUGGING (2) #define EXEC_FLAG_ALLOW_DEBUGGING (1 << 1)
#define EXEC_FLAG_IS_REPL (4) #define EXEC_FLAG_IS_REPL (1 << 2)
#define EXEC_FLAG_SOURCE_IS_RAW_CODE (8) #define EXEC_FLAG_SOURCE_IS_RAW_CODE (1 << 3)
#define EXEC_FLAG_SOURCE_IS_VSTR (16) #define EXEC_FLAG_SOURCE_IS_VSTR (1 << 4)
#define EXEC_FLAG_SOURCE_IS_FILENAME (32) #define EXEC_FLAG_SOURCE_IS_FILENAME (1 << 5)
#define EXEC_FLAG_SOURCE_IS_READER (64) #define EXEC_FLAG_SOURCE_IS_READER (1 << 6)
#define EXEC_FLAG_SOURCE_IS_ATEXIT (128) #define EXEC_FLAG_SOURCE_IS_ATEXIT (1 << 7)
// parses, compiles and executes the code in the lexer // parses, compiles and executes the code in the lexer
// frees the lexer before returning // frees the lexer before returning
// EXEC_FLAG_PRINT_EOF prints 2 EOF chars: 1 after normal output, 1 after exception output // EXEC_FLAG_PRINT_EOF prints 2 EOF chars: 1 after normal output, 1 after exception output
// EXEC_FLAG_ALLOW_DEBUGGING allows debugging info to be printed after executing the code // EXEC_FLAG_ALLOW_DEBUGGING allows debugging info to be printed after executing the code
// EXEC_FLAG_IS_REPL is used for REPL inputs (flag passed on to mp_compile) // EXEC_FLAG_IS_REPL is used for REPL inputs (flag passed on to mp_compile)
STATIC int parse_compile_execute(const void *source, mp_parse_input_kind_t input_kind, int exec_flags, pyexec_result_t *result) { STATIC int parse_compile_execute(const void *source, mp_parse_input_kind_t input_kind, mp_uint_t exec_flags, pyexec_result_t *result) {
int ret = 0; int ret = 0;
#if MICROPY_REPL_INFO #if MICROPY_REPL_INFO
uint32_t start = 0; uint32_t start = 0;
@ -166,6 +166,7 @@ STATIC int parse_compile_execute(const void *source, mp_parse_input_kind_t input
if (exec_flags & EXEC_FLAG_PRINT_EOF) { if (exec_flags & EXEC_FLAG_PRINT_EOF) {
mp_hal_stdout_tx_strn("\x04", 1); mp_hal_stdout_tx_strn("\x04", 1);
} }
// check for SystemExit // check for SystemExit
if (mp_obj_is_subclass_fast(MP_OBJ_FROM_PTR(mp_obj_get_type((mp_obj_t)nlr.ret_val)), MP_OBJ_FROM_PTR(&mp_type_SystemExit))) { if (mp_obj_is_subclass_fast(MP_OBJ_FROM_PTR(mp_obj_get_type((mp_obj_t)nlr.ret_val)), MP_OBJ_FROM_PTR(&mp_type_SystemExit))) {
// at the moment, the value of SystemExit is unused // at the moment, the value of SystemExit is unused
@ -745,7 +746,7 @@ int pyexec_file(const char *filename, pyexec_result_t *result) {
int pyexec_file_if_exists(const char *filename, pyexec_result_t *result) { int pyexec_file_if_exists(const char *filename, pyexec_result_t *result) {
#if MICROPY_MODULE_FROZEN #if MICROPY_MODULE_FROZEN
if (mp_frozen_stat(filename) == MP_IMPORT_STAT_FILE) { if (mp_find_frozen_module(filename, NULL, NULL) == MP_IMPORT_STAT_FILE) {
return pyexec_frozen_module(filename, result); return pyexec_frozen_module(filename, result);
} }
#endif #endif
@ -758,7 +759,8 @@ int pyexec_file_if_exists(const char *filename, pyexec_result_t *result) {
#if MICROPY_MODULE_FROZEN #if MICROPY_MODULE_FROZEN
int pyexec_frozen_module(const char *name, pyexec_result_t *result) { int pyexec_frozen_module(const char *name, pyexec_result_t *result) {
void *frozen_data; void *frozen_data;
int frozen_type = mp_find_frozen_module(name, strlen(name), &frozen_data); int frozen_type;
mp_find_frozen_module(name, &frozen_type, &frozen_data);
switch (frozen_type) { switch (frozen_type) {
#if MICROPY_MODULE_FROZEN_STR #if MICROPY_MODULE_FROZEN_STR

3
shared/upytesthelper/upytesthelper.c
View File

@ -94,6 +94,9 @@ void upytest_execute_test(const char *src) {
gc_init(heap_start, heap_end); gc_init(heap_start, heap_end);
mp_init(); mp_init();
mp_obj_list_init(mp_sys_path, 0); mp_obj_list_init(mp_sys_path, 0);
#if MICROPY_MODULE_FROZEN
mp_obj_list_append(mp_sys_path, MP_OBJ_NEW_QSTR(MP_QSTR__dot_frozen));
#endif
mp_obj_list_init(mp_sys_argv, 0); mp_obj_list_init(mp_sys_argv, 0);
nlr_buf_t nlr; nlr_buf_t nlr;

15
tests/basics/int_big_cmp.py
View File

@ -1,10 +1,13 @@
# test bignum comparisons # test bignum comparisons
i = 1 << 65 i = 1 << 65
cases = (0, 1, -1, i, -i, i + 1, -(i + 1))
print(i == 0) for lhs in cases:
print(i != 0) for rhs in cases:
print(i < 0) print("{} == {} = {}".format(lhs, rhs, lhs == rhs))
print(i > 0) print("{} != {} = {}".format(lhs, rhs, lhs != rhs))
print(i <= 0) print("{} < {} = {}".format(lhs, rhs, lhs < rhs))
print(i >= 0) print("{} > {} = {}".format(lhs, rhs, lhs > rhs))
print("{} <= {} = {}".format(lhs, rhs, lhs <= rhs))
print("{} >= {} = {}".format(lhs, rhs, lhs >= rhs))

41
tests/basics/int_big_zeroone.py
View File

@ -1,4 +1,4 @@
# test [0,-0,1,-1] edge cases of bignum # test [0,1,-1] edge cases of bignum
import skip_if import skip_if
skip_if.no_bigint() skip_if.no_bigint()
@ -16,7 +16,7 @@ print([~c for c in cases])
print([c >> 1 for c in cases]) print([c >> 1 for c in cases])
print([c << 1 for c in cases]) print([c << 1 for c in cases])
# comparison of 0/-0/+0 # comparison of 0
print(long_zero == 0) print(long_zero == 0)
print(long_neg_zero == 0) print(long_neg_zero == 0)
print(long_one - 1 == 0) print(long_one - 1 == 0)
@ -29,3 +29,40 @@ print(long_neg_zero < 1)
print(long_neg_zero < -1) print(long_neg_zero < -1)
print(long_neg_zero > 1) print(long_neg_zero > 1)
print(long_neg_zero > -1) print(long_neg_zero > -1)
# generate zeros that involve negative numbers
large = 1 << 70
large_plus_one = large + 1
zeros = (
large - large,
-large + large,
large + -large,
-(large - large),
large - large_plus_one + 1,
-large & (large - large),
-large ^ -large,
-large * (large - large),
(large - large) // -large,
-large // -large_plus_one,
-(large + large) % large,
(large + large) % -large,
-(large + large) % -large,
)
print(zeros)
# compute arithmetic operations that may have problems with -0
# (this checks that -0 is never generated in the zeros tuple)
cases = (0, 1, -1) + zeros
for lhs in cases:
print("-{} = {}".format(lhs, -lhs))
print("~{} = {}".format(lhs, ~lhs))
print("{} >> 1 = {}".format(lhs, lhs >> 1))
print("{} << 1 = {}".format(lhs, lhs << 1))
for rhs in cases:
print("{} == {} = {}".format(lhs, rhs, lhs == rhs))
print("{} + {} = {}".format(lhs, rhs, lhs + rhs))
print("{} - {} = {}".format(lhs, rhs, lhs - rhs))
print("{} * {} = {}".format(lhs, rhs, lhs * rhs))
print("{} | {} = {}".format(lhs, rhs, lhs | rhs))
print("{} & {} = {}".format(lhs, rhs, lhs & rhs))
print("{} ^ {} = {}".format(lhs, rhs, lhs ^ rhs))

7
tests/basics/string_fstring.py
View File

@ -22,6 +22,13 @@ def foo(a, b):
return f'{x}{y}{a}{b}' return f'{x}{y}{a}{b}'
print(foo(7, 8)) print(foo(7, 8))
# ':' character within {...} that should not be interpreted as format specifiers.
print(f"a{[0,1,2][0:2]}")
print(f"a{[0,15,2][0:2][-1]:04x}")
# Nested '{' and '}' characters.
print(f"a{ {0,1,2}}")
# PEP-0498 specifies that '\\' and '#' must be disallowed explicitly, whereas # PEP-0498 specifies that '\\' and '#' must be disallowed explicitly, whereas
# MicroPython relies on the syntax error as a result of the substitution. # MicroPython relies on the syntax error as a result of the substitution.

39
tests/basics/subclass_native_exc_new.py Normal file
View File

@ -0,0 +1,39 @@
# test subclassing exceptions and providing __new__
class Dummy(BaseException):
pass
class GoodException(BaseException):
def __new__(cls, *args, **kwargs):
print("GoodException __new__")
return Dummy(*args, **kwargs)
class BadException(BaseException):
def __new__(cls, *args, **kwargs):
print("BadException __new__")
return 1
try:
raise GoodException("good message")
except BaseException as good:
print(type(good), good.args[0])
try:
raise BadException("bad message")
except Exception as bad:
# Should be TypeError 'exceptions must derive from BaseException'
print(type(bad), bad.args[0])
try:
def gen():
yield
gen().throw(BadException)
except Exception as genbad:
# Should be TypeError 'exceptions must derive from BaseException'
print(type(genbad), genbad.args[0])

6
tests/basics/subclass_native_exc_new.py.exp Normal file
View File

@ -0,0 +1,6 @@
GoodException __new__
<class 'Dummy'> good message
BadException __new__
<class 'TypeError'> exceptions must derive from BaseException
BadException __new__
<class 'TypeError'> exceptions must derive from BaseException

16
tests/basics/sys_path.py Normal file
View File

@ -0,0 +1,16 @@
# test sys.path
try:
import usys as sys
except ImportError:
import sys
# check that this script was executed from a file of the same name
if "__file__" not in globals() or "sys_path.py" not in __file__:
print("SKIP")
raise SystemExit
# test that sys.path[0] is the directory containing this script
with open(sys.path[0] + "/sys_path.py") as f:
for _ in range(4):
print(f.readline())

10
tests/cmdline/cmd_parsetree.py.exp
View File

@ -78,11 +78,11 @@ arg names:
45 STORE_NAME g 45 STORE_NAME g
48 LOAD_CONST_OBJ \.\+ 48 LOAD_CONST_OBJ \.\+
50 LOAD_METHOD format 50 LOAD_METHOD format
53 LOAD_NAME b (cache=0) 53 LOAD_NAME b
57 CALL_METHOD n=1 nkw=0 56 CALL_METHOD n=1 nkw=0
59 STORE_NAME h 58 STORE_NAME h
62 LOAD_CONST_NONE 61 LOAD_CONST_NONE
63 RETURN_VALUE 62 RETURN_VALUE
mem: total=\\d\+, current=\\d\+, peak=\\d\+ mem: total=\\d\+, current=\\d\+, peak=\\d\+
stack: \\d\+ out of \\d\+ stack: \\d\+ out of \\d\+
GC: total: \\d\+, used: \\d\+, free: \\d\+ GC: total: \\d\+, used: \\d\+, free: \\d\+

17
tests/cmdline/cmd_showbc.py
View File

@ -15,9 +15,9 @@ def f():
c = [1, 2] c = [1, 2]
d = {1, 2} d = {1, 2}
e = {} e = {}
f = {1: 2} f = {1:2}
g = "a" g = 'a'
h = b"a" h = b'a'
# unary/binary ops # unary/binary ops
i = 1 i = 1
@ -59,7 +59,7 @@ def f():
# comprehensions # comprehensions
a = (b for c in d if e) a = (b for c in d if e)
a = [b for c in d if e] a = [b for c in d if e]
a = {b: b for c in d if e} a = {b:b for c in d if e}
# function calls # function calls
a() a()
@ -108,10 +108,8 @@ def f():
# closed over variables # closed over variables
x = 1 x = 1
def closure(): def closure():
nonlocal x nonlocal x; a = x + 1
a = x + 1
x = 1 x = 1
del x del x
@ -128,14 +126,12 @@ def f():
return return
return 1 return 1
# function with lots of locals # function with lots of locals
def f(): def f():
l1 = l2 = l3 = l4 = l5 = l6 = l7 = l8 = l9 = l10 = 1 l1 = l2 = l3 = l4 = l5 = l6 = l7 = l8 = l9 = l10 = 1
m1 = m2 = m3 = m4 = m5 = m6 = m7 = m8 = m9 = m10 = 2 m1 = m2 = m3 = m4 = m5 = m6 = m7 = m8 = m9 = m10 = 2
l10 + m10 l10 + m10
# functions with default args # functions with default args
def f(a=1): def f(a=1):
pass pass
@ -143,19 +139,16 @@ def f(a=1):
def f(b=2): def f(b=2):
return b + a return b + a
# function which yields # function which yields
def f(): def f():
yield yield
yield 1 yield 1
yield from 1 yield from 1
# class # class
class Class: class Class:
pass pass
# delete name # delete name
del Class del Class

48
tests/cmdline/cmd_showbc.py.exp
View File

@ -1,13 +1,13 @@
File cmdline/cmd_showbc.py, code block '<module>' (descriptor: \.\+, bytecode @\.\+ bytes) File cmdline/cmd_showbc.py, code block '<module>' (descriptor: \.\+, bytecode @\.\+ bytes)
Raw bytecode (code_info_size=\\d\+, bytecode_size=\\d\+): Raw bytecode (code_info_size=\\d\+, bytecode_size=\\d\+):
######## ########
\.\+51 63 \.\+63
arg names: arg names:
(N_STATE 3) (N_STATE 3)
(N_EXC_STACK 0) (N_EXC_STACK 0)
bc=0 line=1 bc=0 line=1
######## ########
bc=\\d\+ line=167 bc=\\d\+ line=160
00 MAKE_FUNCTION \.\+ 00 MAKE_FUNCTION \.\+
\\d\+ STORE_NAME f \\d\+ STORE_NAME f
\\d\+ MAKE_FUNCTION \.\+ \\d\+ MAKE_FUNCTION \.\+
@ -45,7 +45,7 @@ Raw bytecode (code_info_size=\\d\+, bytecode_size=\\d\+):
(INIT_CELL 16) (INIT_CELL 16)
bc=0 line=1 bc=0 line=1
######## ########
bc=\\d\+ line=129 bc=\\d\+ line=127
00 LOAD_CONST_NONE 00 LOAD_CONST_NONE
01 LOAD_CONST_FALSE 01 LOAD_CONST_FALSE
02 BINARY_OP 27 __add__ 02 BINARY_OP 27 __add__
@ -92,7 +92,7 @@ Raw bytecode (code_info_size=\\d\+, bytecode_size=\\d\+):
58 BINARY_OP 27 __add__ 58 BINARY_OP 27 __add__
\\d\+ STORE_FAST 8 \\d\+ STORE_FAST 8
\\d\+ LOAD_FAST 0 \\d\+ LOAD_FAST 0
\\d\+ UNARY_OP 1 \\d\+ UNARY_OP 1 __neg__
\\d\+ STORE_FAST 9 \\d\+ STORE_FAST 9
\\d\+ LOAD_FAST 0 \\d\+ LOAD_FAST 0
\\d\+ UNARY_OP 3 \\d\+ UNARY_OP 3
@ -119,11 +119,11 @@ Raw bytecode (code_info_size=\\d\+, bytecode_size=\\d\+):
\\d\+ UNARY_OP 3 \\d\+ UNARY_OP 3
\\d\+ STORE_FAST 10 \\d\+ STORE_FAST 10
\\d\+ LOAD_DEREF 14 \\d\+ LOAD_DEREF 14
\\d\+ LOAD_ATTR c (cache=0) \\d\+ LOAD_ATTR c
\\d\+ STORE_FAST 11 \\d\+ STORE_FAST 11
\\d\+ LOAD_FAST 11 \\d\+ LOAD_FAST 11
\\d\+ LOAD_DEREF 14 \\d\+ LOAD_DEREF 14
\\d\+ STORE_ATTR c (cache=0) \\d\+ STORE_ATTR c
\\d\+ LOAD_DEREF 14 \\d\+ LOAD_DEREF 14
\\d\+ LOAD_CONST_SMALL_INT 0 \\d\+ LOAD_CONST_SMALL_INT 0
\\d\+ LOAD_SUBSCR \\d\+ LOAD_SUBSCR
@ -233,7 +233,7 @@ Raw bytecode (code_info_size=\\d\+, bytecode_size=\\d\+):
\\d\+ LOAD_DEREF 16 \\d\+ LOAD_DEREF 16
\\d\+ POP_TOP \\d\+ POP_TOP
\\d\+ JUMP \\d\+ \\d\+ JUMP \\d\+
\\d\+ LOAD_GLOBAL y (cache=0) \\d\+ LOAD_GLOBAL y
\\d\+ POP_TOP \\d\+ POP_TOP
\\d\+ JUMP \\d\+ \\d\+ JUMP \\d\+
\\d\+ LOAD_DEREF 14 \\d\+ LOAD_DEREF 14
@ -320,7 +320,7 @@ Raw bytecode (code_info_size=\\d\+, bytecode_size=\\d\+):
(N_EXC_STACK 0) (N_EXC_STACK 0)
bc=0 line=1 bc=0 line=1
######## ########
bc=\\d\+ line=136 bc=\\d\+ line=133
00 LOAD_CONST_SMALL_INT 1 00 LOAD_CONST_SMALL_INT 1
01 DUP_TOP 01 DUP_TOP
02 STORE_FAST 0 02 STORE_FAST 0
@ -376,7 +376,7 @@ arg names: a
(N_EXC_STACK 0) (N_EXC_STACK 0)
(INIT_CELL 0) (INIT_CELL 0)
######## ########
bc=\\d\+ line=143 bc=\\d\+ line=139
00 LOAD_CONST_SMALL_INT 2 00 LOAD_CONST_SMALL_INT 2
01 BUILD_TUPLE 1 01 BUILD_TUPLE 1
03 LOAD_NULL 03 LOAD_NULL
@ -393,9 +393,9 @@ arg names:
(N_STATE 2) (N_STATE 2)
(N_EXC_STACK 0) (N_EXC_STACK 0)
bc=0 line=1 bc=0 line=1
bc=0 line=149 bc=0 line=144
bc=3 line=150 bc=3 line=145
bc=6 line=151 bc=6 line=146
00 LOAD_CONST_NONE 00 LOAD_CONST_NONE
01 YIELD_VALUE 01 YIELD_VALUE
02 POP_TOP 02 POP_TOP
@ -418,13 +418,13 @@ arg names:
(N_EXC_STACK 0) (N_EXC_STACK 0)
bc=0 line=1 bc=0 line=1
######## ########
bc=13 line=156 bc=12 line=150
00 LOAD_NAME __name__ (cache=0) 00 LOAD_NAME __name__
04 STORE_NAME __module__ 03 STORE_NAME __module__
07 LOAD_CONST_STRING 'Class' 06 LOAD_CONST_STRING 'Class'
10 STORE_NAME __qualname__ 09 STORE_NAME __qualname__
13 LOAD_CONST_NONE 12 LOAD_CONST_NONE
14 RETURN_VALUE 13 RETURN_VALUE
File cmdline/cmd_showbc.py, code block 'f' (descriptor: \.\+, bytecode @\.\+ bytes) File cmdline/cmd_showbc.py, code block 'f' (descriptor: \.\+, bytecode @\.\+ bytes)
Raw bytecode (code_info_size=\\d\+, bytecode_size=\\d\+): Raw bytecode (code_info_size=\\d\+, bytecode_size=\\d\+):
######## ########
@ -433,9 +433,9 @@ arg names: self
(N_STATE 4) (N_STATE 4)
(N_EXC_STACK 0) (N_EXC_STACK 0)
bc=0 line=1 bc=0 line=1
bc=0 line=164 bc=0 line=157
00 LOAD_GLOBAL super (cache=0) 00 LOAD_GLOBAL super
\\d\+ LOAD_GLOBAL __class__ (cache=0) \\d\+ LOAD_GLOBAL __class__
\\d\+ LOAD_FAST 0 \\d\+ LOAD_FAST 0
\\d\+ LOAD_SUPER_METHOD f \\d\+ LOAD_SUPER_METHOD f
\\d\+ CALL_METHOD n=0 nkw=0 \\d\+ CALL_METHOD n=0 nkw=0
@ -517,7 +517,7 @@ arg names: *
(N_EXC_STACK 0) (N_EXC_STACK 0)
bc=0 line=1 bc=0 line=1
######## ########
bc=\\d\+ line=116 bc=\\d\+ line=114
00 LOAD_DEREF 0 00 LOAD_DEREF 0
02 LOAD_CONST_SMALL_INT 1 02 LOAD_CONST_SMALL_INT 1
03 BINARY_OP 27 __add__ 03 BINARY_OP 27 __add__
@ -536,7 +536,7 @@ arg names: * b
(N_EXC_STACK 0) (N_EXC_STACK 0)
bc=0 line=1 bc=0 line=1
######## ########
bc=\\d\+ line=144 bc=\\d\+ line=140
00 LOAD_FAST 1 00 LOAD_FAST 1
01 LOAD_DEREF 0 01 LOAD_DEREF 0
03 BINARY_OP 27 __add__ 03 BINARY_OP 27 __add__

12
tests/cmdline/cmd_verbose.py.exp
View File

@ -8,12 +8,12 @@ arg names:
(N_EXC_STACK 0) (N_EXC_STACK 0)
bc=0 line=1 bc=0 line=1
bc=0 line=3 bc=0 line=3
00 LOAD_NAME print (cache=0) 00 LOAD_NAME print
04 LOAD_CONST_SMALL_INT 1 03 LOAD_CONST_SMALL_INT 1
05 CALL_FUNCTION n=1 nkw=0 04 CALL_FUNCTION n=1 nkw=0
07 POP_TOP 06 POP_TOP
08 LOAD_CONST_NONE 07 LOAD_CONST_NONE
09 RETURN_VALUE 08 RETURN_VALUE
1 1
mem: total=\\d\+, current=\\d\+, peak=\\d\+ mem: total=\\d\+, current=\\d\+, peak=\\d\+
stack: \\d\+ out of \\d\+ stack: \\d\+ out of \\d\+

15
tests/cpydiff/core_fstring_concat.py
View File

@ -1,12 +1,13 @@
""" """
categories: Core categories: Core
description: f-strings don't support concatenation with adjacent literals if the adjacent literals contain braces description: f-strings don't support concatenation with adjacent literals if the adjacent literals contain braces or are f-strings
cause: MicroPython is optimised for code space. cause: MicroPython is optimised for code space.
workaround: Use the + operator between literal strings when either is an f-string workaround: Use the + operator between literal strings when either or both are f-strings
""" """
x = 1 x, y = 1, 2
print("aa" f"{x}") print("aa" f"{x}") # works
print(f"{x}" "ab") print(f"{x}" "ab") # works
print("a{}a" f"{x}") print("a{}a" f"{x}") # fails
print(f"{x}" "a{}b") print(f"{x}" "a{}b") # fails
print(f"{x}" f"{y}") # fails

8
tests/cpydiff/core_fstring_parser.py
View File

@ -1,9 +1,9 @@
""" """
categories: Core categories: Core
description: f-strings cannot support expressions that require parsing to resolve nested braces description: f-strings cannot support expressions that require parsing to resolve unbalanced nested braces and brackets
cause: MicroPython is optimised for code space. cause: MicroPython is optimised for code space.
workaround: Only use simple expressions inside f-strings workaround: Always use balanced braces and brackets in expressions inside f-strings
""" """
f'{"hello {} world"}' print(f'{"hello { world"}')
f"{repr({})}" print(f'{"hello ] world"}')

10
tests/extmod/uasyncio_gather.py
View File

@ -22,8 +22,9 @@ async def factorial(name, number):
async def task(id): async def task(id):
print("start", id) print("start", id)
await asyncio.sleep(0.2) await asyncio.sleep(0.02)
print("end", id) print("end", id)
return id
async def gather_task(): async def gather_task():
@ -36,12 +37,17 @@ async def main():
# Simple gather with return values # Simple gather with return values
print(await asyncio.gather(factorial("A", 2), factorial("B", 3), factorial("C", 4))) print(await asyncio.gather(factorial("A", 2), factorial("B", 3), factorial("C", 4)))
# Test return_exceptions, where one task is cancelled and the other finishes normally
tasks = [asyncio.create_task(task(1)), asyncio.create_task(task(2))]
tasks[0].cancel()
print(await asyncio.gather(*tasks, return_exceptions=True))
# Cancel a multi gather # Cancel a multi gather
# TODO doesn't work, Task should not forward cancellation from gather to sub-task # TODO doesn't work, Task should not forward cancellation from gather to sub-task
# but rather CancelledError should cancel the gather directly, which will then cancel # but rather CancelledError should cancel the gather directly, which will then cancel
# all sub-tasks explicitly # all sub-tasks explicitly
# t = asyncio.create_task(gather_task()) # t = asyncio.create_task(gather_task())
# await asyncio.sleep(0.1) # await asyncio.sleep(0.01)
# t.cancel() # t.cancel()
# await asyncio.sleep(0.01) # await asyncio.sleep(0.01)

3
tests/extmod/uasyncio_gather.py.exp
View File

@ -8,3 +8,6 @@ Task B: factorial(3) = 6
Task C: Compute factorial(4)... Task C: Compute factorial(4)...
Task C: factorial(4) = 24 Task C: factorial(4) = 24
[2, 6, 24] [2, 6, 24]
start 2
end 2
[CancelledError(), 2]

4
tests/extmod/uselect_poll_udp.py
View File

@ -5,7 +5,9 @@ try:
except ImportError: except ImportError:
try: try:
import socket, select import socket, select
except ImportError:
select.poll # Raises AttributeError for CPython implementations without poll()
except (ImportError, AttributeError):
print("SKIP") print("SKIP")
raise SystemExit raise SystemExit

125
tests/import/mpy_native.py
View File

@ -1,125 +0,0 @@
# test importing of .mpy files with native code (x64 only)
import sys, uio
try:
uio.IOBase
import uos
uos.mount
except (ImportError, AttributeError):
print("SKIP")
raise SystemExit
if not (sys.platform == "linux" and sys.maxsize > 2**32):
print("SKIP")
raise SystemExit
class UserFile(uio.IOBase):
def __init__(self, data):
self.data = data
self.pos = 0
def read(self):
return self.data
def readinto(self, buf):
n = 0
while n < len(buf) and self.pos < len(self.data):
buf[n] = self.data[self.pos]
n += 1
self.pos += 1
return n
def ioctl(self, req, arg):
return 0
class UserFS:
def __init__(self, files):
self.files = files
def mount(self, readonly, mksfs):
pass
def umount(self):
pass
def stat(self, path):
if path in self.files:
return (32768, 0, 0, 0, 0, 0, 0, 0, 0, 0)
raise OSError
def open(self, path, mode):
return UserFile(self.files[path])
# these are the test .mpy files
user_files = {
# bad architecture
"/mod0.mpy": b"C\x05\xff\x00\x10",
# test loading of viper and asm
"/mod1.mpy": (
b"C\x05\x0b\x1f\x20" # header
b"\x20" # n bytes, bytecode
b"\x00\x08\x02m\x02m" # prelude
b"\x51" # LOAD_CONST_NONE
b"\x63" # RETURN_VALUE
b"\x00\x02" # n_obj, n_raw_code
b"\x22" # n bytes, viper code
b"\x00\x00\x00\x00\x00\x00" # dummy machine code
b"\x00\x00" # qstr0
b"\x01\x0c\x0aprint" # n_qstr, qstr0
b"\x00\x00\x00" # scope_flags, n_obj, n_raw_code
b"\x23" # n bytes, asm code
b"\x00\x00\x00\x00\x00\x00\x00\x00" # dummy machine code
b"\x00\x00\x00" # scope_flags, n_pos_args, type_sig
),
# test loading viper with truncated data
"/mod2.mpy": (
b"C\x05\x0b\x1f\x20" # header
b"\x20" # n bytes, bytecode
b"\x00\x08\x02m\x02m" # prelude
b"\x51" # LOAD_CONST_NONE
b"\x63" # RETURN_VALUE
b"\x00\x01" # n_obj, n_raw_code
b"\x12" # n bytes(=4), viper code
),
# test loading viper with additional scope flags and relocation
"/mod3.mpy": (
b"C\x05\x0b\x1f\x20" # header
b"\x20" # n bytes, bytecode
b"\x00\x08\x02m\x02m" # prelude
b"\x51" # LOAD_CONST_NONE
b"\x63" # RETURN_VALUE
b"\x00\x01" # n_obj, n_raw_code
b"\x12" # n bytes(=4), viper code
b"\x00\x00\x00\x00" # dummy machine code
b"\x00" # n_qstr
b"\x81\x60" # scope_flags: VIPERBSS | VIPERRODATA | VIPERRELOC (0xe0 encoded over two bytes)
b"\x00\x00" # n_obj, n_raw_code
b"\x06rodata" # rodata, 6 bytes
b"\x04" # bss, 4 bytes
b"\x03\x01\x00" # dummy relocation of rodata
),
}
# create and mount a user filesystem
uos.mount(UserFS(user_files), "/userfs")
sys.path.append("/userfs")
# import .mpy files from the user filesystem
for i in range(len(user_files)):
mod = "mod%u" % i
try:
__import__(mod)
print(mod, "OK")
except ValueError as er:
print(mod, "ValueError", er)
except RuntimeError as er:
print(mod, "RuntimeError", er)
# unmount and undo path addition
uos.umount("/userfs")
sys.path.pop()

4
tests/import/mpy_native.py.exp
View File

@ -1,4 +0,0 @@
mod0 ValueError incompatible native .mpy architecture
mod1 OK
mod2 RuntimeError Corrupt .mpy file
mod3 OK

15
tests/io/resource_stream.py
View File

@ -1,15 +0,0 @@
import uio
import sys
try:
uio.resource_stream
except AttributeError:
print("SKIP")
raise SystemExit
buf = uio.resource_stream("data", "file2")
print(buf.read())
# resource_stream(None, ...) look ups from current dir, hence sys.path[0] hack
buf = uio.resource_stream(None, sys.path[0] + "/data/file2")
print(buf.read())

2
tests/io/resource_stream.py.exp
View File

@ -1,2 +0,0 @@
1234
1234

1
tests/micropython/const.py
View File

@ -1,4 +1,5 @@
# test constant optimisation # test constant optimisation
# This test will only work when MICROPY_COMP_CONST is enabled.
from micropython import const from micropython import const

4
tests/micropython/import_mpy_native_gc.py
View File

@ -48,8 +48,8 @@ class UserFS:
# Pre-compiled examples/natmod/features0 example for various architectures, keyed # Pre-compiled examples/natmod/features0 example for various architectures, keyed
# by the required value of sys.implementation.mpy. # by the required value of sys.implementation.mpy.
features0_file_contents = { features0_file_contents = {
# -march=x64 -mcache-lookup-bc # -march=x64
0xB05: b'C\x05\x0b\x1f \x84b\xe9/\x00\x00\x00SH\x8b\x1ds\x00\x00\x00\xbe\x02\x00\x00\x00\xffS\x18\xbf\x01\x00\x00\x00H\x85\xc0u\x0cH\x8bC \xbe\x02\x00\x00\x00[\xff\xe0H\x0f\xaf\xf8H\xff\xc8\xeb\xe6ATUSH\x8b\x1dA\x00\x00\x00H\x8b\x7f\x08H\x8bk(\xff\xd5H\x8d5 \x00\x00\x00I\x89\xc4H\x8b\x05.\x00\x00\x00\x0f\xb78\xffShL\x89\xe7\xff\xd5H\x8b\x03[]A\\\xc3\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x90\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x84@\x12factorial \x00\x00\r \x01"\xa1\x1c\x01\x1e\xff', 0xA05: b'C\x05\x0a\x1f \x84b\xe9/\x00\x00\x00SH\x8b\x1ds\x00\x00\x00\xbe\x02\x00\x00\x00\xffS\x18\xbf\x01\x00\x00\x00H\x85\xc0u\x0cH\x8bC \xbe\x02\x00\x00\x00[\xff\xe0H\x0f\xaf\xf8H\xff\xc8\xeb\xe6ATUSH\x8b\x1dA\x00\x00\x00H\x8b\x7f\x08L\x8bc(A\xff\xd4H\x8d5\x1f\x00\x00\x00H\x89\xc5H\x8b\x05-\x00\x00\x00\x0f\xb78\xffShH\x89\xefA\xff\xd4H\x8b\x03[]A\\\xc3\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x90\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x84@\x12factorial \x00\x00\r \x01"\xa1\x1c\x01\x1e\xff',
# -march=armv7m # -march=armv7m
0x1605: b"C\x05\x16\x1f \x84\x12\x1a\xe0\x00\x00\x13\xb5\nK\nJ{D\x9cX\x02!\xe3h\x98G\x03F\x01 3\xb9\x02!#i\x01\x93\x02\xb0\xbd\xe8\x10@\x18GXC\x01;\xf4\xe7\x00\xbfj\x00\x00\x00\x00\x00\x00\x00\xf8\xb5\tN\tK~D\xf4X@hgi\xb8G\x05F\x07K\x07I\xf2XyD\x10\x88ck\x98G(F\xb8G h\xf8\xbd6\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x1c\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00\x80\x00\x00\x00\x00\x00\x00\x00\x01\x84\x00\x12factorial \x00\x00\r<\x01>\xa18\x01:\xff", 0x1605: b"C\x05\x16\x1f \x84\x12\x1a\xe0\x00\x00\x13\xb5\nK\nJ{D\x9cX\x02!\xe3h\x98G\x03F\x01 3\xb9\x02!#i\x01\x93\x02\xb0\xbd\xe8\x10@\x18GXC\x01;\xf4\xe7\x00\xbfj\x00\x00\x00\x00\x00\x00\x00\xf8\xb5\tN\tK~D\xf4X@hgi\xb8G\x05F\x07K\x07I\xf2XyD\x10\x88ck\x98G(F\xb8G h\xf8\xbd6\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x1c\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00\x80\x00\x00\x00\x00\x00\x00\x00\x01\x84\x00\x12factorial \x00\x00\r<\x01>\xa18\x01:\xff",
} }

8
tests/micropython/import_mpy_native_x64.py
View File

@ -9,7 +9,7 @@ except (ImportError, AttributeError):
print("SKIP") print("SKIP")
raise SystemExit raise SystemExit
if not (sys.platform == "linux" and sys.maxsize > 2**32): if not (sys.platform == "linux" and sys.maxsize > 2 ** 32):
print("SKIP") print("SKIP")
raise SystemExit raise SystemExit
@ -52,11 +52,11 @@ class UserFS:
# fmt: off # fmt: off
user_files = { user_files = {
# bad architecture # bad architecture
'/mod0.mpy': b'C\x05\xff\x00\x10', '/mod0.mpy': b'C\x05\xfe\x00\x10',
# test loading of viper and asm # test loading of viper and asm
'/mod1.mpy': ( '/mod1.mpy': (
b'C\x05\x0b\x1f\x20' # header b'C\x05\x0a\x1f\x20' # header
b'\x20' # n bytes, bytecode b'\x20' # n bytes, bytecode
b'\x00\x08\x02m\x02m' # prelude b'\x00\x08\x02m\x02m' # prelude
@ -78,7 +78,7 @@ user_files = {
# test loading viper with additional scope flags and relocation # test loading viper with additional scope flags and relocation
'/mod2.mpy': ( '/mod2.mpy': (
b'C\x05\x0b\x1f\x20' # header b'C\x05\x0a\x1f\x20' # header
b'\x20' # n bytes, bytecode b'\x20' # n bytes, bytecode
b'\x00\x08\x02m\x02m' # prelude b'\x00\x08\x02m\x02m' # prelude

Some files were not shown because too many files have changed in this diff Show More