Commit Graph

1044 Commits

Author SHA1 Message Date
Scott Shawcroft 623b6fad16
Fix nested categories in update_sdkconfig.py
Also, mark QTPy C3 as BIN only.
2022-03-21 17:04:19 -07:00
Jeff Epler 4465adfe01
Merge pull request #6175 from tannewt/mdns
Add mdns module
2022-03-21 15:55:03 -05:00
Scott Shawcroft 8319faa032
Merge pull request #6155 from prplz/lilygo_ttgo_t-01c3
Add board lilygo_ttgo_t-01c3
2022-03-17 18:41:05 -07:00
Jeff Epler e9d81c2826
Add mdns module
This allows for CircuitPython to resolve a .local domain and find
other devices with MDNS services.

First step for #6174
2022-03-17 18:16:16 -07:00
Michael Himing 196bb4d1f4 lilygo_ttgo_t-01c3: only require .bin file 2022-03-17 17:24:24 +11:00
Scott Shawcroft dcc3ec171e
Merge remote-tracking branch 'adafruit/7.2.x' into merge_7.2.2 2022-03-15 13:56:11 -07:00
Dan Halbert f13d218749
Merge pull request #6125 from tannewt/usb_host
Start of USB host API
2022-03-11 21:41:28 -05:00
Scott Shawcroft a719fabbb6
Shrink bluemicro833 build 2022-03-11 16:04:12 -08:00
Dan Halbert c70425ab6b add annotations to __future__; don't generate __future__ stubs 2022-03-10 11:19:35 -05:00
Dan Halbert dd239efc94 add annotations to __future__; don't generate __future__ stubs 2022-03-07 17:43:15 -05:00
Dan Halbert 548d6beb4c
Merge pull request #6085 from rimwolf-redux/msgpack
changed msgpack to use ByteStream type annotations
2022-03-01 10:17:10 -05:00
Dan Halbert 38e9fd703b add circuitpython_typing. to some annotations; generalize extract_pyi.pi 2022-02-28 21:27:24 -05:00
Dan Halbert a45f9a83ea Merge remote-tracking branch 'adafruit/7.2.x' into merge-7.2.x-to-main 2022-02-24 13:07:56 -05:00
Dan Halbert 7107692678 use --long for describe in setup.py-stubs 2022-02-18 21:22:02 -05:00
Jeff Epler 4c9d14b73f Merge remote-tracking branch 'origin/main' into merge-1.18 2022-02-16 11:43:53 -06:00
Jeff Epler d8569cc923 build out some additional submodules for testing 2022-02-16 11:41:01 -06:00
Scott Shawcroft 947a53c8d1
Merge pull request #6033 from jepler/better-describe
Improve use of `git describe`
2022-02-15 10:48:55 -08:00
Jeff Epler 01cabb0324 Merge tag 'v1.18'
Boosted performance, board.json metadata, more mimxrt, rp2, samd features

This release of MicroPython sees a boost to the overall performance of the
VM and runtime.  This is achieved by the addition of an optional cache to
speed up general hash table lookups, as well as a fast path in the VM for
the LOAD_ATTR opcode on instance types.  The new configuration options are
MICROPY_OPT_MAP_LOOKUP_CACHE and MICROPY_OPT_LOAD_ATTR_FAST_PATH.  As part
of this improvement the MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE option has
been removed, which provided a similar map caching mechanism but with the
cache stored in the bytecode, which made it not useful on bare metal ports.
The new mechanism is measured to be at least as good as the old one,
applies to more map lookups, has a constant RAM overhead, and applies to
native code as well as bytecode.

These performance options are enabled on the esp32, mimxrt, rp2, stm32 and
unix ports.  For esp32 and mimxrt some code is also moved to RAM to further
boost performance.  On stm32, performance increases by about 20% for
benchmarks that are heavy on name lookups, like misc_pystone.py and
misc_raytrace.py.  On esp32 performance can increase by 2-3x, and on mimxrt
it is up to 6x.

All boards in all ports now have a board.json metadata file, which is used
to automatically build firmware and generate a webpage for that board
(among other possibilities).  Auto-build scripts have been added for this
purpose and they build all esp32, mimxrt, rp2, samd and stm32 boards.  The
generated output is available at https://micropython.org/download.

Support for FROZEN_DIR and FROZEN_MPY_DIR has been deprecated for some time
and was finally removed in this release.  Instead of these, FROZEN_MANIFEST
can be used.  The io.resource_stream() function is also removed, replaced
by the pure Python version in micropython-lib.

The search order for importing frozen Python modules is now controlled by
the ".frozen" entry in sys.path.  This string is added by default in the
second position in sys.path.  User code should adjust sys.path depending on
the desired behaviour.  Putting ".frozen" first in sys.path will speed up
importing frozen modules.

A bug in multiple precision integers with bitwise of -0 was fixed in commit
2c139bbf4e.

The platform module has been added to allow querying the compiler and
underlying SDK/HAL/libc version.  This is enabled on esp32, mimxrt and
stm32 ports.

The mpremote tool now supports seek, flush, mkdir and rmdir on PC-mounted
filesystems.  And a help command has been added.

The documentation has seen many additions and improvements thanks (for a
second time) to the Google Season of Docs project.  The rp2 documentation
now includes a reference for PIO assembly instructions, a PIO quick
reference and a PIO tutorial.  The random and stm modules have been
documented, along with sys.settrace, manifest.py files and mpremote.  There
is also now more detail about the differences between MicroPython and
standard Python 3.5 and above.

The esp32 port sees support for ESP32-S3 SoCs, and new boards GENERIC_S3,
ESP32_S2_WROVER, LOLIN_S2_MINI, LOLIN_S2_PICO and UM_FEATHERS2NEO.  The PWM
driver has been improved and now supports all PWM timers and channels, and
the duty_u16() and duty_ns() methods, and it keeps the duty constant when
changing frequency.  The machine.bitstream() function has been improved to
use RMT, with an option to select the original bit-banging implementation.

The mimxrt port gained new hardware features: SDRAM and SD card support, as
well as network integration with a LAN driver.  The machine.WDT class was
added along with the machine.reset_cause(), machine.soft_reset(),
machine.unique_id() add machine.bitstream() functions.  DHT sensor support
was added, and f-strings were enabled.

The rp2 port now has support for networking, and bluetooth using NimBLE.
The Nina-W10 WiFi/BT driver is fully integrated and supported by the new
Arduino Nano RP2040 connect board.  I2S protocol support is added along
with a machine.bitstream() driver and DHT sensor support.  The PWM driver
had a bug fix with the accuracy of setting/getting the frequency, and the
duty value is now retained when changing the frequency.

On the samd port there is now support for the internal flash being a block
device, and for filesystems and the os module.  Pin and LED classes have
been implemented.  There are more time functions, more Python features
enabled, and the help() function is added.  SEEED_WIO_TERMINAL and
SEEED_XIAO board definitions are now available.

The stm32 port now has support for F427, F479 and H7A3(Q)/H7B3(Q) MCUs, and
new board definitions for VCC_GND_H743VI, OLIMEX_H407, MIKROE_QUAIL,
GARATRONIC_PYBSTICK26_F411, STM32H73B3I_DK.  A bug was fixed in the SPI
driver where a SPI transfer could fail if the CYW43 WiFi driver was also
active at the same time.

On the windows port the help() function has been enabled, and support for
build variants added, to match the unix port.

The zephyr port upgraded Zephyr to v2.7.0.

The change in code size since the previous release for various ports is
(absolute and percentage change in the text section):

       bare-arm:  -1520  -2.605%
    minimal x86:  -2256  -1.531%
       unix x64:   -457  -0.089%
    unix nanbox:   -925  -0.204%
          stm32:   +312  +0.079% PYBV10
         cc3200:   -176  -0.096%
        esp8266:   +532  +0.076% GENERIC
          esp32: +27096  +1.820% GENERIC
            nrf:   -212  -0.121% pca10040
            rp2:  +9904  +2.051% PICO
           samd: +35332 +33.969% ADAFRUIT_ITSYBITSY_M4_EXPRESS

The changes that dominate these numbers are:
- bare-arm, minimal: use of new MICROPY_CONFIG_ROM_LEVEL_MINIMUM option and
  subsequent disabling of remaining optional features
- unix, cc3200, nrf: general code size reductions of the core
- stm32: performance improvements, addition of platform module
- esp8266: enabling f-strings
- esp32: use of -O2 instead of -Os
- rp2: machine.I2S and other new hardware features
- samd: filesystem support and other new hardware features

Thanks to everyone who contributed to this release: Alan Dragomirecký,
Alexey Shvetsov, Andrew Leech, Andrew Scheller, Antoine Aubert, Boris
Vinogradov, Chris Boudacoff, Chris Fiege, Christian Decker, Damien George,
Daniel Gorny, Dave Hylands, David Michieli, Emilie Feral, Frédéric Pierson,
gibbonsc, Henk Vergonet, iabdalkader, Ihor Nehrutsa, Jan Hrudka, Jan Staal,
jc_.kim, Jim Mussared, Jonathan Hogg, Laurens Valk, leo chung, Lorenzo
Cappelletti, Magnus von Wachenfeldt, Matt Trentini, Matt van de Werken,
Maureen Helm, Michael Bentley, Michael Buesch, Mike Causer, Mike Teachman,
Mike Wadsten, Ned Konz, NitiKaur, oli, patrick, Patrick Van Oosterwijck,
Peter Boin, Peter Hinch, Peter van der Burg, Philipp Ebensberger, Pooya
Moradi, retsyo, robert-hh, roland van straten, Scott Armitage, Sebastian
Wicki, Seon Rozenblum, Sergei Silnov, Simon Baatz, Stewart Bonnick, stijn,
Tobias Thyrrestrup, Tomas Vanek, YoungJoon Chun.

What follows is a detailed list of changes, generated from the git commit
history, and organised into sections.

Main components
===============

all:
- remove MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE
- update Python formatting to latest Black version 21.12b0
- remove support for FROZEN_DIR and FROZEN_MPY_DIR

py core:
- parse: simplify parse nodes representing a list
- emitnative: ensure load_subscr does not clobber existing REG_RET
- mpconfig.h: define initial templates for "feature levels"
- vm: add a fast path for LOAD_ATTR on instance types
- map: add an optional cache of (map+index) to speed up map lookups
- builtinimport: forward all debug printing to MICROPY_DEBUG_PRINTER
- add wrapper macros so hot VM functions can go in fast code location
- runtime: fix crash when exc __new__ doesn't return an exc instance
- mpconfig.h: define the "extra" feature level
- mpconfig.h: revert MICROPY_REPL_INFO to disabled at all levels
- gc: add hook to run code during time consuming GC operations
- showbc: print unary-op string when dumping bytecode
- modsys: replace non-ASCII quote char with ASCII char
- runtime: allow types to use both .attr and .locals_dict
- lexer: support nested [] and {} characters within f-string params
- objfun.h: remove obsolete comments about entries in extra_args
- builtinimport: refactor module importing
- showbc: fix printing of raw bytecode header on nanbox builds
- modio: remove io.resource_stream function
- only search frozen modules when '.frozen' is found in sys.path
- mkrules.cmake: set frozen preprocessor defs early
- runtime: allow initialising sys.path/argv with defaults
- mpstate.h: only include sys.path/argv objects in state when enabled
- mpz: fix bugs with bitwise of -0 by ensuring all 0's are positive
- qstr: reset mpstate.qstr_last_chunk before raising an error
- modbuiltins: add additional macro for extending builtins
- mpconfig.h: define MICROPY_PY_USSL_FINALISER only if not defined

extmod:
- machine_i2c: make SoftI2C configurable via macro option
- machine_spi: make SoftSPI configurable via macro option
- modonewire: make _onewire module configurable via macro option
- machine_pwm: factor out machine.PWM bindings to common code
- move modnetwork and modusocket from stm32 to extmod
- modnetwork: add STA_IF and AP_IF constants
- modnetwork: add extended socket state
- modusocket: add read/write stream methods to socket object
- modnetwork: define network interfaces in port config files
- network_cyw43: make consistent use of STA and AP constants
- modnetwork: remove STM32 references
- modnetwork: remove modnetwork socket u_state member
- mpbthci.h: add mp_bluetooth_hci_uart_any prototype
- nimble: add nimble CMake fragment file
- add platform module
- moduplatform: improve implementation for PC ports
- vfs_posix_file: support MP_STREAM_POLL in vfs_posix_file_ioctl
- modbluetooth: add connection interval to gap_connect
- nimble: update to NimBLE v1.4
- nimble: remove workaround for OS_ENOMEM
- uasyncio: fix gather returning exceptions from a cancelled task
- uplatform: remove unused definitions
- uplatform: use generic custom platform string
- network_ninaw10: fix scan list order to match other NICs
- modbluetooth: support gap_connect(None) to cancel a connection
- modure: redirect regex debug printing to mp_printf
- network_ninaw10: fix config of AP mode
- network_ninaw10: disable active connections before connecting
- network_ninaw10: make NIC state persistent
- network_ninaw10: return -1 on timeout from recv/send
- network_ninaw10: make recv/recvfrom interchangeable
- moduplatform: detect xtensa arch
- modusocket: allow setting timeout on unbound sockets
- modusocket: initialise accepted socket state
- network_ninaw10: use socket timeout preset in modusocket
- modbluetooth: fix conditional compilation of ringbuf_put_uuid
- modbluetooth: put declaration of connect_cancel in correct place

shared:
- libc/string0: don't include string.h, and provide __memcpy_chk
- runtime/pyexec: cleanup EXEC_FLAG flag constants

drivers:
- ninaw10: add ublox Nina-W10 WiFi/BT module driver
- lsm6dsox: add LSM6DSOX driver and examples
- neopixel: avoid heap alloc in fill()
- ninaw10: fix BSSID byte order, and add null byte to ESSID
- ninaw10/nina_wifi_drv: fix DNS resolution

mpy-cross: no changes specific to this component/port

lib:
- mynewt-nimble: switch to the MicroPython fork of NimBLE
- asf4: point submodule to latest commit on circuitpython branch
- update pico-sdk to 1.3.0 and tinyusb to 0.12.0
- stm32lib: update library for L4 v1.17.0, new G4, WL, and MMC fixes
- stm32lib: update library for fix to F7 USB HS

Support components
==================

docs:
- library/os.rst: clarify littlefs requirements for block erase
- library/bluetooth.rst: update incorrect link to gatts_write
- make.bat: change Windows output dir from '_build' to 'build'
- library/machine.I2S.rst: specify that I2S.shift args are kw-only
- esp32: explain ESP32 PWM modes, timers, and channels
- rp2: add reference for PIO assembly instructions, and PIO tutorial
- library/random.rst: document the random module
- reference/mpremote.rst: add docs for mpremote
- reference/manifest.rst: add docs for manifest.py files
- library/stm.rst: document the stm module
- esp32/tutorial: add an example of peripheral control via regs
- rp2/general.rst: fix typo with missing spaces
- library/framebuf.rst: adjust dimensions in example
- library/rp2.rst: update function asm_pio_encode to add sideset_opt
- reference/filesystem.rst: add detail on how to use littlefs fuse
- rp2/quickref.rst: add section on PIO
- library/sys.rst: add docs for sys.settrace
- esp8266/tutorial: fix comments of FrameBuffer examples
- library/uasyncio.rst: detail exception behaviour in cancel/timeout
- library/machine.Timer.rst: document 'id' as positional-only arg
- library/machine.SPI.rst: add example SPI usage
- library/machine.Timer.rst: document `period` and `callback` args
- library/machine.Pin.rst: add Pin.ANALOG mode constant
- remove trailing spaces and convert tabs to spaces
- library/sys.rst: add note about '.frozen' as an entry in sys.path
- differences: document details of new PEPs/features in Python 3.5+
- update copyright year range to include 2022
- esp32: update RMT quickref example to match latest code

examples: no changes specific to this component/port

tests:
- perf_bench: use math.log instead of math.log2
- basics: add tests for type-checking subclassed exc instances
- micropython/const.py: add comment about required config for test
- cpydiff: clarify f-string diffs regarding concatenation
- basics/int_big_cmp.py: add more tests for big-int comparison
- extmod: skip uselect_poll_udp when poll() is not available

tools:
- autobuild: add auto build for GENERIC_C3_USB
- ci.sh: use IDF v4.4 as part of esp32 CI and build GENERIC_S3
- autobuild: add the MIMXRT1010_EVK board to autobuild
- ci.sh: use a specific ESP IDF v4.4 commit
- autobuild: add script to generate website board metadata
- dfu.py: make tool work with python3 when parsing DFU files
- autobuild: automatically build all mimxrt, rp2 and samd boards
- autobuild: automatically build all stm32 boards
- mpremote: implement seek and flush in ioctl method
- autobuild: automatically build all esp32 boards
- upip.py: support == to specify exact package version
- makemanifest.py: make str conversion compatible with Python 2
- makemanifest.py: merge make-frozen.py
- mpremote: add mkdir and rmdir to RemoteFS
- mpremote: add help command
- mpremote: add link to mpremote docs URL in help message
- upip.py: skip '.frozen' entry in sys.path for install path
- autobuild: build esp8266 OTA image with GENERIC_1M board
- ci.sh: upgrade Zephyr docker image to v0.21.0
- ci.sh: build zephyr nucleo_wb55rg to test zephyr bluetooth build

CI:
- workflows: use Python 3.8 for macos workflow
- workflows: add new workflow to build ports download metadata

The ports
=========

all ports:
- add board.json for all boards
- add images, features and urls to board.json
- add '.frozen' as the first entry in sys.path
- move '.frozen' to second entry in sys.path

bare-arm port:
- mpconfigport.h: use MICROPY_CONFIG_ROM_LEVEL_MINIMUM
- mpconfigport.h: disable remaining optional features

cc3200 port: no changes specific to this component/port

esp8266 port:
- boards/GENERIC: enable f-strings
- extract qstr from object when comparing keys in config()
- etshal.h: remove unneeded function declarations
- allow building a board to any dest directory

esp32 port:
- boards: add new FeatherS2-Neo board definition
- machine_timer: use tx_update member for IDF 4.4 and above
- add support for ESP32-S3 SoCs
- boards: add new GENERIC_S3 board definition
- machine_hw_spi: fix hardware SPI DMA channels for S2/S3
- boards: add board definition for ESP32-S2-WROVER module
- boards: add LOLIN_S2_MINI ESP32-S2 board
- machine_pwm: add support for all PWM timers and channels
- README: updated readme with req IDF vers for ESP32-S2, C3 and S3
- usb: add USB host connection detection for CDC serial output
- machine_pin: block out IO16 and IO17 when using SPIRAM on ESP32
- mpthreadport: fix TCB cleanup function so thread_mutex is ready
- main: add option for a board to hook code into startup sequence
- split out WLAN code from modnetwork.c to network_wlan.c
- enable optimisations and move code to iRAM to boost performance
- usb: improve speed of USB CDC output
- add specific deploy_s2.md instructions for esp32-s2
- boards/LOLIN_S2_MINI: add image to board.json
- boards: update board and deploy metadata for UM_xxx boards
- usb: further improve speed of USB CDC output
- boards/LOLIN_S2_PICO: add LOLIN_S2_PICO board definition files
- boards/ESP32_S2_WROVER: link to specific deploy_s2 instructions
- support building with latest IDF v5
- in machine_i2s, send null samples in underflow situations
- in machine_i2s, make object reference arrays root pointers
- add SDCard support for S3, and a GENERIC_S3_SPIRAM board
- boards/GENERIC_S3: enable BLE on ESP32 S3
- machine_pwm: implement duty_u16() and duty_ns() PWM methods
- extract qstr from object when comparing keys in config()
- machine_pin: make GPIO 26 usable for S2,S3 if SPIRAM not config'd
- machine_hw_spi: fix SPI default pins reordering on ESP32-S2/S3
- machine_hw_spi: set proper default SPI(id=1) pins on S2,S3 and C3
- machine_hw_spi: set proper default SPI(id=2) pins on S2 and S3
- boards: remove SPI pin defaults from GENERIC S2/S3 boards
- modnetwork: synchronize WiFi AUTH_xxx constants with IDF values
- machine_pwm: keep duty constant when changing frequency
- machine_bitstream: replace bit-bang code with RMT-based driver
- machine_i2s: add support for ESP-IDF 4.4
- machine_bitstream: fix signal duplication on output pins
- esp32: enable platform module with IDF version
- boards/GENERIC_D2WD: build with -Os optimisation
- esp32_rmt: install RMT driver on core 1
- machine_bitstream: reinstate bitstream bit-bang implementation

javascript port: no changes specific to this component/port

mimxrt port:
- sdcard: implement SDCard driver
- machine_bitstream: add bitstream function to machine module
- rework flash configuration
- sdram: add SDRAM support
- eth: add LAN support and integrate the network module
- modmachine: implement machine.WDT() and machine.reset_cause()
- boards: fix the D14/D15 pin assignment of MIMXRT1050/60/64_EVK
- hal: remove duplicate definitions from flexspi_hyper_flash.h
- dma_channel: fix the DMA channel management
- fix cycle counter for time.ticks_cpu() and machine.bitstream()
- add dht_readinto() to the mimxrt module, and freeze dht.py
- extend the help() message and README.md
- mpconfigport.h: enable f-strings
- modmachine: implement soft_reset() and unique_id() functions
- boards/make-pins.py: allow empty lines and comments in pins.csv
- optimize the runtime speed
- enable the platform module
- boards: add the Seeed ARCH MIX board
- boards: update the board.json files and add deploy_xx.md files
- fix mp_hal_quiet_timing_enter()/exit() so timer still runs
- support PWM using the FLEXPWM and QTMR modules
- define UART 0 on MIMXRT boards
- support selection of PHY type and address
- re-enable eth checksum creation by HW
- fix a tiny unnoticed bug in sdcard.c
- add a driver for the DP83848 PHY device
- refactor the reading of the machine id
- enable ticks_cpu at boot time for NDEBUG builds only
- use -Og instead of -O0 for DEBUG builds
- tidy up the board flash related files
- hal: allow readSampleClkSrc to be configured by a board
- enable MICROPY_PY_USSL_FINALISER

minimal port:
- mpconfigport.h: use MICROPY_CONFIG_ROM_LEVEL_MINIMUM
- Makefile: don't force a 32-bit build
- mpconfigport.h: disable features that are not needed

nrf port:
- Makefile: improve Black Magic Probe commands
- main: use VFS helper function to mount fs and chdir

pic16bit port: no changes specific to this component/port

powerpc port: no changes specific to this component/port

qemu-arm port: no changes specific to this component/port

rp2 port:
- mpconfigport.h: enable heapq module
- add support for bluetooth module using NimBLE
- add framework for networking
- mpconfigport.h: use the "extra" feature level
- enable optimisations (comp goto, map cache, fast attr)
- machine_i2s: add I2S protocol support
- add support for Nina-W10 WiFi/BT module
- boards: add support for Arduino Nano RP2040
- machine_bitstream: implement the machine.bitstream driver
- boards: add neopixel.py to manifest.py
- rp2_pio: support exec with sideset
- boards/PIMORONI_PICOLIPO_16MB: fix 16MB flash size
- boards: add PYBSTICK26 RP2040 board definition
- machine_uart: handle and clear UART RX timeout IRQ
- boards/ARDUINO_NANO_RP2040_CONNECT: set default I2C pins
- machine_pwm: fix PWM frequency setting
- machine_pwm: keep duty value when changing the frequency
- add support for DHT11 and DHT22 sensors
- CMakeLists.txt: allow a board to override PICO_BOARD
- boards/GARATRONIC_PYBSTICK26_RP2040: use correct pico-sdk board cfg

samd port:
- integrate latest asf4, add help, more time funcs and uPy features
- samd_soc: allow a board to configure the low-level MCU config
- add internal flash block device, filesystem and uos support
- add Pin and LED classes, and machine.unique_id
- boards/ADAFRUIT_FEATHER_M0_EXPRESS: update for flash and pins
- boards/ADAFRUIT_ITSYBITSY_M4_EXPRESS: update for flash and pins
- boards/MINISAM_M4: update for flash and pins
- boards/ADAFRUIT_TRINKET_M0: update for flash and pins
- boards/SAMD21_XPLAINED_PRO: update for flash and pins
- boards/SEEED_WIO_TERMINAL: add new board definition
- boards/SEEED_XIAO: add new board definition
- README.md: update README to reflect new features and boards

stm32 port:
- pin: enable GPIO clock of pin if it's constructed without init
- main: don't unconditionally enable GPIO A,B,C,D clocks
- boards/VCC_GND_H743VI: add board definition for VCC_GND_H743VI
- boards/OLIMEX_E407: add Ethernet RMII support
- boards/LEGO_HUB_NO6: remove user paths from cc2564 init file
- boards: remove trailing spaces, and add newline at end of file
- add basic support for STM32H750
- add support for H7A3(Q)/H7B3(Q), and STM32H73B3I_DK board defn
- suggest putting code in main.py not boot.py
- boards/make-pins.py: allow a CPU pin to be hidden
- boards/make-pins.py: allow empty lines and comments in pins.csv
- dma: add functions for external users of DMA to enable clock
- enable LOAD_ATTR fast path, and map lookup caching on >M0
- boards: add OLIMEX H407 board definition
- enable platform module
- extended flash filesystem space to 512K on H743 boards
- boards/NUCLEO_H743ZI: enable VfsLfs2 on NUCLEO_H743ZI(2) boards
- boards: add PF11-BOOT0 to stm32f091_af.csv
- machine_i2c: use hardware I2C for STM32H7
- sdram: enforce gcc opt, and use volatile and DSB in sdram_test
- usbd_cdc_interface: allow a board to hook into USBD CDC RX events
- mpbthciport: allow a board to hook BT HCI poll functions
- pendsv: allow a board to add entries for pendsv_schedule_dispatch
- boards: add images to board.json for Adafruit and VCC_GND boards
- uart: fix race conditions and clearing status in IRQ handler
- mpconfigport.h: use the "extra" feature level
- in machine_i2s, send null samples in underflow situations
- in machine_i2s, make object reference arrays root pointers
- led: support an extra 2 LEDs in board configuration
- boards/MIKROE_CLICKER2_STM32: add more detail to board.json
- boards: add new board MikroElektronika Quail, and F427 support
- main: run optional frozen module at boot
- sdio: don't explicitly disable DMA2 on deinit of SDIO
- dma: make DMA2_Stream3 exclusive to SDIO when CYW43 enabled
- boards: build NUCLEO_WB55 and STM32F769DISC without mboot enabled
- boards: add PYBSTICK26 F411 board definition
- boards/NADHAT_PYBF405: rename board to GARATRONIC_NADHAT_F405
- usb: use a table of allowed values to simplify usb_mode get/set
- boards/NUCLEO_WB55: update rfcore_firmwre for new WS
- flashbdev: support generic flash storage config via link symbols
- boards: convert F413,F439,H743,L4xx,WB55 to new flash FS config
- add support for F479 MCUs
- include HAL MMC code in F4 builds
- boards/make-pins.py: use cpu pins to define static alt-fun macros
- boards/NUCLEO_WB55: fix LED ordering
- boards/LEGO_HUB_NO6: set filesystem label as HUB_NO6
- boards: remove stray '+' characters at start of lines in ld files
- boards: remove unused MICROPY_HW_ENABLE_TIMER config
- boards: enable MICROPY_HW_ENABLE_SERVO on various boards
- update L4 code to build with latest stm32lib and L4 HAL 1.17.0
- main: call sdcard_init when only MICROPY_HW_ENABLE_MMCARD enabled
- sdcard: support 8-bit wide SDIO bus
- sdcard: add config option to force MM card capacity
- factoryreset: init vfs flags before calling pyb_flash_init_vfs
- qspi: fix typo in address comment
- boards/make-pins.py: generate empty ADC table if needed
- boards/OLIMEX_H407: fix typo in OLIMEX H407 board.json
- network_wiznet5k: fix build error with wiznet5k and lwip enabled
- enable MICROPY_PY_USSL_FINALISER

teensy port:
- switch to use manifest.py instead of FROZEN_DIR

unix port:
- enable LOAD_ATTR fast path, and map lookup caching
- modusocket: support MP_STREAM_POLL in unix socket_ioctl
- modos: add support for uos.urandom(n)
- coverage: change remaining printf to mp_printf
- Makefile: use -Og instead of -O0 for debug builds

windows port:
- README: remove unsupported Python instructions for Cygwin
- mpconfigport.h: enable help and help("modules")
- add support for build variants to windows port
- run tests via Makefile
- appveyor: build both standard and dev variants
- appveyor: build mpy-cross only once for mingw-w64
- msvc: run qstr preprocessing phase in parallel

zephyr port:
- mphalport.h: remove unused and unimplemented C-level pin API
- increase minimum CMake version to 3.20.0
- update include path to reboot.h
- get UART console device from devicetree instead of Kconfig
- use CONFIG_USB_DEVICE_STACK for conditional USB device support
- upgrade to Zephyr v2.7.0
- modbluetooth_zephyr: provide dummy connect_cancel function
2022-02-15 12:36:26 -06:00
Jeff Epler ddfcfca65d Don't remove all micropython examples, we want natmod and usercmodule 2022-02-15 10:14:47 -06:00
Michael Himing 90e94dc485 Make ai_thinker_esp32-c3s-2m .bin only 2022-02-15 08:41:42 +11:00
Dan Halbert cb80b01251 Fix and revise convert-release-notes.py 2022-02-11 16:21:20 -05:00
anecdata 9342f0880d
added Saola to ignore duplicate PID 2022-02-09 00:11:46 -06:00
anecdata 0dc89a588a ignore duplicate PID 2022-02-08 23:50:08 -06:00
Jeff Epler 07e710b0a0
Merge remote-tracking branch 'origin/main' into floppy 2022-02-04 09:01:36 -06:00
Jeff Epler 6a7742ba88
Merge remote-tracking branch 'origin/main' into floppy 2022-02-04 09:00:32 -06:00
Jeff Epler f2c31c8d82
add adafruit_floppy, put in correct position 2022-02-02 10:14:19 -06:00
Dan Halbert 748834c6ba update to py tests and scripts due to black 22.1.0 2022-01-29 17:14:01 -05:00
Scott Shawcroft ce3f1dda16
C3s are bin only 2022-01-19 10:16:04 -08:00
Damien George 38054a57f3 tools/mpremote: Bump version to 0.1.0.
Signed-off-by: Damien George <damien@micropython.org>
2022-01-17 09:39:38 +11:00
Jeff Epler 88abbb0029
submodules required must be listed explicitly 2022-01-13 20:40:30 -06:00
Damien George 49325de475 tools/ci.sh: Build zephyr nucleo_wb55rg to test zephyr bluetooth build.
And eliminate one build to reduce CI time.

Signed-off-by: Damien George <damien@micropython.org>
2022-01-13 13:46:03 +11:00
Michael Himing 1cd98fed26 Add esp32s3 dev kits to USB ID ignore list 2022-01-13 13:45:46 +11:00
Jeff Epler e76495605b
Merge pull request #5840 from tannewt/enable_tr_builds
Enable Turkish language builds
2022-01-12 09:02:39 -06:00
Scott Shawcroft 1220346195
Enable Turkish language builds 2022-01-10 11:08:29 -08:00
Scott Shawcroft 44997523e1
Fix Pi Zero file extension 2022-01-07 16:31:21 -08:00
Scott Shawcroft 06ddf06a49
Merge pull request #5817 from EmergReanimator/file_permission_fix
Files' execution permission fix
2022-01-07 10:45:10 -08:00
Dan Halbert afd686986a
Merge pull request #5800 from tannewt/rpi_zero
Initial Pi Zero support
2022-01-07 09:05:30 -05:00
EmergReanimator 54065a7719 Updated to latest uf2 commit where file permissions are corrected. 2022-01-07 10:04:11 +01:00
EmergReanimator 990c00209f Removed obsolete file. 2022-01-07 09:47:25 +01:00
EmergReanimator 0b5681a310 Added execution permission for board and memory info python scripts 2022-01-07 09:46:16 +01:00
Scott Shawcroft fa272f5a70
Fix iMX and zero_w build extension 2022-01-06 10:24:02 -08:00
Maureen Helm c6d26bc524 zephyr: Upgrade to Zephyr v2.7.0.
Updates the Zephyr port build instructions and CI to use the latest
Zephyr release tag.

Signed-off-by: Maureen Helm <maureen.helm@intel.com>
2022-01-06 14:09:39 +11:00
Maureen Helm 1469e29905 tools/ci.sh: Upgrade Zephyr docker image to v0.21.0.
As a prerequisite to upgrading to Zephyr v2.7.0, upgrade CI to use
Zephyr docker image v0.21.0. In particular, this is needed to pick up a
newer CMake version because Zephyr v2.7.0 increased the minimum CMake
version required to 3.20.0.

Signed-off-by: Maureen Helm <maureen.helm@intel.com>
2022-01-06 14:09:39 +11:00
Jeff Epler 4a23742100
Use the first-parent heuristic for describing the version 2022-01-04 11:14:26 -06:00
Jeff Epler 23022a5a7f
Add, use tools/describe script
This isn't intended to make any overt behavioral change, but
there is a slight one in the value of CP_VERSION that will be used
during CI, because it will now include `--always --match "..."`,
so there could be a change to the uploaded name of the mpy-cross
artifacts on s3.
2022-01-04 11:14:03 -06:00
Damien George bfe9eba484 tools/autobuild: Build esp8266 OTA image with GENERIC_1M board.
Because the GENERIC board won't fit in the flash defined by esp8266_ota.ld.

Signed-off-by: Damien George <damien@micropython.org>
2021-12-30 15:47:49 +11:00
Damien George a29c70c9b4 esp8266: Allow building a board to any dest directory.
Signed-off-by: Damien George <damien@micropython.org>
2021-12-30 15:47:49 +11:00
Damien George 959e6f7da9 tools/upip.py: Skip '.frozen' entry in sys.path for install path.
Signed-off-by: Damien George <damien@micropython.org>
2021-12-29 23:55:37 +11:00
Damien George 028776d97b tools/mpremote: Add link to mpremote docs URL in help message.
Signed-off-by: Damien George <damien@micropython.org>
2021-12-29 23:35:24 +11:00
Sergei Silnov 74e995dfd2 tools/mpremote: Add help command.
Fixes issue #7480
2021-12-29 09:46:33 +01:00
Michael Himing ad63150e6d Don't freeze utils directories 2021-12-29 11:56:30 +11:00
Jeff Epler 166793f843 circuitpython_stubs: Use `circuitpython_typing` for cp-specific typing
Installing stubs for a module named `_typing` seems like a dubious
decision.
2021-12-22 11:14:21 -06:00
Michael Bentley 7566d107d5 tools/mpremote: Add mkdir and rmdir to RemoteFS.
This allows the remote MicroPython instance to create and delete
directories from the mounted host filesystem in addition to the already
existing functionality of reading, creating, and modifying files.

Signed-off-by: Michael Bentley <mikebentley15@gmail.com>
2021-12-22 12:13:39 +11:00
Jeff Epler 7f189b3e30
Merge pull request #5747 from adafruit/7.1.x
Merge 7.1.x fixes and updates into main
2021-12-17 20:44:43 -07:00
Jeff Epler bd282bb1be
do not backslash-escape forward slashes 2021-12-17 09:34:47 -06:00
Jeff Epler da4f5bdaef
fix ci matrix when ulab is modified
When the submodule is modified, the modified 'filename' is simply the
submodule path `extmod/ulab`, not the specific file(s) in the submodule
that are modified.
2021-12-17 09:32:53 -06:00
Jim Mussared e0bf4611c3 py: Only search frozen modules when '.frozen' is found in sys.path.
This changes makemanifest.py & mpy-tool.py to merge string and mpy names
into the same list (now mp_frozen_names).

The various paths for loading a frozen module (mp_find_frozen_module) and
checking existence of a frozen module (mp_frozen_stat) use a common
function that searches this list.

In addition, the frozen lookup will now only take place if the path starts
with ".frozen", which needs to be added to sys.path.

This fixes issues #1804, #2322, #3509, #6419.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-12-18 00:01:59 +11:00
Jim Mussared f853e3e106 tools/makemanifest.py: Merge make-frozen.py.
Takes the functionality from tools/make-frozen.py, adds support for
multiple frozen directories, and moves it to tools/makemanifest.py.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-12-17 23:54:05 +11:00
Scott Shawcroft 3f50453dd0
Merge pull request #5727 from tannewt/rpi_neopixel
Add neopixel support on PWM capable pins
2021-12-16 11:44:02 -08:00
Scott Shawcroft 190aee551e
Merge pull request #5699 from jpconstantineau/board-pykey87
Add support for PyKey family of keyboards
2021-12-15 10:59:55 -08:00
Scott Shawcroft 4e41bf1dc7
Checkout ulab for build 2021-12-15 10:56:27 -08:00
Damien George 598618e8cf tools/makemanifest.py: Make str conversion compatible with Python 2.
Signed-off-by: Damien George <damien@micropython.org>
2021-12-15 16:51:08 +11:00
Christian Decker e9f880482c tools/upip.py: Support == to specify exact package version. 2021-12-15 12:49:14 +11:00
Dan Halbert dcc61d3532 remove whitespace 2021-12-13 15:44:55 -05:00
Scott Shawcroft d14868d540 Exclude ci checks and tests from building boards 2021-12-13 15:44:55 -05:00
Scott Shawcroft 9f0c0720f9 Fetch tools/adabot submodule for website build
It's used by build_board_info.py
2021-12-13 15:43:36 -05:00
Jeff Epler 1dce9de480
reflect that the default branch of circuitpython-org is updated 2021-12-13 07:58:26 -06:00
Dan Halbert 05e0734157
Merge pull request #5702 from adafruit/tannewt-patch-2
Exclude ci checks and tests from building boards
2021-12-11 23:00:41 -05:00
Dan Halbert c3a90e38f7
remove whitespace 2021-12-11 18:02:55 -05:00
Pierre Constantineau 86ddecfd4f merge main and resolve conflict 2021-12-10 16:57:50 -06:00
Pierre Constantineau dac71e684d moved aliased boards down to actual builds 2021-12-10 16:34:06 -06:00
Scott Shawcroft cf826d33d1
Exclude ci checks and tests from building boards 2021-12-10 10:28:49 -08:00
Scott Shawcroft 74fa2e9edd
Fetch tools/adabot submodule for website build
It's used by build_board_info.py
2021-12-10 10:20:36 -08:00
Damien George 3f589e2f39 tools/autobuild: Automatically build all esp32 boards.
Any board with a board.json file will be built.  ESP32-based boards will be
built using the IDF at $IDF_PATH_V42, all other MCU variants (S2, S3, C3)
will be built using the IDF at $IDF_PATH_V44.

Signed-off-by: Damien George <damien@micropython.org>
2021-12-10 21:29:27 +11:00
Jeff Epler 9ec9a6d500
these duplicates are OK 2021-12-08 20:47:02 -06:00
Damien George 67f66795c0 tools/mpremote: Implement seek and flush in ioctl method.
Fixes issue #8058.

Signed-off-by: Damien George <damien@micropython.org>
2021-12-09 13:35:59 +11:00
Jim Mussared 3770fab334 all: Update Python formatting to latest Black version 21.12b0.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-12-09 12:09:40 +11:00
Dan Halbert 7cbb30806a minor .py changes due to blacken update 2021-12-06 08:36:24 -05:00
Dan Halbert 4a4c5d7ab8 formatting updates for updated black 2021-12-05 21:37:00 -05:00
Dan Halbert 26f23e44c4 don't fetch submodules for circuitpython.org update 2021-11-30 15:49:20 -05:00
Timon a1052d5f73
Initial broadcom port for Raspberry Pi
This targets the 64-bit CPU Raspberry Pis. The BCM2711 on the Pi 4
and the BCM2837 on the Pi 3 and Zero 2W. There are 64-bit fixes
outside of the ports directory for it.

There are a couple other cleanups that were incidental:
* Use const mcu_pin_obj_t instead of omitting the const. The structs
  themselves are const because they are in ROM.
* Use PTR <-> OBJ conversions in more places. They were found when
  mp_obj_t was set to an integer type rather than pointer.
* Optimize submodule checkout because the Pi submodules are heavy
  and unnecessary for the vast majority of builds.

Fixes #4314
2021-11-22 14:54:44 -08:00
Damien George bb7aae557b tools/autobuild: Automatically build all stm32 boards.
Any board with a board.json file will be built.  Additional variants for
certain pyboards will also be built by the explicit build-stm32-extra.sh
script.  Both .dfu and .hex files will be made available.

Also build boards in a sorted order, and don't stop building if a single
board fails.

Signed-off-by: Damien George <damien@micropython.org>
2021-11-22 17:12:16 +11:00
Damien George 90554d03c0 stm32/boards: Build NUCLEO_WB55 and STM32F769DISC without mboot enabled.
This is to make the builds for all nucleo/discovery boards uniform, so they
can be treated the same by the auto build scripts.

The CI script is updated to explicitly enable mboot and packing, to test
these features.

Signed-off-by: Damien George <damien@micropython.org>
2021-11-22 17:12:16 +11:00
Dan Halbert 98bd5dc204 add ru to language_allow_list 2021-11-16 10:45:28 -05:00
iabdalkader 43079aaf86 drivers/ninaw10: Add ublox Nina-W10 WiFi/BT module driver.
- Add WiFi/BT drivers for ublox Nina-W10 (esp32 based) module.
- Add ublox Nina-W10 Python module in extmod.
2021-11-13 23:01:03 +11:00
Jeff Epler 6afe732151
changes in ports/unix don't trigger any board builds 2021-11-12 13:09:33 -06:00
Jeff Epler 2bcfb15283 Add main.c to codeformat 2021-11-08 08:21:43 -06:00
Damien George 1bd47db688 tools/autobuild: Automatically build all mimxrt, rp2 and samd boards.
Any board with a board.json file will be automatically built.

Signed-off-by: Damien George <damien@micropython.org>
2021-11-05 13:50:08 +11:00
Dave Hylands cb99ca9862 tools/dfu.py: Make tool work with python3 when parsing DFU files. 2021-11-01 15:46:59 +11:00
Jim Mussared ab754d5924 tools/autobuild: Add script to generate website board metadata.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-10-27 14:04:53 +11:00
Jeff Epler 30f3cf31e2 account for deleted boards 2021-10-26 14:06:07 -05:00
Damien George 64e4bae129 tools/ci.sh: Use a specific ESP IDF v4.4 commit.
There is no release of IDF v4.4 yet but master is now on v5.0-dev so a
specific commit must be chosen to stick to v4.4.

Signed-off-by: Damien George <damien@micropython.org>
2021-10-25 23:58:44 +11:00
robert-hh dc8be7ccad tools/autobuild: Add the MIMXRT1010_EVK board to autobuild.
Having a board now available for testing, this binary can be provided with
good confidence.
2021-10-25 23:52:23 +11:00
microDev 6c94234b72
allow any character except / in port or board name 2021-10-22 20:08:02 +05:30
Jeff Epler 8d8f83bc05 Merge tag 'v1.17' into merge-1.17
F-strings, new machine.I2S class, ESP32-C3 support and LEGO_HUB_NO6 board

This release of MicroPython adds support for f-strings (PEP-498), with a
few limitations compared to normal Python.  F-strings are essentially
syntactic sugar for "".format() and make formatting strings a lot more
convenient.  Other improvements to the core runtime include pretty printing
OSError when it has two arguments (an errno code and a string), scheduling
of KeyboardInterrupt on the main thread, and support for a single argument
to the optimised form of StopIteration.

In the machine module a new I2S class has been added, with support for
esp32 and stm32 ports.  This provides a consistent API for transmit and
receive of audio data in blocking, non-blocking and asyncio-based
operation.  Also, the json module has support for the "separators" argument
in the dump and dumps functions, and framebuf now includes a way to blit
between frame buffers of different formats using a palette.  A new,
portable machine.bitstream function is also added which can output a stream
of bits with configurable timing, and is used as the basis for driving
WS2812 LEDs in a common way across ports.

There has been some restructuring of the repository directory layout, with
all third-party code now in the lib/ directory.  And a new top-level
directory shared/ has been added with first-party code that was previously
in lib/ moved there.

The docs have seen further improvement with enhancements and additions to
the rp2 parts, as well as a new quick reference for the zephyr port.
The terms master/slave have been replaced with controller/peripheral,
mainly relating to I2C and SPI usage.  And u-module references have been
replaced with just the module name without the u-prefix to help clear up
the intended usage of modules in MicroPython.

For the esp8266 and esp32 ports, hidden networks are now included in WLAN
scan results.  On the esp32 the RMT class is enhanced with idle_level and
write_pulses modes.  There is initial support for ESP32-C3 chips with
GENERIC_C3 and GENERIC_C3_USB boards.

The javascript port has had its Makefile and garbage collector
implementation reworked so it compiles and runs with latest the Emscripten
using asyncify.

The mimxrt port sees the addition of hardware I2C and SPI support, as well
as some additional methods to the machine module.  There is also support
for Hyperflash chips.

The nrf port now has full VFS storage support, enables source-line on
traceback, and has .mpy features consistent with other ports.

For the rp2 port there is now more configurability for boards, and more
boards added.

The stm32 port has a new LEGO_HUB_NO6 board definition with detailed
information how to get this LEGO Hub running stock MicroPython.  There is
also now support to change the CPU frequency on STM32WB MCUs.  And USBD_xxx
descriptor options have been renamed to MICROPY_HW_USB_xxx.

Thanks to everyone who contributed to this release: Amir Gonnen, Andrew
Scheller, Bryan Tong Minh, Chris Wilson, Damien George, Daniel Mizyrycki,
David Lechner, David P, Fernando, finefoot, Frank Pilhofer, Glenn Ruben
Bakke, iabdalkader, Jeff Epler, Jim Mussared, Jonathan Hogg, Josh Klar,
Josh Lloyd, Julia Hathaway, Krzysztof Adamski, Matúš Olekšák, Michael
Weiss, Michel Bouwmans, Mike Causer, Mike Teachman, Ned Konz, NitiKaur,
oclyke, Patrick Van Oosterwijck, Peter Hinch, Peter Züger, Philipp
Ebensberger, robert-hh, Roberto Colistete Jr, Sashkoiv, Seon Rozenblum,
Tobias Thyrrestrup, Tom McDermott, Will Sowerbutts, Yonatan Goldschmidt.

What follows is a detailed list of changes, generated from the git commit
history, and organised into sections.

Main components
===============

all:
- fix signed shifts and NULL access errors from -fsanitize=undefined
- update to point to files in new shared/ directory

py core:
- mpstate: make exceptions thread-local
- mpstate: schedule KeyboardInterrupt on main thread
- mperrno: add MP_ECANCELED error code
- makeqstrdefs.py: don't include .h files explicitly in preprocessing
- mark unused arguments from bytecode decoding macros
- objexcept: pretty print OSError also when it has 2 arguments
- makeversionhdr: add --tags arg to git describe
- vm: simplify handling of MP_OBJ_STOP_ITERATION in yield-from opcode
- objexcept: make mp_obj_exception_get_value support subclassed excs
- support single argument to optimised MP_OBJ_STOP_ITERATION
- introduce and use mp_raise_type_arg helper
- modsys: optimise sys.exit for code size by using exception helpers
- objexcept: make mp_obj_new_exception_arg1 inline
- obj: fix formatting of comment for mp_obj_is_integer
- emitnative: reuse need_reg_all func in need_stack_settled
- emitnative: ensure stack settling is safe mid-branch
- runtime: fix bool unary op for subclasses of native types
- builtinimport: fix condition for including do_execute_raw_code()
- mkrules: automatically build mpy-cross if it doesn't exist
- implement partial PEP-498 (f-string) support
- lexer: clear fstring_args vstr on lexer free
- mkrules.mk: do submodule sync in "make submodules"

extmod:
- btstack: add missing call to mp_bluetooth_hci_uart_deinit
- btstack: check that BLE is active before performing operations
- uasyncio: get addr and bind server socket before creating task
- axtls-include: add axtls_os_port.h to customise axTLS
- update for move of crypto-algorithms, re1.5, uzlib to lib
- moduselect: conditionally compile select()
- nimble: fix leak in l2cap_send if send-while-stalled
- btstack/btstack.mk: use -Wno-implicit-fallthrough, not =0
- utime: always invoke mp_hal_delay_ms when >= to 0ms
- modbluetooth: clamp MTU values to 32->UINT16_MAX
- nimble: allow modbluetooth binding to hook "sent HCI packet"
- nimble: add "memory stalling" mechanism for l2cap_send
- uasyncio: in open_connection use address info in socket creation
- modujson: add support for dump/dumps separators keyword-argument
- modlwip: fix close and clean up of UDP and raw sockets
- modbluetooth: add send_update arg to gatts_write
- add machine.bitstream
- modframebuf: enable blit between different formats via a palette

lib:
- tinyusb: update to version 0.10.1
- pico-sdk: update to version 1.2.0
- utils/stdout_helpers: make mp_hal_stdout_tx_strn_cooked efficient
- axtls: switch to repo at micropython/axtls
- axtls: update to latest axtls 2.1.5 wih additional commits
- re1.5: move re1.5 code from extmod to lib
- uzlib: move uzlib code from extmod to lib
- crypto-algorithms: move crypto-algorithms code from extmod to lib
- update README's based on contents of these dirs

drivers:
- neopixel: add common machine.bitstream-based neopixel module
- neopixel: optimize fill() for speed
- neopixel: reduce code size of driver
- cyw43: fix cyw43_deinit so it can be called many times in a row
- cyw43: make wifi join fail if interface is not active

mpy-cross:
- disable stack check when building with Emscripten

Support components
==================

docs:
- library: document new esp32.RMT features and fix wait_done
- library: warn that ustruct doesn't handle spaces in format strings
- esp8266/tutorial: change flash mode from dio to dout
- replace master/slave with controller/peripheral in I2C and SPI
- rp2: enhance quickref and change image to Pico pinout
- rp2: update general section to give a brief technical overview
- library/utime.rst: clarify behaviour and precision of sleep ms/us
- library/uasyncio.rst: document stream readexactly() method
- library/machine.I2S.rst: fix use of sd pin in examples
- zephyr: add quick reference for the Zephyr port
- library/zephyr: add libraries specific to the Zephyr port
- templates: add unix and zephyr quickref links to top-index
- rename ufoo.rst to foo.rst
- replace ufoo with foo in all docs
- library/index.rst: clarify module naming and purpose
- library/builtins.rst: add module title
- library/network.rst: simplify socket import
- add docs for machine.bitstream and neopixel module
- library: fix usage of :term: for frozen module reference
- esp8266: use monospace for software tools
- reference: mention that slicing a memoryview causes allocation

examples: no changes specific to this component/port

tests:
- extmod: make uasyncio_heaplock test more deterministic
- cpydiff/modules_struct_whitespace_in_format: run black
- extmod/ujson: add tests for dump/dumps separators argument
- run-multitests.py: add broadcast and wait facility
- multi_bluetooth/ble_subscribe.py: add test for subscription
- extmod/vfs_fat_finaliser.py: ensure alloc at never-used GC blocks
- basics: split f-string debug printing to separate file with .exp
- pybnative: make while.py test run on boards without pyb.delay

tools:
- autobuild: add scripts to build release firmware
- remove obsolete build-stm-latest.sh script
- ci.sh: run apt-get update in ci_powerpc_setup
- makemanifest.py: allow passing flags to mpy-tool.py
- autobuild: add mimxrt port to build scripts for nightly builds
- pyboard.py: add cmd-line option to make soft reset configurable
- mpremote: swap order of PID and VID in connect-list output
- ci.sh: build unix dev variant as part of macOS CI
- ci.sh: build GENERIC_C3 board as part of esp32 CI
- autobuild: use separate IDF version to build newer esp32 SoCs
- autobuild: add FeatherS2 and TinyS2 to esp32 auto builds
- mpremote: add seek whence for mounted files
- mpremote: raise OSError on unsupported RemoteFile.seek
- autobuild: add the MIMXRT1050_EVKB board to the daily builds
- ci.sh: add mpy-cross build to nrf port
- codeformat.py: include ports/nrf/modules/nrf in code formatting
- gen-cpydiff.py: don't rename foo to ufoo in diff output
- autobuild: add auto build for Silicognition wESP32
- mpremote: fix connect-list in case VID/PID are None
- mpremote: add "devs" shortcut for "connect list"
- mpremote: remove support for pyb.USB_VCP in/out specialisation
- autobuild: don't use "-B" for make, it's already a fresh build
- pyboard.py: move --no-exclusive/--soft-reset out of mutex group
- pyboard.py: make --no-follow use same variable as --follow
- pyboard.py: add --exclusive to match --no-exclusive
- pyboard.py: make --no-soft-reset consistent with other args
- uncrustify: force 1 newline at end of file
- mpremote: bump version to 0.0.6

CI:
- workflows: add workflow to build and test javascript port
- workflows: switch from Coveralls to Codecov
- workflows: switch from lcov to gcov
- workflows: add workflow to build and test unix dev variant

The ports
=========

all ports:
- use common mp_hal_stdout_tx_strn_cooked instead of custom one
- update for move of crypto-algorithms, uzlib to lib
- rename USBD_VID/PID config macros to MICROPY_HW_USB_VID/PID

bare-arm port: no changes specific to this component/port

cc3200 port: no changes specific to this component/port

esp8266 port:
- add __len__ to NeoPixel driver to support iterating
- Makefile: add more libm files to build
- include hidden networks in WLAN.scan results
- replace esp.neopixel with machine.bitstream
- remove dead code for end_ticks in machine_bitstream

esp32 port:
- boards/sdkconfig.base: disable MEMPROT_FEATURE to alloc from IRAM
- add __len__ to NeoPixel driver to support iterating
- main: allow MICROPY_DIR to be overridden
- esp32_rmt: fix RMT looping in newer IDF versions
- esp32_rmt: enhance RMT with idle_level and write_pulses modes
- add new machine.I2S class for I2S protocol support
- machine_spi: calculate actual attained baudrate
- machine_hw_spi: use a 2 item SPI queue for long transfers
- machine_dac: add MICROPY_PY_MACHINE_DAC option, enable by default
- machine_i2s: add MICROPY_PY_MACHINE_I2S option, enable by default
- fix use of mp_int_t, size_t and uintptr_t
- add initial support for ESP32C3 SoCs
- boards/GENERIC_C3: add generic C3-based board
- modmachine: release the GIL in machine.idle()
- mphalport: always yield at least once in delay_ms
- machine_uart: add flow kw-arg to enable hardware flow control
- boards: add Silicognition wESP32 board configuration
- mpconfigport.h: enable reverse and inplace special methods
- include hidden networks in WLAN.scan results
- makeimg.py: get bootloader and partition offset from sdkconfig
- enable MICROPY_PY_FSTRINGS by default
- machine_hw_spi: release GIL during transfers
- machine_pin: make check for non-output pins respect chip variant
- replace esp.neopixel with machine.bitstream
- remove dead code for end_ticks in machine_bitstream
- boards: add GENERIC_C3_USB board with USB serial/JTAG support

javascript port:
- rework Makefile and GC so it works with latest Emscripten
- Makefile: suppress compiler errors from array bounds
- Makefile: change variable to EXPORTED_RUNTIME_METHODS

mimxrt port:
- move calc_weekday helper function to timeutils
- machine_spi: add the SPI class to the machine module
- moduos: seed the PRNG on boot using the TRNG
- boards: set vfs partition start to 1 MBbyte
- main: skip running main.py if boot.py failed
- main: extend the information returned by help()
- mimxrt_flash: remove commented-out code
- modmachine: add a few minor methods to the machine module
- machine_led: use mp_raise_msg_varg helper
- machine_i2c: add hardware-based machine.I2C to machine module
- add support for Hyperflash chips
- boards: add support for the MIMXRT1050_EVKB board
- machine_pin: implement ioctl for Pin

minimal port:
- Makefile: add support for building with user C modules

nrf port:
- modules: replace master/slave with controller/peripheral in SPI
- boards/common.ld: calculate unused flash region
- modules/nrf: add new nrf module with flash block device
- drivers: add support for using flash block device with SoftDevice
- mpconfigport.h: expose nrf module when MICROPY_PY_NRF is set
- README: update README.md to reflect internal file systems
- mpconfigport.h: tune FAT FS configuration
- Makefile: add _fs_size linker script override from make
- modules/uos: allow a board to configure MICROPY_VFS_FAT/LFS1/LFS2
- mpconfigport.h: enable MICROPY_PY_IO_FILEIO when an FS is enabled
- qstrdefsport.h: add entries for in-built FS mount points
- main: add auto mount and auto format hook for internal flash FS
- boards: enable needed features for FAT/LFS1/LFS2
- facilitate use of freeze manifest
- boards: set FROZEN_MANIFEST blank when SD present on nrf51 targets
- modules/scripts: add file system formatting script
- Makefile: set default manifest file for all targets
- mphalport: add dummy function for mp_hal_time_ns()
- boards: enable MICROPY_VFS_LFS2 for all target boards
- modules/uos: add ilistdir to uos module
- modules/nrf: add function to enable/disable DCDC
- enable source line on tracebacks
- set .mpy features consistent with documentation and other ports

pic16bit port: no changes specific to this component/port

powerpc port: no changes specific to this component/port

qemu-arm port: no changes specific to this component/port

rp2 port:
- use 0=Monday datetime convention in RTC
- machine_rtc: in RTC.datetime, compute weekday automatically
- CMakeLists.txt: suppress compiler errors for pico-sdk and tinyusb
- tusb_config.h: set CFG_TUD_CDC_EP_BUFSIZE to 256
- machine_uart: add hardware flow control support
- machine_uart: allow overriding default machine UART pins
- machine_i2c: allow boards to configure I2C pins using new macros
- machine_spi: allow boards to configure SPI pins using new macros
- machine_uart: fix poll ioctl to also check hardware FIFO
- machine_uart: fix read when FIFO has chars but ringbuf doesn't
- tusb_port: allow boards to configure USB VID and PID
- boards/ADAFRUIT_FEATHER_RP2040: configure custom VID/PID
- boards/ADAFRUIT_FEATHER_RP2040: configure I2C/SPI default pins
- boards/SPARKFUN_PROMICRO: configure UART/I2C/SPI default pins
- boards/SPARKFUN_THINGPLUS: configure I2C/SPI default pins
- boards: add Adafruit ItsyBitsy RP2040
- boards: add Adafruit QT Py RP2040
- boards: add Pimoroni Pico LiPo 4MB
- boards: add Pimoroni Pico LiPo 16MB
- boards: add Pimoroni Tiny 2040
- CMakeLists.txt: allow a board's cmake to set the manifest path
- enable MICROPY_PY_FSTRINGS by default
- Makefile: add "submodules" target, to match other ports
- rp2_flash: disable IRQs while calling flash_erase/program
- CMakeLists.txt: add option to enable double tap reset to bootrom
- mpconfigport.h: allow boards to add root pointers

samd port:
- add support for building with user C modules

stm32 port:
- softtimer: add soft_timer_reinsert() helper function
- mpbthciport: change from systick to soft-timer for BT scheduling
- provide a custom BTstack runloop that integrates with soft timer
- usb: make irq's default trigger enable all events
- boardctrl: skip running main.py if boot.py had an error
- sdio: fix undefined reference to DMA stream on H7
- dma: add DMAMUX configuration for H7 to fix dma_nohal_init
- main: call mp_deinit() at end of main
- adc: allow using ADC12 and ADC3 for H7
- adc: define the ADC instance used for internal channels
- adc: simplify and generalise how pin_adcX table is defined
- add new machine.I2S class for I2S protocol support
- boards/NUCLEO_F446RE: fix I2C1 pin assignment to match datasheet
- replace master/slave with controller/peripheral in I2C and SPI
- systick: always POLL_HOOK when delaying for milliseconds
- sdram: make SDRAM test cache aware, and optional failure with msg
- boards/NUCLEO_F446RE: enable CAN bus support
- boards: add support for SparkFun STM32 MicroMod Processor board
- uart: fix LPUART1 baudrate set/get
- uart: support low baudrates on LPUART1
- boards/STM32F429DISC: set correct UART2 pins and add UART3/6
- boards/NUCLEO_F439ZI: add board definition for NUCLEO_F439ZI
- boards/LEGO_HUB_NO6: add board definition for LEGO_HUB_NO6
- Makefile: update to only pull in used Bluetooth library
- README.md: update supported MCUs, and submodule and mboot use
- usbd_desc: rename USBD_xxx descriptor opts to MICROPY_HW_USB_xxx
- usbd_cdc_interface: rename USBD_CDC_xx opts to MICROPY_HW_USB_xx
- powerctrl: support changing frequency on WB MCUs
- boards/NUCLEO_H743ZI2: add modified version of NUCLEO_H743ZI
- mbedtls: fix compile warning about uninitialized val
- enable MICROPY_PY_FSTRINGS by default
- add implementation of machine.bitstream
- Makefile: allow GIT_SUBMODULES and LIBS to be extended
- stm32_it: support TIM17 IRQs on WB MCUs
- disable computed goto on constrained boards
- storage: make extended-block-device more configurable
- boards/LEGO_HUB_NO6: change SPI flash storage to use hardware SPI
- boards/LEGO_HUB_NO6: skip first 1MiB of SPI flash for storage
- boards/LEGO_HUB_NO6: add make commands to backup/restore firmware

teensy port: no changes specific to this component/port

unix port:
- modffi: add option to lock GC in callback, and cfun access
- Makefile: add back LIB_SRC_C to list of object files
- variants: enable help and help("modules") on standard and dev
- Makefile: disable error compression on arm-linux-gnueabi-gcc

windows port:
- Makefile: add .exe extension to executables name
- appveyor: update to VS 2017 and use Python 3.8 for build/test

zephyr port:
- machine_spi: add support for hardware SPI
2021-10-15 08:20:54 -05:00
Scott Shawcroft 04459f5e2a
Merge branch 'main' into ai_thinker_esp32-c3s 2021-10-05 14:28:17 -07:00
Bruce Segal 01d7861fd9 Only build .bin in CI 2021-09-29 20:33:50 -07:00
microDev 317751bd79
add board microdev_micro_c3 2021-09-29 00:14:00 +05:30
Scott Shawcroft 276d425899
Merge pull request #5302 from maholli/pycubed_v05c
updating pycubed firmware for mainboardv05
2021-09-20 15:00:25 -07:00
Max Holliday f16a4754ba changed boot counter #ifdef to safer #if 2021-09-17 14:40:16 -06:00
microDev 87dc84af3c
more ci docs build changes
- some refactoring
- add doc build tools to search pattern
- build all if trigger event is not a pr
2021-09-17 19:13:03 +05:30
Damien George da4593f937 tools/ci.sh: Use IDF v4.4 as part of esp32 CI and build GENERIC_S3.
IDF v4.4 does not have an official release so for now use the latest
master.  Also remove building GENERIC with no options (all the other boards
are no-option builds), to keep CI time reasonable.

Signed-off-by: Damien George <damien@micropython.org>
2021-09-16 22:59:05 +10:00
Jim Mussared b326edf68c all: Remove MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE.
This commit removes all parts of code associated with the existing
MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE optimisation option, including the
-mcache-lookup-bc option to mpy-cross.

This feature originally provided a significant performance boost for Unix,
but wasn't able to be enabled for MCU targets (due to frozen bytecode), and
added significant extra complexity to generating and distributing .mpy
files.

The equivalent performance gain is now provided by the combination of
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE (which has
been enabled on the unix port in the previous commit).

It's hard to provide precise performance numbers, but tests have been run
on a wide variety of architectures (x86-64, ARM Cortex, Aarch64, RISC-V,
xtensa) and they all generally agree on the qualitative improvements seen
by the combination of MICROPY_OPT_LOAD_ATTR_FAST_PATH and
MICROPY_OPT_MAP_LOOKUP_CACHE.

For example, on a "quiet" Linux x64 environment (i3-5010U @ 2.10GHz) the
change from CACHE_MAP_LOOKUP_IN_BYTECODE, to LOAD_ATTR_FAST_PATH combined
with MAP_LOOKUP_CACHE is:

diff of scores (higher is better)
N=2000 M=2000       bccache -> attrmapcache      diff      diff% (error%)
bm_chaos.py        13742.56 ->   13905.67 :   +163.11 =  +1.187% (+/-3.75%)
bm_fannkuch.py        60.13 ->      61.34 :     +1.21 =  +2.012% (+/-2.11%)
bm_fft.py         113083.20 ->  114793.68 :  +1710.48 =  +1.513% (+/-1.57%)
bm_float.py       256552.80 ->  243908.29 : -12644.51 =  -4.929% (+/-1.90%)
bm_hexiom.py         521.93 ->     625.41 :   +103.48 = +19.826% (+/-0.40%)
bm_nqueens.py     197544.25 ->  217713.12 : +20168.87 = +10.210% (+/-3.01%)
bm_pidigits.py      8072.98 ->    8198.75 :   +125.77 =  +1.558% (+/-3.22%)
misc_aes.py        17283.45 ->   16480.52 :   -802.93 =  -4.646% (+/-0.82%)
misc_mandel.py     99083.99 ->  128939.84 : +29855.85 = +30.132% (+/-5.88%)
misc_pystone.py    83860.10 ->   82592.56 :  -1267.54 =  -1.511% (+/-2.27%)
misc_raytrace.py   21490.40 ->   22227.23 :   +736.83 =  +3.429% (+/-1.88%)

This shows that the new optimisations are at least as good as the existing
inline-bytecode-caching, and are sometimes much better (because the new
ones apply caching to a wider variety of map lookups).

The new optimisations can also benefit code generated by the native
emitter, because they apply to the runtime rather than the generated code.
The improvement for the native emitter when LOAD_ATTR_FAST_PATH and
MAP_LOOKUP_CACHE are enabled is (same Linux environment as above):

diff of scores (higher is better)
N=2000 M=2000        native -> nat-attrmapcache  diff      diff% (error%)
bm_chaos.py        14130.62 ->   15464.68 :  +1334.06 =  +9.441% (+/-7.11%)
bm_fannkuch.py        74.96 ->      76.16 :     +1.20 =  +1.601% (+/-1.80%)
bm_fft.py         166682.99 ->  168221.86 :  +1538.87 =  +0.923% (+/-4.20%)
bm_float.py       233415.23 ->  265524.90 : +32109.67 = +13.756% (+/-2.57%)
bm_hexiom.py         628.59 ->     734.17 :   +105.58 = +16.796% (+/-1.39%)
bm_nqueens.py     225418.44 ->  232926.45 :  +7508.01 =  +3.331% (+/-3.10%)
bm_pidigits.py      6322.00 ->    6379.52 :    +57.52 =  +0.910% (+/-5.62%)
misc_aes.py        20670.10 ->   27223.18 :  +6553.08 = +31.703% (+/-1.56%)
misc_mandel.py    138221.11 ->  152014.01 : +13792.90 =  +9.979% (+/-2.46%)
misc_pystone.py    85032.14 ->  105681.44 : +20649.30 = +24.284% (+/-2.25%)
misc_raytrace.py   19800.01 ->   23350.73 :  +3550.72 = +17.933% (+/-2.79%)

In summary, compared to MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE, the new
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE options:
- are simpler;
- take less code size;
- are faster (generally);
- work with code generated by the native emitter;
- can be used on embedded targets with a small and constant RAM overhead;
- allow the same .mpy bytecode to run on all targets.

See #7680 for further discussion.  And see also #7653 for a discussion
about simplifying mpy-cross options.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-09-16 16:04:03 +10:00
Max Holliday 4f10a91f85 moving boot counter to main() and re-adding old pin names for SD_CS 2021-09-15 14:33:54 -06:00
Max Holliday 5b69aa0a58
Merge branch 'adafruit:main' into pycubed_v05c 2021-09-15 14:23:56 -06:00
microDev e91fb247a3
build docs dynamically 2021-09-16 00:05:42 +05:30
Scott Shawcroft 6d43dd0dca
Try and fix build 2021-09-14 11:10:55 -07:00
Philipp Ebensberger 0d7366c912 mimxrt: Rework flash configuration.
- Moves definition of BOARD_FLASH_SIZE and other header files related to
	flash configuration into the Makefile.
- Adds board specific clock_config.h.
- Adds board.h, pin_mux.h, and peripherals.h as they are
	required by NXP MCU SDK in order to use our own clock_config.h.
- Renames board specific FlexSPI configuration files.
- Updates flash frequency of MIMXRT1020_EVK
- Creates separated flash_config files for QSPI NOR and
	QSPI Hyper flash.
- Unifies VFS start address to be @ 1M for 1010 and 1020 boards.
- Unifies 1050EVK boards
- Adds support to both NOR and HyperFlash on boards with
	both capabilities.
- Adds automatic FlexRAM initialization to start-up code based on
	linker script and NXP HAL.
- Applies code formatting to all files in mimxrt port.

With this change the flash configuration is restructured and
organized. This simplifies the configuration process and
provides a better overview of each board's settings. With the integration
of clock_config.h, board.h, pin_mux.h, and peripherals.h we gain better
control of the settings and clock configurations. Furthermore the
implementation of an explicit FlexRAM setup improves the system
performance and allows for performance tuning.

Signed-off-by: Philipp Ebensberger
2021-09-14 13:52:52 +02:00
Scott Shawcroft 973a90f2aa
Rename esp32s2 port to espressif
This is in preparation for ESP32-S3 support and potentially others.

Related to #4363
2021-09-13 16:44:55 -07:00
Max Holliday 34c53c101a fixing ducplicate PID for pycubed boards 2021-09-12 20:15:43 -06:00
Patrick Van Oosterwijck 0ec5052f62 tools/autobuild: Add auto build for GENERIC_C3_USB. 2021-09-10 15:41:52 +10:00
Seon Rozenblum 9a7f77bfbc esp32/boards: Add new FeatherS2-Neo board definition. 2021-09-10 15:40:32 +10:00
Scott Shawcroft a874043e1b
Selectively build boards based on changed files
This uses a step output from the test job to set the build matrix
for the board build jobs. The built boards depends on which files
were changed.

* Changes contained within ports/*/boards/ will build each board.
* Changes contained within ports/* will build all boards from the
  port.
* All other file changes will build all boards.

All boards will be build for pushes in `adafruit/circuitpython` as
well.

A side-effect is that we no longer need to explicitly list the
boards to build. It will automatically update as new folders are
added.

Related to #4009
2021-09-07 14:59:45 -07:00
sommersoft 37007b4158 Revert "black formatted 'build_board_info.py'"
Originally ran black with the incorrect line length

This reverts commit cb586a4018.
2021-09-05 18:52:37 -05:00
sommersoft cb586a4018 black formatted 'build_board_info.py' 2021-09-05 18:49:55 -05:00
sommersoft 791937b149 black formatted 'build_board_info.py' 2021-09-05 18:42:05 -05:00
sommersoft 10706e9376 add frozen libraries to the board info 2021-09-05 18:40:58 -05:00
Damien George 89145c6aad tools/mpremote: Bump version to 0.0.6.
Signed-off-by: Damien George <damien@micropython.org>
2021-09-01 00:49:16 +10:00
James Carr 3e543264a5
Convert all_characters to a list before using extend() 2021-08-31 15:26:28 +01:00
David Lechner 86371781e9 tools/uncrustify: Force 1 newline at end of file.
To keep things neat and tidy, we ensure that each file has 1 and only 1
newline at the end of each file.

Signed-off-by: David Lechner <david@pybricks.com>
2021-08-31 13:14:45 +10:00
Damien George 8c4ba575fd tests/basics: Split f-string debug printing to separate file with .exp.
This feature {x=} was introduced in Python 3.8 so needs a separate .exp
file to run on earlier Python versions.

See https://bugs.python.org/issue36817

Signed-off-by: Damien George <damien@micropython.org>
2021-08-26 23:56:02 +10:00
Jim Mussared 145fedef8d tools/pyboard.py: Make --no-soft-reset consistent with other args.
This makes it work like --no-follow and --no-exclusive using a mutex group
and dest.  Although the current implementation with BooleanOptionAction is
neater it requires Python 3.9, so don't use this feature.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-25 15:47:01 +10:00
Jim Mussared 064a145097 tools/pyboard.py: Add --exclusive to match --no-exclusive.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-25 15:46:38 +10:00
Jim Mussared be43164d82 tools/pyboard.py: Make --no-follow use same variable as --follow.
You can set one or the other (or neither) but not both.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-25 15:46:00 +10:00
Jim Mussared 2a290bbfe1 tools/pyboard.py: Move --no-exclusive/--soft-reset out of mutex group.
The --no-exclusive flag was accidentally added to the mutex group in
178198a01d.

The --soft-reset flag was accidentally added to the mutex group in
41adf17830.

These flags can be specified independently to --[no-]follow so should not
be in that mutex group.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-25 15:36:00 +10:00
Damien George ed42002c39 tools/autobuild: Don't use "-B" for make, it's already a fresh build.
And using "-B" means mpy-cross is forcefully rebuilt, sometimes with
invalid CFLAGS_EXTRA options which makes the auto-build fail.

Signed-off-by: Damien George <damien@micropython.org>
2021-08-19 23:30:17 +10:00
Damien George 226c0341ca tools/mpremote: Remove support for pyb.USB_VCP in/out specialisation.
The sys.stdin.buffer and sys.stdout.buffer streams work just as well (and
are just as fast) as pyb.USB_VCP on stm32 devices, so there's no need to
have the USB_VCP specialisation code, which just adds complexity.

Also, on stm32 devices with both USB and UART (or other serial interface),
if something other than the USB_VCP port is used for the serial connection
then mpremote mount will not work because it will default to reading and
writing on USB_VCP instead of the other connected serial stream.

As part of this simplification, support for a second port as input is
removed (this feature was never exposed to the user).

Signed-off-by: Damien George <damien@micropython.org>
2021-08-19 18:56:11 +10:00
Jeff Epler 0553561975
Merge pull request #5078 from rpavlik/patch-1
Update ter-u12n.bdf
2021-08-15 19:57:35 -05:00
Ryan Pavlik 08f43d6766 Fix number of chars in default font to make it valid bdf. 2021-08-15 14:31:58 -05:00
Jim Mussared 692d36d779 py: Implement partial PEP-498 (f-string) support.
This implements (most of) the PEP-498 spec for f-strings and is based on
https://github.com/micropython/micropython/pull/4998 by @klardotsh.

It is implemented in the lexer as a syntax translation to `str.format`:
  f"{a}" --> "{}".format(a)

It also supports:
  f"{a=}" --> "a={}".format(a)

This is done by extracting the arguments into a temporary vstr buffer,
then after the string has been tokenized, the lexer input queue is saved
and the contents of the temporary vstr buffer are injected into the lexer
instead.

There are four main limitations:
- raw f-strings (`fr` or `rf` prefixes) are not supported and will raise
  `SyntaxError: raw f-strings are not supported`.

- literal concatenation of f-strings with adjacent strings will fail
    "{}" f"{a}" --> "{}{}".format(a)    (str.format will incorrectly use
                                         the braces from the non-f-string)
    f"{a}" f"{a}" --> "{}".format(a) "{}".format(a) (cannot concatenate)

- PEP-498 requires the full parser to understand the interpolated
  argument, however because this entirely runs in the lexer it cannot
  resolve nested braces in expressions like
    f"{'}'}"

- The !r, !s, and !a conversions are not supported.

Includes tests and cpydiffs.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-14 16:58:40 +10:00
Damien George 162bf3c5d8 tools/mpremote: Add "devs" shortcut for "connect list".
See issue #7480.

Signed-off-by: Damien George <damien@micropython.org>
2021-08-14 00:13:35 +10:00
Damien George 1f48934312 tools/mpremote: Fix connect-list in case VID/PID are None.
Which can be the case on Windows and macOS for certain serial devices.

Fixes issue #7636.

Signed-off-by: Damien George <damien@micropython.org>
2021-08-14 00:13:35 +10:00
Patrick Van Oosterwijck e49f609186 tools/autobuild: Add auto build for Silicognition wESP32. 2021-08-13 23:25:24 +10:00
Jim Mussared ee549d725a tools/gen-cpydiff.py: Don't rename foo to ufoo in diff output.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2021-08-13 23:14:08 +10:00
Jeff Epler a36b930ead check uncrustify version 2021-08-12 14:19:29 -05:00
Dan Halbert 05205259c6 update tools/adabot submodule 2021-08-11 23:14:58 -04:00
Glenn Ruben Bakke 85f0ce214e tools/codeformat.py: Include ports/nrf/modules/nrf in code formatting. 2021-08-08 23:17:55 +10:00
Glenn Ruben Bakke 4f76f66185 tools/ci.sh: Add mpy-cross build to nrf port. 2021-08-08 23:17:55 +10:00
Damien George 96c6b8cae3 ports: Rename USBD_VID/PID config macros to MICROPY_HW_USB_VID/PID.
For consistency with other board-level config macros that begin with
MICROPY_HW_USB.

Also allow boards in the mimxrt, nrf and samd ports to configure these
values.

Signed-off-by: Damien George <damien@micropython.org>
2021-08-07 23:13:55 +10:00
Scott Shawcroft 1e225610cc
Add ability to disable BLE workflow
Call `supervisor.disable_ble_workflow()` and the BLE workflow will
be disabled until the chip is reset.

This also includes a couple fixes:
1. Terminals can now be deinit by setting the tilegrid to NULL. This
  prevents using the tilegrid before display is init.
2. Fix BLE serial send amount when sending more than a single packet.

Fixes #5049
2021-08-02 18:37:19 -07:00
robert-hh 4445c73b11 tools/autobuild: Add the MIMXRT1050_EVKB board to the daily builds. 2021-07-31 16:27:47 +10:00
Tobias Thyrrestrup 60e3e51753 stm32/Makefile: Update to only pull in used Bluetooth library. 2021-07-31 15:11:46 +10:00
Ryan A. Pavlik c54b86dd0c
Update ter-u12n.bdf
Remove duplicate ENDCHAR that made it invalid
2021-07-30 10:41:38 -05:00
Jeff Epler 5e07b7959f check vid/pid: find topdir relative to this script
@dhalbert noticed that if a fork was called `micropython`, the
existing check could fail during CI because it will be cloned into a
directory also called `micropython`.

Instead of hardcoding a range of strings that are OK as top directories,
find the location of the top directory relative to the script.

This will be conflicty after #5069 is merged, but it should be easy to
fix.
2021-07-27 15:20:58 -05:00
Dan Halbert a974a2d3c1 allow micropython as top-level dir name in duplicate pid/vid check 2021-07-27 09:04:44 -04:00
Michel Bouwmans 92464f11b0 tools/mpremote: Raise OSError on unsupported RemoteFile.seek.
Signed-off-by: Michel Bouwmans <m.bouwmans@ep-games.eu>
2021-07-23 13:21:53 +10:00
Michel Bouwmans 7870ec0370 tools/mpremote: Add seek whence for mounted files.
Fixes issue #7534.

Signed-off-by: Michel Bouwmans <m.bouwmans@ep-games.eu>
2021-07-23 13:21:42 +10:00
Seon Rozenblum 25159129dd tools/autobuild: Add FeatherS2 and TinyS2 to esp32 auto builds. 2021-07-22 22:42:37 +10:00
Damien George 0256e1ab8b tools/autobuild: Use separate IDF version to build newer esp32 SoCs.
Signed-off-by: Damien George <damien@micropython.org>
2021-07-22 22:42:37 +10:00
Jeff Epler 755caddc89 extract_py: Import Dict if needed 2021-07-20 19:49:25 -05:00
Damien George 9f71a11d3f tools/ci.sh: Build GENERIC_C3 board as part of esp32 CI.
Signed-off-by: Damien George <damien@micropython.org>
2021-07-18 23:58:24 +10:00
Damien George cbc8d5b61f tools/ci.sh: Build unix dev variant as part of macOS CI.
To test BTstack build on macOS.

Signed-off-by: Damien George <damien@micropython.org>
2021-07-17 23:55:25 +10:00
Scott Shawcroft c16f559574
Merge pull request #4907 from jepler/refine-stubs
refine stubs-building procedure
2021-07-15 12:11:58 -07:00
Scott Shawcroft 0fadf028ef
Create first BLE-only board, Micro:Bit v2
This fixes build issues with USB off, tweaks the README to allow
for BLE-only boards and adds the Micro:Bit v2 definition.

Fixes #4546
2021-07-14 09:55:51 -07:00
Damien George 0ee256b8b1 github/workflows: Add workflow to build and test unix dev variant.
Signed-off-by: Damien George <damien@micropython.org>
2021-07-13 15:58:39 +10:00
Damien George 136369d72f all: Update to point to files in new shared/ directory.
Signed-off-by: Damien George <damien@micropython.org>
2021-07-12 17:08:10 +10:00
Jeff Epler fd4a7fce91 add tool to parse out firmware sizes for comparison 2021-07-10 19:11:47 -05:00
Jeff Epler 3191357807 Merge remote-tracking branch 'origin/main' into refine-stubs 2021-07-08 19:05:26 -05:00
Andrew Scheller d1decdfa93 tools/mpremote: Swap order of PID and VID in connect-list output.
Fixes issue #7481.
2021-07-02 23:27:07 +10:00
Frank Pilhofer 41adf17830 tools/pyboard.py: Add cmd-line option to make soft reset configurable.
Leaves the default as-is, but allows using --no-soft-reset to disable the
soft reset when connecting.
2021-07-01 12:37:08 +10:00
robert-hh efa97beb99 tools/autobuild: Add mimxrt port to build scripts for nightly builds.
The firmware for Teensy 4.0, Teensy 4.1 and MIMXRT1020_EVK are created.
Users of other MIMXRT10xx_EVK boards should be able to build the firmware
themselves, they might need specific DEBUG settings.

The Makefile had to be changed in order to build the .bin file as well.
2021-07-01 12:29:16 +10:00
Yonatan Goldschmidt 4ada56d4cb tools/makemanifest.py: Allow passing flags to mpy-tool.py. 2021-06-28 01:50:00 +03:00
Damien George f3e4ed82a1 github/workflows: Switch from lcov to gcov.
Coverage calculated by Codecov has the same reliability/deterministic
issues as Coveralls did, so the problem is likely to do with the output of
lcov/gcov, rather than the analysis and display of the data.

Switch from lcov to gcov for data generation to try and simplify this
process of computing coverage.

Signed-off-by: Damien George <damien@micropython.org>
2021-06-26 21:34:37 +10:00
Damien George 2dc4f843bc github/workflows: Add workflow to build and test javascript port.
Signed-off-by: Damien George <damien@micropython.org>
2021-06-25 11:31:00 +10:00
David Lechner 8cebd56a11 tools/ci.sh: Run apt-get update in ci_powerpc_setup.
This fixes failing builds when the GitHub CI image lags behind Ubuntu
security updates.
2021-06-25 10:34:45 +10:00
Damien George c99e4995fd tools: Remove obsolete build-stm-latest.sh script.
The tools/autobuild/ scripts replace this.

Signed-off-by: Damien George <damien@micropython.org>
2021-06-23 16:59:20 +10:00
Damien George 3588d14ae6 tools/autobuild: Add scripts to build release firmware.
Signed-off-by: Damien George <damien@micropython.org>
2021-06-23 16:59:20 +10:00
Jeff Epler f51ca53553 refine stubs-building procedure
* so that excess files are not included in sdist, perform build down
   in circuitpython-stubs
 * This means we need to
   * Remove the need-pypi check
   * Copy a setup.py, README, and MANIFEST.in into the stubs build location
   * Revamp how the overall `mypy --strict` check lists its inputs
 * Add a new test that actually installing the stubs lets us do type
   checking (tools/test-stubs.sh)
 * Add a missing return type to a __init__ function (why was this not
   an error under `mypy --strict`, I wonder)
2021-06-22 08:59:05 -05:00
Jeff Epler b1785eb3f1 finish copyright-notice suggestion 2021-06-21 19:52:08 -05:00
Jeff Epler 680ac9388f review suggestion: remove files 2021-06-21 19:51:03 -05:00
Jeff Epler 4401d73efd
Update tools/pyboard.py
Co-authored-by: Scott Shawcroft <scott@tannewt.org>
2021-06-21 19:49:56 -05:00
Jeff Epler 87d3740c64 Merge tag 'v1.16' 2021-06-18 10:54:19 -05:00
Damien George d2dcd05adc tools/mpremote: Use signal to capture and handle ctrl-C on Windows.
Now a ctrl-C will not stop mpremote, rather this character will be passed
through to the attached device.

The mpremote version is also increased to 0.0.5.

Signed-off-by: Damien George <damien@micropython.org>
2021-06-15 13:52:31 +10:00
Damien George 77c529e8be tools/mpremote: Use available ports instead of auto-connect list.
Using just the list of available ports, instead of a hard-coded list of
possible ports, means that all ports will be available for auto connection.
And the order that they will be attempted in will match what's printed by
"mpremote connect list" (and will be the same as before, trying ACMx before
USBx).  Auto-connect will also now work on Mac, and will allow all COM
ports on Windows.

Signed-off-by: Damien George <damien@micropython.org>
2021-06-15 13:52:31 +10:00
Jeff Epler 124c7b785a Add kaluga version 1.3
.. this board is nearly the same as the "1.2" version originally
released, but makes a few changes to reduce pin conflicts between the 13-pin
camera header and the bootstrapping pins.  "1.3" was introduced in summer
2020-- check the bottom of your board silk to find out whether you have a 1.2
or a 1.3, because it's not clear whether there is remaining 1.2 stock.

Another vexing fact about the 1.3 is that its LCD can have one of two
chipets, and the chipset used is not marked anywhere!
2021-06-08 13:28:48 -05:00
Damien George 20a8f4f7ec tests/unix: Add ffi test for integer types.
Signed-off-by: Damien George <damien@micropython.org>
2021-06-06 22:52:25 +10:00
Maureen Helm 5cb2ade65b zephyr: Update to Zephyr v2.6.0.
Updates the zephyr port build instructions and CI to use the latest
zephyr release tag.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2021-06-06 20:17:42 +10:00
Damien George b15e1ef5a6 github/workflows: Add workflow to build and run unix port on ARM.
Following on from ef16834887, this adds a
coverage build and running of the test suite on an ARM 32-bit Linux-based
architecture.

Signed-off-by: Damien George <damien@micropython.org>
2021-06-05 11:03:09 +10:00
Damien George 934505ac33 tools/ci.sh: Build mpy-cross as part of ci_mimxrt_build.
Signed-off-by: Damien George <damien@micropython.org>
2021-06-02 00:02:21 +10:00
Damien George a60ad33641 tools/mpremote: Add new CLI utility to interact with remote device.
This has been under development since April 2017.  See #3034 and #6375.

Signed-off-by: Damien George <damien@micropython.org>
2021-05-29 17:17:22 +10:00
Damien George e4ba57c5cd tools/pyboard.py: Add "soft_reset" option to Pyboard.enter_raw_repl().
Signed-off-by: Damien George <damien@micropython.org>
2021-05-29 17:17:22 +10:00
Damien George 4982d0920e tools/pyboard.py: Track raw REPL state via in_raw_repl variable.
Signed-off-by: Damien George <damien@micropython.org>
2021-05-29 17:17:22 +10:00
Damien George ef16834887 github/workflows: Add workflow to build and run unix port on MIPS.
This adds a coverage build and running of the test suite on a MIPS 32-bit
big endian architecture.  It uses the feature of qemu to execute foreign
code as though it were native to the system (using qemu user mode).  The
code compiled for MIPS will run under the qemu VM, but all syscalls made by
this code go to the host (Linux) system.

See related #7268 and #7273.

Signed-off-by: Damien George <damien@micropython.org>
2021-05-26 16:33:18 +10:00
Damien George e7c0a8bca3 tools/ci.sh: Build Cortex-A9 sabrelite board as part of qemu-arm CI.
Signed-off-by: Damien George <damien@micropython.org>
2021-05-26 16:24:00 +10:00
Damien George f5cba77e50 tools/tinytest-codegen.py: Add command-line option to exclude tests.
Signed-off-by: Damien George <damien@micropython.org>
2021-05-26 16:24:00 +10:00
Damien George 2c1a6a237d tools/mpy-tool.py: Support relocating ARMv6 arch.
Signed-off-by: Damien George <damien@micropython.org>
2021-05-26 16:24:00 +10:00
Philipp Ebensberger 5f68f0d08a github/workflows: Add CI workflow for mimxrt port. 2021-05-26 00:12:45 +10:00
Damien George 3b950ed295 tools/ci.sh: Use FROZEN_MANIFEST in an esp32 build to test feature.
This tests that FROZEN_MANIFEST works with cmake (on esp32 at least).

Signed-off-by: Damien George <damien@micropython.org>
2021-05-23 00:01:37 +10:00
Maureen Helm a7a9f2fe89 tools/ci.sh: Update zephyr docker image to v0.17.3.
Updates the zephyr docker image and SDK to the latest versions.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2021-05-21 21:55:46 +10:00
Tobias Thyrrestrup 247d7e2e8e tools/pydfu.py: Remove default VID/PID values.
As the new default behaviour, this allows PyDFU to be used with all
devices, not just the ones matching a specific set of VID/PID values.  But
it's still possible to specify VID/PID if needed to narrow down the
selection of the USB device.

Signed-off-by: Tobias Thyrrestrup <tt@LEGO.com>
2021-05-21 15:39:40 +10:00
Damien George 04927dfaca tools/mpy_ld.py: Support R_X86_64_GOTPCREL reloc for x86-64 arch.
This can be treated by the linker the same as R_X86_64_REX_GOTPCRELX,
according to https://reviews.llvm.org/D18301.

Signed-off-by: Damien George <damien@micropython.org>
2021-05-20 23:34:20 +10:00
Jeff Epler 47947dc23c
Merge pull request #4749 from tannewt/merge_1.15
Merge in MicroPython 1.15
2021-05-12 20:52:11 -05:00
Scott Shawcroft 966d25c6a5
Merge MicroPython v1.15 into CircuitPython 2021-05-12 17:51:42 -07:00
Jeff Epler 35f1da4c1a gen_display_resources: Always treat the sample file as utf-8.
This may help resolve the build problems reported at
#4750.
2021-05-11 21:48:06 -05:00
Scott Shawcroft e02a26453c
Merge MicroPython 1.14 into CircuitPython 2021-05-11 15:07:40 -07:00
Jeff Epler d9ee63b0c1 run updated black 2021-05-08 20:42:52 -05:00
Jeff Epler aca9d5bc40 Fix up all natmod examples
* modframebuf: _mp_framebuf_p_t is not "really" a protocol, but the
   QSTR assignment caused problems when building as a dynamic module
 * modure: str_index_to_ptr is not in the natmod API, disable URE match
   spans when dynamic.  mp_obj_len() is a bugfix, we should throw here
   if the object is not string-like
 * moduzlib: Correct paths to uzlib headers & sources.  this relative
   path (from moduzlib.c to the referenced file) works in all cases,
   the other only worked from ports/PORTNAME.
 * dynruntime: Handle 2-arg m_malloc, assert_native_inited, add a
   micropythonish mp_arg_check_num_mp, fix mp_raise_msg to use dumb
   strings, add mp_raise_arg1
 * nativeglue: ad assert_native_inited
 * translate: MP_ERROR_TEXT evaluates to its argument for DYNRUNTIME
 * mpy-tool: A straggling magic number change
 * mpy_ld: Have to renumber manually after dynruntime change
 * import_mpy_native_gc.py: Update copy of features0 baked into this test
2021-05-08 20:17:16 -05:00
Jeff Epler 026cd5914d
Merge pull request #4725 from tannewt/merge_helper
Add Python helper tool for MicroPython merges.
2021-05-08 11:10:42 -05:00
Jeff Epler 5e06c91244
Merge pull request #4716 from tyomitch/main
s/Circuit Python/CircuitPython/g
2021-05-08 08:47:29 -05:00
Scott Shawcroft 20c4a325a4
Add Python helper tool for MicroPython merges.
It automates ignoring parts of MP we don't use.

Related to #2999
2021-05-07 10:21:35 -07:00