Commit Graph

78 Commits

Author SHA1 Message Date
Timon a1052d5f73
Initial broadcom port for Raspberry Pi
This targets the 64-bit CPU Raspberry Pis. The BCM2711 on the Pi 4
and the BCM2837 on the Pi 3 and Zero 2W. There are 64-bit fixes
outside of the ports directory for it.

There are a couple other cleanups that were incidental:
* Use const mcu_pin_obj_t instead of omitting the const. The structs
  themselves are const because they are in ROM.
* Use PTR <-> OBJ conversions in more places. They were found when
  mp_obj_t was set to an integer type rather than pointer.
* Optimize submodule checkout because the Pi submodules are heavy
  and unnecessary for the vast majority of builds.

Fixes #4314
2021-11-22 14:54:44 -08:00
Jeff Epler 8d8f83bc05 Merge tag 'v1.17' into merge-1.17
F-strings, new machine.I2S class, ESP32-C3 support and LEGO_HUB_NO6 board

This release of MicroPython adds support for f-strings (PEP-498), with a
few limitations compared to normal Python.  F-strings are essentially
syntactic sugar for "".format() and make formatting strings a lot more
convenient.  Other improvements to the core runtime include pretty printing
OSError when it has two arguments (an errno code and a string), scheduling
of KeyboardInterrupt on the main thread, and support for a single argument
to the optimised form of StopIteration.

In the machine module a new I2S class has been added, with support for
esp32 and stm32 ports.  This provides a consistent API for transmit and
receive of audio data in blocking, non-blocking and asyncio-based
operation.  Also, the json module has support for the "separators" argument
in the dump and dumps functions, and framebuf now includes a way to blit
between frame buffers of different formats using a palette.  A new,
portable machine.bitstream function is also added which can output a stream
of bits with configurable timing, and is used as the basis for driving
WS2812 LEDs in a common way across ports.

There has been some restructuring of the repository directory layout, with
all third-party code now in the lib/ directory.  And a new top-level
directory shared/ has been added with first-party code that was previously
in lib/ moved there.

The docs have seen further improvement with enhancements and additions to
the rp2 parts, as well as a new quick reference for the zephyr port.
The terms master/slave have been replaced with controller/peripheral,
mainly relating to I2C and SPI usage.  And u-module references have been
replaced with just the module name without the u-prefix to help clear up
the intended usage of modules in MicroPython.

For the esp8266 and esp32 ports, hidden networks are now included in WLAN
scan results.  On the esp32 the RMT class is enhanced with idle_level and
write_pulses modes.  There is initial support for ESP32-C3 chips with
GENERIC_C3 and GENERIC_C3_USB boards.

The javascript port has had its Makefile and garbage collector
implementation reworked so it compiles and runs with latest the Emscripten
using asyncify.

The mimxrt port sees the addition of hardware I2C and SPI support, as well
as some additional methods to the machine module.  There is also support
for Hyperflash chips.

The nrf port now has full VFS storage support, enables source-line on
traceback, and has .mpy features consistent with other ports.

For the rp2 port there is now more configurability for boards, and more
boards added.

The stm32 port has a new LEGO_HUB_NO6 board definition with detailed
information how to get this LEGO Hub running stock MicroPython.  There is
also now support to change the CPU frequency on STM32WB MCUs.  And USBD_xxx
descriptor options have been renamed to MICROPY_HW_USB_xxx.

Thanks to everyone who contributed to this release: Amir Gonnen, Andrew
Scheller, Bryan Tong Minh, Chris Wilson, Damien George, Daniel Mizyrycki,
David Lechner, David P, Fernando, finefoot, Frank Pilhofer, Glenn Ruben
Bakke, iabdalkader, Jeff Epler, Jim Mussared, Jonathan Hogg, Josh Klar,
Josh Lloyd, Julia Hathaway, Krzysztof Adamski, Matúš Olekšák, Michael
Weiss, Michel Bouwmans, Mike Causer, Mike Teachman, Ned Konz, NitiKaur,
oclyke, Patrick Van Oosterwijck, Peter Hinch, Peter Züger, Philipp
Ebensberger, robert-hh, Roberto Colistete Jr, Sashkoiv, Seon Rozenblum,
Tobias Thyrrestrup, Tom McDermott, Will Sowerbutts, Yonatan Goldschmidt.

What follows is a detailed list of changes, generated from the git commit
history, and organised into sections.

Main components
===============

all:
- fix signed shifts and NULL access errors from -fsanitize=undefined
- update to point to files in new shared/ directory

py core:
- mpstate: make exceptions thread-local
- mpstate: schedule KeyboardInterrupt on main thread
- mperrno: add MP_ECANCELED error code
- makeqstrdefs.py: don't include .h files explicitly in preprocessing
- mark unused arguments from bytecode decoding macros
- objexcept: pretty print OSError also when it has 2 arguments
- makeversionhdr: add --tags arg to git describe
- vm: simplify handling of MP_OBJ_STOP_ITERATION in yield-from opcode
- objexcept: make mp_obj_exception_get_value support subclassed excs
- support single argument to optimised MP_OBJ_STOP_ITERATION
- introduce and use mp_raise_type_arg helper
- modsys: optimise sys.exit for code size by using exception helpers
- objexcept: make mp_obj_new_exception_arg1 inline
- obj: fix formatting of comment for mp_obj_is_integer
- emitnative: reuse need_reg_all func in need_stack_settled
- emitnative: ensure stack settling is safe mid-branch
- runtime: fix bool unary op for subclasses of native types
- builtinimport: fix condition for including do_execute_raw_code()
- mkrules: automatically build mpy-cross if it doesn't exist
- implement partial PEP-498 (f-string) support
- lexer: clear fstring_args vstr on lexer free
- mkrules.mk: do submodule sync in "make submodules"

extmod:
- btstack: add missing call to mp_bluetooth_hci_uart_deinit
- btstack: check that BLE is active before performing operations
- uasyncio: get addr and bind server socket before creating task
- axtls-include: add axtls_os_port.h to customise axTLS
- update for move of crypto-algorithms, re1.5, uzlib to lib
- moduselect: conditionally compile select()
- nimble: fix leak in l2cap_send if send-while-stalled
- btstack/btstack.mk: use -Wno-implicit-fallthrough, not =0
- utime: always invoke mp_hal_delay_ms when >= to 0ms
- modbluetooth: clamp MTU values to 32->UINT16_MAX
- nimble: allow modbluetooth binding to hook "sent HCI packet"
- nimble: add "memory stalling" mechanism for l2cap_send
- uasyncio: in open_connection use address info in socket creation
- modujson: add support for dump/dumps separators keyword-argument
- modlwip: fix close and clean up of UDP and raw sockets
- modbluetooth: add send_update arg to gatts_write
- add machine.bitstream
- modframebuf: enable blit between different formats via a palette

lib:
- tinyusb: update to version 0.10.1
- pico-sdk: update to version 1.2.0
- utils/stdout_helpers: make mp_hal_stdout_tx_strn_cooked efficient
- axtls: switch to repo at micropython/axtls
- axtls: update to latest axtls 2.1.5 wih additional commits
- re1.5: move re1.5 code from extmod to lib
- uzlib: move uzlib code from extmod to lib
- crypto-algorithms: move crypto-algorithms code from extmod to lib
- update README's based on contents of these dirs

drivers:
- neopixel: add common machine.bitstream-based neopixel module
- neopixel: optimize fill() for speed
- neopixel: reduce code size of driver
- cyw43: fix cyw43_deinit so it can be called many times in a row
- cyw43: make wifi join fail if interface is not active

mpy-cross:
- disable stack check when building with Emscripten

Support components
==================

docs:
- library: document new esp32.RMT features and fix wait_done
- library: warn that ustruct doesn't handle spaces in format strings
- esp8266/tutorial: change flash mode from dio to dout
- replace master/slave with controller/peripheral in I2C and SPI
- rp2: enhance quickref and change image to Pico pinout
- rp2: update general section to give a brief technical overview
- library/utime.rst: clarify behaviour and precision of sleep ms/us
- library/uasyncio.rst: document stream readexactly() method
- library/machine.I2S.rst: fix use of sd pin in examples
- zephyr: add quick reference for the Zephyr port
- library/zephyr: add libraries specific to the Zephyr port
- templates: add unix and zephyr quickref links to top-index
- rename ufoo.rst to foo.rst
- replace ufoo with foo in all docs
- library/index.rst: clarify module naming and purpose
- library/builtins.rst: add module title
- library/network.rst: simplify socket import
- add docs for machine.bitstream and neopixel module
- library: fix usage of :term: for frozen module reference
- esp8266: use monospace for software tools
- reference: mention that slicing a memoryview causes allocation

examples: no changes specific to this component/port

tests:
- extmod: make uasyncio_heaplock test more deterministic
- cpydiff/modules_struct_whitespace_in_format: run black
- extmod/ujson: add tests for dump/dumps separators argument
- run-multitests.py: add broadcast and wait facility
- multi_bluetooth/ble_subscribe.py: add test for subscription
- extmod/vfs_fat_finaliser.py: ensure alloc at never-used GC blocks
- basics: split f-string debug printing to separate file with .exp
- pybnative: make while.py test run on boards without pyb.delay

tools:
- autobuild: add scripts to build release firmware
- remove obsolete build-stm-latest.sh script
- ci.sh: run apt-get update in ci_powerpc_setup
- makemanifest.py: allow passing flags to mpy-tool.py
- autobuild: add mimxrt port to build scripts for nightly builds
- pyboard.py: add cmd-line option to make soft reset configurable
- mpremote: swap order of PID and VID in connect-list output
- ci.sh: build unix dev variant as part of macOS CI
- ci.sh: build GENERIC_C3 board as part of esp32 CI
- autobuild: use separate IDF version to build newer esp32 SoCs
- autobuild: add FeatherS2 and TinyS2 to esp32 auto builds
- mpremote: add seek whence for mounted files
- mpremote: raise OSError on unsupported RemoteFile.seek
- autobuild: add the MIMXRT1050_EVKB board to the daily builds
- ci.sh: add mpy-cross build to nrf port
- codeformat.py: include ports/nrf/modules/nrf in code formatting
- gen-cpydiff.py: don't rename foo to ufoo in diff output
- autobuild: add auto build for Silicognition wESP32
- mpremote: fix connect-list in case VID/PID are None
- mpremote: add "devs" shortcut for "connect list"
- mpremote: remove support for pyb.USB_VCP in/out specialisation
- autobuild: don't use "-B" for make, it's already a fresh build
- pyboard.py: move --no-exclusive/--soft-reset out of mutex group
- pyboard.py: make --no-follow use same variable as --follow
- pyboard.py: add --exclusive to match --no-exclusive
- pyboard.py: make --no-soft-reset consistent with other args
- uncrustify: force 1 newline at end of file
- mpremote: bump version to 0.0.6

CI:
- workflows: add workflow to build and test javascript port
- workflows: switch from Coveralls to Codecov
- workflows: switch from lcov to gcov
- workflows: add workflow to build and test unix dev variant

The ports
=========

all ports:
- use common mp_hal_stdout_tx_strn_cooked instead of custom one
- update for move of crypto-algorithms, uzlib to lib
- rename USBD_VID/PID config macros to MICROPY_HW_USB_VID/PID

bare-arm port: no changes specific to this component/port

cc3200 port: no changes specific to this component/port

esp8266 port:
- add __len__ to NeoPixel driver to support iterating
- Makefile: add more libm files to build
- include hidden networks in WLAN.scan results
- replace esp.neopixel with machine.bitstream
- remove dead code for end_ticks in machine_bitstream

esp32 port:
- boards/sdkconfig.base: disable MEMPROT_FEATURE to alloc from IRAM
- add __len__ to NeoPixel driver to support iterating
- main: allow MICROPY_DIR to be overridden
- esp32_rmt: fix RMT looping in newer IDF versions
- esp32_rmt: enhance RMT with idle_level and write_pulses modes
- add new machine.I2S class for I2S protocol support
- machine_spi: calculate actual attained baudrate
- machine_hw_spi: use a 2 item SPI queue for long transfers
- machine_dac: add MICROPY_PY_MACHINE_DAC option, enable by default
- machine_i2s: add MICROPY_PY_MACHINE_I2S option, enable by default
- fix use of mp_int_t, size_t and uintptr_t
- add initial support for ESP32C3 SoCs
- boards/GENERIC_C3: add generic C3-based board
- modmachine: release the GIL in machine.idle()
- mphalport: always yield at least once in delay_ms
- machine_uart: add flow kw-arg to enable hardware flow control
- boards: add Silicognition wESP32 board configuration
- mpconfigport.h: enable reverse and inplace special methods
- include hidden networks in WLAN.scan results
- makeimg.py: get bootloader and partition offset from sdkconfig
- enable MICROPY_PY_FSTRINGS by default
- machine_hw_spi: release GIL during transfers
- machine_pin: make check for non-output pins respect chip variant
- replace esp.neopixel with machine.bitstream
- remove dead code for end_ticks in machine_bitstream
- boards: add GENERIC_C3_USB board with USB serial/JTAG support

javascript port:
- rework Makefile and GC so it works with latest Emscripten
- Makefile: suppress compiler errors from array bounds
- Makefile: change variable to EXPORTED_RUNTIME_METHODS

mimxrt port:
- move calc_weekday helper function to timeutils
- machine_spi: add the SPI class to the machine module
- moduos: seed the PRNG on boot using the TRNG
- boards: set vfs partition start to 1 MBbyte
- main: skip running main.py if boot.py failed
- main: extend the information returned by help()
- mimxrt_flash: remove commented-out code
- modmachine: add a few minor methods to the machine module
- machine_led: use mp_raise_msg_varg helper
- machine_i2c: add hardware-based machine.I2C to machine module
- add support for Hyperflash chips
- boards: add support for the MIMXRT1050_EVKB board
- machine_pin: implement ioctl for Pin

minimal port:
- Makefile: add support for building with user C modules

nrf port:
- modules: replace master/slave with controller/peripheral in SPI
- boards/common.ld: calculate unused flash region
- modules/nrf: add new nrf module with flash block device
- drivers: add support for using flash block device with SoftDevice
- mpconfigport.h: expose nrf module when MICROPY_PY_NRF is set
- README: update README.md to reflect internal file systems
- mpconfigport.h: tune FAT FS configuration
- Makefile: add _fs_size linker script override from make
- modules/uos: allow a board to configure MICROPY_VFS_FAT/LFS1/LFS2
- mpconfigport.h: enable MICROPY_PY_IO_FILEIO when an FS is enabled
- qstrdefsport.h: add entries for in-built FS mount points
- main: add auto mount and auto format hook for internal flash FS
- boards: enable needed features for FAT/LFS1/LFS2
- facilitate use of freeze manifest
- boards: set FROZEN_MANIFEST blank when SD present on nrf51 targets
- modules/scripts: add file system formatting script
- Makefile: set default manifest file for all targets
- mphalport: add dummy function for mp_hal_time_ns()
- boards: enable MICROPY_VFS_LFS2 for all target boards
- modules/uos: add ilistdir to uos module
- modules/nrf: add function to enable/disable DCDC
- enable source line on tracebacks
- set .mpy features consistent with documentation and other ports

pic16bit port: no changes specific to this component/port

powerpc port: no changes specific to this component/port

qemu-arm port: no changes specific to this component/port

rp2 port:
- use 0=Monday datetime convention in RTC
- machine_rtc: in RTC.datetime, compute weekday automatically
- CMakeLists.txt: suppress compiler errors for pico-sdk and tinyusb
- tusb_config.h: set CFG_TUD_CDC_EP_BUFSIZE to 256
- machine_uart: add hardware flow control support
- machine_uart: allow overriding default machine UART pins
- machine_i2c: allow boards to configure I2C pins using new macros
- machine_spi: allow boards to configure SPI pins using new macros
- machine_uart: fix poll ioctl to also check hardware FIFO
- machine_uart: fix read when FIFO has chars but ringbuf doesn't
- tusb_port: allow boards to configure USB VID and PID
- boards/ADAFRUIT_FEATHER_RP2040: configure custom VID/PID
- boards/ADAFRUIT_FEATHER_RP2040: configure I2C/SPI default pins
- boards/SPARKFUN_PROMICRO: configure UART/I2C/SPI default pins
- boards/SPARKFUN_THINGPLUS: configure I2C/SPI default pins
- boards: add Adafruit ItsyBitsy RP2040
- boards: add Adafruit QT Py RP2040
- boards: add Pimoroni Pico LiPo 4MB
- boards: add Pimoroni Pico LiPo 16MB
- boards: add Pimoroni Tiny 2040
- CMakeLists.txt: allow a board's cmake to set the manifest path
- enable MICROPY_PY_FSTRINGS by default
- Makefile: add "submodules" target, to match other ports
- rp2_flash: disable IRQs while calling flash_erase/program
- CMakeLists.txt: add option to enable double tap reset to bootrom
- mpconfigport.h: allow boards to add root pointers

samd port:
- add support for building with user C modules

stm32 port:
- softtimer: add soft_timer_reinsert() helper function
- mpbthciport: change from systick to soft-timer for BT scheduling
- provide a custom BTstack runloop that integrates with soft timer
- usb: make irq's default trigger enable all events
- boardctrl: skip running main.py if boot.py had an error
- sdio: fix undefined reference to DMA stream on H7
- dma: add DMAMUX configuration for H7 to fix dma_nohal_init
- main: call mp_deinit() at end of main
- adc: allow using ADC12 and ADC3 for H7
- adc: define the ADC instance used for internal channels
- adc: simplify and generalise how pin_adcX table is defined
- add new machine.I2S class for I2S protocol support
- boards/NUCLEO_F446RE: fix I2C1 pin assignment to match datasheet
- replace master/slave with controller/peripheral in I2C and SPI
- systick: always POLL_HOOK when delaying for milliseconds
- sdram: make SDRAM test cache aware, and optional failure with msg
- boards/NUCLEO_F446RE: enable CAN bus support
- boards: add support for SparkFun STM32 MicroMod Processor board
- uart: fix LPUART1 baudrate set/get
- uart: support low baudrates on LPUART1
- boards/STM32F429DISC: set correct UART2 pins and add UART3/6
- boards/NUCLEO_F439ZI: add board definition for NUCLEO_F439ZI
- boards/LEGO_HUB_NO6: add board definition for LEGO_HUB_NO6
- Makefile: update to only pull in used Bluetooth library
- README.md: update supported MCUs, and submodule and mboot use
- usbd_desc: rename USBD_xxx descriptor opts to MICROPY_HW_USB_xxx
- usbd_cdc_interface: rename USBD_CDC_xx opts to MICROPY_HW_USB_xx
- powerctrl: support changing frequency on WB MCUs
- boards/NUCLEO_H743ZI2: add modified version of NUCLEO_H743ZI
- mbedtls: fix compile warning about uninitialized val
- enable MICROPY_PY_FSTRINGS by default
- add implementation of machine.bitstream
- Makefile: allow GIT_SUBMODULES and LIBS to be extended
- stm32_it: support TIM17 IRQs on WB MCUs
- disable computed goto on constrained boards
- storage: make extended-block-device more configurable
- boards/LEGO_HUB_NO6: change SPI flash storage to use hardware SPI
- boards/LEGO_HUB_NO6: skip first 1MiB of SPI flash for storage
- boards/LEGO_HUB_NO6: add make commands to backup/restore firmware

teensy port: no changes specific to this component/port

unix port:
- modffi: add option to lock GC in callback, and cfun access
- Makefile: add back LIB_SRC_C to list of object files
- variants: enable help and help("modules") on standard and dev
- Makefile: disable error compression on arm-linux-gnueabi-gcc

windows port:
- Makefile: add .exe extension to executables name
- appveyor: update to VS 2017 and use Python 3.8 for build/test

zephyr port:
- machine_spi: add support for hardware SPI
2021-10-15 08:20:54 -05:00
Jeff Epler 52540a9830 Rename EXTENDED_FIELDS -> MP_TYPE_EXTENDED_FIELDS 2021-07-12 06:57:59 -05:00
Damien George 136369d72f all: Update to point to files in new shared/ directory.
Signed-off-by: Damien George <damien@micropython.org>
2021-07-12 17:08:10 +10:00
Jeff Epler 7302bc09a7 rename the type flag to EXTENDED for consistency 2021-07-09 14:59:37 -05:00
Jeff Epler 0d756066e8 milestone: coverage make test_full passes 2021-07-06 10:37:32 -05:00
Jeff Epler df56ba207f WIP 2021-07-06 09:25:56 -05:00
Jeff Epler 87d3740c64 Merge tag 'v1.16' 2021-06-18 10:54:19 -05:00
Damien George 4791d290c6 extmod: Remove old comments used for auto-doc generation.
They are no longer used, and the text in the docs is more up to date.

Signed-off-by: Damien George <damien@micropython.org>
2021-05-06 12:27:31 +10:00
Scott Shawcroft 3fda0c0a1b
Fix board builds and use MP_ERROR_TEXT in py and extmod 2021-05-05 17:51:52 -07:00
Scott Shawcroft f0bb26d70f
Merge MicroPython 1.13 into CircuitPython 2021-05-04 18:06:33 -07:00
Scott Shawcroft b35fa44c8a
Merge MicroPython 1.12 into CircuitPython 2021-05-03 14:01:18 -07:00
Scott Shawcroft 76033d5115
Merge MicroPython v1.11 into CircuitPython 2021-04-26 15:47:41 -07:00
microDev a52eb88031
run code formatting script 2021-03-15 19:27:36 +05:30
Damien George 8f20cdc353 all: Rename absolute time-based functions to include "epoch".
For time-based functions that work with absolute time there is the need for
an Epoch, to set the zero-point at which the absolute time starts counting.
Such functions include time.time() and filesystem stat return values.  And
different ports may use a different Epoch.

To make it clearer what functions use the Epoch (whatever it may be), and
make the ports more consistent with their use of the Epoch, this commit
renames all Epoch related functions to include the word "epoch" in their
name (and remove references to "2000").

Along with this rename, the following things have changed:

- mp_hal_time_ns() is now specified to return the number of nanoseconds
  since the Epoch, rather than since 1970 (but since this is an internal
  function it doesn't change anything for the user).

- littlefs timestamps on the esp8266 have been fixed (they were previously
  off by 30 years in nanoseconds).

Otherwise, there is no functional change made by this commit.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-18 17:20:34 +10:00
Damien George 2a72e90ab8 extmod/vfs: Add option to use 1970 as Epoch.
By setting MICROPY_EPOCH_IS_1970 a port can opt to use 1970/1/1 as the
Epoch for timestamps returned by stat().  And this setting is enabled on
the unix and windows ports because that's what they use.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-01 12:36:28 +10:00
Damien George c70e599659 extmod/vfs: Support larger integer range in VFS stat time fields.
On ports like unix where the Epoch is 1970/1/1 and atime/mtime/ctime are in
seconds since the Epoch, this value will overflow a small-int on 32-bit
systems.  So far this is only an issue on 32-bit unix builds that use the
VFS layer (eg dev and coverage unix variants) but the fix (using
mp_obj_new_int_from_uint instead of MP_OBJ_NEW_SMALL_INT) is there for all
ports so as to not complicate the code, and because they will need the
range one day.

Also apply a similar fix to other fields in VfsPosix.stat because they may
also be large.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-01 12:36:28 +10:00
Diego Elio Pettenò 34b4993d63 Add license to some obvious files. 2020-07-06 19:16:25 +01:00
Damien George 69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
Dan Halbert e0753c4ad2 avoid os.stat() int overflow on smallint-only builds 2020-02-14 18:33:37 -05:00
Jeff Epler 238e121236 protocols: Allow them to be (optionally) type-safe
Protocols are nice, but there is no way for C code to verify whether
a type's "protocol" structure actually implements some particular
protocol.  As a result, you can pass an object that implements the
"vfs" protocol to one that expects the "stream" protocol, and the
opposite of awesomeness ensues.

This patch adds an OPTIONAL (but enabled by default) protocol identifier
as the first member of any protocol structure.  This identifier is
simply a unique QSTR chosen by the protocol designer and used by each
protocol implementer.  When checking for protocol support, instead of
just checking whether the object's type has a non-NULL protocol field,
use `mp_proto_get` which implements the protocol check when possible.

The existing protocols are now named:
    protocol_framebuf
    protocol_i2c
    protocol_pin
    protocol_stream
    protocol_spi
    protocol_vfs
(most of these are unused in CP and are just inherited from MP; vfs and
stream are definitely used though)

I did not find any crashing examples, but here's one to give a flavor of what
is improved, using `micropython_coverage`.  Before the change,
the vfs "ioctl" protocol is invoked, and the result is not intelligible
as json (but it could have resulted in a hard fault, potentially):

    >>> import uos, ujson
    >>> u = uos.VfsPosix('/tmp')
    >>> ujson.load(u)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: syntax error in JSON

After the change, the vfs object is correctly detected as not supporting
the stream protocol:
    >>> ujson.load(p)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    OSError: stream operation not supported
2019-12-04 09:29:57 -06:00
Damien George cfe1c5abf8 extmod/vfs: Rename BP_IOCTL_xxx constants to MP_BLOCKDEV_IOCTL_xxx.
Also rename SEC_COUNT to BLOCK_COUNT and SEC_SIZE to BLOCK_SIZE.
2019-10-29 14:17:29 +11:00
Damien George e1c7b1cb43 extmod/vfs_blockdev: Factor out block device interface code. 2019-10-29 12:55:17 +11:00
Damien George 9aabb6c01b extmod: Factor out block-device struct to make independent of fatfs. 2019-10-29 12:12:37 +11:00
Andrew Leech 74d07469f2 extmod/vfs_fat: Fallback to FAT32 if standard FAT16/SFD format fails.
This allows formatting SD cards, larger flash etc which do not support the
default FAT16/SFD format mode.
2019-03-26 17:15:23 +11:00
Scott Shawcroft 6c01aba05b
Prevent other filesystem changes besides file writes
Such as moving files, adding directories and changing the label.
2019-03-13 16:10:53 -07:00
Scott Shawcroft 7482870164
Throw an error when moving a directory into itself.
Fixes #1192
2019-03-13 16:10:52 -07:00
Damien George e959f21986 extmod/vfs_fat: Update for new oofatfs version. 2019-03-05 15:56:39 +11:00
Scott Shawcroft 72d993d60c
py changes for supporting superclass constructors that take kwargs 2019-01-14 17:29:19 -08:00
Noralf Trønnes 39ee12d1ac Fix os.stat() to use 1970 epoch
Commit 95e70cd0ea 'time: Use 1970 epoch' changed epoch for the time
module, but not for other users. This patch does the same for the only
other core timeutils user: extmod/vfs_fat.c:fat_vfs_stat().
Other timeutils users: cc3200, esp8266 and stm32, are not changed.

Ports that don't use long ints, will still get wrong time values from
os.stat().
2018-08-25 20:43:02 +02:00
Scott Shawcroft 96ebf5bc3f
Two fixes and translate more strings.
* Fix finding translations with escaped characters.
* Add back \r to translations since its needed by screen.
2018-08-09 13:29:30 -07:00
Dan Halbert e2e01efa84 compiles and runs; hangs on import storage;storage.VfsFat.<tab> 2018-07-13 22:51:10 -04:00
Dan Halbert 7c219600a2 WIP: after merge; before testing 2018-07-11 16:45:30 -04:00
Damien George c117effddd extmod/vfs: Introduce a C-level VFS protocol, with fast import_stat.
Following other C-level protocols, this VFS protocol is added to help
abstract away implementation details of the underlying VFS in an efficient
way.  As a starting point, the import_stat function is put into this
protocol so that the VFS sub-system does not need to know about every VFS
implementation in order to do an efficient stat for importing files.

In the future it might be worth adding other functions to this protocol.
2018-06-06 14:33:42 +10:00
Jeff Epler 9c47fd9c13 Specifically give a better error message for setlabel on RO fs 2018-03-27 21:28:19 -05:00
Jeff Epler fe7f405fc7 Add VfsFat.label property
These allow accessing the filesystem label.  For instance,
in boot.py, you can set the label on the built-in storage with:
   storage.remount('/', False)
   storage.getmount('/').label = "NEWLABEL"
   storage.remount('/', True)

Users with multiple CIRCUITPY boards may find it desirable to
choose a different label for each board they own.
2018-03-27 21:28:18 -05:00
Tom Collins 4d3a92c67c extmod/vfs_fat: Add file size as 4th element of uos.ilistdir tuple. 2018-03-12 12:26:36 +11:00
Damien George eb570f47a2 extmod/vfs_fat: Make fat_vfs_open_obj wrapper public, not its function.
This patch just moves the definition of the wrapper object fat_vfs_open_obj
to the location of the definition of its function, which matches how it's
done in most other places in the code base.
2018-02-23 17:33:26 +11:00
Damien George 638b860066 extmod/vfs_fat: Merge remaining vfs_fat_misc.c code into vfs_fat.c.
The only function left in vfs_fat_misc.c is fat_vfs_import_stat() which
can logically go into vfs_fat.c, allowing to remove vfs_fat_misc.c.
2018-02-23 17:24:57 +11:00
Damien George ae4a07730a extmod/vfs_fat: Move ilistdir implementation from misc to main file.
The fat_vfs_ilistdir2() function was only used by fat_vfs_ilistdir_func()
so moving the former into the same file as the latter allows it to be
placed directly into the latter function, thus saving code size.
2018-02-23 17:17:32 +11:00
Damien George 12ad64bc55 extmod/vfs_fat: Mount FatFS on creation so VFS methods can be used.
It's possible to use the methods (eg ilistdir) of a VFS FatFS object
without it being mounted in the VFS itself.  This previously worked but
only because FatFS was "mounting" the filesystem automatically when any
function (eg f_opendir) was called.  But it didn't work for ports that used
synchronisation objects (_FS_REENTRANT) because they are only initialised
via a call to f_mount.  So, call f_mount explicitly when creating a new
FatFS object so that everything is set up correctly.  Then also provide a
finaliser to do the f_umount call, but only if synchronisation objects are
enabled (since otherwise the f_umount call does nothing).
2017-11-20 11:46:40 +11:00
Damien George 4601759bf5 py/objstr: Remove "make_qstr_if_not_already" arg from mp_obj_new_str.
This patch simplifies the str creation API to favour the common case of
creating a str object that is not forced to be interned.  To force
interning of a new str the new mp_obj_new_str_via_qstr function is added,
and should only be used if warranted.

Apart from simplifying the mp_obj_new_str function (and making it have the
same signature as mp_obj_new_bytes), this patch also reduces code size by a
bit (-16 bytes for bare-arm and roughly -40 bytes on the bare-metal archs).
2017-11-16 13:17:51 +11:00
Damien George a3dc1b1957 all: Remove inclusion of internal py header files.
Header files that are considered internal to the py core and should not
normally be included directly are:
    py/nlr.h - internal nlr configuration and declarations
    py/bc0.h - contains bytecode macro definitions
    py/runtime0.h - contains basic runtime enums

Instead, the top-level header files to include are one of:
    py/obj.h - includes runtime0.h and defines everything to use the
        mp_obj_t type
    py/runtime.h - includes mpstate.h and hence nlr.h, obj.h, runtime0.h,
        and defines everything to use the general runtime support functions

Additional, specific headers (eg py/objlist.h) can be included if needed.
2017-10-04 12:37:50 +11:00
Damien George 4c736ea8fc extmod,unix: For uos.stat interpret st_size member as an unsigned int.
This prevents large files (eg larger than 2gb on a 32-bit arch) from
showing up as having a negative size.  Fixes issue #3227.
2017-08-21 20:47:22 +10:00
Damien George d70f688f25 extmod/vfs: Use MP_S_IFDIR, MP_S_IFREG consts instead of magic numbers. 2017-05-10 12:30:34 +10:00
Damien George d4cd4831b0 extmod/vfs_fat: Replace listdir() with implementation of ilistdir().
VfsFat no longer has the listdir() method.  Rather, if listdir()
functionality is needed then one should use uos.listdir() which will call
VfsFat.ilistdir().
2017-05-10 11:39:28 +10:00
Damien George f7816188b7 extmod/vfs_fat: Fix calculation of total blocks in statvfs. 2017-03-29 12:53:35 +11:00
Damien George 12d0731b91 extmod/vfs_fat: Remove obsolete and unused str/len members. 2017-03-10 19:09:42 +11:00
Andrew Gatt 10dbf2383f extmod/vfs_fat.c: Use explicit include path for timeutils.h. 2017-01-30 23:10:16 +11:00
Damien George b697c89009 extmod: Merge old fsusermount.h header into vfs.h and vfs_fat.h.
vfs.h is for generic VFS declarations, and vfs_fat.h is for VfsFat
specific things.
2017-01-30 12:26:08 +11:00