djsundog
/
circuitpython
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs: Add littlefs docs and a filesystem tutorial.

This commit is contained in:
Jim Mussared 2019-12-04 10:42:07 +11:00 committed by Damien George
parent f2650be844
commit 9a849cc7ca
4 changed files with 322 additions and 63 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
docs/library/esp32.rst
View File

@ -56,10 +56,14 @@ This class gives access to the partitions in the device's flash memory.
Returns a 6-tuple ``(type, subtype, addr, size, label, encrypted)``.
.. method:: Partition.readblocks(block_num, buf)
.. method:: Partition.readblocks(block_num, buf, offset)
.. method:: Partition.writeblocks(block_num, buf)
.. method:: Partition.writeblocks(block_num, buf, offset)
.. method:: Partition.ioctl(cmd, arg)
These methods implement the block protocol defined by :class:`uos.AbstractBlockDev`.
These methods implement the simple and :ref:`extended
<block-device-interface>` block protocol defined by
:class:`uos.AbstractBlockDev`.
.. method:: Partition.set_boot()

100
docs/library/uos.rst
View File

@ -178,6 +178,35 @@ represented by VFS classes.
Build a FAT filesystem on *block_dev*.
.. class:: VfsLfs1(block_dev)
Create a filesystem object that uses the `littlefs v1 filesystem format`_.
Storage of the littlefs filesystem is provided by *block_dev*, which must
support the :ref:`extended interface <block-device-interface>`.
Objects created by this constructor can be mounted using :func:`mount`.
See :ref:`filesystem` for more information.
.. staticmethod:: mkfs(block_dev)
Build a Lfs1 filesystem on *block_dev*.
.. class:: VfsLfs2(block_dev)
Create a filesystem object that uses the `littlefs v2 filesystem format`_.
Storage of the littlefs filesystem is provided by *block_dev*, which must
support the :ref:`extended interface <block-device-interface>`.
Objects created by this constructor can be mounted using :func:`mount`.
See :ref:`filesystem` for more information.
.. staticmethod:: mkfs(block_dev)
Build a Lfs2 filesystem on *block_dev*.
.. _littlefs v1 filesystem format: https://github.com/ARMmbed/littlefs/tree/v1
.. _littlefs v2 filesystem format: https://github.com/ARMmbed/littlefs
Block devices
-------------
@ -187,9 +216,15 @@ implementation of this class will usually allow access to the memory-like
functionality a piece of hardware (like flash memory). A block device can be
used by a particular filesystem driver to store the data for its filesystem.
.. _block-device-interface:
Simple and extended interface
.............................
There are two compatible signatures for the ``readblocks`` and ``writeblocks``
methods (see below), in order to support a variety of use cases. A given block
device may implement one form or the other, or both at the same time.
device may implement one form or the other, or both at the same time. The second
form (with the offset parameter) is referred to as the "extended interface".
.. class:: AbstractBlockDev(...)
@ -247,64 +282,5 @@ device may implement one form or the other, or both at the same time.
(*arg* is unused)
- 6 -- erase a block, *arg* is the block number to erase
By way of example, the following class will implement a block device that stores
its data in RAM using a ``bytearray``::
class RAMBlockDev:
def __init__(self, block_size, num_blocks):
self.block_size = block_size
self.data = bytearray(block_size * num_blocks)
def readblocks(self, block_num, buf):
for i in range(len(buf)):
buf[i] = self.data[block_num * self.block_size + i]
def writeblocks(self, block_num, buf):
for i in range(len(buf)):
self.data[block_num * self.block_size + i] = buf[i]
def ioctl(self, op, arg):
if op == 4: # get number of blocks
return len(self.data) // self.block_size
if op == 5: # get block size
return self.block_size
It can be used as follows::
import uos
bdev = RAMBlockDev(512, 50)
uos.VfsFat.mkfs(bdev)
vfs = uos.VfsFat(bdev)
uos.mount(vfs, '/ramdisk')
An example of a block device that supports both signatures and behaviours of
the :meth:`readblocks` and :meth:`writeblocks` methods is::
class RAMBlockDev:
def __init__(self, block_size, num_blocks):
self.block_size = block_size
self.data = bytearray(block_size * num_blocks)
def readblocks(self, block, buf, offset=0):
addr = block_num * self.block_size + offset
for i in range(len(buf)):
buf[i] = self.data[addr + i]
def writeblocks(self, block_num, buf, offset=None):
if offset is None:
# do erase, then write
for i in range(len(buf) // self.block_size):
self.ioctl(6, block_num + i)
offset = 0
addr = block_num * self.block_size + offset
for i in range(len(buf)):
self.data[addr + i] = buf[i]
def ioctl(self, op, arg):
if op == 4: # block count
return len(self.data) // self.block_size
if op == 5: # block size
return self.block_size
if op == 6: # block erase
return 0
See :ref:`filesystem` for example implementations of block devices using both
protocols.

278
docs/reference/filesystem.rst Normal file
View File

@ -0,0 +1,278 @@
.. _filesystem:
Working with filesystems
========================
.. contents::
This tutorial describes how MicroPython provides an on-device filesystem,
allowing standard Python file I/O methods to be used with persistent storage.
MicroPython automatically creates a default configuration and auto-detects the
primary filesystem, so this tutorial will be mostly useful if you want to modify
the partitioning, filesystem type, or use custom block devices.
The filesystem is typically backed by internal flash memory on the device, but
can also use external flash, RAM, or a custom block device.
On some ports (e.g. STM32), the filesystem may also be available over USB MSC to
a host PC. :ref:`pyboard_py` also provides a way for the host PC to access to
the filesystem on all ports.
Note: This is mainly for use on bare-metal ports like STM32 and ESP32. On ports
with an operating system (e.g. the Unix port) the filesystem is provided by the
host OS.
VFS
---
MicroPython implements a Unix-like Virtual File System (VFS) layer. All mounted
filesystems are combined into a single virtual filesystem, starting at the root
``/``. Filesystems are mounted into directories in this structure, and at
startup the working directory is changed to where the primary filesystem is
mounted.
On STM32 / Pyboard, the internal flash is mounted at ``/flash``, and optionally
the SDCard at ``/sd``. On ESP8266/ESP32, the primary filesystem is mounted at
``/``.
Block devices
-------------
A block device is an instance of a class that implements the
:class:`uos.AbstractBlockDev` protocol.
Built-in block devices
~~~~~~~~~~~~~~~~~~~~~~
Ports provide built-in block devices to access their primary flash.
On power-on, MicroPython will attempt to detect the filesystem on the default
flash and configure and mount it automatically. If no filesystem is found,
MicroPython will attempt to create a FAT filesystem spanning the entire flash.
Ports can also provide a mechanism to "factory reset" the primary flash, usually
by some combination of button presses at power on.
STM32 / Pyboard
...............
The :ref:`pyb.Flash <pyb.Flash>` class provides access to the internal flash. On some
boards which have larger external flash (e.g. Pyboard D), it will use that
instead. The ``start`` kwarg should always be specified, i.e.
``pyb.Flash(start=0)``.
Note: For backwards compatibility, when constructed with no arguments (i.e.
``pyb.Flash()``), it only implements the simple block interface and reflects the
virtual device presented to USB MSC (i.e. it includes a virtual partition table
at the start).
ESP8266
.......
The internal flash is exposed as a block device object which is created in the
``flashbdev`` module on start up. This object is by default added as a global
variable so it can usually be accessed simply as ``bdev``. This implements the
extended interface.
ESP32
.....
The :class:`esp32.Partition` class implements a block device for partitions
defined for the board. Like ESP8266, there is a global variable ``bdev`` which
points to the default partition. This implements the extended interface.
Custom block devices
~~~~~~~~~~~~~~~~~~~~
The following class implements a simple block device that stores its data in
RAM using a ``bytearray``::
class RAMBlockDev:
def __init__(self, block_size, num_blocks):
self.block_size = block_size
self.data = bytearray(block_size * num_blocks)
def readblocks(self, block_num, buf):
for i in range(len(buf)):
buf[i] = self.data[block_num * self.block_size + i]
def writeblocks(self, block_num, buf):
for i in range(len(buf)):
self.data[block_num * self.block_size + i] = buf[i]
def ioctl(self, op, arg):
if op == 4: # get number of blocks
return len(self.data) // self.block_size
if op == 5: # get block size
return self.block_size
It can be used as follows::
import os
bdev = RAMBlockDev(512, 50)
os.VfsFat.mkfs(bdev)
os.mount(bdev, '/ramdisk')
An example of a block device that supports both the simple and extended
interface (i.e. both signatures and behaviours of the
:meth:`uos.AbstractBlockDev.readblocks` and
:meth:`uos.AbstractBlockDev.writeblocks` methods) is::
class RAMBlockDev:
def __init__(self, block_size, num_blocks):
self.block_size = block_size
self.data = bytearray(block_size * num_blocks)
def readblocks(self, block, buf, offset=0):
addr = block_num * self.block_size + offset
for i in range(len(buf)):
buf[i] = self.data[addr + i]
def writeblocks(self, block_num, buf, offset=None):
if offset is None:
# do erase, then write
for i in range(len(buf) // self.block_size):
self.ioctl(6, block_num + i)
offset = 0
addr = block_num * self.block_size + offset
for i in range(len(buf)):
self.data[addr + i] = buf[i]
def ioctl(self, op, arg):
if op == 4: # block count
return len(self.data) // self.block_size
if op == 5: # block size
return self.block_size
if op == 6: # block erase
return 0
As it supports the extended interface, it can be used with :class:`littlefs
<uos.VfsLfs2>`::
import os
bdev = RAMBlockDev(512, 50)
os.VfsLfs2.mkfs(bdev)
os.mount(bdev, '/ramdisk')
Filesystems
-----------
MicroPython ports can provide implementations of :class:`FAT <uos.VfsFat>`,
:class:`littlefs v1 <uos.VfsLfs1>` and :class:`littlefs v2 <uos.VfsLfs2>`.
The following table shows which filesystems are included in the firmware by
default for given port/board combinations, however they can be optionally
enabled in a custom firmware build.
==================== ===== =========== ===========
Board FAT littlefs v1 littlefs v2
==================== ===== =========== ===========
pyboard 1.0, 1.1, D Yes No Yes
Other STM32 Yes No No
ESP8266 Yes No No
ESP32 Yes No Yes
==================== ===== =========== ===========
FAT
~~~
The main advantage of the FAT filesystem is that it can be accessed over USB MSC
on supported boards (e.g. STM32) without any additional drivers required on the
host PC.
However, FAT is not tolerant to power failure during writes and this can lead to
filesystem corruption. For applications that do not require USB MSC, it is
recommended to use littlefs instead.
To format the entire flash using FAT::
# ESP8266 and ESP32
import os
os.umount('/')
os.VfsFat.mkfs(bdev)
os.mount(bdev, '/')
# STM32
import os, pyb
os.umount('/flash')
os.VfsFat.mkfs(pyb.Flash(start=0))
os.mount(pyb.Flash(start=0), '/flash')
os.chdir('/flash')
Littlefs
~~~~~~~~
Littlefs_ is a filesystem designed for flash-based devices, and is much more
resistant to filesystem corruption.
Note: It can be still be accessed over USB MSC using the `littlefs FUSE
driver`_. Note that you must use the ``-b=4096`` option to override the block
size.
.. _littlefs FUSE driver: https://github.com/ARMmbed/littlefs-fuse/tree/master/littlefs
.. _Littlefs: https://github.com/ARMmbed/littlefs
To format the entire flash using littlefs v2::
# ESP8266 and ESP32
import os
os.umount('/')
os.VfsLfs2.mkfs(bdev)
os.mount(bdev, '/')
# STM32
import os, pyb
os.umount('/flash')
os.VfsLfs2.mkfs(pyb.Flash(start=0))
os.mount(pyb.Flash(start=0), '/flash')
os.chdir('/flash')
Hybrid (STM32)
~~~~~~~~~~~~~~
By using the ``start`` and ``len`` kwargs to :class:`pyb.Flash`, you can create
block devices spanning a subset of the flash device.
For example, to configure the first 256kiB as FAT (and available over USB MSC),
and the remainder as littlefs::
import os, pyb
os.umount('/flash')
p1 = pyb.Flash(start=0, len=256*1024)
p2 = pyb.Flash(start=256*1024)
os.VfsFat.mkfs(p1)
os.VfsLfs2.mkfs(p2)
os.mount(p1, '/flash')
os.mount(p2, '/data')
os.chdir('/flash')
This might be useful to make your Python files, configuration and other
rarely-modified content available over USB MSC, but allowing for frequently
changing application data to reside on littlefs with better resilience to power
failure, etc.
The partition at offset ``0`` will be mounted automatically (and the filesystem
type automatically detected), but you can add::
import os, pyb
p2 = pyb.Flash(start=256*1024)
os.mount(p2, '/data')
to ``boot.py`` to mount the data partition.
Hybrid (ESP32)
~~~~~~~~~~~~~~
On ESP32, if you build custom firmware, you can modify ``partitions.csv`` to
define an arbitrary partition layout.
At boot, the partition named "vfs" will be mounted at ``/`` by default, but any
additional partitions can be mounted in your ``boot.py`` using::
import esp32, os
p = esp32.Partition.find(esp32.Partition.TYPE_DATA, label='foo')
os.mount(p, '/foo')

1
docs/reference/index.rst
View File

@ -26,4 +26,5 @@ implementation and the best practices to use them.
constrained.rst
packages.rst
asm_thumb2_index.rst
filesystem.rst
pyboard.py.rst