This commit adds full support to the unix port for Bluetooth using the
common extmod/modbluetooth Python bindings. This uses the libusb HCI
transport, which supports many common USB BT adaptors.
This commit adds an idf_heap_info(capabilities) method to the esp32 module
which returns info about the ESP-IDF heaps. It's useful to get a bit of a
picture of what's going on when code fails because ESP-IDF can't allocate
memory anymore. Includes documentation and a test.
This commit adds Loop.new_event_loop() which is used to reset the singleton
event loop. This functionality is put here instead of in Loop.close() to
make it possible to write code that is compatible with CPython.
This commit changes the esp8266 boards to use littlefs v2 as the
filesystem, rather than FAT. Since the esp8266 doesn't expose the
filesystem to the PC over USB there's no strong reason to keep it as FAT.
Littlefs is smaller in code size, is more efficient in use of flash to
store data, is resilient over power failure, and using it saves about 4k of
heap RAM, which can now be used for other things.
This is a backwards incompatible change because all existing esp8266 boards
will need to update their filesystem after installing new firmware (eg
backup old files, install firmware, restore files to new filesystem).
As part of this commit the memory layout of the default board (GENERIC) has
changed. It now allocates all 1M of memory-mapped flash to the firmware,
so the filesystem area starts at the 2M point. This is done to allow more
frozen bytecode to be stored in the 1M of memory-mapped flash. This
requires an esp8266 module with 2M or more of flash to work, so a new board
called GENERIC_1M is added which has the old memory-mapping (but still
changed to use littlefs for the filesystem).
In summary there are now 3 esp8266 board definitions:
- GENERIC_512K: for 512k modules, doesn't have a filesystem.
- GENERIC_1M: for 1M modules, 572k for firmware+frozen code, 396k for
filesystem (littlefs).
- GENERIC: for 2M (or greater) modules, 968k for firmware+frozen code,
1M+ for filesystem (littlefs), FAT driver also included in firmware for
use on, eg, external SD cards.
This commit adds micropython.heap_locked() which returns the current
lock-depth of the heap, and can be used by Python code to check if the heap
is locked or not. This new function is configured via
MICROPY_PY_MICROPYTHON_HEAP_LOCKED and is disabled by default.
This commit also changes the return value of micropython.heap_unlock() so
it returns the current lock-depth as well.
This commit changes the BLE _IRQ_SCAN_RESULT data from:
addr_type, addr, connectable, rssi, adv_data
to:
addr_type, addr, adv_type, rssi, adv_data
This allows _IRQ_SCAN_RESULT to handle all scan result types (not just
connectable and non-connectable passive scans), and to distinguish between
them using adv_type which is an integer taking values 0x00-0x04 per the BT
specification.
This is a breaking change to the API, albeit a very minor one: the existing
connectable value was a boolean and True now becomes 0x00, False becomes
0x02.
Documentation is updated and a test added.
Fixes#5738.
The default value for MICROPYPATH used in unix/main.c is
"~/.micropython/lib:/usr/lib/micropython" which has 2 problems when used in
the Windows port:
- it has a ':' as path separator but the port uses ';' so the entire string
is effectively discarded since it gets interpreted as a single path which
doesn't exist
- /usr/lib/micropython is not a valid path in a standard Windows
environment
Override the value with a suitable default.
This fix can be demonstrated by the following:
b = bytearray(32)
f = framebuf.FrameBuffer(b, 32, 8, framebuf.MONO_HLSB)
f.pixel(0, 0, 1)
print('MONO_HLSB', hex(b[0]))
b = bytearray(32)
f = framebuf.FrameBuffer(b, 32, 8, framebuf.MONO_HMSB)
f.pixel(0, 0, 1)
print('MONO_HMSB', hex(b[0]))
Outcome:
MONO_HLSB 0x80
MONO_HMSB 0x1
This adds a -h option to print the usage help text and adds a new, shorter
error message that is printed when invalid arguments are given. This
behaviour follows CPython (and other tools) more closely.
Show how to send an HTTP response code and content-type. Without the
response code Safari/iOS will fail. Without the content-type Lynx/Links
will fail.
This adds a short paragraph on how to hook readthedocs.org up. The main
goal is to make people aware of the option, to help with contributing to
the documentation.
The size of the event ringbuf was previously fixed to compile-time config
value, but it's necessary to sometimes increase this for applications that
have large characteristic buffers to read, or many events at once.
With this commit the size can be set via BLE.config(rxbuf=512), for
example. This also resizes the internal event data buffer which sets the
maximum size of incoming data passed to the event handler.
This allows the user to explicitly select the behaviour of the write to the
remote peripheral. This is needed for peripherals that have
characteristics with WRITE_NO_RESPONSE set (instead of normal WRITE). The
function's signature is now:
BLE.gattc_write(conn_handle, value_handle, data, mode=0)
mode=0 means write without response, while mode=1 means write with
response. The latter was the original behaviour so this commit is a change
in behaviour of this method, and one should specify 1 as the 4th argument
to get back the old behaviour.
In the future there could be more modes supported, such as long writes.
The address, adv payload and uuid fields of the event are pre-allocated by
modbluetooth, and reused in the IRQ handler. Simplify this and move all
storage into the `mp_obj_bluetooth_ble_t` instance.
This now allows users to hold on to a reference to these instances without
crashes, although they may be overwritten by future events. If they want
to hold onto the values longer term they need to copy them.
defindex.html (used by topindex.html) is deprecated, but topindex.html was
already identical other than setting the title, so just inherit directly
from layout.html.
Behaviour was changed from stack to queue in
8977c7eb58, and this updates variable names
to match. Also updates other references (docs, error messages).
This allows to efficiently send to an I2C slave data that is made up of
more than one buffer. Instead of needing to allocate temporary memory to
combine buffers together this new method allows to pass in a tuple or list
of buffers. The name is based on the POSIX function writev() which has
similar intentions and signature.
The reasons for taking this approach (compared to having an interface with
separate start/write/stop methods) are:
- It's a backwards compatible extension.
- It's convenient for the user.
- It's efficient because there is only one Python call, then the C code can
do everything in one go.
- It's efficient on the I2C bus because the implementation can do
everything in one go without pauses between blocks of bytes.
- It should be possible to implement this extension in all ports, for
hardware and software I2C.
Further discussion is found in issue #3482, PR #4020 and PR #4763.
It's more common to need non-blocking behaviour when reading from a UART,
rather than having a large timeout like 1000ms (the original behaviour).
With a large timeout it's 1) likely that the function will read forever if
characters keep trickling it; or 2) the function will unnecessarily wait
when characters come sporadically, eg at a REPL prompt.
This system makes it a lot easier to include external libraries as static,
native modules in MicroPython. Simply pass USER_C_MODULES (like
FROZEN_MPY_DIR) as a make parameter.
This is only correct for the extmod/uos_dupterm.c implementation however,
as e.g cc3200 implementation does the mp_load_method() itself, and anyway
requires `read` instead of `readinto`.
Replaces "PYB: soft reboot" with "MPY: soft reboot", etc.
Having a consistent prefix across ports reduces the difference between
ports, which is a general goal. And this change won't break pyboard.py
because that tool only looks for "soft reboot".
The machine.sleep() function can be misleading because it clashes with
time.sleep() which has quite different semantics. So change it to
machine.lightsleep() which shows that it is closer in behaviour to
machine.deepsleep().
Also, add an optional argument to these two sleep functions to specify a
maximum time to sleep for. This is a common operation and underlying
hardware usually has a special way of performing this operation.
The existing machine.sleep() function will remain for backwards
compatibility purposes, and it can simply be an alias for
machine.lightsleep() without arguments. The behaviour will be the same.
Examples are added to the beginning of the module docs, similarly to docs
for many other modules.
Improvements to grammar, style, and clarity. Some paragraphs are updated
with better suggestions. A warning added of the effect incorrect usage of
the module may have. Describe the fact that offset range used in one
defined structure is limited.
Otherwise there is really nothing that can be done, it can't be unlocked by
the user because there is no way to allocate memory to execute the unlock.
See issue #4205 and #4209.
With this commit there is now only one entry point into the whole
documentation, which describes the general MicroPython language, and then
from there there are links to information about specific platforms/ports.
This commit doesn't change content (almost, it does fix a few internal
links), it just reorganises things.
All concrete network classes are now moved to their own file (eg
network.WLAN.rst) and deconditionalised (remove ..only:: directives). This
makes the network documentation the same for all ports. After this change
there are no more "..only::" directives for different ports, and the only
difference among ports is the very front page of the docs.
The WiPy machine.Timer class is very different to the esp8266 and esp32
implementations which are better candidates for a general Timer class. By
moving the WiPy Timer docs to a completely separate file, under a new name
machine.TimerWiPy, it gives a clean slate to define and write the docs for
a better, general machine.Timer class. This is with the aim of eventually
providing documentation that does not have conditional parts to it,
conditional on the port.
While the new docs are being defined it makes sense to keep the WiPy docs,
since they describe its behaviour. Once the new Timer behaviour is defined
the WiPy code can be changed to match it, and then the TimerWiPy docs would
be removed.
The machine module should be standard across all ports so should have the
same set of classes in the docs. A special warning is added to the top of
the machine.SD class because it is not standardised and only available on
the cc3200 port.
It's fair to just provide a link to all available modules, regardless of
the port. Most of the existing ports (unix, stm32, esp8266, esp32) share
most of the same set of modules anyway, so no need to maintain separate
lists for them. And there's a big discussion at the start of this index
about modules not being available on a given port.
For port-specific modules, they can also be listed unconditionally because
they have headings that explicitly state they are only available on certain
ports.
The UART.init() method is now included unconditionally and its wording
adjusted to better describe ports other than the cc3200.
UART.irq() is also included unconditionally, but this is currently only
available on the WiPy target.
By virtue of its name, the pyb module would only be available on a pyboard
and so does not need to have conditional "only" directives throughout its
documentation.
These conditionals were added mostly in
cfcf47c064 in the initial development of the
cc3200 port, which had the pyb module before it switched to the machine
module. And wipy only conditionals were removed from the pyb module
documentation in 4542643025, so there's no
need to retain any more conditionals.
Allow including crypto consts based on compilation settings. Disabled by
default to reduce code size; if one wants extra code readability, can
enable them.
These can be optionally specified, but all ports are expected to be able to
accept them, at the very least ignore, though handling of "type" param
(SOCK_STREAM vs SOCK_DGRAM) is recommended.
- moving Makefile to parent folder. This resolves some of the weird path/build issues
- remove trace references to cpydiff and original file (no longer used anywhere)
- converting SOURCEDIR to a changeable variable, passed through all sphinx-build calls
- adding path to conf.py, in case it moves again
- making `-v` default with VERBOSE
- making `-E` default with FORCE
- creating BASEOPTS to store all the dirs, paths, and settings, passing them to sphinx-build in one long chain, instead of individually
- updating README to use the make command as default. Also added text explaining some of the customization you can add into a make command
This can be used to select the output buffer behaviour of the DAC. The
default values are chosen to retain backwards compatibility with existing
behaviour.
Thanks to @peterhinch for the initial idea to add this feature.
/docs/design_guide: added links to firmware build learning guides for SAMD21 & ESP8266. Changes were placed in the "Adding native modules" section, since that seemed to me the best place based on target audience.
Updated documentation for `delay()` which fixes#243.
The parameter order in the example for ticks_diff was incorrect. If it's
"too early" that means that scheduled time is greater than current time and
if it's "running late" then scheduled time would be less than current time.
Put offset first in OR expressions, and use "offset" var instead of
hardcoded numbers. Hopefully, this will make it more self-describing
and show patterns better.
Introduction of ports subdirectory where all ports are moved to
The main change in this release is the introduction of a "ports/"
subdirectory at the top-level of the repository, and all of the ports are
moved here. In the process the "stmhal" port is renamed to "stm32" to
better reflect the MCU that it targets. In addition, the STM32 CMSIS and
HAL sources are moved to a new submodule called "stm32lib".
The bytecode has changed in this release, compared to the previous release,
and as a consequence the .mpy version number has increased to version 3.
This means that scripts compiled with the previous mpy-cross must be
recompiled to work with this new version.
There have also been various enhancements and optimisations, such as:
check for valid UTF-8 when creating str objects, support for reverse
special binary operations like __radd__, full domain checking in the math
module, support for floor-division and modulo in the viper emitter,
and addition of stack overflow checking when executing a regex.
The stm32 port sees improved support for F7 MCUs, addition of a new board
B_L475E_IOT01A based on the STM32L475, and support for the Wiznet W5500
chipset along with improved socket behaviour.
A detailed list of changes follows.
py core:
- objstr: startswith, endswith: check arg to be a string
- nlrx86,x64: replace #define of defined() with portable macro usage
- objtype: handle NotImplemented return from binary special methods
- objtype: mp_obj_class_lookup: improve debug logging
- map: remove unused new/free functions
- make m_malloc_fail() have void return type, since it doesn't return
- modstruct: in struct.pack, stop converting if there are no args left
- modstruct: check and prevent buffer-read overflow in struct unpacking
- modstruct: check and prevent buffer-write overflow in struct packing
- nlrthumb: get working again on standard Thumb arch (ie not Thumb2)
- objfloat: fix binary ops with incompatible objects
- obj: fix comparison of float/complex NaN with itself
- objtype: implement fallback for instance inplace special methods
- objtuple: properly implement comparison with incompatible types
- objstr: add check for valid UTF-8 when making a str from bytes
- objlist: properly implement comparison with incompatible types
- runtime0.h: move relational ops to the beginning of mp_binary_op_t
- runtime0.h: move MP_BINARY_OP_DIVMOD to the end of mp_binary_op_t
- objtype: make sure mp_binary_op_method_name has full size again
- runtime0.h: put inplace arith ops in front of normal operations
- builtinhelp: simplify code slightly by extracting object type
- runtime: implement dispatch for "reverse op" special methods
- nlrx86: fix building for Android/x86
- builtinhelp: change signature of help text var from pointer to array
- runtime.h: change empty mp_warning macro so var-args are non empty
- modbuiltins: implement abs() by dispatching to MP_UNARY_OP_ABS
- {objfloat,objcomplex}: optimise MP_UNARY_OP_ABS by reusing variables
- mpconfig.h: add note that using computed gotos in VM is not C99
- objstr: strip: don't strip "\0" by default
- objexcept: prevent infinite recursion when allocating exceptions
- stream: remove unnecessary checks for NULL return from vstr_add_len
- vstr: raise a RuntimeError if fixed vstr buffer overflows
- vm: use lowercase letter at start of exception message
- persistentcode: define mp_raw_code_save_file() for any unix target
- add config option to print warnings/errors to stderr
- objfloat: support raising a negative number to a fractional power
- objset: simplify set and frozenset by separating their locals dicts
- objset: check that RHS of a binary op is a set/frozenset
- objset: include the failed key in a KeyError raised from set.remove
- objtype: change type of enum-to-qstr table to uint16_t to save space
- objstr: make empty bytes object have a null-terminating byte
- mpprint: only check for null string printing when NDEBUG not defined
- objtype: clean up unary- and binary-op enum-to-qstr mapping tables
- persistentcode: bump .mpy version number to version 3
- bc: update opcode_format_table to match the bytecode
- modmath: add full checks for math domain errors
- modmath: convert log2 macro into a function
- formatfloat: don't print the negative sign of a NaN value
- formatfloat: use standard isinf, isnan funcs instead of custom ones
- modbuiltins: use existing utf8_get_char helper in builtin ord func
- emitnative: implement floor-division and modulo for viper emitter
- objtype: use CPython compatible method name for sizeof
- objtype: fit qstrs for special methods in byte type
- objtype: define all special methods if requested
- objtype: introduce MICROPY_PY_ALL_INPLACE_SPECIAL_METHODS
extmod:
- modubinascii: only include uzlib/tinf.h when it's really needed
- modussl_mbedtls: allow to compile with MBEDTLS_DEBUG_C disabled
- machine_pinbase: put PinBase singleton in ROM
- re1.5: upgrade to v0.8.2, adds hook for stack overflow checking
- modure: add stack overflow checking when executing a regex
- uos_dupterm: update uos.dupterm() and helper funcs to have index
- uos_dupterm: swallow any errors from dupterm closing the stream
- vfs: replace VLA in proxy func with small, static sized array
- modussl: add finaliser support for ussl objects
- modussl_mbedtls: allow to compile with unix coverage build
lib:
- add new submodule, stm32lib containing STM32 CMSIS and HAL source
- embed/abort_: use mp_raise_msg helper function
- libm: fix tanhf so that it correctly handles +/- infinity args
- libm: remove implementation of log2f, use MP_NEED_LOG2 instead
- axtls: update, support for SSL_EAGAIN return code
- berkeley-db-1.xx: update, allow to override MINCACHE, DEFPSIZE
drivers:
- memory/spiflash: change from hard-coded soft SPI to generic SPI
- display/ssd1306.py: improve performance of graphics methods
- nrf24l01: make nRF24L01 test script more portable
- display/ssd1306: implement SSD1306_I2C poweron method
- display/ssd1306: make poweron() work the same with SSD1306_SPI
- wiznet5k: improve the performance of socket ops with threading
- wiznet5k: get low-level W5500 driver working
tools:
- upip: upgrade to 1.2.2
- pyboard: use repr() when quoting data in error messages
- pyboard: update docstring for additional device support
tests:
- object_new: better messages, check user __new__() method
- class_new: add checks for __init__ being called and other improvements
- class_new: add another testcase for __new__/__init__ interaction
- class_inplace_op: test for inplace op fallback to normal one
- run-bench-tests: update locations of executables, now in ports/
- class_reverse_op: test for reverse arith ops special methods
- run-tests: skip class_inplace_op for minimal profile
- run-tests: fix copy-paste mistake in var name
- cpydiff: add cases for locals() discrepancies
- extmod: add test for ure regexes leading to infinite recursion
- extmod: add test for '-' in character class in regex
- run-tests: close device under test using "finally"
- net_inet: update tls test to work with CPython and incl new site
unix port:
- rename modsocket.c to modusocket.c
- modusocket: remove #if MICROPY_SOCKET_EXTRA code blocks
- enable MICROPY_PY_REVERSE_SPECIAL_METHODS
stm32 port:
- modmachine: make machine.bootloader() work when MPU is enabled
- modmachine: improve support for sleep/deepsleep on F7 MCUs
- compute PLL freq table during build instead of at run time
- modmachine: for F7 MCU, save power by reducing internal volt reg
- boards/pllvalues.py: make script work with both Python 2 and 3
- Makefile: use lib/stm32lib instead of local cmsis and hal files
- remove cmsis and hal files, they are now a submodule
- Makefile: automatically fetch stm32lib submodule if needed
- update to new STM Cube HAL library
- fix clock initialisation of L4 MCUs
- rename stmhal port directory to stm32
- remove unused usbd_msc.c file
- boards: change remaining stm32f4xx_hal_conf.h to unix line ending
- boards: change linker scripts to use "K" instead of hex byte size
- boards: fix I2C1 pin mapping on NUCLEO_F401RE/F411RE boards
- i2c: when scanning for I2C devices only do 1 probe per address
- modnwwiznet5k: release the GIL on blocking network operations
- boards: add new board B_L475E_IOT01A based on STM32L475
- make-stmconst.py: make sure mpz const data lives in ROM
- timer: make pyb.Timer() instances persistent
- mpconfigport.h: add configuration for max periphs on L4 series
- usbdev: make the USBD callback struct const so it can go in ROM
- usbdev: change static function variable to non-static
- usbdev: put all CDC state in a struct
- usbdev: put all HID state in a struct
- usbdev: simplify CDC tx/rx buffer passing
- usbdev: simplify HID tx/rx buffer passing
- usbdev/core: add state parameter to all callback functions
- usbdev: put all state for the USB device driver in a struct
- usbdev: simplify pointers to MSC state and block dev operations
- usbdev: merge all global USB device state into a single struct
- usbdev: make device descriptor callbacks take a state pointer
- usbdev: move all the USB device descriptor state into its struct
- timer: enable ARPE so that timer freq can be changed smoothly
- modnwwiznet5k: get the IP address of an established socket
- boards: fix typos in stm32f767_af.csv table
- usbd_cdc_interface: don't reset CDC output buf on initialisation
- modnwwiznet5k: implement WIZNET5K.isconnected() method
- modusocket: make getaddrinfo() work when passed an IP address
- modusocket: return OSError(-2) if getaddrinfo fails
- mpconfigport.h: add MICROPY_THREAD_YIELD() macro
- modnwwiznet5k: add support for W5500 Ethernet chip
- modnwwiznet5k: increase SPI bus speed to 42MHz
- modnwwiznet5k: implement stream ioctl for the Wiznet driver
- mphalport: improve efficiency of mp_hal_stdout_tx_strn_cooked
- make uos.dupterm() conform to specs by using extmod version
cc3200 port:
- enable micropython.kbd_intr() method
- use standard implementation of keyboard interrupt
esp8266 port:
- rename axtls_helpers.c to posix_helpers.c
- posix_helpers: set ENOMEM on memory alloc failure
- set DEFPSIZE=1024, MINCACHE=3 for "btree" module
- esp_mphal: send data in chunks to mp_uos_dupterm_tx_strn
- modnetwork: add "bssid" keyword arg to WLAN.connect() method
- modules/webrepl_setup: add info about allowed password length
zephyr port:
- Makefile: revamp "test" target after ports were moved to ports/
- use CONFIG_NET_APP_SETTINGS to setup initial network addresses
- switch to interrupt-driven pull-style console
pic16bit port:
- add definition of SEEK_SET to unistd.h
docs:
- pyboard/tutorial: add "timeout=0" to UART in pass-through example
- more xrefs to "MicroPython port" in glossary
- library/network: fix ref to "socket" module (should be "usocket")
- machine.Signal: improve style/grammar and add usage example
- library: add description of "index" parameter to uos.dupterm()
- library/micropython: fix typo in RST formatting
- library/framebuf.rst: generalise constructor to all colour formats
- btree: describe page caching policy of the underlying implementation
- esp8266/tutorial: update neopixel with example of using 4 bbp
- library/network: clarify usage of "bssid" arg in connect() method
- pyboard/quickref: add info for Switch, RTC, CAN, Accel classes
- pyboard/tutorial: update now that yellow LED also supports PWM
- esp8266/quickref: add quickref info for RTC class
- library: add missing cross-ref links for classes in pyb module
- library/network: update docs to state that W5500 is supported
- uselect: document one-shot polling mode
- usocket: elaborate descriptions
- usocket: document inet_ntop(), inet_pton()
- library/network: add dhcp_hostname parameter
- reference/isr_rules: minor typo correction
- ussl: fix module name refs and use "MicroPython port" term
- esp8266/general: add section on TLS limitations
- usocket: document that settimeout() isn't supported by all ports
- ure: add "|" (alternative) to the list of supported operators
- reference/isr_rules.rst: add tutorial on use of micropython.schedule()
travis:
- use --upgrade when pip is installing cpp-coveralls
- update build command now that stm32 Wiznet config has changed
examples:
- hwconfig_console: add .on()/.off() methods
all:
- convert mp_uint_t to mp_unary_op_t/mp_binary_op_t where appropriate
- convert remaining "mp_uint_t n_args" to "size_t n_args"
- make new ports/ sub-directory and move all ports there
- update Makefiles and others to build with new ports/ dir layout
- remove inclusion of internal py header files
- use NULL instead of "" when calling mp_raise exception helpers
README:
- update "Dependencies" section
- add explicit section on contributing
- add gcc and arm-none-eabi-newlib to list of required components
.gitattributes:
- remove obsolete entries for stmhal/hal, stmhal/cmsis
- add entries for files that will move to ports/ dir
* atmel-samd: Remove ASF3. This will break builds.
* atmel-samd: Add ASF4 for the SAMD21 and SAMD51.
* Introduce the supervisor concept to facilitate porting.
The supervisor is the code which runs individual MicroPython VMs. By
splitting it out we make it more consistent and easier to find.
This also adds very basic SAMD21 and SAMD51 support using the
supervisor. Only the REPL currently works.
This begins the work for #178.
