It's now replaced by cmake/idf.py. But a convenience Makefile is still
provided with traditional targets like "all" and "deploy".
Signed-off-by: Damien George <damien@micropython.org>
This commit adds support for building the esp32 port with cmake, and in
particular it builds MicroPython as a component within the ESP-IDF. Using
cmake and the ESP-IDF build infrastructure makes it much easier to maintain
the port, especially with the various new ESP32 MCUs and their required
toolchains.
Signed-off-by: Damien George <damien@micropython.org>
The "word" referred to by BYTES_PER_WORD is actually the size of mp_obj_t
which is not always the same as the size of a pointer on the target
architecture. So rename this config value to better reflect what it
measures, and also prefix it with MP_.
For uses of BYTES_PER_WORD in setting the stack limit this has been
changed to sizeof(void *), because the stack usually grows with
machine-word sized values (eg an nlr_buf_t has many machine words in it).
Signed-off-by: Damien George <damien@micropython.org>
To simplify config, there's no need to specify MP_PLAT_PRINT_STRN if it's
the same as the default definition in py/mpconfig.h.
Signed-off-by: Damien George <damien@micropython.org>
Hardware I2C implementations must provide a .init() protocol method if they
want to support reconfiguration. Otherwise the default is that i2c.init()
raises an OSError (currently the case for all ports).
mp_machine_soft_i2c_locals_dict is renamed to mp_machine_i2c_locals_dict to
match the generic SPI bindings.
Fixes issue #6623 (where calling .init() on a HW I2C would crash).
Signed-off-by: Damien George <damien@micropython.org>
Support building .cpp files and linking them into the micropython
executable in a way similar to how it is done for .c files. The main
incentive here is to enable user C modules to use C++ files (which are put
in SRC_MOD_CXX by py.mk) since the core itself does not utilize C++.
However, to verify build functionality a unix overage test is added. The
esp32 port already has CXXFLAGS so just add the user modules' flags to it.
For the unix port use a copy of the CFLAGS but strip the ones which are not
usable for C++.
For seeding, the RNG function of the ESP-IDF is used, which is told to be a
true RNG, at least when WiFi or Bluetooth is enabled. Seeding on import is
as per CPython. To obtain a reproducible sequence of pseudo-random numbers
one must explicitly seed with a known value.
It requires mp_hal_time_ns() to be provided by a port. This function
allows very accurate absolute timestamps.
Enabled on unix, windows, stm32, esp8266 and esp32.
Signed-off-by: Damien George <damien@micropython.org>
With a warning that this way of constructing software I2C/SPI is
deprecated. The check and warning will be removed in a future release.
This should help existing code to migrate to the new SoftI2C/SoftSPI types.
Signed-off-by: Damien George <damien@micropython.org>
Previous commits removed the ability for one I2C/SPI constructor to
construct both software- or hardware-based peripheral instances. Such
construction is now split to explicit soft and non-soft types.
This commit makes both types available in all ports that previously could
create both software and hardware peripherals: machine.I2C and machine.SPI
construct hardware instances, while machine.SoftI2C and machine.SoftSPI
create software instances.
This is a breaking change for use of software-based I2C and SPI. Code that
constructed I2C/SPI peripherals in the following way will need to be
changed:
machine.I2C(-1, ...) -> machine.SoftI2C(...)
machine.I2C(scl=scl, sda=sda) -> machine.SoftI2C(scl=scl, sda=sda)
machine.SPI(-1, ...) -> machine.SoftSPI(...)
machine.SPI(sck=sck, mosi=mosi, miso=miso)
-> machine.SoftSPI(sck=sck, mosi=mosi, miso=miso)
Code which uses machine.I2C and machine.SPI classes to access hardware
peripherals does not need to change.
Signed-off-by: Damien George <damien@micropython.org>
The SoftSPI constructor is now used soley to create SoftSPI instances, it
can no longer delegate to create a hardware-based SPI instance.
Signed-off-by: Damien George <damien@micropython.org>
The SoftI2C constructor is now used soley to create SoftI2C instances, it
can no longer delegate to create a hardware-based I2C instance.
Signed-off-by: Damien George <damien@micropython.org>
Also rename machine_i2c_type to mp_machine_soft_i2c_type. These changes
make it clear that it's a soft-I2C implementation, and match SoftSPI.
Signed-off-by: Damien George <damien@micropython.org>
For time-based functions that work with absolute time there is the need for
an Epoch, to set the zero-point at which the absolute time starts counting.
Such functions include time.time() and filesystem stat return values. And
different ports may use a different Epoch.
To make it clearer what functions use the Epoch (whatever it may be), and
make the ports more consistent with their use of the Epoch, this commit
renames all Epoch related functions to include the word "epoch" in their
name (and remove references to "2000").
Along with this rename, the following things have changed:
- mp_hal_time_ns() is now specified to return the number of nanoseconds
since the Epoch, rather than since 1970 (but since this is an internal
function it doesn't change anything for the user).
- littlefs timestamps on the esp8266 have been fixed (they were previously
off by 30 years in nanoseconds).
Otherwise, there is no functional change made by this commit.
Signed-off-by: Damien George <damien@micropython.org>
To portably get the Epoch. This is simply aliased to localtime() on ports
that are not timezone aware.
Signed-off-by: Damien George <damien@micropython.org>
PPP support was disabled in 96008ff59a -
marked as "unsupported" due to an early IDF v4 release. With the currently
supported IDF v4.x version - 4c81978a - it appears to be working just fine.
This commit changes the default logging level on all esp32 boards to ERROR.
The esp32 port is now stable enough that it makes sense to remove the info
logs to make the output cleaner, and to match other ports. More verbose
logging can always be reenabled via esp.osdebug().
This also fixes issue #6354, error messages from NimBLE: the problem is
that ble.active(True) will cause the IDF's NimBLE port to reset the
"NimBLE" tag back to the default level (which was INFO prior to this
commit). Even if the user had previously called esp.osdebug(None), because
the IDF is setting the "NimBLE" tag back to the default (INFO), the
messages will continue to be shown.
The one quirk is that if the user does want to see the additional logging,
then they must call esp.osdebug(0, 3) after ble.active(True) to undo the
IDF setting the level back to the default (now ERROR). This means that
it's impossible (via Python/esp.osdebug) to see stack-startup logging,
you'd have to recompile with the default level changed back to INFO.
MicroPython and NimBLE must be on the same core, for synchronisation of the
BLE ringbuf and the MicroPython scheduler. However, in the current IDF
versions (3.3 and 4.0) there are issues (see e.g. #5489) with running
NimBLE on core 1.
This change - pinning both tasks to core 0 - makes it possible to reliably
run the BLE multitests on esp32 boards.
Previously the interaction between the different layers of the Bluetooth
stack was different on each port and each stack. This commit defines
common interfaces between them and implements them for cyw43, btstack,
nimble, stm32, unix.
gettimeofday returns seconds since 2000/1/1 so needs to be adjusted to
seconds since 1970/1/1 to give the correct return value of mp_hal_time_ns.
Signed-off-by: Damien George <damien@micropython.org>
A previous commit 3a9d948032 can cause
lock-ups of the RMT driver, so this commit reverses that, adds a loop_en
flag, and explicitly controls the TX interrupt in write_pulses(). This
provides correct looping, non-blocking writes and sensible behaviour for
wait_done().
See also #6167.
Otherwise the RMT will repeat pulses when using loop(True). This repeating
is due to a bug in the IDF which will be fixed in an upcoming release, but
for now the accepted workaround is to swap these calls, which should still
work in the fixed version of the IDF.
Fixes issue #6167.
The ESP32 RMT peripheral has hardware support for a carrier frequency, and
this commit exposes it to Python with the keyword arguments carrier_freq
and carrier_duty_percent in the constructor. Example usage:
r = esp32.RMT(0, pin=Pin(2), clock_div=80, carrier_freq=38000, carrier_duty_percent=50)
With this commit the code should work correctly regardless of the size of
StackType_t (it's actually 1 byte in size for the esp32's custom FreeRTOS).
Fixes issue #6072.
The code previously called rtc_get_reset_reason which is a "raw" reset
cause. The ESP-IDF massages that for the proper reset cause available from
esp_reset_reason.
Fixes issue #5134.
This commit allows the user to set/get the GAP device name used by service
0x1800, characteristic 0x2a00. The usage is:
BLE.config(gap_name="myname")
print(BLE.config("gap_name"))
As part of this change the compile-time setting
MICROPY_PY_BLUETOOTH_DEFAULT_NAME is renamed to
MICROPY_PY_BLUETOOTH_DEFAULT_GAP_NAME to emphasise its link to GAP and this
new "gap_name" config value. And the default value of this for the NimBLE
bindings is changed from "PYBD" to "MPY NIMBLE" to be more generic.
This commit fixes the behaviour of socket.getaddrinfo on the ESP32 so it
raises an OSError when the name resolution fails instead of returning a []
or a resolution for 0.0.0.0.
Tests are added (generic and ESP32-specific) to verify behaviour consistent
with CPython, modulo the different types of exceptions per MicroPython
documentation.
This commit adds several small items to improve the support for OTA
updates on an esp32:
- a partition table for 4MB flash modules that has two OTA partitions ready
to go to do updates
- a GENERIC_OTA board that uses that partition table and that enables
automatic roll-back in the bootloader
- a new esp32.Partition.mark_app_valid_cancel_rollback() class-method to
signal that the boot is successful and should not be rolled back at the
next reset
- an automated test for doing an OTA update
- documentation updates
This change is made for two reasons:
1. A 3rd-party library (eg berkeley-db-1.xx, axtls) may use the system
provided errno for certain errors, and yet MicroPython stream objects
that it calls will be using the internal mp_stream_errno. So if the
library returns an error it is not known whether the corresponding errno
code is stored in the system errno or mp_stream_errno. Using the system
errno in all cases (eg in the mp_stream_posix_XXX wrappers) fixes this
ambiguity.
2. For systems that have threading the system-provided errno should always
be used because the errno value is thread-local.
For systems that do not have an errno, the new lib/embed/__errno.c file is
provided.
Note: the uncrustify configuration is explicitly set to 'add' instead of
'force' in order not to alter the comments which use extra spaces after //
as a means of indenting text for clarity.
This commit consolidates a number of check_esp_err functions that check
whether an ESP-IDF return code is OK and raises an exception if not. The
exception raised is an OSError with the error code as the first argument
(negative if it's ESP-IDF specific) and the ESP-IDF error string as the
second argument.
This commit also fixes esp32.Partition.set_boot to use check_esp_err, and
uses that function for a unit test.
This commit adds an idf_heap_info(capabilities) method to the esp32 module
which returns info about the ESP-IDF heaps. It's useful to get a bit of a
picture of what's going on when code fails because ESP-IDF can't allocate
memory anymore. Includes documentation and a test.
Add -Wdouble-promotion and -Wfloat-conversion for most ports to ban out
implicit floating point conversions, and add extra Travis builds using
MICROPY_FLOAT_IMPL_FLOAT to uncover warnings which weren't found
previously. For the unix port -Wsign-comparison is added as well but only
there since only clang supports this but gcc doesn't.
Now that error string compression is supported it's more important to have
consistent error string formatting (eg all lowercase English words,
consistent contractions). This commit cleans up some of the strings to
make them more consistent.
TimeoutError was added back in 077812b2ab for
the cc3200 port. In f522849a4d the cc3200
port enabled use of it in the socket module aliased to socket.timeout. So
it was never added to the builtins. Then it was replaced by
OSError(ETIMEDOUT) in 047af9b10b.
The esp32 port enables this exception, since the very beginning of that
port, but it could never be accessed because it's not in builtins.
It's being removed: 1) to not encourage its use; 2) because there are a lot
of other OSError subclasses which are not defined at all, and having
TimeoutError is a bit inconsistent.
Note that ports can add anything to the builtins via MICROPY_PORT_BUILTINS.
And they can also define their own exceptions using the
MP_DEFINE_EXCEPTION() macro.
This commit changes the default filesystem type for esp32 to littlefs v2.
This port already enables both VfsFat and VfsLfs2, so either can be used
for the filesystem, and existing systems that use FAT will still work.
Move extmod/modbluetooth_nimble.* to extmod/nimble. And move common
Makefile lines to extmod/nimble/nimble.mk (which was previously only used
by stm32). This allows (upcoming) btstack to follow a similar structure.
Work done in collaboration with Jim Mussared aka @jimmo.
This string is recognised by uncrustify, to disable formatting in the
region marked by these comments. This is necessary in the qstrdef*.h files
to prevent modification of the strings within the Q(...). In other places
it is used to prevent excessive reformatting that would make the code less
readable.
This provides a more consistent C-level API to raise exceptions, ie moving
away from nlr_raise towards mp_raise_XXX. It also reduces code size by a
small amount on some ports.
Previous behaviour is when this argument is set to "true", in which case
the function will raise any pending exception. Setting it to "false" will
cancel any pending exception.
This modifies the signature of mp_thread_set_state() to use
mp_state_thread_t* instead of void*. This matches the return type of
mp_thread_get_state(), which returns the same value.
`struct _mp_state_thread_t;` had to be moved before
`#include <mpthreadport.h>` since the stm32 port uses it in its
mpthreadport.h file.
Make version 4.1 and lower does not allow $call as the main expression on a
line, so assign the result of the $call to a dummy variable.
Fixes issue #5426.
Instances of the slice class are passed to __getitem__() on objects when
the user indexes them with a slice. In practice the majority of the time
(other than passing it on untouched) is to work out what the slice means in
the context of an array dimension of a particular length. Since Python 2.3
there has been a method on the slice class, indices(), that takes a
dimension length and returns the real start, stop and step, accounting for
missing or negative values in the slice spec. This commit implements such
a indices() method on the slice class.
It is configurable at compile-time via MICROPY_PY_BUILTINS_SLICE_INDICES,
disabled by default, enabled on unix, stm32 and esp32 ports.
This commit also adds new tests for slice indices and for slicing unicode
strings.
Move webrepl support code from ports/esp8266/modules into extmod/webrepl
(to be alongside extmod/modwebrepl.c), and use frozen manifests to include
it in the build on esp8266 and esp32.
A small modification is made to webrepl.py to make it work on non-ESP
ports, i.e. don't call dupterm_notify if not available.
Implements text, rodata and bss generalised relocations, as well as generic
qstr-object linking. This allows importing dynamic native modules on all
supported architectures in a unified way.
The compile-time configuration value MICROPY_HW_RTC_USER_MEM_MAX can now be
used to define the amount of memory set aside for RTC.memory(). If this
value is configured to zero then the RTC.memory functionality is not
included in the build.
The IDF heap is more fragmented with IDF 4 and mbedtls cannot allocate
enough RAM with 16+16kiB for both in and out buffers, so reduce output
buffer size.
Fixes issue #5303.
This commit removes the Makefile-level MICROPY_FATFS config and moves the
MICROPY_VFS_FAT config to the Makefile level to replace it. It also moves
the include of the oofatfs source files in the build from each port to a
central place in extmod/extmod.mk.
For a port to enabled VFS FAT support it should now set MICROPY_VFS_FAT=1
at the level of the Makefile. This will include the relevant oofatfs files
in the build and set MICROPY_VFS_FAT=1 at the C (preprocessor) level.
This commit adds support for littlefs (v2) on all esp32 boards.
The original FAT filesystem still works and any board with a preexisting
FAT filesystem will still work as normal. It's possible to switch to
littlefs by reformatting the block device using:
import uos, flashbdev
uos.VfsLfs2.mkfs(flashbdev.bdev)
Then when the board reboots (soft or hard) the new littlefs filesystem will
be mounted. It's possible to switch back to a FAT filesystem by formatting
with uos.VfsFat.mkfs(flashbdev.bdev).
When a SPI bus is initialized with a SPI host that is currently in use the
exception msg incorrectly indicates "SPI device already in use". The
mention of "device" in the exception msg is confusing because the error is
about trying to use a SPI host that is already claimed. A better exception
msg is "SPI host already in use".
For consistency with "umachine". Now that weak links are enabled
by default for built-in modules, this should be a no-op, but allows
extension of the bluetooth module by user code.
Also move registration of ubluetooth to objmodule rather than
port-specific.
This commit implements automatic module weak links for all built-in
modules, by searching for "ufoo" in the built-in module list if "foo"
cannot be found. This means that all modules named "ufoo" are always
available as "foo". Also, a port can no longer add any other weak links,
which makes strict the definition of a weak link.
It saves some code size (about 100-200 bytes) on ports that previously had
lots of weak links.
Some changes from the previous behaviour:
- It doesn't intern the non-u module names (eg "foo" is not interned),
which saves code size, but will mean that "import foo" creates a new qstr
(namely "foo") in RAM (unless the importing module is frozen).
- help('modules') no longer lists non-u module names, only the u-variants;
this reduces duplication in the help listing.
Weak links are effectively the same as having a set of symbolic links on
the filesystem that is searched last. So an "import foo" will search
built-in modules first, then all paths in sys.path, then weak links last,
importing "ufoo" if it exists. Thus a file called "foo.py" somewhere in
sys.path will still have precedence over the weak link of "foo" to "ufoo".
See issues: #1740, #4449, #5229, #5241.
When loading a manifest file, e.g. by include(), it will chdir first to the
directory of that manifest. This means that all file operations within a
manifest are relative to that manifest's location.
As a consequence of this, additional environment variables are needed to
find absolute paths, so the following are added: $(MPY_LIB_DIR),
$(PORT_DIR), $(BOARD_DIR). And rename $(MPY) to $(MPY_DIR) to be
consistent.
Existing manifests are updated to match.
Remove the 240MHz CPU config option from sdkconfig.base and create a new
sdkconfig.240mhz file for those boards that want to use 240MHz on boot.
The default CPU frequency is now 160MHz (was 240MHz), to align with the ESP
IDF and support more boards (eg those with D2WD chips).
Fixes issue #5169.
This prevents issues with concurrent access to the ringbuf.
MICROPY_BEGIN_ATOMIC_SECTION is only atomic to the same core. We could
address this with a mutex, but it's also not safe to call mp_sched_schedule
across cores.
This avoids a confusing ENOMEM raised from gap_advertise if there is
currently an active connection. This refers to the static connection
buffer pre-allocated by Nimble (nothing to do with MicroPython heap
memory).
This commit adds support for a second supported hash (currently set to the
4.0-beta1 tag). When this hash is detected, the relevant changes are
applied.
This allows to start using v4 features (e.g. BLE with Nimble), and also
start doing testing, while still supporting the original, stable, v3.3 IDF.
Note: this feature is experimental, not well tested, and network.LAN and
network.PPP are currently unsupported.
This patch uses the newly-added esp32.Partition class to replace the
existing FlashBdev class. Partition objects implement the block protocol
so can be directly mounted via uos.mount(). This has the following
benefits:
- allows the filesystem partition location and size to be specified in
partitions.csv, and overridden by a particular board
- very easily allows to have multiple filesystems by simply adding extra
entries to partitions.csv
- improves efficiency/speed of filesystem operations because the block
device is implemented fully in C
- opens the possibility to have encrypted flash storage (since Partitions
can be encrypted)
Note that this patch is fully backwards compatible: existing filesystems
remain untouched and work with this new code.
As per PEP 485, this function appeared in for Python 3.5. Configured via
MICROPY_PY_MATH_ISCLOSE which is disabled by default, but enabled for the
ports which already have MICROPY_PY_MATH_SPECIAL_FUNCTIONS enabled.
Replaces the `SDKCONFIG` makefile variable with `BOARD`. Defaults to
BOARD=GENERIC. spiram can be enabled with `BOARD=GENERIC_SPIRAM`
Add example definition for TINYPICO, currently identical to GENERIC_SPIRAM
but with custom board/SoC names for the uPy banner.
They are both enabled by default, but can be disabled by defining
MICROPY_HW_ENABLE_MDNS_QUERIES and/or MICROPY_HW_ENABLE_MDNS_RESPONDER to
0. The hostname for the responder is currently taken from
tcpip_adapter_get_hostname() but should eventually be configurable.
This commit adds the connect() method to the PPP interface and requires
that connect() be called after active(1). This is a breaking change for
the PPP API.
With the connect() method it's now possible to pass in authentication
information for PAP/CHAP, eg:
ppp.active(1)
ppp.connect(authmode=ppp.AUTH_PAP, username="user", "password="password")
If no authentication is needed simply call connect() without any
parameters. This will get the original behaviour of calling active(1).
On this port the GIL is enabled and everything works under the assumption
of the GIL, ie that a given task has exclusive access to the uPy state, and
any ISRs interrupt the current task and therefore the ISR inherits
exclusive access to the uPy state for the duration of its execution.
If the MicroPython tasks are not pinned to a specific core then an ISR may
be executed on a different core to the task, making it possible for the
main task and an ISR to execute in parallel, breaking the assumption of the
GIL.
The easiest and safest fix for this is to pin all MicroPython related code
to the same CPU core, as done by this patch. Then any ISR that accesses
MicroPython state must be registered from a MicroPython task, to ensure it
is invoked on the same core.
See issue #4895.
Without this you often don't get any DNS server from your network provider.
Additionally, setting your own DNS _does not work_ without this option set
(which could be a bug in the PPP stack).
WIFI_REASON_AUTH_FAIL does not necessarily mean the password is wrong, and
a wrong password may not lead to a WIFI_REASON_AUTH_FAIL error code. So to
improve reliability connecting to a WLAN always reconnect regardless of the
error.
This updates ESP IDF to use v3.3-beta3. And also adjusts README.md to
point to stable docs which provide a link to download the correct toolchain
for this IDF version, namely 1.22.0-80-g6c4433a-5.2.0
This adds support for SD cards using the ESP32's built-in hardware SD/MMC
host controller, over either the SDIO bus or SPI. The class is available
as machine.SDCard and using it can be as simple as:
uos.mount(machine.SDCard(), '/sd')
