Commit Graph

599 Commits

Author SHA1 Message Date
Jim Mussared add1200343 all: Remove the zlib module.
This will be replaced with a new deflate module providing the same
functionality, with an optional frozen Python wrapper providing a
replacement zlib module.

binascii.crc32 is temporarily disabled.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 18:48:29 +10:00
Damien George 273ba28c07 tools/autobuild: Update auto-build code to build esp32 port with IDF v5.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-23 15:53:04 +10:00
Damien George 6a9db521ed github/workflows: Update esp32 CI to use IDF v5.0.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-23 15:34:53 +10:00
Damien George 0908d00453 tools/ci.sh: Add a H5 board to stm32 CI build.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-15 11:09:37 +10:00
Damien George 33b403dfb4 tools/ci.sh: Build PICO_W board as part of rp2 CI.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-14 22:20:20 +10:00
Wilko Nienhaus 034502bc72 esp32/esp32_ulp: Fix ULP (FSM) support for S2 and S3.
This change enables the ULP (FSM) for all ESP32 variants rather than
requiring it to be enabled for each board specifically.

It also ensures the correct header file is included for each variant.

Lastly, it updates the IDF version we're builing against to v4.4.2, as that
version contains important fixes to make the ULP actually work on S2/S3
chips. See: https://github.com/espressif/esp-idf/commit/a0e3d48

Signed-off-by: Wilko Nienhaus <wilko.nienhaus@gmail.com>
2023-06-14 16:41:03 +10:00
Jim Mussared b4de697ad1 tools/mpremote: Fix exec_ -> exec in commands.py.
This was missed in the pyboard refactor and is preventing `cp -r` from
working.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-09 13:36:12 +10:00
Damien George 0832fc53ce tools/ci.sh: Add mimxrt and samd ports to code size build.
The automatic code size build and GitHub comment is a really useful
feature.  This commit adds a few more builds to it (mimxrt and samd).

Signed-off-by: Damien George <damien@micropython.org>
2023-06-08 22:47:03 +10:00
Jim Mussared a1fbb1980c extmod/modtimeq: Remove timeq module.
This is a MicroPython-specific module that existed to support the old
version of uasyncio.  It's undocumented and not enabled on all ports and
takes up code size unnecessarily.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:28 +10:00
Jim Mussared 5fd042e7d1 all: Replace all uses of umodule in Python code.
Applies to drivers/examples/extmod/port-modules/tools.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared 4216bc7d13 tests: Replace umodule with module everywhere.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared eb85f4d4c9 examples/natmod: Rename umodule to module.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared 45ac651d1a all: Rename *umodule*.c to remove the "u" prefix.
Updates any includes, and references from Makefiles/CMake.

This essentially reverts what was done long ago in commit
136b5cbd76

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:17 +10:00
Jim Mussared f5f9edf645 all: Rename UMODULE to MODULE in preprocessor/Makefile vars.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:11 +10:00
Damien George a61d40e5e8 tools/mpremote: Fix use of stdout_write_bytes function.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-02 18:03:16 +10:00
Jim Mussared b4d785fa20 tools/mpremote: Detach mpremote from pyboard.py.
This commit just takes the necessary parts of pyboard.py and merges them
with pyboardextended.py to make a new transport_serial.py, and updates the
rest of mpremote to use this instead.

It is difficult to continue to add features to mpremote (which usually
requires modification to pyboard.py) while also maintaining backwards
compatibility for pyboard.py.

The idea is that this provides a starting point for further refactoring of
mpremote to allow different transports (webrepl, BLE, etc).

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 17:42:13 +10:00
Jim Mussared bd5d0163c4 docs/reference/mpremote.rst: Extend the mpremote guide.
Changes in this commit:
- Add a extra detail to each of the commands.
- Add more about handling options and arguments.
- Include shortcut commands that behave like real commands to the command
  list (e.g. bootloader, rtc).
- Add extra information and reword to address common misconceptions, in
  particular how commands chain together.
- Add additional examples showing some more interesting combinations.
- Add descriptions to each of the examples.
- Add pipx installation instructions.
- Describe how user-configuration works.

This work was sponsored by Google Season of Docs.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 16:16:28 +10:00
Jim Mussared b6b19798c2 tools/mpremote: Handle `cp` without destination.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 16:16:24 +10:00
Jim Mussared 46715e370d tools/mpremote: Add `rtc` commands to get and set the RTC.
Replaces the existing functionality provided by the `setrtc` alias to use
the current time, rather than a hard-coded date/time.

Use `rtc` to query the current time.  Use `rtc --set` to set it.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 16:15:48 +10:00
Jim Mussared d736a2f3f3 tools/mpremote: Allow terminator for shortcut commands.
For example, the `reset` shortcut previously allowed an optional delay, but
the argument handling cannot handle `reset next-command` as `next-command`
will be interpreted as the delay argument.  The fix in this commit allows
`reset + next-command`.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 16:14:26 +10:00
Jim Mussared 7c2c9ea21c tools/mpremote: Add `sleep` command.
This allows the sequence to be paused (e.g. wait for device, etc).

Also removes the t_ms arg in reset/bootloader, because these arguments
don't really need to be changed, and keeping them would mean inconsistent
units used for time delays.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 16:11:29 +10:00
Damien George ed7a3b11d9 tools/pydfu.py: Use getattr to retrieve getargspec function.
Since Python 3.11, inspect.getargspec() has been removed.

Signed-off-by: Damien George <damien@micropython.org>
2023-05-24 12:00:32 +10:00
Damien George 2771b20d29 tools/mpremote: Add repl option to escape non-printable characters.
This commit adds the "--escape-non-printable" option to the repl command.
When specified the REPL console will escape non-printable characters,
printing them as their hex value in square brackets.

This escaping behaviour was previously the default and only behaviour, but
it is now opt-in.

As part of this change, the speed of echoing device data to the console is
improved by by reading and writing in chunks.

Signed-off-by: Damien George <damien@micropython.org>
2023-05-22 14:13:24 +10:00
Jim Mussared 46d070bfee tools/manifestfile.py: Fix license capturing.
The license field was incorrectly being set to the version.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-05-19 21:56:38 +10:00
Christian Clauss a000c61d5d tools/pyboard.py: Import errno to fix undefined name in PyboardError.
This will keep line 96 from raising a NameError.

Signed-off-by: Christian Clauss <cclauss@me.com>
2023-05-09 18:49:23 +10:00
Damien George 7c645b52e3 CODECONVENTIONS: Require that commits be signed-off by the author.
And use "must" instead of "should" where appropriate in related text.

Signed-off-by: Damien George <damien@micropython.org>
2023-05-08 12:20:56 +10:00
Damien George 0264465585 tools/pyboard.py: Import serial.tools.list_ports.
This import is needed by newer versions of pyserial.

Signed-off-by: Damien George <damien@micropython.org>
2023-05-04 12:12:29 +10:00
Christian Clauss 2a1db770ce all: Fix cases of Python variable assigned but never used.
This fixes ruff rule F841.
2023-05-02 16:36:05 +10:00
Christian Clauss 8f8bd98164 all: Fix strings with backslash by using raw string literals. 2023-05-02 11:55:02 +10:00
Christian Clauss cda292935d tools/mpremote: Remove unused import of serial. 2023-05-02 11:22:37 +10:00
Damien George b1229efbd1 all: Fix spelling mistakes based on codespell check.
Signed-off-by: Damien George <damien@micropython.org>
2023-04-27 18:03:06 +10:00
Damien George e160fe7bc6 tools/pyboard.py: Rename ProcessPtyToTerminal member "ser" to "serial".
So that this file doesn't need to be excluded from codespell.

Signed-off-by: Damien George <damien@micropython.org>
2023-04-27 18:03:06 +10:00
Damien George d77c35f110 tools/ci.sh: Add functions to check code spelling using codespell.
Signed-off-by: Damien George <damien@micropython.org>

tools/ci.sh: Explicitly specify pyproject.toml.

Signed-off-by: Damien George <damien@micropython.org>

tools/ci.sh: Import tomli.

Signed-off-by: Damien George <damien@micropython.org>
2023-04-27 16:01:14 +10:00
Damien George 07a719a315 tools/ci.sh: Build both SAMD21 and SAMD51 boards as part of samd CI.
Signed-off-by: Damien George <damien@micropython.org>
2023-04-27 15:11:52 +10:00
Takeo Takahashi dc7de6ed83 renesas-ra: Change MICROPY_HW_BOARD_NAME definition to product name.
Changes in this commit:
- Change MICROPY_HW_BOARD_NAME definition to match the product name.
- Rename board folder's name to match the product name style.
- Change related files like Makefile, document descriptions, test cases, CI
  and tools.

Signed-off-by: Takeo Takahashi <takeo.takahashi.xv@renesas.com>
2023-04-27 14:12:53 +10:00
Jim Mussared a42a8022d8 extmod/modbluetooth: Make all HCI transports trace in the same format.
- Use HCI_TRACE macro consistently.
 - Use the same colour formatting.
 - Add a tool to convert to .pcap for Wireshark.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-04-26 11:37:20 +10:00
Jonas Scharpf b525f1c9ec tools/mpremote: Add ctrl-x as additonal mpremote disconnect shortcut.
The mpremote REPL can now be closed with either ctrl+] or ctrl+x, which
gives users a choice, useful if the ']' key is difficult to access.

Fixes issue #11197.

Signed-off-by: Jonas Scharpf <jonas@brainelectronics.de>
2023-04-15 00:07:40 +10:00
Jim Mussared cfd3b70934 tools/manifestfile.py: Add support for publishing packages to PyPI.
This adds a new MODE_PYPROJECT, which gives basic support to allow
packaging a small subset of micropython-lib packages to PyPI.

This change allows a package in micropython-lib to:
- Add a "pypi" name to its metadata indicating that it's based on a PyPI
  package.
- Add "stdlib" to its metadata indicating that it's a micropython version
  of a stdlib package.
- Add a "pypi_publish" name to its metadata to indicate that it can be
  published to PyPI (this can be different to the package name, e.g. "foo"
  might want to be published as "micropython-foo").

When a package requires() another one, if it's in MODE_PYPROJECT then if
the package is from pypi then it will record that as a pypi dependency
instead (or no dependency at all if it's from stdlib).

Also allows require() to explicitly specify the pypi name.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-04-07 12:53:53 +10:00
Jos Verlinde 9f74ffb6eb tools/pyboard.py: Fix ESPxx boards hanging in bootloader after reset.
This is a follow up to d263438a6e, which
solved one problem (reset on disconnect) but introduced a second one (hang
in bootloader).

To solve both probles, False/False is needed for DTR/RTS for ESPxx, but
that would then block stm32 and others.  Any unconditional combination of
DTR/RTS ends up blocking normal operation on some type of board or another.

A simple overview (for windows only):

  DTR          CTS            ESP8266/ESP32          STM32/SAMD51/RP2040
  unspecified  unspecified    Reset on disconnect    OK
  True         False          Hang in bootloader     OK
  False        False          OK                     No Repl
  True         True           Reset on disconnect    No Repl
  False        True           Reset on disconnect    No Repl

  serial.manufacturer:        wch.cn/Silicon Labs    Microsoft

  serial.description:         USB-SERIAL CH340 /     USB Serial Device
                              CP210x USB to UART
                              Bridge

The updated logic will only set the DTR/RTS signals for boards that do not
use standard Microsoft drivers (based on the manufacturer).  It would also
be possible to check against a list of known driver manufactures (like
wch.cn or Silicon Labs) but this would require a list of known drivers for
all ports.

Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
2023-04-04 13:30:05 +10:00
Damien George b5ceb9d577 tools/pyboard.py: Fix joining of path in filesystem_command.
This was broken by 5327cd1021

Signed-off-by: Damien George <damien@micropython.org>
2023-03-22 15:19:09 +11:00
Christian Clauss 4376c969f6 all: Fix Python comparison to None and True, and use "not in".
These are basic PEP8 recommendations.
2023-03-10 13:32:24 +11:00
Damien George 5327cd1021 tools/pyboard.py: Use '/' exclusively when dealing with paths.
Currently, certain mpremote filesystem operations can fail on Windows due
to a mixing of '/' and '\' for path separators.  Eg if filesystem_command()
is called with a destination that ends in / then dest.endswith(os.path.sep)
will return False, which gives the wrong behaviour (it does end in a path
separator).

For similar reasons to 7e9a15966a, it's best
to use '/' everywhere in pyboard.py and mpremote, because the target device
understands only '/'.  mpremote already does this, so the remaining place
to fix it is in pyboard.y, to convert all incoming paths to use '/' instead
of '\'.

This effectively reverts 57fd66b80f which
tried to fix the problem in a different way.

See also related 1f84440538.

Signed-off-by: Damien George <damien@micropython.org>
2023-02-24 13:08:34 +11:00
Jim Mussared ce9f7cd00a tools/mpremote: Use hatch to build mpremote package.
This allows the entire configuration to be defined in a single file,
including the logic for including pyboard.py and automatically versioning
based on the git tag.

Building the package works both via `python -m build` as well as
`hatch build`.  `python -m build ` has the advantage of automatically
fetching all dependencies, you don't need to manually install any hatch
packages.

In order to make the versioning work, and also keep things simpler for end
users, mpremote releases will now be the same as MicroPython releases and
use the same tag.  The version strings for mpremote will look like:
- X.Y.Z -- clean build at the tag
- X.Y.Z.postN+gHASH -- clean build, N revisions from the most recent tag
- X.Y.Z.postN+gHASH.dYYYYMMDD -- dirty build, N revisions from out

This commit extends on the idea from #8404.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-02-24 11:40:22 +11:00
Damien George e0d1127246 tools/ci.sh: Add unix, stm32 and rp2 ports to code size check.
Now that the code-size-check CI action gives a nice report (as a comment)
on the code size difference, it's possible to have a few more ports
reported there.  In this commit, unix, stm32 and rp2 are added.  Unix
represents non-MCU builds, and stm32 and rp2 represent ARM-based builds,
for ports that have lots of features enabled.

Signed-off-by: Damien George <damien@micropython.org>
2023-02-20 11:26:23 +11:00
Damien George 4bd4b6a47e tools/ci.sh: Install pkg-config for macos builds.
Signed-off-by: Damien George <damien@micropython.org>
2023-02-17 13:13:57 +11:00
Martin Milata 850f09b109 tools/mpy-tool.py: Initialize line_info_top.
Without it the line number mapping doesn't work.

Signed-off-by: Martin Milata <martin@martinmilata.cz>
2023-02-01 13:17:22 +11:00
Jos Verlinde d263438a6e tools/pyboard.py: Set DTR on Windows to avoid ESPxx hard reset.
Fixes issue #9659.

Signed-off-by: Jos Verlinde <Jos.Verlinde@Microsoft.com>
2023-01-13 16:51:31 +11:00
Jim Mussared aa64280666 tools/pyboard.py: Add fs_{listdir,readfile,writefile,stat}.
These are for working with the filesystem when using pyboard.py as a
library, rather than at the command line.

- fs_listdir returns a list of tuples, in the same format as os.ilistdir().
- fs_readfile returns the contents of a file as a bytes object.
- fs_writefile allows writing a bytes object to a file.
- fs_stat returns an os.statresult.

All raise FileNotFoundError (or OSError(ENOENT) on Python 2) if the file is
not found (or PyboardError on other errors).

Updated fs_cp and fs_get to use fs_stat to compute file size.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-01-13 16:38:34 +11:00
Jim Mussared 6013d27dd5 tools/pyboard.py: Add parse kwarg to eval.
This is useful when using pyboard.py as a library rather than at the
command line.

    pyb.eval("1+1") --> b"2"
    pyb.eval("{'a': '\x00'}") --> b"{'a': '\\x00'}"

Now you can also do

    pyb.eval("1+1", parse=True) --> 2
    pyb.eval("{'a': '\x00'}", parse=True) --> {'a': '\x00'}

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-01-13 16:38:34 +11:00
Jim Mussared 77002a92bf tools/pyboard.py: Fix Python 2 compatibility.
In Python 2, serial.read()[0] a string, not int. Use struct.unpack to do
this instead.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-01-13 16:38:34 +11:00