Commit Graph

1207 Commits

Author SHA1 Message Date
Damien George
d75f49c0f0 tools/ci.sh: Don't print python2 version.
Because python2 may not be available.

Signed-off-by: Damien George <damien@micropython.org>
2022-11-23 10:32:55 +11:00
Scott Shawcroft
167a50658a
Try and fix ci change script 2022-11-22 09:42:59 -08:00
MicroDev
7ea563e661
fix json parse issue 2022-11-19 23:29:05 +05:30
MicroDev
4f2f489df4
ignore espressif_esp32s3_devkitc_1_n32r8 in vid:pid check 2022-11-18 09:54:40 +05:30
Scott Shawcroft
c525322a29
Merge pull request #7132 from MicroDev1/ci
CI: Schedule PR jobs based on commit specific changes
2022-11-17 10:37:07 -08:00
Dominic Davis-Foster
ca84d79809
Allow duplicate VID and PID in CI check. 2022-11-15 23:06:36 +00:00
Damien George
3927ce6035 tools/ci.sh: Remove one build from zephyr CI to speed it up.
The zephyr CI takes the most time out of all CI jobs, so remove the
standard qemu_x86 build to speed it up.  The remaining builds should still
cover enough cases to catch errors.

Signed-off-by: Damien George <damien@micropython.org>
2022-11-16 10:05:17 +11:00
iabdalkader
3bae10417e tools/ci.sh: Add ARDUINO_NANO_RP2040_CONNECT to rp2 CI build. 2022-11-16 09:47:06 +11:00
Jim Mussared
1504204606 tools/gendoc.py: Remove unused script.
This used to be used to generate .rst docs from inline comments in the C
code (specifically for APIs) but is now unused.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-11-15 23:17:26 +11:00
Andrew Scheller
02ad71468f tools/gen-cpydiff.py: Use os.path.join and os.path.isdir.
Makes path handling clearer and simpler.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-11-15 23:17:26 +11:00
Jeff Epler
dec802a73b
build docs when .github/workflows changes
Closes: #7205
2022-11-13 13:26:46 -06:00
Angus Gratton
2e6e53057b shared/tinyusb: Further refactor static USB device implementation.
App the mp_ prefix to usbd_ symbols and files which are defined here and
not in TinyUSB.

rp2 only for now. This includes some groundwork for dynamic USB devices
(defined in Python).

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2022-11-11 16:47:36 +11:00
Damien George
b52fe52d31 tools/makemanifest.py: Fix printing of mpy_cross.CrossCompileError.
This change was missed by 5852fd7708

Signed-off-by: Damien George <damien@micropython.org>
2022-11-02 14:06:18 +11:00
Damien George
3ed017677b tools/mpremote: Make RemoteFile objects iterable.
So that filesystems mounted with "mpremote mount" can have their files
iterated over, making them consistent with other files.

Signed-off-by: Damien George <damien@micropython.org>
2022-10-31 12:40:17 +11:00
Jim Mussared
3427e12e8f tools/mpremote: Handle FileNotFound exceptions in listdir.
The except handler for OSError didn't include the line that actually calls
os.listdir, so an invalid path wasn't handled correctly.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-31 12:38:25 +11:00
microDev
30f07fbf22
schedule pr jobs based on commit specific changes 2022-10-26 23:19:13 +05:30
Jim Mussared
1ba0e8ff96 py/persistentcode: Only emit sub-version if generated code has native.
In order for v1.19.1 to load a .mpy, the formerly-feature-flags which are
now used for the sub-version must be zero.

The sub-version is only used to indicate a native version change, so it
should be zero when emitting bytecode-only .mpy files.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-25 14:57:04 +11:00
Jeff Epler
d08b43f704
We refer to port multiple times, make it convenient 2022-10-14 11:33:04 -05:00
Jeff Epler
641a398a54
only build subset of boards for bindings changes 2022-10-14 11:33:03 -05:00
Jeff Epler
3b600ac9b3
Potentially compute settings of fewer boards
.. when the file is within ports, just get the settings for boards
within the port
2022-10-14 11:33:03 -05:00
Jeff Epler
ad130e87f0
Parallelize finding board settings
This reduces the _elapsed_ time running the script from ~90s to ~15s
on my AMD Ryzen 7 5700U. The CPU time is still around 2 minutes.
2022-10-14 11:33:03 -05:00
Jeff Epler
cab4063057
Make it easier to locally test ci_set_matrix
Now you can e.g., `tools/ci_set_matrix.py ports/raspberrypi/mpconfigport.h`
and see what outputs would be set.
2022-10-14 11:33:02 -05:00
Jeff Epler
81154b4ee6
Revert "don't build other ports due to common-hal changes"
This reverts commit 91985cef7e.
2022-10-14 11:32:58 -05:00
Dan Halbert
91985cef7e don't build other ports due to common-hal changes 2022-10-14 11:26:37 -04:00
Jeff Epler
a31ad57225
Use non-deprecated way to set build outputs
see https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
2022-10-12 13:45:30 -05:00
Jim Mussared
87011f6353 extmod/extmod.mk: Make extmod.mk handle GIT_SUBMODULES.
This applies to nimble, btstack, axtls, mbedtls, lwip.

Rather than having the ports individually manage GIT_SUBMODULES for these
components, make extmod.mk append them when the relevant feature is
enabled.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-11 23:29:09 +11:00
Dan Halbert
e0517c7379
Merge pull request #6999 from jepler/picow-ssl
pico_w: implement ssl with caveats
2022-10-06 10:51:36 -04:00
Jim Mussared
b8982ec5f9 tools/verifygitlog.py: Add additional help for subject line issues.
This check used to just show the regular expression that failed to match,
but the rules are pretty subtle and hard to interpret from the regular
expression alone.

Add some basic checks for the main things that go wrong:
 - Missing capitalisation.
 - Missing full-stop.
 - Missing path.
 - Single-word subject.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-07 00:26:31 +11:00
Jim Mussared
f6d06b3ce0 tools/verifygitlog.py: Ignore comment lines in commit messages.
The "signed-off" check assumes that the Signed-off-by: line is the last,
but there may me many lines of comments after this.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-07 00:19:40 +11:00
Jeff Epler
ecd140213b
populate nina-fw submodule whe needed 2022-10-06 08:14:50 -05:00
Dan Halbert
3d91ea444a run black on tools/ci_check_duplicate_usb_vid_pid.py 2022-10-05 17:37:33 -04:00
Jeff Epler
905a5957b9
allow gen_crt_bundle to generate a .S file directly
.. this gets rid of one of the steps of converting it
2022-10-05 14:56:25 -05:00
Jeff Epler
b47d1d777b
black and fix doc typo 2022-10-05 14:56:24 -05:00
Jeff Epler
b1ce1d05d7
Copy from esp-idf@d51f7d8821 2022-10-05 14:56:24 -05:00
Jeff Epler
dcb650c513 pico w: add ssl module
Note: at this time, the ssl module on pico_w never verifies the server
certificate. This means it does not actually provide a higher security
level than regular socket / http protocols.
2022-10-05 13:12:43 -04:00
Damien George
46d02c2469 tools/mpremote: Bump version to 0.4.0.
Signed-off-by: Damien George <damien@micropython.org>
2022-10-06 01:02:39 +11:00
Jim Mussared
f13134e403 tools/mpremote: Fix argument handling for follow and add help strings.
Fixes in this commit are:
- Make --follow the default for "run" (accidentally changed in 68d094358).
- Add help strings for "repl": --capture --inject-file --inject-code
- Update help strings for "run".
- Fix encoding for --inject-code (accidentally broken in 68d094358).
- Remove ability to --no-follow for "eval". It was there previously because
  it shared the same code path with "exec" and "run", but makes no sense
  for "eval", so might as well remove.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-06 00:58:31 +11:00
Kyle Mohr
b417eacc5e
Update ci_check_duplicate_usb_vid_pid.py 2022-10-04 00:03:38 -05:00
Kyle Mohr
ba2368c59e
Utilize VID/PID from native USB port 2022-10-03 23:46:23 -05:00
Angus Gratton
0e35c4de9b tools: Add pre-commit support.
Tweak the existing codeformat.py and verifygitlog.py to allow them to be
easily called by pre-commit.

(This turned out to be easier than using any existing pre-commit hooks,
without making subtle changes in the formatting.)

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2022-10-04 14:52:48 +11:00
Kyle Mohr
a948dc399b add esp32-s2-devkitc-1-n8r2 variant 2022-10-04 03:51:51 +00:00
Angus Gratton
bdac8272d8 tools: Add note about uncrustify versions.
Uncrustify versions are not mutually compatible:

1. Version 0.73 or newer produce slightly different formatting. It may be
possible to tweak these by adding more config items, but this will cause
older versions to error out with 'Unknown option'.

2. Version 0.75 prints a range of deprecation warnings due to config file
changes, and returns a non-zero exit code. These are actually fixable
as most are the default value, and pp_indent has changed from 'true' to '1'
which is backwards compatible. However issue 1 remains, so probably better
to have it fail explicitly.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2022-10-04 14:34:54 +11:00
Jim Mussared
7705b9b9d5 tools/pyboard.py: Handle unsupported fs command.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-01 22:44:24 +10:00
Jim Mussared
12ca918eb2 tools/mpremote: Add mpremote mip install to install packages.
This supports the same package sources as the new `mip` tool.
 - micropython-lib (by name)
 - http(s) & github packages with json description
 - directly downloading a .py/.mpy file

The version is specified with an optional `@version` on the end of the
package name. The target dir, index, and mpy/no-mpy can be set through
command line args.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-01 22:44:24 +10:00
Jim Mussared
68d094358e tools/mpremote: Use argparse for command line parsing.
No functional change other than to allow slightly more flexibility in how
--foo arguments are specified.

This removes all custom handling for --foo args in all commands and
replaces it with per-command argparse configs.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-01 22:44:24 +10:00
Jim Mussared
413a69b94b tools/mpremote: Simplify dispatch of commands.
No functional change.

This makes each built-in command defined by just a handler method and
simplifies a lot of the logic around tracking the board state.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-01 22:44:24 +10:00
Jeff Epler
907c5d387f
Tweak black_bindings
Originally, black_bindings found each contiguous "//|" block and sent
it to black independently. This was slower than it needed to be.

Instead, swap the comment prefix: when running black, take off
"//|" prefixes and put "##|" prefixes on all un-prefixed lines.
Then, after black is run, do the opposite operation

This more than doubles the overall speed of "pre-commit run --all",
from 3m20s to 55s CPU time on my local machine (32.5s to under 10s
"elapsed" time)

It also causes a small amount of churn in the bindings, because
black now sees enough context to know whether one 'def' follows another
or ends the 'def's in a 'class'. In the latter case, it adds an extra
newline, which becomes a "//|" line.

I'm less sure why a trailing comma was omitted before down in
rp2pio/StateMachine.c but let's roll with it.
2022-09-30 11:18:13 -05:00
Jim Mussared
282401da5c tools/manifestfile.py: Replace recursive glob with os.walk.
Recursive glob isn't supported before Python 3.5.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-30 17:34:03 +10:00
Jim Mussared
924a3e03ec top: Replace upip with mip everywhere.
Updates all README.md and docs, and manifests to `require("mip")`.

Also extend and improve the documentation on freezing and packaging.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-30 17:34:03 +10:00
Jim Mussared
65ab0ec91c tools/manifestfile.py: Add author kwarg to metadata().
This allows future micropython-lib packages to specify an author.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-29 23:53:35 +10:00
Dan Halbert
db065a299f
Merge pull request #6933 from jepler/🥧🐮
Implement a useful subset of `wifi` and `socketpool` modules on 🥧🐮
2022-09-28 18:09:24 -04:00
Jeff Epler
6c3cdceb45
Implement scan, connect, ping
My pings go out, and then they come back

```py
import os
import wifi
import ipaddress

wifi.radio.connect(os.getenv('WIFI_SSID'), os.getenv('WIFI_PASSWORD'))
ipv4 = ipaddress.ip_address("8.8.4.4")
print("Ping google.com: %f ms" % (wifi.radio.ping(ipv4)*1000))
```
2022-09-28 10:06:33 -05:00
Jeff Epler
3a0d237581
Run black_bindings during pre-commit 2022-09-27 15:22:55 -05:00
Jeff Epler
6a20aef382
new script to blacken docstrings in bindings 2022-09-27 15:20:49 -05:00
Jeff Epler
a769beed3f
Use new unique ID for this board 2022-09-22 16:23:38 -05:00
Jim Mussared
920da9c5e3 unix/variants/coverage: Add test for manifest freeze_mpy().
This uses the frozentest.mpy that is also used by ports/minimal.

Also fixes two bugs that these new tests picked up:
 - File extension matching in manifestfile.py.
 - Handling of freeze_mpy results in makemanifest.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 23:51:10 +10:00
Jim Mussared
6ecdf1a240 tests/frozen: Move frozentest.mpy from ports/ to tests/.
frozentest.mpy was previously duplicated in ports/minimal and
ports/powerpc.

This needs to be re-generated on every .mpy version increase, so might as
well just have a single copy of it.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 23:51:10 +10:00
Jim Mussared
d94141e147 py/persistentcode: Introduce .mpy sub-version.
The intent is to allow us to make breaking changes to the native ABI (e.g.
changes to dynruntime.h) without needing the bytecode version to increment.

With this commit the two bits previously used for the feature flags (but
now unused as of .mpy version 6) encode a sub-version.  A bytecode-only
.mpy file can be loaded as long as MPY_VERSION matches, but a native .mpy
(i.e. one with an arch set) must also match MPY_SUB_VERSION.  This allows 3
additional updates to the native ABI per bytecode revision.

The sub-version is set to 1 because the previous commits that changed the
layout of mp_obj_type_t have changed the native ABI.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
2022-09-19 23:19:55 +10:00
Jim Mussared
c1530a0ce8 unix: Refactor mpconfigport.h and mpconfigvariant.h.
This is a no-op for coverage and minimal.

The standard and dev variants have been merged and enable the same feature
set as a typical bare-metal board.  And remove the CI for the dev build.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-13 17:39:03 +10:00
Wind-stormger
57fd66b80f tools/pyboard.py: Support Windows pathname separators.
Addresses issue #9132.
2022-09-13 13:28:39 +10:00
Andrew Leech
4e0964b59f extmod/vfs: Add finaliser to ilistdir to close directory handle.
When iterating over filesystem/folders with os.iterdir(), an open file
(directory) handle is used internally.  Currently this file handle is only
closed once the iterator is completely drained, eg. once all entries have
been looped over / converted into list etc.

If a program opens an iterdir but does not loop over it, or starts to loop
over the iterator but breaks out of the loop, then the handle never gets
closed.  In this state, when the iter object is cleaned up by the garbage
collector this open handle can cause corruption of the filesystem.

Fixes issues #6568 and #8506.
2022-09-13 13:00:42 +10:00
Jim Mussared
4903e48e34 tools/makemanifest.py: Force the repo version of the mpy_cross package.
In case the version from pypi is installed or some other version is
available in sys.path, prepend `$(TOP)/mpy-cross` to sys.path instead.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-09 18:12:55 +10:00
Dan Halbert
dd37c81136
Merge pull request #6847 from dhalbert/status-bar-control
Allow enabling and disabling of status bar
2022-09-08 10:48:36 -04:00
Jim Mussared
9070a24940 tools/autobuild: Use distinct directory for building stm32 variants.
Previous the build directory just used the board name, now make it use the
variant name too.

This shouldn't have any change because the existing directory should not
exist (all builds run by these scripts remove their build directory after
completion), but it makes debugging easier.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-08 14:54:11 +10:00
Jim Mussared
5852fd7708 tools/manifestfile.py: Allow manifests to set metadata.
The metadata can be version, description, and license.

After executing a manifest, the top-level metadata can be queried, and also
each file output from the manifest will have the metadata of the
containing manifest.

Use the version metadata to "tag" files before freezing such that they have
__version__ available.
2022-09-05 17:07:13 +10:00
Jim Mussared
bc23f207ce tools/manifestfile.py: Allow require() to specify unix packages.
By default, don't include micropython-lib/unix-ffi in the search.

If unix_ffi=True is passed to require(), then include unix-ffi and make it
take precedence over the other locations (e.g. python-stdlib).

This does two things:
 - Prevents non-unix builds from using unix-only packages.
 - Allows the unix build to optionally use a more full-featured (e.g. ffi)
   based package, even with the same name as one from e.g. stdlib.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-05 17:06:52 +10:00
Jim Mussared
e9a28ce312 tools/manifestfile.py: Allow include of directory path.
If an include path is a directory, then it implicitly grabs the manifest.py
file inside that directory. This simplifies most manifest.py files.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-05 17:06:52 +10:00
Jim Mussared
6bd0ec7a70 tools/makemanifest.py: Update to use mpy_cross module.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-05 17:06:43 +10:00
Jim Mussared
f3cdb052db tools/manifestfile.py: Add library for working with manifests.
This splits the manifest file loading logic from makemanifest.py and
updates makemanifest.py to use it.

This will allow non-freezing uses of manifests, such as defining packages
and dependencies in micropython-lib.

Also adds additional methods to the manifest "API":
 - require() - to get a package from micropython-lib.
 - module() - to define a single-file module
 - package() - to define a multi-file package

module() and package() should replace most uses of freeze() and can also
be also used in non-freezing scenarios.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-05 17:00:43 +10:00
Dan Halbert
52080e24eb status bar control 2022-08-30 15:23:44 -04:00
Damien George
730e975091 esp32/boards: Merge manifest_release modules into standard manifest.
Having two separate manifests is confusing.  It's simpler to have the daily
builds use the same configuration as the stable, release builds.

Signed-off-by: Damien George <damien@micropython.org>
2022-08-30 11:34:34 +10:00
Jeff Epler
cb202ea344
Check uniqueness of creatorid:creationid
Turns out we already have one duplicate, so I 'endorsed' it, but probably
one should be reassigned.

Closes: #8628
2022-08-27 21:32:58 -05:00
Damien George
2e386bcf76 tools/mpremote: Print nicer errors for unsupported 'cp -r' arguments.
Also document support for 'cp :a :b'.

Signed-off-by: Damien George <damien@micropython.org>
2022-08-26 13:30:03 +10:00
Damien George
f5fedf4676 tools/pyboard.py: Add fs_cp function for direct device-to-device copy.
Signed-off-by: Damien George <damien@micropython.org>
2022-08-26 13:30:03 +10:00
Damien George
24f1161fe2 tools/pyboard.py: Remove implicit fs_put if source starts with ./.
Signed-off-by: Damien George <damien@micropython.org>
2022-08-26 13:30:03 +10:00
Damien George
858707181d tools/mpremote: Print a nicer error when a filesystem command fails.
Signed-off-by: Damien George <damien@micropython.org>
2022-08-26 13:30:03 +10:00
Damien George
1855df6361 stm32: Remove support for CC3000 WiFi driver.
It has been about 8 years since support for this chip was added.  Reasons
to remove it are:
- It is no longer easy to obtain this part.
- There are now many other options for WiFi.
- It's not a good use of developer time to maintain it.

Signed-off-by: Damien George <damien@micropython.org>
2022-08-26 12:51:37 +10:00
Jim Mussared
492ba5eaf2 stm32: Move board variant config to mpconfigboard.mk.
Rather than having the autobuild know about the particular variants, have
the mpconfigboard.mk describe them and make autobuild discover them
automatically.

Adds a "query-variants" target to stm32/Makefile to allow the set of
possible variants to be queried.

Removes pybv3 from the autobuild as this isn't use by the downloads page.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-26 09:38:38 +10:00
robert-hh
f6ec01d1da tools/ci.sh: Split the stm32 builds for wiznet5k and cc3k.
- Add lib/wiznet5k into the 'make submodules' step.
- Split the stm32 builds for wiznet5k and cc3k.
- Run 'make .... clean' after making the wiznet5k build.
2022-08-23 15:00:00 +10:00
Nicholas H.Tollervey
af54d2ce9f
javascript: Rename this port to 'webassembly'. 2022-08-22 12:03:39 +01:00
Jim Mussared
59e3348c10 tools/mpremote: Add "edit" command.
This allows a remote file to be edited locally by copying it over, running
the local editor, then copying it back.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-18 22:29:13 +10:00
Jim Mussared
263737ecfe tools/pyboard.py: Add "touch" filesystem command.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-18 22:24:25 +10:00
Jim Mussared
a311e9e3d4 tools/mpremote: Allow + terminator for fs commands.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-18 22:22:36 +10:00
Damien George
cbc9f944c4 tests,tools: Update path to unix micropython executable.
These were missed by 47c84286e8

Signed-off-by: Damien George <damien@micropython.org>
2022-08-18 11:47:58 +10:00
Scott Shawcroft
1ae88c3131
Merge remote-tracking branch 'adafruit/main' into design_guide_update 2022-08-11 10:52:06 -07:00
Damien George
d53c3b6ade unix/variants: Remove variant suffix from executable filename.
The executable now lives in the build directory, and since the build
directory already contains the variant name there is no need to also add
it to the executable.

Signed-off-by: Damien George <damien@micropython.org>
2022-08-11 13:34:34 +10:00
Daniel Jour
409995ac68 tools/ci.sh: Force mpy-cross build for samd and teensy. 2022-08-11 13:33:51 +10:00
Daniel Jour
47c84286e8 all: Fix paths to mpy-cross and micropython binaries.
Binaries built using the Make build system now no longer appear in the
working directory of the build, but rather in the build directory.  Thus
some paths had to be adjusted.
2022-08-11 13:31:13 +10:00
Scott Shawcroft
9c6c8b5d62
Don't build boards for docs changes 2022-08-10 16:31:50 -07:00
Dan Ellis
9b5e00fcc5 py/formatfloat: Format all whole-number floats exactly.
Formerly, py/formatfloat would print whole numbers inaccurately with
nonzero digits beyond the decimal place.  This resulted from its strategy
of successive scaling of the argument by 0.1 which cannot be exactly
represented in floating point.  The change in this commit avoids scaling
until the value is smaller than 1, so all whole numbers print with zero
fractional part.

Fixes issue #4212.

Signed-off-by: Dan Ellis dan.ellis@gmail.com
2022-08-09 13:43:47 -04:00
Scott Shawcroft
167108a91f
Fix builds with displayio but not terminalio 2022-08-04 16:34:25 -07:00
Scott Shawcroft
3a2bcbc5c7
Enable title bar on all builds
* Tweak scroll area position so last line is complete and top is
  under the title bar.
* Pick Blinka size based on the font to minimize unused space in
  title bar. Related to #2791
* Update the title bar after terminal is started. Fixes #6078

Fixes #6668
2022-08-04 16:33:10 -07:00
Jim Mussared
58bed5ec14 tools/ci.sh: Initialise submodules for more ports.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-03 00:08:28 +10:00
Scott Shawcroft
51006f5eeb
Merge branch 'main' into webflow_edit_page 2022-07-29 10:03:07 -07:00
Scott Shawcroft
bb734c75d7
Improve build matrix computation
If a changed file is in frozen, supervisor, shared-bindings,
shared-module or common-hal, then compute board settings to
determine if it is impacted.

This should reduce the boards built for changes that are in those
directories.
2022-07-26 15:33:13 -07:00
Neradoc
f472996e80 move get_board_mapping to shared_binding_matrix because it just uses a list from there
uppercase some of the configuration lists and dicts
properly split the extensions into a list
2022-07-26 18:15:59 +02:00
Neradoc
eabe8b971a list extensions instead of macros names ("bin,uf2" not BIN_UF2)
the modules_support_matrix usees a dictionnary per board instead of a list
optionally include the frozen modules URLs in it
2022-07-26 18:15:59 +02:00
Neradoc
d021d9ae4a Move build extensions to board directories
- define CIRCUITPY_BUILD_EXTENSIONS to predefined values
- set CIRCUITPY_BUILD_EXTENSIONS in port and board config
- reuse the support matrix "get_settings_from_makefile" to get it
- move the existing port and board specific values
- remove the C3 specific board values because it's not the default
- update build_release_files.py to use get_settings_from_makefile
2022-07-26 18:15:50 +02:00
Dan Ellis
f9cbe6bc47 py/formatfloat: Format all whole-number floats exactly.
Formerly, py/formatfloat would print whole numbers inaccurately with
nonzero digits beyond the decimal place.  This resulted from its strategy
of successive scaling of the argument by 0.1 which cannot be exactly
represented in floating point.  The change in this commit avoids scaling
until the value is smaller than 1, so all whole numbers print with zero
fractional part.

Fixes issue #4212.

Signed-off-by: Dan Ellis dan.ellis@gmail.com
2022-07-26 22:23:47 +10:00
foamyguy
d0782b60cb Merge branch 'main' into webflow_edit_page
# Conflicts:
#	supervisor/shared/web_workflow/web_workflow.c
2022-07-25 19:20:05 -05:00
Scott Shawcroft
47f718aa92
Merge pull request #6615 from askpatrickw/beetle-esp32-c3
Beetle esp32 c3
2022-07-25 15:00:05 -07:00
Patrick
71c22232f0 board config compelted 2022-07-17 11:48:45 -07:00
foamyguy
584e27139e add back /edit/ handler and specify ES6 quote chars for jsmin 2022-07-15 19:49:03 -05:00
Scott Shawcroft
bad080a7bb
Merge pull request #6588 from jepler/signal-errors-ci-fetch-deps
signal errors in ci_fetch_deps subprocesses
2022-07-15 11:17:59 -07:00
Jeff Epler
8cdbeb62f6
Pass check=False for an expected-fail command 2022-07-14 15:16:28 -05:00
Scott Shawcroft
ac460dd1e1
Merge branch 'main' into esp32 2022-07-13 15:30:53 -07:00
Jeff Epler
104ac5b965
signal errors in ci_fetch_deps subprocesses
A recent build failed. The original error seemed to be during ci_fetch_deps
where a build message said
```
  fatal: reference is not a tree: 346c936e14c6ea3a8d3d65cb1fa46202dc92999d
  fatal: Unable to checkout '346c936e14c6ea3a8d3d65cb1fa46202dc92999d' in submodule path 'extmod/ulab'
```
(along with other problems), but this step didn't signal failure to
github actions.

By adding the check= parameter, a failure of the subprocess will cause
a CalledProcessError to be raised, which will make ci_fetch_deps exit with
nonzero status. In turn, this should let Actions understand that something
went wrong with this step, instead of waiting for some subsequent step(s)
to go wrong.
2022-07-13 14:56:22 -05:00
iabdalkader
2076f2efcc tools/autobuild: Add nrf port to autobuild scripts. 2022-07-13 16:19:22 +10:00
Scott Shawcroft
23bbb19138
Merge remote-tracking branch 'adafruit/main' into websocket_serial 2022-07-12 13:15:38 -07:00
Scott Shawcroft
15fe386457
Fix build and minify html and js 2022-07-12 11:13:17 -07:00
Dan Halbert
fa62c87536 ESP32 .bin only; update nvm.toml 2022-07-08 20:37:47 -04:00
Dan Halbert
2c7ad7a39f make translate and pre-commit formatting fixes 2022-07-08 16:53:53 -04:00
Dan Halbert
a9d4f09360 Update ci_check_duplicate_usb_vid_pid.py for ESP32 2022-07-08 16:11:01 -04:00
Damien George
80bc9b3dee tools/mpremote: Don't be verbose when using cat command.
Fixes issue #8828.

Signed-off-by: Damien George <damien@micropython.org>
2022-07-08 22:28:26 +10:00
Damien George
7e5137e0ae tools/pyboard.py: Add verbose option to filesystem_command.
Signed-off-by: Damien George <damien@micropython.org>
2022-07-08 22:26:41 +10:00
Patrick
03c1871d6b Folders have underscores not dashes 2022-07-01 14:50:46 -07:00
Patrick
f95edd93a0 C3 boards are Bin only 2022-07-01 14:50:46 -07:00
Scott Shawcroft
3cd05291d0
Static files + welcome page 2022-06-27 13:39:09 -07:00
Scott Shawcroft
7543dd9af0
CORS works and autoreload too 2022-06-27 13:39:08 -07:00
Scott Shawcroft
eeb9b7881e
Disable/enable buttons 2022-06-27 13:39:08 -07:00
Scott Shawcroft
d6eaaafee8
Basic JS 2022-06-27 13:39:08 -07:00
Matthew McGowan
d03bba3adb feat(swan_r5): Enable UF2 support for Swan R5 board 2022-06-21 14:47:33 -07:00
David Lechner
17f0297c1a tools/ci.sh: Drop ppa requirement for code formatting CI.
The CI scripts were using a PPA to get a backported version of uncrustify
on Ubuntu 20.04.  However, this causes CI to intermittently fail due to
connection issues to launchpad.net or the key server.

Ubuntu 22.04 has a newer version of uncrustify removing the need for the
PPA.  Ubuntu 22.04 is now in beta on GitHub actions, so it can be used.

Signed-off-by: David Lechner <david@pybricks.com>
2022-06-17 16:56:41 +10:00
Maureen Helm
bada8b3cad zephyr: Upgrade to Zephyr v3.1.0.
Updates the Zephyr port build instructions and CI to use the latest Zephyr
release tag.

Tested on frdm_k64f.

Signed-off-by: Maureen Helm <maureen.helm@intel.com>
2022-06-17 16:51:34 +10:00
Damien George
dea1ba3309 tools/metrics.py: Add mimxrt and renesas-ra to list of available ports.
Signed-off-by: Damien George <damien@micropython.org>
2022-06-17 14:57:58 +10:00
Scott Shawcroft
4e70781c45
Remove extra glyph padding 2022-06-13 09:21:41 -07:00
Damien George
1f1afae622 examples/natmod/features3: Add example to test more natmod features.
Signed-off-by: Damien George <damien@micropython.org>
2022-06-10 16:42:43 +10:00
Damien George
17a0d65ee4 tools/mpy_ld.py: Support GOT entries that reference inside mp_fun_table.
Signed-off-by: Damien George <damien@micropython.org>
2022-06-10 16:42:43 +10:00
Scott Shawcroft
6446010753
Wi-Fi autoconnect and title bar status
This adds support for CIRCUITPY_WIFI_SSID and CIRCUITPY_WIFI_PASSWORD
in `/.env`. When both are defined, CircuitPython will attempt to
connect to the network even when user code isn't running. If the
user code attempts to a network with the same SSID, it will return
immediately. Connecting to another SSID will disconnect from the
auto-connected network. If the user code initiates the connection,
then it will be shutdown after user code exits. (Should match <8
behavior.)

This PR also reworks the default displayio terminal. It now supports
a title bar TileGrid in addition to the (newly renamed) scroll area.
The default title bar is the top row of the display and is positioned
to the right of the Blinka logo when it is enabled. The scroll area
is now below the Blinka logo.

The Wi-Fi auto-connect code now uses the title bar to show its
state including the IP address when connected. It does this through
the "standard" OSC control sequence `ESC ] 0 ; <s> ESC \` where <s>
is the title bar string. This is commonly supported by terminals
so it should work over USB and UART as well.

Related to #6174
2022-06-09 14:55:54 -07:00
Damien George
313f082896 tools/ci.sh: Build native .mpy examples for armv7emsp, xtensawin in CI.
Signed-off-by: Damien George <damien@micropython.org>
2022-06-09 13:24:08 +10:00
Damien George
abb3850398 tools/mpy_ld.py: Support R_XTENSA_PDIFF32 relocation.
Newer versions of the ESP-IDF's toolchain use this relocation.

Fixes issue #8436.

Signed-off-by: Damien George <damien@micropython.org>
2022-06-09 11:57:22 +10:00
Damien George
6a1dbaad1c tools/ci.sh: Build NUCLEO_H743ZI with -O2 optimisation level.
This tests the build when -O2 is used, which can lead to additional
compiler analysis and warnings.

Signed-off-by: Damien George <damien@micropython.org>
2022-06-08 15:00:59 +10:00
Damien George
5290bfaefe tools/mpremote: Bump version to 0.3.0.
Signed-off-by: Damien George <damien@micropython.org>
2022-06-07 23:23:14 +10:00
Damien George
646fcdadbf tools/mpremote: Add command to print the version.
Signed-off-by: Damien George <damien@micropython.org>
2022-06-07 23:22:04 +10:00
Damien George
2fb413b265 tools/mpy-tool.py: Improve generated frozen identifiers.
Frozen identifiers now include their full name hierarchy, eg their class
name.  This makes it easier to understand the generated code.

Signed-off-by: Damien George <damien@micropython.org>
2022-06-07 16:55:18 +10:00
Damien George
9670a156da all: Rename MICROPY_PY_WIZNET5K to MICROPY_PY_NETWORK_WIZNET5K.
To match MICROPY_PY_NETWORK_CYW43 and MICROPY_PY_NETWORK_NINAW10.

Signed-off-by: Damien George <damien@micropython.org>
2022-06-07 16:55:18 +10:00
Damien George
bf92b0cbf2 tools/ci.sh: Add test for mpy-tool's merging feature.
Signed-off-by: Damien George <damien@micropython.org>
2022-06-07 13:51:45 +10:00
Damien George
599a22e569 tools/mpy-tool.py: Rework .mpy merging feature.
Now that the native qstr link table is gone, merging a native .mpy file
with a bytecode .mpy file is not as simple as concatenating the .mpy data.
The qstr_table and obj_table tables from all merged .mpy files must now be
joined together, because they are global to the .mpy file (and hence global
to the merged .mpy file).  This means the bytecode needs to be be decoded,
qstr_table and obj_table indices updated to point to the correct entries in
the new tables, and then the bytecode re-encoded.

This commit makes this change to the merging feature in mpy-tool.py.  This
can now merge an arbitrary number of bytecode .mpy files, and up to one
native .mpy file.

Signed-off-by: Damien George <damien@micropython.org>
2022-06-07 13:51:45 +10:00
Damien George
f506bf342a py/bc: Remove unused mp_opcode_format function.
This was made redundant by f2040bfc7e, which
also did not update this function for the change to qstr-opcode encoding,
so it does not work correctly anyway.

Signed-off-by: Damien George <damien@micropython.org>
2022-06-07 13:32:38 +10:00
Damien George
b37b578214 py/persistentcode: Remove remaining native qstr linking support.
Support for architecture-specific qstr linking was removed in
d4d53e9e11, where native code was changed to
access qstr values via qstr_table.  The only remaining use for the special
qstr link table in persistentcode.c is to support native module written in
C, linked via mpy_ld.py.  But native modules can also use the standard
module-level qstr_table (and obj_table) which was introduced in the .mpy
file reworking in f2040bfc7e.

This commit removes the remaining native qstr liking support in
persistentcode.c's load_raw_code function, and adds two new relocation
options for constants.qstr_table and constants.obj_table.  mpy_ld.py is
updated to use these relocations options instead of the native qstr link
table.

Signed-off-by: Damien George <damien@micropython.org>
2022-06-07 13:19:55 +10:00
Nathan Young
761e66e85b Enable UF2 builds for STM32 Thing Plus 2022-06-04 15:21:06 +02:00
Andrew Leech
b92e51efe7 tools/ci.sh: Add build of W5100S_EVB_PICO board to rp2 CI.
This exercises the inclusion of a number of libraries in the rp2 port
including mbedtls and lwip.

Signed-off-by: Andrew Leech <andrew@alelec.net>
2022-06-03 14:35:52 +10:00
Damien George
1d047617bb tools/mpy-tool.py: Remove obsolete unicode flag in .mpy header.
This was removed in c49d5207e9

Signed-off-by: Damien George <damien@micropython.org>
2022-05-26 11:43:46 +10:00
David Lechner
d42d35f56d tests/run-tests.py: Enable -X realtime option for macOS tests.
This enables the new `-X realtime` runtime option when running tests on
macOS.  This causes MicroPython to configure all threads to be high
priority so that they are allowed to use high precision timers.  This
makes tests that depend on the passage of time more likely to succeed.

CI tests that were disabled because of this are now enabled again.

Signed-off-by: David Lechner <david@pybricks.com>
2022-05-24 00:52:44 +10:00
Damien George
c1b9d2259e py/dynruntime.mk: Add basic support for armv6m architecture.
The examples/natmod features0 and features1 examples now build and run on
ARMv6-M platforms.  More complicated examples are not yet supported because
the compiler emits references to built-in functions like __aeabi_uidiv.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-23 23:01:25 +10:00
Damien George
d4d53e9e11 py/emitnative: Access qstr values using indirection table qstr_table.
This changes the native emitter to access qstr values using the qstr
indirection table qstr_table, but only when generating native code that
will be saved to a .mpy file.  This makes the resulting native code fully
static, ie it does not require any fix-ups or rewriting when it is
imported.

The performance of native code is more or less unchanged.  Benchmark
results on PYBv1.0 (using --via-mpy and --emit native) are:

N=100 M=100          baseline -> this-commit     diff      diff% (error%)
bm_chaos.py            407.16 ->     411.85 :   +4.69 =  +1.152% (+/-0.01%)
bm_fannkuch.py         100.89 ->     101.20 :   +0.31 =  +0.307% (+/-0.01%)
bm_fft.py             3521.17 ->    3441.72 :  -79.45 =  -2.256% (+/-0.00%)
bm_float.py           6707.29 ->    6644.83 :  -62.46 =  -0.931% (+/-0.00%)
bm_hexiom.py            55.91 ->      55.41 :   -0.50 =  -0.894% (+/-0.00%)
bm_nqueens.py         5343.54 ->    5326.17 :  -17.37 =  -0.325% (+/-0.00%)
bm_pidigits.py         603.89 ->     632.79 :  +28.90 =  +4.786% (+/-0.33%)
core_qstr.py            64.18 ->      64.09 :   -0.09 =  -0.140% (+/-0.01%)
core_yield_from.py     313.61 ->     311.11 :   -2.50 =  -0.797% (+/-0.03%)
misc_aes.py            654.29 ->     659.75 :   +5.46 =  +0.834% (+/-0.02%)
misc_mandel.py        4205.10 ->    4272.08 :  +66.98 =  +1.593% (+/-0.01%)
misc_pystone.py       3077.79 ->    3128.39 :  +50.60 =  +1.644% (+/-0.01%)
misc_raytrace.py       388.45 ->     393.71 :   +5.26 =  +1.354% (+/-0.01%)
viper_call0.py         576.83 ->     566.76 :  -10.07 =  -1.746% (+/-0.05%)
viper_call1a.py        550.39 ->     540.12 :  -10.27 =  -1.866% (+/-0.11%)
viper_call1b.py        438.32 ->     432.09 :   -6.23 =  -1.421% (+/-0.11%)
viper_call1c.py        442.96 ->     436.11 :   -6.85 =  -1.546% (+/-0.08%)
viper_call2a.py        536.31 ->     527.37 :   -8.94 =  -1.667% (+/-0.04%)
viper_call2b.py        378.99 ->     377.50 :   -1.49 =  -0.393% (+/-0.08%)

Signed-off-by: Damien George <damien@micropython.org>
2022-05-23 15:43:06 +10:00
Damien George
1fb01bd6c5 py/emitnative: Put a pointer to the native prelude in child_table array.
Some architectures (like esp32 xtensa) cannot read byte-wise from
executable memory.  This means the prelude for native functions -- which is
usually located after the machine code for the native function -- must be
placed in separate memory that can be read byte-wise.  Prior to this commit
this was achieved by enabling N_PRELUDE_AS_BYTES_OBJ for the emitter and
MICROPY_EMIT_NATIVE_PRELUDE_AS_BYTES_OBJ for the runtime.  The prelude was
then placed in a bytes object, pointed to by the module's constant table.

This behaviour is changed by this commit so that a pointer to the prelude
is stored either in mp_obj_fun_bc_t.child_table, or in
mp_obj_fun_bc_t.child_table[num_children] if num_children > 0.  The reasons
for doing this are:

1. It decouples the native emitter from runtime requirements, the emitted
   code no longer needs to know if the system it runs on can/can't read
   byte-wise from executable memory.

2. It makes all ports have the same emitter behaviour, there is no longer
   the N_PRELUDE_AS_BYTES_OBJ option.

3. The module's constant table is now used only for actual constants in the
   Python code.  This allows further optimisations to be done with the
   constants (eg constant deduplication).

Code size change for those ports that enable the native emitter:
   unix x64:   +80 +0.015%
      stm32:   +24 +0.004% PYBV10
    esp8266:   +88 +0.013% GENERIC
      esp32:   -20 -0.002% GENERIC[incl -112(data)]
        rp2:   +32 +0.005% PICO

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 16:44:49 +10:00
Damien George
8725a32f41 tools/ci.sh: For esp8266 CI install esptool 3.3.1.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 16:01:56 +10:00
Damien George
f8d3956c63 tools/ci.sh: Run full test suite on stackless and settrace builds.
Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 14:25:51 +10:00
Damien George
c49d5207e9 py/persistentcode: Remove unicode feature flag from .mpy file.
Prior to this commit, even with unicode disabled .py and .mpy files could
contain unicode characters, eg by entering them directly in a string as
utf-8 encoded.

The only thing the compiler disallowed (with unicode disabled) was using
\uxxxx and \Uxxxxxxxx notation to specify a character within a string with
value >= 0x100; that would give a SyntaxError.

With this change mpy-cross will now accept \u and \U notation to insert a
character with value >= 0x100 into a string (because the -mno-unicode
option is now gone, there's no way to forbid this).  The runtime will
happily work with strings with such characters, just like it already works
with strings with characters that were utf-8 encoded directly.

This change simplifies things because there are no longer any feature
flags in .mpy files, and any bytecode .mpy will now run on any target.

Signed-off-by: Damien George <damien@micropython.org>
2022-05-17 12:51:54 +10:00