Commit Graph

558 Commits

Author SHA1 Message Date
Damien George aa3d6b6aa5 tools/ci.sh: Change esp32 CI to work with idf.py and IDF v4.0.2.
Signed-off-by: Damien George <damien@micropython.org>
2021-02-15 16:40:11 +11:00
Brianna Laugher d128999938 tools: Add filesystem action examples to pyboard.py help.
Signed-off-by: Brianna Laugher <brianna.laugher@gmail.com>
2021-02-13 14:37:28 +11:00
Damien George c9260dda23 rp2: Use local tinyusb instead of the one in pico-sdk.
So that all MicroPython ports that use tinyusb use the same version.  Also
requires fewer submodule checkouts when building rp2 along with other ports
that use tinyusb.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-12 12:56:28 +11:00
stijn c2b5bfcc0c tools: Remove obsolete upip bootstrap script.
The upip module is frozen into ports supporting it, and it is included in
the source tree, so there is no need to get it from PyPi.  Moreover the
PyPi package referred to is an out-of-date version of upip which is
basically unrelated to our upip.py because the source is taken from a fork
of micropython-lib instead of this repository.
2021-01-30 14:39:22 +11:00
Jim Mussared 2aecf378be tools/makemanifest.py: Add check that freeze path is a directory.
Avoids accidentally writing

    freeze("path/to/file.py")

and getting unexpected results.
2021-01-30 14:15:33 +11:00
stijn 37c2f507a0 github/workflows: Add workflow to verify commit message format.
Using the new tools/verifygitlog.py script.
2021-01-30 14:09:21 +11:00
stijn d48860c7dd tools/verifygitlog.py: Add script for verifying commit message format.
The main rules enforced are:
- At most 72 characters in the subject line, with a ": " in it.
- At most 75 characters per line in the body.
- No "noreply" email addresses.
2021-01-30 14:08:29 +11:00
Damien George b8f4c623f9 github/workflows: Add CI workflow for rp2 port.
Signed-off-by: Damien George <damien@micropython.org>
2021-01-30 00:42:29 +11:00
Damien George fe16e785fe tools/mpy-tool.py: List frozen modules in MICROPY_FROZEN_LIST_ITEM.
Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 23:57:10 +11:00
Damien George 5d68b5e22c tools/ci.sh: For ci_code_size_setup, update apt to install gcc-multilib.
Signed-off-by: Damien George <damien@micropython.org>
2021-01-29 23:57:10 +11:00
Damien George 203e1d2a65 tools/ci.sh: For code size build, fetch history of master branch only.
It's not necessary to fetch all branches.

Signed-off-by: Damien George <damien@micropython.org>
2021-01-24 15:02:20 +11:00
Damien George de2374cdc6 tools/ci.sh: Pip install pyhy for stm32 builds.
Signed-off-by: Damien George <damien@micropython.org>
2021-01-18 12:43:01 +11:00
Damien George ee52f89224 tools/ci.sh: Use pip-install to get latest version of esptool.py.
Because the version included in xtensa-lx106-elf-standalone.tar.gz needs
Python 2 (and pyserial for Python 2).

Signed-off-by: Damien George <damien@micropython.org>
2020-12-14 13:09:33 +11:00
Damien George 69262a11dc tools/ci.sh: Put echo of CI path in a separate function.
Because the setup functions may print other information which should not be
added to the path.

Signed-off-by: Damien George <damien@micropython.org>
2020-12-14 13:05:43 +11:00
Damien George cb1bb7592e stm32/Makefile: Change -O0 to -Og for DEBUG=1 builds.
The -Og optimisation level produces a more realistic build, gives a better
debugging experience, and generates smaller code than -O0, allowing debug
builds to fit in flash.

This commit also assigns variables in can.c to prevent warnings when -Og is
used, and builds a board in CI with DEBUG=1 enabled.

Signed-off-by: Damien George <damien@micropython.org>
2020-12-07 22:27:38 +11:00
Damien George a59282b9bf tools/pyboard.py: Add fast raw-paste mode.
This commit adds support to pyboard.py for the new raw REPL paste mode.

Note that this new pyboard.py is fully backwards compatible with old
devices (it detects if the device supports the new raw REPL paste mode).

Signed-off-by: Damien George <damien@micropython.org>
2020-12-01 22:35:13 +11:00
Damien George 547e8a9fe7 tools/ci.sh: Add helper script to run CI tasks.
The aim is for this script to be used on any CI platform, as well as run
locally.

Signed-off-by: Damien George <damien@micropython.org>
2020-11-29 22:21:28 +11:00
awachtler 56e0932485 tools/upip.py: Support explicit port number in host.
Adding a port number other then 443 to a PyPI URL may be needed if a local
server like devpi is used.
2020-10-20 12:44:30 +11:00
stijn 4b35aa5730 tools: Write msvc-compatible frozen content.
The msvc compiler doesn't accept zero-sized arrays so let the freezing
process generate compatible C code in case no modules are found and the
involved arrays are all empty.  This doesn't affect the functionality in
any way because those arrays only get accessed when mp_frozen_mpy_names
contains names, i.e.  when modules are actually found.
2020-09-11 10:51:55 +10:00
Damien George 4f2fe34623 tools/mpy-tool.py: Fix merge of multiple mpy files to POP_TOP correctly.
MP_BC_CALL_FUNCTION will leave the result on the Python stack, so that
result must be discarded by MP_BC_POP_TOP.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-09 00:11:51 +10:00
Damien George 06659077a8 all: Update Python code to conform to latest black formatting.
Updating to Black v20.8b1 there are two changes that affect the code in
this repository:

- If there is a trailing comma in a list (eg [], () or function call) then
  that list is now written out with one line per element.  So remove such
  trailing commas where the list should stay on one line.

- Spaces at the start of """ doc strings are removed.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-29 15:18:01 +10:00
Damien George 448319a745 tools/makemanifest.py: Use os.makedirs to make path for generated files.
The existing implementation of mkdir() in this file is not sophisticated
enough to work correctly on all operating systems (eg Mac can raise
EISDIR).  Using the standard os.makedirs() function handles all cases
correctly.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 11:18:48 +10:00
Martin Milata 492cf34fd8 tools/mpy-tool.py: Fix offset of line number info.
Signed-off-by: Martin Milata <martin@martinmilata.cz>
2020-08-21 16:17:07 +10:00
Michael Buesch 60cf2c0959 tools/pyboard.py: Replace eval() of received data with alternative.
Prior to this commit, pyboard.py used eval() to "parse" file data received
from the board.  Using eval() on received data from a device is dangerous,
because a malicious device may inject arbitrary code execution on the PC
that is doing the operation.

Consider the following scenario:

Eve may write a malicious script to Bob's board in his absence.  On return
Bob notices that something is wrong with the board, because it doesn't work
as expected anymore.  He wants to read out boot.py (or any other file) to
see what is wrong.  What he gets is a remote code execution on his PC.

Proof of concept:

Eve:

  $ cat boot.py
  _print = print
  print = lambda *x, **y: _print("os.system('ls /; echo Pwned!')", end="\r\n\x04")
  $ ./pyboard.py -f cp boot.py :
  cp boot.py :boot.py

Bob:

  $ ./pyboard.py -f cp :boot.py /tmp/foo
  cp :boot.py /tmp/foo
  bin   chroot  dev  home  lib32  media  opt   root  sbin  sys  usr
  boot  config  etc  lib   lib64  mnt    proc  run   srv   tmp  var
  Pwned!

There's also the possibility that the device is malfunctioning and sends
random and possibly dangerous data back to the PC, to be eval'd.

Fix this problem by using ast.literal_eval() to parse the received bytes,
instead of eval().

Signed-off-by: Michael Buesch <m@bues.ch>
2020-08-21 16:08:03 +10:00
Damien George b731bd0ce6 tools/makemanifest.py: Print nicely formatted errors from mpy-cross.
If mpy-cross exits with an error be sure to print that error in a way that
is readable, instead of a long bytes object.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-08 14:46:05 +10:00
Damien George 952de5cb77 tools/makemanifest.py: Use errno.EEXIST instead of number 17.
To make this code more portable, across different platforms.

Signed-off-by: Damien George <damien@micropython.org>
2020-07-26 10:56:24 +10:00
Thorsten von Eicken 3e758ef235 lib/mbedtls_errors: Add code to patch mbedtls for shortened error strs.
The file `mbedtls_errors/mp_mbedtls_errors.c` can be used instead of
`mbedtls/library/error.c` to give shorter error strings, reducing the build
size of the error strings from about 12-16kB down to about 2-5kB.
2020-07-20 23:53:27 +10:00
Andrew Leech 07f181a216 Revert "tools/pydfu.py: Respect longer timeouts requested by DFU dev..."
This reverts commit 4d6f60d428.

This implementation used the timeout as a maximum amount of time needed for
the operation, when actually the spec and other tools suggest that it's the
minumum delay needed between subsequent USB transfers.
2020-07-01 16:54:03 +10:00
Andrew Leech 4d6f60d428 tools/pydfu.py: Respect longer timeouts requested by DFU device/mboot. 2020-06-30 21:22:00 +10:00
Damien George 76faeed098 tools/makemanifest.py: Support freezing a subdirectory recursively.
This adds support for freezing an entire directory while keeping the
directory as part of the import path.  For example

    freeze("path/to/library", "module")

will recursively freeze all scripts in "path/to/library/module" and have
them importable as "from module import ...".

Signed-off-by: Damien George <damien@micropython.org>
2020-06-24 12:21:30 +10:00
David Lechner 77ed6f69ac tools/uncrustify: Enable more opts to remove space between func and '('.
With only `sp_func_proto_paren = remove` set there are some cases where
uncrustify misses removing a space between the function name and the
opening '('.  This sets all of the related options to `force` as well.
2020-06-19 22:07:32 +10:00
Damien George 026fda605e tools/codeformat.py: Include extmod/{btstack,nimble} in code formatting.
Signed-off-by: Damien George <damien@micropython.org>
2020-06-18 22:20:20 +10:00
David Lechner b4d0d7bf03 tools/uncrustify: Update config for v0.71.0.
This is the result of running...

    uncrustify -c tools/uncrustify.cfg --update-config-with-doc -o tools/uncrustify.cfg

...with some manual fixups to correct places where it changed things it
should not have.

Essentially it just adds new config parameters introduced in v0.71.0
with their default values.

Signed-off-by: David Lechner <david@pybricks.com>
2020-06-14 11:08:05 -05:00
David Lechner ecd7826316 tools/codeformat.py: Remove sizeof fixup.
Formatting for `* sizeof` was fixed in uncrustify v0.71, so we no longer
need the fixups for it.  Also, there was one file where the updated
uncrustify caught a problem that the regex didn't pick up, which is updated
in this commit.

Signed-off-by: David Lechner <david@pybricks.com>
2020-06-14 20:24:18 +10:00
David Lechner bd7c92e17d tools/uncrustify.cfg: Remove deprecated sp_word_brace option.
This option was removed in uncrustify v0.71.

Signed-off-by: David Lechner <david@pybricks.com>
2020-06-14 20:24:03 +10:00
David Lechner 8f642677f7 tools/codeformat.py: Add verbose option to pass to uncrustify and black.
This adds a new command line option `-v` to `tools/codeformat.py` to enable
verbose printing of all files that are scanned.

Normally `uncrustify` and `black` are called with the `-q` option so
setting verbose suppresses the `-q` option and on `black` also enables the
`-v` option which makes it print out all file names matching the filter
similar to how `uncrustify` does by default.
2020-05-29 22:59:56 +10:00
David Lechner 2d1fef7096 tools/codeformat.py: Use -q option on uncrustify to make output quiet.
This suppresses the Parsing: <file> as language C lines.  This makes
parsing run a bit faster and on CI it makes for less scrolling through logs
(and black already uses the -q option).
2020-05-28 10:08:38 +10:00
stijn 84fa3312cf all: Format code to add space after C++-style comment start.
Note: the uncrustify configuration is explicitly set to 'add' instead of
'force' in order not to alter the comments which use extra spaces after //
as a means of indenting text for clarity.
2020-04-23 11:24:25 +10:00
Damien George 6e2871df59 tools/check_code_size.sh: Remove unused script.
It's now replaced by tools/metrics.py.
2020-04-22 14:07:56 +10:00
Damien George 17dc86369f tools/metrics.py: Use OrderedDict when reading build log.
So that the output (eg of the diff command) always has the lines in the
same order.
2020-04-22 14:03:26 +10:00
Damien George 1cc24cd39a tools/metrics.py: Don't build mpy-cross if not needed by any ports.
To save build time.
2020-04-22 14:03:07 +10:00
Damien George a4423570e2 tools/metrics.py: Add option to diff to error if delta above threshold.
Useful for things like CI where the size check is automated.
2020-04-22 14:02:14 +10:00
stijn bcf01d1686 all: Fix implicit conversion from double to float.
These are found when building with -Wfloat-conversion.
2020-04-18 22:42:24 +10:00
Jim Mussared 85858e72df py/objexcept: Allow compression of exception message text.
The decompression of error-strings is only done if the string is accessed
via printing or via er.args.  Tests are added for this feature to ensure
the decompression works.
2020-04-05 15:02:06 +10:00
David Lechner 3dc324d3f1 tests: Format all Python code with black, except tests in basics subdir.
This adds the Python files in the tests/ directory to be formatted with
./tools/codeformat.py.  The basics/ subdirectory is excluded for now so we
aren't changing too much at once.

In a few places `# fmt: off`/`# fmt: on` was used where the code had
special formatting for readability or where the test was actually testing
the specific formatting.
2020-03-30 13:21:58 +11:00
Lars Kellogg-Stedman 3a0f64fc7a tools/pyboard.py: Add -d as an alias for --device. 2020-03-30 11:37:32 +11:00
Lars Kellogg-Stedman 1cf994c48b tools/pyboard.py: Support setting device/baudrate from shell env vars.
Allow defaults for --device and --baudrate to be set in the environment
using PYBOARD_DEVICE and PYBOARD_BAUDRATE.
2020-03-30 11:37:32 +11:00
Damien George 1a3e386c67 all: Remove spaces inside and around parenthesis.
Using new options enabled in the uncrustify configuration.
2020-03-28 23:36:44 +11:00
Damien George feb2577585 all: Remove spaces between nested paren and inside function arg paren.
Using new options enabled in the uncrustify configuration.
2020-03-25 00:39:46 +11:00
stijn 1b3e0e10b9 tools/codeformat.py: Include all msvc C code in auto-format. 2020-03-25 00:36:42 +11:00
Andrew Leech 38ccb4c643 tools/pydfu.py: Display any error strings from device/mboot. 2020-03-22 15:24:57 +11:00
Andrew Leech 2966d83a65 tools/pydfu.py: Add args for VID/PID & exit with cleaner error handling. 2020-03-22 13:33:04 +11:00
David Lechner 8a4ce6b79a tools/codeformat.py: Eliminate need for sizeof fixup.
This eliminates the need for the sizeof regex fixup by rearranging things a
bit.  All other bitfields already use the parentheses around expressions
with sizeof, so one case is fixed by following this convention.

VM_MAX_STATE_ON_STACK is the only remaining problem and it can be worked
around by changing the order of the operands.
2020-03-11 14:34:40 +11:00
Damien George 554c01fc25 tools/metrics.py: Use check_call instead of run to error out on error. 2020-03-11 14:23:39 +11:00
MikeTeachman 8db5d2d1f1 tools/makemanifest.py: Fix build on Windows by adding .exe to mpy-cross.
When using a manifest on Windows the reference to mpy-cross compiler was
missing the .exe file extension, so add it when appropriate.

Also allow the default path to mpy-cross to be overridden by the (optional)
MICROPY_MPYCROSS environment variable, to allow full flexibility on any OS.
2020-03-05 10:26:31 +11:00
Damien George 69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
Damien George 4b23e98fb0 tools/codeformat.py: Add formatter using uncrustify for C, black for Py.
This commit adds a tool, codeformat.py, which will reformat C and Python
code to fit a certain style.  By default the tool will reformat (almost)
all the original (ie not 3rd-party) .c, .h and .py files in this
repository.  Passing filenames on the command-line to codeformat.py will
reformat only those.  Reformatting is done in-place.

uncrustify is used for C reformatting, which is available for many
platforms and can be easily built from source, see
https://github.com/uncrustify/uncrustify.  The configuration for uncrustify
is also added in this commit and values are chosen to best match the
existing code style.  A small post-processing stage on .c and .h files is
done by codeformat.py (after running uncrustify) to fix up some minor
items:
- space inserted after * when used as multiplication with sizeof
- #if/ifdef/ifndef/elif/else/endif are dedented by one level when they are
  configuring if-blocks and case-blocks.

For Python code, the formatter used is black, which can be pip-installed;
see https://github.com/psf/black.  The defaults are used, except for line-
length which is set at 99 characters to match the "about 100" line-length
limit used in C code.

The formatting tools used and their configuration were chosen to strike a
balance between keeping existing style and not changing too many lines of
code, and enforcing a relatively strict style (especially for Python code).
This should help to keep the code consistent across everything, and reduce
cognitive load when writing new code to match the style.
2020-02-28 10:14:28 +11:00
Damien George a636837987 tools/makemanifest.py: Support freezing with empty list of mpy files.
Fixes issue #5655.
2020-02-20 10:41:50 +11:00
Michael Buesch 1604606238 tools/pyboard.py: Change shebang to use python3.
This script still works with Python 2 but Python 3 is recommended.
2020-02-01 00:06:26 +11:00
Michael Buesch 1cadb12d1c tools/pyboard.py: Use slice del instead of list.clear() for Py2 compat.
Python 2 does not have list.clear().
2020-02-01 00:05:29 +11:00
Michael Buesch 83afd48ad9 tools/pyboard.py: Add option --no-follow to detach after sending script.
This option makes pyboard.py exit as soon as the script/command is
successfully sent to the device, ie it does not wait for any output.  This
can help to avoid hangs if the board is being rebooted with --comman (for
example).

Example usage:

    $ python3 ./tools/pyboard.py --device /dev/ttyUSB0 --no-follow \
        --command 'import machine; machine.reset()'
2020-02-01 00:03:37 +11:00
c0rejump 6db5cede06 tools/pydfu.py: Clean up syntax, update comments and docstrings.
Some parts of code have been aligned to increase readability.  In general
'' instead of "" were used wherever possible to keep the same convention
for entire file.  Import inspect line has been moved to the top according
to hints reported by pep8 tools.  A few extra spaces were removed, a few
missing spaces were added.  Comments have been updated, mostly in
"read_dfu_file" function.  Some other comments have been capitalized and/or
slightly updated.  A few docstrings were fixed as well.  No real code
changes intended.
2020-01-23 13:23:12 +11:00
Damien George a11e306227 tools: Add metrics.py script to build and compute port sizes/metrics. 2020-01-23 12:48:52 +11:00
Jason Neal de78a9e317 tools/gen-cpydiff.py: Adjust subsections to sentence case. 2020-01-06 22:16:18 +11:00
David Lechner fd0ba7be07 tools/tinytest-codegen.py: Add extra newline and result message.
This is an alternative to f4ed2df that adds a newline so that the output of
the test starts on a new line and the result of the test is prefixed with
"result: " to distinguish it from the test output.

Suggested-by: @dpgeorge
2019-12-19 17:55:50 +11:00
Damien George b3b9b11596 tools/pyboard.py: Support executing .mpy files directly.
This patch allows executing .mpy files (including native ones) directly on
a target, eg a board over a serial connection.  So there's no need to copy
the file to its filesystem to test it.

For example:

    $ mpy-cross foo.py
    $ pyboard.py foo.mpy
2019-12-19 17:00:52 +11:00
Damien George 0bd7d1f7f0 py/persistentcode: Move loading of rodata/bss to before obj/raw-code.
This makes the loading of viper-code-with-relocations a bit neater and
easier to understand, by treating the rodata/bss like a special object to
be loaded into the constant table (which is how it behaves).
2019-12-17 13:22:11 +11:00
Damien George fc97d6d1b5 tools/mpy-tool.py: Raise exception if trying to freeze relocatable mpy. 2019-12-12 20:15:28 +11:00
Damien George abc642973d py/dynruntime: Add support for float API to make/get floats.
We don't want to add a feature flag to .mpy files that indicate float
support because it will get complex and difficult to use.  Instead the .mpy
is built using whatever precision it chooses (float or double) and the
native glue API will convert between this choice and what the host runtime
actually uses.
2019-12-12 20:15:28 +11:00
Damien George aad79adab7 tools/mpy_ld.py: Add new mpy_ld.py tool and associated build files.
This commit adds a new tool called mpy_ld.py which is essentially a linker
that builds .mpy files directly from .o files.  A new header file
(dynruntime.h) and makefile fragment (dynruntime.mk) are also included
which allow building .mpy files from C source code.  Such .mpy files can
then be dynamically imported as though they were a normal Python module,
even though they are implemented in C.

Converting .o files directly (rather than pre-linked .elf files) allows the
resulting .mpy to be more efficient because it has more control over the
relocations; for example it can skip PLT indirection.  Doing it this way
also allows supporting more architectures, such as Xtensa which has
specific needs for position-independent code and the GOT.

The tool supports targets of x86, x86-64, ARM Thumb and Xtensa (windowed
and non-windowed).  BSS, text and rodata sections are supported, with
relocations to all internal sections and symbols, as well as relocations to
some external symbols (defined by dynruntime.h), and linking of qstrs.
2019-12-12 20:15:28 +11:00
Damien George 27879844d2 tools/mpy-tool.py: Add ability to merge multiple .mpy files into one.
Usage:

    mpy-tool.py -o merged.mpy --merge mod1.mpy mod2.mpy

The constituent .mpy files are executed sequentially when the merged file
is imported, and they all use the same global namespace.
2019-12-12 20:15:28 +11:00
Damien George 360d972c16 py/nativeglue: Add new header file with native function table typedef. 2019-12-12 20:15:28 +11:00
Damien George 7f24c29778 tools/mpy-tool.py: Support qstr linking when freezing Xtensa native mpy. 2019-11-28 13:11:51 +11:00
Yonatan Goldschmidt b2dd443d92 tools/makemanifest.py: Use sys.executable when invoking Python scripts.
So the version of Python used to run makemanifest.py is also used for the
sub-scripts.
2019-11-13 13:44:19 +11:00
Andrew Leech d2e6cfd8fd tools/makemanifest.py: Skip freezing unsupported files with warning. 2019-11-07 12:34:57 +11:00
Mirko Vogt 2f71d66ef7 tools/makemanifest.py: Follow symlinks when freezing linked directories.
While the new manifest.py style got introduced for freezing python code
into the resulting binary, the old way - where files and modules within
ports/*/modules where baked into the resulting binary - was still
supported via `freeze('$(PORT_DIR)/modules')` within manifest.py.

However behaviour changed for symlinked directories (=modules), as those
links weren't followed anymore.

This commit restores the original behaviour by explicitly following
symlinks within a modules/ directory
2019-11-06 11:41:06 +11:00
Damien George 36c9be6f60 tools/mpy-tool.py: Use "@progbits #" attribute for native xtensa code. 2019-11-04 15:31:42 +11:00
Jim Mussared 8ba963cfa3 tools/makemanifest.py: Eval relative paths w.r.t. current manifest file.
When loading a manifest file, e.g. by include(), it will chdir first to the
directory of that manifest.  This means that all file operations within a
manifest are relative to that manifest's location.

As a consequence of this, additional environment variables are needed to
find absolute paths, so the following are added: $(MPY_LIB_DIR),
$(PORT_DIR), $(BOARD_DIR).  And rename $(MPY) to $(MPY_DIR) to be
consistent.

Existing manifests are updated to match.
2019-10-21 23:01:41 +11:00
Damien George e81f538e25 tools: Add mechanism to provide a manifest of frozen files.
This introduces a new build variable FROZEN_MANIFEST which can be set to a
manifest listing (written in Python) that describes the set of files to be
frozen in to the firmware.
2019-10-15 21:34:23 +11:00
Damien George 8e8cfa6f53 tools/make-frozen.py: Allow to run with no directory passed in.
In which case it will just emit empty frozen C definitions.
2019-10-15 21:33:49 +11:00
Damien George 23f0691fdd py/persistentcode: Make .mpy more compact with qstr directly in prelude.
Instead of encoding 4 zero bytes as placeholders for the simple_name and
source_file qstrs, and storing the qstrs after the bytecode, store the
qstrs at the location of these 4 bytes.  This saves 4 bytes per bytecode
function stored in a .mpy file (for example lcd160cr.mpy drops by 232
bytes, 4x 58 functions).  And resulting code size is slightly reduced on
ports that use this feature.
2019-10-15 16:56:27 +11:00
Damien George 9adedce42e py: Add new Xtensa-Windowed arch for native emitter.
Enabled via the configuration MICROPY_EMIT_XTENSAWIN.
2019-10-05 13:44:53 +10:00
Damien George c8c0fd4ca3 py: Rework and compress second part of bytecode prelude.
This patch compresses the second part of the bytecode prelude which
contains the source file name, function name, source-line-number mapping
and cell closure information.  This part of the prelude now begins with a
single varible length unsigned integer which encodes 2 numbers, being the
byte-size of the following 2 sections in the header: the "source info
section" and the "closure section".  After decoding this variable unsigned
integer it's possible to skip over one or both of these sections very
easily.

This scheme saves about 2 bytes for most functions compared to the original
format: one in the case that there are no closure cells, and one because
padding was eliminated.
2019-10-01 12:26:22 +10:00
Damien George b5ebfadbd6 py: Compress first part of bytecode prelude.
The start of the bytecode prelude contains 6 numbers telling the amount of
stack needed for the Python values and exceptions, and the signature of the
function.  Prior to this patch these numbers were all encoded one after the
other (2x variable unsigned integers, then 4x bytes), but using so many
bytes is unnecessary.

An entropy analysis of around 150,000 bytecode functions from the CPython
standard library showed that the optimal Shannon coding would need about
7.1 bits on average to encode these 6 numbers, compared to the existing 48
bits.

This patch attempts to get close to this optimal value by packing the 6
numbers into a single, varible-length unsigned integer via bit-wise
interleaving.  The interleaving scheme is chosen to minimise the average
number of bytes needed, and at the same time keep the scheme simple enough
so it can be implemented without too much overhead in code size or speed.
The scheme requires about 10.5 bits on average to store the 6 numbers.

As a result most functions which originally took 6 bytes to encode these 6
numbers now need only 1 byte (in 80% of cases).
2019-10-01 12:26:22 +10:00
Damien George 5716c5cf65 py/persistentcode: Bump .mpy version to 5.
The bytecode opcodes have changed (there are more, and they have been
reordered).
2019-09-26 16:39:37 +10:00
Josh Lloyd 7d58a197cf py: Rename MP_QSTR_NULL to MP_QSTRnull to avoid intern collisions.
Fixes #5140.
2019-09-26 16:04:56 +10:00
Damien George 1f7202d122 py/bc: Replace big opcode format table with simple macro. 2019-09-26 15:27:11 +10:00
Damien George 5889cf58db py/bc0: Order opcodes into groups based on their size and format. 2019-09-26 15:27:10 +10:00
Damien George c69f58e6b9 tools/mpy-tool.py: Fix freezing of non-bytecode funcs with settrace.
Only bytecode functions can be profiled at this stage.  Native functions
(eg inline assembler) may not even have a valid prelude.

Fixes issue #5075.
2019-09-06 23:55:15 +10:00
Damien George b29fae0c56 py/bc: Fix size calculation of UNWIND_JUMP opcode in mp_opcode_format.
Prior to this patch mp_opcode_format would calculate the incorrect size of
the MP_BC_UNWIND_JUMP opcode, missing the additional byte.  But, because
opcodes below 0x10 are unused and treated as bytes in the .mpy load/save
and freezing code, this bug did not show any symptoms, since nested unwind
jumps would rarely (if ever) reach a depth of 16 (so the extra byte of this
opcode would be between 0x01 and 0x0f and be correctly loaded/saved/frozen
simply as an undefined opcode).

This patch fixes this bug by correctly accounting for the additional byte.
        .
2019-09-02 13:30:16 +10:00
Damien George 4691b43c8a tools/mpy-tool.py: Add initial support for frozen with settrace. 2019-08-30 16:49:13 +10:00
Milan Rossa 498e35219e tests: Add tests for sys.settrace feature. 2019-08-30 16:48:22 +10:00
Jim Mussared 0bd1eb80ff qemu-arm: Add testing of frozen native modules.
- Split 'qemu-arm' from 'unix' for generating tests.
- Add frozen module to the qemu-arm test build.
- Add test that reproduces the requirement to half-word align native
  function data.
2019-08-20 15:14:08 +10:00
Jim Mussared 4ab5156c01 tools/mpy-tool.py: Force native func alignment to halfword/word on ARM.
This is necessary for ARMV6 and V7.  Without this change, calling a frozen
native/viper function that is misaligned will crash.
2019-08-20 15:13:17 +10:00
Damien George 4d94fae833 tools/pyboard.py: Add filesystem commands to ls/cat/cp/rm remote files.
Use "-f" to select filesystem mode, followed by the command to execute.
Optionally put ":" at the start of a filename to indicate that it's on the
remote device, if it would otherwise be ambiguous.

Examples:

    $ pyboard.py -f ls
    $ pyboard.py -f cat main.py
    $ pyboard.py -f cp :main.py .   # get from device
    $ pyboard.py -f cp main.py :    # put to device
    $ pyboard.py -f rm main.py
2019-07-25 15:56:01 +10:00
Damien George f073f2b543 tools: Add uf2conv.py from Microsoft/uf2 repository.
Repository https://github.com/Microsoft/uf2 commit
19615407727073e36d81bf239c52108ba92e7660
2019-07-01 17:18:44 +10:00
Jun Wu b152bbddd1 py: Define EMIT_MACHINE_CODE as EMIT_NATIVE || EMIT_INLINE_ASM.
The combination MICROPY_EMIT_NATIVE || MICROPY_EMIT_INLINE_ASM is used in
many places, so define a new macro for it.
2019-06-28 13:54:45 +10:00
Damien George 9d3031cc9d tools/mpy-tool.py: Fix linking of qstr objects in native ARM Thumb code.
Previously, when linking qstr objects in native code for ARM Thumb, the
index into the machine code was being incremented by 4, not 8.  It should
be 8 to account for the size of the two machine instructions movw and movt.
This patch makes sure the index into the machine code is incremented by the
correct amount for all variations of qstr linking.

See issue #4829.
2019-06-11 11:36:39 +10:00
Damien George faf3d3e9e9 tools/mpy-tool.py: Fix linking qstrs in native code, and multiple files.
Fixes errors in the tool when 1) linking qstrs in native ARM-M code; 2)
freezing multiple files some of which use native code and some which don't.

Fixes issue #4829.
2019-06-04 22:21:01 +10:00
Damien George 993ca572ca tools/upip.py: Add support for multiple index URLs with custom default.
The user can now select their own package index by either passing the "-i"
command line option, or setting the upip.index_urls variable (before doing
an install).

The https://micropython.org/pi package index hosts packages from
micropython-lib and will be searched first when installing a package.  If a
package is not found here then it will fallback to PyPI.
2019-05-15 15:46:16 +10:00
Damien George 56f6ceba7f tools/pyboard.py: Don't accumulate output data if data_consumer used.
Prior to this patch, when a lot of data was output by a running script
pyboard.py would try to capture all of this output into the "data"
variable, which would gradually slow down pyboard.py to the point where it
would have large CPU and memory usage (on the host) and potentially lose
data.

This patch fixes this problem by not accumulating the data in the case that
the data is not needed, which is when "data_consumer" is used.
2019-04-25 13:24:32 +10:00
Damien George 74ed06828f tools/mpy-tool.py: Fix init of QStrWindow, and remove unused variable.
The qstr window size is not log-2 encoded, it's just the actual number (but
in mpy-tool.py this didn't lead to an error because the size is just used
to truncate the window so it doesn't grow arbitrarily large in memory).

Addresses issue #4635.
2019-04-08 15:24:24 +10:00
Damien George 643d2a0e86 tools/mpy-tool.py: Adjust use of super() to make it work with Python 2.
Fixes the regression introduced in ea3c80a514
2019-04-08 11:21:18 +10:00
rhubarbdog 869a8b70ce tools/pyboard.py: Add missing line from example usage comments. 2019-03-26 16:52:41 +11:00
Andrew Leech 89ff506513 py: Update and rework build system for including external C modules.
How to use this feature is documented in docs/develop/cmodules.rst.
2019-03-08 22:58:42 +11:00
Ayke van Laethem 2e516074da py: Implement a module system for external, user C modules.
This system makes it a lot easier to include external libraries as static,
native modules in MicroPython.  Simply pass USER_C_MODULES (like
FROZEN_MPY_DIR) as a make parameter.
2019-03-08 22:49:00 +11:00
Damien George 6e11d86318 tools/upip.py: Use "raise arg" instead of no-arg raise form, for native. 2019-03-08 16:51:09 +11:00
Damien George 9a5f92ea72 py/persistentcode: Bump .mpy version to 4. 2019-03-08 15:53:05 +11:00
Damien George ea3c80a514 tools/mpy-tool.py: Add support for freezing native code.
This adds support to freeze .mpy files that contain native code blocks.
2019-03-08 15:53:05 +11:00
Damien George 636ed0ff8d py/emitglue: Remove union in mp_raw_code_t to combine bytecode & native. 2019-03-08 15:53:04 +11:00
Damien George 4f0931b21f py/persistentcode: Define static qstr set to reduce size of mpy files.
When encoded in the mpy file, if qstr <= QSTR_LAST_STATIC then store two
bytes: 0, static_qstr_id.  Otherwise encode the qstr as usual (either with
string data or a reference into the qstr window).

Reduces mpy file size by about 5%.
2019-03-05 16:32:05 +11:00
Damien George 992a6e1dea py/persistentcode: Pack qstrs directly in bytecode to reduce mpy size.
Instead of emitting two bytes in the bytecode for where the linked qstr
should be written to, it is now replaced by the actual qstr data, or a
reference into the qstr window.

Reduces mpy file size by about 10%.
2019-03-05 16:27:34 +11:00
Damien George 5996eeb48f py/persistentcode: Add a qstr window to save mpy files more efficiently.
This is an implementation of a sliding qstr window used to reduce the
number of qstrs stored in a .mpy file.  The window size is configured to 32
entries which takes a fixed 64 bytes (16-bits each) on the C stack when
loading/saving a .mpy file.  It allows to remember the most recent 32 qstrs
so they don't need to be stored again in the .mpy file.  The qstr window
uses a simple least-recently-used mechanism to discard the least recently
used qstr when the window overflows (similar to dictionary compression).
This scheme only needs a single pass to save/load the .mpy file.

Reduces mpy file size by about 25% with a window size of 32.
2019-03-05 16:25:07 +11:00
Damien George 5a2599d962 py: Replace POP_BLOCK and POP_EXCEPT opcodes with POP_EXCEPT_JUMP.
POP_BLOCK and POP_EXCEPT are now the same, and are always followed by a
JUMP.  So this optimisation reduces code size, and RAM usage of bytecode by
two bytes for each try-except handler.
2019-03-05 16:09:58 +11:00
Dave Hylands c932639063 tools/pydfu.py: Fix regression so tool runs under Python 2 again.
Under python3 (tested with 3.6.7) bytes with a list of integers as an
argument returns a different result than under python 2.7 (tested with
2.7.15rc1) which causes pydfu.py to fail when run under 2.7.  Changing
bytes to bytearray makes pydfu work properly under both Python 2.7 and
Python 3.6.
2018-12-30 01:20:48 +11:00
Dave Hylands 39eef27083 tools/mpy-tool.py: Fix build error when no qstrs present in frozen mpy.
If you happen to only have a really simple frozen file that doesn't contain
any new qstrs then the generated frozen_mpy.c file contains an empty
enumeration which causes a C compile time error.
2018-12-15 14:36:08 +11:00
Damien George 814d580a15 tools/mpy-tool.py: Fix calc of opcode size for opcodes with map caching.
Following an equivalent fix to py/bc.c.  The reason the incorrect values
for the opcode constants were not previously causing a bug is because they
were never being used: these opcodes always have qstr arguments so the part
of the code that was comparing them would never be reached.

Thanks to @malinah for finding the problem and providing the initial patch.
2018-12-13 01:26:55 +11:00
Damien George 4f25a8b6a4 tools/pydfu.py: Improve DFU reset, and auto-detect USB transfer size.
A DFU device must be in the idle state before it can be programmed, and
this requires either clearing the status or aborting, depending on its
current state.  Code is added to do this.  And the USB transfer size is now
automatically detected so devices with a size less than 2048 bytes work
correctly.
2018-11-27 16:19:27 +11:00
Martin Dybdal 7795b2e5c3 tools/pyboard.py: In TelnetToSerial.close replace try/except with if.
Some Python linters don't like unconditional except clauses because they
catch SystemExit and KeyboardInterrupt, which usually is not the intended
behaviour.
2018-10-19 23:46:10 +11:00
Andrew Leech a2703649ea tools/pydfu: Workaround stdio flush error on Windows with Python 3.6.
There appears to be an issue on Windows with CPython >= 3.6,
sys.stdout.flush() raises an exception:

    OSError: [WinError 87] The parameter is incorrect

It works fine to just catch and ignore the error on the flush line.  Tested
on Windows 10 x64 1803 (Build 17134.228), Python 3.6.4 amd64.
2018-09-21 11:57:24 +10:00
Martin Dybdal 5ed8226e02 tools/pyboard.py: Change base class of PyboardError to Exception.
Following standard practice for defining custom exceptions.
2018-08-10 16:23:38 +10:00
Ayke van Laethem 0d7a088039 tools/pyboard: Run exec: command as a string.
The Python documentation recommends to pass the command as a string when
using Popen(..., shell=True).  This is because "sh -c <string>" is used to
execute the command and additional arguments after the command string are
passed to the shell itself (not the executing command).

https://docs.python.org/3.5/library/subprocess.html#subprocess.Popen
2018-08-04 15:45:23 +10:00
Rich Barlow 6e5a40cf3c tools/mpy-tool: Set sane initial dynamic qstr pool size with frozen mods
The first dynamic qstr pool is double the size of the 'alloc' field of
the last const qstr pool. The built in const qstr pool
(mp_qstr_const_pool) has a hardcoded alloc size of 10, meaning that the
first dynamic pool is allocated space for 20 entries. The alloc size
must be less than or equal to the actual number of qstrs in the pool
(the 'len' field) to ensure that the first dynamically created qstr
triggers the creation of a new pool.

When modules are frozen a second const pool is created (generally
mp_qstr_frozen_const_pool) and linked to the built in pool. However,
this second const pool had its 'alloc' field set to the number of qstrs
in the pool. When freezing a large quantity of modules this can result
in thousands of qstrs being in the pool. This means that the first
dynamically created qstr results in a massive allocation. This commit
sets the alloc size of the frozen qstr pool to 10 or less (if the number
of qstrs in the pool is less than 10). The result of this is that the
allocation behaviour when a dynamic qstr is created is identical with an
without frozen code.

Note that there is the potential for a slight memory inefficiency if the
frozen modules have less than 10 qstrs, as the first few dynamic
allocations will have quite a large overhead, but the geometric growth
soon deals with this.
2018-08-01 18:59:31 +10:00
roland 11a38d5dc5 tools/pydfu.py: Make the DFU tool work again with Python 2.
This patch will work for both Python 2 and 3.
2018-07-27 16:53:50 +10:00
roland feec0a6909 tools/pydfu.py: Use getfullargspec instead of getargspec for newer pyusb
pyusb v1.0.2 warns about `getargspec` as being deprecated.
2018-07-20 14:28:17 +10:00
roland 6e50df4e21 tools/dfu.py: Pad image data to 8 byte alignment to support L476.
Thanks to @dhylands for this patch to pad the image to 8-byte boundaries.
2018-07-20 14:23:11 +10:00
Damien George 44fc92ea7c tools/mpy-tool.py: Put frozen bignum digit data in ROM, not in RAM. 2018-07-09 13:43:34 +10:00
Damien George 929d10acf7 tools/mpy-tool.py: Support freezing of floats in obj representation D. 2018-07-09 12:22:40 +10:00
Damien George 92667dc2e5 tools/pydfu.py: Add support for multiple memory segments.
Segments are separated by / and begin with the memory address.  This
follows how the ST DFU tool works.
2018-06-22 15:32:32 +10:00
Damien George 8fb95d6520 tools/pydfu.py: Increase download packet size to full 2048 bytes.
The ST DFU bootloader supports a transfer size up to 2048 bytes, so send
that much data on each download (to device) packet.  This almost halves
total download time.
2018-06-08 15:32:49 +10:00
Keith Wiley 3ea0862a6e tools/pydfu.py: Fix typo in comments. 2018-05-18 23:33:19 +10:00
Damien George f7be5f9bfa tools/upip: Upgrade upip to 1.2.4.
Uses new pypi.org URL, and now creates a socket with the address parameters
returned by getaddrinfo().
2018-04-23 16:11:27 +10:00
Paul Sokolovsky 4475f32420 tools/tinytest-codegen: Ignore system locale, write output in UTF-8.
Way to reproduce a UnicodeEncodeError without this patch:

    LC_ALL=C tinytest-codegen.py ...
2017-12-15 12:04:10 +02:00
Paul Sokolovsky 325d0fc74b tools/tinytest-codegen: Add --stdin switch instead of recently added --target.
Instead of passing thru more and more options from tinytest-codegen to
run-tests --list-tests, pipe output of run-tests --list-tests into
tinytest-codegen.
2017-12-14 12:26:59 +02:00
Damien George bb516af1eb tools/pydfu.py: Call set_configuration() on fresh USB device object.
This call is required before using the device (some operating systems don't
need it but others do).  Fixes issue #3476.
2017-12-14 10:08:37 +11:00
Paul Sokolovsky 43141ddb55 tools/tinytest-codegen: Take --target= option for test set selection.
Gets passed to run-tests --list-tests to get actual list of tests to use.
If --target= is not given, legacy set hardcoded in tinytest-codegen itself
is used.

Also, get rid of tinytest test groups - they aren't really used for
anything, and only complicate processing. Besides, one of the next
step is to limit number of tests per a generated file to control
the binary size, which also will require "flat" list of tests.
2017-12-13 18:35:37 +02:00
Paul Sokolovsky e6f0d547ab tools/tinytest-codegen: More excludes after enabling expected output match. 2017-12-13 09:07:51 +02:00
Paul Sokolovsky 4db6a7adbe tools/tinytest-codegen: Wrap lines of exclude_tests.
So it was manageable and extensible.
2017-12-13 09:07:51 +02:00
Paul Sokolovsky 48e931e1d3 tools/tinytest-codegen.py: Generate code for upytesthelper.
The way tinytest was used in qemu-arm test target is that it didn't test
much. MicroPython tests are based on matching the test output against
reference output, but qemu-arm's implementation didn't do that, it
effectively tested just that there was no exception during test
execution. "upytesthelper" wrapper was introduce to fix it, and so
test generator is now switched to generate test code for it.

Also, fix PEP8 and other codestyle issues.
2017-12-13 09:07:51 +02:00
Paul Sokolovsky e9d29c9ba9 lib/tinytest: Move from tools/tinytest.
Tinytest library was misplaced under tools/. By convention, any target
libraries belong to lib/, while tools/ contains host-side tools.
2017-12-08 19:26:15 +02:00
Damien George 7cf446f3da tools/gen-cpydiff.py: Update executable paths to point to new ports dir. 2017-11-28 10:50:32 +11:00
Damien George 9ba3de6ea1 tools/mpy-tool.py: Implement freezing of Ellipsis const object. 2017-11-15 12:46:08 +11:00
Damien George 933eab46fc py/bc: Update opcode_format_table to match the bytecode. 2017-10-10 10:37:38 +11:00
Paul Sokolovsky c15be989ee tools/pyboard: Update docstring for additional device support. 2017-10-08 00:04:57 +03:00
Paul Sokolovsky ea6692a83e tools/pyboard: Use repr() when quoting data in error messages.
As it may contain newlines, etc.
2017-10-05 23:40:19 +03:00
Damien George ff93fd4f50 py/persistentcode: Bump .mpy version number to version 3.
The binary and unary ops have changed bytecode encoding.
2017-10-05 10:49:44 +11:00
Paul Sokolovsky 2f7827ba8f tools/upip: Upgrade to 1.2.2.
TLS SNI support, fixes after making str.rstrip() behavior compliant.
2017-09-29 18:24:11 -07:00
Damien George 4a93801c12 all: Update Makefiles and others to build with new ports/ dir layout.
Also renames "stmhal" to "stm32" in documentation and everywhere else.
2017-09-06 14:09:13 +10:00
stijn e4ab404780 tools/mpy-tool.py: Fix missing argument in dump() function
This makes the -d commandline argument usable again.
Pass empty string as parent name as listing starts from the root.
2017-08-16 10:38:19 +02:00
Damien George b6a3289564 tools/mpy-tool.py: Don't generate const_table if it's empty. 2017-08-12 22:26:18 +10:00
Alexander Steffen 55f33240f3 all: Use the name MicroPython consistently in comments
There were several different spellings of MicroPython present in comments,
when there should be only one.
2017-07-31 18:35:40 +10:00
Paul Sokolovsky a3cd349eaf tools/mpy_bin2res: Tools to convert binary resources to Python module.
Afterwards, they can be access using pkg_resource module from
micropython-lib.
2017-07-27 14:41:27 +03:00
Paul Sokolovsky 7901741bf1 tools/pyboard: Add license header. 2017-07-22 17:12:15 +03:00
Damien George 761e4c7ff6 all: Remove trailing spaces, per coding conventions. 2017-07-19 13:12:10 +10:00
Paul Sokolovsky ad5e7a0e6f tools/gen-cpydiff: Use case description as 3rd-level heading.
This is required to easily giving links to a particular difference case.
Also, add RST anchors to allow cases to cross-reference each other.
2017-07-09 13:51:40 +03:00
Ville Skyttä ca16c38210 various: Spelling fixes 2017-05-29 11:36:05 +03:00
Damien George 88c51c3592 tools/mpy-tool.py: Fix regression with freezing floats in obj repr C.
Regression was introduced by ec534609f6
2017-05-16 18:53:02 +10:00
Paul Sokolovsky 55491031be tools/mpy_cross_all.py: Helper tool to run mpy-cross on the entire project. 2017-05-14 17:51:12 +03:00
Damien George ec534609f6 tools/mpy-tool.py: Use MP_ROM_xxx macros to support nanbox builds. 2017-05-13 10:08:13 +10:00
Paul Sokolovsky d4c070415a tools/upip: Upgrade to 1.2.
Memory optimizations and error handling improvements.
2017-05-05 13:12:19 +03:00
Paul Sokolovsky e81f46940e tools/upip: Upgrade to 1.1.6, supports commented lines in requirements.txt. 2017-05-01 00:03:45 +03:00
Paul Sokolovsky 473e85e2da tools/mpy-tool: Make work if run from another directory.
By making sure we don't add relative paths to sys.path.
2017-05-01 00:01:30 +03:00
Damien George dd11af209d py: Add LOAD_SUPER_METHOD bytecode to allow heap-free super meth calls.
This patch allows the following code to run without allocating on the heap:

    super().foo(...)

Before this patch such a call would allocate a super object on the heap and
then load the foo method and call it right away.  The super object is only
needed to perform the lookup of the method and not needed after that.  This
patch makes an optimisation to allocate the super object on the C stack and
discard it right after use.

Changes in code size due to this patch are:

   bare-arm: +128
    minimal: +232
   unix x64: +416
unix nanbox: +364
     stmhal: +184
    esp8266: +340
     cc3200: +128
2017-04-22 23:39:20 +10:00
Paul Sokolovsky 3e1310d6e2 tools/pyboard: Provide more details when expected reply not received.
When trying to execute a command via raw REPL and expected "OK" reply
not received, show what was received instead.
2017-04-07 01:04:47 +03:00
Paul Sokolovsky 2cbe997834 tools/pyboard: ProcessPtyToTerminal: Add workaround for PySerial bug.
When working with a "virtual" port, like PTY. The issue described in
http://stackoverflow.com/questions/34831131/pyserial-does-not-play-well-with-virtual-port
2017-04-05 12:30:39 +03:00
Paul Sokolovsky 546ef301a1 tools/pyboard: execpty: Use shell=False to workaround some curdir issues.
Without this, Zephyr's port "make test" doesn't work.
2017-04-05 00:46:12 +03:00
Paul Sokolovsky 647e72ca63 tools/pyboard: Add "exec" and "execpty" pseudo-devices support.
This allows to execute a command and communicate with its stdin/stdout
via pipes ("exec") or with command-created pseudo-terminal ("execpty"),
to emulate serial access. Immediate usecase is controlling a QEMU process
which emulates board's serial via normal console, but it could be used
e.g. with helper binaries to access real board over other hadware
protocols, etc.

An example of device specification for these cases is:

	--device exec:../zephyr/qemu.sh
	--device execpty:../zephyr/qemu2.sh

Where qemu.sh contains long-long qemu startup line, or calls another
command. There's a special support in this patch for running the command
in a new terminal session, to support shell wrappers like that (without
new terminal session, only wrapper script would be terminated, but its
child processes would continue to run).
2017-04-04 17:46:02 +03:00
Paul Sokolovsky 9b3f423c14 tools/pyboard: Tighten up Pyboard object closure on errors.
Some "device" implementations may be sensitive to this.
2017-04-02 20:48:53 +03:00
Paul Sokolovsky 320099aab9 tools/tinytest-codegen: Update for recent test renaming ("intbig" suffix). 2017-03-06 22:40:04 +01:00
Damien George e4be56a0ea qemu-arm: Enable machine module and associated tests. 2017-03-02 16:39:58 +11:00
Krzysztof Blazewicz 23ccb3e12e tools/gen-cpydiff.py: configurable CPython and micropython executables 2017-02-27 15:39:55 +11:00
Damien George 1034d9acc8 tools/gen-cpydiff.py: Set the Python import path to find test modules. 2017-02-22 15:50:58 +11:00
Rami Ali b7fa63c7ce tools: Add gen-cpydiff.py to generate docs differences.
This patch introduces the a small framework to track differences between
uPy and CPython.  The framework consists of:

- A set of "tests" which test for an individual feature that differs between
  uPy and CPy.  Each test is like a normal uPy test in the test suite, but
  has a special comment at the start with some meta-data: a category (eg
  syntax, core language), a human-readable description of the difference, a
  cause, and a workaround.  Following the meta-data there is a short code
  snippet which demonstrates the difference.  See tests/cpydiff directory
  for the initial set of tests.

- A program (this patch) which runs all the tests (on uPy and CPy) and
  generates nicely-formated .rst documenting the differences.

- Integration into the docs build so that everything is automatic, and the
  differences appear in a way that is easy for users to read/reference (see
  latter commits).

The idea with using this new framework is:

- When a new difference is found it's easy to write a short test for it,
  along with a description, and add it to the existing ones.  It's also easy
  for contributors to submit tests for differences they find.

- When something is no longer different the tool will give an error and
  difference can be removed (or promoted to a proper feature test).
2017-02-20 17:14:34 +11:00
Damien George 6a11048af1 py/persistentcode: Bump .mpy version due to change in bytecode. 2017-02-17 00:19:34 +11:00
Paul Sokolovsky aac2db9aaf tools/upip: Update to 1.1.5. Better and more user-friendly error handling. 2017-02-10 20:18:05 +03:00
Damien George f1db8a3097 qemu-arm: Don't compile tests in "REPL" mode.
Previous to this patch the qemu-arm tests were compiled with is_relp=true
meaning that the __repl_print__ function was called for all lines of code
in the outer scope.  This is not the right behaviour for scripts that are
executed as though they were a file (eg tests).

With this fix the micropython/heapalloc_str.py test now works so it is
removed from the test blacklist.
2017-01-27 12:35:46 +11:00
Paul Sokolovsky 4614403f63 tools/tinytest-codegen.py: Blacklist heapalloc_str.py test for qemu-arm. 2017-01-27 01:19:36 +03:00
Damien George 98458a46ec tools/mpy-tool.py: Add support for OPT_CACHE_MAP_LOOKUP_IN_BYTECODE.
With caching of map lookups in the bytecode, frozen bytecode can still
work but must be stored in RAM, not ROM.  This patch allows mpy-tool.py to
generate code that works with this optimisation, but it's not recommended
to use it on embedded targets (because of lack of RAM).
2017-01-05 15:52:52 +11:00
Damien George 29b5879613 tools/pyboard.py: Refactor so target is not reset between scripts/cmd.
Previous to this patch pyboard.py would open a new serial connection to
the target for each script that was run, and for any command that was run.
Apart from being inefficient, this meant that the board was soft-reset
between scripts/commands, which precludes scripts from accessing variables
set in a previous one.

This patch changes the behaviour of pyboard.py so that the connection to
the target is created only once, and it's not reset between scripts or any
command that is sent with the -c option.
2016-12-15 11:29:33 +11:00
Paul Sokolovsky 30cfdc29ed tools/tinytest-codegen: Blacklist recently added uheapq_timeq test (qemu-arm). 2016-11-12 03:24:36 +03:00
Paul Sokolovsky 61d74fdef8 tools, unix: Replace upip tarball with just source files.
To make its inclusion as frozen modules in multiple ports less magic.
Ports are just expected to symlink 2 files into their scripts/modules
subdirs.

Unix port updated to use this and in general follow frozen modules setup
tested and tried on baremetal ports, where there's "scripts" predefined
dir (overridable with FROZEN_DIR make var), and a user just drops Python
files there.
2016-11-07 18:39:41 +03:00
Damien George ca973bd308 qemu-arm: Enable software floating point support, and float tests.
This helps to test floating point code on Cortex-M hardware.

As part of this patch the link-time-optimisation was disabled because it
wasn't compatible with software FP support.  In particular, the linker
could not find the __aeabi_f2d, __aeabi_d2f etc functions even though they
were provided by lib/libm/math.c.
2016-11-03 12:28:31 +11:00
Paul Sokolovsky 796b720dbc tools/tinytest-codegen: Exclude ticks_diff test for qemu-arm port. 2016-10-30 22:24:07 +03:00
Alex March 52aa532050 qemu-arm: Exclude new vfs_fat tests. 2016-10-27 12:25:28 +11:00
Paul Sokolovsky f7aa692093 tools/check_code_size.sh: Code size validation script for CI. 2016-10-25 11:43:55 +03:00
Paul Sokolovsky b78144c64d tools/pip-micropython: Remove deprecated wrapper tool.
Deprecated for long time, pip-micropython now can't install packages
optimized for low-heap ports (like whole of micropython-lib).
2016-10-24 16:52:15 +03:00
Alex March 06e7032906 qemu-arm: Exclude extmod/vfs_fat_fileio.py test. 2016-10-24 12:49:19 +11:00
Paul Sokolovsky 197a5724d8 tools: Upgrade upip to 1.1.4.
Fix error on unix when installing to non-existing absolute path.
2016-10-22 21:14:58 +03:00
Damien George b0a15aa735 qemu-arm: Enable lots of extmods and enable tests for them.
The qemu-arm port is used for testing of ARM Thumb architecture on a
desktop so should have many features enabled.
2016-10-14 00:08:19 +11:00
Paul Sokolovsky cdbeee0c50 tools: Upgrade upip to 1.1.3.
Initial support for running on a baremetal, low-heap systems (like esp8266),
using Python module interface.
2016-10-12 18:55:31 +03:00
Paul Sokolovsky cecf6bee97 tools: Upgrade upip to 1.0.
Fully self-hosted release (without fallbacks), and uses stream gzip
decompression (step towards support for limited-heap baremetal systems).
2016-10-05 00:34:09 +03:00
Paul Sokolovsky 4c63986101 tools: Update upip to 0.8. Fixes IPv6 support. 2016-09-23 15:35:06 +03:00
Damien George 7df9291b6c py: Update opcode format table because 3 opcodes were removed, 1 added.
LIST_APPEND, MAP_ADD and SET_ADD have been removed, and STORE_COMP has
been added in adaf0d865c.
2016-09-23 12:48:57 +10:00
Damien George c51c883cc8 tools/mpy-tool.py: Support freezing of complex numbers. 2016-09-03 00:19:02 +10:00
Damien George b6bdf18deb tools/mpy-tool.py: Compute the hash value for str/bytes objects.
This makes it more efficient at runtime to hash str/bytes objects.
2016-09-02 15:10:45 +10:00
Damien George b4790afdaf tools/mpy-tool.py: Store qstr config values in global config object.
Makes it easier to access them without passing around another dict of the
config values.
2016-09-02 15:09:21 +10:00
Damien George 72ae3c72c7 tools/mpy-tool.py: Support freezing float literals with obj-repr C.
The tool now generates code for freezing floats in obj-repr A, B or C,
with the specific representation detected at compile time using macros.
2016-08-10 13:26:11 +10:00
Paul Sokolovsky cbe544a76d tools: Upgrade upip to 0.7.
SSL certificate warning, switch to uerrno, and better usage message.
2016-05-28 02:57:53 +03:00
Damien George 25a42fb6ef tools/mpy-tool.py: Don't strip directories from the frozen source name.
Directories are now supported by the frozen import system (to implement
frozen packages) so we should keep them.
2016-05-23 13:29:03 +01:00
Damien George 9b4c013823 tools/mpy-tool.py: Include .py extension in frozen filename.
So that it can be correctly stat'd when looking for frozen files.
2016-05-23 12:46:02 +01:00
Paul Sokolovsky daa4793578 tools/make-frozen: Update for latest changes in frozen modules support.
Frozen modules are now stored with extensions and with '/' as path
separator. In other words, frozen modules paths stored as they are
in normal filesystem.
2016-05-21 21:39:27 +03:00
Damien George 99b4719357 tools/mpy-tool.py: Add checks for critical configuration vars.
When an mpy file is frozen it must know the values of certain
configuration variables.  This patch provides an explicit check in the
generated C file that the configuration variables are what they are
supposed to be.
2016-05-16 23:13:30 +01:00
Paul Sokolovsky 7b1bf0c308 tools/make-frozen.py: Quick fix to support package-modules.
It allows to "import foo.bar", but not "from foo import bar".
2016-05-14 16:30:02 +03:00
Damien George 5985e41afc tools/make-frozen.py: Properly escape hex chars when making C strings. 2016-05-13 13:12:01 +01:00
Damien George 02fd83bcbc tools/mpy-tool: Make sure that all C-level variables are unique.
Fixes issue #2023.
2016-05-03 12:24:39 +01:00
Damien George c3beb16db3 tools/mpy-tool.py: Add support for Python 2.7. 2016-04-15 11:56:10 +01:00
Damien George 0a2e9650f5 py: Add ability to have frozen persistent bytecode from .mpy files.
The config variable MICROPY_MODULE_FROZEN is now made of two separate
parts: MICROPY_MODULE_FROZEN_STR and MICROPY_MODULE_FROZEN_MPY.  This
allows to have none, either or both of frozen strings and frozen mpy
files (aka frozen bytecode).
2016-04-13 16:07:47 +01:00
Damien George 0699c6bf9e tools: Add mpy-tool.py, to work with .mpy files.
Currently it can freeze .mpy files.
2016-04-13 16:05:43 +01:00
Damien George be989be861 qemu-arm: Enable builtin override feature, and enable more tests.
Hopefully these tests run reliably on Travis.
2016-03-15 13:45:32 +00:00
Paul Sokolovsky 17f324b836 py/frozenmod: Store frozen module names together, to quickly scan them. 2016-01-03 18:08:45 +02:00
Paul Sokolovsky 0992588811 py/frozenmod: Make frozen module content be 0-terminated.
To allow simple zero-terminated lexers.
2016-01-01 07:51:07 +02:00
Damien George 4bd95f8b44 tools: Add C middle-processor to make builtin tables proper hash tables. 2015-12-31 00:24:33 +00:00
Paul Sokolovsky bb7f5b5501 tools: Upgrade upip to 0.6.3.
Updated for _os -> uos builtin module rename.
2015-12-12 00:20:10 +02:00
Damien George bbe2e22fcb tools: Fix pyboard.py to work under Python 3. 2015-12-08 12:55:28 +00:00
Peter Hinch f4fcc14cfb tools: Add option to pyboard.py to wait for serial device to be ready.
Also prints a nicer error message if the serial connection could not be
established.
2015-12-08 11:10:09 +00:00
Dave Hylands 519cef813e tools: Allow pyboard.py to work when boot.py prints things. 2015-12-06 21:45:10 +00:00
Paul Sokolovsky f8bc3f6964 tools: Update to upip 0.6.2. Fixes issue due to MacOSX undocumented behavior. 2015-11-15 00:04:18 +02:00
Paul Sokolovsky 3862ef9a9f tools: Update upip to 0.6.1. Fixes normal installs without -p switch. 2015-11-14 17:54:30 +02:00
Paul Sokolovsky 0aff87b808 tools: Update upip to 0.6.
Removes FFI dependency, instead uses builtin os module. Thus can work on
systems where dynamic library loading is not available.
2015-11-14 00:05:24 +02:00
Paul Sokolovsky 89f2ddd2d0 tools/pyboard.py: Don't add terminating \x04 character to stdout output. 2015-11-07 13:48:35 +00:00
Paul Sokolovsky e6dccaf18e tools/pyboard.py: Make -c (inline Python code) option compatible with python2. 2015-11-07 12:16:11 +03:00
Tom Soulanille a787467569 tools/pyboard: Add -c argument to run a program passed as a string. 2015-10-19 23:20:59 +01:00
Paul Sokolovsky 6a515b95a8 tools/upip: Update to 0.5.9.
MICROPYPATH environment variable is now honored, package are installed to
first path specified in it.
2015-10-17 20:16:41 +03:00
Damien George 0334058fa4 Rename "Micro Python" to "MicroPython" in REPL, help, readme's and misc. 2015-10-12 00:06:25 +01:00
Dave Hylands 01d64914c5 stmhal: Fix USB CDC-only mode under Windows.
This fix adds PIDs 9801 and 9802 to the pybcdc.inf file.

When in CDC only mode, it presents itself as a Communcations
device rather than as a composite device. Presenting as a
composite device with only the CDC interface seems to confuse
windows.

To test and make sure that the correct pybcdc.inf was being used,
I used USBDeview from http://www.nirsoft.net/utils/usb_devices_view.html
to uninstall any old pyboard drivers (Use Control-F and search
for pyboard). I found running USBDeview as administrator worked best.

Installing the driver in CDC+MSC mode first is recommended (since the
pybcdc.inf file in on the internal flash drive). Then when you switch
modes everything seems to work properly.

I used https://github.com/dhylands/upy-examples/blob/master/boot_switch.py
to easily switch the pyboard between the various USB modes for testing.
2015-10-09 00:18:01 +01:00
Paul Sokolovsky a5deadf082 tools: Upgrade upip to 0.5.8.
Adds support for ussl module (which is experimental and not fully
implemented yet itself).
2015-09-13 09:49:09 +03:00
Paul Sokolovsky e8ad47a6ca tools: Upgrade to upip 0.5.7.
Just dependent micropython-lib modules update for upip, no new
functionality.
2015-08-30 11:04:38 +03:00
Damien George b7d59060e2 tools: Make gen-changelog.sh print more lines from the tag annotation. 2015-08-11 13:50:40 +01:00
Damien George e86b47175f tools: Make pyboard.py Python2 compatible. 2015-08-08 13:03:08 +01:00
Damien George 7ccdf8be77 tools/pyboard.py: Fix read timeout calc to work with shorter sleep. 2015-07-29 00:21:22 +01:00
Daniel Campora cd14188bc8 tools: Add telnet support to pyboard.py.
The adapter class "TelnetToSerial" is used to access the Telnet
connection using the same API as with the serial connection. The
function pyboard.run-test() has been removed to made the module
generic and because this small test is no longer needed.
2015-07-26 14:02:34 +01:00
Damien George db109ca0fc tools/pyboard.py: Speed up reading of chars by decreasing sleep period. 2015-07-25 22:49:25 +01:00
Damien George f1236734bb tools/pyboard.py: Make enter_raw_repl stricter and more reliable.
When looking for chars to indicate raw repl is active, look for the full
string of chars to improve reliability of entering raw repl correctly.

Previous to this patch there was the possibility that raw repl was
entered in a dirty state, where not all input chars from previous
invocation were drained.
2015-07-25 22:46:07 +01:00
Damien George 9de53bf788 tools/pyboard.py: Fix parsing of returned error so last chr is not lost. 2015-07-25 22:44:56 +01:00
Dave Hylands c91727b75a tools: Fix pydfu.py to work with old and new versions of PyUSB
Update pydfu.py to match with the version from openmv.

I just updated the openmv version to work with both of the
PyUSB 1.0.0.b1 and 1.0.0.b2

See: https://github.com/walac/pyusb/blob/master/ReleaseNotes.rst
2015-07-06 09:56:12 -07:00
Paul Sokolovsky 6ab8b63bdd tools/make-frozen.py: Use Python2-compatible shebang. 2015-07-06 14:48:29 +03:00
Paul Sokolovsky de575c80b9 tools/make-frozen.py: Actually make Python2-compatible. 2015-07-06 14:27:57 +03:00
Paul Sokolovsky 3a2e9f20f6 tools/make-frozen.py: Add Python2 compatibility. 2015-07-06 14:09:16 +03:00
Paul Sokolovsky 484adac0bb tools: Update upip to 0.5.4.
Recognize and handle "package not found" error.
2015-06-25 15:32:14 +03:00
Damien George d59ca4efdb tools/pyboard.py: Change logic for when raw ">" prompt is parsed.
In raw REPL ">" indicates the prompt.  We originally read this character
upon entering the raw REPL, and after reading the last bit of the
output.  This patch changes the logic so the ">" is read only just
before trying to send the next command.  To make this work (and as an
added feature) the input buffer is now flushed upon entering raw REPL.

The main reason for this change is so that pyboard.py recognises the EOF
when sys.exit() is called on the pyboard.  Ie, if you run pyboard.py
with a script that calls sys.exit(), then pyboard.py will exit after
the sys.exit() is called.
2015-06-15 23:28:04 +01:00
Paul Sokolovsky dd0e24f4b0 pip-micropython: Add deprecation notice, but still leave for reference. 2015-06-13 21:39:57 +03:00
Paul Sokolovsky 098f5ae221 micropython-upip: Mark as binary file and re-commit. 2015-06-06 23:09:23 +03:00
Paul Sokolovsky 1406d9ccde upip: Upgrade to 0.5.3 to follow uctypes.struct() signature change. 2015-06-06 23:03:28 +03:00
Paul Sokolovsky f8a39e3bb1 tools/make-frozen.py: Handle trailing slash in argument more reliably. 2015-06-03 19:28:31 +03:00
Damien George 0e6c89a5c8 tools: Add codestats.sh to compute code statistics such as size, speed. 2015-06-03 15:38:43 +00:00
Paul Sokolovsky 0df20da4dd tools: Add upip 0.5.2 tarball.
So unix version, which now includes upip as a frozen module, can be built
without Internet connection.
2015-06-02 01:32:07 +03:00
Paul Sokolovsky c02dc8b2c9 tools/make-frozen.py: Open files in binary mode. 2015-05-30 00:39:00 +03:00
Paul Sokolovsky a8e7c03171 tools/make-frozen.py: Preserve directory hierarchy.
Currently, frozen packages are not supported, but eventually they should be,
so make sure to store complete directory hierarchy.
2015-05-30 00:23:46 +03:00
Paul Sokolovsky f5ae384d4f bootstrap_upip.sh: Allow to install into a path given by argument. 2015-05-29 22:35:45 +03:00
Dave Hylands 669dbca959 tools: Allow pyboard constructor to take a baudrate parameter.
This allows pyboard.py to be used over a UART interface
rather than just over a USB serial interface.
2015-05-27 23:01:28 +01:00
Dave Hylands 97ce883217 tools: Add exec_raw_no_follow to pyboard.py. 2015-05-18 11:39:57 +01:00
Paul Sokolovsky 76abb2e623 tools: Add script to install "upip" package manager.
upip is a simple and light-weight package manager for MicroPython modules,
offering subset of pip functionality. upip is part of micropython-lib
project: https://github.com/micropython/micropython-lib/tree/master/upip
This script bootstraps upip by downloading and unpacking it directly from
PyPI repository, with all other packages to be installed with upip itself.
2015-05-07 16:00:40 +03:00
Damien George 4865a22f78 tools/pyboard.py: Add "--follow" option to wait for output indefinitely.
Also flush stdout so you can see output as it comes.
2015-05-07 10:14:51 +01:00
Damien George 23a2b11abf tools: Add STM32F4DISC and ESPRUINO_PICO to stmhal build script. 2015-04-21 13:16:41 +00:00
Damien George 2bb5f41611 tools/pyboard.py: Make it 8-bit clean, so it works with unicode chars.
Addresses issue #1190.
2015-04-19 21:31:28 +01:00
Damien George d8475092d1 qemu-arm: Disable inline asm tests that use floating point. 2015-04-19 15:50:24 +01:00
Damien George 39ce2db181 stmhal: Add "CDC" option to pyb.usb_mode, for CDC device only. 2015-02-13 14:02:51 +00:00
Damien George fb2006cc69 tools: Make gen-changelog.sh sort version strings correctly. 2015-02-13 13:31:02 +00:00
Damien George 2f8c409a4f qemu-arm: Disable try_finally1.py test (it fails randomly on Travis). 2015-01-25 18:48:03 +00:00
Paul Sokolovsky 640e0b221e py: Implement very simple frozen modules support.
Only modules (not packages) supported now. Source modules can be converted
to frozen module structures using tools/make-frozen.py script.
2015-01-20 11:52:12 +02:00
Damien George b58da9420c qemu-arm: Disable basics/memoryerror.py test. 2015-01-12 16:32:14 +00:00
Damien George 99dde4ed1f qemu-arm: Enable GC and native code-gen; enable more tests. 2015-01-12 12:07:42 +00:00
Damien George 85e8e2ed5b qemu-arm: Add 'test' target to Makefile to run and verify test suite.
Replaces RUN_TEST=1 definition; now "make test" in qemu-arm directory
will run tests/basics/ and check that they all succeed.

This patch also enables the test on Travis CI.
2015-01-09 00:03:21 +00:00
Damien George e3fa8278b4 tools: Add script to generate a ChangeLog file.
We don't have an explicit ChangeLog file, but don't really need one
because we use a good version control system.  This script is useful if
you need a pretty-printed ChangeLog for some reason.
2014-12-29 19:03:25 +00:00
Damien George 4fd7c1a2ac tools, pyboard.py: Write data to pyboard in chunks of 256 bytes.
This speeds up writes significantly.
2014-12-20 18:09:04 +00:00
Damien George 17c5ce3727 tools: Make pyboard.py have infinite timeout when running script.
This makes pyboard.py much more useful for long running scripts.  When
running a script via pyboard.py, it now waits until the script finishes,
with no timeout.  CTRL-C can be used to break out of the waiting if
needed.
2014-11-30 21:30:53 +00:00
Damien George 7c4445afe1 tools: Make gendoc.py able to output RST format. 2014-10-31 01:36:11 +00:00
Damien George bc1488a05f stmhal: Improve REPL control codes; improve pyboard.py script.
Improvements are:

2 ctrl-C's are now needed to truly kill running script on pyboard, so
make CDC interface allow multiple ctrl-C's through at once (ie sending
b'\x03\x03' to pyboard now counts as 2 ctrl-C's).

ctrl-C in friendly-repl can now stop multi-line input.

In raw-repl mode, use ctrl-D to indicate end of running script, and also
end of any error message.  Thus, output of raw-repl is always at least 2
ctrl-D's and it's much easier to parse.

pyboard.py is now a bit faster, handles exceptions from pyboard better
(prints them and exits with exit code 1), prints out the pyboard output
while the script is running (instead of waiting till the end), and
allows to follow the output of a previous script when run with no
arguments.
2014-10-26 15:39:22 +00:00
Damien George 5fc42a6c97 tools, pydfu: Some fixes to support Python 3. 2014-10-22 20:27:43 +01:00
Dave Hylands 842210f53a Add pydfu.py to the micropython tree. Use dfu_util bgy default
You can do:

make USE_PYDFU=1 deploy

to use pydfu.py
2014-10-22 20:18:38 +01:00
Damien George 9c9db3a7a1 tools, pyboard.py: Allow exec argument to be bytes or str. 2014-10-19 14:54:52 +01:00
blmorris 3b064370f8 Enable device keyword option when running pyboard.py --tests and run-tests --pyboard 2014-10-01 23:31:52 +01:00
Damien George c737086e1c tools: Add network build to auto-build script. 2014-09-30 23:04:08 +00:00
Damien George 2842945e76 stmhal: Fix bugs in documentation so it compiles. 2014-09-17 23:27:42 +00:00
Paul Sokolovsky f3c3010ffc pip-micropython: Revert to using PIP_MICROPY_DEST environment var.
-t/--target is a pip option. Trying to use pip options for different meanings
in pip-micropython may lead to big confusion. That's why the original passed
any extra parameters using environment variables. "All options belong to pip."
2014-08-27 02:53:06 +03:00
Damien George db63660c19 Add pip-micropython to unix make install.
Also add -t/--target option to pip-micropython to allowing installing to
the pyboard.

Thanks to turbinenreiter/Sebastian Plamauer for the patch.
2014-08-26 16:03:57 +01:00
Damien George c4ee39dd63 tools, gendoc: Output small descr about module TOC. 2014-08-10 16:51:26 +01:00
Damien George 590b2abdfc tools, gendoc: Put module TOC/index in module/ directory. 2014-08-08 23:34:39 +01:00
Damien George ef7a066c9c tools, gendoc: Allow constants at module level; gen module index.
Addresses some issues from #585.
2014-08-08 23:29:05 +01:00
Paul Sokolovsky c10a4405cd gendoc.py: Support modules w/o functions and/or classes.
I.e. don't assume that both are always present.
2014-06-29 15:48:30 +03:00
Damien George 3793830ed9 tools: Move gendoc.py to tools, and make it a little more generic. 2014-05-10 19:12:47 +01:00
Ilya Dmitrichenko be86596bb9 tools: inline test suite generator. 2014-05-08 01:41:22 +03:00
Ilya Dmitrichenko b1442e04d1 tools: check-in errordeveloper/tinytest@eb2dbc858f 2014-05-08 01:41:21 +03:00
Damien George 37936bebbf tools: In build-stm-latest, replace git hash with git tag. 2014-05-04 00:11:09 +00:00
Damien George 3244123031 tools: pyboard.py now acts as a command-line program to run scripts.
You can run a local script on the pyboard using:

    python pyboard.py test.py

where test.py is the local script you want to run.
2014-05-03 18:14:34 +01:00
Damien George aad1204b8e stmhal: Change names: flash to firmware, flashboard to deploy.
Since firmware lives in a build directory which already specifies the
board name, no real reason to also have the firmware have the board
name.
2014-05-03 17:46:45 +01:00
Damien George e88814a274 Merge pull request #543 from lurch/patch-4
Make pyboard.enter_raw_repl more robust
2014-05-03 13:09:16 +01:00
Damien George d9f4deef8c Merge pull request #547 from cjbarnes18/pyboard_module_fixes
Fix pyboard.py tests.
2014-05-03 00:11:43 +01:00
Craig Barnes 71a1e35206 Fix pyboard.py tests. 2014-05-02 21:36:07 +01:00
Damien George ba742ed639 Add shell script to automatically build stmhal/ for all pyboards. 2014-05-02 13:49:44 +00:00
Andrew Scheller 1f740bdaf4 Make pyboard.enter_raw_repl more robust
In case there's a program in the microcontroller's main.py running in an infinite loop
2014-05-01 20:42:23 +01:00
Damien George ca045d97b9 tools: Update pyboard.py to work with latest changes to RTC. 2014-04-21 20:18:37 +01:00
Damien George ecdf6cdc5b tools: Fix up pybcdc.inf generation: new lines and hex digits.
Using Python's file open in 'r' mode opens it for text reading, which
converts all new lines to \n.  Could use 'rb' binary mode, but then
don't have access to the string Template replacement functions.  Thus,
force the output to have '\\r\\n' ending.

Also fix regex to match hex digits.
2014-04-16 11:45:52 +01:00
Andrew Scheller 1452221aca Auto-generate the stmhal/pybcdc_inf header file from static files
The USB VID&PID are automatically extracted from usbd_desc_cdc_msc.c
and inserted into pybcdc_inf.template, ensuring that the same USB
IDs get used everywhere
2014-04-16 02:41:26 +01:00
Paul Sokolovsky cc8c0882a0 pip-micropython: Fix inverted condition. 2014-04-15 02:32:41 +03:00
Andrew Scheller 6fc40cc3da pip-micropython: revert $HOME back to ~ 2014-04-14 22:26:34 +01:00
Damien George a7239b2fc2 Merge pull request #480 from lurch/patch-2
Make pip-micropython more portable
2014-04-14 21:39:31 +01:00
Andrew Scheller 0aac7ba344 Make pip-micropython more portable
Replace /tmp with a $TMPDIR variable
Replace multiple occurrences of /tmp/pip-micropy-venv with a $TMPVENV variable
Replace ~ with $HOME
2014-04-14 12:26:12 +01:00
Damien George 48a9b3fd11 tools: Improve timout/reading of pyboard.py. 2014-04-14 01:47:36 +01:00
Damien George b636d024d2 Make pyboard.py have its own exception; update run-tests for pyboard. 2014-04-13 13:48:33 +01:00