Commit Graph

536 Commits

Author SHA1 Message Date
Damien George 7f24c29778 tools/mpy-tool.py: Support qstr linking when freezing Xtensa native mpy. 2019-11-28 13:11:51 +11:00
Yonatan Goldschmidt b2dd443d92 tools/makemanifest.py: Use sys.executable when invoking Python scripts.
So the version of Python used to run makemanifest.py is also used for the
sub-scripts.
2019-11-13 13:44:19 +11:00
Andrew Leech d2e6cfd8fd tools/makemanifest.py: Skip freezing unsupported files with warning. 2019-11-07 12:34:57 +11:00
Mirko Vogt 2f71d66ef7 tools/makemanifest.py: Follow symlinks when freezing linked directories.
While the new manifest.py style got introduced for freezing python code
into the resulting binary, the old way - where files and modules within
ports/*/modules where baked into the resulting binary - was still
supported via `freeze('$(PORT_DIR)/modules')` within manifest.py.

However behaviour changed for symlinked directories (=modules), as those
links weren't followed anymore.

This commit restores the original behaviour by explicitly following
symlinks within a modules/ directory
2019-11-06 11:41:06 +11:00
Damien George 36c9be6f60 tools/mpy-tool.py: Use "@progbits #" attribute for native xtensa code. 2019-11-04 15:31:42 +11:00
Jim Mussared 8ba963cfa3 tools/makemanifest.py: Eval relative paths w.r.t. current manifest file.
When loading a manifest file, e.g. by include(), it will chdir first to the
directory of that manifest.  This means that all file operations within a
manifest are relative to that manifest's location.

As a consequence of this, additional environment variables are needed to
find absolute paths, so the following are added: $(MPY_LIB_DIR),
$(PORT_DIR), $(BOARD_DIR).  And rename $(MPY) to $(MPY_DIR) to be
consistent.

Existing manifests are updated to match.
2019-10-21 23:01:41 +11:00
Damien George e81f538e25 tools: Add mechanism to provide a manifest of frozen files.
This introduces a new build variable FROZEN_MANIFEST which can be set to a
manifest listing (written in Python) that describes the set of files to be
frozen in to the firmware.
2019-10-15 21:34:23 +11:00
Damien George 8e8cfa6f53 tools/make-frozen.py: Allow to run with no directory passed in.
In which case it will just emit empty frozen C definitions.
2019-10-15 21:33:49 +11:00
Damien George 23f0691fdd py/persistentcode: Make .mpy more compact with qstr directly in prelude.
Instead of encoding 4 zero bytes as placeholders for the simple_name and
source_file qstrs, and storing the qstrs after the bytecode, store the
qstrs at the location of these 4 bytes.  This saves 4 bytes per bytecode
function stored in a .mpy file (for example lcd160cr.mpy drops by 232
bytes, 4x 58 functions).  And resulting code size is slightly reduced on
ports that use this feature.
2019-10-15 16:56:27 +11:00
Damien George 9adedce42e py: Add new Xtensa-Windowed arch for native emitter.
Enabled via the configuration MICROPY_EMIT_XTENSAWIN.
2019-10-05 13:44:53 +10:00
Damien George c8c0fd4ca3 py: Rework and compress second part of bytecode prelude.
This patch compresses the second part of the bytecode prelude which
contains the source file name, function name, source-line-number mapping
and cell closure information.  This part of the prelude now begins with a
single varible length unsigned integer which encodes 2 numbers, being the
byte-size of the following 2 sections in the header: the "source info
section" and the "closure section".  After decoding this variable unsigned
integer it's possible to skip over one or both of these sections very
easily.

This scheme saves about 2 bytes for most functions compared to the original
format: one in the case that there are no closure cells, and one because
padding was eliminated.
2019-10-01 12:26:22 +10:00
Damien George b5ebfadbd6 py: Compress first part of bytecode prelude.
The start of the bytecode prelude contains 6 numbers telling the amount of
stack needed for the Python values and exceptions, and the signature of the
function.  Prior to this patch these numbers were all encoded one after the
other (2x variable unsigned integers, then 4x bytes), but using so many
bytes is unnecessary.

An entropy analysis of around 150,000 bytecode functions from the CPython
standard library showed that the optimal Shannon coding would need about
7.1 bits on average to encode these 6 numbers, compared to the existing 48
bits.

This patch attempts to get close to this optimal value by packing the 6
numbers into a single, varible-length unsigned integer via bit-wise
interleaving.  The interleaving scheme is chosen to minimise the average
number of bytes needed, and at the same time keep the scheme simple enough
so it can be implemented without too much overhead in code size or speed.
The scheme requires about 10.5 bits on average to store the 6 numbers.

As a result most functions which originally took 6 bytes to encode these 6
numbers now need only 1 byte (in 80% of cases).
2019-10-01 12:26:22 +10:00
Damien George 5716c5cf65 py/persistentcode: Bump .mpy version to 5.
The bytecode opcodes have changed (there are more, and they have been
reordered).
2019-09-26 16:39:37 +10:00
Josh Lloyd 7d58a197cf py: Rename MP_QSTR_NULL to MP_QSTRnull to avoid intern collisions.
Fixes #5140.
2019-09-26 16:04:56 +10:00
Damien George 1f7202d122 py/bc: Replace big opcode format table with simple macro. 2019-09-26 15:27:11 +10:00
Damien George 5889cf58db py/bc0: Order opcodes into groups based on their size and format. 2019-09-26 15:27:10 +10:00
Damien George c69f58e6b9 tools/mpy-tool.py: Fix freezing of non-bytecode funcs with settrace.
Only bytecode functions can be profiled at this stage.  Native functions
(eg inline assembler) may not even have a valid prelude.

Fixes issue #5075.
2019-09-06 23:55:15 +10:00
Damien George b29fae0c56 py/bc: Fix size calculation of UNWIND_JUMP opcode in mp_opcode_format.
Prior to this patch mp_opcode_format would calculate the incorrect size of
the MP_BC_UNWIND_JUMP opcode, missing the additional byte.  But, because
opcodes below 0x10 are unused and treated as bytes in the .mpy load/save
and freezing code, this bug did not show any symptoms, since nested unwind
jumps would rarely (if ever) reach a depth of 16 (so the extra byte of this
opcode would be between 0x01 and 0x0f and be correctly loaded/saved/frozen
simply as an undefined opcode).

This patch fixes this bug by correctly accounting for the additional byte.
        .
2019-09-02 13:30:16 +10:00
Damien George 4691b43c8a tools/mpy-tool.py: Add initial support for frozen with settrace. 2019-08-30 16:49:13 +10:00
Milan Rossa 498e35219e tests: Add tests for sys.settrace feature. 2019-08-30 16:48:22 +10:00
Jim Mussared 0bd1eb80ff qemu-arm: Add testing of frozen native modules.
- Split 'qemu-arm' from 'unix' for generating tests.
- Add frozen module to the qemu-arm test build.
- Add test that reproduces the requirement to half-word align native
  function data.
2019-08-20 15:14:08 +10:00
Jim Mussared 4ab5156c01 tools/mpy-tool.py: Force native func alignment to halfword/word on ARM.
This is necessary for ARMV6 and V7.  Without this change, calling a frozen
native/viper function that is misaligned will crash.
2019-08-20 15:13:17 +10:00
Damien George 4d94fae833 tools/pyboard.py: Add filesystem commands to ls/cat/cp/rm remote files.
Use "-f" to select filesystem mode, followed by the command to execute.
Optionally put ":" at the start of a filename to indicate that it's on the
remote device, if it would otherwise be ambiguous.

Examples:

    $ pyboard.py -f ls
    $ pyboard.py -f cat main.py
    $ pyboard.py -f cp :main.py .   # get from device
    $ pyboard.py -f cp main.py :    # put to device
    $ pyboard.py -f rm main.py
2019-07-25 15:56:01 +10:00
Damien George f073f2b543 tools: Add uf2conv.py from Microsoft/uf2 repository.
Repository https://github.com/Microsoft/uf2 commit
19615407727073e36d81bf239c52108ba92e7660
2019-07-01 17:18:44 +10:00
Jun Wu b152bbddd1 py: Define EMIT_MACHINE_CODE as EMIT_NATIVE || EMIT_INLINE_ASM.
The combination MICROPY_EMIT_NATIVE || MICROPY_EMIT_INLINE_ASM is used in
many places, so define a new macro for it.
2019-06-28 13:54:45 +10:00
Damien George 9d3031cc9d tools/mpy-tool.py: Fix linking of qstr objects in native ARM Thumb code.
Previously, when linking qstr objects in native code for ARM Thumb, the
index into the machine code was being incremented by 4, not 8.  It should
be 8 to account for the size of the two machine instructions movw and movt.
This patch makes sure the index into the machine code is incremented by the
correct amount for all variations of qstr linking.

See issue #4829.
2019-06-11 11:36:39 +10:00
Damien George faf3d3e9e9 tools/mpy-tool.py: Fix linking qstrs in native code, and multiple files.
Fixes errors in the tool when 1) linking qstrs in native ARM-M code; 2)
freezing multiple files some of which use native code and some which don't.

Fixes issue #4829.
2019-06-04 22:21:01 +10:00
Damien George 993ca572ca tools/upip.py: Add support for multiple index URLs with custom default.
The user can now select their own package index by either passing the "-i"
command line option, or setting the upip.index_urls variable (before doing
an install).

The https://micropython.org/pi package index hosts packages from
micropython-lib and will be searched first when installing a package.  If a
package is not found here then it will fallback to PyPI.
2019-05-15 15:46:16 +10:00
Damien George 56f6ceba7f tools/pyboard.py: Don't accumulate output data if data_consumer used.
Prior to this patch, when a lot of data was output by a running script
pyboard.py would try to capture all of this output into the "data"
variable, which would gradually slow down pyboard.py to the point where it
would have large CPU and memory usage (on the host) and potentially lose
data.

This patch fixes this problem by not accumulating the data in the case that
the data is not needed, which is when "data_consumer" is used.
2019-04-25 13:24:32 +10:00
Damien George 74ed06828f tools/mpy-tool.py: Fix init of QStrWindow, and remove unused variable.
The qstr window size is not log-2 encoded, it's just the actual number (but
in mpy-tool.py this didn't lead to an error because the size is just used
to truncate the window so it doesn't grow arbitrarily large in memory).

Addresses issue #4635.
2019-04-08 15:24:24 +10:00
Damien George 643d2a0e86 tools/mpy-tool.py: Adjust use of super() to make it work with Python 2.
Fixes the regression introduced in ea3c80a514
2019-04-08 11:21:18 +10:00
rhubarbdog 869a8b70ce tools/pyboard.py: Add missing line from example usage comments. 2019-03-26 16:52:41 +11:00
Andrew Leech 89ff506513 py: Update and rework build system for including external C modules.
How to use this feature is documented in docs/develop/cmodules.rst.
2019-03-08 22:58:42 +11:00
Ayke van Laethem 2e516074da py: Implement a module system for external, user C modules.
This system makes it a lot easier to include external libraries as static,
native modules in MicroPython.  Simply pass USER_C_MODULES (like
FROZEN_MPY_DIR) as a make parameter.
2019-03-08 22:49:00 +11:00
Damien George 6e11d86318 tools/upip.py: Use "raise arg" instead of no-arg raise form, for native. 2019-03-08 16:51:09 +11:00
Damien George 9a5f92ea72 py/persistentcode: Bump .mpy version to 4. 2019-03-08 15:53:05 +11:00
Damien George ea3c80a514 tools/mpy-tool.py: Add support for freezing native code.
This adds support to freeze .mpy files that contain native code blocks.
2019-03-08 15:53:05 +11:00
Damien George 636ed0ff8d py/emitglue: Remove union in mp_raw_code_t to combine bytecode & native. 2019-03-08 15:53:04 +11:00
Damien George 4f0931b21f py/persistentcode: Define static qstr set to reduce size of mpy files.
When encoded in the mpy file, if qstr <= QSTR_LAST_STATIC then store two
bytes: 0, static_qstr_id.  Otherwise encode the qstr as usual (either with
string data or a reference into the qstr window).

Reduces mpy file size by about 5%.
2019-03-05 16:32:05 +11:00
Damien George 992a6e1dea py/persistentcode: Pack qstrs directly in bytecode to reduce mpy size.
Instead of emitting two bytes in the bytecode for where the linked qstr
should be written to, it is now replaced by the actual qstr data, or a
reference into the qstr window.

Reduces mpy file size by about 10%.
2019-03-05 16:27:34 +11:00
Damien George 5996eeb48f py/persistentcode: Add a qstr window to save mpy files more efficiently.
This is an implementation of a sliding qstr window used to reduce the
number of qstrs stored in a .mpy file.  The window size is configured to 32
entries which takes a fixed 64 bytes (16-bits each) on the C stack when
loading/saving a .mpy file.  It allows to remember the most recent 32 qstrs
so they don't need to be stored again in the .mpy file.  The qstr window
uses a simple least-recently-used mechanism to discard the least recently
used qstr when the window overflows (similar to dictionary compression).
This scheme only needs a single pass to save/load the .mpy file.

Reduces mpy file size by about 25% with a window size of 32.
2019-03-05 16:25:07 +11:00
Damien George 5a2599d962 py: Replace POP_BLOCK and POP_EXCEPT opcodes with POP_EXCEPT_JUMP.
POP_BLOCK and POP_EXCEPT are now the same, and are always followed by a
JUMP.  So this optimisation reduces code size, and RAM usage of bytecode by
two bytes for each try-except handler.
2019-03-05 16:09:58 +11:00
Dave Hylands c932639063 tools/pydfu.py: Fix regression so tool runs under Python 2 again.
Under python3 (tested with 3.6.7) bytes with a list of integers as an
argument returns a different result than under python 2.7 (tested with
2.7.15rc1) which causes pydfu.py to fail when run under 2.7.  Changing
bytes to bytearray makes pydfu work properly under both Python 2.7 and
Python 3.6.
2018-12-30 01:20:48 +11:00
Dave Hylands 39eef27083 tools/mpy-tool.py: Fix build error when no qstrs present in frozen mpy.
If you happen to only have a really simple frozen file that doesn't contain
any new qstrs then the generated frozen_mpy.c file contains an empty
enumeration which causes a C compile time error.
2018-12-15 14:36:08 +11:00
Damien George 814d580a15 tools/mpy-tool.py: Fix calc of opcode size for opcodes with map caching.
Following an equivalent fix to py/bc.c.  The reason the incorrect values
for the opcode constants were not previously causing a bug is because they
were never being used: these opcodes always have qstr arguments so the part
of the code that was comparing them would never be reached.

Thanks to @malinah for finding the problem and providing the initial patch.
2018-12-13 01:26:55 +11:00
Damien George 4f25a8b6a4 tools/pydfu.py: Improve DFU reset, and auto-detect USB transfer size.
A DFU device must be in the idle state before it can be programmed, and
this requires either clearing the status or aborting, depending on its
current state.  Code is added to do this.  And the USB transfer size is now
automatically detected so devices with a size less than 2048 bytes work
correctly.
2018-11-27 16:19:27 +11:00
Martin Dybdal 7795b2e5c3 tools/pyboard.py: In TelnetToSerial.close replace try/except with if.
Some Python linters don't like unconditional except clauses because they
catch SystemExit and KeyboardInterrupt, which usually is not the intended
behaviour.
2018-10-19 23:46:10 +11:00
Andrew Leech a2703649ea tools/pydfu: Workaround stdio flush error on Windows with Python 3.6.
There appears to be an issue on Windows with CPython >= 3.6,
sys.stdout.flush() raises an exception:

    OSError: [WinError 87] The parameter is incorrect

It works fine to just catch and ignore the error on the flush line.  Tested
on Windows 10 x64 1803 (Build 17134.228), Python 3.6.4 amd64.
2018-09-21 11:57:24 +10:00
Martin Dybdal 5ed8226e02 tools/pyboard.py: Change base class of PyboardError to Exception.
Following standard practice for defining custom exceptions.
2018-08-10 16:23:38 +10:00
Ayke van Laethem 0d7a088039 tools/pyboard: Run exec: command as a string.
The Python documentation recommends to pass the command as a string when
using Popen(..., shell=True).  This is because "sh -c <string>" is used to
execute the command and additional arguments after the command string are
passed to the shell itself (not the executing command).

https://docs.python.org/3.5/library/subprocess.html#subprocess.Popen
2018-08-04 15:45:23 +10:00
Rich Barlow 6e5a40cf3c tools/mpy-tool: Set sane initial dynamic qstr pool size with frozen mods
The first dynamic qstr pool is double the size of the 'alloc' field of
the last const qstr pool. The built in const qstr pool
(mp_qstr_const_pool) has a hardcoded alloc size of 10, meaning that the
first dynamic pool is allocated space for 20 entries. The alloc size
must be less than or equal to the actual number of qstrs in the pool
(the 'len' field) to ensure that the first dynamically created qstr
triggers the creation of a new pool.

When modules are frozen a second const pool is created (generally
mp_qstr_frozen_const_pool) and linked to the built in pool. However,
this second const pool had its 'alloc' field set to the number of qstrs
in the pool. When freezing a large quantity of modules this can result
in thousands of qstrs being in the pool. This means that the first
dynamically created qstr results in a massive allocation. This commit
sets the alloc size of the frozen qstr pool to 10 or less (if the number
of qstrs in the pool is less than 10). The result of this is that the
allocation behaviour when a dynamic qstr is created is identical with an
without frozen code.

Note that there is the potential for a slight memory inefficiency if the
frozen modules have less than 10 qstrs, as the first few dynamic
allocations will have quite a large overhead, but the geometric growth
soon deals with this.
2018-08-01 18:59:31 +10:00
roland 11a38d5dc5 tools/pydfu.py: Make the DFU tool work again with Python 2.
This patch will work for both Python 2 and 3.
2018-07-27 16:53:50 +10:00
roland feec0a6909 tools/pydfu.py: Use getfullargspec instead of getargspec for newer pyusb
pyusb v1.0.2 warns about `getargspec` as being deprecated.
2018-07-20 14:28:17 +10:00
roland 6e50df4e21 tools/dfu.py: Pad image data to 8 byte alignment to support L476.
Thanks to @dhylands for this patch to pad the image to 8-byte boundaries.
2018-07-20 14:23:11 +10:00
Damien George 44fc92ea7c tools/mpy-tool.py: Put frozen bignum digit data in ROM, not in RAM. 2018-07-09 13:43:34 +10:00
Damien George 929d10acf7 tools/mpy-tool.py: Support freezing of floats in obj representation D. 2018-07-09 12:22:40 +10:00
Damien George 92667dc2e5 tools/pydfu.py: Add support for multiple memory segments.
Segments are separated by / and begin with the memory address.  This
follows how the ST DFU tool works.
2018-06-22 15:32:32 +10:00
Damien George 8fb95d6520 tools/pydfu.py: Increase download packet size to full 2048 bytes.
The ST DFU bootloader supports a transfer size up to 2048 bytes, so send
that much data on each download (to device) packet.  This almost halves
total download time.
2018-06-08 15:32:49 +10:00
Keith Wiley 3ea0862a6e tools/pydfu.py: Fix typo in comments. 2018-05-18 23:33:19 +10:00
Damien George f7be5f9bfa tools/upip: Upgrade upip to 1.2.4.
Uses new pypi.org URL, and now creates a socket with the address parameters
returned by getaddrinfo().
2018-04-23 16:11:27 +10:00
Paul Sokolovsky 4475f32420 tools/tinytest-codegen: Ignore system locale, write output in UTF-8.
Way to reproduce a UnicodeEncodeError without this patch:

    LC_ALL=C tinytest-codegen.py ...
2017-12-15 12:04:10 +02:00
Paul Sokolovsky 325d0fc74b tools/tinytest-codegen: Add --stdin switch instead of recently added --target.
Instead of passing thru more and more options from tinytest-codegen to
run-tests --list-tests, pipe output of run-tests --list-tests into
tinytest-codegen.
2017-12-14 12:26:59 +02:00
Damien George bb516af1eb tools/pydfu.py: Call set_configuration() on fresh USB device object.
This call is required before using the device (some operating systems don't
need it but others do).  Fixes issue #3476.
2017-12-14 10:08:37 +11:00
Paul Sokolovsky 43141ddb55 tools/tinytest-codegen: Take --target= option for test set selection.
Gets passed to run-tests --list-tests to get actual list of tests to use.
If --target= is not given, legacy set hardcoded in tinytest-codegen itself
is used.

Also, get rid of tinytest test groups - they aren't really used for
anything, and only complicate processing. Besides, one of the next
step is to limit number of tests per a generated file to control
the binary size, which also will require "flat" list of tests.
2017-12-13 18:35:37 +02:00
Paul Sokolovsky e6f0d547ab tools/tinytest-codegen: More excludes after enabling expected output match. 2017-12-13 09:07:51 +02:00
Paul Sokolovsky 4db6a7adbe tools/tinytest-codegen: Wrap lines of exclude_tests.
So it was manageable and extensible.
2017-12-13 09:07:51 +02:00
Paul Sokolovsky 48e931e1d3 tools/tinytest-codegen.py: Generate code for upytesthelper.
The way tinytest was used in qemu-arm test target is that it didn't test
much. MicroPython tests are based on matching the test output against
reference output, but qemu-arm's implementation didn't do that, it
effectively tested just that there was no exception during test
execution. "upytesthelper" wrapper was introduce to fix it, and so
test generator is now switched to generate test code for it.

Also, fix PEP8 and other codestyle issues.
2017-12-13 09:07:51 +02:00
Paul Sokolovsky e9d29c9ba9 lib/tinytest: Move from tools/tinytest.
Tinytest library was misplaced under tools/. By convention, any target
libraries belong to lib/, while tools/ contains host-side tools.
2017-12-08 19:26:15 +02:00
Damien George 7cf446f3da tools/gen-cpydiff.py: Update executable paths to point to new ports dir. 2017-11-28 10:50:32 +11:00
Damien George 9ba3de6ea1 tools/mpy-tool.py: Implement freezing of Ellipsis const object. 2017-11-15 12:46:08 +11:00
Damien George 933eab46fc py/bc: Update opcode_format_table to match the bytecode. 2017-10-10 10:37:38 +11:00
Paul Sokolovsky c15be989ee tools/pyboard: Update docstring for additional device support. 2017-10-08 00:04:57 +03:00
Paul Sokolovsky ea6692a83e tools/pyboard: Use repr() when quoting data in error messages.
As it may contain newlines, etc.
2017-10-05 23:40:19 +03:00
Damien George ff93fd4f50 py/persistentcode: Bump .mpy version number to version 3.
The binary and unary ops have changed bytecode encoding.
2017-10-05 10:49:44 +11:00
Paul Sokolovsky 2f7827ba8f tools/upip: Upgrade to 1.2.2.
TLS SNI support, fixes after making str.rstrip() behavior compliant.
2017-09-29 18:24:11 -07:00
Damien George 4a93801c12 all: Update Makefiles and others to build with new ports/ dir layout.
Also renames "stmhal" to "stm32" in documentation and everywhere else.
2017-09-06 14:09:13 +10:00
stijn e4ab404780 tools/mpy-tool.py: Fix missing argument in dump() function
This makes the -d commandline argument usable again.
Pass empty string as parent name as listing starts from the root.
2017-08-16 10:38:19 +02:00
Damien George b6a3289564 tools/mpy-tool.py: Don't generate const_table if it's empty. 2017-08-12 22:26:18 +10:00
Alexander Steffen 55f33240f3 all: Use the name MicroPython consistently in comments
There were several different spellings of MicroPython present in comments,
when there should be only one.
2017-07-31 18:35:40 +10:00
Paul Sokolovsky a3cd349eaf tools/mpy_bin2res: Tools to convert binary resources to Python module.
Afterwards, they can be access using pkg_resource module from
micropython-lib.
2017-07-27 14:41:27 +03:00
Paul Sokolovsky 7901741bf1 tools/pyboard: Add license header. 2017-07-22 17:12:15 +03:00
Damien George 761e4c7ff6 all: Remove trailing spaces, per coding conventions. 2017-07-19 13:12:10 +10:00
Paul Sokolovsky ad5e7a0e6f tools/gen-cpydiff: Use case description as 3rd-level heading.
This is required to easily giving links to a particular difference case.
Also, add RST anchors to allow cases to cross-reference each other.
2017-07-09 13:51:40 +03:00
Ville Skyttä ca16c38210 various: Spelling fixes 2017-05-29 11:36:05 +03:00
Damien George 88c51c3592 tools/mpy-tool.py: Fix regression with freezing floats in obj repr C.
Regression was introduced by ec534609f6
2017-05-16 18:53:02 +10:00
Paul Sokolovsky 55491031be tools/mpy_cross_all.py: Helper tool to run mpy-cross on the entire project. 2017-05-14 17:51:12 +03:00
Damien George ec534609f6 tools/mpy-tool.py: Use MP_ROM_xxx macros to support nanbox builds. 2017-05-13 10:08:13 +10:00
Paul Sokolovsky d4c070415a tools/upip: Upgrade to 1.2.
Memory optimizations and error handling improvements.
2017-05-05 13:12:19 +03:00
Paul Sokolovsky e81f46940e tools/upip: Upgrade to 1.1.6, supports commented lines in requirements.txt. 2017-05-01 00:03:45 +03:00
Paul Sokolovsky 473e85e2da tools/mpy-tool: Make work if run from another directory.
By making sure we don't add relative paths to sys.path.
2017-05-01 00:01:30 +03:00
Damien George dd11af209d py: Add LOAD_SUPER_METHOD bytecode to allow heap-free super meth calls.
This patch allows the following code to run without allocating on the heap:

    super().foo(...)

Before this patch such a call would allocate a super object on the heap and
then load the foo method and call it right away.  The super object is only
needed to perform the lookup of the method and not needed after that.  This
patch makes an optimisation to allocate the super object on the C stack and
discard it right after use.

Changes in code size due to this patch are:

   bare-arm: +128
    minimal: +232
   unix x64: +416
unix nanbox: +364
     stmhal: +184
    esp8266: +340
     cc3200: +128
2017-04-22 23:39:20 +10:00
Paul Sokolovsky 3e1310d6e2 tools/pyboard: Provide more details when expected reply not received.
When trying to execute a command via raw REPL and expected "OK" reply
not received, show what was received instead.
2017-04-07 01:04:47 +03:00
Paul Sokolovsky 2cbe997834 tools/pyboard: ProcessPtyToTerminal: Add workaround for PySerial bug.
When working with a "virtual" port, like PTY. The issue described in
http://stackoverflow.com/questions/34831131/pyserial-does-not-play-well-with-virtual-port
2017-04-05 12:30:39 +03:00
Paul Sokolovsky 546ef301a1 tools/pyboard: execpty: Use shell=False to workaround some curdir issues.
Without this, Zephyr's port "make test" doesn't work.
2017-04-05 00:46:12 +03:00
Paul Sokolovsky 647e72ca63 tools/pyboard: Add "exec" and "execpty" pseudo-devices support.
This allows to execute a command and communicate with its stdin/stdout
via pipes ("exec") or with command-created pseudo-terminal ("execpty"),
to emulate serial access. Immediate usecase is controlling a QEMU process
which emulates board's serial via normal console, but it could be used
e.g. with helper binaries to access real board over other hadware
protocols, etc.

An example of device specification for these cases is:

	--device exec:../zephyr/qemu.sh
	--device execpty:../zephyr/qemu2.sh

Where qemu.sh contains long-long qemu startup line, or calls another
command. There's a special support in this patch for running the command
in a new terminal session, to support shell wrappers like that (without
new terminal session, only wrapper script would be terminated, but its
child processes would continue to run).
2017-04-04 17:46:02 +03:00
Paul Sokolovsky 9b3f423c14 tools/pyboard: Tighten up Pyboard object closure on errors.
Some "device" implementations may be sensitive to this.
2017-04-02 20:48:53 +03:00
Paul Sokolovsky 320099aab9 tools/tinytest-codegen: Update for recent test renaming ("intbig" suffix). 2017-03-06 22:40:04 +01:00
Damien George e4be56a0ea qemu-arm: Enable machine module and associated tests. 2017-03-02 16:39:58 +11:00
Krzysztof Blazewicz 23ccb3e12e tools/gen-cpydiff.py: configurable CPython and micropython executables 2017-02-27 15:39:55 +11:00
Damien George 1034d9acc8 tools/gen-cpydiff.py: Set the Python import path to find test modules. 2017-02-22 15:50:58 +11:00
Rami Ali b7fa63c7ce tools: Add gen-cpydiff.py to generate docs differences.
This patch introduces the a small framework to track differences between
uPy and CPython.  The framework consists of:

- A set of "tests" which test for an individual feature that differs between
  uPy and CPy.  Each test is like a normal uPy test in the test suite, but
  has a special comment at the start with some meta-data: a category (eg
  syntax, core language), a human-readable description of the difference, a
  cause, and a workaround.  Following the meta-data there is a short code
  snippet which demonstrates the difference.  See tests/cpydiff directory
  for the initial set of tests.

- A program (this patch) which runs all the tests (on uPy and CPy) and
  generates nicely-formated .rst documenting the differences.

- Integration into the docs build so that everything is automatic, and the
  differences appear in a way that is easy for users to read/reference (see
  latter commits).

The idea with using this new framework is:

- When a new difference is found it's easy to write a short test for it,
  along with a description, and add it to the existing ones.  It's also easy
  for contributors to submit tests for differences they find.

- When something is no longer different the tool will give an error and
  difference can be removed (or promoted to a proper feature test).
2017-02-20 17:14:34 +11:00
Damien George 6a11048af1 py/persistentcode: Bump .mpy version due to change in bytecode. 2017-02-17 00:19:34 +11:00
Paul Sokolovsky aac2db9aaf tools/upip: Update to 1.1.5. Better and more user-friendly error handling. 2017-02-10 20:18:05 +03:00
Damien George f1db8a3097 qemu-arm: Don't compile tests in "REPL" mode.
Previous to this patch the qemu-arm tests were compiled with is_relp=true
meaning that the __repl_print__ function was called for all lines of code
in the outer scope.  This is not the right behaviour for scripts that are
executed as though they were a file (eg tests).

With this fix the micropython/heapalloc_str.py test now works so it is
removed from the test blacklist.
2017-01-27 12:35:46 +11:00
Paul Sokolovsky 4614403f63 tools/tinytest-codegen.py: Blacklist heapalloc_str.py test for qemu-arm. 2017-01-27 01:19:36 +03:00
Damien George 98458a46ec tools/mpy-tool.py: Add support for OPT_CACHE_MAP_LOOKUP_IN_BYTECODE.
With caching of map lookups in the bytecode, frozen bytecode can still
work but must be stored in RAM, not ROM.  This patch allows mpy-tool.py to
generate code that works with this optimisation, but it's not recommended
to use it on embedded targets (because of lack of RAM).
2017-01-05 15:52:52 +11:00
Damien George 29b5879613 tools/pyboard.py: Refactor so target is not reset between scripts/cmd.
Previous to this patch pyboard.py would open a new serial connection to
the target for each script that was run, and for any command that was run.
Apart from being inefficient, this meant that the board was soft-reset
between scripts/commands, which precludes scripts from accessing variables
set in a previous one.

This patch changes the behaviour of pyboard.py so that the connection to
the target is created only once, and it's not reset between scripts or any
command that is sent with the -c option.
2016-12-15 11:29:33 +11:00
Paul Sokolovsky 30cfdc29ed tools/tinytest-codegen: Blacklist recently added uheapq_timeq test (qemu-arm). 2016-11-12 03:24:36 +03:00
Paul Sokolovsky 61d74fdef8 tools, unix: Replace upip tarball with just source files.
To make its inclusion as frozen modules in multiple ports less magic.
Ports are just expected to symlink 2 files into their scripts/modules
subdirs.

Unix port updated to use this and in general follow frozen modules setup
tested and tried on baremetal ports, where there's "scripts" predefined
dir (overridable with FROZEN_DIR make var), and a user just drops Python
files there.
2016-11-07 18:39:41 +03:00
Damien George ca973bd308 qemu-arm: Enable software floating point support, and float tests.
This helps to test floating point code on Cortex-M hardware.

As part of this patch the link-time-optimisation was disabled because it
wasn't compatible with software FP support.  In particular, the linker
could not find the __aeabi_f2d, __aeabi_d2f etc functions even though they
were provided by lib/libm/math.c.
2016-11-03 12:28:31 +11:00
Paul Sokolovsky 796b720dbc tools/tinytest-codegen: Exclude ticks_diff test for qemu-arm port. 2016-10-30 22:24:07 +03:00
Alex March 52aa532050 qemu-arm: Exclude new vfs_fat tests. 2016-10-27 12:25:28 +11:00
Paul Sokolovsky f7aa692093 tools/check_code_size.sh: Code size validation script for CI. 2016-10-25 11:43:55 +03:00
Paul Sokolovsky b78144c64d tools/pip-micropython: Remove deprecated wrapper tool.
Deprecated for long time, pip-micropython now can't install packages
optimized for low-heap ports (like whole of micropython-lib).
2016-10-24 16:52:15 +03:00
Alex March 06e7032906 qemu-arm: Exclude extmod/vfs_fat_fileio.py test. 2016-10-24 12:49:19 +11:00
Paul Sokolovsky 197a5724d8 tools: Upgrade upip to 1.1.4.
Fix error on unix when installing to non-existing absolute path.
2016-10-22 21:14:58 +03:00
Damien George b0a15aa735 qemu-arm: Enable lots of extmods and enable tests for them.
The qemu-arm port is used for testing of ARM Thumb architecture on a
desktop so should have many features enabled.
2016-10-14 00:08:19 +11:00
Paul Sokolovsky cdbeee0c50 tools: Upgrade upip to 1.1.3.
Initial support for running on a baremetal, low-heap systems (like esp8266),
using Python module interface.
2016-10-12 18:55:31 +03:00
Paul Sokolovsky cecf6bee97 tools: Upgrade upip to 1.0.
Fully self-hosted release (without fallbacks), and uses stream gzip
decompression (step towards support for limited-heap baremetal systems).
2016-10-05 00:34:09 +03:00
Paul Sokolovsky 4c63986101 tools: Update upip to 0.8. Fixes IPv6 support. 2016-09-23 15:35:06 +03:00
Damien George 7df9291b6c py: Update opcode format table because 3 opcodes were removed, 1 added.
LIST_APPEND, MAP_ADD and SET_ADD have been removed, and STORE_COMP has
been added in adaf0d865c.
2016-09-23 12:48:57 +10:00
Damien George c51c883cc8 tools/mpy-tool.py: Support freezing of complex numbers. 2016-09-03 00:19:02 +10:00
Damien George b6bdf18deb tools/mpy-tool.py: Compute the hash value for str/bytes objects.
This makes it more efficient at runtime to hash str/bytes objects.
2016-09-02 15:10:45 +10:00
Damien George b4790afdaf tools/mpy-tool.py: Store qstr config values in global config object.
Makes it easier to access them without passing around another dict of the
config values.
2016-09-02 15:09:21 +10:00
Damien George 72ae3c72c7 tools/mpy-tool.py: Support freezing float literals with obj-repr C.
The tool now generates code for freezing floats in obj-repr A, B or C,
with the specific representation detected at compile time using macros.
2016-08-10 13:26:11 +10:00
Paul Sokolovsky cbe544a76d tools: Upgrade upip to 0.7.
SSL certificate warning, switch to uerrno, and better usage message.
2016-05-28 02:57:53 +03:00
Damien George 25a42fb6ef tools/mpy-tool.py: Don't strip directories from the frozen source name.
Directories are now supported by the frozen import system (to implement
frozen packages) so we should keep them.
2016-05-23 13:29:03 +01:00
Damien George 9b4c013823 tools/mpy-tool.py: Include .py extension in frozen filename.
So that it can be correctly stat'd when looking for frozen files.
2016-05-23 12:46:02 +01:00
Paul Sokolovsky daa4793578 tools/make-frozen: Update for latest changes in frozen modules support.
Frozen modules are now stored with extensions and with '/' as path
separator. In other words, frozen modules paths stored as they are
in normal filesystem.
2016-05-21 21:39:27 +03:00
Damien George 99b4719357 tools/mpy-tool.py: Add checks for critical configuration vars.
When an mpy file is frozen it must know the values of certain
configuration variables.  This patch provides an explicit check in the
generated C file that the configuration variables are what they are
supposed to be.
2016-05-16 23:13:30 +01:00
Paul Sokolovsky 7b1bf0c308 tools/make-frozen.py: Quick fix to support package-modules.
It allows to "import foo.bar", but not "from foo import bar".
2016-05-14 16:30:02 +03:00
Damien George 5985e41afc tools/make-frozen.py: Properly escape hex chars when making C strings. 2016-05-13 13:12:01 +01:00
Damien George 02fd83bcbc tools/mpy-tool: Make sure that all C-level variables are unique.
Fixes issue #2023.
2016-05-03 12:24:39 +01:00
Damien George c3beb16db3 tools/mpy-tool.py: Add support for Python 2.7. 2016-04-15 11:56:10 +01:00
Damien George 0a2e9650f5 py: Add ability to have frozen persistent bytecode from .mpy files.
The config variable MICROPY_MODULE_FROZEN is now made of two separate
parts: MICROPY_MODULE_FROZEN_STR and MICROPY_MODULE_FROZEN_MPY.  This
allows to have none, either or both of frozen strings and frozen mpy
files (aka frozen bytecode).
2016-04-13 16:07:47 +01:00
Damien George 0699c6bf9e tools: Add mpy-tool.py, to work with .mpy files.
Currently it can freeze .mpy files.
2016-04-13 16:05:43 +01:00
Damien George be989be861 qemu-arm: Enable builtin override feature, and enable more tests.
Hopefully these tests run reliably on Travis.
2016-03-15 13:45:32 +00:00
Paul Sokolovsky 17f324b836 py/frozenmod: Store frozen module names together, to quickly scan them. 2016-01-03 18:08:45 +02:00
Paul Sokolovsky 0992588811 py/frozenmod: Make frozen module content be 0-terminated.
To allow simple zero-terminated lexers.
2016-01-01 07:51:07 +02:00
Damien George 4bd95f8b44 tools: Add C middle-processor to make builtin tables proper hash tables. 2015-12-31 00:24:33 +00:00
Paul Sokolovsky bb7f5b5501 tools: Upgrade upip to 0.6.3.
Updated for _os -> uos builtin module rename.
2015-12-12 00:20:10 +02:00
Damien George bbe2e22fcb tools: Fix pyboard.py to work under Python 3. 2015-12-08 12:55:28 +00:00
Peter Hinch f4fcc14cfb tools: Add option to pyboard.py to wait for serial device to be ready.
Also prints a nicer error message if the serial connection could not be
established.
2015-12-08 11:10:09 +00:00
Dave Hylands 519cef813e tools: Allow pyboard.py to work when boot.py prints things. 2015-12-06 21:45:10 +00:00
Paul Sokolovsky f8bc3f6964 tools: Update to upip 0.6.2. Fixes issue due to MacOSX undocumented behavior. 2015-11-15 00:04:18 +02:00
Paul Sokolovsky 3862ef9a9f tools: Update upip to 0.6.1. Fixes normal installs without -p switch. 2015-11-14 17:54:30 +02:00
Paul Sokolovsky 0aff87b808 tools: Update upip to 0.6.
Removes FFI dependency, instead uses builtin os module. Thus can work on
systems where dynamic library loading is not available.
2015-11-14 00:05:24 +02:00
Paul Sokolovsky 89f2ddd2d0 tools/pyboard.py: Don't add terminating \x04 character to stdout output. 2015-11-07 13:48:35 +00:00
Paul Sokolovsky e6dccaf18e tools/pyboard.py: Make -c (inline Python code) option compatible with python2. 2015-11-07 12:16:11 +03:00
Tom Soulanille a787467569 tools/pyboard: Add -c argument to run a program passed as a string. 2015-10-19 23:20:59 +01:00
Paul Sokolovsky 6a515b95a8 tools/upip: Update to 0.5.9.
MICROPYPATH environment variable is now honored, package are installed to
first path specified in it.
2015-10-17 20:16:41 +03:00
Damien George 0334058fa4 Rename "Micro Python" to "MicroPython" in REPL, help, readme's and misc. 2015-10-12 00:06:25 +01:00
Dave Hylands 01d64914c5 stmhal: Fix USB CDC-only mode under Windows.
This fix adds PIDs 9801 and 9802 to the pybcdc.inf file.

When in CDC only mode, it presents itself as a Communcations
device rather than as a composite device. Presenting as a
composite device with only the CDC interface seems to confuse
windows.

To test and make sure that the correct pybcdc.inf was being used,
I used USBDeview from http://www.nirsoft.net/utils/usb_devices_view.html
to uninstall any old pyboard drivers (Use Control-F and search
for pyboard). I found running USBDeview as administrator worked best.

Installing the driver in CDC+MSC mode first is recommended (since the
pybcdc.inf file in on the internal flash drive). Then when you switch
modes everything seems to work properly.

I used https://github.com/dhylands/upy-examples/blob/master/boot_switch.py
to easily switch the pyboard between the various USB modes for testing.
2015-10-09 00:18:01 +01:00
Paul Sokolovsky a5deadf082 tools: Upgrade upip to 0.5.8.
Adds support for ussl module (which is experimental and not fully
implemented yet itself).
2015-09-13 09:49:09 +03:00
Paul Sokolovsky e8ad47a6ca tools: Upgrade to upip 0.5.7.
Just dependent micropython-lib modules update for upip, no new
functionality.
2015-08-30 11:04:38 +03:00
Damien George b7d59060e2 tools: Make gen-changelog.sh print more lines from the tag annotation. 2015-08-11 13:50:40 +01:00
Damien George e86b47175f tools: Make pyboard.py Python2 compatible. 2015-08-08 13:03:08 +01:00
Damien George 7ccdf8be77 tools/pyboard.py: Fix read timeout calc to work with shorter sleep. 2015-07-29 00:21:22 +01:00
Daniel Campora cd14188bc8 tools: Add telnet support to pyboard.py.
The adapter class "TelnetToSerial" is used to access the Telnet
connection using the same API as with the serial connection. The
function pyboard.run-test() has been removed to made the module
generic and because this small test is no longer needed.
2015-07-26 14:02:34 +01:00
Damien George db109ca0fc tools/pyboard.py: Speed up reading of chars by decreasing sleep period. 2015-07-25 22:49:25 +01:00
Damien George f1236734bb tools/pyboard.py: Make enter_raw_repl stricter and more reliable.
When looking for chars to indicate raw repl is active, look for the full
string of chars to improve reliability of entering raw repl correctly.

Previous to this patch there was the possibility that raw repl was
entered in a dirty state, where not all input chars from previous
invocation were drained.
2015-07-25 22:46:07 +01:00
Damien George 9de53bf788 tools/pyboard.py: Fix parsing of returned error so last chr is not lost. 2015-07-25 22:44:56 +01:00
Dave Hylands c91727b75a tools: Fix pydfu.py to work with old and new versions of PyUSB
Update pydfu.py to match with the version from openmv.

I just updated the openmv version to work with both of the
PyUSB 1.0.0.b1 and 1.0.0.b2

See: https://github.com/walac/pyusb/blob/master/ReleaseNotes.rst
2015-07-06 09:56:12 -07:00
Paul Sokolovsky 6ab8b63bdd tools/make-frozen.py: Use Python2-compatible shebang. 2015-07-06 14:48:29 +03:00
Paul Sokolovsky de575c80b9 tools/make-frozen.py: Actually make Python2-compatible. 2015-07-06 14:27:57 +03:00
Paul Sokolovsky 3a2e9f20f6 tools/make-frozen.py: Add Python2 compatibility. 2015-07-06 14:09:16 +03:00
Paul Sokolovsky 484adac0bb tools: Update upip to 0.5.4.
Recognize and handle "package not found" error.
2015-06-25 15:32:14 +03:00
Damien George d59ca4efdb tools/pyboard.py: Change logic for when raw ">" prompt is parsed.
In raw REPL ">" indicates the prompt.  We originally read this character
upon entering the raw REPL, and after reading the last bit of the
output.  This patch changes the logic so the ">" is read only just
before trying to send the next command.  To make this work (and as an
added feature) the input buffer is now flushed upon entering raw REPL.

The main reason for this change is so that pyboard.py recognises the EOF
when sys.exit() is called on the pyboard.  Ie, if you run pyboard.py
with a script that calls sys.exit(), then pyboard.py will exit after
the sys.exit() is called.
2015-06-15 23:28:04 +01:00
Paul Sokolovsky dd0e24f4b0 pip-micropython: Add deprecation notice, but still leave for reference. 2015-06-13 21:39:57 +03:00
Paul Sokolovsky 098f5ae221 micropython-upip: Mark as binary file and re-commit. 2015-06-06 23:09:23 +03:00
Paul Sokolovsky 1406d9ccde upip: Upgrade to 0.5.3 to follow uctypes.struct() signature change. 2015-06-06 23:03:28 +03:00
Paul Sokolovsky f8a39e3bb1 tools/make-frozen.py: Handle trailing slash in argument more reliably. 2015-06-03 19:28:31 +03:00
Damien George 0e6c89a5c8 tools: Add codestats.sh to compute code statistics such as size, speed. 2015-06-03 15:38:43 +00:00
Paul Sokolovsky 0df20da4dd tools: Add upip 0.5.2 tarball.
So unix version, which now includes upip as a frozen module, can be built
without Internet connection.
2015-06-02 01:32:07 +03:00
Paul Sokolovsky c02dc8b2c9 tools/make-frozen.py: Open files in binary mode. 2015-05-30 00:39:00 +03:00
Paul Sokolovsky a8e7c03171 tools/make-frozen.py: Preserve directory hierarchy.
Currently, frozen packages are not supported, but eventually they should be,
so make sure to store complete directory hierarchy.
2015-05-30 00:23:46 +03:00
Paul Sokolovsky f5ae384d4f bootstrap_upip.sh: Allow to install into a path given by argument. 2015-05-29 22:35:45 +03:00
Dave Hylands 669dbca959 tools: Allow pyboard constructor to take a baudrate parameter.
This allows pyboard.py to be used over a UART interface
rather than just over a USB serial interface.
2015-05-27 23:01:28 +01:00
Dave Hylands 97ce883217 tools: Add exec_raw_no_follow to pyboard.py. 2015-05-18 11:39:57 +01:00
Paul Sokolovsky 76abb2e623 tools: Add script to install "upip" package manager.
upip is a simple and light-weight package manager for MicroPython modules,
offering subset of pip functionality. upip is part of micropython-lib
project: https://github.com/micropython/micropython-lib/tree/master/upip
This script bootstraps upip by downloading and unpacking it directly from
PyPI repository, with all other packages to be installed with upip itself.
2015-05-07 16:00:40 +03:00
Damien George 4865a22f78 tools/pyboard.py: Add "--follow" option to wait for output indefinitely.
Also flush stdout so you can see output as it comes.
2015-05-07 10:14:51 +01:00
Damien George 23a2b11abf tools: Add STM32F4DISC and ESPRUINO_PICO to stmhal build script. 2015-04-21 13:16:41 +00:00
Damien George 2bb5f41611 tools/pyboard.py: Make it 8-bit clean, so it works with unicode chars.
Addresses issue #1190.
2015-04-19 21:31:28 +01:00
Damien George d8475092d1 qemu-arm: Disable inline asm tests that use floating point. 2015-04-19 15:50:24 +01:00
Damien George 39ce2db181 stmhal: Add "CDC" option to pyb.usb_mode, for CDC device only. 2015-02-13 14:02:51 +00:00
Damien George fb2006cc69 tools: Make gen-changelog.sh sort version strings correctly. 2015-02-13 13:31:02 +00:00
Damien George 2f8c409a4f qemu-arm: Disable try_finally1.py test (it fails randomly on Travis). 2015-01-25 18:48:03 +00:00
Paul Sokolovsky 640e0b221e py: Implement very simple frozen modules support.
Only modules (not packages) supported now. Source modules can be converted
to frozen module structures using tools/make-frozen.py script.
2015-01-20 11:52:12 +02:00
Damien George b58da9420c qemu-arm: Disable basics/memoryerror.py test. 2015-01-12 16:32:14 +00:00
Damien George 99dde4ed1f qemu-arm: Enable GC and native code-gen; enable more tests. 2015-01-12 12:07:42 +00:00
Damien George 85e8e2ed5b qemu-arm: Add 'test' target to Makefile to run and verify test suite.
Replaces RUN_TEST=1 definition; now "make test" in qemu-arm directory
will run tests/basics/ and check that they all succeed.

This patch also enables the test on Travis CI.
2015-01-09 00:03:21 +00:00
Damien George e3fa8278b4 tools: Add script to generate a ChangeLog file.
We don't have an explicit ChangeLog file, but don't really need one
because we use a good version control system.  This script is useful if
you need a pretty-printed ChangeLog for some reason.
2014-12-29 19:03:25 +00:00
Damien George 4fd7c1a2ac tools, pyboard.py: Write data to pyboard in chunks of 256 bytes.
This speeds up writes significantly.
2014-12-20 18:09:04 +00:00
Damien George 17c5ce3727 tools: Make pyboard.py have infinite timeout when running script.
This makes pyboard.py much more useful for long running scripts.  When
running a script via pyboard.py, it now waits until the script finishes,
with no timeout.  CTRL-C can be used to break out of the waiting if
needed.
2014-11-30 21:30:53 +00:00
Damien George 7c4445afe1 tools: Make gendoc.py able to output RST format. 2014-10-31 01:36:11 +00:00
Damien George bc1488a05f stmhal: Improve REPL control codes; improve pyboard.py script.
Improvements are:

2 ctrl-C's are now needed to truly kill running script on pyboard, so
make CDC interface allow multiple ctrl-C's through at once (ie sending
b'\x03\x03' to pyboard now counts as 2 ctrl-C's).

ctrl-C in friendly-repl can now stop multi-line input.

In raw-repl mode, use ctrl-D to indicate end of running script, and also
end of any error message.  Thus, output of raw-repl is always at least 2
ctrl-D's and it's much easier to parse.

pyboard.py is now a bit faster, handles exceptions from pyboard better
(prints them and exits with exit code 1), prints out the pyboard output
while the script is running (instead of waiting till the end), and
allows to follow the output of a previous script when run with no
arguments.
2014-10-26 15:39:22 +00:00
Damien George 5fc42a6c97 tools, pydfu: Some fixes to support Python 3. 2014-10-22 20:27:43 +01:00
Dave Hylands 842210f53a Add pydfu.py to the micropython tree. Use dfu_util bgy default
You can do:

make USE_PYDFU=1 deploy

to use pydfu.py
2014-10-22 20:18:38 +01:00
Damien George 9c9db3a7a1 tools, pyboard.py: Allow exec argument to be bytes or str. 2014-10-19 14:54:52 +01:00
blmorris 3b064370f8 Enable device keyword option when running pyboard.py --tests and run-tests --pyboard 2014-10-01 23:31:52 +01:00
Damien George c737086e1c tools: Add network build to auto-build script. 2014-09-30 23:04:08 +00:00
Damien George 2842945e76 stmhal: Fix bugs in documentation so it compiles. 2014-09-17 23:27:42 +00:00
Paul Sokolovsky f3c3010ffc pip-micropython: Revert to using PIP_MICROPY_DEST environment var.
-t/--target is a pip option. Trying to use pip options for different meanings
in pip-micropython may lead to big confusion. That's why the original passed
any extra parameters using environment variables. "All options belong to pip."
2014-08-27 02:53:06 +03:00
Damien George db63660c19 Add pip-micropython to unix make install.
Also add -t/--target option to pip-micropython to allowing installing to
the pyboard.

Thanks to turbinenreiter/Sebastian Plamauer for the patch.
2014-08-26 16:03:57 +01:00
Damien George c4ee39dd63 tools, gendoc: Output small descr about module TOC. 2014-08-10 16:51:26 +01:00
Damien George 590b2abdfc tools, gendoc: Put module TOC/index in module/ directory. 2014-08-08 23:34:39 +01:00
Damien George ef7a066c9c tools, gendoc: Allow constants at module level; gen module index.
Addresses some issues from #585.
2014-08-08 23:29:05 +01:00
Paul Sokolovsky c10a4405cd gendoc.py: Support modules w/o functions and/or classes.
I.e. don't assume that both are always present.
2014-06-29 15:48:30 +03:00
Damien George 3793830ed9 tools: Move gendoc.py to tools, and make it a little more generic. 2014-05-10 19:12:47 +01:00
Ilya Dmitrichenko be86596bb9 tools: inline test suite generator. 2014-05-08 01:41:22 +03:00
Ilya Dmitrichenko b1442e04d1 tools: check-in errordeveloper/tinytest@eb2dbc858f 2014-05-08 01:41:21 +03:00
Damien George 37936bebbf tools: In build-stm-latest, replace git hash with git tag. 2014-05-04 00:11:09 +00:00
Damien George 3244123031 tools: pyboard.py now acts as a command-line program to run scripts.
You can run a local script on the pyboard using:

    python pyboard.py test.py

where test.py is the local script you want to run.
2014-05-03 18:14:34 +01:00
Damien George aad1204b8e stmhal: Change names: flash to firmware, flashboard to deploy.
Since firmware lives in a build directory which already specifies the
board name, no real reason to also have the firmware have the board
name.
2014-05-03 17:46:45 +01:00
Damien George e88814a274 Merge pull request #543 from lurch/patch-4
Make pyboard.enter_raw_repl more robust
2014-05-03 13:09:16 +01:00
Damien George d9f4deef8c Merge pull request #547 from cjbarnes18/pyboard_module_fixes
Fix pyboard.py tests.
2014-05-03 00:11:43 +01:00
Craig Barnes 71a1e35206 Fix pyboard.py tests. 2014-05-02 21:36:07 +01:00
Damien George ba742ed639 Add shell script to automatically build stmhal/ for all pyboards. 2014-05-02 13:49:44 +00:00
Andrew Scheller 1f740bdaf4 Make pyboard.enter_raw_repl more robust
In case there's a program in the microcontroller's main.py running in an infinite loop
2014-05-01 20:42:23 +01:00
Damien George ca045d97b9 tools: Update pyboard.py to work with latest changes to RTC. 2014-04-21 20:18:37 +01:00
Damien George ecdf6cdc5b tools: Fix up pybcdc.inf generation: new lines and hex digits.
Using Python's file open in 'r' mode opens it for text reading, which
converts all new lines to \n.  Could use 'rb' binary mode, but then
don't have access to the string Template replacement functions.  Thus,
force the output to have '\\r\\n' ending.

Also fix regex to match hex digits.
2014-04-16 11:45:52 +01:00
Andrew Scheller 1452221aca Auto-generate the stmhal/pybcdc_inf header file from static files
The USB VID&PID are automatically extracted from usbd_desc_cdc_msc.c
and inserted into pybcdc_inf.template, ensuring that the same USB
IDs get used everywhere
2014-04-16 02:41:26 +01:00
Paul Sokolovsky cc8c0882a0 pip-micropython: Fix inverted condition. 2014-04-15 02:32:41 +03:00
Andrew Scheller 6fc40cc3da pip-micropython: revert $HOME back to ~ 2014-04-14 22:26:34 +01:00
Damien George a7239b2fc2 Merge pull request #480 from lurch/patch-2
Make pip-micropython more portable
2014-04-14 21:39:31 +01:00
Andrew Scheller 0aac7ba344 Make pip-micropython more portable
Replace /tmp with a $TMPDIR variable
Replace multiple occurrences of /tmp/pip-micropy-venv with a $TMPVENV variable
Replace ~ with $HOME
2014-04-14 12:26:12 +01:00
Damien George 48a9b3fd11 tools: Improve timout/reading of pyboard.py. 2014-04-14 01:47:36 +01:00
Damien George b636d024d2 Make pyboard.py have its own exception; update run-tests for pyboard. 2014-04-13 13:48:33 +01:00
Paul Sokolovsky 06526e7ccc pip-micropython: Apply more workarounds for setuptools, pip, etc misfeatures. 2014-04-12 23:33:14 +03:00
Paul Sokolovsky 9f043da5c3 pip-micropython: Require command verb, but restrict to "install" only.
So, pip-micropython can/should be run as normal pip:

    pip-micropython install micropython-unittest
2014-04-07 02:53:41 +03:00
Damien George d71fc37af7 tools: Improve reading back from pyboard. 2014-04-03 22:44:37 +01:00
Damien George 0e49642965 tools: pyboard.py can now execute a file remotely! 2014-04-03 22:12:01 +01:00
Paul Sokolovsky 802d505756 tools: Add pip-micropython to install packages in MicroPython-friendly way. 2014-04-02 12:49:07 +03:00
Damien George 41ea651839 Add tools/pyboard.py, a simple module to communicate with the pyboard.
Using pyboard.py you can use Python running on your PC to execute
commands on the connected pyboard.  See examples in that file.
2014-03-24 12:42:06 +00:00
mux 1a1ba4d555 Change dfu.py to be Python 2/3 compatible
* Chane dfu.py to use Python 3 syntax to avoid dependency on Python 2.
* Update Makefile to call python instead of python2
* Fix #33
2014-01-03 08:51:02 +02:00
Damien f0d0940638 Add tools/dfu.py, and use it in stm/Makefile. 2013-12-20 12:07:50 +00:00