This patch adds a function utf8_check() to check for a valid UTF-8 encoded
string, and calls it when constructing a str from raw bytes. The feature
is selectable at compile time via MICROPY_PY_BUILTINS_STR_UNICODE_CHECK and
is enabled if unicode is enabled. It costs about 110 bytes on Thumb-2, 150
bytes on Xtensa and 170 bytes on x86-64.
IEEE floating point is specified such that a comparison of NaN with itself
returns false, and Python respects these semantics. This patch makes uPy
also have these semantics. The fix has a minor impact on the speed of the
object-equality fast-path, but that seems to be unavoidable and it's much
more important to have correct behaviour (especially in this case where
the wrong answer for nan==nan is silently returned).
These are now returned as "operation not supported" instead of raising
TypeError. In particular, this fixes equality for float vs incompatible
types, which now properly results in False instead of exception. This
also paves the road to support reverse operation (e.g. __radd__) with
float objects.
This is achieved by introducing mp_obj_get_float_maybe(), similar to
existing mp_obj_get_int_maybe().
Prior to this patch, the size of the buffer given to pack_into() was checked
for being too small by using the count of the arguments, not their actual
size. For example, a format spec of '4I' would only check that there was 4
bytes available, not 16; and 'I' would check for 1 byte, not 4.
The pack() function is ok because its buffer is created to be exactly the
correct size.
The fix in this patch calculates the total size of the format spec at the
start of pack_into() and verifies that the buffer is large enough. This
adds some computational overhead, to iterate through the whole format spec.
The alternative is to check during the packing, but that requires extra
code to handle alignment, and the check is anyway not needed for pack().
So to maintain minimal code size the check is done using struct_calcsize.
Prior to this patch, the size of the buffer given to unpack/unpack_from was
checked for being too small by using the count of the arguments, not their
actual size. For example, a format spec of '4I' would only check that
there was 4 bytes available, not 16; and 'I' would check for 1 byte, not 4.
This bug is fixed in this patch by calculating the total size of the format
spec at the start of the unpacking function. This function anyway needs to
calculate the number of items at the start, so calculating the total size
can be done at the same time.
This patch makes a repeat counter behave the same as repeating the
typecode, when there are not enough args. For example:
struct.pack('2I', 1) now behave the same as struct.pack('II', 1).
Similar to the existing testcase, but test that returning both value of
native type and instance of another user class from __new__ lead to
__init__ not being called, for better coverage.
NotImplemented means "try other fallbacks (like calling __rop__
instead of __op__) and if nothing works, raise TypeError". As
MicroPython doesn't implement any fallbacks, signal to raise
TypeError right away.
Otherwise, it will silently get incorrect result on other values types,
including CPython tuple form like "foo.png".endswith(("png", "jpg"))
(which MicroPython doesn't support for unbloatedness).
* atmel-samd: Introduce a nvm module for non-volatile byte-level memory access.
This allows for persisting small configuration values even when the file system
is read-only from CircuitPython.
Fixes#160
* Review feedback:
* Add tests.
* Fix non-zero index.
* Fix len()
This implementation ignores invalid characters in the input. This allows
it to decode the output of b2a_base64, and also mimics the behavior of
CPython.
The value of 0 can't be used because otherwise mp_binary_get_size will let
a null byte through as the type code (intepreted as byterray). This can
lead to invalid type-specifier strings being let through without an error
in the struct module, and even buffer overruns.
This introduces a skip_if module that can be used by tests to
determine when they should be skipped due to the environment.
Some tests have been split in order to have finer grained skip
control.
The idea is that these tests can be run with just a test server running
on a test host, with device under test connecting to it, instead of
requiring Internet connection for testing.
Such setup is however WIP, and some tests in net_hosted/ are so far
written to connect to Internet, as there're not test server written
yet. This is expected to evolve over time.
This attempts to bootstrap network tests for MicroPython. This commits
sets test/net_inet/ as place for tests which require access to wide
Internet. They aren't intended to be run as part of the main testsuite,
instead to be run manually on demand.
test_tls_sites.py in particular check that it's possible to establish
SSL/TLS connection to select sites on the Internet: few references ones,
plus those for which problems were reported, and resolved.
Fixes for stmhal USB mass storage, lwIP bindings and VFS regressions
This release provides an important fix for the USB mass storage device in
the stmhal port by implementing the SCSI SYNCHRONIZE_CACHE command, which
is now require by some Operating Systems. There are also fixes for the
lwIP bindings to improve non-blocking sockets and error codes. The VFS has
some regressions fixed including the ability to statvfs the root.
All changes are listed below.
py core:
- modbuiltins: add core-provided version of input() function
- objstr: catch case of negative "maxsplit" arg to str.rsplit()
- persistentcode: allow to compile with complex numbers disabled
- objstr: allow to compile with obj-repr D, and unicode disabled
- modsys: allow to compile with obj-repr D and PY_ATTRTUPLE disabled
- provide mp_decode_uint_skip() to help reduce stack usage
- makeqstrdefs.py: make script run correctly with Python 2.6
- objstringio: if created from immutable object, follow copy on write policy
extmod:
- modlwip: connect: for non-blocking mode, return EINPROGRESS
- modlwip: fix error codes for duplicate calls to connect()
- modlwip: accept: fix error code for non-blocking mode
- vfs: allow to statvfs the root directory
- vfs: allow "buffering" and "encoding" args to VFS's open()
- modframebuf: fix signed/unsigned comparison pendantic warning
lib:
- libm: use isfinite instead of finitef, for C99 compatibility
- utils/interrupt_char: remove support for KBD_EXCEPTION disabled
tests:
- basics/string_rsplit: add tests for negative "maxsplit" argument
- float: convert "sys.exit()" to "raise SystemExit"
- float/builtin_float_minmax: PEP8 fixes
- basics: convert "sys.exit()" to "raise SystemExit"
- convert remaining "sys.exit()" to "raise SystemExit"
unix port:
- convert to use core-provided version of built-in import()
- Makefile: replace references to make with $(MAKE)
windows port:
- convert to use core-provided version of built-in import()
qemu-arm port:
- Makefile: adjust object-file lists to get correct dependencies
- enable micropython.mem_*() functions to allow more tests
stmhal port:
- boards: enable DAC for NUCLEO_F767ZI board
- add support for NUCLEO_F446RE board
- pass USB handler as parameter to allow more than one USB handler
- usb: use local USB handler variable in Start-of-Frame handler
- usb: make state for USB device private to top-level USB driver
- usbdev: for MSC implement SCSI SYNCHRONIZE_CACHE command
- convert from using stmhal's input() to core provided version
cc3200 port:
- convert from using stmhal's input() to core provided version
teensy port:
- convert from using stmhal's input() to core provided version
esp8266 port:
- Makefile: replace references to make with $(MAKE)
- Makefile: add clean-modules target
- convert from using stmhal's input() to core provided version
zephyr port:
- modusocket: getaddrinfo: Fix mp_obj_len() usage
- define MICROPY_PY_SYS_PLATFORM (to "zephyr")
- machine_pin: use native Zephyr types for Zephyr API calls
docs:
- machine.Pin: remove out_value() method
- machine.Pin: add on() and off() methods
- esp8266: consistently replace Pin.high/low methods with .on/off
- esp8266/quickref: polish Pin.on()/off() examples
- network: move confusingly-named cc3200 Server class to its reference
- uos: deconditionalize, remove minor port-specific details
- uos: move cc3200 port legacy VFS mounting functions to its ref doc
- machine: sort machine classes in logical order, not alphabetically
- network: first step to describe standard network class interface
examples:
- embedding: use core-provided KeyboardInterrupt object
In CPython 3.4 this raises a SyntaxError. In CPython 3.5+ having a
positional after * is allowed but uPy has the wrong semantics and passes
the arguments in the incorrect order. To prevent incorrect use of a
function going unnoticed it is important to raise the SyntaxError in uPy,
until the behaviour is fixed to follow CPython 3.5+.
This patch fixes 2 things when printing a floating-point number that
requires rounding up of the mantissa:
- retain the correct precision; eg 0.99 becomes 1.0, not 1.00
- if the exponent goes from -1 to 0 then render it as +0, not -0
Tests for an issue with line continuation failing in paste mode due to the
lexer only checking for \n in the "following" character position, before
next_char() has had a chance to convert \r and \r\n to \n.
This patch refactors the handling of the special super() call within the
compiler. It removes the need for a global (to the compiler) state variable
which keeps track of whether the subject of an expression is super. The
handling of super() is now done entirely within one function, which makes
the compiler a bit cleaner and allows to easily add more optimisations to
super calls.
Changes to the code size are:
bare-arm: +12
minimal: +0
unix x64: +48
unix nanbox: -16
stmhal: +4
cc3200: +0
esp8266: -56
If we got a CRASH result, return early, similar to SKIP. This is important
because previous refactor changed branching logic a bit, so CRASH now gets
post-processed into CRASH\n, which broke remote hardware tests.
A shorter name takes less code size, less room in scripts and is faster to
type at the REPL.
Tests and HW-API examples are updated to reflect the change.
MONO_xxx is much easier to read if you're not familiar with the code.
MVLSB is deprecated but kept for backwards compatibility, for the time
being.
This patch also updates the associated docs and tests.
The 'S' typecode is a uPy extension so it should be grouped with the other
extension (namely 'O' typecode). Testing 'S' needs uctypes which is an
extmod module and not always available, so this test is made optional and
will only be run on ports that have (u)struct and uctypes. Otherwise it
will be silently skipped.
This is so that the filename of the test doesn't clash with the module name
itself (being "websocket"), and lead to potential problems executing the
test.
MICROPY_LONGINT_IMPL_LONGLONG doesn't have overflow detection, so just
parsing a large number won't give an error, we need to print it out
to check that the whole number was parsed.
These short unit tests test the base uPy methods as well as parts of the
websocket protocol, as implemented by uPy.
@dpgeorge converted the original socket based tests by @hosaka to ones
that only require io.BytesIO.
This test just tests that the basic functions/methods can be called with
the appropriate arguments. There is no real test of underlying
functionality.
Thanks to @hosaka for the initial implementation of this test.
I.e. they don't run successfully with MICROPY_LONGINT_IMPL_NONE
and MICROPY_LONGINT_IMPL_LONGLONG (the problem is that they generate
different output than CPython, TODO to fix that).
The use of large literal numbers is a big no-no when it comes to writing
programs which work with different int representations. Also, some checks
are pretty adhoc (e.g using struct module to check for 64-bitness). This
change bases entire detection on sys.maxsize and integer operarions, and
thus more correct, even if longer.
Note that this change doesn't mean that any of these tests can pass with
anything but MPZ - even despite checking for various int representations,
the tests aren't written to be portable among them.
Tests which don't work with small ints are suffixed with _intbig.py. Some
of these may still work with long long ints and need to be reclassified
later.
Previous to this patch any non-interned str/bytes objects would create a
special parse node that held a copy of the str/bytes data. Then in the
compiler this data would be turned into a str/bytes object. This actually
lead to 2 copies of the data, one in the parse node and one in the object.
The parse node's copy of the data would be freed at the end of the compile
stage but nevertheless it meant that the peak memory usage of the
parse/compile stage was higher than it needed to be (by an amount equal to
the number of bytes in all the non-interned str/bytes objects).
This patch changes the behaviour so that str/bytes objects are created
directly in the parser and the object stored in a const-object parse node
(which already exists for bignum, float and complex const objects). This
reduces peak RAM usage of the parse/compile stage, simplifies the parser
and compiler, and reduces code size by about 170 bytes on Thumb2 archs,
and by about 300 bytes on Xtensa archs.
These tests are intended to fail, as they provide a programatic record of
differences between uPy and CPython. They also contain a special comment
at the start of the file which has meta-data describing the difference,
including known causes and known workarounds.
Depending on the thread scheduler, a busy-wait loop can hog the CPU and
make the tests very slow. So convert such loops to loops that have an
explicit sleep, allowing the worker threads to do their job.
This allows using the test runner for other scenarios than just
testing uPy itself.
The principle of comparing either to CPython or else to a .exp
file is really handy but to be able to test custom modules not
built into micropython.exe one needs to be able to specify the
module search path a.k.a MICROPYPATH.
This patch implements support for class methods __delattr__ and __setattr__
for customising attribute access. It is controlled by the config option
MICROPY_PY_DELATTR_SETATTR and is disabled by default.
A few tests still fail on PYBLITE, and that's due to differences in the
available peripheral block numbers on the different MCUs (eg I2C(2)
exists on one, but it's I2C(3) on the other).
This new function controls what happens on a hard-fault:
- debugging disabled: board will do a reset
- debugging enabled: board will print registers and stack and flash LEDs
The default is disabled, ie to do a reset. This is different to previous
behaviour which flashed the LEDs and waited indefinitely.
machine.time_pulse_us() is intended to provide very fine timing, including
while working with signal bursts, where each transition is tracked in row.
Throwing and handling an exception may take too much time and "signal loss".
So instead, in case of a timeout, just return negative value. Cases of
timeout while waiting for initial signal stabilization, and during actual
timing, are recognized.
The documentation is updated accordingly, and rewritten somewhat to clarify
the function behavior.
Updated modbuiltin.c to add conditional support for 3-arg calls to
pow() using MICROPY_PY_BUILTINS_POW3 config parameter. Added support in
objint_mpz.c for for optimised implementation.
CPython 3.6 has a few changes that, when run on uPy's test suite, give a
different output to CPython 3.5. uPy currently officially supports the
3.4 language definition, but it's useful to be able to run the test suite
with 3.4/3.5/3.6 versions of CPython. This patch makes such changes to
support 3.6.
If sets are not enabled, set literals lead to SyntaxError during parsing,
so it requires feature_check. Set tests are skipped based on set_*.py
pattern.
