Commit Graph

767 Commits

Author SHA1 Message Date
Scott Shawcroft 9435e01f9e
Support __bytes
Fixes #1763
2019-10-14 16:05:17 -07:00
Dan Halbert 8664a6574b use approx of original @godlygeek code for smallints; add tests 2019-05-12 11:17:29 -04:00
Matt Wozniski e041df73bb Add tests for overflows converting ints to bytes 2019-05-09 03:22:24 -04:00
Dan Halbert 7c219600a2 WIP: after merge; before testing 2018-07-11 16:45:30 -04:00
Damien George 6a445b60fa py/lexer: Add support for underscores in numeric literals.
This is a very convenient feature introduced in Python 3.6 by PEP 515.
2018-06-12 12:17:43 +10:00
Damien George 36c1052183 py/objtype: Optimise instance get/set/del by skipping special accessors.
This patch is a code optimisation, trading text bytes for speed.  On
pyboard it's an increase of 0.06% in code size for a gain (in pystone
performance) of roughly 6.5%.

The patch optimises load/store/delete of attributes in user defined classes
by not looking up special accessors (@property, __get__, __delete__,
__set__, __setattr__ and __getattr_) if they are guaranteed not to exist in
the class.

Currently, if you do my_obj.foo() then the runtime has to do a few checks
to see if foo is a property or has __get__, and if so delegate the call.
And for stores things like my_obj.foo = 1 has to first check if foo is a
property or has __set__ defined on it.

Doing all those checks each and every time the attribute is accessed has a
performance penalty.  This patch eliminates all those checks for cases when
it's guaranteed that the checks will always fail, ie no attributes are
properties nor have any special accessor methods defined on them.

To make this guarantee it checks all attributes of a user-defined class
when it is first created.  If any of the attributes of the user class are
properties or have special accessors, or any of the base classes of the
user class have them, then it sets a flag in the class to indicate that
special accessors must be checked for.  Then in the load/store/delete code
it checks this flag to see if it can take the shortcut and optimise the
lookup.

It's an optimisation that's pretty widely applicable because it improves
lookup performance for all methods of user defined classes, and stores of
attributes, at least for those that don't have special accessors.  And, it
allows to enable descriptors with minimal additional runtime overhead if
they are not used for a particular user class.

There is one restriction on dynamic class creation that has been introduced
by this patch: a user-defined class cannot go from zero special accessors
to one special accessor (or more) after that class has been subclassed.  If
the script attempts this an AttributeError is raised (see addition to
tests/misc/non_compliant.py for an example of this case).

The cost in code space bytes for the optimisation in this patch is:

   unix x64:  +528
unix nanbox:  +508
      stm32:  +192
     cc3200:  +200
    esp8266:  +332
      esp32:  +244

Performance tests that were done:

- on unix x86-64, pystone improved by about 5%
- on pyboard, pystone improved by about 6.5%, from 1683 up to 1794
- on pyboard, bm_chaos (from CPython benchmark suite) improved by about 5%
- on esp32, pystone improved by about 30% (but there are caching effects)
- on esp32, bm_chaos improved by about 11%
2018-06-08 12:12:08 +10:00
Jeff Epler c60589c02b py/objtype: Fix assertion failures in super_attr by checking type.
Fixes assertion failures and segmentation faults when making calls like:

    super(1, 1).x
2018-05-30 11:14:07 +10:00
Jeff Epler 05b13fd292 py/objtype: Fix assertion failures in mp_obj_new_type by checking types.
Fixes assertion failures when the arguments to type() were not of valid
types, e.g., when making calls like:

    type("", (), 3)
    type("", 3, {})
2018-05-30 11:11:24 +10:00
Damien George dfeaea1441 py/objtype: Remove TODO comment about needing to check for property.
Instance members are always treated as values, even if they are properties.
A test is added to show this is the case.
2018-05-25 10:59:40 +10:00
Damien George 400273a799 py/objgenerator: Protect against reentering a generator.
Generators that are already executing cannot be reexecuted.  This patch
puts in a check for such a case.

Thanks to @jepler for finding the bug.
2018-05-22 16:54:03 +10:00
Jan Klusacek b318ebf101 py/modbuiltins: Add support for rounding integers.
As per CPython semantics.  This feature is controlled by
MICROPY_PY_BUILTINS_ROUND_INT which is disabled by default.
2018-05-22 14:18:16 +10:00
Damien George 1ad0013dec tests: Add some tests for bigint hash, float hash and float parsing.
Following outcome of recent fuzz testing and sanitizing by @jepler.
2018-05-21 13:05:40 +10:00
Dan Halbert 54293397c5
Merge pull request #837 from godlygeek/human_readable_oserror
Human readable OSError messages
2018-05-15 10:12:09 -04:00
Matt Wozniski 086bffb594 Update errno tests to match new expected behavior
Signed-off-by: Matt Wozniski <mwozniski@bloomberg.net>
2018-05-14 23:16:18 -04:00
bildzeitung bf26ffbf56 Updated tests; removed try/catch for ucollections 2018-05-14 14:43:34 -04:00
Damien George 7541be5637 tests/basics/special_methods2: Enable some additional tests that work.
These special methods are all available if MICROPY_PY_ALL_SPECIAL_METHODS
is enabled.
2018-05-11 17:37:16 +10:00
Damien George 3678a6bdc6 py/modbuiltins: Make built-in dir support the __dir__ special method.
If MICROPY_PY_ALL_SPECIAL_METHODS is enabled then dir() will now delegate
to the special method __dir__ if the object it is listing has this method.
2018-05-10 23:14:23 +10:00
Damien George 529860643b py/modbuiltins: Make built-in hasattr work properly for user types.
It now allows __getattr__ in a user type to raise AttributeError when the
attribute does not exist.
2018-05-10 23:03:30 +10:00
Jeff Epler d6cf5c6749 py/objstr: In find/rfind, don't crash when end < start. 2018-04-05 16:14:17 +10:00
Damien George 1bfc774a08 tests/basics/string_compare.py: Add test with string that hashes to 0.
The string "Q+?" is special in that it hashes to zero with the djb2
algorithm (among other strings), and a zero hash should be incremented to a
hash of 1.
2018-04-05 01:04:38 +10:00
Damien George 22161acf47 tests/basics/class_super.py: Add tests for store/delete of super attr. 2018-04-05 01:03:57 +10:00
Damien George 7b7bbd0ee7 tests/basics: Add tests for edge cases of nan-box's 47-bit small int. 2018-04-05 00:59:49 +10:00
Damien George dd48ccb1e3 tests/basics: Add test for subclassing an iterable native type. 2018-04-04 15:26:18 +10:00
Damien George df02f5620a tests/basics/int_big1.py: Add test for big int in mp_obj_get_int_maybe. 2018-04-04 15:23:32 +10:00
Damien George 7d5c753b17 tests/basics: Modify int-big tests to prevent constant folding.
So that these tests test the runtime behaviour, not the compiler (which may
be executed offline).
2018-04-04 13:57:22 +10:00
Damien George f684e9e1ab tests/basics/int_big1.py: Add test converting str with non-print chars. 2018-04-04 13:56:00 +10:00
Damien George 430efb0444 tests/basics: Add test for use of return within try-except.
The case of a return statement in the try suite of a try-except statement
was previously only tested by builtin_compile.py, and only then in the part
of this test which checked for the existence of the compile builtin.  So
this patch adds an explicit unit test for this case.
2018-04-04 01:43:16 +10:00
Scott Shawcroft e4ae1e3d59
Merge pull request #734 from jepler/str-find-backwards-circuitpython
py/objstr: Don't crash when end < start
2018-04-01 22:41:59 -07:00
Jeff Epler a909007fef py/objgenerator: Check stack before resuming a generator
This turns a hard crash in a recursive generator into
a 'maximum recursion depth exceeded' exception.
2018-04-01 16:40:15 -05:00
Jeff Epler 0041df0c6b py/objstr: Don't crash when end < start
.. and add testcases for the same.

(crash found by afl-fuzz)
2018-03-31 22:17:11 -05:00
Scott Shawcroft 3215b85568
Merge pull request #728 from jepler/double-splat-crash-circuitpython
py/bc: Turn assertion error into exception
2018-03-31 09:57:25 -07:00
Jeff Epler 853f7ac4d0 py/bc: Turn assertion error into exception 2018-03-31 08:44:29 -05:00
Damien George bcfff4fc98 tests/basics/iter1.py: Add more tests for walking a user-defined iter.
Some code in mp_iternext() was only tested by the native emitter, so the
tests added here test this function using just the bytecode emitter.
2018-03-30 14:23:13 +11:00
Damien George f50b64cab5 py/runtime: Be sure that non-intercepted thrown object is an exception.
The VM expects that, if mp_resume() returns MP_VM_RETURN_EXCEPTION, then
the returned value is an exception instance (eg to add a traceback to it).
It's possible that a value passed to a generator's throw() is not an
exception so must be explicitly checked for if the thrown value is not
intercepted by the generator.

Thanks to @jepler for finding the bug.
2018-03-30 12:43:38 +11:00
Damien George 3280788195 py/runtime: Check that keys in dicts passed as ** args are strings.
Prior to this patch the code would crash if a key in a ** dict was anything
other than a str or qstr.  This is because mp_setup_code_state() assumes
that keys in kwargs are qstrs (for efficiency).

Thanks to @jepler for finding the bug.
2018-03-30 11:13:32 +11:00
Jeff Epler ff06a45599 Fix assertion failures in super_attr
micropython: ../../py/objtype.c:1100: super_attr: Assertion `MP_OBJ_IS_TYPE(self->type, &mp_type_type)' failed.

e.g., when making calls like
    super(1, 1).x
2018-03-29 06:42:10 -05:00
Jeff Epler 6da8d7c465 Fix assertion failures in mp_obj_new_type
Fixes the following assertion failures when the arguments to type()
were not of valid types:
micropython: ../../py/objtype.c:984: mp_obj_new_type: Assertion `MP_OBJ_IS_TYPE(bases_tuple, &mp_type_tuple)' failed.
micropython: ../../py/objtype.c:994: mp_obj_new_type: Assertion `MP_OBJ_IS_TYPE(items[i], &mp_type_type)' failed.

e.g., when making calls like
    type("", (), 3)
    type("", 3, {})
2018-03-29 06:42:10 -05:00
Jeff Epler a55988a547 Fix assertion failure in mpz_divmod_inpl
.. turning this from an assertion failure into an exception:
    pow(1,1,0)
2018-03-29 06:42:10 -05:00
Scott Shawcroft 4517ab8ba4
Merge pull request #709 from jepler/core-class-superproperty
Make test core_class_superproperty.py succeed
2018-03-26 21:59:05 -07:00
Jeff Epler 047a4f59c5 This test now passes, make it run regularly 2018-03-26 18:47:06 -05:00
Jeff Epler 355bf8b553 Conditionally compile out nonstandard array/struct typecodes
.. defaulting to off for circuitpython-supported boards, on for others.

.. fixing up the tests that fail when it is turned off, so that they skip
instead of failing
2018-03-26 18:13:49 -05:00
Jeff Epler cdb83b18ec Implement * and *= for array.array 2018-03-23 07:37:07 -05:00
Damien George 72adc381fb tests/basics/builtin_enumerate: Add test for many pos args to enumerate. 2018-03-08 12:51:06 +11:00
Damien George 439acddc60 tests/basics/gc1: Add test which triggers GC threshold. 2018-02-27 22:39:17 +11:00
Damien George 22ade2f5c4 py/vm: Fix case of handling raised StopIteration within yield from.
This patch concerns the handling of an NLR-raised StopIteration, raised
during a call to mp_resume() which is handling the yield from opcode.

Previously, commit 6738c1dded introduced code
to handle this case, along with a test.  It seems that it was lucky that
the test worked because the code did not correctly handle the stack pointer
(sp).

Furthermore, commit 79d996a57b improved the
way mp_resume() propagated certain exceptions: it changed raising an NLR
value to returning MP_VM_RETURN_EXCEPTION.  This change meant that the
test introduced in gen_yield_from_ducktype.py was no longer hitting the
code introduced in 6738c1dded.

The patch here does two things:

1. Fixes the handling of sp in the VM for the case that yield from is
   interrupted by a StopIteration raised via NLR.

2. Introduces a new test to check this handling of sp and re-covers the
   code in the VM.
2018-02-27 15:39:31 +11:00
Damien George 90da791a08 tests/basics: Add test for calling a subclass of a native class.
Adding this test gets py/objtype.c to 100% coverage.
2018-02-24 23:13:42 +11:00
Damien George 160d670868 py/objdeque: Protect against negative maxlen in deque constructor.
Otherwise passing -1 as maxlen will lead to a zero allocation and
subsequent unbound buffer overflow in deque.append() because i_put is
allowed to grow without bound.
2018-02-21 23:34:17 +11:00
Damien George 8f9b113be2 tests/basics: Add tests to improve coverage of py/objdeque.c. 2018-02-21 23:19:06 +11:00
Paul Sokolovsky 4668ec801e tests/basics/deque*: Tests for ucollections.deque. 2018-02-21 22:58:14 +11:00
Damien George 4e469085c1 py/objstr: Protect against creating bytes(n) with n negative.
Prior to this patch uPy (on a 32-bit arch) would have severe issues when
calling bytes(-1): such a call would call vstr_init_len(vstr, -1) which
would then +1 on the len and call vstr_init(vstr, 0), which would then
round this up and allocate a small amount of memory for the vstr.  The
bytes constructor would then attempt to zero out all this memory, thinking
it had allocated 2^32-1 bytes.
2018-02-19 16:25:30 +11:00