Commit Graph

537 Commits

Author SHA1 Message Date
Scott Shawcroft
b057fb8a4b
codeformat 2021-04-19 22:22:44 -07:00
matejcik
1a2ffda175 py/runtime: Make sys.modules preallocate to a configurable size.
This allows configuring the pre-allocated size of sys.modules dict, in
order to prevent unwanted reallocations at run-time (3 sys-modules is
really not quite enough for a larger project).
2021-04-12 22:36:16 +10:00
microDev
a52eb88031
run code formatting script 2021-03-15 19:27:36 +05:30
Damien George
7e956fae28 py: Rename BITS_PER_BYTE to MP_BITS_PER_BYTE.
To give this macro a standard MP_ prefix.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-04 22:46:42 +11:00
Damien George
8a41ee19c2 py: Remove BITS_PER_WORD definition.
It's only used in one location, to test if << or >> will overflow when
shifting mp_uint_t.  For such a test it's clearer to use sizeof(lhs_val),
which will be valid even if the type of lhs_val changes.

Signed-off-by: Damien George <damien@micropython.org>
2021-02-04 22:46:42 +11:00
Jeff Epler
b2b8520880 Always use preprocessor for MICROPY_ERROR_REPORTING
This ensures that only the translate("") alternative that will be used
is seen after preprocessing.  Improves the quality of the Huffman encoding
and reduces binary size slightly.

Also makes one "enhanced" error message only occur when ERROR_REPORTING_DETAILED:
Instead of the word-for-word python3 error message
"Type object has no attribute '%q'", the message will be
"'type' object has no attribute '%q'".  Also reduces binary size.
(that's rolled into this commit as it was right next to a change to
use the preprocessor for MICROPY_ERROR_REPORTING)

Note that the odd semicolon after "value_error:" in parsenum.c is necessary
due to a detail of the C grammar, in which a declaration cannot follow
a label directly.
2020-11-19 16:18:52 -06:00
Jeff Epler
c06fc8e02d Introduce, use mp_raise_arg1
This raises an exception with a given object value.  Saves a bit of
code size.
2020-11-19 16:15:06 -06:00
Damien George
5f9b105244 py/runtime: Fix builtin compile() in "single" mode so it prints exprs.
As per CPython behaviour, compile(stmt, "file", "single") should create
code which prints to stdout (via __repl_print__) the results of any
expressions in stmt.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-22 11:38:46 +10:00
Scott Shawcroft
701e80a025
Make socket reads interruptable 2020-08-21 11:00:02 -07:00
Scott Shawcroft
eb8b42aff1
Add basic error handling 2020-08-19 14:23:28 -07:00
Jeff Epler
89797fd3f9 various: Use mp_obj_get_type_qstr more widely
This removes runtime allocations of the cstring version of the qstring.

It is not a size improvement
2020-08-04 14:45:45 -05:00
Jeff Epler
67eb93fc98 py: introduce, use mp_raise_msg_vlist
This saves a very small amount of flash, 8 bytes on trinket_m0
2020-08-04 13:34:29 -05:00
Jeff Epler
dddd25a776 Combine similar strings to reduce size of translations
This is a slight trade-off with code size, in places where a "_varg"
mp_raise variant is now used.  The net savings on trinket_m0 is
just 32 bytes.

It also means that the translation will include the original English
text, and cannot be translated.  These are usually names of Python
types such as int, set, or dict or special values such as "inf" or
"Nan".
2020-08-04 13:34:29 -05:00
Damien George
9883d8e818 py/persistentcode: Maintain root ptr list of imported native .mpy code.
On ports where normal heap memory can contain executable code (eg ARM-based
ports such as stm32), native code loaded from an .mpy file may be reclaimed
by the GC because there's no reference to the very start of the native
machine code block that is reachable from root pointers (only pointers to
internal parts of the machine code block are reachable, but that doesn't
help the GC find the memory).

This commit fixes this issue by maintaining an explicit list of root
pointers pointing to native code that is loaded from an .mpy file.  This
is not needed for all ports so is selectable by the new configuration
option MICROPY_PERSISTENT_CODE_TRACK_RELOC_CODE.  It's enabled by default
if a port does not specify any special functions to allocate or commit
executable memory.

A test is included to test that native code loaded from an .mpy file does
not get reclaimed by the GC.

Fixes #6045.

Signed-off-by: Damien George <damien@micropython.org>
2020-08-02 22:34:09 +10:00
Diego Elio Pettenò
34b4993d63 Add license to some obvious files. 2020-07-06 19:16:25 +01:00
Damien George
332d83343f py: Rework mp_convert_member_lookup to properly handle built-ins.
This commit fixes lookups of class members to make it so that built-in
functions that are used as methods/functions of a class work correctly.

The mp_convert_member_lookup() function is pretty much completely changed
by this commit, but for the most part it's just reorganised and the
indenting changed.  The functional changes are:

- staticmethod and classmethod checks moved to later in the if-logic,
  because they are less common and so should be checked after the more
  common cases.

- The explicit mp_obj_is_type(member, &mp_type_type) check is removed
  because it's now subsumed by other, more general tests in this function.

- MP_TYPE_FLAG_BINDS_SELF and MP_TYPE_FLAG_BUILTIN_FUN type flags added to
  make the checks in this function much simpler (now they just test this
  bit in type->flags).

- An extra check is made for mp_obj_is_instance_type(type) to fix lookup of
  built-in functions.

Fixes #1326 and #6198.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-30 23:55:32 +10:00
stijn
84fa3312cf all: Format code to add space after C++-style comment start.
Note: the uncrustify configuration is explicitly set to 'add' instead of
'force' in order not to alter the comments which use extra spaces after //
as a means of indenting text for clarity.
2020-04-23 11:24:25 +10:00
stijn
70affd9ba2 all: Fix implicit floating point to integer conversions.
These are found when building with -Wfloat-conversion.
2020-04-18 22:42:24 +10:00
Damien George
8e048d2548 all: Clean up error strings to use lowercase and change cannot to can't.
Now that error string compression is supported it's more important to have
consistent error string formatting (eg all lowercase English words,
consistent contractions).  This commit cleans up some of the strings to
make them more consistent.
2020-04-13 22:19:37 +10:00
Jim Mussared
def76fe4d9 all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
Jim Mussared
85858e72df py/objexcept: Allow compression of exception message text.
The decompression of error-strings is only done if the string is accessed
via printing or via er.args.  Tests are added for this feature to ensure
the decompression works.
2020-04-05 15:02:06 +10:00
Jim Mussared
a9a745e4b4 py: Use preprocessor to detect error reporting level (terse/detailed).
Instead of compiler-level if-logic.  This is necessary to know what error
strings are included in the build at the preprocessor stage, so that string
compression can be implemented.
2020-04-05 14:11:51 +10:00
Jeff Epler
9d2270a979 runtime: Improve error message when setting read-only property
Formerly, if you wrote
   SPI.frequency = 0
you would get the sightly erroneous error message
   AttributeError: 'SPI' object has no attribute 'frequency'
In this case, a better message would read
   AttributeError: 'SPI' object cannot assign attribute 'frequency'

This new message will both be used in the case where the attribute doesn't
exist at all (and the object has no dynamic attributes; most instances of
built in types behave this way), or if the attribute exists but is
read-only.
2020-03-20 17:57:28 -05:00
Damien George
69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
Damien George
17aeb43e18 py: Un-nest configuration #if/#endif's for selection of complex code.
Because un-nested #if's are simpler to handle with formatting tools.
2020-02-28 10:30:34 +11:00
Damien George
a1b18b3ba7 py: Removing dangling "else" to improve code format consistency. 2020-02-28 10:29:27 +11:00
Damien George
1fccda049f py/objexcept: Rename mp_obj_new_exception_msg_varg2 to ..._vlist.
Follow up to recent commit ad7213d3c3, the
name "varg2" is misleading, vlist describes better that the argument is a
va_list.  This name also matches CircuitPython, which already has such
helper functions.
2020-02-18 21:00:42 +11:00
Damien George
ad7213d3c3 py: Add mp_raise_msg_varg helper and use it where appropriate.
This commit adds mp_raise_msg_varg(type, fmt, ...) as a helper for
nlr_raise(mp_obj_new_exception_msg_varg(type, fmt, ...)).  It makes the
C-level API for raising exceptions more consistent, and reduces code size
on most ports:

   bare-arm:   +28 +0.042%
minimal x86:  +100 +0.067%
   unix x64:   -56 -0.011%
unix nanbox:  -300 -0.068%
      stm32:  -204 -0.054% PYBV10
     cc3200:    +0 +0.000%
    esp8266:   -64 -0.010% GENERIC
      esp32:  -104 -0.007% GENERIC
        nrf:  -136 -0.094% pca10040
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2020-02-13 11:52:40 +11:00
Dan Halbert
5164719e67 preserve errno in OSError 2020-02-04 16:19:40 -05:00
Dan Halbert
57b566e736 Include filename for 'No such file/directory', etc. 2020-02-04 13:32:39 -05:00
Nicko van Someren
3aab54bf43 py: Support non-boolean results for equality and inequality tests.
This commit implements a more complete replication of CPython's behaviour
for equality and inequality testing of objects.  This addresses the issues
discussed in #5382 and a few other inconsistencies.  Improvements over the
old code include:

- Support for returning non-boolean results from comparisons (as used by
  numpy and others).
- Support for non-reflexive equality tests.
- Preferential use of __ne__ methods and MP_BINARY_OP_NOT_EQUAL binary
  operators for inequality tests, when available.
- Fallback to op2 == op1 or op2 != op1 when op1 does not implement the
  (in)equality operators.

The scheme here makes use of a new flag, MP_TYPE_FLAG_NEEDS_FULL_EQ_TEST,
in the flags word of mp_obj_type_t to indicate if various shortcuts can or
cannot be used when performing equality and inequality tests.  Currently
four built-in classes have the flag set: float and complex are
non-reflexive (since nan != nan) while bytearray and frozenszet instances
can equal other builtin class instances (bytes and set respectively).  The
flag is also set for any new class defined by the user.

This commit also includes a more comprehensive set of tests for the
behaviour of (in)equality operators implemented in special methods.
2020-01-30 14:53:07 +11:00
Scott Shawcroft
7d8dac9211
Refine iMX RT memory layout and add three boards
Introduces a way to place CircuitPython code and data into
tightly coupled memory (TCM) which is accessible by the CPU in a
single cycle. It also frees up room in the corresponding cache for
intermittent data. Loading from external flash is slow!

The data cache is also now enabled.

Adds support for the iMX RT 1021 chip. Adds three new boards:
* iMX RT 1020 EVK
* iMX RT 1060 EVK
* Teensy 4.0

Related to #2492, #2472 and #2477. Fixes #2475.
2020-01-17 17:36:08 -08:00
David Lechner
339d0816c5 py/runtime: Move MICROPY_PORT_INIT_FUNC near the end of mp_init().
This moves the MICROPY_PORT_INIT_FUNC hook to the end of mp_init(), just
before MP_THREAD_GIL_ENTER(), so that everything (in particular the GIL
mutex) is intialized before the hook is called.  MICROPY_PORT_DEINIT_FUNC
is also moved to be symmetric (but there is no functional change there).

If a port needs to perform initialisation earlier than
MICROPY_PORT_INIT_FUNC then it can do it before calling mp_init().
2020-01-12 13:27:04 +11:00
Damien George
bfbd94401d py: Make mp_obj_get_type() return a const ptr to mp_obj_type_t.
Most types are in rodata/ROM, and mp_obj_base_t.type is a constant pointer,
so enforce this const-ness throughout the code base.  If a type ever needs
to be modified (eg a user type) then a simple cast can be used.
2020-01-09 11:25:26 +11:00
Damien George
aacd618939 py/runtime: Don't allocate iter buf for user-defined types.
A user-defined type that defines __iter__ doesn't need any memory to be
pre-allocated for its iterator (because it can't use such memory).  So
optimise for this case by not allocating the iter-buf.
2019-12-27 12:34:22 +11:00
Yonatan Goldschmidt
07ccb5588c py/objobject: Add object.__setattr__ function.
Allows assigning attributes on class instances that implement their own
__setattr__.  Both object.__setattr__ and super(A, b).__setattr__ will work
with this commit.
2019-12-21 00:12:08 +11:00
Scott Shawcroft
5e857fdb67
Use BluetoothError in _bleio
This better differentiates errors than using OSError everywhere.
2019-11-20 14:02:15 -08:00
Damien George
323d47887f py/runtime: Reorder some binary ops so they don't require conditionals.
runtime0.h is part of the MicroPython ABI so it's simpler if it's
independent of config options, like MICROPY_PY_REVERSE_SPECIAL_METHODS.

What's effectively done here is to move MP_BINARY_OP_DIVMOD and
MP_BINARY_OP_CONTAINS up in the enum, then remove the #if
MICROPY_PY_REVERSE_SPECIAL_METHODS conditional.

Without this change .mpy files would need to have a feature flag for
MICROPY_PY_REVERSE_SPECIAL_METHODS (when embedding native code that uses
this enum).

This commit has no effect when MICROPY_PY_REVERSE_SPECIAL_METHODS is
disabled.  With this option enabled this commit reduces code size by about
60 bytes.
2019-10-29 23:13:51 +11:00
Damien George
809d89c794 py/runtime: Fix PEP479 behaviour throwing StopIteration into yield from.
Commit 3f6ffe059f implemented PEP479 but did
not catch the case fixed in this commit.  Found by coverage analysis, that
the VM had uncovered code.
2019-10-04 23:27:00 +10:00
Jim Mussared
16f8ceeaaa extmod/modbluetooth: Add low-level Python BLE API. 2019-10-01 09:51:02 +10:00
Milan Rossa
310b3d1b81 py: Integrate sys.settrace feature into the VM and runtime.
This commit adds support for sys.settrace, allowing to install Python
handlers to trace execution of Python code.  The interface follows CPython
as closely as possible.  The feature is disabled by default and can be
enabled via MICROPY_PY_SYS_SETTRACE.
2019-08-30 16:44:12 +10:00
Damien George
af20c2ead3 py: Add global default_emit_opt variable to make emit kind persistent.
mp_compile no longer takes an emit_opt argument, rather this setting is now
provided by the global default_emit_opt variable.

Now, when -X emit=native is passed as a command-line option, the emitter
will be set for all compiled modules (included imports), not just the
top-level script.

In the future there could be a way to also set this variable from a script.

Fixes issue #4267.
2019-08-28 12:47:58 +10:00
Damien George
bc9b656f35 py/runtime: Remove obsolete comment about mp_parse_compile_execute.
mp_locals_get/set and mp_globals_get/set are now static-inline functions so
this comment is no longer correct.
2019-08-22 15:59:14 +10:00
Milan Rossa
cb3647004f py: Implement new sys.atexit feature.
This patch implements a new sys.atexit function which registers a function
that is later executed when the main script ends.  It is configurable via
MICROPY_PY_SYS_ATEXIT, disabled by default.

This is not compliant with CPython, rather it can be used to implement a
CPython compatible "atexit" module if desired (similar to how
sys.print_exception can be used to implement functionality of the
"traceback" module).
2019-08-15 17:30:50 +10:00
Paul m. p. P
60f1063797 py/runtime: Allow to override builtins.__import__ with Python func.
This patch adds a simple but powerful hook into the import system, in a
CPython compatible way, by allowing to override builtins.__import__.

This does introduce some overhead to all imports but it's minor:
- the dict lookup of __import__ is bypassed if there are no modifications
  to the builtins module (which is the case at start up);
- imports are not performance critical, usually done just at the start of a
  script;
- compared to how much work is done in an import, looking up a value in a
  dict is a relatively small additional piece of work.
2019-07-31 22:36:00 +10:00
Matt Wozniski
f325bd848b Use OverflowError exception type for overflows
Initially I used one of the existing exception raise functions, but it's
more correct and not much more code to raise OverflowError instead.
2019-05-09 03:22:26 -04:00
Terri Oda
a9b05d37d7 Make status light flash blue for incompatible mpy (fixes #1369) 2019-05-08 11:54:08 -07:00
Jun Wu
089c9b71d1 py: remove "if (0)" and "if (false)" branches.
Prior to this commit, building the unix port with `DEBUG=1` and
`-finstrument-functions` the compilation would fail with an error like
"control reaches end of non-void function".  This change fixes this by
removing the problematic "if (0)" branches.  Not all branches affect
compilation, but they are all removed for consistency.
2019-05-06 18:28:28 +10:00
Damien George
9ce25d7022 py/runtime: Fix mp_unpack_ex so seq can't be reclaimed by GC during use.
The issue described in the comment added here can be seen by forcing a
gc_collect() at the start of each call to gc_alloc().
2019-04-15 11:30:19 +10:00
Damien George
1754c71f45 py/runtime: Optimise to not create temp float for int to power negative. 2019-04-15 11:04:59 +10:00
Scott Shawcroft
72992070c5
Fix boards with no shared busses. 2019-04-09 11:36:10 -07:00
Andrew Leech
8977c7eb58 py/scheduler: Convert micropythyon.schedule() to a circular buffer.
This means the schedule operates on a first-in, first-executed manner
rather than the current last-in, first executed.
2019-03-26 16:35:42 +11:00
Damien George
440462b18e py/runtime: Remove long-obsolete MICROPY_FSUSERMOUNT init code.
In 1808b2e8d5 it was replaced by MICROPY_VFS
and related code.
2019-03-20 00:16:37 +11:00
Scott Shawcroft
46fd60c703
Prevent infinite display update recursion and fix VFS mounting
Fixes #1529
2019-02-19 14:50:31 -08:00
Damien George
054dd33eba py: Downcase MP_xxx_SLOT_IS_FILLED inline functions. 2019-02-12 14:54:51 +11:00
Damien George
eee1e8841a py: Downcase all MP_OBJ_IS_xxx macros to make a more consistent C API.
These macros could in principle be (inline) functions so it makes sense to
have them lower case, to match the other C API functions.

The remaining macros that are upper case are:
- MP_OBJ_TO_PTR, MP_OBJ_FROM_PTR
- MP_OBJ_NEW_SMALL_INT, MP_OBJ_SMALL_INT_VALUE
- MP_OBJ_NEW_QSTR, MP_OBJ_QSTR_VALUE
- MP_OBJ_FUN_MAKE_SIG
- MP_DECLARE_CONST_xxx
- MP_DEFINE_CONST_xxx

These must remain macros because they are used when defining const data (at
least, MP_OBJ_NEW_SMALL_INT is so it makes sense to have
MP_OBJ_SMALL_INT_VALUE also a macro).

For those macros that have been made lower case, compatibility macros are
provided for the old names so that users do not need to change their code
immediately.
2019-02-12 14:54:51 +11:00
Paul Sokolovsky
8fea833e3f py: Update my copyright info on some files.
Based on git history.
2019-02-06 00:19:00 +11:00
Dan Halbert
4167bf5b24 wip: advertising works, but not connection 2018-12-27 00:04:04 -05:00
Damien George
fa50047bbc py/runtime: Unlock the GIL in mp_deinit function.
This mirrors what is done in mp_init.  Some RTOSs require this symmetry to
get back to a clean state (when doing a soft reset, for example).
2018-12-27 14:20:31 +11:00
Paul Sokolovsky
b1d08726ee py/obj: Add support for __int__ special method.
Based on the discussion, this special method is available unconditionally,
as converting to int is a common operation.
2018-12-07 17:28:04 +11:00
Dan Halbert
1763ffe245 More UUID work; use mp_raise for exceptions 2018-11-20 23:04:58 -05:00
Paul Sokolovsky
5c18730f28 py/runtime: Fix qstr assumptions when handling "import *".
There was an assumption that all names in a module dict are qstr's.
However, they can be dynamically generated (by assigning to globals()),
and in case of a long name, it won't be a qstr. Handle this situation
properly, including taking care of not creating superfluous qstr's for
names starting with "_" (which aren't imported by "import *").
2018-11-01 13:33:16 +11:00
Damien George
a9237cee82 py/runtime: Remove comment in mp_import_name about level being 0.
A non-zero level has been supported for some time now.
2018-10-01 15:35:10 +10:00
Damien George
4ab397576f py/runtime: Use mp_import_name to implement tail of mp_import_from. 2018-10-01 15:22:03 +10:00
Damien George
2c7a3061d5 py/runtime: Remove nlr protection when calling __next__ in mp_resume.
And remove related comment about needing such protection when calling send.

Reasoning for removal is as follows:
- mp_resume is only called by the VM in YIELD_FROM opcode
- if send_value != MP_OBJ_NULL then throw_value == MP_OBJ_NULL
- so if __next__ or send are called then throw_value == MP_OBJ_NULL
- if __next__ or send raise an exception without nlr protection then the
  exception will be handled by the global exception handler of the VM
- this handler already has code to handle exceptions raised in YIELD_FROM,
  including correct handling of StopIteration
- this handler doesn't handle the case of injection of GeneratorExit, but
  this won't be needed because throw_value == MP_OBJ_NULL

Note that it's already possible for mp_resume() to raise an exception
(including StopIteration) from the unprotected call to type->iternext(), so
that's why the VM already has code to handle the case of exceptions coming
out of mp_resume().

This commit reduces code size by a bit, and significantly reduces C stack
usage when using yield-from, from 88 bytes down to 40 for Thumb2, and 152
down to 72 bytes for x86-64 (better than half).  (Note that gcc doesn't
seem to tail-call optimise the call from mp_resume() to mp_obj_gen_resume()
so this saving in C stack usage helps all uses of yield-from.)
2018-09-28 22:16:56 +10:00
Damien George
b01f66c5f1 py: Shorten error messages by using contractions and some rewording. 2018-09-20 14:33:10 +10:00
stijn
89516b2b62 py/runtime: Fix incorrect test for MICROPY_PORT_DEINIT_FUNC. 2018-09-11 00:38:31 +10:00
Scott Shawcroft
de5a9d72dc
Compress all translated strings with Huffman coding.
This saves code space in builds which use link-time optimization.
The optimization drops the untranslated strings and replaces them
with a compressed_string_t struct. It can then be decompressed to
a c string.

Builds without LTO work as well but include both untranslated
strings and compressed strings.

This work could be expanded to include QSTRs and loaded strings if
a compress method is added to C. Its tracked in #531.
2018-08-16 17:40:57 -07:00
Scott Shawcroft
96ebf5bc3f
Two fixes and translate more strings.
* Fix finding translations with escaped characters.
* Add back \r to translations since its needed by screen.
2018-08-09 13:29:30 -07:00
Dan Halbert
2809b4f9dd Merge branch 'master' into micropython-25ae98f-merge 2018-07-16 19:36:40 -04:00
Dan Halbert
6dfa527f79 do not call property if referenced by class, not object 2018-07-14 22:17:25 -04:00
Damien George
e94d644a81 py/runtime: Use mp_obj_new_int_from_ll when return int is not small.
There's no need to call mp_obj_new_int() which will just fail the check for
small int and call mp_obj_new_int_from_ll() anyway.

Thanks to @Jongy for prompting this change.
2018-07-14 23:05:25 +10:00
Dan Halbert
e2e01efa84 compiles and runs; hangs on import storage;storage.VfsFat.<tab> 2018-07-13 22:51:10 -04:00
Dan Halbert
7c219600a2 WIP: after merge; before testing 2018-07-11 16:45:30 -04:00
Damien George
a6ea6b08bc py: Simplify some cases of accessing the map of module and type dict.
mp_obj_module_get_globals() returns a mp_obj_dict_t*, and type->locals_dict
is a mp_obj_dict_t*, so access the map entry of the dict directly instead
of needing to cast this mp_obj_dict_t* up to an object and then calling the
mp_obj_dict_get_map() helper function.
2018-07-08 21:31:09 +10:00
Damien George
035906419d extmod/uos_dupterm: Use native C stream methods on dupterm object.
This patch changes dupterm to call the native C stream methods on the
connected stream objects, instead of calling the Python readinto/write
methods.  This is much more efficient for native stream objects like UART
and webrepl and doesn't require allocating a special dupterm array.

This change is a minor breaking change from the user's perspective because
dupterm no longer accepts pure user stream objects to duplicate on.  But
with the recent addition of uio.IOBase it is possible to still create such
classes just by inheriting from uio.IOBase, for example:

    import uio, uos

    class MyStream(uio.IOBase):
        def write(self, buf):
            # existing write implementation
        def readinto(self, buf):
            # existing readinto implementation

    uos.dupterm(MyStream())
2018-06-12 15:06:11 +10:00
Roy Hooper
567f3e30a7 Initial implementation of supervisor.reload() 2018-05-14 16:57:50 -04:00
Damien George
bc87b862fd py/runtime: Add mp_load_method_protected helper which catches exceptions
This new helper function acts like mp_load_method_maybe but is wrapped in
an NLR handler so it can catch exceptions.  It prevents AttributeError from
propagating out, and optionally all other exceptions.  This helper can be
used to fully implement hasattr (see follow-up commit), and also for cases
where mp_load_method_maybe is used but it must now raise an exception.
2018-05-10 23:00:04 +10:00
Damien George
3f420c0c27 py: Don't include mp_optimise_value or opt_level() if compiler disabled.
Without the compiler enabled the mp_optimise_value is unused, and the
micropython.opt_level() function is not useful, so exclude these from the
build to save RAM and code size.
2018-04-04 14:24:03 +10:00
Damien George
f50b64cab5 py/runtime: Be sure that non-intercepted thrown object is an exception.
The VM expects that, if mp_resume() returns MP_VM_RETURN_EXCEPTION, then
the returned value is an exception instance (eg to add a traceback to it).
It's possible that a value passed to a generator's throw() is not an
exception so must be explicitly checked for if the thrown value is not
intercepted by the generator.

Thanks to @jepler for finding the bug.
2018-03-30 12:43:38 +11:00
Damien George
3280788195 py/runtime: Check that keys in dicts passed as ** args are strings.
Prior to this patch the code would crash if a key in a ** dict was anything
other than a str or qstr.  This is because mp_setup_code_state() assumes
that keys in kwargs are qstrs (for efficiency).

Thanks to @jepler for finding the bug.
2018-03-30 11:13:32 +11:00
Damien George
a9f6d49218 py/vm: Simplify handling of special-case STOP_ITERATION in yield from.
There's no need to have MP_OBJ_NULL a special case, the code can re-use
the MP_OBJ_STOP_ITERATION value to signal the special case and the VM can
detect this with only one check (for MP_OBJ_STOP_ITERATION).
2018-02-27 15:48:09 +11:00
Damien George
209936880d py/builtinimport: Add compile-time option to disable external imports.
The new option is MICROPY_ENABLE_EXTERNAL_IMPORT and is enabled by default
so that the default behaviour is the same as before.  With it disabled
import is only supported for built-in modules, not for external files nor
frozen modules.  This allows to support targets that have no filesystem of
any kind and that only have access to pre-supplied built-in modules
implemented natively.
2018-02-20 18:00:44 +11:00
Dan Halbert
302ad40fb2 Revert "alloca seems buggy on M4"
This reverts commit 4b1e9d8f92.
2018-01-24 19:24:36 -05:00
Dan Halbert
4b1e9d8f92 alloca seems buggy on M4 2018-01-23 20:54:02 -05:00
Damien George
f5fb68e94f py/runtime: Remove unnecessary break statements from switch. 2017-12-19 13:13:21 +11:00
Damien George
30fd8484eb py/runtime: Use the Python stack when building *arg and **kwarg state.
With MICROPY_ENABLE_PYSTACK enabled the following language constructs no
longer allocate on the heap: f(*arg), f(**kwarg).
2017-12-11 13:49:09 +11:00
Damien George
1e5a33df41 py: Convert all uses of alloca() to use new scoped allocation API. 2017-12-11 13:49:09 +11:00
Damien George
02d830c035 py: Introduce a Python stack for scoped allocation.
This patch introduces the MICROPY_ENABLE_PYSTACK option (disabled by
default) which enables a "Python stack" that allows to allocate and free
memory in a scoped, or Last-In-First-Out (LIFO) way, similar to alloca().

A new memory allocation API is introduced along with this Py-stack.  It
includes both "local" and "nonlocal" LIFO allocation.  Local allocation is
intended to be equivalent to using alloca(), whereby the same function must
free the memory.  Nonlocal allocation is where another function may free
the memory, so long as it's still LIFO.

Follow-up patches will convert all uses of alloca() and VLA to the new
scoped allocation API.  The old behaviour (using alloca()) will still be
available, but when MICROPY_ENABLE_PYSTACK is enabled then alloca() is no
longer required or used.

The benefits of enabling this option are (or will be once subsequent
patches are made to convert alloca()/VLA):
- Toolchains without alloca() can use this feature to obtain correct and
  efficient scoped memory allocation (compared to using the heap instead
  of alloca(), which is slower).
- Even if alloca() is available, enabling the Py-stack gives slightly more
  efficient use of stack space when calling nested Python functions, due to
  the way that compilers implement alloca().
- Enabling the Py-stack with the stackless mode allows for even more
  efficient stack usage, as well as retaining high performance (because the
  heap is no longer used to build and destroy stackless code states).
- With Py-stack and stackless enabled, Python-calling-Python is no longer
  recursive in the C mp_execute_bytecode function.

The micropython.pystack_use() function is included to measure usage of the
Python stack.
2017-12-11 13:49:09 +11:00
Damien George
5b8998da6d py/runtime: Move mp_exc_recursion_depth to runtime and rename to raise.
For consistency this helper function is renamed to match the other
exception helpers, and moved to their location in runtime.c.
2017-12-11 13:49:09 +11:00
Paul Sokolovsky
39dd89fe31 py/runtime: When tracing unary/binary ops, output op (method) name.
E.g.:

    >>> 1+1
    binary 26 __add__ 3 3

Output is similar to bytecode dump (numeric code, then op name).
2017-12-09 01:28:16 +02:00
Damien George
5e34a113ea py/runtime: Add MP_BINARY_OP_CONTAINS as reverse of MP_BINARY_OP_IN.
Before this patch MP_BINARY_OP_IN had two meanings: coming from bytecode it
meant that the args needed to be swapped, but coming from within the
runtime meant that the args were already in the correct order.  This lead
to some confusion in the code and comments stating how args were reversed.
It also lead to 2 bugs: 1) containment for a subclass of a native type
didn't work; 2) the expression "{True} in True" would illegally succeed and
return True.  In both of these cases it was because the args to
MP_BINARY_OP_IN ended up being reversed twice.

To fix these things this patch introduces MP_BINARY_OP_CONTAINS which
corresponds exactly to the __contains__ special method, and this is the
operator that built-in types should implement.  MP_BINARY_OP_IN is now only
emitted by the compiler and is converted to MP_BINARY_OP_CONTAINS by
swapping the arguments.
2017-11-24 14:48:23 +11:00
Damien George
9783ac282e py/runtime: Simplify handling of containment binary operator.
In mp_binary_op, there is no need to explicitly check for type->getiter
being non-null and raising an exception because this is handled exactly by
mp_getiter().  So just call the latter unconditionally.
2017-11-24 12:07:12 +11:00
Scott Shawcroft
7b393bc406 Merge tag 'v1.9.3'
Introduction of ports subdirectory where all ports are moved to

The main change in this release is the introduction of a "ports/"
subdirectory at the top-level of the repository, and all of the ports are
moved here.  In the process the "stmhal" port is renamed to "stm32" to
better reflect the MCU that it targets.  In addition, the STM32 CMSIS and
HAL sources are moved to a new submodule called "stm32lib".

The bytecode has changed in this release, compared to the previous release,
and as a consequence the .mpy version number has increased to version 3.
This means that scripts compiled with the previous mpy-cross must be
recompiled to work with this new version.

There have also been various enhancements and optimisations, such as:
check for valid UTF-8 when creating str objects, support for reverse
special binary operations like __radd__, full domain checking in the math
module, support for floor-division and modulo in the viper emitter,
and addition of stack overflow checking when executing a regex.

The stm32 port sees improved support for F7 MCUs, addition of a new board
B_L475E_IOT01A based on the STM32L475, and support for the Wiznet W5500
chipset along with improved socket behaviour.

A detailed list of changes follows.

py core:
- objstr: startswith, endswith: check arg to be a string
- nlrx86,x64: replace #define of defined() with portable macro usage
- objtype: handle NotImplemented return from binary special methods
- objtype: mp_obj_class_lookup: improve debug logging
- map: remove unused new/free functions
- make m_malloc_fail() have void return type, since it doesn't return
- modstruct: in struct.pack, stop converting if there are no args left
- modstruct: check and prevent buffer-read overflow in struct unpacking
- modstruct: check and prevent buffer-write overflow in struct packing
- nlrthumb: get working again on standard Thumb arch (ie not Thumb2)
- objfloat: fix binary ops with incompatible objects
- obj: fix comparison of float/complex NaN with itself
- objtype: implement fallback for instance inplace special methods
- objtuple: properly implement comparison with incompatible types
- objstr: add check for valid UTF-8 when making a str from bytes
- objlist: properly implement comparison with incompatible types
- runtime0.h: move relational ops to the beginning of mp_binary_op_t
- runtime0.h: move MP_BINARY_OP_DIVMOD to the end of mp_binary_op_t
- objtype: make sure mp_binary_op_method_name has full size again
- runtime0.h: put inplace arith ops in front of normal operations
- builtinhelp: simplify code slightly by extracting object type
- runtime: implement dispatch for "reverse op" special methods
- nlrx86: fix building for Android/x86
- builtinhelp: change signature of help text var from pointer to array
- runtime.h: change empty mp_warning macro so var-args are non empty
- modbuiltins: implement abs() by dispatching to MP_UNARY_OP_ABS
- {objfloat,objcomplex}: optimise MP_UNARY_OP_ABS by reusing variables
- mpconfig.h: add note that using computed gotos in VM is not C99
- objstr: strip: don't strip "\0" by default
- objexcept: prevent infinite recursion when allocating exceptions
- stream: remove unnecessary checks for NULL return from vstr_add_len
- vstr: raise a RuntimeError if fixed vstr buffer overflows
- vm: use lowercase letter at start of exception message
- persistentcode: define mp_raw_code_save_file() for any unix target
- add config option to print warnings/errors to stderr
- objfloat: support raising a negative number to a fractional power
- objset: simplify set and frozenset by separating their locals dicts
- objset: check that RHS of a binary op is a set/frozenset
- objset: include the failed key in a KeyError raised from set.remove
- objtype: change type of enum-to-qstr table to uint16_t to save space
- objstr: make empty bytes object have a null-terminating byte
- mpprint: only check for null string printing when NDEBUG not defined
- objtype: clean up unary- and binary-op enum-to-qstr mapping tables
- persistentcode: bump .mpy version number to version 3
- bc: update opcode_format_table to match the bytecode
- modmath: add full checks for math domain errors
- modmath: convert log2 macro into a function
- formatfloat: don't print the negative sign of a NaN value
- formatfloat: use standard isinf, isnan funcs instead of custom ones
- modbuiltins: use existing utf8_get_char helper in builtin ord func
- emitnative: implement floor-division and modulo for viper emitter
- objtype: use CPython compatible method name for sizeof
- objtype: fit qstrs for special methods in byte type
- objtype: define all special methods if requested
- objtype: introduce MICROPY_PY_ALL_INPLACE_SPECIAL_METHODS

extmod:
- modubinascii: only include uzlib/tinf.h when it's really needed
- modussl_mbedtls: allow to compile with MBEDTLS_DEBUG_C disabled
- machine_pinbase: put PinBase singleton in ROM
- re1.5: upgrade to v0.8.2, adds hook for stack overflow checking
- modure: add stack overflow checking when executing a regex
- uos_dupterm: update uos.dupterm() and helper funcs to have index
- uos_dupterm: swallow any errors from dupterm closing the stream
- vfs: replace VLA in proxy func with small, static sized array
- modussl: add finaliser support for ussl objects
- modussl_mbedtls: allow to compile with unix coverage build

lib:
- add new submodule, stm32lib containing STM32 CMSIS and HAL source
- embed/abort_: use mp_raise_msg helper function
- libm: fix tanhf so that it correctly handles +/- infinity args
- libm: remove implementation of log2f, use MP_NEED_LOG2 instead
- axtls: update, support for SSL_EAGAIN return code
- berkeley-db-1.xx: update, allow to override MINCACHE, DEFPSIZE

drivers:
- memory/spiflash: change from hard-coded soft SPI to generic SPI
- display/ssd1306.py: improve performance of graphics methods
- nrf24l01: make nRF24L01 test script more portable
- display/ssd1306: implement SSD1306_I2C poweron method
- display/ssd1306: make poweron() work the same with SSD1306_SPI
- wiznet5k: improve the performance of socket ops with threading
- wiznet5k: get low-level W5500 driver working

tools:
- upip: upgrade to 1.2.2
- pyboard: use repr() when quoting data in error messages
- pyboard: update docstring for additional device support

tests:
- object_new: better messages, check user __new__() method
- class_new: add checks for __init__ being called and other improvements
- class_new: add another testcase for __new__/__init__ interaction
- class_inplace_op: test for inplace op fallback to normal one
- run-bench-tests: update locations of executables, now in ports/
- class_reverse_op: test for reverse arith ops special methods
- run-tests: skip class_inplace_op for minimal profile
- run-tests: fix copy-paste mistake in var name
- cpydiff: add cases for locals() discrepancies
- extmod: add test for ure regexes leading to infinite recursion
- extmod: add test for '-' in character class in regex
- run-tests: close device under test using "finally"
- net_inet: update tls test to work with CPython and incl new site

unix port:
- rename modsocket.c to modusocket.c
- modusocket: remove #if MICROPY_SOCKET_EXTRA code blocks
- enable MICROPY_PY_REVERSE_SPECIAL_METHODS

stm32 port:
- modmachine: make machine.bootloader() work when MPU is enabled
- modmachine: improve support for sleep/deepsleep on F7 MCUs
- compute PLL freq table during build instead of at run time
- modmachine: for F7 MCU, save power by reducing internal volt reg
- boards/pllvalues.py: make script work with both Python 2 and 3
- Makefile: use lib/stm32lib instead of local cmsis and hal files
- remove cmsis and hal files, they are now a submodule
- Makefile: automatically fetch stm32lib submodule if needed
- update to new STM Cube HAL library
- fix clock initialisation of L4 MCUs
- rename stmhal port directory to stm32
- remove unused usbd_msc.c file
- boards: change remaining stm32f4xx_hal_conf.h to unix line ending
- boards: change linker scripts to use "K" instead of hex byte size
- boards: fix I2C1 pin mapping on NUCLEO_F401RE/F411RE boards
- i2c: when scanning for I2C devices only do 1 probe per address
- modnwwiznet5k: release the GIL on blocking network operations
- boards: add new board B_L475E_IOT01A based on STM32L475
- make-stmconst.py: make sure mpz const data lives in ROM
- timer: make pyb.Timer() instances persistent
- mpconfigport.h: add configuration for max periphs on L4 series
- usbdev: make the USBD callback struct const so it can go in ROM
- usbdev: change static function variable to non-static
- usbdev: put all CDC state in a struct
- usbdev: put all HID state in a struct
- usbdev: simplify CDC tx/rx buffer passing
- usbdev: simplify HID tx/rx buffer passing
- usbdev/core: add state parameter to all callback functions
- usbdev: put all state for the USB device driver in a struct
- usbdev: simplify pointers to MSC state and block dev operations
- usbdev: merge all global USB device state into a single struct
- usbdev: make device descriptor callbacks take a state pointer
- usbdev: move all the USB device descriptor state into its struct
- timer: enable ARPE so that timer freq can be changed smoothly
- modnwwiznet5k: get the IP address of an established socket
- boards: fix typos in stm32f767_af.csv table
- usbd_cdc_interface: don't reset CDC output buf on initialisation
- modnwwiznet5k: implement WIZNET5K.isconnected() method
- modusocket: make getaddrinfo() work when passed an IP address
- modusocket: return OSError(-2) if getaddrinfo fails
- mpconfigport.h: add MICROPY_THREAD_YIELD() macro
- modnwwiznet5k: add support for W5500 Ethernet chip
- modnwwiznet5k: increase SPI bus speed to 42MHz
- modnwwiznet5k: implement stream ioctl for the Wiznet driver
- mphalport: improve efficiency of mp_hal_stdout_tx_strn_cooked
- make uos.dupterm() conform to specs by using extmod version

cc3200 port:
- enable micropython.kbd_intr() method
- use standard implementation of keyboard interrupt

esp8266 port:
- rename axtls_helpers.c to posix_helpers.c
- posix_helpers: set ENOMEM on memory alloc failure
- set DEFPSIZE=1024, MINCACHE=3 for "btree" module
- esp_mphal: send data in chunks to mp_uos_dupterm_tx_strn
- modnetwork: add "bssid" keyword arg to WLAN.connect() method
- modules/webrepl_setup: add info about allowed password length

zephyr port:
- Makefile: revamp "test" target after ports were moved to ports/
- use CONFIG_NET_APP_SETTINGS to setup initial network addresses
- switch to interrupt-driven pull-style console

pic16bit port:
- add definition of SEEK_SET to unistd.h

docs:
- pyboard/tutorial: add "timeout=0" to UART in pass-through example
- more xrefs to "MicroPython port" in glossary
- library/network: fix ref to "socket" module (should be "usocket")
- machine.Signal: improve style/grammar and add usage example
- library: add description of "index" parameter to uos.dupterm()
- library/micropython: fix typo in RST formatting
- library/framebuf.rst: generalise constructor to all colour formats
- btree: describe page caching policy of the underlying implementation
- esp8266/tutorial: update neopixel with example of using 4 bbp
- library/network: clarify usage of "bssid" arg in connect() method
- pyboard/quickref: add info for Switch, RTC, CAN, Accel classes
- pyboard/tutorial: update now that yellow LED also supports PWM
- esp8266/quickref: add quickref info for RTC class
- library: add missing cross-ref links for classes in pyb module
- library/network: update docs to state that W5500 is supported
- uselect: document one-shot polling mode
- usocket: elaborate descriptions
- usocket: document inet_ntop(), inet_pton()
- library/network: add dhcp_hostname parameter
- reference/isr_rules: minor typo correction
- ussl: fix module name refs and use "MicroPython port" term
- esp8266/general: add section on TLS limitations
- usocket: document that settimeout() isn't supported by all ports
- ure: add "|" (alternative) to the list of supported operators
- reference/isr_rules.rst: add tutorial on use of micropython.schedule()

travis:
- use --upgrade when pip is installing cpp-coveralls
- update build command now that stm32 Wiznet config has changed

examples:
- hwconfig_console: add .on()/.off() methods

all:
- convert mp_uint_t to mp_unary_op_t/mp_binary_op_t where appropriate
- convert remaining "mp_uint_t n_args" to "size_t n_args"
- make new ports/ sub-directory and move all ports there
- update Makefiles and others to build with new ports/ dir layout
- remove inclusion of internal py header files
- use NULL instead of "" when calling mp_raise exception helpers

README:
- update "Dependencies" section
- add explicit section on contributing
- add gcc and arm-none-eabi-newlib to list of required components

.gitattributes:
- remove obsolete entries for stmhal/hal, stmhal/cmsis
- add entries for files that will move to ports/ dir
2017-11-02 12:41:50 -07:00
Scott Shawcroft
73c15dcf8b Merge commit 'f869d6b2e339c04469c6c9ea3fb2fabd7bbb2d8c' into nrf2_merge
This is prep for merging in the NRF5 pull request.
2017-10-24 22:31:16 -07:00
Damien George
37282f8fc1 extmod/uos_dupterm: Update uos.dupterm() and helper funcs to have index.
The uos.dupterm() signature and behaviour is updated to reflect the latest
enhancements in the docs.  It has minor backwards incompatibility in that
it no longer accepts zero arguments.

The dupterm_rx helper function is moved from esp8266 to extmod and
generalised to support multiple dupterm slots.

A port can specify multiple slots by defining the MICROPY_PY_OS_DUPTERM
config macro to an integer, being the number of slots it wants to have;
0 means to disable the dupterm feature altogether.

The unix and esp8266 ports are updated to work with the new interface and
are otherwise unchanged with respect to functionality.
2017-10-13 20:01:57 +11:00
Damien George
a3dc1b1957 all: Remove inclusion of internal py header files.
Header files that are considered internal to the py core and should not
normally be included directly are:
    py/nlr.h - internal nlr configuration and declarations
    py/bc0.h - contains bytecode macro definitions
    py/runtime0.h - contains basic runtime enums

Instead, the top-level header files to include are one of:
    py/obj.h - includes runtime0.h and defines everything to use the
        mp_obj_t type
    py/runtime.h - includes mpstate.h and hence nlr.h, obj.h, runtime0.h,
        and defines everything to use the general runtime support functions

Additional, specific headers (eg py/objlist.h) can be included if needed.
2017-10-04 12:37:50 +11:00
Paul Sokolovsky
9dce823cfd py/modbuiltins: Implement abs() by dispatching to MP_UNARY_OP_ABS.
This allows user classes to implement __abs__ special method, and saves
code size (104 bytes for x86_64), even though during refactor, an issue
was fixed and few optimizations were made:

* abs() of minimum (negative) small int value is calculated properly.
* objint_longlong and objint_mpz avoid allocating new object is the
  argument is already non-negative.
2017-09-18 00:06:43 +03:00
Paul Sokolovsky
eb84a830df py/runtime: Implement dispatch for "reverse op" special methods.
If, for class X, X.__add__(Y) doesn't exist (or returns NotImplemented),
try Y.__radd__(X) instead.

This patch could be simpler, but requires undoing operand swap and
operation switch to get non-confusing error message in case __radd__
doesn't exist.
2017-09-10 17:05:57 +03:00
Damien George
ca21aed0a1 py: Make m_malloc_fail() have void return type, since it doesn't return. 2017-08-31 17:00:14 +10:00