Commit Graph

352 Commits

Author SHA1 Message Date
Dan Halbert 2e260a0eab Merge remote-tracking branch 'adafruit/master' into micropython-25ae98f-merge 2018-07-28 14:13:54 -04:00
Dan Halbert be1d882a8b Prevent repetitive recursive scanning of dicts when making them long-lived 2018-07-19 15:19:21 -04:00
Dan Halbert 0d27f4d9a6 continued WIP: almost compiling 2018-07-12 14:13:51 -04:00
Dan Halbert 7c219600a2 WIP: after merge; before testing 2018-07-11 16:45:30 -04:00
Damien George db5d8c97f1 py/obj.h: Introduce a "flags" entry in mp_obj_type_t. 2018-06-08 11:48:25 +10:00
Damien George 1427f8f593 py/stream: Move definition of mp_stream_p_t from obj.h to stream.h.
Since a long time now, mp_obj_type_t no longer refers explicitly to
mp_stream_p_t but rather to an abstract "const void *protocol".  So there's
no longer any need to define mp_stream_p_t in obj.h and it can go with all
its associated definitions in stream.h.  Pretty much all users of this type
will already include the stream header.
2018-06-04 16:53:17 +10:00
Roy Hooper 567f3e30a7 Initial implementation of supervisor.reload() 2018-05-14 16:57:50 -04:00
Damien George 6b4b6d388b py/obj.h: Fix math.e constant for nan-boxing builds.
Due to a typo, math.e was too small by around 6e-11.
2018-05-01 23:25:18 +10:00
Damien George 3280788195 py/runtime: Check that keys in dicts passed as ** args are strings.
Prior to this patch the code would crash if a key in a ** dict was anything
other than a str or qstr.  This is because mp_setup_code_state() assumes
that keys in kwargs are qstrs (for efficiency).

Thanks to @jepler for finding the bug.
2018-03-30 11:13:32 +11:00
Damien George e0bc438e4b py/obj.h: Move declaration of mp_obj_list_init to objlist.h.
If this function is used then objlist.h is already included to get the
definition of mp_obj_list_t.
2018-03-13 14:03:15 +11:00
Damien George 9f811e9096 py/obj.h: Clean up by removing commented-out inline versions of macros. 2018-03-13 14:01:55 +11:00
Paul Sokolovsky 970eedce8f py/objdeque: Implement ucollections.deque type with fixed size.
So far, implements just append() and popleft() methods, required for
a normal queue. Constructor doesn't accept an arbitarry sequence to
initialize from (am empty deque is always created), so an empty tuple
must be passed as such. Only fixed-size deques are supported, so 2nd
argument (size) is required.

There's also an extension to CPython - if True is passed as 3rd argument,
append(), instead of silently overwriting the oldest item on queue
overflow, will throw IndexError. This behavior is desired in many
cases, where queues should store information reliably, instead of
silently losing some items.
2018-02-21 22:39:25 +11:00
Damien George 73d1d20b46 py/objexcept: Remove long-obsolete mp_const_MemoryError_obj.
This constant exception instance was once used by m_malloc_fail() to raise
a MemoryError without allocating memory, but it was made obsolete long ago
by 3556e45711.  The functionality is now
replaced by the use of mp_emergency_exception_obj which lives in the global
uPy state, and which can handle any exception type, not just MemoryError.
2018-02-15 16:50:02 +11:00
Scott Shawcroft 416abe33ed Introduce a long lived section of the heap.
This adapts the allocation process to start from either end of the heap
when searching for free space. The default behavior is identical to the
existing behavior where it starts with the lowest block and looks higher.
Now it can also look from the highest block and lower depending on the
long_lived parameter to gc_alloc. As the heap fills, the two sections may
overlap. When they overlap, a collect may be triggered in order to keep
the long lived section compact. However, free space is always eligable
for each type of allocation.

By starting from either of the end of the heap we have ability to separate
short lived objects from long lived ones. This separation reduces heap
fragmentation because long lived objects are easy to densely pack.

Most objects are short lived initially but may be made long lived when
they are referenced by a type or module. This involves copying the
memory and then letting the collect phase free the old portion.

QSTR pools and chunks are always long lived because they are never freed.

The reallocation, collection and free processes are largely unchanged. They
simply also maintain an index to the highest free block as well as the lowest.
These indices are used to speed up the allocation search until the next collect.

In practice, this change may slightly slow down import statements with the
benefit that memory is much less fragmented afterwards. For example, a test
import into a 20k heap that leaves ~6k free previously had the largest
continuous free space of ~400 bytes. After this change, the largest continuous
free space is over 3400 bytes.
2018-01-24 10:33:46 -08:00
Damien George 2759bec858 py: Extend nan-boxing config to have 47-bit small integers.
The nan-boxing representation has an extra 16-bits of space to store
small-int values, and making use of it allows to create and manipulate full
32-bit positive integers (ie up to 0xffffffff) without using the heap.
2017-12-11 22:39:12 +11:00
Damien George 4601759bf5 py/objstr: Remove "make_qstr_if_not_already" arg from mp_obj_new_str.
This patch simplifies the str creation API to favour the common case of
creating a str object that is not forced to be interned.  To force
interning of a new str the new mp_obj_new_str_via_qstr function is added,
and should only be used if warranted.

Apart from simplifying the mp_obj_new_str function (and making it have the
same signature as mp_obj_new_bytes), this patch also reduces code size by a
bit (-16 bytes for bare-arm and roughly -40 bytes on the bare-metal archs).
2017-11-16 13:17:51 +11:00
Scott Shawcroft 73c15dcf8b Merge commit 'f869d6b2e339c04469c6c9ea3fb2fabd7bbb2d8c' into nrf2_merge
This is prep for merging in the NRF5 pull request.
2017-10-24 22:31:16 -07:00
Damien George 77a48e8cd4 py/obj: Remove declaration for mp_obj_new_none(), it's never defined. 2017-09-04 23:35:46 +10:00
Paul Sokolovsky 9950865c39 py/objfloat: Fix binary ops with incompatible objects.
These are now returned as "operation not supported" instead of raising
TypeError. In particular, this fixes equality for float vs incompatible
types, which now properly results in False instead of exception. This
also paves the road to support reverse operation (e.g. __radd__) with
float objects.

This is achieved by introducing mp_obj_get_float_maybe(), similar to
existing mp_obj_get_int_maybe().
2017-09-02 23:05:24 +03:00
Paul Sokolovsky 784909ce16 py/objtype: Handle NotImplemented return from binary special methods.
NotImplemented means "try other fallbacks (like calling __rop__
instead of __op__) and if nothing works, raise TypeError". As
MicroPython doesn't implement any fallbacks, signal to raise
TypeError right away.
2017-08-30 01:39:24 +03:00
Damien George 58321dd985 all: Convert mp_uint_t to mp_unary_op_t/mp_binary_op_t where appropriate
The unary-op/binary-op enums are already defined, and there are no
arithmetic tricks used with these types, so it makes sense to use the
correct enum type for arguments that take these values.  It also reduces
code size quite a bit for nan-boxing builds.
2017-08-29 13:16:30 +10:00
Dan Halbert ef61b5ecb5 Initial merge of micropython v1.9.2 into circuitpython 2.0.0 (in development) master.
cpx build compiles and loads and works in repl; test suite not run yet
esp8266 not tested yet
2017-08-25 22:17:07 -04:00
Alexander Steffen 55f33240f3 all: Use the name MicroPython consistently in comments
There were several different spellings of MicroPython present in comments,
when there should be only one.
2017-07-31 18:35:40 +10:00
Alexander Steffen 299bc62586 all: Unify header guard usage.
The code conventions suggest using header guards, but do not define how
those should look like and instead point to existing files. However, not
all existing files follow the same scheme, sometimes omitting header guards
altogether, sometimes using non-standard names, making it easy to
accidentally pick a "wrong" example.

This commit ensures that all header files of the MicroPython project (that
were not simply copied from somewhere else) follow the same pattern, that
was already present in the majority of files, especially in the py folder.

The rules are as follows.

Naming convention:
* start with the words MICROPY_INCLUDED
* contain the full path to the file
* replace special characters with _

In addition, there are no empty lines before #ifndef, between #ifndef and
one empty line before #endif. #endif is followed by a comment containing
the name of the guard macro.

py/grammar.h cannot use header guards by design, since it has to be
included multiple times in a single C file. Several other files also do not
need header guards as they are only used internally and guaranteed to be
included only once:
* MICROPY_MPHALPORT_H
* mpconfigboard.h
* mpconfigport.h
* mpthreadport.h
* pin_defs_*.h
* qstrdefs*.h
2017-07-18 11:57:39 +10:00
Scott Shawcroft 30ee7019ca Merge tag 'v1.9.1'
Fixes for stmhal USB mass storage, lwIP bindings and VFS regressions

This release provides an important fix for the USB mass storage device in
the stmhal port by implementing the SCSI SYNCHRONIZE_CACHE command, which
is now require by some Operating Systems.  There are also fixes for the
lwIP bindings to improve non-blocking sockets and error codes.  The VFS has
some regressions fixed including the ability to statvfs the root.

All changes are listed below.

py core:
- modbuiltins: add core-provided version of input() function
- objstr: catch case of negative "maxsplit" arg to str.rsplit()
- persistentcode: allow to compile with complex numbers disabled
- objstr: allow to compile with obj-repr D, and unicode disabled
- modsys: allow to compile with obj-repr D and PY_ATTRTUPLE disabled
- provide mp_decode_uint_skip() to help reduce stack usage
- makeqstrdefs.py: make script run correctly with Python 2.6
- objstringio: if created from immutable object, follow copy on write policy

extmod:
- modlwip: connect: for non-blocking mode, return EINPROGRESS
- modlwip: fix error codes for duplicate calls to connect()
- modlwip: accept: fix error code for non-blocking mode
- vfs: allow to statvfs the root directory
- vfs: allow "buffering" and "encoding" args to VFS's open()
- modframebuf: fix signed/unsigned comparison pendantic warning

lib:
- libm: use isfinite instead of finitef, for C99 compatibility
- utils/interrupt_char: remove support for KBD_EXCEPTION disabled

tests:
- basics/string_rsplit: add tests for negative "maxsplit" argument
- float: convert "sys.exit()" to "raise SystemExit"
- float/builtin_float_minmax: PEP8 fixes
- basics: convert "sys.exit()" to "raise SystemExit"
- convert remaining "sys.exit()" to "raise SystemExit"

unix port:
- convert to use core-provided version of built-in import()
- Makefile: replace references to make with $(MAKE)

windows port:
- convert to use core-provided version of built-in import()

qemu-arm port:
- Makefile: adjust object-file lists to get correct dependencies
- enable micropython.mem_*() functions to allow more tests

stmhal port:
- boards: enable DAC for NUCLEO_F767ZI board
- add support for NUCLEO_F446RE board
- pass USB handler as parameter to allow more than one USB handler
- usb: use local USB handler variable in Start-of-Frame handler
- usb: make state for USB device private to top-level USB driver
- usbdev: for MSC implement SCSI SYNCHRONIZE_CACHE command
- convert from using stmhal's input() to core provided version

cc3200 port:
- convert from using stmhal's input() to core provided version

teensy port:
- convert from using stmhal's input() to core provided version

esp8266 port:
- Makefile: replace references to make with $(MAKE)
- Makefile: add clean-modules target
- convert from using stmhal's input() to core provided version

zephyr port:
- modusocket: getaddrinfo: Fix mp_obj_len() usage
- define MICROPY_PY_SYS_PLATFORM (to "zephyr")
- machine_pin: use native Zephyr types for Zephyr API calls

docs:
- machine.Pin: remove out_value() method
- machine.Pin: add on() and off() methods
- esp8266: consistently replace Pin.high/low methods with .on/off
- esp8266/quickref: polish Pin.on()/off() examples
- network: move confusingly-named cc3200 Server class to its reference
- uos: deconditionalize, remove minor port-specific details
- uos: move cc3200 port legacy VFS mounting functions to its ref doc
- machine: sort machine classes in logical order, not alphabetically
- network: first step to describe standard network class interface

examples:
- embedding: use core-provided KeyboardInterrupt object
2017-06-20 10:56:05 -07:00
Damien George 4df013c8cc py/objtype: mp_obj_new_super doesn't need to be public, so inline it.
Saves code size (20 bytes on bare-arm) and makes it a tiny bit more
efficient.
2017-04-22 12:14:04 +10:00
Damien George a73501b1d6 py/objfloat: Add implementation of high-quality float hashing.
Disabled by default.
2017-04-12 13:38:17 +10:00
Damien George 816413e4b2 py: Optimise types for common case where type has a single parent type.
The common cases for inheritance are 0 or 1 parent types, for both built-in
types (eg built-in exceptions) as well as user defined types.  So it makes
sense to optimise the case of 1 parent type by storing just the type and
not a tuple of 1 value (that value being the single parent type).

This patch makes such an optimisation.  Even though there is a bit more
code to handle the two cases (either a single type or a tuple with 2 or
more values) it helps reduce overall code size because it eliminates the
need to create a static tuple to hold single parents (eg for the built-in
exceptions).  It also helps reduce RAM usage for user defined types that
only derive from a single parent.

Changes in code size (in bytes) due to this patch:

    bare-arm:       -16
    minimal (x86): -176
    unix (x86-64): -320
    unix nanbox:   -384
    stmhal:         -64
    cc3200:         -32
    esp8266:       -108
2017-04-12 13:22:21 +10:00
Damien George fc710169b7 py/obj: Clean up and add comments describing mp_obj_type_t struct. 2017-04-12 13:20:26 +10:00
Damien George 19f2e47d59 py: Add very simple but correct hashing for float and complex numbers.
Hashing of float and complex numbers that are exact (real) integers should
return the same integer hash value as hashing the corresponding integer
value.  Eg hash(1), hash(1.0) and hash(1+0j) should all be the same (this
is how Python is specified: if x==y then hash(x)==hash(y)).

This patch implements the simplest way of doing float/complex hashing by
just converting the value to int and returning that value.
2017-04-04 11:57:21 +10:00
Damien George a5500a8aad py/obj.h: Make sequence grow more efficient and support overlapping.
The first memmove now copies less bytes in some cases (because len_adj <=
slice_len), and the memcpy is replaced with memmove to support the
possibility that dest and slice regions are overlapping.
2017-04-02 17:28:24 +10:00
Damien George 6b34107537 py: Change mp_uint_t to size_t for mp_obj_str_get_data len arg. 2017-03-29 12:56:45 +11:00
Damien George 6213ad7f46 py: Convert mp_uint_t to size_t for tuple/list accessors.
This patch changes mp_uint_t to size_t for the len argument of the
following public facing C functions:

mp_obj_tuple_get
mp_obj_list_get
mp_obj_get_array

These functions take a pointer to the len argument (to be filled in by the
function) and callers of these functions should update their code so the
type of len is changed to size_t.  For ports that don't use nan-boxing
there should be no change in generate code because the size of the type
remains the same (word sized), and in a lot of cases there won't even be a
compiler warning if the type remains as mp_uint_t.

The reason for this change is to standardise on the use of size_t for
variables that count memory (or memory related) sizes/lengths.  It helps
builds that use nan-boxing.
2017-03-29 12:56:17 +11:00
Damien George 3f810daeb3 py/obj: Change mp_uint_t to size_t for mp_obj_get_array_fixed_n len arg. 2017-03-26 19:20:06 +11:00
Damien George 60656eaea4 py: Define and use MP_OBJ_ITER_BUF_NSLOTS to get size of stack iter buf.
It improves readability of code and reduces the chance to make a mistake.

This patch also fixes a bug with nan-boxing builds by rounding up the
calculation of the new NSLOTS variable, giving the correct number of slots
(being 4) even if mp_obj_t is larger than the native machine size.
2017-03-23 16:36:08 +11:00
Damien George 507119f4d8 py/sequence: Convert mp_uint_t to size_t where appropriate. 2017-03-23 16:23:20 +11:00
Damien George c88cfe165b py: Use size_t as len argument and return type of mp_get_index.
These values are used to compute memory addresses and so size_t is the
more appropriate type to use.
2017-03-23 16:17:40 +11:00
Scott Shawcroft aef0586ee1 py: Fix varg helpers by adding vlist variant of mp_obj_new_exception_msg 2017-02-24 17:47:02 +01:00
Damien George ae8d867586 py: Add iter_buf to getiter type method.
Allows to iterate over the following without allocating on the heap:
- tuple
- list
- string, bytes
- bytearray, array
- dict (not dict.keys, dict.values, dict.items)
- set, frozenset

Allows to call the following without heap memory:
- all, any, min, max, sum

TODO: still need to allocate stack memory in bytecode for iter_buf.
2017-02-16 18:38:06 +11:00
Damien George da36f5232d py/objint: Convert mp_uint_t to size_t where appropriate. 2017-02-16 16:51:17 +11:00
Damien George fa5a591757 py/objexcept: Convert mp_uint_t to size_t where appropriate. 2017-02-16 16:51:17 +11:00
Damien George efa629028a py/objclosure: Convert mp_uint_t to size_t where appropriate. 2017-02-16 16:51:17 +11:00
Damien George dbcdb9f8d8 py/objfun: Convert mp_uint_t to size_t where appropriate. 2017-02-16 16:51:16 +11:00
Damien George ccc5254224 py/objarray: Convert mp_uint_t to size_t where appropriate. 2017-02-16 16:51:16 +11:00
Damien George c0d9500eee py/objstr: Convert mp_uint_t to size_t (and use int) where appropriate. 2017-02-16 16:51:16 +11:00
Damien George 68cd3a93f0 py/objset: Convert mp_uint_t to size_t where appropriate. 2017-02-16 16:51:16 +11:00
Damien George 1ea2f7a8ce py/objdict: Convert mp_uint_t to size_t where appropriate. 2017-02-16 16:51:16 +11:00
Damien George 58d9eeb8d9 py/objlist: Convert mp_uint_t to size_t where appropriate. 2017-02-16 16:51:16 +11:00
Damien George 229823942c py/objtuple: Convert mp_uint_t to size_t where appropriate. 2017-02-16 16:51:16 +11:00
Damien George af622eb2a6 py/map: Change mp_uint_t to size_t where appropriate.
The internal map/set functions now use size_t exclusively for computing
addresses.  size_t is enough to reach all of available memory when
computing addresses so is the right type to use.  In particular, for
nanbox builds it saves quite a bit of code size and RAM compared to the
original use of mp_uint_t (which is 64-bits on nanbox builds).
2017-02-08 11:00:15 +11:00