Commit Graph

2600 Commits

Author SHA1 Message Date
Chris Packham
8c6856d2e7 py/emitglue.c: provide mp_raw_code_load_file for any unix architecture
Signed-off-by: Chris Packham <chris.packham@alliedtelesis.co.nz>
2016-09-04 16:17:46 +10:00
Damien George
5f3bda422a py: If str/bytes hash is 0 then explicitly compute it. 2016-09-02 14:49:50 +10:00
Damien George
f127bef3e4 py/makeqstrdata.py: Compute the qstr hash from bytes, not characters. 2016-09-02 14:32:47 +10:00
Delio Brignoli
f98bb2ddcb py/mpprint: Fail an assertion with unsupported format specifiers.
Arguments of an unknown type cannot be skipped and continuing to parse a
format string after encountering an unknown format specifier leads to
undefined behaviour.  This patch helps to find use of unsupported formats.
2016-09-01 18:09:44 +10:00
Damien George
0823c1baf8 extmod: Add machine_spi with generic SPI C-protocol and helper methods.
The idea is that all ports can use these helper methods and only need to
provide initialisation of the SPI bus, as well as a single transfer
function.  The coding pattern follows the stream protocol and helper
methods.
2016-09-01 15:07:20 +10:00
Paul Sokolovsky
1f61fe07a2 py/mkrules.mk: Allow to override name of libmicropython.a
Or alternatively, refer to an exact library file, not just phony target
"lib".
2016-08-30 00:59:17 +03:00
Damien George
f4ee9f8853 py/bc.h: Rename _mp_code_state to _mp_code_state_t.
This rename was missed in the previous patch.
2016-08-27 23:23:51 +10:00
Damien George
581a59a456 py: Rename struct mp_code_state to mp_code_state_t.
Also at _t to mp_exc_stack pre-declaration in struct typedef.
2016-08-27 23:21:00 +10:00
Damien George
26295e04ff py/emitglue: Use more compact mp_raise_ValueError function. 2016-08-26 23:45:07 +10:00
Damien George
fe3cc5bb53 py/modstruct: Use more compact mp_raise_ValueError function.
Saves a few bytes of code size.
2016-08-26 23:42:40 +10:00
Damien George
24df30c133 py/compile: Don't compile assert statements when optimisations enabled.
As per CPython.
2016-08-26 22:28:22 +10:00
Damien George
5ffe1d8dc0 py/gc: Add MICROPY_GC_CONSERVATIVE_CLEAR option to always zero memory.
There can be stray pointers in memory blocks that are not properly zero'd
after allocation.  This patch adds a new config option to always zero all
allocated memory (via gc_alloc and gc_realloc) and hence help to eliminate
stray pointers.

See issue #2195.
2016-08-26 15:35:26 +10:00
Paul Sokolovsky
c428367543 extmod/modubinascii: Make crc32() support configurable.
Disable by default, enable in unix port.
2016-08-24 18:28:43 +03:00
Krzysztof Blazewicz
6562076454 py/stream.c: use mp_obj_get_type in mp_get_stream_raise
In current state `mp_get_stream_raise` assumes that `self_in` is an object
and always performs a pointer derefence which may cause a segfault.

This function shall throw an exception whenever `self_in` does not implement
a stream protocol, that includes qstr's and numbers.

fixes #2331
2016-08-24 01:33:31 +03:00
Damien George
095e43a9a5 py/sequence: Allow to use bignums as indices in slice objects.
See issue #2264.
2016-08-15 23:26:34 +10:00
Damien George
41fceae559 py/obj.h: For obj reprs A,B,C use void* explicitly for mp_obj_t typedef.
The machine_ptr_t type is long obsolete as the type of mp_obj_t is now
defined by the object representation, ie by MICROPY_OBJ_REPR.  So just use
void* explicitly for the typedef of mp_obj_t.

If a port wants to use something different then they should define a new
object representation.
2016-08-15 10:56:55 +10:00
Damien George
2196799051 py/objstr: Use mp_raise_{Type,Value}Error instead of mp_raise_msg.
This patch does further refactoring using the new mp_raise_TypeError
and mp_raise_ValueError functions.
2016-08-14 16:51:54 +10:00
Damien George
1a0d3fd632 py/runtime.h: Move comment about mp_not_implemented to correct place. 2016-08-14 16:35:10 +10:00
Damien George
4aaa5adf9f py/objtuple: In tuple_cmp_helper, use mp_check_self instead of raising.
Only tuple, namedtuple and attrtuple use the tuple_cmp_helper function,
and they all have getiter=mp_obj_tuple_getiter, so the check here is only
to ensure that the self object is consistent.  Hence use mp_check_self.
2016-08-14 16:28:05 +10:00
Paul Sokolovsky
c4a8004933 py: Get rid of assert() in method argument checking functions.
Checks for number of args removes where guaranteed by function descriptor,
self checking is replaced with mp_check_self(). In few cases, exception
is raised instead of assert.
2016-08-12 22:39:03 +03:00
Paul Sokolovsky
83e0ebabb4 py/objdict: Get rid of asserts (remove/replace with mp_check_self()). 2016-08-12 22:01:11 +03:00
Paul Sokolovsky
8c50f93a41 py/runtime.h: Define mp_check_self(pred) helper macro.
Indended to replace raw asserts in bunch of files. Expands to empty
if MICROPY_BUILTIN_METHOD_CHECK_SELF_ARG is defined, otehrwise by
default still to assert, though a particular port may define it to
something else.
2016-08-12 21:58:56 +03:00
Paul Sokolovsky
9e1b61dedd py/runtime: Factor out exception raising helpers.
Introduce mp_raise_msg(), mp_raise_ValueError(), mp_raise_TypeError()
instead of previous pattern nlr_raise(mp_obj_new_exception_msg(...)).
Save few bytes on each call, which are many.
2016-08-12 21:28:45 +03:00
Paul Sokolovsky
1bc2911174 py/mpconfig.h: Define MP_ALWAYSINLINE for reuse.
Similar to existing MP_NOINLINE.
2016-08-07 22:36:05 +03:00
Paul Sokolovsky
1563388001 py/objstr,objstrunicode: Fix inconistent #if indentation. 2016-08-07 15:24:57 +03:00
Paul Sokolovsky
56eb25f049 py/objstr: Make .partition()/.rpartition() methods configurable.
Default is disabled, enabled for unix port. Saves 600 bytes on x86.
2016-08-07 06:46:55 +03:00
Paul Sokolovsky
7fb31479bf py/mkrules.mk: Allow to add more items for "clean" target using CLEAN_EXTRA. 2016-08-04 23:46:08 +03:00
Paul Sokolovsky
0dfe849413 py/py.mk: Extra switches to build "embedded" BerkeleyDB BTree lib. 2016-07-31 00:39:09 +03:00
Paul Sokolovsky
61e77a4e88 py/mpconfig.h: Add MICROPY_STREAMS_POSIX_API setting.
To filter out even prototypes of mp_stream_posix_*() functions, which
require POSIX types like ssize_t & off_t, which may be not available in
some ports.
2016-07-30 20:05:56 +03:00
Paul Sokolovsky
c8b80e4740 lib/embed/abort_: Implementation of abort_() function raising uPy exception.
Helpful when porting existing C libraries to MicroPython. abort()ing in
embedded environment isn't a good idea, so when compiling such library,
-Dabort=abort_ option can be given to redirect standard abort() to this
"safe" version.
2016-07-30 00:35:50 +03:00
Paul Sokolovsky
4f1b0292db py/stream: Add adapter methods with POSIX-compatible signatures.
Previoussly such read() and write() methods were used by modussl_axtls,
move to py/stream for reuse.
2016-07-30 00:25:06 +03:00
Paul Sokolovsky
3990b1715d py/objstringio: Implement MP_STREAM_SEEK ioctl and add seek() method. 2016-07-28 01:53:44 +03:00
Paul Sokolovsky
f039ac5bd7 py/objstringio: Add MP_STREAM_FLUSH ioctl and flush() method.
No-op for this object.
2016-07-28 01:14:32 +03:00
Paul Sokolovsky
a60b0263ba py/stream: Implement generic flush() method, in terms of C-level ioctl. 2016-07-27 00:39:10 +03:00
Paul Sokolovsky
f2f761c0c3 py/stream: Stream module works with errno's, so should include mperrno.h. 2016-07-26 15:05:15 +03:00
Paul Sokolovsky
ed1c194ebf py/objstrunicode: str_index_to_ptr: Implement positive indexing properly.
Order out-of-bounds check, completion check, and increment in the right way.
2016-07-25 19:28:04 +03:00
Paul Sokolovsky
6af90b2972 py/objstrunicode: str_index_to_ptr: Should handle bytes too.
There's single str_index_to_ptr() function, called for both bytes and
unicode objects, so should handle each properly.
2016-07-25 14:45:08 +03:00
Paul Sokolovsky
16f324641f py/stream.h: Remove dated comment of POSIX-specificity of EAGAIN.
We have adopted POSIX-compatible error numbers as MicroPython's native.
2016-07-25 00:47:00 +03:00
Paul Sokolovsky
1a7e28d8b7 py/stream.h: Move mp_stream_write_adaptor() inside ifdef block. 2016-07-25 00:16:51 +03:00
Paul Sokolovsky
918851e836 unix: Enable MICROPY_PY_STR_BYTES_CMP_WARN.
Also, fix a warning text (remove "duplicate" BytesWarning).
2016-07-22 00:52:07 +03:00
Paul Sokolovsky
a1b442bc07 py/mpconfig.h: Fix description for MICROPY_PY_STR_BYTES_CMP_WARN. 2016-07-22 00:46:24 +03:00
Paul Sokolovsky
707cae7494 py/obj: Issue a warning when str and bytes objects are compared.
Something like:

if foo == "bar":

will be always false if foo is b"bar". In CPython, warning is issued if
interpreter is started as "python3 -b". In MicroPython,
MICROPY_PY_STR_BYTES_CMP_WARN setting controls it.
2016-07-22 00:34:34 +03:00
Dave Hylands
26b7d8a7be py: Fix nlrthumb.c when DEBUG=1 is defined 2016-07-21 00:54:21 +03:00
Paul Sokolovsky
93e353e384 py/gc: Implement GC running by allocation threshold.
Currently, MicroPython runs GC when it could not allocate a block of memory,
which happens when heap is exhausted. However, that policy can't work well
with "inifinity" heaps, e.g. backed by a virtual memory - there will be a
lot of swap thrashing long before VM will be exhausted. Instead, in such
cases "allocation threshold" policy is used: a GC is run after some number of
allocations have been made. Details vary, for example, number or total amount
of allocations can be used, threshold may be self-adjusting based on GC
outcome, etc.

This change implements a simple variant of such policy for MicroPython. Amount
of allocated memory so far is used for threshold, to make it useful to typical
finite-size, and small, heaps as used with MicroPython ports. And such GC policy
is indeed useful for such types of heaps too, as it allows to better control
fragmentation. For example, if a threshold is set to half size of heap, then
for an application which usually makes big number of small allocations, that
will (try to) keep half of heap memory in a nice defragmented state for an
occasional large allocation.

For an application which doesn't exhibit such behavior, there won't be any
visible effects, except for GC running more frequently, which however may
affect performance. To address this, the GC threshold is configurable, and
by default is off so far. It's configured with gc.threshold(amount_in_bytes)
call (can be queries without an argument).
2016-07-21 00:37:30 +03:00
Paul Sokolovsky
6aa7c805cc esp8266: Cache Xtensa-built libaxtls.a in local build dir.
Allows to build the library variant for other archs in parallel.
2016-07-16 04:56:23 +03:00
Paul Sokolovsky
ad9b9c7621 py/stream: Implement 2- and 3-arg write() method as an extension to CPython.
3-arg form:

stream.write(data, offset, length)

2-arg form:

stream.write(data, length)

These allow efficient buffer writing without incurring extra memory
allocation for slicing or creating memoryview() object, what is
important for low-memory ports.

All arguments must be positional. It might be not so bad idea to standardize
on 3-arg form, but 2-arg case would need check and raising an exception
anyway then, so instead it was just made to work.
2016-07-14 01:44:50 +03:00
Paul Sokolovsky
20283aec10 extmod/modussl_axtls: Further changes to allow alternative SSL modules.
Make variable MICROPY_SSL_AXTLS=1 should be defined to activate modussl_axtls
and link with -laxtls.
2016-07-13 01:49:38 +03:00
Paul Sokolovsky
e32d1e17bb extmod/modussl: Rename to modussl_axtls.c, to allow impl using other SSL libs. 2016-07-13 01:35:59 +03:00
Damien George
3593d8e10c py/nlrx64.S: Prefix mp_thread_get_state with an underscore on Mac. 2016-07-11 14:27:47 +01:00
Daniel Tralamazza
9626662819 qemu-arm: Enable gcc LTO option for nlrthumb.c
LTO can't "see" inside naked functions, but we can mark `nlr_push_tail` as used.
2016-07-11 12:04:56 +01:00
Paul Sokolovsky
ec7fe92531 py/objarray: Split out header to allow direct access to object.
This follows source code/header file organization similar to few other
objects, and intended to be used only is special cases, where efficiency/
simplicity matters.
2016-07-06 18:19:27 +03:00
Paul Sokolovsky
426112ce88 extmod/moduos_dupterm: Reserve buffer bytearray object for dupterm.
Allocating it for each read/write operation is a memory fragmentation
hazard.
2016-07-04 13:32:30 +03:00
Paul Sokolovsky
737bd9c314 py/mpconfig.h: Mention MICROPY_PY_BTREE config option.
However, as it requires linking with external libraries, it actually
should be ste on Makefile level.
2016-07-02 14:57:42 +03:00
Paul Sokolovsky
ae184cb9de py/builtinimport: Disable "imported as namespace package" warning.
Namespace packages are natural part of Python3, CPython3 doesn't have such
warning, it made sense only from point of view of Python2 legacy.
2016-07-02 14:45:49 +03:00
Paul Sokolovsky
749cbaca7f py/gc: Calculate (and report) maximum contiguous free block size.
Just as maximum allocated block size, it's reported in allocation units
(not bytes).
2016-07-01 00:09:55 +03:00
Paul Sokolovsky
6a6e0b7e05 py/gc: Be sure to count last allocated block at heap end in stats.
Previously, if there was chain of allocated blocks ending with the last
block of heap, it wasn't included in number of 1/2-block or max block
size stats.
2016-06-30 12:56:21 +03:00
Damien George
77e37ff98b py/mpthread: Include mpstate.h when defining GIL macros. 2016-06-28 11:28:52 +01:00
Damien George
94238d4ae5 py/nlrsetjmp: Update to take into account new location of nlr_top.
It's now accessed via the MP_STATE_THREAD macro.
2016-06-28 11:28:52 +01:00
Damien George
1df4168c33 py/nlrthumb: Convert NLR thumb funcs from asm to C with inline-asm.
Now only the bits that really need to be written in assembler are written
in it, otherwise C is used.  This means that the assembler code no longer
needs to know about the global state structure which makes it much easier
to maintain.
2016-06-28 11:28:52 +01:00
Damien George
df95f52583 py/modthread: Allow to properly set the stack limit of a thread.
We rely on the port setting and adjusting the stack size so there is
enough room to recover from hitting the stack limit.
2016-06-28 11:28:51 +01:00
Damien George
e90b6ce0b5 py/mpthread.h: Move GIL macros outside MICROPY_PY_THREAD block.
The GIL macros are needed even if threading is not enabled.
2016-06-28 11:28:50 +01:00
Damien George
c567afc5fa py/modthread: Make Lock objects work when GIL is enabled. 2016-06-28 11:28:50 +01:00
Damien George
a1c93a62b1 py: Don't use gc or qstr mutex when the GIL is enabled.
There is no need since the GIL already makes gc and qstr operations
atomic.
2016-06-28 11:28:50 +01:00
Damien George
4cec63a9db py: Implement a simple global interpreter lock.
This makes the VM/runtime thread safe, at the cost of not being able to
run code in parallel.
2016-06-28 11:28:50 +01:00
Damien George
1f54ad2aed py: Make interning of qstrs thread safe. 2016-06-28 11:28:50 +01:00
Damien George
3653f5144a py/gc: Fix GC+thread bug where ptr gets lost because it's not computed.
GC_EXIT() can cause a pending thread (waiting on the mutex) to be
scheduled right away.  This other thread may trigger a garbage
collection.  If the pointer to the newly-allocated block (allocated by
the original thread) is not computed before the switch (so it's just left
as a block number) then the block will be wrongly reclaimed.

This patch makes sure the pointer is computed before allowing any thread
switch to occur.
2016-06-28 11:28:49 +01:00
Damien George
9172c0cb25 py/modthread: Call mp_thread_start/mp_thread_finish around threads.
So the underlying thread implementation can do any necessary bookkeeping.
2016-06-28 11:28:49 +01:00
Damien George
722cff5fd0 py/modthread: Be more careful with root pointers when creating a thread. 2016-06-28 11:28:49 +01:00
Damien George
e33806aaff py/gc: Fix 2 cases of concurrent access to ATB and FTB. 2016-06-28 11:28:49 +01:00
Damien George
7f4658a7ee py/modthread: Satisfy unused-args warning. 2016-06-28 11:28:49 +01:00
Damien George
c93d9caa8b py/gc: Make memory manager and garbage collector thread safe.
By using a single, global mutex, all memory-related functions (alloc,
free, realloc, collect, etc) are made thread safe.  This means that only
one thread can be in such a function at any one time.
2016-06-28 11:28:49 +01:00
Damien George
34fc006f5e py/modthread: Add with-context capabilities to lock object. 2016-06-28 11:28:49 +01:00
Damien George
801d1b3803 py/modthread: Implement lock object, for creating a mutex. 2016-06-28 11:28:49 +01:00
Damien George
2dacd604c5 py/modthread: Add exit() function.
Simply raises the SystemExit exception.
2016-06-28 11:28:48 +01:00
Damien George
707f98f207 py/modthread: Add stack_size() function. 2016-06-28 11:28:48 +01:00
Damien George
3eb7a26809 py/modthread: Properly cast concrete exception pointer to an object. 2016-06-28 11:28:48 +01:00
Damien George
27cc07721b py: Add basic _thread module, with ability to start a new thread. 2016-06-28 11:28:48 +01:00
Damien George
330165a2cc py: Add MP_STATE_THREAD to hold state specific to a given thread. 2016-06-28 11:09:31 +01:00
Paul Sokolovsky
c7fba524cb py/objtype: Inherit protocol vtable from base class only if it exists. 2016-06-19 00:56:06 +03:00
Paul Sokolovsky
41167554e4 py/mphal.h: If virtpin API is used, automagically include its header. 2016-06-19 00:12:37 +03:00
Paul Sokolovsky
413c3e10b4 py/objtype: instance: Inherit protocol vtable from a base class.
This allows to define an  abstract base class which would translate
C-level protocol to Python method calls, and any subclass inheriting
from it will support this feature. This in particular actually enables
recently introduced machine.PinBase class.
2016-06-19 00:01:59 +03:00
Paul Sokolovsky
3fecbb2462 extmod/machine_pinbase: Implementation of PinBase class.
Allows to translate C-level pin API to Python-level pin API. In other
words, allows to implement a pin class and Python which will be usable
for efficient C-coded algorithms, like bitbanging SPI/I2C, time_pulse,
etc.
2016-06-18 23:40:28 +03:00
Paul Sokolovsky
07209f8592 all: Rename mp_obj_type_t::stream_p to protocol.
It's now used for more than just stream protocol (e.g. pin protocol), so
don't use false names.
2016-06-18 18:44:57 +03:00
Paul Sokolovsky
f469c76442 py: Rename __QSTR_EXTRACT flag to NO_QSTR.
It has more usages than just qstr extraction, for example, embedding (where
people don't care about efficient predefined qstrs).
2016-06-16 01:42:48 +03:00
Paul Sokolovsky
230d5cda05 py/mkrules.mk: Define "lib" outside conditional block.
"lib" happened to be defined inside block conditional on $(PROG).
2016-06-16 01:11:24 +03:00
Paul Sokolovsky
dcb904416a py/makeqstrdefs.py: Remove restriction that source path can't be absolute.
That's arbitrary restriction, in case of embedding, a source file path may
be absolute. For the purpose of filtering out system includes, checking
for ".c" suffix is enough.
2016-06-16 01:04:42 +03:00
Paul Sokolovsky
0f5bf1aafe py/mpconfig.h: MP_NOINLINE is universally useful, move from unix port. 2016-06-15 23:52:00 +03:00
Paul Sokolovsky
337111ba9e py: Support to build berkeley db 1.85 and "btree" module. 2016-06-15 00:52:45 +03:00
Mark Anthony Palomer
3131053e1a py/objdict: Implemented OrderedDict equality check. 2016-06-12 17:33:48 +03:00
Damien George
b1533c4366 py/parse: Treat constants that start with underscore as private.
Assignments of the form "_id = const(value)" are treated as private
(following a similar CPython convention) and code is no longer emitted
for the assignment to a global variable.

See issue #2111.
2016-06-06 17:28:32 +01:00
Damien George
33168081f4 extmod/machine: Add MICROPY_PY_MACHINE_PULSE config for time_pulse_us.
Since not all ports that enable the machine module have the pin HAL
functions.
2016-05-31 14:25:19 +01:00
Damien George
4940bee62a extmod: Add machine time_pulse_us function (at C and Python level).
The C implementation is taken from the DHT driver.
2016-05-31 13:58:48 +01:00
Damien George
715ee9d925 py/modstruct: Allow to have "0s" in struct format. 2016-05-28 23:27:38 +01:00
Paul Sokolovsky
2ec0ee082a py/moduerrno: Add ECONNREFUSED, one of frequent networking errors. 2016-05-28 00:17:19 +03:00
Paul Sokolovsky
751e3b7a82 extmod/virtpin: Initial implementation of open-ended C-level Pin interface.
Using usual method of virtual method tables. Single virtual method,
ioctl, is defined currently for all operations. This universal and
extensible vtable-based method is also defined as a default MPHAL
GPIO implementation, but a specific port may override it with its
own implementation (e.g. close-ended, but very efficient, e.g. avoiding
virtual method dispatch).
2016-05-27 01:08:43 +03:00
Paul Sokolovsky
c76acd8c23 py/moduerrno: Add EEXIST, EISDIR.
Useful to check file/dir operations result, in particular used by upip.
2016-05-25 01:23:57 +03:00
Antonin ENFRUN
ca41dc2750 py/objnamedtuple: Allow passing field names as a tuple.
So the documentation's example works.  Besides, a tuple can be more
memory efficient.
2016-05-23 21:08:07 +01:00
Damien George
202d5acd06 py/makeqstrdata.py: Allow to have double-quote characters in qstrs.
When rendering the qstr for a C header file, the double-quate character
must be escaped.
2016-05-23 15:18:55 +01:00
Damien George
274952a117 py: Allow to stat and import frozen mpy files using new frozen "VFS".
Freezing mpy files using mpy-tool.py now works again.
2016-05-23 12:42:23 +01:00
Paul Sokolovsky
9dde6062cc py/objstr: Fix mix-signed comparison in str.center(). 2016-05-22 02:22:14 +03:00