Commit Graph

2856 Commits

Author SHA1 Message Date
Paul Sokolovsky
6aa7c805cc esp8266: Cache Xtensa-built libaxtls.a in local build dir.
Allows to build the library variant for other archs in parallel.
2016-07-16 04:56:23 +03:00
Paul Sokolovsky
ad9b9c7621 py/stream: Implement 2- and 3-arg write() method as an extension to CPython.
3-arg form:

stream.write(data, offset, length)

2-arg form:

stream.write(data, length)

These allow efficient buffer writing without incurring extra memory
allocation for slicing or creating memoryview() object, what is
important for low-memory ports.

All arguments must be positional. It might be not so bad idea to standardize
on 3-arg form, but 2-arg case would need check and raising an exception
anyway then, so instead it was just made to work.
2016-07-14 01:44:50 +03:00
Paul Sokolovsky
20283aec10 extmod/modussl_axtls: Further changes to allow alternative SSL modules.
Make variable MICROPY_SSL_AXTLS=1 should be defined to activate modussl_axtls
and link with -laxtls.
2016-07-13 01:49:38 +03:00
Paul Sokolovsky
e32d1e17bb extmod/modussl: Rename to modussl_axtls.c, to allow impl using other SSL libs. 2016-07-13 01:35:59 +03:00
Damien George
3593d8e10c py/nlrx64.S: Prefix mp_thread_get_state with an underscore on Mac. 2016-07-11 14:27:47 +01:00
Daniel Tralamazza
9626662819 qemu-arm: Enable gcc LTO option for nlrthumb.c
LTO can't "see" inside naked functions, but we can mark `nlr_push_tail` as used.
2016-07-11 12:04:56 +01:00
Paul Sokolovsky
ec7fe92531 py/objarray: Split out header to allow direct access to object.
This follows source code/header file organization similar to few other
objects, and intended to be used only is special cases, where efficiency/
simplicity matters.
2016-07-06 18:19:27 +03:00
Paul Sokolovsky
426112ce88 extmod/moduos_dupterm: Reserve buffer bytearray object for dupterm.
Allocating it for each read/write operation is a memory fragmentation
hazard.
2016-07-04 13:32:30 +03:00
Paul Sokolovsky
737bd9c314 py/mpconfig.h: Mention MICROPY_PY_BTREE config option.
However, as it requires linking with external libraries, it actually
should be ste on Makefile level.
2016-07-02 14:57:42 +03:00
Paul Sokolovsky
ae184cb9de py/builtinimport: Disable "imported as namespace package" warning.
Namespace packages are natural part of Python3, CPython3 doesn't have such
warning, it made sense only from point of view of Python2 legacy.
2016-07-02 14:45:49 +03:00
Paul Sokolovsky
749cbaca7f py/gc: Calculate (and report) maximum contiguous free block size.
Just as maximum allocated block size, it's reported in allocation units
(not bytes).
2016-07-01 00:09:55 +03:00
Paul Sokolovsky
6a6e0b7e05 py/gc: Be sure to count last allocated block at heap end in stats.
Previously, if there was chain of allocated blocks ending with the last
block of heap, it wasn't included in number of 1/2-block or max block
size stats.
2016-06-30 12:56:21 +03:00
Damien George
77e37ff98b py/mpthread: Include mpstate.h when defining GIL macros. 2016-06-28 11:28:52 +01:00
Damien George
94238d4ae5 py/nlrsetjmp: Update to take into account new location of nlr_top.
It's now accessed via the MP_STATE_THREAD macro.
2016-06-28 11:28:52 +01:00
Damien George
1df4168c33 py/nlrthumb: Convert NLR thumb funcs from asm to C with inline-asm.
Now only the bits that really need to be written in assembler are written
in it, otherwise C is used.  This means that the assembler code no longer
needs to know about the global state structure which makes it much easier
to maintain.
2016-06-28 11:28:52 +01:00
Damien George
df95f52583 py/modthread: Allow to properly set the stack limit of a thread.
We rely on the port setting and adjusting the stack size so there is
enough room to recover from hitting the stack limit.
2016-06-28 11:28:51 +01:00
Damien George
e90b6ce0b5 py/mpthread.h: Move GIL macros outside MICROPY_PY_THREAD block.
The GIL macros are needed even if threading is not enabled.
2016-06-28 11:28:50 +01:00
Damien George
c567afc5fa py/modthread: Make Lock objects work when GIL is enabled. 2016-06-28 11:28:50 +01:00
Damien George
a1c93a62b1 py: Don't use gc or qstr mutex when the GIL is enabled.
There is no need since the GIL already makes gc and qstr operations
atomic.
2016-06-28 11:28:50 +01:00
Damien George
4cec63a9db py: Implement a simple global interpreter lock.
This makes the VM/runtime thread safe, at the cost of not being able to
run code in parallel.
2016-06-28 11:28:50 +01:00
Damien George
1f54ad2aed py: Make interning of qstrs thread safe. 2016-06-28 11:28:50 +01:00
Damien George
3653f5144a py/gc: Fix GC+thread bug where ptr gets lost because it's not computed.
GC_EXIT() can cause a pending thread (waiting on the mutex) to be
scheduled right away.  This other thread may trigger a garbage
collection.  If the pointer to the newly-allocated block (allocated by
the original thread) is not computed before the switch (so it's just left
as a block number) then the block will be wrongly reclaimed.

This patch makes sure the pointer is computed before allowing any thread
switch to occur.
2016-06-28 11:28:49 +01:00
Damien George
9172c0cb25 py/modthread: Call mp_thread_start/mp_thread_finish around threads.
So the underlying thread implementation can do any necessary bookkeeping.
2016-06-28 11:28:49 +01:00
Damien George
722cff5fd0 py/modthread: Be more careful with root pointers when creating a thread. 2016-06-28 11:28:49 +01:00
Damien George
e33806aaff py/gc: Fix 2 cases of concurrent access to ATB and FTB. 2016-06-28 11:28:49 +01:00
Damien George
7f4658a7ee py/modthread: Satisfy unused-args warning. 2016-06-28 11:28:49 +01:00
Damien George
c93d9caa8b py/gc: Make memory manager and garbage collector thread safe.
By using a single, global mutex, all memory-related functions (alloc,
free, realloc, collect, etc) are made thread safe.  This means that only
one thread can be in such a function at any one time.
2016-06-28 11:28:49 +01:00
Damien George
34fc006f5e py/modthread: Add with-context capabilities to lock object. 2016-06-28 11:28:49 +01:00
Damien George
801d1b3803 py/modthread: Implement lock object, for creating a mutex. 2016-06-28 11:28:49 +01:00
Damien George
2dacd604c5 py/modthread: Add exit() function.
Simply raises the SystemExit exception.
2016-06-28 11:28:48 +01:00
Damien George
707f98f207 py/modthread: Add stack_size() function. 2016-06-28 11:28:48 +01:00
Damien George
3eb7a26809 py/modthread: Properly cast concrete exception pointer to an object. 2016-06-28 11:28:48 +01:00
Damien George
27cc07721b py: Add basic _thread module, with ability to start a new thread. 2016-06-28 11:28:48 +01:00
Damien George
330165a2cc py: Add MP_STATE_THREAD to hold state specific to a given thread. 2016-06-28 11:09:31 +01:00
Paul Sokolovsky
c7fba524cb py/objtype: Inherit protocol vtable from base class only if it exists. 2016-06-19 00:56:06 +03:00
Paul Sokolovsky
41167554e4 py/mphal.h: If virtpin API is used, automagically include its header. 2016-06-19 00:12:37 +03:00
Paul Sokolovsky
413c3e10b4 py/objtype: instance: Inherit protocol vtable from a base class.
This allows to define an  abstract base class which would translate
C-level protocol to Python method calls, and any subclass inheriting
from it will support this feature. This in particular actually enables
recently introduced machine.PinBase class.
2016-06-19 00:01:59 +03:00
Paul Sokolovsky
3fecbb2462 extmod/machine_pinbase: Implementation of PinBase class.
Allows to translate C-level pin API to Python-level pin API. In other
words, allows to implement a pin class and Python which will be usable
for efficient C-coded algorithms, like bitbanging SPI/I2C, time_pulse,
etc.
2016-06-18 23:40:28 +03:00
Paul Sokolovsky
07209f8592 all: Rename mp_obj_type_t::stream_p to protocol.
It's now used for more than just stream protocol (e.g. pin protocol), so
don't use false names.
2016-06-18 18:44:57 +03:00
Paul Sokolovsky
f469c76442 py: Rename __QSTR_EXTRACT flag to NO_QSTR.
It has more usages than just qstr extraction, for example, embedding (where
people don't care about efficient predefined qstrs).
2016-06-16 01:42:48 +03:00
Paul Sokolovsky
230d5cda05 py/mkrules.mk: Define "lib" outside conditional block.
"lib" happened to be defined inside block conditional on $(PROG).
2016-06-16 01:11:24 +03:00
Paul Sokolovsky
dcb904416a py/makeqstrdefs.py: Remove restriction that source path can't be absolute.
That's arbitrary restriction, in case of embedding, a source file path may
be absolute. For the purpose of filtering out system includes, checking
for ".c" suffix is enough.
2016-06-16 01:04:42 +03:00
Paul Sokolovsky
0f5bf1aafe py/mpconfig.h: MP_NOINLINE is universally useful, move from unix port. 2016-06-15 23:52:00 +03:00
Paul Sokolovsky
337111ba9e py: Support to build berkeley db 1.85 and "btree" module. 2016-06-15 00:52:45 +03:00
Mark Anthony Palomer
3131053e1a py/objdict: Implemented OrderedDict equality check. 2016-06-12 17:33:48 +03:00
Damien George
b1533c4366 py/parse: Treat constants that start with underscore as private.
Assignments of the form "_id = const(value)" are treated as private
(following a similar CPython convention) and code is no longer emitted
for the assignment to a global variable.

See issue #2111.
2016-06-06 17:28:32 +01:00
Damien George
33168081f4 extmod/machine: Add MICROPY_PY_MACHINE_PULSE config for time_pulse_us.
Since not all ports that enable the machine module have the pin HAL
functions.
2016-05-31 14:25:19 +01:00
Damien George
4940bee62a extmod: Add machine time_pulse_us function (at C and Python level).
The C implementation is taken from the DHT driver.
2016-05-31 13:58:48 +01:00
Damien George
715ee9d925 py/modstruct: Allow to have "0s" in struct format. 2016-05-28 23:27:38 +01:00
Paul Sokolovsky
2ec0ee082a py/moduerrno: Add ECONNREFUSED, one of frequent networking errors. 2016-05-28 00:17:19 +03:00
Paul Sokolovsky
751e3b7a82 extmod/virtpin: Initial implementation of open-ended C-level Pin interface.
Using usual method of virtual method tables. Single virtual method,
ioctl, is defined currently for all operations. This universal and
extensible vtable-based method is also defined as a default MPHAL
GPIO implementation, but a specific port may override it with its
own implementation (e.g. close-ended, but very efficient, e.g. avoiding
virtual method dispatch).
2016-05-27 01:08:43 +03:00
Paul Sokolovsky
c76acd8c23 py/moduerrno: Add EEXIST, EISDIR.
Useful to check file/dir operations result, in particular used by upip.
2016-05-25 01:23:57 +03:00
Antonin ENFRUN
ca41dc2750 py/objnamedtuple: Allow passing field names as a tuple.
So the documentation's example works.  Besides, a tuple can be more
memory efficient.
2016-05-23 21:08:07 +01:00
Damien George
202d5acd06 py/makeqstrdata.py: Allow to have double-quote characters in qstrs.
When rendering the qstr for a C header file, the double-quate character
must be escaped.
2016-05-23 15:18:55 +01:00
Damien George
274952a117 py: Allow to stat and import frozen mpy files using new frozen "VFS".
Freezing mpy files using mpy-tool.py now works again.
2016-05-23 12:42:23 +01:00
Paul Sokolovsky
9dde6062cc py/objstr: Fix mix-signed comparison in str.center(). 2016-05-22 02:22:14 +03:00
Dave Hylands
6a60fb3cf4 py/objstr*: Properly ifdef str.center(). 2016-05-22 01:54:41 +03:00
Paul Sokolovsky
1b5abfcaae py/objstr: Implement str.center().
Disabled by default, enabled in unix port. Need for this method easily
pops up when working with text UI/reporting, and coding workalike
manually again and again counter-productive.
2016-05-22 00:13:44 +03:00
Paul Sokolovsky
2c573f00b8 py/builtinimport: Unbreak bare-arm build. 2016-05-21 22:37:58 +03:00
Paul Sokolovsky
8a2970e136 py/builtinimport: Unbreak minimal build.
These are workarounds required until frozen .mpy loading following standard
frozen modules code path.
2016-05-21 22:23:08 +03:00
Paul Sokolovsky
fb742cdc12 py/{builtinimport,frozenmod}: Rework frozen modules support to support packages.
Now frozen modules is treated just as a kind of VFS, and all operations
performed on it correspond to operations on normal filesystem. This allows
to support packages properly, and potentially also data files.

This change also have changes to rework frozen bytecode modules support to
use the same framework, but it's not finished (and actually may not work,
as older adhox handling of any type of frozen modules is removed).
2016-05-21 21:38:50 +03:00
Paul Sokolovsky
5a2a4e9452 py/mphal.h: Provide default prototypes for mp_hal_delay_us/mp_hal_ticks_us.
Similar to existing mp_hal_delay_ms/mp_hal_ticks_ms.
2016-05-21 02:13:50 +03:00
Paul Sokolovsky
497660fcda py/stream: Add mp_stream_close() helper function. 2016-05-20 21:18:49 +03:00
Damien George
3ff16ff52e py: Declare constant data as properly constant.
Otherwise some compilers (eg without optimisation) will put this read-only
data in RAM instead of ROM.
2016-05-20 12:46:20 +01:00
Paul Sokolovsky
7f7c84b10a py/stream: Support both "exact size" and "one underlying call" operations.
Both read and write operations support variants where either a) a single
call is made to the undelying stream implementation and returned buffer
length may be less than requested, or b) calls are repeated until requested
amount of data is collected, shorter amount is returned only in case of
EOF or error.

These operations are available from the level of C support functions to be
used by other C modules to implementations of Python methods to be used in
user-facing objects.

The rationale of these changes is to allow to write concise and robust
code to work with *blocking* streams of types prone to short reads, like
serial interfaces and sockets. Particular object types may select "exact"
vs "once" types of methods depending on their needs. E.g., for sockets,
revc() and send() methods continue to be "once", while read() and write()
thus converted to "exactly" versions.

These changes don't affect non-blocking handling, e.g. trying "exact"
method on the non-blocking socket will return as much data as available
without blocking. No data available is continued to be signaled as None
return value to read() and write().

From the point of view of CPython compatibility, this model is a cross
between its io.RawIOBase and io.BufferedIOBase abstract classes. For
blocking streams, it works as io.BufferedIOBase model (guaranteeing
lack of short reads/writes), while for non-blocking - as io.RawIOBase,
returning None in case of lack of data (instead of raising expensive
exception, as required by io.BufferedIOBase). Such a cross-behavior
should be optimal for MicroPython needs.
2016-05-18 02:41:45 +03:00
Paul Sokolovsky
e53fb1bf03 py/modstruct: Raise ValueError on unsupported format char. 2016-05-14 15:47:08 +03:00
Paul Sokolovsky
2ae6697300 py/objstringio: Add TODO comment about avoiding copying on .getvalue(). 2016-05-14 14:46:13 +03:00
Damien George
cc80c4dd59 py/objstr: Make dedicated splitlines function, supporting diff newlines.
It now supports \n, \r and \r\n as newline separators.

Adds 56 bytes to stmhal and 80 bytes to unix x86-64.

Fixes issue #1689.
2016-05-13 12:21:32 +01:00
Paul Sokolovsky
68a7a92cec py/gc: gc_dump_alloc_table(): Dump heap offset instead of actual address.
Address printed was truncated anyway and in general confusing to outsider.
A line which dumps it is still left in the source, commented, for peculiar
cases when it may be needed (e.g. when running under debugger).
2016-05-13 00:16:38 +03:00
Paul Sokolovsky
9a8751b006 gc: gc_dump_alloc_table(): Use '=' char for tail blocks.
'=' is pretty natural character for tail, and gives less dense picture
where it's easier to see what object types are actually there.
2016-05-13 00:16:38 +03:00
Paul Sokolovsky
10503f3534 py/moduerrno: Add EACCES, pretty common error on Unix. 2016-05-13 00:15:38 +03:00
Damien George
9a92499641 py/objexcept: Don't convert errno to str in constructor, do it in print.
OSError's are now printed like:

    OSError: [Errno 1] EPERM

but only if the string corresponding to the errno is found.
2016-05-12 14:27:52 +01:00
Paul Sokolovsky
a314b842bb py/emitglue: Fix build on AArch64 (ARMv8, etc.) related to loading .mpy files.
Actual loading of .mpy files isn't tested.
2016-05-12 16:00:57 +03:00
Colin Hogben
a896951a9a py/objfloat, py/modmath: Ensure M_PI and M_E defined.
In some compliation enviroments (e.g. mbed online compiler) with
strict standards compliance, <math.h> does not define constants such
as M_PI.  Provide fallback definitions of M_E and M_PI where needed.
2016-05-12 13:28:45 +01:00
Damien George
d45e5f8c35 py: Add mp_errno_to_str() and use it to provide nicer OSError msgs.
If an OSError is raised with an integer argument, and that integer
corresponds to an errno, then the string for the errno is used as the
argument to the exception, instead of the integer.  Only works if
the uerrno module is enabled.
2016-05-12 13:20:40 +01:00
Damien George
47bf6ba61a py/moduerrno: Add more constants to the errno module. 2016-05-12 13:18:48 +01:00
Damien George
c9a7430dbe py/mperrno: Add some more MP_Exxx constants, related to networking. 2016-05-12 12:48:47 +01:00
Paul Sokolovsky
bc04dc277e py/gc: Make (byte)array type dumping conditional on these types being enabled. 2016-05-11 19:21:53 +03:00
Paul Sokolovsky
3d7f3f00e0 py/gc: gc_dump_alloc_table(): Show byte/str and (byte)array objects.
These are typical consumers of large chunks of memory, so it's useful to
see at least their number (how much memory isn't clearly shown, as the data
for these objects is allocated elsewhere).
2016-05-11 19:00:15 +03:00
Damien George
80a8d473f6 py/repl: Fix handling of backslash in quotes when checking continuation. 2016-05-11 16:05:22 +01:00
Damien George
83a9a723b9 py/mperrno: Add EAFNOSUPPORT definition. 2016-05-10 23:44:59 +01:00
Damien George
e36ff98c80 py/parse: Add uerrno to list of modules to look for constants in. 2016-05-10 23:30:39 +01:00
Damien George
596a3feb8f py: Add uerrno module, with errno constants and dict. 2016-05-10 23:30:39 +01:00
Damien George
3f56fd64b8 py: Add mperrno.h file with uPy defined errno constants. 2016-05-10 23:30:39 +01:00
Paul Sokolovsky
6f34e138f1 py/vstr: Change allocation policy, +16 to requested size, instead of *2.
Effect measured on esp8266 port:

Before:
>>> pystone_lowmem.main(10000)
Pystone(1.2) time for 10000 passes = 44214 ms
This machine benchmarks at 226 pystones/second
>>> pystone_lowmem.main(10000)
Pystone(1.2) time for 10000 passes = 44246 ms
This machine benchmarks at 226 pystones/second

After:
>>> pystone_lowmem.main(10000)
Pystone(1.2) time for 10000 passes = 44343ms
This machine benchmarks at 225 pystones/second
>>> pystone_lowmem.main(10000)
Pystone(1.2) time for 10000 passes = 44376ms
This machine benchmarks at 225 pystones/second
2016-05-10 00:56:51 +03:00
Paul Sokolovsky
40f0096ee7 Revert "py/objstr: .format(): Avoid call to vstr_null_terminated_str()."
This reverts commit 6de8dbb488. The change
was incorrect (correct change would require comparing with end pointer in
each if statement in the block).
2016-05-09 23:42:42 +03:00
Paul Sokolovsky
a1f2245a81 py/vstr: vstr_null_terminated_str(): Extend string by at most one byte.
vstr_null_terminated_str is almost certainly a vstr finalization operation,
so it should add the requested NUL byte, and not try to pre-allocate more.
The previous implementation could actually allocate double of the buffer
size.
2016-05-09 22:39:57 +03:00
Paul Sokolovsky
6de8dbb488 py/objstr: .format(): Avoid call to vstr_null_terminated_str().
By comparing with string end pointer instead of checking for NUL byte.
Should alleviate reallocations and fragmentation a tiny bit.
2016-05-09 21:55:09 +03:00
Damien George
460b086333 py/mpz: Fix mpn_div so that it doesn't modify memory of denominator.
Previous to this patch bignum division and modulo would temporarily
modify the RHS argument to the operation (eg x/y would modify y), but on
return the RHS would be restored to its original value.  This is not
allowed because arguments to binary operations are const, and in
particular might live in ROM.  The modification was to normalise the arg
(and then unnormalise before returning), and this patch makes it so the
normalisation is done on the fly and the arg is now accessed as read-only.

This change doesn't increase the order complexity of the operation, and
actually reduces code size.
2016-05-09 17:21:42 +01:00
Damien George
65402ab1ec py/mpz: Do Python style division/modulo within bignum divmod routine.
This patch consolidates the Python logic for division/modulo to one place
within the bignum code.
2016-05-08 22:21:21 +01:00
Damien George
dc3faea040 py/mpz: Fix bug with overflowing C-shift in division routine.
When DIG_SIZE=32, a uint32_t is used to store limbs, and no normalisation
is needed because the MSB is already set, then there will be left and
right shifts (in C) by 32 of a 32-bit variable, leading to undefined
behaviour.  This patch fixes this bug.
2016-05-08 21:38:43 +01:00
Paul Sokolovsky
d59c2e5e45 py/repl: If there're no better alternatives, try to complete "import".
Also do that only for the first word in a line. The idea is that when you
start up interpreter, high chance that you want to do an import. With this
patch, this can be achieved with "i<tab>".
2016-05-08 20:40:47 +03:00
Damien George
470c429ee1 py/runtime: Properly handle passing user mappings to ** keyword args. 2016-05-07 22:02:46 +01:00
Damien George
12dd8df375 py/objstr: Binary type of str/bytes for buffer protocol is 'B'.
The type is an unsigned 8-bit value, since bytes objects are exactly
that.  And it's also sensible for unicode strings to return unsigned
values when accessed in a byte-wise manner (CPython does not allow this).
2016-05-07 21:18:17 +01:00
Damien George
eb54e4d065 py/obj: Add warning note about get_array return value and GC blocks. 2016-05-04 10:19:08 +01:00
Paul Sokolovsky
9549590fc6 py/modcollections: Rename module name have "u" prefix for consistency. 2016-05-02 13:57:46 +03:00
Paul Sokolovsky
ddb9dba2f7 py/modio: Rename module name to "uio" for consistency with other modules. 2016-05-02 13:56:33 +03:00
Paul Sokolovsky
25d0f7d59d extmod/modwebrepl: Module to handle WebREPL protocol.
While just a websocket is enough for handling terminal part of WebREPL,
handling file transfer operations requires demultiplexing and acting
upon, which is encapsulated in _webrepl class provided by this module,
which wraps a websocket object.
2016-04-29 00:52:52 +03:00
Paul Sokolovsky
eff85bb1dc py/vm: "yield from" didn't handle MP_OBJ_STOP_ITERATION optimization.
E.g. crashed when yielding from already stopped generators.
2016-04-28 02:08:43 +03:00
Paul Sokolovsky
51cee4495e py/mkrules.mk: Typo fixes in comments. 2016-04-26 12:39:28 +03:00
Damien George
23df4b08fb py/emitnative: Use MP_OBJ_NEW_SMALL_INT instead of manual bit shifting. 2016-04-26 10:02:32 +01:00
Damien George
2bddfd4922 py/obj.h: When constructing a small-int cast to mp_uint_t for bit-shift.
The C standard says that left-shifting a signed value (on the LHS of the
operator) is undefined.  So we cast to an unsigned integer before the
shift.  gcc does not issue a warning about this, but clang does.
2016-04-26 09:51:37 +01:00
stijn
9264d42e2a py/makeqstrdefs.py: Windows compatibility.
- msvc preprocessor output contains full paths with backslashes so the
  ':' and '\' characters needs to be erased from the paths as well
- use a regex for extraction of filenames from preprocessor output so it
  can handle both gcc and msvc preprocessor output, and spaces in paths
  (also thanks to a PR from @travnicekivo for part of that regex)
- os.rename will fail on windows if the destination file already exists,
  so simply attempt to delete that file first
2016-04-25 22:34:22 +01:00
stijn
b2b771ca02 py/makeqstrdefs.py: Remove unused function/variable/import. 2016-04-25 22:34:20 +01:00
Paul Sokolovsky
bababce6de py/runtime_utils: Fix nanbox build. 2016-04-25 20:03:14 +03:00
Paul Sokolovsky
6d103b6548 py: Move call_function_*_protected() functions to py/ for reuse.
They almost certainly needed by any C code which calls Python callbacks.
2016-04-25 19:31:17 +03:00
Damien George
51dca54cd0 py/mkrules.mk: Remove obsolete rules for auto qstr generation. 2016-04-22 11:36:19 +01:00
Damien George
109990fc32 py/mkenv.mk: Remove -s and -S args from PYTHON variable.
Qstr auto-generation is now much faster so this optimisation for start-up
time is no longer needed.  And passing "-s -S" breaks some things, like
stmhal's "make deploy".
2016-04-21 22:25:35 +01:00
Damien George
fea40ad468 py: Fix bug passing a string as a keyword arg in a dict.
Addresses issue #1998.
2016-04-21 16:51:36 +01:00
Paul Sokolovsky
1b60a6dc4e py: Divide "split" and "cat" phases of qstr extraction for better efficiency.
E.g. for stmhal, accumulated preprocessed output may grow large due to
bloated vendor headers, and then reprocessing tens of megabytes on each
build make take couple of seconds on fast hardware (=> potentially dozens
of seconds on slow hardware). So instead, split once after each change,
and only cat repetitively (guaranteed to be fast, as there're thousands
of lines involved at most).
2016-04-19 14:39:08 +03:00
Paul Sokolovsky
8dd704b019 py/makeqstrdefs.py: Process only CPP line-numbering info.
Not stuff like "#pragma", etc.
2016-04-19 12:52:57 +03:00
Paul Sokolovsky
4494b521ea py/mkrules.mk: Fix Bashism. 2016-04-19 12:32:23 +03:00
Paul Sokolovsky
21ab304c41 py/mkrules.mk: Cleanup command passed to shell. 2016-04-19 12:28:30 +03:00
Paul Sokolovsky
0dc85c9f86 py/mkrules.mk: Try to detect and emulate make -B behavior for qstr extraction.
If make -B is run, the rule is run with $? empty. Extract fron all file in
this case. But this gets fragile, really "make clean" should be used instead
with such build complexity.
2016-04-19 12:18:46 +03:00
Paul Sokolovsky
c618f91e22 py: Rework QSTR extraction to work in simple and obvious way.
When there're C files to be (re)compiled, they're all passed first to
preprocessor. QSTR references are extracted from preprocessed output and
split per original C file. Then all available qstr files (including those
generated previously) are catenated together. Only if the resulting content
has changed, the output file is written (causing almost global rebuild
to pick up potentially renumbered qstr's). Otherwise, it's not updated
to not cause spurious rebuilds. Related make rules are split to minimize
amount of commands executed in the interim case (when some C files were
updated, but no qstrs were changed).
2016-04-19 11:37:56 +03:00
Paul Sokolovsky
50e4fa5e19 py/mkenv.mk: Optimize Python startup type during make process.
By skipping loading site.py, etc.
2016-04-17 16:11:44 +03:00
Damien George
31480fb91b py/frozenmod: Pass the source name of the frozen module to the lexer.
This allows for better error messages, since the name of the file (sans
.py) can now be printed when an exception occurs within a frozen script.
2016-04-17 12:37:00 +01:00
Jan Čapek
53e3770b15 py/mkrules.mk: Suppress line-no output from CPP for qstr auto-gen. 2016-04-16 13:19:35 +01:00
Jan Čapek
000eae121c py/py.mk: Add makefile variable for qstr autogeneration control.
- any architecture may explicitely build with qstring make
  QSTR_AUTOGEN_DISABLE=1 autogeneration disabled and provide its
  own list of qstrings by the standard
  mechanisms (qstrdefsport.h).
2016-04-16 13:19:23 +01:00
Jan Čapek
61b560f63f py/mkrules.mk: Add mpconfig[port].h dependency to qstr generating rule. 2016-04-16 13:18:51 +01:00
Jan Čapek
d76c65f599 py: Add rules for automated extraction of qstrs from sources.
- add template rule that converts a specified source file into a qstring file

- add special rule for generating a central header that contains all
  extracted/autogenerated strings - defined by QSTR_DEFS_COLLECTED
  variable. Each platform appends a list of sources that may contain
  qstrings into a new build variable: SRC_QSTR. Any autogenerated
  prerequisities are should be appened to SRC_QSTR_AUTO_DEPS variable.

- remove most qstrings from py/qstrdefs, keep only qstrings that
  contain special characters - these cannot be easily detected in the
  sources without additional annotations

- remove most manual qstrdefs, use qstrdef autogen for: py, cc3200,
  stmhal, teensy, unix, windows, pic16bit:

   - remove all micropython generic qstrdefs except for the special strings that contain special characters (e.g. /,+,<,> etc.)
   - remove all port specific qstrdefs except for special strings
   - append sources for qstr generation in platform makefiles (SRC_QSTR)
2016-04-16 13:18:09 +01:00
Pavel Moravec
dbbf082786 py/makeqstrdefs: Add script to automate extraction of qstr from sources.
This script will search for patterns of the form Q(...) and generate a
list of them.

The original code by Pavel Moravec has been significantly simplified to
remove the part that searched for C preprocessor directives (eg #if).
This is because all source is now run through CPP before being fed into
this script.
2016-04-16 13:13:52 +01:00
Damien George
00137b8c11 py/map: Change hash-table allocation policy to be less aggressive.
Small hash tables (eg those used in user class instances that only have a
few members) now only use the minimum amount of memory necessary to hold
the key/value pairs.  This can reduce performance for instances that have
many members (because then there are many reallocations/rehashings of the
table), but helps to conserve memory.

See issue #1760.
2016-04-15 16:24:46 +01:00
Paul Sokolovsky
9b0714b24c py: Declare help, input, open builtins in core.
These are *defined* per-port, but why redeclare them again and again.
2016-04-15 00:07:56 +03:00
Damien George
a649d72606 py/makeqstrdata: Add special case to handle \n qstr. 2016-04-14 15:22:36 +01:00
Damien George
2243d68345 py/makeqstrdata: Reinstate Python2 compatibility. 2016-04-14 14:37:04 +01:00
Damien George
49bb04ee64 py/makeqstrdata: Fix rendering of qstrs that have non-printable ASCII.
The qstr data needs to be turned into a proper C string so non-ASCII
chars must be properly escaped according to C rules.
2016-04-14 14:20:25 +01:00
Damien George
0c1de1cdee py: Simplify "and" action within parser by making ident-rules explicit.
Most grammar rules can optimise to the identity if they only have a single
argument, saving a lot of RAM building the parse tree.  Previous to this
patch, whether a given grammar rule could be optimised was defined (mostly
implicitly) by a complicated set of logic rules.  With this patch the
definition is always specified explicitly by using "and_ident" in the rule
definition in the grammar.  This simplifies the logic of the parser,
making it a bit smaller and faster.  RAM usage in unaffected.
2016-04-14 13:49:23 +01:00
Damien George
f30b6f0af5 py/makeqstrdata: Add more names for escaped chars and esc non-printable.
Non-printable characters are escaped as 0xXX, where XX are the hex
digits of the character value.
2016-04-13 22:12:39 +01:00
Damien George
0a2e9650f5 py: Add ability to have frozen persistent bytecode from .mpy files.
The config variable MICROPY_MODULE_FROZEN is now made of two separate
parts: MICROPY_MODULE_FROZEN_STR and MICROPY_MODULE_FROZEN_MPY.  This
allows to have none, either or both of frozen strings and frozen mpy
files (aka frozen bytecode).
2016-04-13 16:07:47 +01:00
Damien George
594fa73411 py/makeqstrdata: Factor out some code to functions that can be reused. 2016-04-13 16:05:43 +01:00
Damien George
ed0c11236f py/emitglue: Make mp_raw_code_t* arguments constant pointers. 2016-04-13 16:05:43 +01:00
Damien George
6d24dc23b8 py/emitglue: Move typedef of mp_raw_code_t from .c to .h file.
It's needed by frozen bytecode.
2016-04-13 16:05:43 +01:00
Damien George
eacbd7aeba py: Fix constant folding and inline-asm to work with new async grammar. 2016-04-13 15:26:39 +01:00
pohmelie
81ebba7e02 py: add async/await/async for/async with syntax
They are sugar for marking function as generator, "yield from"
and pep492 python "semantically equivalents" respectively.

@dpgeorge was the original author of this patch, but @pohmelie made
changes to implement `async for` and `async with`.
2016-04-13 15:26:38 +01:00
Paul Sokolovsky
b67d098841 py/modbuiltins: __repl_print__: Add comment about setting "_" special var. 2016-04-13 00:59:41 +03:00
Damien George
eec8a94f04 extmod/machine_i2c: Implement I2C memory reading/writing. 2016-04-12 15:52:17 +01:00
Damien George
d083712224 extmod: Add generic machine.I2C class, with bit-bang I2C.
Should work on any machine that provides the correct pin functions.
2016-04-12 14:06:54 +01:00
Damien George
53ad681ed1 extmod: Add initial framebuf module. 2016-04-12 14:06:53 +01:00
Damien George
358e5d8bad py/stream: Move uPy func obj wrappers to below their respective funcs. 2016-04-10 12:41:28 +01:00
Damien George
657aef66ff py/stream: Simplify arg extraction logic for stream_ioctl.
Saves 16 bytes of code.

Also, use mp_obj_get_int_truncated to allow integers as big as a machine
word to be passed as the value.
2016-04-10 12:37:59 +01:00
Paul Sokolovsky
558fd5d228 py/stream: ioctl(): Properly support 2-arg form. 2016-04-10 13:36:44 +03:00
Paul Sokolovsky
6c3db26ab7 py/stream: Fix signed comparison issue. 2016-04-10 13:31:52 +03:00
Paul Sokolovsky
0c97e4c414 py/stream: Add Python-level ioctl() method.
Will call underlying C virtual methods of stream interface. This isn't
intended to be added to every stream object (it's not in CPython), but
is convenient way to expose extra operation on Python side without
adding bunch of Python-level methods.
2016-04-10 12:45:46 +03:00
Paul Sokolovsky
a45e280c58 py/stream.h: Add bigger inventory of stream ioctl's. 2016-04-10 12:42:41 +03:00
Damien George
04d5e644fc py/objarray: Fix array.append so it doesn't extend if append fails.
Addresses issue #1965.
2016-04-07 09:03:33 +01:00
Damien George
2c915e1ae6 py: Implement basic with support in native emitter. 2016-04-07 08:53:24 +01:00
Damien George
ce8b4e8749 py: Combine continuous block of emit steps into with_cleanup emit call.
Because different emitters need to handle with-cleanup in different ways.
2016-04-07 08:50:38 +01:00
Paul Sokolovsky
e6a4d4e23c py: Move stream-related declarations from obj.h to stream.h. 2016-04-05 22:06:52 +03:00
Paul Sokolovsky
069654f2be py/obj.h: Add comment why mp_fun_kw_t takes non-const mp_map_t*.
mp_fun_kw_t takes mp_map_t* (and not const mp_map_t*) to ease passing
this arg to mp_map_lookup(), which may modify its arg, depending on
flags.
2016-04-04 15:37:19 +03:00
Stephen Kyle
b475327ffa py/map: Prevent map resize failure from destroying map. 2016-04-01 16:36:00 +03:00
Paul Sokolovsky
b1dfdaf6cb py/ringbuf.h: Add reusable ring buffer class.
Features inline get/put operations for the highest performance. Locking
is not part of implementation, operation should be wrapped with locking
externally as needed.
2016-03-30 14:48:31 +03:00
Damien George
2599672384 py/parsenum: Use pow function to apply exponent to decimal number.
Pow is already a dependency when compiling with floats, so may as well
use it here to reduce code size and speed up the conversion for most
cases.
2016-03-29 22:12:07 +01:00
Damien George
e1e7657277 py/formatfloat: Fix further cases of buffer overflow in formatting.
Includes extensive test cases to catch hopefully all cases where
buffer might overflow.
2016-03-29 22:07:15 +01:00
Damien George
03b8bb7ec9 py/formatfloat: Fix case of float format where leading digit was "10".
When taking the logarithm of the float to determine the exponent, there
are some edge cases that finish the log loop too large.  Eg for an
input value of 1e32-epsilon, this is actually less than 1e32 from the
log-loop table and finishes as 10.0e31 when it should be 1.0e32.  It
is thus rendered as :e32 (: comes after 9 in ascii).

There was the same problem with numbers less than 1.
2016-03-29 22:03:13 +01:00
Paul Sokolovsky
53ad5edc01 py/stream: Fix stupid thinko with variable naming/shadowing. 2016-03-27 12:58:33 +03:00
Paul Sokolovsky
8f1854ad2d extmod/modlwip: Add SOL_SOCKET and SO_REUSEADDR constants for setsockopt(). 2016-03-25 15:06:12 +02:00
Paul Sokolovsky
2c81b9be28 py/modio: io.BufferedWriter: Describe flushing policy. 2016-03-25 14:59:30 +02:00
Paul Sokolovsky
063e6e7d0a py/modio: Implement io.BufferedWriter.flush(). 2016-03-25 14:33:38 +02:00
Paul Sokolovsky
5d93dfbc2c py/modio: Initial implementation of io.BufferedWriter class.
Just .write() method implemented currently.
2016-03-25 01:10:49 +02:00
Paul Sokolovsky
4a02a8f74d py/stream: Fix object vs ptr usecase in mp_stream_writeall(). 2016-03-24 19:43:08 +02:00
Paul Sokolovsky
24342dd65e extmod/modwebsocket: Start module for WebSocket helper functions.
Currently, only write support is implemented (of limited buffer size).
2016-03-24 19:16:00 +02:00
Paul Sokolovsky
d4c8e626f2 py/stream: Add mp_stream_writeall() helper function.
Spools entire output buffer to a blocking stream (chunk by chunk if
needed).
2016-03-24 19:09:00 +02:00
Damien George
8d4d6731f5 py/parse: When looking up consts, check they exist before checking type. 2016-03-19 21:36:32 +00:00
Damien George
3acaa28b52 py: Don't allocate an extra parse node for power exponent.
Previous to this patch, the "**b" in "a**b" had its own parse node with
just one item (the "b").  Now, the "b" is just the last element of the
power parse-node.  This saves (a tiny bit of) RAM when compiling.
2016-03-16 13:04:51 +00:00
Paul Sokolovsky
52e062ef33 py/frozenmod: Allow port to override lexer to use for frozen modules. 2016-03-16 17:42:37 +07:00
Damien George
9a58316de2 py/objfun: Allow inline-asm functions to be called with 4 arguments. 2016-03-16 08:22:26 +00:00
Damien George
cea6cf8a5e py/formatfloat: Fix buffer overflow when formatting tiny numbers. 2016-03-15 12:21:56 +00:00
Damien George
0d1f8868b6 py: For mp_buffer_info_t, change len type from mp_uint_t to size_t. 2016-03-15 12:20:57 +00:00
Damien George
77f85db41e py/objarray: Fix array slice assignment when array is reallocated.
Addresses issue #1898.
2016-03-14 23:12:54 +00:00
Damien George
06b398489e py/parsenum: Fix compiler warnings for no decl and signed comparison. 2016-03-14 22:52:52 +00:00
Damien George
0be6359f39 py: When printf'ing an object as a pointer, pass the concrete pointer. 2016-03-14 22:41:14 +00:00
Damien George
2a1cca20b1 py: Fix passing of some wide int types to printf varg format list.
Passing an mp_uint_t to a %d printf format is incorrect for builds where
mp_uint_t is larger than word size (eg a nanboxing build).  This patch
adds some simple casting to int in these cases.
2016-03-14 22:40:39 +00:00
Damien George
e7cd1699df py/argcheck: Use size_t instead of mp_uint_t to count number of args. 2016-03-14 22:35:48 +00:00
Damien George
dddb98db8b py/parsenum: Use size_t to count bytes, and int for type of base arg.
size_t is the proper type to count number of bytes in a string.  The base
argument does not need to be a full mp_uint_t, int is enough.
2016-03-14 22:34:03 +00:00
Damien George
9ae51257bd py: Use MP_SMALL_INT_POSITIVE_MASK to check if uint fits in a small int.
Using the original WORD_MSBIT_HIGH-logic resulted in errors when the
object model is not REPR_A or REPR_C.
2016-03-10 21:52:56 +00:00
Paul Sokolovsky
f1919b7c98 extmod/modlwip: Add dummy .makefile() method. 2016-03-09 09:14:45 +07:00
Damien George
dcdf8f2d14 py/objboundmeth: Allocate arg state on stack if heap alloc fails.
If the heap is locked, or memory allocation fails, then calling a bound
method will still succeed by allocating the argument state on the stack.

The new code also allocates less stack than before if less than 4
arguments are passed.  It's also a tiny bit smaller in code size.

This was done as part of the ESA project.
2016-03-08 15:36:53 +00:00
Paul Sokolovsky
ed593780bf extmod/modlwip: Implement dummy setsockopt(). 2016-03-08 14:24:49 +07:00
Markus Fix
4f0080346b py/emitglue: Get persistent bytecode working on Linux ARM platform. 2016-03-07 15:15:23 +00:00
Paul Sokolovsky
1d5d4f49d9 py/stackctrl: Add mp_stack_set_top() to explicitly set stack top value.
Useful for embedded targets with fixed stack layout.
2016-03-07 14:12:24 +07:00
Paul Sokolovsky
453a2a3d7c extmod/vfs_fat: Add lexer, move from stmhal port for reuse. 2016-03-03 13:25:44 +02:00
Damien George
4f72aa86bf py/qstrdefs: Add mkdir and remove qstrs for user-mountable filesystems. 2016-02-29 10:54:00 +00:00
Paul Sokolovsky
e0821830b0 extmod/vfs_fat: Add .rename() method. 2016-02-29 01:23:53 +02:00
Paul Sokolovsky
cd6d189f48 extmod/vfs_fat: Move listdir() method from stmhal for reuse. 2016-02-28 17:17:24 +02:00
Damien George
ea23520403 py: Add MICROPY_DYNAMIC_COMPILER option to config compiler at runtime.
This new compile-time option allows to make the bytecode compiler
configurable at runtime by setting the fields in the mp_dynamic_compiler
structure.  By using this feature, the compiler can generate bytecode
that targets any MicroPython runtime/VM, regardless of the host and
target compile-time settings.

Options so far that fall under this dynamic setting are:
- maximum number of bits that a small int can hold;
- whether caching of lookups is used in the bytecode;
- whether to use unicode strings or not (lexer behaviour differs, and
  therefore generated string constants differ).
2016-02-25 10:05:46 +00:00
Damien George
28adab36c7 py/emitinlinethumb: Use qstrs instead of char* for names of asm ops.
Reduces code size by 112 bytes on Thumb2 arch, and makes assembler faster
because comparison can be a simple equals instead of a string compare.

Not all ops have been converted, only those that were simple to convert
and reduced code size.
2016-02-23 15:20:39 +00:00
Damien George
e9d1a94bf0 py/malloc: Provide a proper malloc-based implementation of realloc_ext. 2016-02-23 13:53:38 +00:00
Damien George
d6c558c0aa py/parse: Use m_renew_maybe to ensure that memory is shrunk in-place.
The chunks of memory that the parser allocates contain parse nodes and
are pointed to from many places, so these chunks cannot be relocated
by the memory manager.  This patch makes it so that when a chunk is
shrunk to fit, it is not relocated.
2016-02-23 13:44:29 +00:00
Damien George
40d8430ee3 py/vm: Add macros to hook into various points in the VM.
These can be used to insert arbitrary checks, polling, etc into the VM.
They are left general because the VM is a highly tuned loop and it should
be up to a given port how that port wants to modify the VM internals.

One common use would be to insert a polling check, but only done after
a certain number of opcodes were executed, so as not to slow down the VM
too much.  For example:

 #define MICROPY_VM_HOOK_COUNT (30)
 #define MICROPY_VM_HOOK_INIT static uint vm_hook_divisor = MICROPY_VM_HOOK_COUNT
 #define MICROPY_VM_HOOK_POLL if (--vm_hook_divisor == 0) { \
     vm_hook_divisor = MICROPY_VM_HOOK_COUNT;
     extern void vm_hook_function(void);
     vm_hook_function();
 }
 #define MICROPY_VM_HOOK_LOOP MICROPY_VM_HOOK_POLL
 #define MICROPY_VM_HOOK_RETURN MICROPY_VM_HOOK_POLL
2016-02-17 09:02:19 +00:00
Alex March
69d9e7d27d py/repl: Check for an identifier char after the keyword.
- As described in the #1850.
- Add cmdline tests.
2016-02-17 08:56:15 +00:00
Paul Sokolovsky
6cee869feb py/qstrdefs.h: qstrs for VfsFat. 2016-02-15 12:49:32 +02:00
Damien George
9598f36a84 py/emitnative: Add check that RHS of viper store is of integral type. 2016-02-15 09:38:02 +00:00
Damien George
94e4bd456f py/asmx64: Support all 16 regs in reg to memory move instructions. 2016-02-15 09:05:50 +00:00
Damien George
46fc7a3d75 py/asmx64: Add helper macro for generating REX_[WRXB] bits from a reg64. 2016-02-15 09:02:13 +00:00
Paul Sokolovsky
46a0ac02c5 extmod/vfs_fat_ffconf: Reusable FatFs module, move from stmhal/ffconf.
TODO: Probably merge into vfs_fat_diskio.
2016-02-15 00:19:27 +02:00
Paul Sokolovsky
6b0c88256b extmod/vfs_fat_file: Reusable FatFs module, move from stmhal/file. 2016-02-15 00:16:46 +02:00
Paul Sokolovsky
8cb78e0e53 extmod/vfs_fat_diskio: Reusable FatFs module, move from stmhal/diskio. 2016-02-15 00:08:37 +02:00
Paul Sokolovsky
72085a669b py/mpstate.h: fs_user_mount is now standard, reusable uPy functionality. 2016-02-15 00:02:03 +02:00
Paul Sokolovsky
8a43a41b3a unix: Enable VfsFat support. 2016-02-14 19:15:22 +02:00
Paul Sokolovsky
dc3eb55e6a py/obj.h: If not float support is enabled, define mp_obj_is_float(o) to false.
We have so many configuration options, that finally having shortcuts like
this is helpful and cuts on number of ifdef's.
2016-02-14 19:12:57 +02:00
Paul Sokolovsky
c38809e26b py/objarray: Implement "in" operator for bytearray. 2016-02-14 18:57:11 +02:00
Damien George
0e0ce47e65 extmod/fsusermount: Expose umount as a public function. 2016-02-10 23:40:10 +00:00
Damien George
c33ad60a67 extmod/fsusermount: Change block protocol to support ioctl method.
The new block protocol is:
- readblocks(self, n, buf)
- writeblocks(self, n, buf)
- ioctl(self, cmd, arg)

The new ioctl method handles the old sync and count methods, as well as
a new "get sector size" method.

The old protocol is still supported, and used if the device doesn't have
the ioctl method.
2016-02-10 08:59:31 +00:00
Damien George
a2e5e4c3d8 py/viper: Allow uint as index to load/store, and give better error msg. 2016-02-09 13:46:49 +00:00
Damien George
3e02b1d19a py/viper: Allow casting of Python integers to viper pointers.
This allows you to pass a number (being an address) to a viper function
that expects a pointer, and also allows casting of integers to pointers
within viper functions.

This was actually the original behaviour, but it regressed due to native
type identifiers being promoted to 4 bits in width.
2016-02-09 13:29:20 +00:00
Damien George
ff1a96ce2c py/mpz: Add commented-out mpz_pow3_inpl function, to compute (x**y)%z.
This function computes (x**y)%z in an efficient way.  For large arguments
this operation is otherwise not computable by doing x**y and then %z.

It's currently not used, but is added in case it's useful one day.
2016-02-03 22:30:49 +00:00
Doug Currie
2e2e15cec2 py/mpz: Complete implementation of mpz_{and,or,xor} for negative args.
For these 3 bitwise operations there are now fast functions for
positive-only arguments, and general functions for arbitrary sign
arguments (the fast functions are the existing implementation).

By default the fast functions are not used (to save space) and instead
the general functions are used for all operations.

Enable MICROPY_OPT_MPZ_BITWISE to use the fast functions for positive
arguments.
2016-02-03 22:13:39 +00:00
Damien George
5f3e005b67 py: Extend native type-sig to use 4 bits, so uint is separate to ptr.
Before this patch, the native types for uint and ptr/ptr8/ptr16/ptr32
all overlapped and it was possible to make a mistake in casting.  Now,
these types are all separate and any coding mistakes will be raised
as runtime errors.
2016-02-02 23:16:05 +00:00
Damien George
086d98cbde py/objstr: Make mp_obj_str_format_helper static. 2016-02-02 16:51:52 +00:00
Damien George
87e07ea943 py/objstr: For str.format, don't allocate on the heap for field name. 2016-02-02 16:26:21 +00:00
pohmelie
e3a29de1dc py/objstr: For str.format, add nested/computed fields support.
Eg: '{:{}}'.format(123, '>20')

@pohmelie was the original author of this patch, but @dpgeorge made
significant changes to reduce code size and improve efficiency.
2016-02-02 16:25:24 +00:00
Damien George
93bb7dffd2 py/vm: Fix popping of exception block in UNWIND_JUMP opcode.
Fixes issue #1812.
2016-02-01 16:07:21 +00:00
Damien George
9e677114e4 py/mpprint: Fix sign extension when printf'ing %u, %x and %X. 2016-02-01 15:08:42 +00:00
Damien George
7417ccfb0d py/formatfloat: Add ability to format doubles with exponents > 99.
For single prec, exponents never get larger than about 37.  For double
prec, exponents can be larger than 99 and need 3 bytes to format.  This
patch makes the number of bytes needed configurable.

Addresses issue #1772.
2016-01-29 11:39:12 +00:00
Paul Sokolovsky
d3b1f0b627 py/runtime: mp_stack_ctrl_init() should be called immediately on startup.
Calling it from mp_init() is too late for some ports (like Unix), and leads
to incomplete stack frame being captured, with following GC issues. So, now
each port should call mp_stack_ctrl_init() on its own, ASAP after startup,
and taking special precautions so it really was called before stack variables
get allocated (because if such variable with a pointer is missed, it may lead
to over-collecting (typical symptom is segfaulting)).
2016-01-29 02:13:42 +02:00
Damien George
7b05b1b225 py/bc: Update opcode format table now that MP_BC_NOT opcode is gone.
MP_BC_NOT was removed and the "not" operation made a proper unary
operator, and the opcode format table needs to be updated to reflect
this change (but actually the change is only cosmetic).
2016-01-28 16:11:41 +00:00
Damien George
8f54c08691 py/inlineasm: Add ability to specify return type of asm_thumb funcs.
Supported return types are: object, bool, int, uint.

For example:

@micropython.asm_thumb
def foo(r0, r1) -> uint:
    add(r0, r0, r1)
2016-01-27 14:27:10 +00:00
Damien George
a53af6c875 extmod/modurandom: Add some extra random functions.
Functions added are:
- randint
- randrange
- choice
- random
- uniform

They are enabled with configuration variable
MICROPY_PY_URANDOM_EXTRA_FUNCS, which is disabled by default.  It is
enabled for unix coverage build and stmhal.
2016-01-26 15:27:00 +00:00
Paul Sokolovsky
ee1656e148 extmod/moduhashlib: Add support for SHA1 (based on axTLS).
SHA1 is used in a number of protocols and algorithm originated 5 years ago
or so, in other words, it's in "wide use", and only newer protocols use
SHA2.

The implementation depends on axTLS enabled. TODO: Make separate config
option specifically for sha1().
2016-01-24 20:55:09 +02:00
Damien George
df4ce930e3 py/modmicropython: Add stack_use, heap_lock and heap_unlock functions.
micropython.stack_use() returns an integer being the number of bytes used
on the stack.

micropython.heap_lock() and heap_unlock() can be used to prevent the
memory manager from allocating anything on the heap.  Calls to these are
allowed to be nested.
2016-01-24 09:12:06 +00:00
Paul Sokolovsky
6c8b3a7fee py/objgetitemiter: Typo fix in comment. 2016-01-23 23:30:06 +02:00
Dave Hylands
a17755ee8b py: Add ustruct.pack_into and unpack_from 2016-01-19 12:25:28 +00:00
Paul Sokolovsky
a58a91eb04 extmod/modurandom: Add "urandom" module.
Seedable and reproducible pseudo-random number generator. Implemented
functions are getrandbits(n) (n <= 32) and seed().

The algorithm used is Yasmarang by Ilya Levin:
http://www.literatecode.com/yasmarang
2016-01-17 12:13:01 +02:00
Damien George
ff133cdb07 py/objproperty: Fix static struct to build with OBJ_REPR_D. 2016-01-15 11:09:20 +00:00
chrysn
f8ba2eca80 builtin property: accept keyword arguments
this allows python code to use property(lambda:..., doc=...) idiom.

named versions for the fget, fset and fdel arguments are left out in the
interest of saving space; they are rarely used and easy to enable when
actually needed.

a test case is included.
2016-01-14 21:15:18 +02:00
Damien George
22d85ec5be py: Use new code pattern for parsing kw args with mp_arg_parse_all.
Makes code easier to read and more maintainable.
2016-01-13 15:47:56 +00:00
Damien George
8bb4931fec py/emitglue: Use mp_obj_is_float instead of MP_OBJ_IS_TYPE. 2016-01-13 15:24:41 +00:00
Antonin ENFRUN
efc971e8f9 py: unary_op enum type fix, and a cast to remove clang warning 2016-01-12 22:06:39 +01:00
Damien George
5b3f0b7f39 py: Change first arg of type.make_new from mp_obj_t to mp_obj_type_t*.
The first argument to the type.make_new method is naturally a uPy type,
and all uses of this argument cast it directly to a pointer to a type
structure.  So it makes sense to just have it a pointer to a type from
the very beginning (and a const pointer at that).  This patch makes
such a change, and removes all unnecessary casting to/from mp_obj_t.
2016-01-11 00:49:27 +00:00
Damien George
4b72b3a133 py: Change type signature of builtin funs that take variable or kw args.
With this patch the n_args parameter is changed type from mp_uint_t to
size_t.
2016-01-11 00:49:27 +00:00
Damien George
a0c97814df py: Change type of .make_new and .call args: mp_uint_t becomes size_t.
This patch changes the type signature of .make_new and .call object method
slots to use size_t for n_args and n_kw (was mp_uint_t.  Makes code more
efficient when mp_uint_t is larger than a machine word.  Doesn't affect
ports when size_t and mp_uint_t have the same size.
2016-01-11 00:48:41 +00:00
Damien George
fe6756aa2d py: Remove long-obsolete mp_method_t typedef. 2016-01-10 00:06:36 +00:00
Damien George
da3dffa79d py/objint: Fix classification of float so it works for OBJ_REPR_D. 2016-01-08 17:57:30 +00:00
Damien George
2adf7ec3dd py/mpz: Fix conversion of float to mpz so it works on big endian archs. 2016-01-08 17:56:58 +00:00
Damien George
978d2e55ef py/runtime: Use appropriate printf fmt for malloc num_bytes. 2016-01-08 13:49:58 +00:00
Damien George
c1c57eafac py/smallint: Allow to override MP_SMALL_INT_MIN et al.
This allows a port to specify exactly how many bits are in a small
int (eg for a uPy bytecode cross compiler).
2016-01-08 13:43:56 +00:00
Damien George
1fa6be5264 py/obj: For OBJ_REPR_D, use uint32_t cast when extracting qstr value. 2016-01-08 13:43:13 +00:00
Damien George
7dbf74c5b9 py/parse: Include unistd.h for ssize_t definition.
In some cases ssize_t is not defined by already included headers.
2016-01-08 13:42:00 +00:00
Damien George
1404d620c0 py/emitglue: Add more feature flags to .mpy persistent bytecode output.
Need to record in .mpy file whether unicode is enabled, and how many bits
are in a small int.
2016-01-08 13:35:35 +00:00
Damien George
dd0a0f79d7 py/viper: Truncate viper integer args so they can be up to 32-bit. 2016-01-07 16:48:20 +00:00
Damien George
ea8be373a9 py/inlinethumb: Remove 30-bit restriction on movwt instruction.
movwt can now move a full 32-bit constant into a register.
2016-01-07 16:34:11 +00:00
Damien George
47dc5922ca py/inlinethumb: Allow assembler to use big ints as args to instructions. 2016-01-07 16:21:07 +00:00
Damien George
22b2265053 py/parse: Improve constant folding to operate on small and big ints.
Constant folding in the parser can now operate on big ints, whatever
their representation.  This is now possible because the parser can create
parse nodes holding arbitrary objects.  For the case of small ints the
folding is still efficient in RAM because the folded small int is stored
inplace in the parse node.

Adds 48 bytes to code size on Thumb2 architecture.  Helps reduce heap
usage because more constants can be computed at compile time, leading to
a smaller parse tree, and most importantly means that the constants don't
have to be computed at runtime (perhaps more than once).  Parser will now
be a little slower when folding due to calls to runtime to do the
arithmetic.
2016-01-07 14:40:35 +00:00
Damien George
d6b31e4578 py: Change mp_obj_int_is_positive to more general mp_obj_int_sign.
This function returns the sign (-1, 0 or 1) of the integer object.
2016-01-07 14:29:12 +00:00
Damien George
93b3726240 py/parse: Optimise away parse node that's just parenthesis around expr.
Before this patch, (x+y)*z would be parsed to a tree that contained a
redundant identity parse node corresponding to the parenthesis.  With
this patch such nodes are optimised away, which reduces memory
requirements for expressions with parenthesis, and simplifies the
compiler because it doesn't need to handle this identity case.

A parenthesis parse node is still needed for tuples.
2016-01-07 13:07:52 +00:00
Damien George
43cab7c283 py/modbuiltins: Fix access of mp_obj_t variable, wrap in MP_OBJ_TO_PTR. 2016-01-04 14:19:33 +00:00
Damien George
d4df8f4925 py/objstr: In str.format, handle case of no format spec for string arg.
Handles, eg, "{:>20}".format("foo"), where there is no explicit spec for
the type of the argument.
2016-01-04 13:13:39 +00:00
stijn
3c014a67ea py: Implement __dict__ for instances.
Note that even though wrapped in MICROPY_CPYTHON_COMPAT, it is not
fully compatible because the modifications to the dictionary do not
propagate to the actual instance members.
2016-01-03 20:51:26 +02:00
Dave Hylands
7281d95aee py: Make dir report instance members 2016-01-03 20:37:18 +02:00
Damien George
8212d97317 py: Use polymorphic iterator type where possible to reduce code size.
Only types whose iterator instances still fit in 4 machine words have
been changed to use the polymorphic iterator.

Reduces Thumb2 arch code size by 264 bytes.
2016-01-03 16:27:55 +00:00
Paul Sokolovsky
17f324b836 py/frozenmod: Store frozen module names together, to quickly scan them. 2016-01-03 18:08:45 +02:00
Damien George
1b0aab621b py: Change struct and macro for builtin fun so they can be type checked. 2016-01-03 11:53:44 +00:00
Damien George
3d2daa2d03 py: Change exception traceback data to use size_t instead of mp_uint_t.
The traceback array stores qstrs and line numbers.  qstrs are typed as
size_t, and line numbers should safely fit in size_t as well.
2016-01-02 22:04:12 +00:00
Paul Sokolovsky
00ee84e1e1 py: Clean up instantiation of dupterm object.
To comply with already established scheme for extmod's.
2016-01-01 14:22:57 +02:00
Paul Sokolovsky
9bbfd5efd4 py/mpconfig: Make configuration of dupterm object reusable. 2016-01-01 13:16:18 +02:00
Paul Sokolovsky
ddea7cb702 extmod/moduos_dupterm: Make uos.dupterm() implementation reusable.
That's just function which sets/gets dup terminal object, and can be
easily reused across ports.
2016-01-01 13:04:32 +02:00
Paul Sokolovsky
0992588811 py/frozenmod: Make frozen module content be 0-terminated.
To allow simple zero-terminated lexers.
2016-01-01 07:51:07 +02:00
Damien George
6dde019d93 py/map: In map lookup, check for fixed map independent of ordered map.
It's possible to have a fixed map that is properly hashed (ie not
simply ordered).
2015-12-31 00:24:33 +00:00
Dave Hylands
0dce9a21ce stmhal: Add struct qstr to block of qstrs needed for MICROPY_PY_STRUCT.
Otherwise build fails if uctypes is disabled.
2015-12-28 00:28:56 +00:00
Paul Sokolovsky
3ea03a1188 py/gc: Improve mark/sweep debug output.
Previously, mark operation weren't logged at all, while it's quite useful
to see cascade of marks in case of over-marking (and in other cases too).
Previously, sweep was logged for each block of object in memory, but that
doesn't make much sense and just lead to longer output, harder to parse
by a human. Instead, log sweep only once per object. This is similar to
other memory manager operations, e.g. an object is allocated, then freed.
Or object is allocated, then marked, otherwise swept (one log entry per
operation, with the same memory address in each case).
2015-12-27 20:40:36 +02:00
Damien George
bbe8d51bd9 py/map: Add fast-path for hashing of map index when it is a qstr.
Map indicies are most commonly a qstr, and adding a fast-path for hashing
of a qstr increases overall performance of the runtime.

On pyboard there is a 4% improvement in the pystone benchmark for a cost
of 20 bytes of code size.  It's about a 2% improvement on unix.
2015-12-26 21:15:47 +00:00
Damien George
78913211a9 py: Be more restrictive binding self when looking up instance attrs.
When looking up and extracting an attribute of an instance, some
attributes must bind self as the first argument to make a working method
call.  Previously to this patch, any attribute that was callable had self
bound as the first argument.  But Python specs require the check to be
more restrictive, and only functions, closures and generators should have
self bound as the first argument

Addresses issue #1675.
2015-12-26 12:41:31 +00:00
Paul Sokolovsky
b4c65c253f extmod/fsusermount: Move module qstrs to global pool from stmhal port. 2015-12-25 19:22:16 +02:00
Damien George
8047340d75 py: Handle case of return within the finally block of try-finally.
Addresses issue #1636.
2015-12-24 12:47:39 +00:00
Paul Sokolovsky
1c9210bc2b unix/unix_mphal: Raise KeyboardInterrupt straight from signal handler.
POSIX doesn't guarantee something like that to work, but it works on any
system with careful signal implementation. Roughly, the requirement is
that signal handler is executed in the context of the process, its main
thread, etc. This is true for Linux. Also tested to work without issues
on MacOSX.
2015-12-23 00:07:00 +02:00
Paul Sokolovsky
d50f649cf8 py/objstr: Applying % (format) operator to bytes should return bytes, not str. 2015-12-20 16:52:11 +02:00
Paul Sokolovsky
ef63ab5724 py/objstr: Make sure that b"%s" % b"foo" uses undecorated bytes value.
I.e. the expected result for above is b"foo", whereas previously we got
b"b'foo'".
2015-12-20 16:51:59 +02:00
stijn
0a4eb4dbf2 py/mpprint: Fix printing of 64bit integers for 64bit windows builds
This makes all tests pass again for 64bit windows builds which would
previously fail for anything printing ranges (builtin_range/unpack1)
because they were printed as range( ld, ld ).

This is done by reusing the mp_vprintf implementation for MICROPY_OBJ_REPR_D
for 64bit windows builds (both msvc and mingw-w64) since the format specifier
used for 64bit integers is also %lld, or %llu for the unsigned version.

Note these specifiers used to be fetched from inttypes.h, which is the
C99 way of working with printf/scanf in a portable way, but mingw-w64
wants to be backwards compatible with older MS C runtimes and uses
the non-portable %I64i instead of %lld in inttypes.h, so remove the use
of said header again in mpconfig.h and define the specifiers manually.
2015-12-19 01:15:58 +00:00
Damien George
fe03e7bcb7 py/obj: Fix float constants for MICROPY_OBJ_REPR_C. 2015-12-18 21:44:01 +00:00
Damien George
acaccb37ec py/gc: When printing info, use %u instead of UINT_FMT for size_t args.
Ideally we'd use %zu for size_t args, but that's unlikely to be supported
by all runtimes, and we would then need to implement it in mp_printf.
So simplest and most portable option is to use %u and cast the argument
to uint(=unsigned int).

Note: reason for the change is that UINT_FMT can be %llu (size suitable
for mp_uint_t) which is wider than size_t and prints incorrect results.
2015-12-18 12:52:45 +00:00
Damien George
dd5353a405 py: Add MICROPY_ENABLE_COMPILER and MICROPY_PY_BUILTINS_EVAL_EXEC opts.
MICROPY_ENABLE_COMPILER can be used to enable/disable the entire compiler,
which is useful when only loading of pre-compiled bytecode is supported.
It is enabled by default.

MICROPY_PY_BUILTINS_EVAL_EXEC controls support of eval and exec builtin
functions.  By default they are only included if MICROPY_ENABLE_COMPILER
is enabled.

Disabling both options saves about 40k of code size on 32-bit x86.
2015-12-18 12:35:44 +00:00
Damien George
ab8012bd80 py/emitglue: Add include of unistd.h for read function. 2015-12-18 12:33:08 +00:00
Damien George
a83124361e py/compile: Simplify compilation of comprehension iterators.
Saves 88 bytes on Thumb2, and 200 bytes on x86-64 archs.
2015-12-18 01:37:55 +00:00
Paul Sokolovsky
2eb844e0b4 ports: Rename "machine" module to "umachine".
To let unix port implement "machine" functionality on Python level, and
keep consistent naming in other ports (baremetal ports will use magic
module "symlinking" to still load it on "import machine").

Fixes #1701.
2015-12-18 02:39:52 +02:00
Damien George
2fe7e6bef0 py/parse: Include stddef.h for definition of size_t. 2015-12-17 22:17:26 +00:00
Damien George
02fa8dff85 py/modgc: Remove obsolete extern declaration. 2015-12-17 13:20:40 +00:00
Damien George
831137b807 py/compile: Use size_t or uintptr_t instead of mp_uint_t. 2015-12-17 13:13:18 +00:00
Damien George
16a6a47a7b py/parse: Replace mp_int_t/mp_uint_t with size_t etc, where appropriate. 2015-12-17 13:06:05 +00:00
Damien George
6e2fb56d40 py/qstr: Change type of qstr from mp_uint_t to size_t.
For builds where mp_uint_t is larger than size_t, it doesn't make
sense to use such a wide type for qstrs.  There can only be as many
qstrs as there is address space on the machine, so size_t is the correct
type to use.

Saves about 3000 bytes of code size when building unix/ port with
MICROPY_OBJ_REPR_D.
2015-12-17 12:45:22 +00:00
Damien George
257848587f py/qstr: Use size_t instead of mp_uint_t when counting allocated bytes. 2015-12-17 12:41:40 +00:00
Damien George
1d899e1783 py/bc: Use size_t instead of mp_uint_t to count size of state and args. 2015-12-17 12:33:42 +00:00
Damien George
7a30e87d2b py: Fix MICROPY_STACKLESS mode to compile with MICROPY_OBJ_REPR_D. 2015-12-17 12:32:41 +00:00
Damien George
ab1e36dcf9 py/mpprint: Implement %llu and %lld format specifiers for mp_printf.
Only enabled for MICROPY_OBJ_REPR_D.
2015-12-17 11:41:10 +00:00
Damien George
d977d268e8 py/gc: Use size_t instead of mp_uint_t to count things related to heap.
size_t is the correct type to use to count things related to the size of
the address space.  Using size_t (instead of mp_uint_t) is important for
the efficiency of ports that configure mp_uint_t to larger than the
machine word size.
2015-12-16 20:09:11 -05:00
Damien George
f7782f8082 py/gc: For finaliser, interpret a pointer into the heap as concrete obj. 2015-12-16 19:45:42 -05:00
Damien George
969e4bbe6a py/gc: Scan GC blocks as an array of pointers, not an array of objects.
The GC should search for pointers within the heap.  This patch makes a
difference when an object is larger than a pointer (eg 64-bit NaN
boxing).
2015-12-16 19:41:37 -05:00
Damien George
3911d5af32 py/modsys: Fix module globals table to use MP_ROM_QSTR. 2015-12-16 19:40:14 -05:00
Paul Sokolovsky
1a1cceaf6f py/objpolyiter: Implement instance-polymorphic iterator type.
This allows to have single itertaor type for various internal iterator
types (save rodata space by not having repeating almost-empty type
structures). It works by looking "iternext" method stored in particular
object instance (should be first object field after "base").
2015-12-14 23:48:12 +02:00
Dave Hylands
f925165942 unix: Move modmachine into unix directory
This leaves behind the common functionality in extmod/machine_mem.c
which can be used by all ports.
2015-12-13 01:21:36 +02:00
Michael Buesch
17298af61e py/modmath: Add domain error checking to sqrt, log, log2, log10.
These functions will raise 'ValueError: math domain error' on invalid
input.
2015-12-12 15:12:54 +00:00
Damien George
29e9db0c58 py: Fix compiler to handle lambdas used as default arguments.
Addresses issue #1709.
2015-12-12 13:42:51 +00:00
Damien George
bdbe8c9ae2 py: Make UNARY_OP_NOT a first-class op, to agree with Py not semantics.
Fixes #1684 and makes "not" match Python semantics.  The code is also
simplified (the separate MP_BC_NOT opcode is removed) and the patch saves
68 bytes for bare-arm/ and 52 bytes for minimal/.

Previously "not x" was implemented as !mp_unary_op(x, MP_UNARY_OP_BOOL),
so any given object only needs to implement MP_UNARY_OP_BOOL (and the VM
had a special opcode to do the ! bit).

With this patch "not x" is implemented as mp_unary_op(x, MP_UNARY_OP_NOT),
but this operation is caught at the start of mp_unary_op and dispatched as
!mp_obj_is_true(x).  mp_obj_is_true has special logic to test for
truthness, and is the correct way to handle the not operation.
2015-12-10 22:19:48 +00:00
Henrik Sölver
e242b1785f py/emitinlinethumb: Add support for MRS instruction.
Only IPSR and BASEPRI special registers supported at the moment, but easy
to extend in the future.
2015-12-10 17:32:54 +00:00
Paul Sokolovsky
645045a799 py/mkrules.mk: Don't pass COPT to linker.
Oftentimes, libc, libm, etc. don't come compiled with CPU compressed code
option (Thumb, MIPS16, etc.), but we may still want to use such compressed
code for MicroPython itself.
2015-12-10 00:49:25 +02:00
Damien George
e84325bd1d py: Add mp_get_stream_raise to factor out check for stream methods. 2015-12-09 18:47:43 +00:00
Damien George
3ff259a262 py: Fix calling of parent classmethod from instance of subclass.
Addresses issue #1697.
2015-12-09 17:30:01 +00:00
Damien George
33ac0fd09f py: Don't try to optimise for+range when args are not simple expressions.
Addresses issue #1693.
2015-12-08 21:05:14 +00:00
Paul Sokolovsky
946f870e3c py/misc.h: Include stdint.h only once (unconditionally at the top). 2015-12-08 02:23:58 +02:00
Paul Sokolovsky
9f001b09a8 py/misc.h: Include stdint.h, as large share of code now depends on it. 2015-12-07 20:08:07 +02:00
pohmelie
c6ee273410 py: Add min/max "default" keyword argument 2015-12-07 18:56:25 +02:00
pohmelie
354e688d8e py: Add MICROPY_PY_BUILTINS_MIN_MAX, disable for minimal ports. 2015-12-07 18:56:25 +02:00
Paul Sokolovsky
2b7236d249 py: Make it easy to build without MICROPY_PY_BUILTINS_COMPLEX.
Automagically skip related modules.
2015-12-07 00:19:24 +02:00
Paul Sokolovsky
f2d532c404 py/modsys: Use MP_ROM_PTR() initializer for sys.modules.
Based on similar usage for sys.argv/sys.path.
2015-12-05 00:27:04 +02:00
Paul Sokolovsky
1a1d11fa32 py/modsys: Implement sys.modules.
This for example will allow people to reload modules which didn't load
successfully (e.g. due to syntax error).
2015-12-05 00:13:29 +02:00
Damien George
7a99639cff py: Fix function calls that have positional and a star-arg-with-iterator.
Addresses issue #1678.
2015-12-03 17:59:49 +00:00
Paul Sokolovsky
b4eccfd02d py/mpconfig: Actually allow to override MICROPY_BYTES_PER_GC_BLOCK. 2015-12-03 01:58:25 +02:00
Paul Sokolovsky
75feece208 py/gc: Make GC block size be configurable. 2015-12-03 01:40:52 +02:00
fabien.lementec
e042f485ed py/mpprint: Printing of doubles is now supported (by uPy own routine). 2015-12-02 14:21:36 +02:00
Damien George
b8cfb0d7b2 py: Add support for 64-bit NaN-boxing object model, on 32-bit machine.
To use, put the following in mpconfigport.h:

    #define MICROPY_OBJ_REPR (MICROPY_OBJ_REPR_D)
    #define MICROPY_FLOAT_IMPL (MICROPY_FLOAT_IMPL_DOUBLE)
    typedef int64_t mp_int_t;
    typedef uint64_t mp_uint_t;
    #define UINT_FMT "%llu"
    #define INT_FMT "%lld"

Currently does not work with native emitter enabled.
2015-11-29 14:25:36 +00:00
Damien George
999cedb90f py: Wrap all obj-ptr conversions in MP_OBJ_TO_PTR/MP_OBJ_FROM_PTR.
This allows the mp_obj_t type to be configured to something other than a
pointer-sized primitive type.

This patch also includes additional changes to allow the code to compile
when sizeof(mp_uint_t) != sizeof(void*), such as using size_t instead of
mp_uint_t, and various casts.
2015-11-29 14:25:35 +00:00
Damien George
cbf7674025 py: Add MP_ROM_* macros and mp_rom_* types and use them. 2015-11-29 14:25:04 +00:00
Damien George
94fe6e523d py/gc: Move away from using mp_uint_t, instead use uintptr_t and size_t.
The GC works with concrete pointers and so the types should reflect this.
2015-11-29 14:25:04 +00:00
Damien George
254cfa6c31 py: Use uintptr_t instead of mp_uint_t in MP_TAGPTR_* macros. 2015-11-29 14:25:04 +00:00
Damien George
9f6976b74e py: Make mp_setup_code_state take concrete pointer for func arg. 2015-11-29 14:25:04 +00:00
Damien George
5d66b427e2 py/emit: Change type of arg of load_const_obj from void* to mp_obj_t. 2015-11-29 14:25:04 +00:00
Damien George
c3f64d9799 py: Change qstr_* functions to use size_t as the type for str len arg. 2015-11-29 14:25:04 +00:00
Damien George
4e7107a572 py: Change mp_print_strn_t func type to use size_t for the str length. 2015-11-29 14:25:04 +00:00
Damien George
0786716c5a py/asmx86: Fix function definition to use int32_t instead of int. 2015-11-27 14:06:53 +00:00
Damien George
88a9103b3c py/binary: Make use of MP_ALIGN. 2015-11-27 12:05:11 +00:00
Paul Sokolovsky
f32020ef3d py/mpconfig.h: Allow to build without alloca() for ANSI C compliance.
Define MICROPY_NO_ALLOCA=1 and memory will be allocated from heap instead
and freed by garbage collection.
2015-11-25 23:24:51 +02:00
Paul Sokolovsky
f0fbab7ca7 extmod/fsusermount: Make configurable with MICROPY_FSUSERMOUNT. 2015-11-25 13:19:36 +02:00
Paul Sokolovsky
3a1bbcc2ef extmod: Move fsusermount.c from stmhal for cross-port reuse. 2015-11-25 13:19:36 +02:00
stijn
3baf6b5319 windows/py: Support 64bit mingw-w64 builds
- add mp_int_t/mp_uint_t typedefs in mpconfigport.h
- fix integer suffixes/formatting in mpconfig.h and mpz.h
- use MICROPY_NLR_SETJMP=1 in Makefile since the current nlrx64.S
  implementation causes segfaults in gc_free()
- update README
2015-11-24 17:34:14 +02:00
stijn
bf1570cb25 nlr: Use single preprocessor symbol to check if building on Windows 2015-11-24 17:34:14 +02:00
Damien George
9a56912ad1 py/compile: Do proper checking of * and ** in function definition.
This patch checks that there is only one *, and that ** is last in the
arg list.
2015-11-23 16:50:42 +00:00
Damien George
0e3f29cc99 py: Check that second argument to hasattr is actually a string.
Fixes issue #1623.
2015-11-23 15:57:00 +00:00
Damien George
44e6e348d6 py/emitglue: Implement persistent saving and loading of const objects. 2015-11-23 11:54:12 +00:00
Damien George
39a8deb95f py/emitglue: Add feature-flag header to .mpy to detect bytecode compat.
Loading .mpy files will now check to make sure that the target VM can
support the bytecode.
2015-11-23 10:58:16 +00:00
Paul Sokolovsky
b3be4710aa py/mpz: Normalize (remove leading zeros) xor operation result. 2015-11-22 22:03:18 +02:00
Paul Sokolovsky
3d6240ba1b py/formatfloat: Handle calculation of integer digit for %f format properly.
%f prints true integer digit, so its calculation should happen before any
exponential scaling.
2015-11-22 20:05:08 +02:00
Paul Sokolovsky
3c4c069802 py/formatfloat: Workaround (fix?) incorrect rounding for %f format. 2015-11-22 18:09:28 +02:00
Paul Sokolovsky
9aaccd4735 py/formatfloat: Convert to fully portable implementation.
This takes previous IEEE-754 single precision float implementation, and
converts it to fully portable parametrizable implementation using C99
functions like signbit(), isnan(), isinf(). As long as those functions
are available (they can be defined in adhoc manner of course), and
compiler can perform standard arithmetic and comparison operations on a
float type, this implementation will work with any underlying float type
(including types whose mantissa is larger than available intergral integer
type).
2015-11-22 17:54:05 +02:00
Paul Sokolovsky
295ea12411 py/emitglue: Host definition of mp_verbose_flag.
This may not seem like the ideal place, but is actually the only place
in py/ where it gets referenced, so is just right.
2015-11-21 16:54:15 +02:00
Paul Sokolovsky
72bd172b30 py/modsys: Consistently use indented #if's. 2015-11-21 16:03:37 +02:00
Damien George
83229d3ffe py: Use MP_OBJ_NULL instead of NULL when appropriate. 2015-11-20 14:09:20 +00:00
Damien George
b5b1f2c527 py/emitglue: Add mp_raw_code_load_mem to load raw-code from memory. 2015-11-20 12:44:20 +00:00
Damien George
f148727b78 py/emitglue: Only compile raw-code fatfs loader when on thumb2 platform.
Here we are assuming that a thumb2 port will have fatfs, which is only
roughly true.  We need a better way of enabling specific raw-code file
readers.
2015-11-20 12:42:26 +00:00
Damien George
d4dba88236 py/compile: Add mp_compile_to_raw_code() to return raw code object.
This can then be passed to mp_raw_code_save_file to save a .mpy file.
2015-11-20 12:30:37 +00:00
Igor Gatis
f5c554dfe3 py: Added Cygwin support to py/nlrx86.S. 2015-11-19 20:23:43 +02:00
Damien George
593faf14c4 py/map: Store key/value in earliest possible slot in hash table.
This change makes the code behave how it was supposed to work when first
written.  The avail_slot variable is set to the first free slot when
looking for a key (which would come from deleting an entry).  So it's
more efficient (for subsequent lookups) to insert a new key into such a
slot, rather than the very last slot that was searched.
2015-11-19 01:27:28 +00:00
Damien George
994ff738c8 py/mpstate: Make mp_pending_exception volatile.
It can change asynchronously.
2015-11-17 14:27:21 +00:00
Damien George
a24eafacc9 py/modmath: Make log2, log10 and hyperbolic funcs be SPECIAL_FUNCTIONS.
Will be included only when MICROPY_PY_MATH_SPECIAL_FUNCTIONS is enabled.

Also covers cmath module (but only log10 is there at the moment).
2015-11-17 14:10:13 +00:00
Damien George
2c83894257 py: Implement default and star args for lambdas. 2015-11-17 14:00:14 +00:00
Damien George
cbd9ae5256 py/compile: Don't unnecessarily save state when compiling param list.
Parameter lists can't be nested so there is no need to save the global
state when compiling them.
2015-11-17 12:37:02 +00:00
Paul Sokolovsky
863d4cd862 py/modmath: Don't create symbol entry for expm1() if not needed. 2015-11-14 00:55:54 +02:00
Paul Sokolovsky
0d6116d86b py/modmath: Make expm1() be in MICROPY_PY_MATH_SPECIAL_FUNCTIONS. 2015-11-14 00:46:52 +02:00
Paul Sokolovsky
6d1eabfeaa unix/mpconfigport: Move log2() definition to modmath.c.
It's safer to define it where it's used, defining it for all source files
may lead to hard to diagnose conflicts in corner cases.
2015-11-13 15:24:48 +02:00
Damien George
432e8275a9 py: Allow to import compiled bytecode files. 2015-11-13 12:49:18 +00:00
Damien George
d8c834c95d py: Add MICROPY_PERSISTENT_CODE_LOAD/SAVE to load/save bytecode.
MICROPY_PERSISTENT_CODE must be enabled, and then enabling
MICROPY_PERSISTENT_CODE_LOAD/SAVE (either or both) will allow loading
and/or saving of code (at the moment just bytecode) from/to a .mpy file.
2015-11-13 12:49:18 +00:00
Damien George
c8e9c0d89a py: Add MICROPY_PERSISTENT_CODE so code can persist beyond the runtime.
Main changes when MICROPY_PERSISTENT_CODE is enabled are:

- qstrs are encoded as 2-byte fixed width in the bytecode
- all pointers are removed from bytecode and put in const_table (this
  includes const objects and raw code pointers)

Ultimately this option will enable persistence for not just bytecode but
also native code.
2015-11-13 12:49:18 +00:00
Damien George
713ea1800d py: Add constant table to bytecode.
Contains just argument names at the moment but makes it easy to add
arbitrary constants.
2015-11-13 12:49:18 +00:00
Damien George
3a3db4dcf0 py: Put all bytecode state (arg count, etc) in bytecode. 2015-11-13 12:49:18 +00:00
Damien George
9b7f583b0c py: Reorganise bytecode layout so it's more structured, easier to edit. 2015-11-13 12:49:18 +00:00
Damien George
1f92ffb5b7 py/emitinlinethumb: Allow to compile with -Wsign-compare. 2015-11-09 14:11:47 +00:00
Damien George
723d598d32 py/asmthumb: Allow to compile with -Wsign-compare and -Wunused-parameter. 2015-11-09 14:11:21 +00:00
Paul Sokolovsky
50f56227c6 py/objint_longlong: Instead of assert, throw OverflowError. 2015-11-09 01:34:56 +02:00
Dave Hylands
7f3c0d1ea8 py: Clear finalizer flag when calling gc_free.
Currently, the only place that clears the bit is in gc_collect.
So if a block with a finalizer is allocated, and subsequently
freed, and then the block is reallocated with no finalizer then
the bit remains set.

This could also be fixed by having gc_alloc clear the bit, but
I'm pretty sure that free is called way less than alloc, so doing
it in free is more efficient.
2015-11-07 14:26:11 +00:00
Damien George
8b8d189bc0 py: Adjust object repr C (30-bit stuffed float) to reduce code size.
This patch adds/subtracts a constant from the 30-bit float representation
so that str/qstr representations are favoured: they now have all the high
bits set to zero.  This makes encoding/decoding qstr strings more
efficient (and they are used more often than floats, which are now
slightly less efficient to encode/decode).

Saves about 300 bytes of code space on Thumb 2 arch.
2015-11-06 23:25:10 +00:00
Damien George
731f359292 all: Add py/mphal.h and use it in all ports.
py/mphal.h contains declarations for generic mp_hal_XXX functions, such
as stdio and delay/ticks, which ports should provide definitions for.  A
port will also provide mphalport.h with further HAL declarations.
2015-10-31 19:14:30 +03:00
Paul Sokolovsky
2b080cf599 py/modstruct: Support repetition counters for all types, not just string.
This makes format specifiers ~ fully compatible with CPython.

Adds 24 bytes for stmhal port (because previosuly we had to catch and report
it's unsupported to user).
2015-10-31 18:47:08 +03:00
adminpete
d6201fc4b7 py: In inline asm, vldr and vstr offsets now in bytes not words.
As per ARM convention.
2015-10-31 10:50:45 +00:00
omtinez
17c649da3d py/makeversionhdr.py: Work with backslashes in paths.
This script may be called by Windows IDEs (e.g. Visual Studio) and be passed
paths with backslashes.
2015-10-30 11:34:53 +03:00
Paul Sokolovsky
e0d7740a22 extmod/modlwip: slip: Use stream protocol and be port-independent.
Based on the original patch by Galen Hazelwood:
https://github.com/micropython/micropython/pull/1517 .
2015-10-27 00:04:59 +03:00
Paul Sokolovsky
0dbd928cee Makefiles: Remove duplicate object files when linking.
Scenario: module1 depends on some common file from lib/, so specifies it
in its SRC_MOD, and the same situation with module2, then common file
from lib/ eventually ends up listed twice in $(OBJ), which leads to link
errors.

Make is equipped to deal with such situation easily, quoting the manual:
"The value of $^ omits duplicate prerequisites, while $+ retains them and
preserves their order." So, just use $^ consistently in all link targets.
2015-10-24 15:46:53 +03:00
Damien George
22521ea9e2 py/nlrthumb: Make compatible with Cortex-M0 (ARMv6M instr set). 2015-10-20 13:26:34 +01:00
Damien George
04353cc85e py: With obj repr "C", change raw str accessor from macro to function.
This saves around 1000 bytes (Thumb2 arch) because in repr "C" it is
costly to check and extract a qstr.  So making such check/extract a
function instead of a macro saves lots of code space.
2015-10-20 12:38:54 +01:00
Damien George
183edefddd py: Add object repr "C", where 30-bit floats are stuffed in obj word.
This new object representation puts floats into the object word instead
of on the heap, at the expense of reducing their precision to 30 bits.
It only makes sense when the word size is 32-bits.
2015-10-20 12:38:54 +01:00
Damien George
aedb859177 py: Make float representation configurable with object representation. 2015-10-20 12:35:40 +01:00
Damien George
7e359c648b py: Move float e/pi consts to objfloat and make mp_obj_float_t private. 2015-10-20 12:35:17 +01:00
Damien George
aaef1851a7 py: Add mp_obj_is_float function (macro) and use it where appropriate. 2015-10-20 12:35:17 +01:00
Paul Sokolovsky
e0f5df579b all: Make netutils.h available to all ports by default.
Generally, ports should inherit INC from py.mk, append to it, not
overwrite it. TODO: Likely should do the same for other vars too.
2015-10-19 18:32:42 +03:00
Paul Sokolovsky
8ee153f234 unix/modtime: Implement ticks_ms(), ticks_us() and ticks_diff().
All of these functions return positive small int, thus range is 2 bits less
than word size (30 bit on 32-bit systems, 62 bit on 64-bit systems).
2015-10-19 17:48:27 +03:00
Damien George
096d1e4512 py: Add lsl/lsr/asr opcode support to inline Thumb2 assembler. 2015-10-19 14:26:19 +01:00
Paul Sokolovsky
7799410950 py/stream: Allow to reuse is_nonblocking_error(). 2015-10-18 15:39:33 +03:00
Damien George
035a0a2b6e py: Add support for _ in REPL to hold last computed value.
Only available when MICROPY_CAN_OVERRIDE_BUILTINS is enabled.
2015-10-17 12:55:25 +01:00
Damien George
e813541e3f py: Add option for inline assembler to support ARMv7-M instructions.
Cortex-M0, M0+ and M1 only have ARMv6-M Thumb/Thumb2 instructions.  M3,
M4 and M7 have a superset of these, named ARMv7-M.  This patch adds a
config option to enable support of the superset of instructions.
2015-10-16 22:08:57 +01:00
Damien George
4bf3f2d3c0 py: Fix with+for+return bug by popping for-iter when unwinding exc stack.
Addresses issue #1182.
2015-10-15 17:48:28 +01:00
Damien George
4300c7dba2 py: Remove dependency on printf/fwrite in mp_plat_print.
See issue #1500.
2015-10-15 00:05:55 +01:00
Damien George
d7e3b36a09 py/compile: Remove unnecessary label in compilation of for statement. 2015-10-14 15:51:12 +01:00
Damien George
fcce1483fa py: Fix build of ARM native emitter due to recent viper changes.
Addresses #1510.
2015-10-14 12:40:54 +01:00
Damien George
59a41e8fcd py/qstr: Fix calc of qstr memory usage, due to new qstr chunk allocation. 2015-10-13 15:52:06 +01:00
Damien George
b8f9ac5411 py: Implement ptr32 load and store in viper emitter. 2015-10-13 00:50:17 +01:00
Damien George
3c9c3687d6 py: Add support to call __init__ from a builtin module on first import. 2015-10-12 13:46:01 +01:00
Paul Sokolovsky
408b74d74c py: Allow to to build MicroPython as a static library.
The whole current port gets slurped into a static lib named
"libmicropython.a". Maybe that's not ideal, but at least something
to start with.
2015-10-12 15:32:06 +03:00
Damien George
fdfcee7b1e py/parse: Make parser error handling cleaner, less spaghetti-like. 2015-10-12 12:59:18 +01:00
Damien George
64f2b213bb py: Move constant folding from compiler to parser.
It makes much more sense to do constant folding in the parser while the
parse tree is being built.  This eliminates the need to create parse
nodes that will just be folded away.  The code is slightly simpler and a
bit smaller as well.

Constant folding now has a configuration option,
MICROPY_COMP_CONST_FOLDING, which is enabled by default.
2015-10-12 12:58:45 +01:00
Paul Sokolovsky
91fc075a33 py/objarray: Allow to create array of void pointers, as extension to CPython.
Using 'P' format specifier (matches struct module). This is another shortcut
for FFI, just as previously introduced "array of objects" ('O').
2015-10-12 10:13:51 +03:00
Paul Sokolovsky
1b586f3a73 py: Rename MP_BOOL() to mp_obj_new_bool() for consistency in naming. 2015-10-11 15:18:15 +03:00
Paul Sokolovsky
53ca6ae1f3 py/makeqstrdata.py: Catch and report case of empty input file.
The usual cause would be that a cross-compiler for a port is not in PATH.
2015-10-11 11:09:57 +03:00
Damien George
366239b8b9 py/parse: Factor logic when creating parse node from and-rule. 2015-10-08 23:13:18 +01:00
Damien George
b948de36fb py: Don't generate unnecessary parse nodes for assignment or kwargs.
This patch eliminates the need for a nested parse node for assignments
and keyword arguments.  It saves a little bit of RAM when parsing.
2015-10-08 14:26:01 +01:00
Damien George
9f5f156b9d py/emitnative: Raise ViperTypeError for unsupported unary ops. 2015-10-08 13:08:59 +01:00
Damien George
7e12a601b8 py/compile: Fix edge case when constant-folding negation of integer.
Also adds tests specifically for testing constant folding.
2015-10-08 13:02:00 +01:00
Paul Sokolovsky
aaa8867d4a modussl: SSL socket wrapper module based on axTLS. 2015-10-06 18:10:39 +03:00
Damien George
0496de26d3 py: Allow to enable inline assembler without native emitter. 2015-10-03 17:07:54 +01:00
Damien George
58e0f4ac50 py: Allocate parse nodes in chunks to reduce fragmentation and RAM use.
With this patch parse nodes are allocated sequentially in chunks.  This
reduces fragmentation of the heap and prevents waste at the end of
individually allocated parse nodes.

Saves roughly 20% of RAM during parse stage.
2015-10-02 00:11:11 +01:00
Damien George
e5635f4ab3 py: Catch all cases of integer (big and small) division by zero. 2015-10-01 22:48:48 +01:00
Damien George
2065373f67 py/mpz: Fix bignum anding of large negative with smaller positive int. 2015-10-01 22:35:06 +01:00
Damien George
2f4e8511cd py/mpz: Force rhs of mpz_shl_inpl/mpz_shr_inpl to be unsigned.
Python semantics are that rhs of shift must be non-negative, so there's
no need to handle negative values in the underlying mpz implementation.
2015-10-01 18:01:37 +01:00