Commit Graph

2639 Commits

Author SHA1 Message Date
Scott Shawcroft
a8fbad5d8b Add heap analysis scripts based on GDB breakpoint logs. 2016-11-23 17:31:53 -08:00
Scott Shawcroft
ccbb5e84f9 This introduces an alternative hardware API called nativeio structured around different functions that are typically accelerated by native hardware. Its not meant to reflect the structure of the hardware.
Docs are here: http://tannewt-micropython.readthedocs.io/en/microcontroller/

It differs from upstream's machine in the following ways:

* Python API is identical across ports due to code structure. (Lives in shared-bindings)
* Focuses on abstracting common functionality (AnalogIn) and not representing structure (ADC).
* Documentation lives with code making it easy to ensure they match.
* Pin is split into references (board.D13 and microcontroller.pin.PA17) and functionality (DigitalInOut).
* All nativeio classes claim underlying hardware resources when inited on construction, support Context Managers (aka with statements) and have deinit methods which release the claimed hardware.
* All constructors take pin references rather than peripheral ids. Its up to the implementation to find hardware or throw and exception.
2016-11-21 14:11:52 -08:00
Scott Shawcroft
9321828158 py: Add ability to manually mark blocks during collect. 2016-11-21 14:11:49 -08:00
Scott Shawcroft
f03a9ac505 py: Add support for properties on native objects when they are in the locals dict. 2016-11-18 16:18:14 -08:00
Scott Shawcroft
3cfa5b993d py: Change frozen build to not use -printf which isn't available on Mac OSX. 2016-11-18 16:18:12 -08:00
Scott Shawcroft
84edec4fad py: Fix typo in comment. 2016-11-18 16:17:56 -08:00
Scott Shawcroft
bd36873362 Merge tag 'v1.8.6'
ESP8266 port uses SDK 2.0, has more heap, has support for 512k devices

This release brings some code size reductions to the core as well as
more tests and improved coverage which is now at 94.3%.

The time.ticks_diff(a, b) function has changed: the order of the arguments
has been swapped so that it behaves like "a - b", and it can now return a
negative number if "a" came before "b" (modulo the period of the ticks
functions).

For the ESP8266 port the Espressif SDK has been updated to 2.0.0, the
heap has been increased from 28k to 36k, and there is support for 512k
devices via "make 512k".  upip is included by default as frozen bytecode.
The network module now allows access-point reconnection without WiFi
credentials, and exposes configuration for the station DHCP hostname.  The
DS18B20 driver now handles negative temperatures, and NeoPixel and APA102
drivers handle 4 bytes-per-pixel LEDs.

For the CC3200 port there is now support for loading of precompiled .mpy
files and threading now works properly with interrupts.

A detailed list of changes follows.

py core:
- py.mk: automatically add frozen.c to source list if FROZEN_DIR is defined
- be more specific with MP_DECLARE_CONST_FUN_OBJ macros
- specialise builtin funcs to use separate type for fixed arg count
- {modbuiltins,obj}: use MP_PYTHON_PRINTER where possible
- modbuiltins: add builtin "slice", pointing to existing slice type
- add "delattr" builtin, conditional on MICROPY_CPYTHON_COMPAT
- sequence: fix reverse slicing of lists
- fix null pointer dereference in mpz.c, fix missing va_end in warning.c
- remove asserts that are always true in emitbc.c
- fix wrong assumption that m_renew will not move if shrinking
- change config default so m_malloc0 uses memset if GC not enabled
- add MICROPY_FLOAT_CONST macro for defining float constants
- move frozen bytecode Makefile rules from ports to common mk files
- strip leading dirs from frozen mpy files, so any path can be used

extmod:
- vfs_fat_file: check fatfs f_sync() and f_close() returns for errors
- vfs_fat_file: make file.close() a no-op if file already closed
- utime_mphal: ticks_diff(): switch arg order, return signed value
- utime_mphal: add MP_THREAD_GIL_EXIT/ENTER warppers for sleep functions
- utime_mphal: implement ticks_add(), add to all maintained ports
- utime_mphal: allow ticks functions period be configurable by a port

lib:
- utils/pyhelp.c: use mp_printf() instead of printf()
- utils/pyexec: add mp_hal_set_interrupt_char() prototype
- libm: move Thumb-specific sqrtf function to separate file

drivers:
- add "from micropython import const" when const is used

tools:
- upgrade upip to 1.1.4: fix error on unix when installing to non-existing
  absolute path
- pip-micropython: remove deprecated wrapper tool
- check_code_size.sh: code size validation script for CI
- replace upip tarball with just source file, to make its inclusion as
  frozen modules in multiple ports less magic

tests:
- extmod/vfs_fat: improve VFS test coverage
- basics/builtin_slice: add test for "slice" builtin name
- basics: add test for builtin "delattr"
- extmod/vfs_fat_fsusermount: improve fsusermount test coverage
- extmod/vfs_fat_oldproto: test old block device protocol
- basics/gc1: garbage collector threshold() coverage
- extmod/uhashlib_sha1: coverage for SHA1 algorithm
- extmod/uhashlib_sha256: rename sha256.py test
- btree1: fix out of memory error running on esp8266
- extmod/ticks_diff: test for new semantics of ticks_diff()
- extmod/framebuf1: test framebuffer pixel clear, and text function

minimal port:
- Makefile: split rule for firmware.bin generation

unix port:
- Makefile: remove references to deprecated pip-micropython
- modtime: use ticks_diff() implementation from extmod/utime_mphal.c
- mphalport.h: add warning of mp_hal_delay_ms() implementation
- modtime: switch ticks/sleep_ms/us() to utime_mphal
- fix symbol references for x86 Mac
- replace upip tarball with just source file

windows port:
- enable utime_mphal following unix, define mp_hal_ticks_*
- fix utime_mphal compilation for msvc
- implement mp_hal_ticks_cpu in terms of QueryPerformanceCounter

qemu-arm port:
- exclude ticks_diff test for qemu-arm port
- exclude extmod/vfs_fat_fileio.py test
- exclude new vfs_fat tests
- enable software floating point support, and float tests

stmhal port:
- modutime: refactor to use extmod's version of ticks_cpu
- refactor pin usage to use mp_hal_pin API
- led: refactor LED to use mp_hal_pin_output() init function
- Makefile: use standard rules for frozen module generation
- modutime: consistently convert to MP_ROM_QSTR/MP_ROM_PTR
- enable SD power save (disable CLK on idle)

cc3200 port:
- use mp_raise_XXX helper functions to reduce code size
- mods/pybspi: allow "write" arg of read/readinto to be positional
- enable loading of precompiled .mpy files
- fix thread mutex's so threading works with interrupts

teensy port:
- update to provide new mp_hal_pin_XXX functions following stmhal

esp8266 port:
- Makefile: use latest esptool.py flash size auto-detection
- esp_init_data: auto-initialize system params with vendor SDK 2.0.0
- esp8266.ld: move help.o to iROM
- esp8266.ld: move modmachine.o to iROM
- esp8266.ld: move main.o to iROM
- add MP_FASTCODE modifier to put a function to iRAM
- main: mark nlr_jump_fail() as MP_FASTCODE
- modules/webrepl: enforce only one concurrent WebREPL connection
- etshal.h: add few more ESP8266 vendor lib prototypes
- modesp: add flash_user_start() function
- add support for building firmware version for 512K modules
- scripts: make neopixel/apa102 handle 4bpp LEDs with common code
- modutime: consistently convert to MP_ROM_QSTR/MP_ROM_PTR
- modnetwork: config(): fix copy-paste error in setting "mac"
- scripts/port_diag: add descriptions for esf_buf types
- modnetwork.c: allows AP reconnection without WiFi credentials
- main: bump heap size to 36K
- etshal.h: add prototypes for SPIRead/SPIWrite/SPIEraseSector
- etshal.h: adjust size of MD5_CTX structure
- modules: fix negative temperature in ds18x20 driver
- rename "machine" module implementation to use contemporary naming
- rework webrepl_setup to run over wired REPL
- espneopixel.c: solve glitching LED issues with cpu at 80MHz
- include upip as a standard frozen bytecode module
- update docs for esptool 1.2.1/SDK 2.0 (--flash_size=detect)
- modnetwork.c: expose configuration for station DHCP hostname

zephyr port:
- implement utime module
- use board/SoC values for startup banner based on Zephyr config
- initial implementation of machine.Pin
- zephyr_getchar: update for recent Zephyr refactor of console hooks
- support time -> utime module "weaklink"
- README: update for the current featureset, add more info
- mpconfigport.h: move less important params to the bottom
- Makefile: allow to adjust heap size from make command line
- Makefile: update comments to the current state of affairs
- Makefile: allow to override Zephyr config from make command line
- Makefile: add minimal port
- Makefile: add -fomit-frame-pointer to reduce code size
- mphalport.h: update for new "unified" kernal API (sleep functions)

docs:
- machine.SPI: bring up to date with Hardware API, make vendor-neutral
- machine.SPI: improve descriptions of xfer methods
- library/builtins: add docs for delattr and slice
- library/network: reword intro paragraph
- library/network: typo fixes, consistent acronym capitalization
- library/index: update TOCs so builtins sorted before modules
- utime: document ticks_cpu() in more detail
- utime: describe new semantics of ticks_diff() (signed ring arithmetics)
- utime: add docs for ticks_add(), improvements for other ticks_*()
- esp8266: update for new WebREPL setup procedure
- */quickref.rst: use new semantics of ticks_diff()
- library/machine.Pin: update Pin docs to align with new HW API

travis:
- integrate tools/check_code_size.sh
- minimal: Use CROSS=1, for binary size check

examples:
- http_server_simplistic: add "not suitable for real use" note
- hwapi: example showing best practices for HW API usage in apps
- hwapi: add hwconfig for DragonBoard 410c
2016-11-14 11:10:07 -08:00
Damien George
fcda6a2a78 py: Strip leading dirs from frozen mpy files, so any path can be used.
With this patch one can now do "make FROZEN_MPY_DIR=../../frozen" to
specify a directory containing scripts to be frozen (as well as absolute
paths).

The compiled .mpy files are now stored in $(BUILD)/frozen_mpy/.
2016-11-08 14:36:06 +11:00
Damien George
bdf33bc136 py: Move frozen bytecode Makefile rules from ports to common mk files.
Now, to use frozen bytecode all a port needs to do is define
FROZEN_MPY_DIR to the directory containing the .py files to freeze, and
define MICROPY_MODULE_FROZEN_MPY and MICROPY_QSTR_EXTRA_POOL.
2016-11-08 14:28:30 +11:00
Damien George
561844f3ba py: Add MICROPY_FLOAT_CONST macro for defining float constants.
All float constants in the core should use this macro to prevent
unnecessary creation of double-precision floats, which makes code less
efficient.
2016-11-03 12:33:01 +11:00
Colin Hogben
828df54bfe py: Change config default so m_malloc0 uses memset if GC not enabled.
With MICROPY_ENABLE_GC set to false the alternate memory manager may not
clear all memory that is allocated, so it must be cleared in m_malloc0.
2016-11-03 10:16:31 +11:00
Colin Hogben
f9b6b37cf6 py: Fix wrong assumption that m_renew will not move if shrinking
In both parse.c and qstr.c, an internal chunking allocator tidies up
by calling m_renew to shrink an allocated chunk to the size used, and
assumes that the chunk will not move.  However, when MICROPY_ENABLE_GC
is false, m_renew calls the system realloc, which does not guarantee
this behaviour.  Environments where realloc may return a different
pointer include:

(1) mbed-os with MBED_HEAP_STATS_ENABLED (which adds a wrapper around
malloc & friends; this is where I was hit by the bug);

(2) valgrind on linux (how I diagnosed it).

The fix is to call m_renew_maybe with allow_move=false.
2016-11-02 23:15:41 +11:00
Jan Pochyla
ffb04a5845 unix: fix symbol references for x86 Mac 2016-11-02 00:42:04 +03:00
Pavol Rusnak
7ffc959c00 py: remove asserts that are always true in emitbc.c 2016-10-31 23:21:22 +03:00
Pavol Rusnak
3679ee9b52 py: fix null pointer dereference in mpz.c, fix missing va_end in warning.c 2016-10-31 23:21:15 +03:00
Fabio Utzig
8908e505ce py/sequence: Fix reverse slicing of lists. 2016-10-30 15:54:19 -02:00
Paul Sokolovsky
76146b3d9a extmod/utime_mphal: Allow ticks functions period be configurable by a port.
Using MICROPY_PY_UTIME_TICKS_PERIOD config var.
2016-10-30 03:02:07 +03:00
Paul Sokolovsky
8a49905a2f py/stream: Typo fix in comment. 2016-10-27 22:13:45 +03:00
Scott Shawcroft
d189a3f3cf atmel-samd: Support auto-reset based on USB write activity.
It will soft-reboot micropython after a burst of writes to the
file system. This means that after you save files on your computer
they will be automatically rerun.

This can be disabled in the build by unsetting AUTORESET_TIMER in
mpconfigboard.h.

Using the REPL will also prevent the soft resets until you reset
with CTRL-D manually.
2016-10-25 18:36:37 -07:00
Paul Sokolovsky
f00ecdb54d extmod/moduos_dupterm: Renamed to uos_dupterm.
As part of file naming clean up (moduos_dupterm doesn't implement a
full module, so should skip "mod" prefix, similar to other files in
extmod/).
2016-10-26 02:08:37 +03:00
Damien George
5076e5c339 py: Add "delattr" builtin, conditional on MICROPY_CPYTHON_COMPAT. 2016-10-24 13:50:03 +11:00
Damien George
bdb0d2d0bc py/modbuiltins: Add builtin "slice", pointing to existing slice type. 2016-10-24 13:35:39 +11:00
Paul Sokolovsky
3730090d8f py/{modbuiltins,obj}: Use MP_PYTHON_PRINTER where possible. 2016-10-22 01:07:07 +03:00
Erik Moqvist
f64e806f50 lib/utils/pyhelp.c: Use mp_printf() instead of printf()
This patch introduces MP_PYTHON_PRINTER for general use.
2016-10-21 18:30:58 +11:00
Damien George
571e6f26db py: Specialise builtin funcs to use separate type for fixed arg count.
Builtin functions with a fixed number of arguments (0, 1, 2 or 3) are
quite common.  Before this patch the wrapper for such a function cost
3 machine words.  After this patch it only takes 2, which can reduce the
code size by quite a bit (and pays off even more, the more functions are
added).  It also makes function dispatch slightly more efficient in CPU
usage, and furthermore reduces stack usage for these cases.  On x86 and
Thumb archs the dispatch functions are now tail-call optimised by the
compiler.

The bare-arm port has its code size increase by 76 bytes, but stmhal drops
by 904 bytes.  Stack usage by these builtin functions is decreased by 48
bytes on Thumb2 archs.
2016-10-21 16:26:01 +11:00
Damien George
4ebdb1f2b2 py: Be more specific with MP_DECLARE_CONST_FUN_OBJ macros.
In order to have more fine-grained control over how builtin functions are
constructed, the MP_DECLARE_CONST_FUN_OBJ macros are made more specific,
with suffix of _0, _1, _2, _3, _VAR, _VAR_BETEEN or _KW.  These names now
match the MP_DEFINE_CONST_FUN_OBJ macros.
2016-10-21 16:26:01 +11:00
Tony DiCola
00a44fa36c Merge remote-tracking branch 'micropython/master'
Conflicts:
	README.md - Kept Adafruit README.md intact.
	py/emitglue.c - Added xtensa architecture as an OR of the define.
	zephyr/README.md - Fixed spelling mistake.
2016-10-20 23:01:13 +00:00
Paul Sokolovsky
b440307b4a py/py.mk: Automatically add frozen.c to source list if FROZEN_DIR is defined.
Now frozen modules generation handled fully by py.mk and available for reuse
by any port.
2016-10-21 01:08:43 +03:00
Damien George
7d0d7215d2 py: Use mp_raise_msg helper function where appropriate.
Saves the following number of bytes of code space: 176 for bare-arm, 352
for minimal, 272 for unix x86-64, 140 for stmhal, 120 for esp8266.
2016-10-17 12:17:37 +11:00
Damien George
2750a7b38e py/objdict: Actually provide the key that failed in KeyError exception.
The failed key is available as exc.args[0], as per CPython.
2016-10-17 12:00:19 +11:00
Damien George
a3edeb9ea5 py/objdict: Fix optimisation for allocating result in fromkeys.
Iterables don't respond to __len__, so call __len__ on the original
argument.
2016-10-17 11:58:57 +11:00
Scott Shawcroft
c69ea1a143 py/makeversionhdr.py: Use any tag to describe the version. 2016-10-14 13:10:16 -07:00
Paul Sokolovsky
a97284423e extmod/utime_mphal: Factor out implementations in terms of mp_hal_* for reuse.
As long as a port implement mp_hal_sleep_ms(), mp_hal_ticks_ms(), etc.
functions, it can just use standard implementations of utime.sleel_ms(),
utime.ticks_ms(), etc. Python-level functions.
2016-10-14 20:14:01 +03:00
Damien George
824f5c5a32 py/vstr: Combine vstr_new_size with vstr_new since they are rarely used.
Now there is just one function to allocate a new vstr, namely vstr_new
(in addition to vstr_init etc).  The caller of this function should know
what initial size to allocate for the buffer, or at least have some policy
or config option, instead of leaving it to a default (as it was before).
2016-10-14 16:46:34 +11:00
Damien George
e93c1ca5da extmod/modujson: Implement ujson.load() to load JSON from a stream.
This refactors ujson.loads(s) to behave as ujson.load(StringIO(s)).

Increase in code size is: 366 bytes for unix x86-64, 180 bytes for
stmhal, 84 bytes for esp8266.
2016-10-13 11:46:14 +11:00
Damien George
af8d791bd0 esp8266: Enable importing of precompiled .mpy files. 2016-10-12 11:03:58 +11:00
Damien George
31101d91ce py/lexer: Remove unnecessary code, and unreachable code.
Setting emit_dent=0 is unnecessary because arriving in that part of the
if-logic will guarantee that emit_dent is already zero.

The block to check indent_top(lex)>0 is unreachable because a newline is
always inserted an the end of the input stream, and hence dedents are
always processed before EOF.
2016-10-12 11:00:17 +11:00
Damien George
deaa57acf3 py/compile: Remove debugging code for compiler dispatch.
It was a relic from the days of developing the compiler and is no longer
needed, and it's impossible to trigger via a test.
2016-10-12 10:20:48 +11:00
Damien George
6dff3df501 py/objint: Use size_t for arguments that measure bytes/sizes. 2016-10-11 13:20:11 +11:00
Damien George
8bb7d958f1 py: Factor duplicated function to calculate size of formatted int. 2016-10-11 13:11:32 +11:00
Damien George
df3e5d2b2f py/mpz: Use assert to verify mpz does not have a fixed digit buffer. 2016-10-11 13:00:56 +11:00
Damien George
48874942f0 py/mpz: In divmod, replace check for rhs!=0 with assert.
The check for division by zero is made by the caller of this function.
2016-10-11 13:00:01 +11:00
Damien George
e49153fb98 py/compile: Remove unreachable code. 2016-10-11 12:29:54 +11:00
Damien George
7dc2345715 py/modmicropython: Add micropython.opt_level([value]) function.
This allows to get/set at runtime the optimisation level of the compiler.
2016-10-11 10:56:22 +11:00
Scott Shawcroft
a52fd670e6 Merge remote-tracking branch 'micropython/master' 2016-10-10 11:42:59 -07:00
Paul Sokolovsky
d22a04d9c4 py/objstringio: Add readinto() method.
Also, drop deprecated (as for MicroPython) readall() method.
2016-10-09 11:56:11 +03:00
Damien George
3a0a771730 py: Add mp_raise_OSError(errno) helper function.
This is an often used code pattern, and its use reduces code size of the
core by about 100 bytes.
2016-10-07 13:31:59 +11:00
Damien George
dffa383b06 py/modstruct: Remove unreachable code, and add comment about CPy diff.
The deleted code is unreachable because calcsize_items guarantees that
num_items corresponds to how many items there are in fmt to unpack.
2016-10-07 12:54:14 +11:00
Damien George
8de270b4fc py/objbool: Make a slight simplification of bool constructor.
Reduces code size for some archs.
2016-09-30 17:02:07 +10:00
Damien George
eca1408f16 py/objbool: Defer bool's unary op implementation to small int.
Similar to how binary op already works.  Common unary operations already
have fast paths for bool so there's no need to have explicit handling of
ops in bool_unary_op, especially since they have the same behaviour as
integers.
2016-09-30 17:02:06 +10:00