Commit Graph

5588 Commits

Author SHA1 Message Date
Jeff Epler f77e0b8992
fix comment marker 2023-09-22 10:39:50 -05:00
Jeff Epler 60fa26022b
build fixes for persistent code 2023-09-22 10:39:41 -05:00
Jeff Epler 5c0f8f7786
fixes for circuitpython nativeglue 2023-09-22 10:39:28 -05:00
Jeff Epler a285a33076
Restore our defines for exception chaining 2023-09-22 10:39:12 -05:00
Jeff Epler 9104654930
makeqstrdata: ensure _lt and _gt qstrs are sorted early
this fixes a build error because their numbers have to be <256
2023-09-22 10:38:52 -05:00
Dan Halbert 10b95a1998 add translate.h and linker.h includes back to runtime.h; remove linker.h refs; remove top-level lib/cyw43-driver 2023-09-20 22:00:09 -04:00
Jeff Epler a00f2d8940
array extend/append need extern linkage & right names 2023-09-20 11:46:51 -05:00
Jeff Epler 7a386545ca
re-add translation make rules 2023-09-20 11:46:35 -05:00
Jeff Epler a760794449
re-add objtraceback.o 2023-09-20 11:46:22 -05:00
Jeff Epler f83a235c85
re-add mp_obj_exception_get_native 2023-09-20 11:46:14 -05:00
Jeff Epler 76ef77a620
fix exception chaining, preprocessor error 2023-09-20 11:26:57 -05:00
Jeff Epler e068ff7234
fix compile errors, fix mp_raise prototypes 2023-09-20 11:26:41 -05:00
Jeff Epler 1ab884ab0f
fix compile errors 2023-09-20 11:26:26 -05:00
Jeff Epler c87703512a
fix conditional nesting 2023-09-20 11:26:17 -05:00
Jeff Epler 37c8b0df9c
fix argument name 2023-09-20 11:26:03 -05:00
Jeff Epler 6b8ee2ca6f
remove old python style checking for "special accessors"
this appears to be properly handled by other (new?) code, but
tests will tell.
2023-09-20 11:25:58 -05:00
Jeff Epler bb6d530c1d
slots & macro moved to header 2023-09-20 11:25:09 -05:00
Jeff Epler 7806044e5e
string construction changed 2023-09-20 11:25:01 -05:00
Jeff Epler 9cedfc4cbd
implemented as macro now 2023-09-20 11:24:53 -05:00
Jeff Epler ee51968d9d
can't be static, is called from builtins 2023-09-20 11:24:45 -05:00
Jeff Epler 4ed939f26e
fix compile error 2023-09-20 11:24:26 -05:00
Jeff Epler 680b94d463
fix compile erors 2023-09-20 11:24:22 -05:00
Jeff Epler be62395c47
fix indentation 2023-09-20 11:24:15 -05:00
Jeff Epler f9fb567a07
fix CP vs MP differences in exception-throwing functions 2023-09-20 11:24:07 -05:00
Jeff Epler 37a881f4d3
Get the type object for the constructor call 2023-09-20 11:23:49 -05:00
Jeff Epler 082b299a0c
spelling 2023-09-20 11:23:39 -05:00
Jeff Epler 8e00c69cdf
spelling 2023-09-20 11:22:34 -05:00
Jeff Epler cd3ca6b6a6
fix array object name 2023-09-20 11:22:25 -05:00
Jeff Epler fb2eb6a7d9
add mp_obj_is_tuple_compatible 2023-09-20 11:22:01 -05:00
Jeff Epler e4e58ac223
add mp_obj_get_type_qstr 2023-09-20 11:21:56 -05:00
Jeff Epler a9d6661d5b
add and fix exception prototypes 2023-09-20 11:21:50 -05:00
Jeff Epler b499b7fdc2
add missing type objects 2023-09-20 11:21:34 -05:00
Jeff Epler 57c3dfd579
fix missing define of NONSTANDARD_TYPECODES 2023-09-20 11:21:10 -05:00
Jeff Epler 649ea62f57
fix typo 2023-09-20 11:21:02 -05:00
Jeff Epler 73612ac119
Fix availability of MP_ERROR_TEXT in misc.h 2023-09-20 11:20:56 -05:00
Jeff Epler aceb573c88
fix makeversionhdr 2023-09-20 11:20:45 -05:00
Jeff Epler dc277f0aff
fix HEAP_PTR macro 2023-09-20 11:20:39 -05:00
Jeff Epler 36371fe757
fix compiling gc.c 2023-09-20 11:20:29 -05:00
Jeff Epler 0c64275cc6
Add header for PLACE_IN_ITCM 2023-09-20 11:19:57 -05:00
Angus Gratton fa68523968 py/nlrx64: Mark nlr_push() as naked function when possible.
Supported from GCC 8 and up, and Compiler Explorer suggests it works as
expected with Clang since 3.6 (2014).

- Fixes situation where building embedded MicroPython with -O0 and
  MICROPY_NLR_X64 crashes at runtime (due to nlr_push pushing the
  frame pointer register EBP). Closes #12421.

- Allows removing the macOS tweak to undo pushing EBP onto the stack
  in the generated function prelude.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-09-20 11:48:36 +10:00
Dan Halbert 4d175ab41a convert to MP_DEFINED_CONST_OBJ_TYPE() 2023-09-19 21:09:29 -04:00
Dan Halbert 2c0fa0f7dc initial merge from v1.20.0; just satisifying conflicts 2023-09-19 11:10:12 -04:00
Jeff Epler a8b81ce943
Enhance unexpected keyword argument detection
Before, incorrect use when calling a core function would just say
"extra keyword arguments given"; now, it will name the argument:

```python
>>> Synthesizer(bad_kwarg="boo")
TypeError: unexpected keyword argument 'bad_kwarg'
```
2023-09-15 10:39:14 -05:00
Angus Gratton 3e8aed9fcc py/gc: Add "max new split" value in result of gc.mem_free().
Follow-up to 519c24dd48 when MICROPY_GC_SPLIT_HEAP_AUTO is enabled, based
on discussion at
https://github.com/orgs/micropython/discussions/12316#discussioncomment-6858007

gc.mem_free() is always a heuristic, but this makes it a more useful
heuristic for common use cases.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-09-15 12:19:13 +10:00
Jeff Epler 4b41fdb586
Fast(ish) special purpose bitbang spi over i2c
with the i2c bus operating at 400kHz this achieves a 4.8kHz SPI clock
rate which could be worse.

It accepts the same style of init sequence as displayio.

tested by scoping the pins on the espressif lcd dev kit with a dummy init sequence:
```python
dotclockframebuffer.ioexpander_send_init_sequence(
    bus=bus,
    i2c_address=expander_addr,
    gpio_address=1,
    gpio_data_len=1,
    gpio_data=0xff,
    cs_bit=1,
    mosi_bit=3,
    clk_bit=2,
    init_sequence=init_sequence)
```
2023-09-14 14:42:48 -05:00
Damien George 397697aa83 py/persistentcode: Always close reader even if an exception is raised.
Fixes issue #3874.

Signed-off-by: Damien George <damien@micropython.org>
2023-09-14 13:03:31 +10:00
Damien George 5e122b11ea py/parse: Always free lexer even if an exception is raised.
Fixes issue #3843.

Signed-off-by: Damien George <damien@micropython.org>
2023-09-14 13:03:31 +10:00
Damien George c9089e71a1 py/runtime: Add helpers to call a general function on nlr jump callback.
Signed-off-by: Damien George <damien@micropython.org>
2023-09-14 13:03:31 +10:00
Scott Shawcroft 0928a95bb2
Merge pull request #8351 from jepler/dotclockframebuffer
Dotclockframebuffer
2023-09-07 13:21:29 -07:00
IhorNehrutsa 6e79b55dc8 py/malloc: Fix DEBUG_print() args in m_realloc_maybe.
Signed-off-by: Ihor Nehrutsa <IhorNehrutsa@gmail.com>
2023-09-04 10:26:09 +10:00
David Lechner ffb43b2dd3 py/modthread: Return thread id from start_new_thread().
In CPython, `_thread.start_new_thread()` returns an ID that is the same ID
that is returned by `_thread.get_ident()`.  The current MicroPython
implementation of `_thread.start_new_thread()` always returns `None`.

This modifies the required functions to return a value. The native thread
id is returned since this can be used for interop with other functions, for
example, `pthread_kill()` on *nix. `_thread.get_ident()` is also modified
to return the native thread id so that the values match and avoids the need
for a separate `native_id` attribute.

Fixes issue #12153.

Signed-off-by: David Lechner <david@pybricks.com>
2023-09-03 18:49:18 +10:00
Jim Mussared a64f2fdca0 py/dynruntime.h: Implement MP_OBJ_NEW_QSTR.
Because mpy_ld.py doesn't know the target object representation, it emits
instances of `MP_OBJ_NEW_QSTR(MP_QSTR_Foo)` as const string objects, rather
than qstrs. However this doesn't work for map keys (e.g. for a locals dict)
because the map has all_keys_are_qstrs flag is set (and also auto-complete
requires the map keys to be qstrs).

Instead, emit them as regular qstrs, and make a functioning MP_OBJ_NEW_QSTR
function available (via `native_to_obj`, also used for e.g. making
integers).

Remove the code from mpy_ld.py to emit qstrs as constant strings, but leave
behind the scaffold to emit constant objects in case we want to do use this
in the future.

Strictly this should be a .mpy sub-version bump, even though the function
table isn't changing, it does lead to a change in behavior for a new .mpy
running against old MicroPython. `mp_native_to_obj` will incorrectly return
the qstr value directly as an `mp_obj_t`, leading to unexpected results.
But given that it's broken at the moment, it seems unlikely that anyone is
relying on this, so it's not work the other downsides of a sub-version bump
(i.e. breaking pure-Python modules that use @native). The opposite case of
running an old .mpy on new MicroPython is unchanged, and remains broken in
exactly the same way.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-09-01 16:14:22 +10:00
Daniël van de Giessen d1f288c041 py/modstruct: Support pad bytes in struct format.
This adds support for the x format code in struct.pack and struct.unpack.

The primary use case for this is ignoring bytes while unpacking.  When
interfacing with existing systems, it may often happen that you either have
fields in a struct that aren't properly specified or you simply don't care
about them.  Being able to easily skip them is useful.

Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
2023-09-01 14:50:49 +10:00
mcskatkat e0a1480600 py/objstr: Fix `str % {}` edge case.
Eliminate `TypeError` when format string contains no named conversions.
This matches CPython behavior.

Signed-off-by: mcskatkat <mc_skatkat@hotmail.com>
2023-09-01 14:31:57 +10:00
Jeff Epler ed9cacf41d
Add DotClockFramebuffer 2023-08-30 10:30:15 -05:00
Jim Mussared f3eccb154c py/mpconfig: Enable SSL finalizers if finalizers are enabled.
The rp2 port was enabling SSL and had finalizers enabled via the "extra
features" level, but missed explicitly enabling `MICROPY_PY_SSL_FINALISER`
(like esp32, stm32, and mimxrt did).

This commit makes `MICROPY_PY_SSL_FINALISER` default to enabled if
finalizers are enabled, and removes the explicit setting of this for
esp32, stm32, mimxrt (because they all use the "extra features" level).

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-30 12:38:13 +10:00
Jim Mussared b2b5bcce28 py/profile: Remove the requirement to disable MICROPY_COMP_CONST.
The only reason that const had to be disabled was to make the test output
match CPython when const was involved.  Instead, this commit fixes the test
to handle the lines where const is used.

Also:
- remove the special handling for MICROPY_PERSISTENT_CODE_SAVE in
  unix/mpconfigport.h, and make this automatic.
- move the check for MICROPY_PERSISTENT_CODE_SAVE to where it's used (like
  we do for other similar checks) and add a comment explaining it.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-30 10:58:04 +10:00
Jim Mussared bc1af5ab7f cc3200/Makefile: Build firmware.zip.
This allows the cc3200 port to be build with the standard autobuild script
rather than the custom build-cc3200-latest.sh (which is now removed).

This also fixes the path inside the zip file (by using the `-j` flag to
zip).

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-24 15:44:20 +10:00
Jeff Epler 4d8b354c13
Use qstrs to improve compression
any qstr of >= 3 chars long is a candidate for use; the input byte value
(1) is used to indicate a fixed-length qstr number follows.

Before accounting for the code size changes, this saves ~100 bytes
on trinket m0, but it may end up being a wash due to added code.

A cutoff of length>2 is slightly better for ja and ru and worse for others.
This has to do with the relative frequency of latin characters vs non-latin
characters, is my guess.

lang    after   before  change (- = more free space)
en_US   3132    2996    -134
fr      2136    2060    -76
ja      1916    1724    -232
ru      2196    2000    -196
2023-08-23 15:16:53 -05:00
Dan Halbert 88c22d5052 remove last uses of 'u' prefix 2023-08-22 12:57:47 -04:00
Jeff Epler 0cab3359df
ignore some cast-align diagnostics 2023-08-20 14:26:48 -05:00
Jeff Epler 1399d0f25b
guard mp_type_native_gen_wrap declaration with MICROPY_EMIT_NATIVE 2023-08-20 14:26:27 -05:00
Jeff Epler f9e99bd3c3
Fix picow build 2023-08-20 11:30:36 -05:00
Jeff Epler d5d566dc65
Don't assume __ICACHE_PRESENT is defined
This seems to only be defined in cmsis files for M7 family MCUs, so
it's not for e.g., makerdiary_m60_keyboard which enables loading of native
code. Lower MCUs don't ever have icache, so the default is "off".
2023-08-20 10:15:56 -05:00
Jeff Epler d277294a14
Alias m_new_ll to m_new
this saves changing uses of it in ports/ and if we re-add ll later, it'll
make it easier to find the places that needed long-lived allocation.
2023-08-20 10:15:55 -05:00
Jeff Epler 7e38b79e21
Add mp_type_native_coro_wrap 2023-08-20 10:15:55 -05:00
Jeff Epler b0bf36ef3b
Fix building with !MICROPY_PY_ASYNC_AWAIT 2023-08-19 17:17:50 -05:00
Jeff Epler a94301122a
Make natmods work again.
And put back our magic number, because our bytecode format differs
from upstream

drop btree & framebuf natmods, they had additional problems I didn't
want to fix right now.
2023-08-19 16:59:13 -05:00
Dan Halbert bfccb77ec1 asyncio test fixes and asyncio library updates 2023-08-18 13:16:16 -04:00
Jeff Epler 8d5e844b00
Implement traceback length limit 2023-08-15 13:10:31 -05:00
Jeff Epler df1dd6ad3c
make 'import uerrno' work again in unix port
this fixes tests that 'import uerrno'.
2023-08-15 11:50:13 -05:00
Jim Mussared 9573d31071 all: Remove query-variants make target.
This is difficult to implement on cmake-based ports, and having the list
of variants in mpconfigboard.{cmake,mk} duplicates information that's
already in board.json.

This removes the existing query-variants make target from stm32 & rp2
and the definition of BOARD_VARIANTS from the various board files.

Also renames the cmake variable to MICROPY_BOARD_VARIANT to match other
variables such as MICROPY_BOARD. The make variable stays as
BOARD_VARIANT.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-15 17:37:44 +10:00
Jim Mussared ad123ed013 esp32/Makefile: Implement `make submodules` to match other ports.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-15 17:37:44 +10:00
Angus Gratton 519c24dd48 py/gc: Add new MICROPY_GC_SPLIT_HEAP_AUTO "auto grow heap" mode.
When set, the split heap is automatically extended with new areas on
demand, and shrunk if a heap area becomes empty during a GC pass or soft
reset.

To save code size the size allocation for a new heap block (including
metadata) is estimated at 103% of the failed allocation, rather than
working from the more complex algorithm in gc_try_add_heap(). This appears
to work well except in the extreme limit case when almost all RAM is
exhausted (~last few hundred bytes). However in this case some allocation
is likely to fail soon anyhow.

Currently there is no API to manually add a block of a given size to the
heap, although that could easily be added if necessary.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-08-15 10:48:02 +10:00
Angus Gratton d325ee4509 py/gc: Apply some code formatting cleanup.
This commit:
- Breaks up some long lines for readability.
- Fixes a potential macro argument expansion issue.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-08-15 10:41:02 +10:00
Dan Halbert 3d48e87e60 jepler: fix 18 tests (redo commit to be correct) 2023-08-14 18:34:32 -04:00
Dan Halbert b73f05fe64 turn off viper tests; thanks jepler 2023-08-14 16:43:07 -04:00
Dan Halbert 8cf5b6e707 fixes to unix port support; thanks @jepler 2023-08-14 09:51:52 -04:00
Dan Halbert d582407b06 pre-commit fixes 2023-08-14 00:59:22 -04:00
Dan Halbert 2171e67c1b merge latest adafruit/main 2023-08-13 19:43:54 -04:00
Dan Halbert 27c7807693 silabs build needs work; other builds are working 2023-08-13 19:05:16 -04:00
Dan Halbert 850877a85f wip; fix generators; typos 2023-08-13 00:30:59 -04:00
Dan Halbert 433059023d fix -j1 builds, particularly CPX 2023-08-12 16:49:03 -04:00
Dan Halbert 48e404ab90 wip; fix frozen; still need to fix -j1 for frozen 2023-08-12 15:44:27 -04:00
Dan Halbert 1b19ed4c39 change compilation verbosity options; fix mpy-cross 2023-08-12 13:38:17 -04:00
Dan Halbert 8a89a3d425 sort TRANSLATION()'s 2023-08-11 13:36:03 -04:00
Dan Halbert 10b94796c0 wip; fix more m_malloc and gc_alloc 2023-08-11 00:30:34 -04:00
Dan Halbert fe0e2f13bc wip; fix qstr processing 2023-08-10 20:06:32 -04:00
Dan Halbert 0dca530b70 wip; Trinket M0 compiles 2023-08-09 15:20:10 -04:00
Dan Halbert 2ff8667e75 wip; remove long-lived functionality; remove PR#2614
Trinket M0 compile has no compilation errors, but has link errors to fix.
2023-08-08 20:41:17 -04:00
Dan Halbert 0d2c3c3f08 wip: continuing compilation fixes; mp_obj_alloc everywhere 2023-08-07 20:45:57 -04:00
Damien George ef71028f77 extmod/modselect: Add optimisation to use system poll when possible.
A previous commit removed the unix-specific select module implementation
and made unix use the common one.

This commit adds an optimisation so that the system poll function is used
when polling objects that have a file descriptor.  With this optimisation
enabled, if code registers both file-descriptor-based objects, and non-
file-descriptor-based objects with select.poll() then the following occurs:

- the system poll is called for all file-descriptor-based objects with a
  timeout of 1ms

- then the bare-metal polling implementation is used for remaining objects,
  which calls into their ioctl method (which can be in C or Python)

In the case where all objects have file descriptors, the system poll is
called with the full timeout requested by the caller.  That makes it as
efficient as possible in the case everything has a file descriptor.

Benefits of this approach:

- all ports use the same select module implementation

- the unix port now supports polling of all objects and matches bare metal
  implementations

- it's still efficient for existing cases where only files and sockets are
  polled (on unix)

- the bare metal implementation does not change

- polling of SSL objects will now work on unix by calling in to the ioctl
  method on SSL objects (this is required for asyncio ssl support)

Note that extmod/vfs_posix_file.c has poll disable when the optimisation is
enabled, because the code is not reachable when the optimisation is used.

Signed-off-by: Damien George <damien@micropython.org>
2023-08-07 12:11:40 +10:00
Jeff Epler 46db0366b8
OrderedDict.move_to_end: alternate implementation of #8234
this implementation is hoped to be smaller. (feather_m4_express/fr fits
unlike the other PR; approximate savings ~600 bytes)

Minor difference to standard Python: A `dict` object has a
`move_to_end` method. However, calling this method always results in
TypeError.

Implementing it this way means that the method table can still be shared
between OrderedDict and builtin dict.

Closes #4408.
2023-08-04 09:26:26 -05:00
Damien Tournoud 2dcd745434 py/gc: Speed up incremental GC cycles by tracking the last used block.
In applications that use little memory and run GC regularly, the cost of
the sweep phase quickly becomes prohibitives as the amount of RAM
increases.

On an ESP32-S3 with 2 MB of external SPIRAM, for example, a trivial GC
cycle takes a minimum of 40ms, virtually all of it in the sweep phase.

Similarly, on the UNIX port with 1 GB of heap, a trivial GC takes 47 ms,
again virtually all of it in the sweep phase.

This commit speeds up the sweep phase in the case most of the heap is empty
by keeping track of the ID of the highest block we allocated in an area
since the last GC.

The performance benchmark run on PYBV10 shows between +0 and +2%
improvement across the existing performance tests.  These tests don't
really stress the GC, so they were also run with gc.threshold(30000) and
gc.threshold(10000).  For the 30000 case, performance improved by up to
+10% with this commit.  For the 10000 case, performance improved by at
least +10% on 6 tests, and up to +25%.

Signed-off-by: Damien George <damien@micropython.org>
2023-08-04 17:25:16 +10:00
Dan Halbert 465d1c673e wip 2023-08-03 23:29:00 -04:00
Dan Halbert 0d4bc8c163 initial v1.19.1 merge; not compiled yet 2023-08-01 13:50:05 -04:00
Jeff Epler 272a2dcdb0
Merge pull request #8218 from jepler/esp32-camera
Update the camera board to use ESP32-S3 microcontroller
2023-07-31 14:48:50 -05:00
Jeff Epler cd181eb5b4
expand on the 0/1 requirement 2023-07-31 10:44:06 -05:00
Jeff Epler e8307a4c02
enable-if-any/all deserve some commentary. 2023-07-31 10:21:53 -05:00
Dan Halbert f466a4373e
Merge pull request #8222 from tannewt/fat_window_align
Align fatfs window buffer for tinyusb
2023-07-27 20:35:23 -04:00