Commit Graph

362 Commits

Author SHA1 Message Date
Damien George
f01d5fb657 py/mkrules.mk: Automatically configure frozen options when manifest set.
Following how mkrules.cmake works.  This makes it easy for a port to enable
frozen code, by defining FROZEN_MANIFEST in its Makefile.

Signed-off-by: Damien George <damien@micropython.org>
2023-06-08 23:12:56 +10:00
Jim Mussared
a1fbb1980c extmod/modtimeq: Remove timeq module.
This is a MicroPython-specific module that existed to support the old
version of uasyncio.  It's undocumented and not enabled on all ports and
takes up code size unnecessarily.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:28 +10:00
Jim Mussared
4216bc7d13 tests: Replace umodule with module everywhere.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared
5e50975a6d py/modsys: Allow sys.path to be assigned to.
Previously sys.path could be modified by append/pop or slice assignment.

This allows `sys.path = [...]`, which can be simpler in many cases, but
also improves CPython compatibility.

It also allows sys.path to be set to a tuple which means that you can
clear sys.path (e.g. temporarily) with no allocations.

This also makes sys.path (and sys.argv for consistency) able to be disabled
via mpconfig. The unix port (and upytesthelper) require them, so they
explicitly verify that they're enabled.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared
2eba98f1e0 all: Use MP_REGISTER_EXTENSIBLE_MODULE for overrideable built-ins.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:21 +10:00
Jim Mussared
45ac651d1a all: Rename *umodule*.c to remove the "u" prefix.
Updates any includes, and references from Makefiles/CMake.

This essentially reverts what was done long ago in commit
136b5cbd76

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:17 +10:00
Jim Mussared
0ceccd4cf8 all: Rename *umodule*.h to remove the "u" prefix.
This work was funded through GitHub Sponsors.

Also updates #includes.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:14 +10:00
Jim Mussared
f5f9edf645 all: Rename UMODULE to MODULE in preprocessor/Makefile vars.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:11 +10:00
Jim Mussared
7f5d5c7271 all: Rename mod_umodule*, ^umodule* to remove the "u" prefix.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:07 +10:00
Jim Mussared
1bf2dcb15e all: Rename mp_umodule*, mp_module_umodule* to remove the "u" prefix.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:04 +10:00
Jim Mussared
30628d1bb7 all: Rename MP_QSTR_umodule to MP_QSTR_module everywhere.
This renames the builtin-modules, such that help('modules') and printing
the module object will show "module" rather than "umodule".

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:53:57 +10:00
Jim Mussared
952a78f88a unix/main: Add NLR scope for checking module __path__.
If the module has a user-defined getattr, this could raise.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-01 16:21:37 +10:00
Jim Mussared
525557738c py/builtinimport: Optimise sub-package loading.
This makes it so that sub-packages are resolved relative to their parent's
`__path__`, rather than re-resolving each parent's filesystem path.

The previous behavior was that `import foo.bar` would first re-search
`sys.path` for `foo`, then use the resulting path to find `bar`.

For already-loaded and u-prefixed modules, because we no longer need to
build the path from level to level, we no longer unnecessarily search
the filesystem. This should improve startup time.

Explicitly makes the resolving process clear:
 - Loaded modules are returned immediately without touching the filesystem.
 - Exact-match of builtins are also returned immediately.
 - Then the filesystem search happens.
 - If that fails, then the weak-link handling is applied.

This maintains the existing behavior: if a user writes `import time` they
will get time.py if it exits, otherwise the built-in utime. Whereas `import
utime` will always return the built-in.

This also fixes a regression from a7fa18c203
where we search the filesystem for built-ins. It is now only possible to
override u-prefixed builtins. This will remove a lot of filesystem stats
at startup, as micropython-specific modules (e.g. `pyb`) will no longer
attempt to look at the filesystem.

Added several improvements to the comments and some minor renaming and
refactoring to make it clearer how the import mechanism works. Overall
code size diff is +56 bytes on STM32.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-01 16:21:21 +10:00
David Lechner
6836acf3dc unix/Makefile: Allow variants to add QSTR_DEFS.
This modifies the unix port Makefile to use += for QSTR_DEFS and
QSTR_GLOBAL_DEPENDENCIES so that variants can add additional files if
needed (similar to stm32 port).

Signed-off-by: David Lechner <david@pybricks.com>
2023-05-11 14:34:01 +10:00
Damien George
b1229efbd1 all: Fix spelling mistakes based on codespell check.
Signed-off-by: Damien George <damien@micropython.org>
2023-04-27 18:03:06 +10:00
Damien George
127fd170c8 unix/modutime: Use extmod version of time module.
No API or functional change.

Signed-off-by: Damien George <damien@micropython.org>
2023-04-27 15:11:52 +10:00
Damien George
9955553001 extmod/modutime: Provide a generic time module.
Based on extmod/utime_mphal.c, with:
- a globals dict added
- time.localtime wrapper added
- time.time wrapper added
- time.time_ns function added

New configuration options are added for this module:
- MICROPY_PY_UTIME (enabled at basic features level)
- MICROPY_PY_UTIME_GMTIME_LOCALTIME_MKTIME
- MICROPY_PY_UTIME_TIME_TIME_NS

Signed-off-by: Damien George <damien@micropython.org>
2023-04-27 15:09:56 +10:00
Jim Mussared
a42a8022d8 extmod/modbluetooth: Make all HCI transports trace in the same format.
- Use HCI_TRACE macro consistently.
 - Use the same colour formatting.
 - Add a tool to convert to .pcap for Wireshark.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-04-26 11:37:20 +10:00
Jim Mussared
256f47e2f8 extmod/btstack: Fix indicate/notify queuing.
This adds a mechanism to track a pending notify/indicate operation that
is deferred due to the send buffer being full. This uses a tracked alloc
that is passed as the content arg to the callback.

This replaces the previous mechanism that did this via the global pending
op queue, shared with client read/write ops.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-04-26 11:37:20 +10:00
Damien George
9e6885ad82 extmod/btstack: Switch to use hci_dump_init instead of hci_dump_open.
The latter is no longer available in the version of BTstack now in use by
this repository.

Signed-off-by: Damien George <damien@micropython.org>
2023-04-22 00:39:31 +10:00
Damien George
b3c8ab37ec py/gc: Make gc_dump_info/gc_dump_alloc_table take a printer as argument.
So that callers can redirect the output if needed.

Signed-off-by: Damien George <damien@micropython.org>
2023-03-10 10:58:10 +11:00
Jim Mussared
69b93527d5 ports: Make BOARD default from BOARD_DIR in Makefile's.
This allows:

    $ make BOARD_DIR=path/to/board

to infer BOARD=board, rather than the previous behavior that required
additionally setting BOARD explicitly.

Also makes the same change for VARIANT_DIR -> VARIANT on Unix.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
2023-02-23 09:47:08 +11:00
小权一句两句
94ee1b629a unix/main: Free pathbuf when there's an error opening a file. 2023-01-20 19:38:14 +11:00
Juan Francisco Cantero Hurtado
720f2cfba9 unix/Makefile: Disable building the shared library of libffi.
MicroPython doesn't need the shared library and disabling it in the
configure could ease the build of a static version of MicroPython.
2023-01-19 15:24:43 +11:00
David Lechner
958f748e53 unix/moduos: Implement 2-arg version of os.getenv().
This adds the `default` argument of `os.getenv(key, default=None)`.

Signed-off-by: David Lechner <david@pybricks.com>
2022-12-14 13:38:39 +11:00
Jeff Epler
d75ff42297 unix/coverage: Add extra GC coverage test for ATB gap byte.
The assertion that is added here (to gc.c) fails when running this new test
if ALLOC_TABLE_GAP_BYTE is set to 0.

Signed-off-by: Jeff Epler <jepler@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
2022-12-08 14:35:08 +11:00
iabdalkader
71881116e6 extmod/extmod.mk: Set default mbedtls config file in extmod Makefile. 2022-11-09 18:43:13 +01:00
Damien George
3bb371c5f2 unix/variants: Change mip package to mip-cmdline.
mip-cmdline adds command-line support to mip, useful for the unix port, via
micropython -m mip ...

Signed-off-by: Damien George <damien@micropython.org>
2022-11-09 14:58:54 +11:00
Damien George
9347545f9e extmod/mbedtls: Enable MBEDTLS_TLS_DEFAULT_ALLOW_SHA1_IN_KEY_EXCHANGE.
This was already enabled on all ports except mimxrt.  Now it's enabled on
all of them.

Signed-off-by: Damien George <damien@micropython.org>
2022-10-22 19:06:21 +11:00
Damien George
b337678964 extmod/mbedtls: Add common configuration file, and use it in all ports.
This is a no-op change.

Signed-off-by: Damien George <damien@micropython.org>
2022-10-22 19:06:21 +11:00
Damien George
67f98ba10c extmod/btstack: Update BTstack bindings to work with latest BTstack.
The following multi-tests pass (eg with PYBD_SF6+LEGO_HUB_NO6):

    ble_gap_advertise.py
    ble_gap_connect.py
    ble_gap_device_name.py
    ble_gattc_discover_services.py
    ble_gatt_data_transfer.py
    perf_gatt_char_write.py
    perf_gatt_notify.py
    stress_log_filesystem.py

These are the same tests that passed prior to this BTstack update.

Also tested on the unix port using H4 transport.

Signed-off-by: Damien George <damien@micropython.org>
2022-10-22 14:28:25 +11:00
Damien George
89b3207376 unix/modffi: Move header includes inside MICROPY_PY_FFI guard.
So ffi.h is not needed if this module is disabled.

Signed-off-by: Damien George <damien@micropython.org>
2022-10-14 14:36:17 +11:00
Jim Mussared
d6d8722558 extmod: Make extmod.mk self-contained.
This makes it so that all a port needs to do is set the relevant variables
and "include extmod.mk" and doesn't need to worry about adding anything to
OBJ, CFLAGS, SRC_QSTR, etc.

Make all extmod variables (src, flags, etc) private to extmod.mk.

Also move common/shared, extmod-related fragments (e.g. wiznet, cyw43,
bluetooth) into extmod.mk.

Now that SRC_MOD, CFLAGS_MOD, CXXFLAGS_MOD are unused by both extmod.mk
(and user-C-modules in a previous commit), remove all uses of them from
port makefiles.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-11 23:31:49 +11:00
Jim Mussared
87011f6353 extmod/extmod.mk: Make extmod.mk handle GIT_SUBMODULES.
This applies to nimble, btstack, axtls, mbedtls, lwip.

Rather than having the ports individually manage GIT_SUBMODULES for these
components, make extmod.mk append them when the relevant feature is
enabled.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-11 23:29:09 +11:00
Jim Mussared
065af04a4e unix/Makefile: Don't use _MOD variable names.
This conflicts with the triple-usage of these variables for user-C-modules
and extmod source.

For CFLAGS_MOD, just use CFLAGS directly. For SRC, use SRC_C directly as
the relevant files are all guarded by the preprocessor anyway.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-11 23:27:01 +11:00
Jim Mussared
8e912a501a unix: Enable sys.executable.
Gives the absolute path to the unix micropython binary.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
2022-10-11 18:10:30 +11:00
Jim Mussared
924a3e03ec top: Replace upip with mip everywhere.
Updates all README.md and docs, and manifests to `require("mip")`.

Also extend and improve the documentation on freezing and packaging.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-30 17:34:03 +10:00
Jim Mussared
920da9c5e3 unix/variants/coverage: Add test for manifest freeze_mpy().
This uses the frozentest.mpy that is also used by ports/minimal.

Also fixes two bugs that these new tests picked up:
 - File extension matching in manifestfile.py.
 - Handling of freeze_mpy results in makemanifest.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 23:51:10 +10:00
Jim Mussared
94beeabd2e py/obj: Convert make_new into a mp_obj_type_t slot.
Instead of being an explicit field, it's now a slot like all the other
methods.

This is a marginal code size improvement because most types have a make_new
(100/138 on PYBV11), however it improves consistency in how types are
declared, removing the special case for make_new.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:15 +10:00
Jim Mussared
6da41b5900 py/obj: Merge getiter and iternext mp_obj_type_t slots.
The goal here is to remove a slot (making way to turn make_new into a slot)
as well as reduce code size by the ~40 references to mp_identity_getiter
and mp_stream_unbuffered_iter.

This introduces two new type flags:
- MP_TYPE_FLAG_ITER_IS_ITERNEXT: This means that the "iter" slot in the
  type is "iternext", and should use the identity getiter.
- MP_TYPE_FLAG_ITER_IS_CUSTOM: This means that the "iter" slot is a pointer
  to a mp_getiter_iternext_custom_t instance, which then defines both
  getiter and iternext.

And a third flag that is the OR of both, MP_TYPE_FLAG_ITER_IS_STREAM: This
means that the type should use the identity getiter, and
mp_stream_unbuffered_iter as iternext.

Finally, MP_TYPE_FLAG_ITER_IS_GETITER is defined as a no-op flag to give
the default case where "iter" is "getiter".

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:13 +10:00
Jim Mussared
a52cd5b07d py/obj: Add accessors for type slots and use everywhere.
This is a no-op, but sets the stage for changing the mp_obj_type_t
representation.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:07 +10:00
Jim Mussared
9dce82776d all: Remove unnecessary locals_dict cast.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:01 +10:00
Jim Mussared
662b9761b3 all: Make all mp_obj_type_t defs use MP_DEFINE_CONST_OBJ_TYPE.
In preparation for upcoming rework of mp_obj_type_t layout.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:01 +10:00
Jim Mussared
fb2a57800a all: Simplify buffer protocol to just a "get buffer" callback.
The buffer protocol type only has a single member, and this existing layout
creates problems for the upcoming split/slot-index mp_obj_type_t layout
optimisations.

If we need to make the buffer protocol more sophisticated in the future
either we can rely on the mp_obj_type_t optimisations to just add
additional slots to mp_obj_type_t or re-visit the buffer protocol then.

This change is a no-op in terms of generated code.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 18:40:39 +10:00
Jim Mussared
51b054dd66 unix: Refactor mpconfigvariant.mk.
All variants (except minimal) enable text compression and fat/lfs, so move
them to the common mpconfigport.mk.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-13 17:40:55 +10:00
Jim Mussared
3e5b1be8ca py/mpconfig: Add "everything" features from unix coverage.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-13 17:40:33 +10:00
Jim Mussared
c1530a0ce8 unix: Refactor mpconfigport.h and mpconfigvariant.h.
This is a no-op for coverage and minimal.

The standard and dev variants have been merged and enable the same feature
set as a typical bare-metal board.  And remove the CI for the dev build.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-13 17:39:03 +10:00
Jim Mussared
f4fed02537 unix/variants: Remove freedos and fast variants.
The freedos variant is untested by CI and is difficult to maintain.  The
fast variant is not a good name for what it does.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-13 17:37:55 +10:00
Andrew Leech
86676a43aa unix/mpconfigport: Enable MICROPY_ENABLE_FINALISER when VFS is used. 2022-09-13 13:06:56 +10:00
Jim Mussared
203dae41fb all: Update all manifest.py files to use new features.
Changes in this commit:
- Manifest include's now use the directory path where possible (no longer
  necessary to include the manifest.py file explicitly).
- Add manifest.py for all drivers and components that are referenced by
  port/board manifests.
- Replace all uses of freeze() with package()/module(), except for port and
  board modules.
- Use opt=3 everywhere, for consistency and to reduce code size.
- Use require() instead of include() for all micropython-lib references.
- Remove support for optional board-level manifest.py in mimxrt port, to
  make it behave the same as other ports (the board must set
  FROZEN_MANIFEST to a custom manifest.py, which can optionally include the
  default, port-level manifest).
- Also reinstates modules that were accidentally removed from the esp8266
  512k build in fbe9417b90.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
2022-09-05 18:43:18 +10:00