Commit Graph

127 Commits

Author SHA1 Message Date
Jim Mussared 3533924c36 extmod/moddeflate: Add deflate module providing the DeflateIO class.
This provides similar functionality to the former zlib.DecompIO and
especially CPython's gzip.GzipFile for both compression and decompression.

This class can be used directly, and also can be used from Python to
implement (via io.BytesIO) zlib.decompress and zlib.compress, as well as
gzip.GzipFile.

Enable/disable this on all ports/boards that zlib was previously configured
for.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 19:32:40 +10:00
Jim Mussared add1200343 all: Remove the zlib module.
This will be replaced with a new deflate module providing the same
functionality, with an optional frozen Python wrapper providing a
replacement zlib module.

binascii.crc32 is temporarily disabled.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 18:48:29 +10:00
Jim Mussared 2fbc08c462 extmod/asyncio: Rename uasyncio to asyncio.
The asyncio module now has much better CPython compatibility and
deserves to be just called "asyncio".

This will avoid people having to write `from uasyncio import asyncio`.

Renames all files, and updates port manifests to use the new path. Also
renames the built-in _uasyncio to _asyncio.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-19 17:33:03 +10:00
Damien George f01d5fb657 py/mkrules.mk: Automatically configure frozen options when manifest set.
Following how mkrules.cmake works.  This makes it easy for a port to enable
frozen code, by defining FROZEN_MANIFEST in its Makefile.

Signed-off-by: Damien George <damien@micropython.org>
2023-06-08 23:12:56 +10:00
Jim Mussared a1fbb1980c extmod/modtimeq: Remove timeq module.
This is a MicroPython-specific module that existed to support the old
version of uasyncio.  It's undocumented and not enabled on all ports and
takes up code size unnecessarily.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:28 +10:00
Jim Mussared 45ac651d1a all: Rename *umodule*.c to remove the "u" prefix.
Updates any includes, and references from Makefiles/CMake.

This essentially reverts what was done long ago in commit
136b5cbd76

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:17 +10:00
Jim Mussared f5f9edf645 all: Rename UMODULE to MODULE in preprocessor/Makefile vars.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:11 +10:00
Jim Mussared dfe232d000 py/builtinimport: Remove weak links.
In order to keep "import umodule" working, the existing mechanism is
replaced with a simple fallback to drop the "u".

This makes importing of built-ins no longer touch the filesystem, which
makes a typical built-in import take ~0.15ms rather than 3-5ms.

(Weak links were added in c14a81662c)

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:04 +10:00
David Lechner 6a4a9bc147 windows/Makefile: Allow variants to add QSTR_DEFS.
This modifies the windows port Makefile to use += for QSTR_DEFS and
QSTR_GLOBAL_DEPENDENCIES so that variants can add additional files if
needed (similar to stm32 port).

Signed-off-by: David Lechner <david@pybricks.com>
2023-05-11 14:34:15 +10:00
Damien George 127fd170c8 unix/modutime: Use extmod version of time module.
No API or functional change.

Signed-off-by: Damien George <damien@micropython.org>
2023-04-27 15:11:52 +10:00
Damien George 9955553001 extmod/modutime: Provide a generic time module.
Based on extmod/utime_mphal.c, with:
- a globals dict added
- time.localtime wrapper added
- time.time wrapper added
- time.time_ns function added

New configuration options are added for this module:
- MICROPY_PY_UTIME (enabled at basic features level)
- MICROPY_PY_UTIME_GMTIME_LOCALTIME_MKTIME
- MICROPY_PY_UTIME_TIME_TIME_NS

Signed-off-by: Damien George <damien@micropython.org>
2023-04-27 15:09:56 +10:00
Jim Mussared 69b93527d5 ports: Make BOARD default from BOARD_DIR in Makefile's.
This allows:

    $ make BOARD_DIR=path/to/board

to infer BOARD=board, rather than the previous behavior that required
additionally setting BOARD explicitly.

Also makes the same change for VARIANT_DIR -> VARIANT on Unix.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
2023-02-23 09:47:08 +11:00
stijn 6abf03b5d0 windows/msvc: Fix module freezing.
Make this more generally useful and in line with what the mingw
and unix ports do: 16bit dig size to work on 32bit ports, a
self-contained qstrdefs.preprocessed.h because makemanifest.py
uses that, and a dev variant which effectively puts this to use:
previously the uasyncio module wasn't frozen but instead tests
ran by importing it from the extmod/ directory.
2023-02-01 13:10:00 +11:00
stijn e145318a81 windows/msvc: Fix qstr generation dependency.
The mpversion.h file must exist before py/ source can be preprocessed,
but this went unnoticed because micropython.vcxproj always calls
MakeVersionHdr before MakeQstrDefs.
2023-02-01 13:10:00 +11:00
stijn 65941ea0e5 windows/msvc: Make mpy-cross independent of micropython variant.
The variant.props may have incompatible build options which break
the mpy-cross build and in any case mpy-cross has nothing to do
with variant support.
2023-02-01 13:10:00 +11:00
stijn 55a76f4edd windows/msvc: Remove variant suffix from executable filename.
This is in line with the change made for other ports in d53c3b6a: since
the default output directory already includes the variant name in it
there's no need to add it to the executable as well.
2023-02-01 13:10:00 +11:00
stijn f6f177807f windows/.gitignore: Simplify by removing build artefacts.
Since all output is now in the build-<variant>/ directory, which is already
excluded by the root .gitignore, we don't need to repeat that.
2022-12-13 17:22:11 +11:00
stijn 9c7ff87643 all: Keep msvc build output in build/ directories.
This follow the change made for Makefile-based projects in b2e82402.
2022-12-13 17:18:53 +11:00
stijn fbc96009f8 windows: Make project file read-only for IDE.
Avoids the 'warning: Wildcards in project items are not supported'
message from the C++ project system in Visual Studio, while otherwise
remaining completely functional.
2022-11-15 16:37:31 +11:00
Jim Mussared d6d8722558 extmod: Make extmod.mk self-contained.
This makes it so that all a port needs to do is set the relevant variables
and "include extmod.mk" and doesn't need to worry about adding anything to
OBJ, CFLAGS, SRC_QSTR, etc.

Make all extmod variables (src, flags, etc) private to extmod.mk.

Also move common/shared, extmod-related fragments (e.g. wiznet, cyw43,
bluetooth) into extmod.mk.

Now that SRC_MOD, CFLAGS_MOD, CXXFLAGS_MOD are unused by both extmod.mk
(and user-C-modules in a previous commit), remove all uses of them from
port makefiles.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-11 23:31:49 +11:00
Jim Mussared 17f2783e4a all: Use += rather than = everywhere for CFLAGS/LDFLAGS/LIBS.
This avoids a surprise where an = can cancel out an earlier +=.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-10-11 23:17:41 +11:00
Jim Mussared 203dae41fb all: Update all manifest.py files to use new features.
Changes in this commit:
- Manifest include's now use the directory path where possible (no longer
  necessary to include the manifest.py file explicitly).
- Add manifest.py for all drivers and components that are referenced by
  port/board manifests.
- Replace all uses of freeze() with package()/module(), except for port and
  board modules.
- Use opt=3 everywhere, for consistency and to reduce code size.
- Use require() instead of include() for all micropython-lib references.
- Remove support for optional board-level manifest.py in mimxrt port, to
  make it behave the same as other ports (the board must set
  FROZEN_MANIFEST to a custom manifest.py, which can optionally include the
  default, port-level manifest).
- Also reinstates modules that were accidentally removed from the esp8266
  512k build in fbe9417b90.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
2022-09-05 18:43:18 +10:00
Damien George 8f4c108025 all: Remove MICROPY_PY_IO_FILEIO config option.
Since commit e65d1e69e8 there is no longer an
io.FileIO class, so this option is no longer needed.

This option also controlled whether or not files supported being opened in
binary mode (eg 'rb'), and could, if disabled, lead to confusion as to why
opening a file in binary mode silently did the wrong thing (it would just
open in text mode if MICROPY_PY_IO_FILEIO was disabled).

The various VFS implementations (POSIX, FAT, LFS) were the only places
where enabling this option made a difference, and in almost all cases where
one of these filesystems were enabled, MICROPY_PY_IO_FILEIO was also
enabled.  So it makes sense to just unconditionally enable this feature
(ability to open a file in binary mode) in all cases, and so just remove
this config option altogether.  That makes configuration simpler and means
binary file support always exists (and opening a file in binary mode is
arguably more fundamental than opening in text mode, so if anything should
be configurable then it should be the ability to open in text mode).

Signed-off-by: Damien George <damien@micropython.org>
2022-08-18 11:54:17 +10:00
Jim Mussared 28aaab9590 py/objstr: Add hex/fromhex to bytes/memoryview/bytearray.
These were added in Python 3.5.

Enabled via MICROPY_PY_BUILTINS_BYTES_HEX, and enabled by default for all
ports that currently have ubinascii.

Rework ubinascii to use the implementation of these methods.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-12 12:44:30 +10:00
Jim Mussared 94a19f1062 windows/Makefile: Update test dependency on $(PROG).
PR #9012 (b2e8240268) changed the output to
$(BUILD)/$(PROG) but the tests are still looking for $(PROG).

Also remove the now-unnecessary override of $(PROG) in the standard
variant.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-11 16:53:43 +10:00
Damien George d53c3b6ade unix/variants: Remove variant suffix from executable filename.
The executable now lives in the build directory, and since the build
directory already contains the variant name there is no need to also add
it to the executable.

Signed-off-by: Damien George <damien@micropython.org>
2022-08-11 13:34:34 +10:00
Jim Mussared 579f330508 py/mkenv.mk: Use micropython-lib from submodule by default.
Also adds micropython-lib to 'make submodules' when using a frozen manifest
(for make and cmake).

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-03 00:08:41 +10:00
Jim Mussared 092784da19 ports: Remove unused mp_type_{fileio/textio} macros in mpconfigport.h.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-07-26 18:07:22 +10:00
stijn d05377c060 windows/msvc: Support compressed ROM text for error messages.
Enable it in the dev variant as well for consistency with the
makefile-based dev variant.
2022-07-18 23:25:38 +10:00
stijn c4adeb2e08 windows: Provide a definition for MP_ALWAYSINLINE. 2022-07-18 23:24:46 +10:00
David Lechner 81dbea1ce3 shared/readline: Use MP_REGISTER_ROOT_POINTER().
This uses MP_REGISTER_ROOT_POINTER() to register the readline_history root
pointer array used by shared/readline.c and removes the registration from
all mpconfigport.h files.

This also required adding a new MICROPY_READLINE_HISTORY_SIZE config option
since not all ports used the same sized array.

Signed-off-by: David Lechner <david@pybricks.com>
2022-07-18 13:48:49 +10:00
David Lechner fc3d7ae11b py/make_root_pointers: Add MP_REGISTER_ROOT_POINTER parser/generator.
This adds new compile-time infrastructure to parse source code files for
`MP_REGISTER_ROOT_POINTER()` and generates a new `root_pointers.h` header
file containing the collected declarations.  This works the same as the
existing `MP_REGISTER_MODULE()` feature.

Signed-off-by: David Lechner <david@pybricks.com>
2022-07-18 13:48:23 +10:00
David Lechner c118b5d0e4 extmod/extmod.mk: Separate out extmod file list from py.mk to extmod.mk.
This separates extmod source files from `py.mk`.  Previously, `py.mk`
assumed that every consumer of the py/ directory also wanted to include
extmod/.  However, this is not the case.  For example, building mpy-cross
uses py/ but doesn't need extmod/.

This commit moves all extmod-specific items from `py.mk` to `extmod.mk` and
explicitly includes `extmod.mk` in ports that use it.

Signed-off-by: David Lechner <david@pybricks.com>
2022-06-21 00:14:34 +10:00
Damien George d2f018bff1 unix,windows: Factor out code that generates random bytes to a new func.
Signed-off-by: Damien George <damien@micropython.org>
2022-06-07 23:41:30 +10:00
stijn 5bb2a85d74 windows: Use BCryptGenRandom to implement os.urandom.
Fix urandom not working on windows (there's no /dev/urandom) by using
a proper cryptographic random function (same one as CPython >= 3.11).
2022-06-07 23:32:30 +10:00
Damien George 47f634300c py: Change makemoduledefs process so it uses output of qstr extraction.
This cleans up the parsing of MP_REGISTER_MODULE() and generation of
genhdr/moduledefs.h so that it uses the same process as compressed error
string messages, using the output of qstr extraction.

This makes sure all MP_REGISTER_MODULE()'s that are part of the build are
correctly picked up.  Previously the extraction would miss some (eg if you
had a mod.c file in the board directory for an stm32 board).

Build speed is more or less unchanged.

Thanks to @stinos for the ports/windows/msvc/genhdr.targets changes.

Signed-off-by: Damien George <damien@micropython.org>
2022-06-02 16:29:53 +10:00
Damien George 6e71cde6aa ports: Use default VFS config for import_stat and builtin_open.
For ports with MICROPY_VFS and MICROPY_PY_IO enabled their configuration
can now be simplified to use the defaults for mp_import_stat and
mp_builtin_open.

This commit makes no functional change, except for the following minor
points:
- the built-in "open" is removed from the minimal port (it previously did
  nothing)
- the duplicate built-in "input" is removed from the esp32 port
- qemu-arm now delegates to VFS import/open

Signed-off-by: Damien George <damien@micropython.org>
2022-05-25 13:04:45 +10:00
Jim Mussared 63431a5536 unix: Make port-specific modules use MP_REGISTER_MODULE.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-05-18 20:57:09 +10:00
stijn 20028c7c80 windows: Avoid busy loop when using scheduler. 2022-05-04 10:22:29 +10:00
stijn 8aa79c95bd windows: Consolidate all sleep-related functions into windows_mphal.c.
Replace the timer-based sleep with the standard win32 call since the former
has no benefits: even though it allows specifying the time in 100uSec
chunks, the actual resolution is still limited by the OS and is never
better than 1mSec.

For clarity move all of this next to the mp_hal_delay_ms definition so all
related functions are in one place.
2022-05-04 10:22:09 +10:00
Damien George 0149cd6b8b windows: Switch to VFS subsystem and use VfsPosix.
Following the unix port.

Signed-off-by: Damien George <damien@micropython.org>
2022-03-10 00:26:36 +11:00
stijn c2e8a5acd2 windows: Unify project file headers.
The xmlns attribute is required for older msbuild version (e.g. for
VS2015).  Add it where needed, and reorder the attributes so all
files look the same.
2022-02-25 16:41:11 +11:00
stijn c14f47faa3 windows/Makefile: Specify CXXFLAGS in the Makefile.
Enables building user modules which use C++ code, like the unix port.
2022-02-18 14:40:36 +11:00
David Lechner 28cb573b89 windows/appveyor: Fix printing of test failures.
In the `after_test` section, the current directory is `ports/windows` when
tests are run, so running `run-tests.py` without changing the directory or
specifying a path causes a file not found error.

This commit fixes the problem by changing the directory before calling
`run-tests.py`.

Signed-off-by: David Lechner <david@pybricks.com>
2022-02-18 14:31:52 +11:00
Andrew Leech 6f7d6c567f windows/uasyncio: Add support for uasyncio to windows dev variant. 2022-02-07 14:39:15 +11:00
Andrew Leech c708262c12 windows/uselect: Enable micropython select in dev variant. 2022-02-07 14:39:15 +11:00
Andrew Leech 69c9a76786 windows/mingw: Include extmod/shared/lib sources properly. 2022-02-07 14:39:15 +11:00
Damien George fbd47fc46c ports: Consolidate inclusion of umachine module in built-ins.
The inclusion of `umachine` in the list of built-in modules is now done
centrally in py/objmodule.c.  Enabling MICROPY_PY_MACHINE will include this
module.

As part of this, all ports now have `umachine` as the core module name
(previously some had only `machine` as the name).

Signed-off-by: Damien George <damien@micropython.org>
2022-02-03 10:08:54 +11:00
Andrew Leech 30b6ce86be windows: Add micropython.schedule support. 2022-01-23 10:55:08 +11:00
stijn dd6967202a py/modmath: Add math.tau, math.nan and math.inf constants.
Configurable by the new MICROPY_PY_MATH_CONSTANTS option.
2022-01-23 09:28:33 +11:00