circuitpython/ports/unix
Damien George ef71028f77 extmod/modselect: Add optimisation to use system poll when possible.
A previous commit removed the unix-specific select module implementation
and made unix use the common one.

This commit adds an optimisation so that the system poll function is used
when polling objects that have a file descriptor.  With this optimisation
enabled, if code registers both file-descriptor-based objects, and non-
file-descriptor-based objects with select.poll() then the following occurs:

- the system poll is called for all file-descriptor-based objects with a
  timeout of 1ms

- then the bare-metal polling implementation is used for remaining objects,
  which calls into their ioctl method (which can be in C or Python)

In the case where all objects have file descriptors, the system poll is
called with the full timeout requested by the caller.  That makes it as
efficient as possible in the case everything has a file descriptor.

Benefits of this approach:

- all ports use the same select module implementation

- the unix port now supports polling of all objects and matches bare metal
  implementations

- it's still efficient for existing cases where only files and sockets are
  polled (on unix)

- the bare metal implementation does not change

- polling of SSL objects will now work on unix by calling in to the ioctl
  method on SSL objects (this is required for asyncio ssl support)

Note that extmod/vfs_posix_file.c has poll disable when the optimisation is
enabled, because the code is not reachable when the optimisation is used.

Signed-off-by: Damien George <damien@micropython.org>
2023-08-07 12:11:40 +10:00
..
mbedtls all: Rename UMODULE to MODULE in preprocessor/Makefile vars. 2023-06-08 17:54:11 +10:00
variants extmod/modselect: Add optimisation to use system poll when possible. 2023-08-07 12:11:40 +10:00
alloc.c unix/alloc: Use MP_REGISTER_ROOT_POINTER() for mmap_region_head. 2022-07-18 13:49:09 +10:00
coverage.c extmod/modtimeq: Remove timeq module. 2023-06-08 17:54:28 +10:00
coveragecpp.cpp esp32,unix: Support building C++ code. 2020-10-29 15:29:50 +11:00
fatfs_port.c unix/fatfs_port: Fix month offset in timestamp calculation. 2020-09-02 00:19:06 +10:00
gccollect.c py/gc: Make gc_dump_info/gc_dump_alloc_table take a printer as argument. 2023-03-10 10:58:10 +11:00
input.c all: Update to point to files in new shared/ directory. 2021-07-12 17:08:10 +10:00
input.h ports: Make new ports/ sub-directory and move all ports there. 2017-09-06 13:40:51 +10:00
main.c py/modsys: Allow sys.path to be assigned to. 2023-06-08 17:54:24 +10:00
Makefile unix/modselect: Remove unix-specific implementation of select module. 2023-08-07 12:09:38 +10:00
modffi.c unix/modffi: Move header includes inside MICROPY_PY_FFI guard. 2022-10-14 14:36:17 +11:00
modjni.c unix/Makefile: Don't use _MOD variable names. 2022-10-11 23:27:01 +11:00
modmachine.c all: Use MP_REGISTER_EXTENSIBLE_MODULE for overrideable built-ins. 2023-06-08 17:54:21 +10:00
modos.c all: Rename *umodule*.c to remove the "u" prefix. 2023-06-08 17:54:17 +10:00
modsocket.c unix/modsocket: Add poll support for missing ERR,HUP,NVAL poll values. 2023-08-06 11:54:06 +10:00
modtermios.c all: Fix spelling mistakes based on codespell check. 2023-04-27 18:03:06 +10:00
modtime.c all: Rename *umodule*.c to remove the "u" prefix. 2023-06-08 17:54:17 +10:00
mpbthciport.c extmod/modbluetooth: Make all HCI transports trace in the same format. 2023-04-26 11:37:20 +10:00
mpbtstackport_common.c extmod/btstack: Switch to use hci_dump_init instead of hci_dump_open. 2023-04-22 00:39:31 +10:00
mpbtstackport_h4.c extmod/btstack: Update BTstack bindings to work with latest BTstack. 2022-10-22 14:28:25 +11:00
mpbtstackport_usb.c extmod/btstack: Update BTstack bindings to work with latest BTstack. 2022-10-22 14:28:25 +11:00
mpbtstackport.h unix: Implement BLE H4 HCI UART for btstack/nimble. 2020-09-08 12:53:24 +10:00
mpconfigport.h py/modsys: Allow sys.path to be assigned to. 2023-06-08 17:54:24 +10:00
mpconfigport.mk all: Rename UMODULE to MODULE in preprocessor/Makefile vars. 2023-06-08 17:54:11 +10:00
mphalport.h unix,windows: Factor out code that generates random bytes to a new func. 2022-06-07 23:41:30 +10:00
mpnimbleport.c extmod/nimble: Make stm32 and unix NimBLE ports use synchronous events. 2020-11-13 17:19:05 +11:00
mpnimbleport.h unix: Implement BLE H4 HCI UART for btstack/nimble. 2020-09-08 12:53:24 +10:00
mpthreadport.c unix: Implement -X realtime command-line option on macOS. 2022-05-24 00:51:47 +10:00
mpthreadport.h unix: Implement -X realtime command-line option on macOS. 2022-05-24 00:51:47 +10:00
qstrdefsport.h all: Add *FORMAT-OFF* in various places. 2020-02-28 10:31:07 +11:00
README.md unix/README: Fix Markdown link markup. 2023-07-27 21:40:41 +10:00
unix_mphal.c all: Rename mod_umodule*, ^umodule* to remove the "u" prefix. 2023-06-08 17:54:07 +10:00

The Unix version

The "unix" port requires a standard Unix-like environment with gcc and GNU make. This includes Linux, BSD, macOS, and Windows Subsystem for Linux. The x86 and x64 architectures are supported (i.e. x86 32- and 64-bit), as well as ARM and MIPS. Making a full-featured port to another architecture requires writing some assembly code for the exception handling and garbage collection. Alternatively, a fallback implementation based on setjmp/longjmp can be used.

To build (see section below for required dependencies):

$ cd ports/unix
$ make submodules
$ make

Then to give it a try:

$ ./build-standard/micropython
>>> list(5 * x + y for x in range(10) for y in [4, 2, 1])

Use CTRL-D (i.e. EOF) to exit the shell.

Learn about command-line options (in particular, how to increase heap size which may be needed for larger applications):

$ ./build-standard/micropython -h

To run the complete testsuite, use:

$ make test

The Unix port comes with a built-in package manager called mip, e.g.:

$ ./build-standard/micropython -m mip install hmac

or

$ ./build-standard/micropython
>>> import mip
>>> mip.install("hmac")

Browse available modules at micropython-lib. See Package management for more information about mip.

External dependencies

The libffi library and pkg-config tool are required. On Debian/Ubuntu/Mint derivative Linux distros, install build-essential(includes toolchain and make), libffi-dev, and pkg-config packages.

Other dependencies can be built together with MicroPython. This may be required to enable extra features or capabilities, and in recent versions of MicroPython, these may be enabled by default. To build these additional dependencies, in the unix port directory first execute:

$ make submodules

This will fetch all the relevant git submodules (sub repositories) that the port needs. Use the same command to get the latest versions of submodules as they are updated from time to time. After that execute:

$ make deplibs

This will build all available dependencies (regardless whether they are used or not). If you intend to build MicroPython with additional options (like cross-compiling), the same set of options should be passed to make deplibs. To actually enable/disable use of dependencies, edit the ports/unix/mpconfigport.mk file, which has inline descriptions of the options. For example, to build the SSL module, MICROPY_PY_SSL should be set to 1.