Commit Graph

37 Commits

Author SHA1 Message Date
Andrew Leech 32c99174e1 unix/mpconfigport.h: Enable MICROPY_PY_DELATTR_SETATTR.
This is a generally useful feature and because it's part of the object
model it cannot be added at runtime by some loadable Python code, so enable
it on the standard unix build.
2020-10-20 23:47:50 +11:00
Jim Mussared 1b1b22905e unix: Implement BLE H4 HCI UART for btstack/nimble.
This commit adds support for using Bluetooth on the unix port via a H4
serial interface (distinct from a USB dongle), with both BTstack and NimBLE
Bluetooth stacks.

Note that MICROPY_PY_BLUETOOTH is now disabled for the coverage variant.
Prior to this commit Bluetooth was anyway not being built on Travis because
libusb was not detected.  But now that bluetooth works in H4 mode it will
be built, and will lead to a large decrease in coverage because Bluetooth
tests cannot be run on Travis.
2020-09-08 12:53:24 +10:00
Jim Mussared ed14435a8e extmod/modbluetooth: Refactor stack/hci/driver/port bindings.
Previously the interaction between the different layers of the Bluetooth
stack was different on each port and each stack.  This commit defines
common interfaces between them and implements them for cyw43, btstack,
nimble, stm32, unix.
2020-09-08 11:41:31 +10:00
Damien George 2a72e90ab8 extmod/vfs: Add option to use 1970 as Epoch.
By setting MICROPY_EPOCH_IS_1970 a port can opt to use 1970/1/1 as the
Epoch for timestamps returned by stat().  And this setting is enabled on
the unix and windows ports because that's what they use.

Signed-off-by: Damien George <damien@micropython.org>
2020-09-01 12:36:28 +10:00
Damien George 5c8bf12acf all: Fix auto-enable of MICROPY_GCREGS_SETJMP to select GC behaviour.
Only enable it if MICROPY_GCREGS_SETJMP is not already defined, and no
supported architecture is defined.
2020-04-30 16:49:42 +10:00
Jim Mussared 710426024a all: Factor gchelper code to one place and use it for unix & ARM ports.
No functionality change is intended with this commit, it just consolidates
the separate implementations of GC helper code to the lib/utils/ directory
as a general set of helper functions useful for any port.  This reduces
duplication of code, and makes it easier for future ports or embedders to
get the GC implementation correct.

Ports should now link against gchelper_native.c and either gchelper_m0.s or
gchelper_m3.s (currently only Cortex-M is supported but other architectures
can follow), or use the fallback gchelper_generic.c which will work on
x86/x64/ARM.

The gc_helper_get_sp function from gchelper_m3.s is not really GC related
and was only used by cc3200, so it has been moved to that port and renamed
to cortex_m3_get_sp.
2020-04-29 23:45:19 +10:00
Jim Mussared 7563d58210 unix: Add support for modbluetooth and BLE using btstack.
This commit adds full support to the unix port for Bluetooth using the
common extmod/modbluetooth Python bindings.  This uses the libusb HCI
transport, which supports many common USB BT adaptors.
2020-04-29 16:45:46 +10:00
Jim Mussared cb5994d96e unix/modmachine: Add machine.idle(), implemented using sched_yield.
Also add a definition of MICROPY_EVENT_POLL_HOOK so the unix port can build
against modules that require this.
2020-04-29 16:37:46 +10:00
stijn 30840ebc99 all: Enable extra conversion warnings where applicable.
Add -Wdouble-promotion and -Wfloat-conversion for most ports to ban out
implicit floating point conversions, and add extra Travis builds using
MICROPY_FLOAT_IMPL_FLOAT to uncover warnings which weren't found
previously.  For the unix port -Wsign-comparison is added as well but only
there since only clang supports this but gcc doesn't.
2020-04-18 22:42:28 +10:00
Jim Mussared c2cfbcc8d4 unix: Implement MICROPY_BEGIN/END_ATOMIC_SECTION protection macros.
This macro is used to implement global serialisation, typically by
disabling IRQs.  On the unix port, if threading is enabled, use the
existing thread mutex (that protects the thread list structure) for this
purpose.  Other places in the code (eg the scheduler) assume this macro
will provide serialisation.
2020-04-13 21:44:09 +10:00
David Lechner b1066a9f96 unix: Remove custom definition of MP_PLAT_PRINT_STRN.
This removes the port-specific definition of MP_PLAT_PRINT_STRN on the unix
port.  Since fee7e5617f this is no longer a
single function call so we are not really optimising anything over using
the default definition of MP_PLAT_PRINT_STRN which calls
mp_hal_stdout_tx_strn_cooked().
2020-03-25 00:54:18 +11:00
Damien George 2cdf1d25f5 unix: Remove custom file implementation to use extmod's VFS POSIX one.
The implementation in extmod/vfs_posix_file.c is now equivalent to that in
ports/unix/file.c, so remove the latter and use the former instead.
2020-03-18 21:01:07 +11:00
Damien George 69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
David Lechner fee7e5617f unix: Release GIL during all system calls.
Addition of GIL EXIT/ENTER pairs are:

- modos: release the GIL during system calls.  CPython does this as well.

- moduselect: release the GIL during the poll() syscall.  This call can be
  blocking, so it is important to allow other threads to run at this time.

- modusocket: release the GIL during system calls.  Many of these calls can
  be blocking, so it is important to allow other threads to run.

- unix_mphal: release the GIL during the read and write syscalls in
  mp_hal_stdin_rx_chr and mp_hal_stdout_tx_strn.  If we don't do this
  threads are blocked when the REPL or the builtin input function are used.

- file, main, mpconfigport.h: release GIL during syscalls in built-in
  functions that could block.
2020-01-26 23:21:29 +11:00
Jim Mussared bd2fff6687 unix: Add build variants, analogous to boards on bare-metal.
Invoking "make" will still build the standard "micropython" executable, but
other variants are now build using, eg, "make VARIANT=minimal".  This
follows how bare-metal ports specify a particular board, and allows running
any make target (eg clean, test) with any variant.

Convenience targets (eg "make coverage") are provided to retain the old
behaviour, at least for now.

See issue #3043.
2020-01-12 10:34:23 +11:00
Nicko van Someren 4c93955b7b py/objslice: Add support for indices() method on slice objects.
Instances of the slice class are passed to __getitem__() on objects when
the user indexes them with a slice.  In practice the majority of the time
(other than passing it on untouched) is to work out what the slice means in
the context of an array dimension of a particular length.  Since Python 2.3
there has been a method on the slice class, indices(), that takes a
dimension length and returns the real start, stop and step, accounting for
missing or negative values in the slice spec.  This commit implements such
a indices() method on the slice class.

It is configurable at compile-time via MICROPY_PY_BUILTINS_SLICE_INDICES,
disabled by default, enabled on unix, stm32 and esp32 ports.

This commit also adds new tests for slice indices and for slicing unicode
strings.
2019-12-28 23:55:15 +11:00
Damien George 799b6d1e0c extmod: Consolidate FAT FS config to MICROPY_VFS_FAT across all ports.
This commit removes the Makefile-level MICROPY_FATFS config and moves the
MICROPY_VFS_FAT config to the Makefile level to replace it.  It also moves
the include of the oofatfs source files in the build from each port to a
central place in extmod/extmod.mk.

For a port to enabled VFS FAT support it should now set MICROPY_VFS_FAT=1
at the level of the Makefile.  This will include the relevant oofatfs files
in the build and set MICROPY_VFS_FAT=1 at the C (preprocessor) level.
2019-11-11 11:37:38 +11:00
Damien George 1582c7eeb0 unix,windows: Enable module weak links. 2019-10-22 16:23:43 +11:00
Jim Mussared 93bd61ca91 unix: Allow building without a manifest. 2019-10-21 23:21:18 +11:00
Milan Rossa 310b3d1b81 py: Integrate sys.settrace feature into the VM and runtime.
This commit adds support for sys.settrace, allowing to install Python
handlers to trace execution of Python code.  The interface follows CPython
as closely as possible.  The feature is disabled by default and can be
enabled via MICROPY_PY_SYS_SETTRACE.
2019-08-30 16:44:12 +10:00
stijn af5c998f37 py/modmath: Implement math.isclose() for non-complex numbers.
As per PEP 485, this function appeared in for Python 3.5.  Configured via
MICROPY_PY_MATH_ISCLOSE which is disabled by default, but enabled for the
ports which already have MICROPY_PY_MATH_SPECIAL_FUNCTIONS enabled.
2019-08-17 23:23:17 +10:00
Milan Rossa 6f0c6bd774 unix: Enable sys.atexit, triggered after the main script ends. 2019-08-15 17:31:04 +10:00
Damien George b5f33ac2cb ports: Update to work with new oofatfs version. 2019-03-05 15:56:39 +11:00
Yonatan Goldschmidt bc4f8b438b extmod/moduwebsocket: Refactor `websocket` to `uwebsocket`.
As mentioned in #4450, `websocket` was experimental with a single intended
user, `webrepl`. Therefore, we'll make this change without a weak
link `websocket` -> `uwebsocket`.
2019-02-14 00:35:45 +11:00
Paul Sokolovsky af2030dec6 unix/mpconfigport.h: Enable MICROPY_PY_UHASHLIB_MD5 for uhashlib.md5.
This will allow to e.g. implement HTTP Digest authentication.

Adds 540 bytes for x86_32, 332 for arm_thumb2 (for Unix port, which already
includes axTLS library).
2018-09-26 15:12:28 +10:00
Damien George da2d2b6d88 py/mpconfig.h: Introduce MICROPY_DEBUG_PRINTER for debugging output.
This patch in effect renames MICROPY_DEBUG_PRINTER_DEST to
MICROPY_DEBUG_PRINTER, moving its default definition from
lib/utils/printf.c to py/mpconfig.h to make it official and documented, and
makes this macro a pointer rather than the actual mp_print_t struct.  This
is done to get consistency with MICROPY_ERROR_PRINTER, and provide this
macro for use outside just lib/utils/printf.c.

Ports are updated to use the new macro name.
2018-08-02 14:04:44 +10:00
Paul Sokolovsky 771911028c unix/mpconfigport.h: Enable MICROPY_PY_UCRYPTOLIB. 2018-06-27 14:56:59 +10:00
Damien George 565f590586 ports: Enable IOBase on unix, stm32, esp8266 and esp32.
It's a core feature, in particular required for user-streams with uasyncio.
2018-06-12 12:29:26 +10:00
Damien George 1d40f12e44 unix: Support MICROPY_VFS_POSIX and enable it in coverage build.
The unix coverage build is now switched fully to the VFS implementation, ie
the uos module is the uos_vfs module.  For example, one can now sandbox uPy
to their home directory via:

    $ ./micropython_coverage

    >>> import uos
    >>> uos.umount('/') # unmount existing root VFS
    >>> vfs = uos.VfsPosix('/home/user') # create new POSIX VFS
    >>> uos.mount(vfs, '/') # mount new POSIX VFS at root

Some filesystem/OS features may no longer work with the coverage build due
to this change, and these need to be gradually fixed.

The standard unix port remains unchanged, it still uses the traditional uos
module which directly accesses the underlying host filesystem.
2018-06-06 14:28:23 +10:00
Damien George 20b4b85f72 ports: Enable MICROPY_PY_BUILTINS_ROUND_INT on selected ports. 2018-05-22 14:18:16 +10:00
Damien George 82828340a0 ports: Enable ucollections.deque on relevant ports.
These ports are all capable of running uasyncio.
2018-02-21 22:55:13 +11:00
Paul Sokolovsky ea742085ed unix/mpconfigport.h: Allow to override stackless options from commandline. 2017-12-16 20:43:04 +02:00
Paul Sokolovsky e7fc765880 unix/mpconfigport: Disable uio.resource_stream().
This function was implemented as an experiment, and was enabled only in
unix port. To remind, it allows to access arbitrary files frozen as
source modules (vs bytecode).

However, further experimentation showed that the same functionality can
be implemented with frozen bytecode. The process requires more steps, but
with suitable toolset it doesn't matter patch. This process is:

1. Convert binary files into "Python resource module" with
tools/mpy_bin2res.py.
2. Freeze as the bytecode.
3. Use micropython-lib's pkg_resources.resource_stream() to access it.

In other words, the extra step is using tools/mpy_bin2res.py (because
there would be wrapper for uio.resource_stream() anyway).

Going frozen bytecode route allows more flexibility, and same/additional
efficiency:

1. Frozen source support can be disabled altogether for additional code
savings.
2. Resources could be also accessed as a buffer, not just as a stream.

There're few caveats too:

1. It wasn't actually profiled the overhead of storing a resource in
"Python resource module" vs storing it directly, but it's assumed that
overhead is small.
2. The "efficiency" claim above applies to the case when resource
file is frozen as the bytecode. If it's not, it actually will take a
lot of RAM on loading. But in this case, the resource file should not
be used (i.e. generated) in the first place, and micropython-lib's
pkg_resources.resource_stream() implementation has the appropriate
fallback to read the raw files instead. This still poses some distribution
issues, e.g. to deployable to baremetal ports (which almost certainly
would require freezeing as the bytecode), a distribution package should
include the resource module. But for non-freezing deployment, presense
of resource module will lead to memory inefficiency.

All the discussion above reminds why uio.resource_stream() was implemented
in the first place - to address some of the issues above. However, since
then, frozen bytecode approach seems to prevail, so, while there're still
some issues to address with it, this change is being made.

This change saves 488 bytes for the unix x86_64 port.
2017-12-10 02:38:23 +02:00
Christopher Cooper 7413b3ce3e extmod/moduhashlib: Enable SHA1 hashing when using "mbedtls" library.
The SHA1 hashing functionality is provided via the "axtls" library's
implementation, and hence is unavailable when the "axtls" library is not being
used.  This change provides the same SHA1 hashing functionality when using the
"mbedtls" library by using its implementation instead.
2017-11-12 21:46:23 +02:00
Paul Sokolovsky 24c8eda744 unix: Enable MICROPY_PY_REVERSE_SPECIAL_METHODS.
With inplace methods now disabled by default, it makes sense to enable
reverse methods, as they allow for more useful features, e.g. allow
for datetime module to implement both 2 * HOUR and HOUR * 2 (where
HOUR is e.g. timedelta object).
2017-10-28 13:05:57 +03:00
David Lechner 62849b7010 py: Add config option to print warnings/errors to stderr.
This adds a new configuration option to print runtime warnings and errors to
stderr. On Unix, CPython prints warnings and unhandled exceptions to stderr,
so the unix port here is configured to use this option.

The unix port already printed unhandled exceptions on the main thread to
stderr. This patch fixes unhandled exceptions on other threads and warnings
(issue #2838) not printing on stderr.

Additionally, a couple tests needed to be fixed to handle this new behavior.
This is done by also capturing stderr when running tests.
2017-09-26 11:59:11 +10:00
Damien George 01dd7804b8 ports: Make new ports/ sub-directory and move all ports there.
This is to keep the top-level directory clean, to make it clear what is
core and what is a port, and to allow the repository to grow with new ports
in a sustainable way.
2017-09-06 13:40:51 +10:00