Commit Graph

30 Commits

Author SHA1 Message Date
stijn cac666f38c extmod/vfs_posix_file: Fix flush handling in msvc builds.
Flushing console output in msvc builds always fails because that
output is not buffered so don't propagate that as an error (in a
simlar way as was done in 1c047742 for macOS).

Signed-off-by: stijn <stijn@ignitron.net>
2023-10-05 10:18:24 +11:00
stephanelsmith db06041d59 extmod/vfs_posix_file: Implement sys.std*.buffer objects.
Add the buffer attribute to sys.stdin, sys.stdout and sys.stderr.  This
provides raw access to underlying stdio streams for the unix port (and
others that use VfsPosix).

Signed-off-by: stephanelsmith <stephane.smith@titansensor.com>
2023-09-01 17:39:38 +10:00
stephanelsmith 1c047742a2 extmod/vfs_posix_file: Fix flush handling on macOS.
On macOS, if running micropython from subprocess.check_output, then a
stdout.flush() raises error 45.

Here's a test case.  This will run fine on linux, but crashes on macOS with
error 45.

    import sys
    import subprocess
    import tempfile
    with tempfile.NamedTemporaryFile('w') as fp:
        fp.write('''
    import sys
    sys.stdout.write('hello world')
    sys.stdout.flush()
    print('')
    ''')
        fp.flush()
        print('py3')
        o = subprocess.check_output(f'python3 {fp.name}'.split())
        print(o)
        print('upy')
        o = subprocess.check_output(f'micropython {fp.name}'.split())
        print(o)

On macOS:

    py3
    b'hello world\n'
    upy
    Traceback (most recent call last):
      File "...", line 4, in <module>
    OSError: 45

On unix:

    py3
    b'hello world\n'
    upy
    b'hello world\n'

Signed-off-by: stephanelsmith <stephane.smith@titansensor.com>
2023-09-01 17:39:38 +10:00
Damien George ef71028f77 extmod/modselect: Add optimisation to use system poll when possible.
A previous commit removed the unix-specific select module implementation
and made unix use the common one.

This commit adds an optimisation so that the system poll function is used
when polling objects that have a file descriptor.  With this optimisation
enabled, if code registers both file-descriptor-based objects, and non-
file-descriptor-based objects with select.poll() then the following occurs:

- the system poll is called for all file-descriptor-based objects with a
  timeout of 1ms

- then the bare-metal polling implementation is used for remaining objects,
  which calls into their ioctl method (which can be in C or Python)

In the case where all objects have file descriptors, the system poll is
called with the full timeout requested by the caller.  That makes it as
efficient as possible in the case everything has a file descriptor.

Benefits of this approach:

- all ports use the same select module implementation

- the unix port now supports polling of all objects and matches bare metal
  implementations

- it's still efficient for existing cases where only files and sockets are
  polled (on unix)

- the bare metal implementation does not change

- polling of SSL objects will now work on unix by calling in to the ioctl
  method on SSL objects (this is required for asyncio ssl support)

Note that extmod/vfs_posix_file.c has poll disable when the optimisation is
enabled, because the code is not reachable when the optimisation is used.

Signed-off-by: Damien George <damien@micropython.org>
2023-08-07 12:11:40 +10:00
Damien George 22106bf2fa extmod/vfs_posix_file: Add poll support for missing ERR,HUP,NVAL values.
Signed-off-by: Damien George <damien@micropython.org>
2023-08-06 11:54:06 +10:00
Jim Mussared 198311c780 py/stream: Add mp_stream___exit___obj that calls mp_stream_close.
There are enough places that implement __exit__ by forwarding directly to
mp_stream_close that this saves code size.

For the cases where __exit__ is a no-op, additionally make their
MP_STREAM_CLOSE ioctl handled as a no-op.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 18:49:03 +10:00
Jim Mussared f5f9edf645 all: Rename UMODULE to MODULE in preprocessor/Makefile vars.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:11 +10:00
stijn 9ae8d38204 extmod/vfs_posix_file: Implement finaliser for files.
Prevent handle leaks when file objects aren't closed explicitly and
fix some MICROPY_CPYTHON_COMPAT issues: this wasn't properly adhered
to because #ifdef was used so it was always on, and closing files
multiple times should be avoided unconditionally.
2022-09-19 23:44:50 +10:00
Jim Mussared 94beeabd2e py/obj: Convert make_new into a mp_obj_type_t slot.
Instead of being an explicit field, it's now a slot like all the other
methods.

This is a marginal code size improvement because most types have a make_new
(100/138 on PYBV11), however it improves consistency in how types are
declared, removing the special case for make_new.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:15 +10:00
Jim Mussared 6da41b5900 py/obj: Merge getiter and iternext mp_obj_type_t slots.
The goal here is to remove a slot (making way to turn make_new into a slot)
as well as reduce code size by the ~40 references to mp_identity_getiter
and mp_stream_unbuffered_iter.

This introduces two new type flags:
- MP_TYPE_FLAG_ITER_IS_ITERNEXT: This means that the "iter" slot in the
  type is "iternext", and should use the identity getiter.
- MP_TYPE_FLAG_ITER_IS_CUSTOM: This means that the "iter" slot is a pointer
  to a mp_getiter_iternext_custom_t instance, which then defines both
  getiter and iternext.

And a third flag that is the OR of both, MP_TYPE_FLAG_ITER_IS_STREAM: This
means that the type should use the identity getiter, and
mp_stream_unbuffered_iter as iternext.

Finally, MP_TYPE_FLAG_ITER_IS_GETITER is defined as a no-op flag to give
the default case where "iter" is "getiter".

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:13 +10:00
Jim Mussared 9dce82776d all: Remove unnecessary locals_dict cast.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:01 +10:00
Jim Mussared 662b9761b3 all: Make all mp_obj_type_t defs use MP_DEFINE_CONST_OBJ_TYPE.
In preparation for upcoming rework of mp_obj_type_t layout.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:01 +10:00
Damien George 8f4c108025 all: Remove MICROPY_PY_IO_FILEIO config option.
Since commit e65d1e69e8 there is no longer an
io.FileIO class, so this option is no longer needed.

This option also controlled whether or not files supported being opened in
binary mode (eg 'rb'), and could, if disabled, lead to confusion as to why
opening a file in binary mode silently did the wrong thing (it would just
open in text mode if MICROPY_PY_IO_FILEIO was disabled).

The various VFS implementations (POSIX, FAT, LFS) were the only places
where enabling this option made a difference, and in almost all cases where
one of these filesystems were enabled, MICROPY_PY_IO_FILEIO was also
enabled.  So it makes sense to just unconditionally enable this feature
(ability to open a file in binary mode) in all cases, and so just remove
this config option altogether.  That makes configuration simpler and means
binary file support always exists (and opening a file in binary mode is
arguably more fundamental than opening in text mode, so if anything should
be configurable then it should be the ability to open in text mode).

Signed-off-by: Damien George <damien@micropython.org>
2022-08-18 11:54:17 +10:00
Damien George 237a393bec extmod/vfs_posix_file: Remove unused MICROPY_VFS_POSIX_FILE.
This was made obsolete by 2b409ef8a4

Signed-off-by: Damien George <damien@micropython.org>
2022-08-18 11:48:45 +10:00
Jim Mussared e65d1e69e8 py/modio: Remove FileIO and TextIOWrapper from io module.
On ports with more than one filesystem, the type will be wrong, for example
if using LFS but FAT enabled, then the type will be FAT.  So it's not
possible to use these classes to identify a file object type.

Furthermore, constructing an io.FileIO currently crashes on FAT, and
make_new isn't supported on LFS.

And the io.TextIOWrapper class does not match CPython at all.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-07-26 17:58:01 +10:00
Andrew Leech 2ceeabf180 extmod/vfs_posix_file: Support MP_STREAM_POLL in vfs_posix_file_ioctl.
Allows asyncio reading of sys.stdin when MICROPY_PY_USELECT is used in the
build configuration.
2021-10-19 22:47:18 +11:00
Damien George 26b4ef4c46 extmod/vfs_posix_file: Allow closing an already closed file.
Signed-off-by: Damien George <damien@micropython.org>
2021-02-11 22:54:41 +11:00
Jim Mussared def76fe4d9 all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
David Lechner 9418611c8a unix: Implement PEP 475 to retry syscalls failing with EINTR.
https://www.python.org/dev/peps/pep-0475/

This implements something similar to PEP 475 on the unix port, and for the
VfsPosix class.

There are a few differences from the CPython implementation:
- Since we call mp_handle_pending() between any ENITR's, additional
  functions could be called if MICROPY_ENABLE_SCHEDULER is enabled, not
  just signal handlers.
- CPython only handles signal on the main thread, so other threads will
  raise InterruptedError instead of retrying.  On MicroPython,
  mp_handle_pending() will currently raise exceptions on any thread.

A new macro MP_HAL_RETRY_SYSCALL is introduced to reduce duplicated code
and ensure that all instances behave the same.  This will also allow other
ports that use POSIX-like system calls (and use, eg, VfsPosix) to provide
their own implementation if needed.
2020-03-27 14:40:46 +11:00
David Lechner 58d9a4815d extmod/vfs_posix_file: Include unistd.h to get STD{IN,OUT,ERR}_FILENO. 2020-03-25 00:59:39 +11:00
Damien George 2cdf1d25f5 unix: Remove custom file implementation to use extmod's VFS POSIX one.
The implementation in extmod/vfs_posix_file.c is now equivalent to that in
ports/unix/file.c, so remove the latter and use the former instead.
2020-03-18 21:01:07 +11:00
Damien George 68b1bc2042 extmod/vfs_posix_file: Lock GIL when writing and allow stdio flush.
Also support MP_STREAM_GET_FILENO ioctl.  The stdio flush change was done
previously for the unix port in 3e0b46b9af.
These changes make this POSIX file implementation equivalent to the unix
file implementation.
2020-03-18 21:01:07 +11:00
Damien George ad9a0ec8ab all: Convert exceptions to use mp_raise_XXX helpers in remaining places. 2020-03-18 17:26:19 +11:00
Damien George 69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
David Lechner 35f66d38b8 extmod/vfs_posix: Release GIL during system calls.
This releases the GIL during syscalls that could block.
2020-01-26 23:26:14 +11:00
Damien George 09376f0e47 py: Introduce MP_ROM_NONE macro for ROM to refer to None object.
This helps to prevent mistakes, and allows easily changing the ROM value of
None if needed.
2019-12-27 22:51:17 +11:00
Damien George c13f9f209d all: Convert nlr_raise(mp_obj_new_exception_msg(x)) to mp_raise_msg(x).
This helper function was added a while ago and these are the remaining
cases to convert, to save a bit of code size.
2019-11-05 11:35:45 +11:00
Damien George 0cc8910bc5 extmod: Give vars/funcs unique names so STATIC can be set to nothing.
Fixes issue #5018.
2019-08-20 15:21:09 +10:00
Damien George f03601779e extmod: Convert legacy uppercase macro names to lowercase. 2019-02-12 14:54:51 +11:00
Damien George 8d82b0edbd extmod: Add VfsPosix filesystem component.
This VFS component allows to mount a host POSIX filesystem within the uPy
VFS sub-system.  All traditional POSIX file access then goes through the
VFS, allowing to sandbox a uPy process to a certain sub-dir of the host
system, as well as mount other filesystem types alongside the host
filesystem.
2018-06-06 14:28:23 +10:00