Commit Graph

110 Commits

Author SHA1 Message Date
Dan Halbert f2ebe6839c Initial MicroPython v1.21.0 merge; not compiled yet 2023-10-18 17:49:14 -04:00
Jeff Epler cd462c51b3
re-fix stream function definition 2023-09-22 14:26:36 -05:00
Jeff Epler 085cf0e3e4
make mp_stream_flush available 2023-09-22 13:52:04 -05:00
Jim Mussared 198311c780 py/stream: Add mp_stream___exit___obj that calls mp_stream_close.
There are enough places that implement __exit__ by forwarding directly to
mp_stream_close that this saves code size.

For the cases where __exit__ is a no-op, additionally make their
MP_STREAM_CLOSE ioctl handled as a no-op.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 18:49:03 +10:00
Jim Mussared a52cd5b07d py/obj: Add accessors for type slots and use everywhere.
This is a no-op, but sets the stage for changing the mp_obj_type_t
representation.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-09-19 19:06:07 +10:00
Jim Mussared 8a0ee5a5c0 py/objstr: Split mp_obj_str_from_vstr into bytes/str versions.
Previously the desired output type was specified.  Now make the type part
of the function name.  Because this function is used in a few places this
saves code size due to smaller call-site.

This makes `mp_obj_new_str_type_from_vstr` a private function of objstr.c
(which is almost the only place where the output type isn't a compile-time
constant).

This saves ~140 bytes on PYBV11.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-26 16:43:55 +10:00
Damien George e08ca78f40 py/stream: Remove mp_stream_errno and use system errno instead.
This change is made for two reasons:

1. A 3rd-party library (eg berkeley-db-1.xx, axtls) may use the system
   provided errno for certain errors, and yet MicroPython stream objects
   that it calls will be using the internal mp_stream_errno.  So if the
   library returns an error it is not known whether the corresponding errno
   code is stored in the system errno or mp_stream_errno.  Using the system
   errno in all cases (eg in the mp_stream_posix_XXX wrappers) fixes this
   ambiguity.

2. For systems that have threading the system-provided errno should always
   be used because the errno value is thread-local.

For systems that do not have an errno, the new lib/embed/__errno.c file is
provided.
2020-04-27 23:58:46 +10:00
Jim Mussared def76fe4d9 all: Use MP_ERROR_TEXT for all error messages. 2020-04-05 15:02:06 +10:00
Damien George 69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
Damien George bfbd94401d py: Make mp_obj_get_type() return a const ptr to mp_obj_type_t.
Most types are in rodata/ROM, and mp_obj_base_t.type is a constant pointer,
so enforce this const-ness throughout the code base.  If a type ever needs
to be modified (eg a user type) then a simple cast can be used.
2020-01-09 11:25:26 +11:00
Paul Sokolovsky 8fea833e3f py: Update my copyright info on some files.
Based on git history.
2019-02-06 00:19:00 +11:00
Damien George 9ab816d676 py/stream: Adjust mp_stream_posix_XXX to take void*, not mp_obj_t.
These POSIX wrappers are assumed to be passed a concrete stream object so
it is more efficient (eg on nan-boxing builds) to pass in the pointer
rather than mp_obj_t, because then the users of these functions only need
to store a void* (and mp_obj_t may be wider than a pointer).  And things
would be further improved if the stream protocol functions eventually took
a pointer as their first argument (instead of an mp_obj_t).

This patch is a step to getting ussl/axtls compiling on nan-boxing builds.

See issue #3085.
2018-08-14 17:36:08 +10:00
Damien George 34344a413f py/stream: Remove stray empty line at start of file.
This was accidentally added in 6abede2ca9
2018-06-20 16:26:12 +10:00
Damien George 2c8d130f70 py/stream: Update comment for mp_stream_write_adaptor. 2018-06-20 15:56:32 +10:00
Damien George 6abede2ca9 py/stream: Introduce and use efficient mp_get_stream to access stream_p.
The existing mp_get_stream_raise() helper does explicit checks that the
input object is a real pointer object, has a non-NULL stream protocol, and
has the desired stream C method (read/write/ioctl).  In most cases it is
not necessary to do these checks because it is guaranteed that the input
object has the stream protocol and desired C methods.  For example, native
objects that use the stream wrappers (eg mp_stream_readinto_obj) in their
locals dict always have the stream protocol (or else they shouldn't have
these wrappers in their locals dict).

This patch introduces an efficient mp_get_stream() which doesn't do any
checks and just extracts the stream protocol struct.  This should be used
in all cases where the argument object is known to be a stream.  The
existing mp_get_stream_raise() should be used primarily to verify that an
object does have the correct stream protocol methods.

All uses of mp_get_stream_raise() in py/stream.c have been converted to use
mp_get_stream() because the argument is guaranteed to be a proper stream
object.

This patch improves efficiency of stream operations and reduces code size.
2018-06-18 12:35:56 +10:00
Ayke van Laethem d43c737756 py/stream: Use uPy errno instead of system's for non-blocking check.
This is a more consistent use of errno codes.  For example, it may be that
a stream returns MP_EAGAIN but the mp_is_nonblocking_error() macro doesn't
catch this value because it checks for EAGAIN instead (which may have a
different value than MP_EAGAIN when MICROPY_USE_INTERNAL_ERRNO is enabled).
2018-05-01 15:54:50 +10:00
Damien George cf31d384f1 py/stream: Switch stream close operation from method to ioctl.
This patch moves the implementation of stream closure from a dedicated
method to the ioctl of the stream protocol, for each type that implements
closing.  The benefits of this are:

1. Rounds out the stream ioctl function, which already includes flush,
   seek and poll (among other things).

2. Makes calling mp_stream_close() on an object slightly more efficient
   because it now no longer needs to lookup the close method and call it,
   rather it just delegates straight to the ioctl function (if it exists).

3. Reduces code size and allows future types that implement the stream
   protocol to be smaller because they don't need a dedicated close method.

Code size reduction is around 200 bytes smaller for x86 archs and around
30 bytes smaller for the bare-metal archs.
2018-04-10 13:41:32 +10:00
Damien George a3dc1b1957 all: Remove inclusion of internal py header files.
Header files that are considered internal to the py core and should not
normally be included directly are:
    py/nlr.h - internal nlr configuration and declarations
    py/bc0.h - contains bytecode macro definitions
    py/runtime0.h - contains basic runtime enums

Instead, the top-level header files to include are one of:
    py/obj.h - includes runtime0.h and defines everything to use the
        mp_obj_t type
    py/runtime.h - includes mpstate.h and hence nlr.h, obj.h, runtime0.h,
        and defines everything to use the general runtime support functions

Additional, specific headers (eg py/objlist.h) can be included if needed.
2017-10-04 12:37:50 +11:00
Damien George 7885a425d7 py/stream: Remove unnecessary checks for NULL return from vstr_add_len.
The vstr argument to the calls to vstr_add_len are dynamically allocated
(ie fixed_buf=false) and so vstr_add_len will never return NULL.  So
there's no need to check for it.  Any out-of-memory errors are raised by
the call to m_renew in vstr_ensure_extra.
2017-09-21 18:22:55 +10:00
Paul Sokolovsky e3383e9352 py/stream: seek: Consistently handle negative offset for SEEK_SET.
Per POSIX, this is EINVAL, so raises OSError(EINVAL).
2017-08-20 22:02:41 +03:00
Alexander Steffen 55f33240f3 all: Use the name MicroPython consistently in comments
There were several different spellings of MicroPython present in comments,
when there should be only one.
2017-07-31 18:35:40 +10:00
Damien George 48d867b4a6 all: Make more use of mp_raise_{msg,TypeError,ValueError} helpers. 2017-06-15 11:54:41 +10:00
Ville Skyttä ca16c38210 various: Spelling fixes 2017-05-29 11:36:05 +03:00
Paul Sokolovsky 59a1201da9 all: Remove readall() method, which is equivalent to read() w/o args.
Its addition was due to an early exploration on how to add CPython-like
stream interface. It's clear that it's not needed and just takes up
bytes in all ports.
2016-11-14 00:24:22 +03:00
Paul Sokolovsky 8a49905a2f py/stream: Typo fix in comment. 2016-10-27 22:13:45 +03:00
Damien George 7d0d7215d2 py: Use mp_raise_msg helper function where appropriate.
Saves the following number of bytes of code space: 176 for bare-arm, 352
for minimal, 272 for unix x86-64, 140 for stmhal, 120 for esp8266.
2016-10-17 12:17:37 +11:00
Damien George 3a0a771730 py: Add mp_raise_OSError(errno) helper function.
This is an often used code pattern, and its use reduces code size of the
core by about 100 bytes.
2016-10-07 13:31:59 +11:00
Damien George 79ec869f95 py/stream: Remove unnecessary check for NULL return from vstr_extend.
vstr_extend will now only return NULL if the vstr is a fixed buffer, which
in this case it is not.
2016-09-22 10:50:47 +10:00
Krzysztof Blazewicz 6562076454 py/stream.c: use mp_obj_get_type in mp_get_stream_raise
In current state `mp_get_stream_raise` assumes that `self_in` is an object
and always performs a pointer derefence which may cause a segfault.

This function shall throw an exception whenever `self_in` does not implement
a stream protocol, that includes qstr's and numbers.

fixes #2331
2016-08-24 01:33:31 +03:00
Paul Sokolovsky 61e77a4e88 py/mpconfig.h: Add MICROPY_STREAMS_POSIX_API setting.
To filter out even prototypes of mp_stream_posix_*() functions, which
require POSIX types like ssize_t & off_t, which may be not available in
some ports.
2016-07-30 20:05:56 +03:00
Paul Sokolovsky 4f1b0292db py/stream: Add adapter methods with POSIX-compatible signatures.
Previoussly such read() and write() methods were used by modussl_axtls,
move to py/stream for reuse.
2016-07-30 00:25:06 +03:00
Paul Sokolovsky a60b0263ba py/stream: Implement generic flush() method, in terms of C-level ioctl. 2016-07-27 00:39:10 +03:00
Paul Sokolovsky ad9b9c7621 py/stream: Implement 2- and 3-arg write() method as an extension to CPython.
3-arg form:

stream.write(data, offset, length)

2-arg form:

stream.write(data, length)

These allow efficient buffer writing without incurring extra memory
allocation for slicing or creating memoryview() object, what is
important for low-memory ports.

All arguments must be positional. It might be not so bad idea to standardize
on 3-arg form, but 2-arg case would need check and raising an exception
anyway then, so instead it was just made to work.
2016-07-14 01:44:50 +03:00
Paul Sokolovsky 07209f8592 all: Rename mp_obj_type_t::stream_p to protocol.
It's now used for more than just stream protocol (e.g. pin protocol), so
don't use false names.
2016-06-18 18:44:57 +03:00
Paul Sokolovsky 497660fcda py/stream: Add mp_stream_close() helper function. 2016-05-20 21:18:49 +03:00
Paul Sokolovsky 7f7c84b10a py/stream: Support both "exact size" and "one underlying call" operations.
Both read and write operations support variants where either a) a single
call is made to the undelying stream implementation and returned buffer
length may be less than requested, or b) calls are repeated until requested
amount of data is collected, shorter amount is returned only in case of
EOF or error.

These operations are available from the level of C support functions to be
used by other C modules to implementations of Python methods to be used in
user-facing objects.

The rationale of these changes is to allow to write concise and robust
code to work with *blocking* streams of types prone to short reads, like
serial interfaces and sockets. Particular object types may select "exact"
vs "once" types of methods depending on their needs. E.g., for sockets,
revc() and send() methods continue to be "once", while read() and write()
thus converted to "exactly" versions.

These changes don't affect non-blocking handling, e.g. trying "exact"
method on the non-blocking socket will return as much data as available
without blocking. No data available is continued to be signaled as None
return value to read() and write().

From the point of view of CPython compatibility, this model is a cross
between its io.RawIOBase and io.BufferedIOBase abstract classes. For
blocking streams, it works as io.BufferedIOBase model (guaranteeing
lack of short reads/writes), while for non-blocking - as io.RawIOBase,
returning None in case of lack of data (instead of raising expensive
exception, as required by io.BufferedIOBase). Such a cross-behavior
should be optimal for MicroPython needs.
2016-05-18 02:41:45 +03:00
Damien George 358e5d8bad py/stream: Move uPy func obj wrappers to below their respective funcs. 2016-04-10 12:41:28 +01:00
Damien George 657aef66ff py/stream: Simplify arg extraction logic for stream_ioctl.
Saves 16 bytes of code.

Also, use mp_obj_get_int_truncated to allow integers as big as a machine
word to be passed as the value.
2016-04-10 12:37:59 +01:00
Paul Sokolovsky 558fd5d228 py/stream: ioctl(): Properly support 2-arg form. 2016-04-10 13:36:44 +03:00
Paul Sokolovsky 6c3db26ab7 py/stream: Fix signed comparison issue. 2016-04-10 13:31:52 +03:00
Paul Sokolovsky 0c97e4c414 py/stream: Add Python-level ioctl() method.
Will call underlying C virtual methods of stream interface. This isn't
intended to be added to every stream object (it's not in CPython), but
is convenient way to expose extra operation on Python side without
adding bunch of Python-level methods.
2016-04-10 12:45:46 +03:00
Paul Sokolovsky 53ad5edc01 py/stream: Fix stupid thinko with variable naming/shadowing. 2016-03-27 12:58:33 +03:00
Paul Sokolovsky 4a02a8f74d py/stream: Fix object vs ptr usecase in mp_stream_writeall(). 2016-03-24 19:43:08 +02:00
Paul Sokolovsky d4c8e626f2 py/stream: Add mp_stream_writeall() helper function.
Spools entire output buffer to a blocking stream (chunk by chunk if
needed).
2016-03-24 19:09:00 +02:00
Damien George 4b72b3a133 py: Change type signature of builtin funs that take variable or kw args.
With this patch the n_args parameter is changed type from mp_uint_t to
size_t.
2016-01-11 00:49:27 +00:00
Damien George e84325bd1d py: Add mp_get_stream_raise to factor out check for stream methods. 2015-12-09 18:47:43 +00:00
Damien George 999cedb90f py: Wrap all obj-ptr conversions in MP_OBJ_TO_PTR/MP_OBJ_FROM_PTR.
This allows the mp_obj_t type to be configured to something other than a
pointer-sized primitive type.

This patch also includes additional changes to allow the code to compile
when sizeof(mp_uint_t) != sizeof(void*), such as using size_t instead of
mp_uint_t, and various casts.
2015-11-29 14:25:35 +00:00
Damien George 4e7107a572 py: Change mp_print_strn_t func type to use size_t for the str length. 2015-11-29 14:25:04 +00:00
Paul Sokolovsky 7799410950 py/stream: Allow to reuse is_nonblocking_error(). 2015-10-18 15:39:33 +03:00
blmorris bdd78c31b6 py: Add stream_tell method, and use for unix and stmhal file tell. 2015-08-13 22:56:32 +01:00