Protocols are nice, but there is no way for C code to verify whether
a type's "protocol" structure actually implements some particular
protocol. As a result, you can pass an object that implements the
"vfs" protocol to one that expects the "stream" protocol, and the
opposite of awesomeness ensues.
This patch adds an OPTIONAL (but enabled by default) protocol identifier
as the first member of any protocol structure. This identifier is
simply a unique QSTR chosen by the protocol designer and used by each
protocol implementer. When checking for protocol support, instead of
just checking whether the object's type has a non-NULL protocol field,
use `mp_proto_get` which implements the protocol check when possible.
The existing protocols are now named:
protocol_framebuf
protocol_i2c
protocol_pin
protocol_stream
protocol_spi
protocol_vfs
(most of these are unused in CP and are just inherited from MP; vfs and
stream are definitely used though)
I did not find any crashing examples, but here's one to give a flavor of what
is improved, using `micropython_coverage`. Before the change,
the vfs "ioctl" protocol is invoked, and the result is not intelligible
as json (but it could have resulted in a hard fault, potentially):
>>> import uos, ujson
>>> u = uos.VfsPosix('/tmp')
>>> ujson.load(u)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: syntax error in JSON
After the change, the vfs object is correctly detected as not supporting
the stream protocol:
>>> ujson.load(p)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OSError: stream operation not supported
This feature is controlled at compile time by MICROPY_PY_URE_SUB, disabled
by default.
Thanks to @dmazzella for the original patch for this feature; see #3770.
This feature is controlled at compile time by
MICROPY_PY_URE_MATCH_SPAN_START_END, disabled by default.
Thanks to @dmazzella for the original patch for this feature; see #3770.
This feature is controlled at compile time by MICROPY_PY_URE_MATCH_GROUPS,
disabled by default.
Thanks to @dmazzella for the original patch for this feature; see #3770.
It's more robust to have the version defined statically in a header file,
rather than dynamically generating it via git using a git tag. In case
git doesn't exist, or a different source control tool is used, it's
important to still have the uPy version number available.
SHORT, INT, LONG, LONGLONG, and unsigned (U*) variants are being defined.
This is done at compile using GCC-style predefined macros like
__SIZEOF_INT__. If the compiler doesn't have such defines, no such types
will be defined.
The recent implementation of the listen backlog meant that the logic to
test for readability of such a socket changed, and this commit updates the
logic to work again.
Array to hold waiting connections is in-place if backlog=1, else is a
dynamically allocated array. Incoming connections are processed FIFO
style to maintain fairness.
sizeof() can work in two ways: a) calculate size of already instantiated
structure ("sizeof variable") - in this case we already no layout; b) size
of structure decsription ("sizeof type"). In the latter case, LAYOUT_NATIVE
was assumed, but there should possibility to calculate size for other
layouts too. So, with this patch, there're now 2 forms:
uctypes.sizeof(struct)
uctypes.sizeof(struct_desc, layout)
This value is unused. It was an artifact of early draft design, but
bitfields were optimized to use scalar one-word encoding, to allow
compact encoding of typical multiple bitfields in MCU control
registers.
Since mbedtls 2.7.0 new digest functions were introduced with a "_ret"
suffix to allow the functions to return an error message (eg, if the
underlying hardware acceleration failed). These new functions must be used
instead of the old ones to prevent deprecation warnings, or link errors for
missing functions, depending on the mbedtls configuration.
This removes the need for a separate axtls build stage, and builds all
axtls object files along with other code. This simplifies and cleans up
the build process, automatically builds axtls when needed, and puts the
axtls object files in the correct $(BUILD) location.
The MicroPython axtls configuration file is provided in
extmod/axtls-include/config.h
Commit 95e70cd0ea 'time: Use 1970 epoch' changed epoch for the time
module, but not for other users. This patch does the same for the only
other core timeutils user: extmod/vfs_fat.c:fat_vfs_stat().
Other timeutils users: cc3200, esp8266 and stm32, are not changed.
Ports that don't use long ints, will still get wrong time values from
os.stat().
This saves code space in builds which use link-time optimization.
The optimization drops the untranslated strings and replaces them
with a compressed_string_t struct. It can then be decompressed to
a c string.
Builds without LTO work as well but include both untranslated
strings and compressed strings.
This work could be expanded to include QSTRs and loaded strings if
a compress method is added to C. Its tracked in #531.
The underlying socket can handling polling, and any other transparent ioctl
requests. Note that CPython handles the case of polling an ssl object by
polling the file descriptor of the underlying socket file, and that
behaviour is emulated here.
Otherwise they may be called on a socket that no longer exists.
For example, if the GC calls the finaliser on the socket and then reuses
its heap memory, the "callback" entry of the old socket may contain invalid
data. If lwIP then calls the TCP callback the code may try to call the
user callback object which is now invalid. The lwIP callbacks must be
deregistered during the closing of the socket, before all the pcb pointers
are set to NULL.
This feature is controlled at compile time by MICROPY_PY_URE_SUB, disabled
by default.
Thanks to @dmazzella for the original patch for this feature; see #3770.
This feature is controlled at compile time by
MICROPY_PY_URE_MATCH_SPAN_START_END, disabled by default.
Thanks to @dmazzella for the original patch for this feature; see #3770.
This feature is controlled at compile time by MICROPY_PY_URE_MATCH_GROUPS,
disabled by default.
Thanks to @dmazzella for the original patch for this feature; see #3770.
This will allow implementations other than axTLS.
This commit includes additions of checks and clarifications of exceptions
related to user input.
To make the interface cleaner, I've disallowed switching from encrypt to
decrypt in the same object, as this is not always possible with other
crypto libraries (not all libraries have AES_convert_key like axTLS).
Allow including crypto consts based on compilation settings. Disabled by
default to reduce code size; if one wants extra code readability, can
enable them.
The API follows guidelines of https://www.python.org/dev/peps/pep-0272/,
but is optimized for code size, with the idea that full PEP 0272
compatibility can be added with a simple Python wrapper mode.
The naming of the module follows (u)hashlib pattern.
At the bare minimum, this module is expected to provide:
* AES128, ECB (i.e. "null") mode, encrypt only
Implementation in this commit is based on axTLS routines, and implements
following:
* AES 128 and 256
* ECB and CBC modes
* encrypt and decrypt
With this patch objects are only checked that they have the stream protocol
at the start of their use as a stream, and afterwards the efficient
mp_get_stream() helper is used to extract the stream protocol C methods.
This patch changes dupterm to call the native C stream methods on the
connected stream objects, instead of calling the Python readinto/write
methods. This is much more efficient for native stream objects like UART
and webrepl and doesn't require allocating a special dupterm array.
This change is a minor breaking change from the user's perspective because
dupterm no longer accepts pure user stream objects to duplicate on. But
with the recent addition of uio.IOBase it is possible to still create such
classes just by inheriting from uio.IOBase, for example:
import uio, uos
class MyStream(uio.IOBase):
def write(self, buf):
# existing write implementation
def readinto(self, buf):
# existing readinto implementation
uos.dupterm(MyStream())
Via the config value MICROPY_PY_UHASHLIB_SHA256. Default to enabled to
keep backwards compatibility.
Also add default value for the sha1 class, to at least document its
existence.
For consistency with other modules, and to help avoid clashes with the
actual underlying functions that do the hashing (eg
crypto-algorithms/sha256.c:sha256_update).
Following other C-level protocols, this VFS protocol is added to help
abstract away implementation details of the underlying VFS in an efficient
way. As a starting point, the import_stat function is put into this
protocol so that the VFS sub-system does not need to know about every VFS
implementation in order to do an efficient stat for importing files.
In the future it might be worth adding other functions to this protocol.
This VFS component allows to mount a host POSIX filesystem within the uPy
VFS sub-system. All traditional POSIX file access then goes through the
VFS, allowing to sandbox a uPy process to a certain sub-dir of the host
system, as well as mount other filesystem types alongside the host
filesystem.
If mbedtls_ctr_drbg_seed() is available in the mbedtls bulid then so should
be mbedtls_entropy_func(). Then it's up to the port to configure a valid
entropy source, eg via MBEDTLS_ENTROPY_HARDWARE_ALT.
Otherwise the "sock" member may have an undefined value if wrap_socket
fails with an exception and exits early, and then if the finaliser runs it
will try to close an invalid stream object.
Fixes issue #3828.
