Commit Graph

45 Commits

Author SHA1 Message Date
Dan Halbert 7a09af73ec Improve struct compatibility with CPython 2019-01-20 15:12:34 -05:00
Scott Shawcroft 933add6cd8
Support internationalisation. 2018-08-07 14:58:57 -07:00
Damien George 2daacc5cee py/modstruct: Check and prevent buffer-write overflow in struct packing.
Prior to this patch, the size of the buffer given to pack_into() was checked
for being too small by using the count of the arguments, not their actual
size.  For example, a format spec of '4I' would only check that there was 4
bytes available, not 16; and 'I' would check for 1 byte, not 4.

The pack() function is ok because its buffer is created to be exactly the
correct size.

The fix in this patch calculates the total size of the format spec at the
start of pack_into() and verifies that the buffer is large enough.  This
adds some computational overhead, to iterate through the whole format spec.
The alternative is to check during the packing, but that requires extra
code to handle alignment, and the check is anyway not needed for pack().
So to maintain minimal code size the check is done using struct_calcsize.
2017-09-01 11:11:09 +10:00
Damien George 79d5acbd01 py/modstruct: Check and prevent buffer-read overflow in struct unpacking
Prior to this patch, the size of the buffer given to unpack/unpack_from was
checked for being too small by using the count of the arguments, not their
actual size.  For example, a format spec of '4I' would only check that
there was 4 bytes available, not 16; and 'I' would check for 1 byte, not 4.

This bug is fixed in this patch by calculating the total size of the format
spec at the start of the unpacking function.  This function anyway needs to
calculate the number of items at the start, so calculating the total size
can be done at the same time.
2017-09-01 10:53:29 +10:00
Damien George 793d826d9d py/modstruct: In struct.pack, stop converting if there are no args left.
This patch makes a repeat counter behave the same as repeating the
typecode, when there are not enough args.  For example:
struct.pack('2I', 1) now behave the same as struct.pack('II', 1).
2017-09-01 10:10:51 +10:00
Alexander Steffen 55f33240f3 all: Use the name MicroPython consistently in comments
There were several different spellings of MicroPython present in comments,
when there should be only one.
2017-07-31 18:35:40 +10:00
Paul Sokolovsky af90461931 py/binary: mp_binary_get_size: Raise error on unsupported typecodes.
Previouly, we had errors checked in callers, which led to duplicate code
or missing checks in some places.
2017-01-17 22:53:06 +03:00
Damien George dffa383b06 py/modstruct: Remove unreachable code, and add comment about CPy diff.
The deleted code is unreachable because calcsize_items guarantees that
num_items corresponds to how many items there are in fmt to unpack.
2016-10-07 12:54:14 +11:00
Damien George 93c4a6a3f7 all: Remove 'name' member from mp_obj_module_t struct.
One can instead lookup __name__ in the modules dict to get the value.
2016-09-22 00:23:16 +10:00
Damien George fe3cc5bb53 py/modstruct: Use more compact mp_raise_ValueError function.
Saves a few bytes of code size.
2016-08-26 23:42:40 +10:00
Damien George 715ee9d925 py/modstruct: Allow to have "0s" in struct format. 2016-05-28 23:27:38 +01:00
Paul Sokolovsky e53fb1bf03 py/modstruct: Raise ValueError on unsupported format char. 2016-05-14 15:47:08 +03:00
Dave Hylands a17755ee8b py: Add ustruct.pack_into and unpack_from 2016-01-19 12:25:28 +00:00
Damien George 4b72b3a133 py: Change type signature of builtin funs that take variable or kw args.
With this patch the n_args parameter is changed type from mp_uint_t to
size_t.
2016-01-11 00:49:27 +00:00
Damien George 999cedb90f py: Wrap all obj-ptr conversions in MP_OBJ_TO_PTR/MP_OBJ_FROM_PTR.
This allows the mp_obj_t type to be configured to something other than a
pointer-sized primitive type.

This patch also includes additional changes to allow the code to compile
when sizeof(mp_uint_t) != sizeof(void*), such as using size_t instead of
mp_uint_t, and various casts.
2015-11-29 14:25:35 +00:00
Damien George cbf7674025 py: Add MP_ROM_* macros and mp_rom_* types and use them. 2015-11-29 14:25:04 +00:00
Paul Sokolovsky 2b080cf599 py/modstruct: Support repetition counters for all types, not just string.
This makes format specifiers ~ fully compatible with CPython.

Adds 24 bytes for stmhal port (because previosuly we had to catch and report
it's unsupported to user).
2015-10-31 18:47:08 +03:00
Paul Sokolovsky 7e66b859b2 modstruct: Raise NotImplementedError for unsupported repeat specification. 2015-07-05 22:44:14 +03:00
Paul Sokolovsky 3d3ef36e97 modstruct: Rename module to "ustruct", to allow full Python-level impl. 2015-05-04 16:53:52 +03:00
Damien George 7d414a1b52 py: Parse big-int/float/imag constants directly in parser.
Previous to this patch, a big-int, float or imag constant was interned
(made into a qstr) and then parsed at runtime to create an object each
time it was needed.  This is wasteful in RAM and not efficient.  Now,
these constants are parsed straight away in the parser and turned into
objects.  This allows constants with large numbers of digits (so
addresses issue #1103) and takes us a step closer to #722.
2015-02-08 01:57:40 +00:00
Damien George 05005f679e py: Remove mp_obj_str_builder and use vstr instead.
With this patch str/bytes construction is streamlined.  Always use a
vstr to build a str/bytes object.  If the size is known beforehand then
use vstr_init_len to allocate only required memory.  Otherwise use
vstr_init and the vstr will grow as needed.  Then use
mp_obj_new_str_from_vstr to create a str/bytes object using the vstr
memory.

Saves code ROM: 68 bytes on stmhal, 108 bytes on bare-arm, and 336 bytes
on unix x64.
2015-01-21 23:18:02 +00:00
Damien George 51dfcb4bb7 py: Move to guarded includes, everywhere in py/ core.
Addresses issue #1022.
2015-01-01 20:32:09 +00:00
Damien George 3b603f29ec Use MP_DEFINE_CONST_DICT macro to define module dicts.
This is just a clean-up of the code.  Generated code is exactly the
same.
2014-11-29 14:39:27 +00:00
Damien George 9336ee320a py: Make mp_binary_set_val work on big endian machine. 2014-10-06 15:05:35 +00:00
Damien George 42f3de924b py: Convert [u]int to mp_[u]int_t where appropriate.
Addressing issue #50.
2014-10-03 17:44:14 +00:00
Damien George 4abff7500f py: Change uint to mp_uint_t in runtime.h, stackctrl.h, binary.h.
Part of code cleanup, working towards resolving issue #50.
2014-08-30 14:59:21 +01:00
Damien George 40f3c02682 Rename machine_(u)int_t to mp_(u)int_t.
See discussion in issue #50.
2014-07-03 13:25:24 +01:00
Paul Sokolovsky 59c675a64c py: Include mpconfig.h before all other includes.
It defines types used by all other headers.

Fixes #691.
2014-06-21 22:43:22 +03:00
Emmanuel Blot f6932d6506 Prefix ARRAY_SIZE with micropython prefix MP_ 2014-06-19 18:54:34 +02:00
Paul Sokolovsky 62798831be modstruct: Add one more extension to typecodes - 'S', a pointer to C string.
Also, add comment with description of extension to CPython's typecodes.
2014-06-02 16:35:56 +03:00
Damien George ee3fd46f13 Rename configuration variables controling Python features.
Now of the form MICROPY_PY_*.  See issue #35.
2014-05-24 23:03:12 +01:00
Paul Sokolovsky 053765414c modstruct: struct_calcsize: Fix case of uninitialized var. 2014-05-20 16:34:05 +03:00
Paul Sokolovsky da9f0924ef py, unix: Add copyright for modules I worked closely on. 2014-05-13 18:41:25 +03:00
Paul Sokolovsky df94b717b4 modstruct: Implement count specifier for strings (e.g. "100s").
Infra for counts of other types is there, need last mile to be implemented.
2014-05-12 23:45:50 +03:00
Paul Sokolovsky 147c80bf7c modstruct: Use MP_OBJ_FUN_ARGS_MAX instead of -1. 2014-05-11 22:50:27 +03:00
Damien George 04b9147e15 Add license header to (almost) all files.
Blanket wide to all .c and .h files.  Some files originating from ST are
difficult to deal with (license wise) so it was left out of those.

Also merged modpyb.h, modos.h, modstm.h and modtime.h in stmhal/.
2014-05-03 23:27:38 +01:00
Damien George 6d3c5e4301 Add ARRAY_SIZE macro, and use it where possible. 2014-04-26 10:47:29 +01:00
Paul Sokolovsky 6204460461 modstruct: Initial implementation of struct.pack(). 2014-04-19 03:22:08 +03:00
Paul Sokolovsky 1355cf42f2 modstruct: Fix .calcsize() to account for struct type/alignment. 2014-04-19 03:21:06 +03:00
Damien George b11b85adaa py: Allow to pass buffer protocol flags to get_buffer helper funcs. 2014-04-18 22:59:24 +01:00
Damien George 57a4b4f178 py: Add typecode to buffer protocol.
When querying an object that supports the buffer protocol, that object
must now return a typecode (as per binary.[ch]).  This does not have to
be honoured by the caller, but can be useful for determining element
size.
2014-04-18 22:29:21 +01:00
Paul Sokolovsky 0c43cf9154 modstruct: Basic implementation of native struct alignment and types. 2014-04-11 03:59:16 +03:00
Paul Sokolovsky 6582d64d01 modstruct: Refactor to support both LE and BE packed structs. 2014-04-11 03:58:49 +03:00
Damien George bf8ae4d96e py: Fix modstruct to compile on 64-bit machine. 2014-04-10 13:53:31 +01:00
Paul Sokolovsky e9db840480 py: Start implementing "struct" module.
Only calcsize() and unpack() functions provided so far, for little-endian
byte order. Format strings don't support repition spec (like "2b3i").

Unfortunately, dealing with all the various binary type sizes and alignments
will lead to quite a bloated "binary" helper functions  - if optimizing for
speed. Need to think if using dynamic parametrized algos makes more sense.
2014-04-10 03:58:03 +03:00