Commit Graph

299 Commits

Author SHA1 Message Date
Damien George
dfa563c71f py/objstr: Make empty bytes object have a null-terminating byte.
Because a lot of string processing functions assume there is a null
terminating byte, so they can work in an efficient way.

Fixes issue #3334.
2017-10-04 17:59:22 +11:00
Damien George
a3dc1b1957 all: Remove inclusion of internal py header files.
Header files that are considered internal to the py core and should not
normally be included directly are:
    py/nlr.h - internal nlr configuration and declarations
    py/bc0.h - contains bytecode macro definitions
    py/runtime0.h - contains basic runtime enums

Instead, the top-level header files to include are one of:
    py/obj.h - includes runtime0.h and defines everything to use the
        mp_obj_t type
    py/runtime.h - includes mpstate.h and hence nlr.h, obj.h, runtime0.h,
        and defines everything to use the general runtime support functions

Additional, specific headers (eg py/objlist.h) can be included if needed.
2017-10-04 12:37:50 +11:00
Paul Sokolovsky
fc9a6dd09e py/objstr: strip: Don't strip "\0" by default.
An issue was due to incorrectly taking size of default strip characters
set.
2017-09-19 21:21:12 +03:00
tll
68c28174d0 py/objstr: Add check for valid UTF-8 when making a str from bytes.
This patch adds a function utf8_check() to check for a valid UTF-8 encoded
string, and calls it when constructing a str from raw bytes.  The feature
is selectable at compile time via MICROPY_PY_BUILTINS_STR_UNICODE_CHECK and
is enabled if unicode is enabled.  It costs about 110 bytes on Thumb-2, 150
bytes on Xtensa and 170 bytes on x86-64.
2017-09-06 16:43:09 +10:00
Damien George
58321dd985 all: Convert mp_uint_t to mp_unary_op_t/mp_binary_op_t where appropriate
The unary-op/binary-op enums are already defined, and there are no
arithmetic tricks used with these types, so it makes sense to use the
correct enum type for arguments that take these values.  It also reduces
code size quite a bit for nan-boxing builds.
2017-08-29 13:16:30 +10:00
Paul Sokolovsky
37379a2974 py/objstr: startswith, endswith: Check arg to be a string.
Otherwise, it will silently get incorrect result on other values types,
including CPython tuple form like "foo.png".endswith(("png", "jpg"))
(which MicroPython doesn't support for unbloatedness).
2017-08-29 00:06:21 +03:00
Javier Candeira
35a1fea90b all: Raise exceptions via mp_raise_XXX
- Changed: ValueError, TypeError, NotImplementedError
  - OSError invocations unchanged, because the corresponding utility
    function takes ints, not strings like the long form invocation.
  - OverflowError, IndexError and RuntimeError etc. not changed for now
    until we decide whether to add new utility functions.
2017-08-13 22:52:33 +10:00
Damien George
3d25d9c7d9 py/objstr: Raise an exception for wrong type on RHS of str binary op.
The main case to catch is invalid types for the containment operator, of
the form str.__contains__(non-str).
2017-08-09 21:25:48 +10:00
Alexander Steffen
55f33240f3 all: Use the name MicroPython consistently in comments
There were several different spellings of MicroPython present in comments,
when there should be only one.
2017-07-31 18:35:40 +10:00
Damien George
9d2c72ad4f py/objstr: Remove unnecessary "sign" variable in formatting code. 2017-07-04 02:13:27 +10:00
Damien George
65417c5ad9 py/objstr: Move uPy function wrappers to just after the C function.
This matches the coding/layout style of all the other objects.
2017-07-02 23:35:42 +10:00
Damien George
326e8860ab py/objstr: Allow to compile with obj-repr D, and unicode disabled. 2017-06-08 00:40:38 +10:00
Damien George
9f85c4fe48 py/objstr: Catch case of negative "maxsplit" arg to str.rsplit().
Negative values mean no limit on the number of splits so should delegate to
the .split() method.
2017-06-02 13:07:22 +10:00
Ville Skyttä
ca16c38210 various: Spelling fixes 2017-05-29 11:36:05 +03:00
Paul Sokolovsky
9a973977bb py/objstr: Use MICROPY_FULL_CHECKS for range checking when constructing bytes.
Split this setting from MICROPY_CPYTHON_COMPAT. The idea is to be able to
keep MICROPY_CPYTHON_COMPAT disabled, but still pass more of regression
testsuite. In particular, this fixes last failing test in basics/ for
Zephyr port.
2017-04-02 21:20:07 +03:00
Damien George
6b34107537 py: Change mp_uint_t to size_t for mp_obj_str_get_data len arg. 2017-03-29 12:56:45 +11:00
Damien George
6213ad7f46 py: Convert mp_uint_t to size_t for tuple/list accessors.
This patch changes mp_uint_t to size_t for the len argument of the
following public facing C functions:

mp_obj_tuple_get
mp_obj_list_get
mp_obj_get_array

These functions take a pointer to the len argument (to be filled in by the
function) and callers of these functions should update their code so the
type of len is changed to size_t.  For ports that don't use nan-boxing
there should be no change in generate code because the size of the type
remains the same (word sized), and in a lot of cases there won't even be a
compiler warning if the type remains as mp_uint_t.

The reason for this change is to standardise on the use of size_t for
variables that count memory (or memory related) sizes/lengths.  It helps
builds that use nan-boxing.
2017-03-29 12:56:17 +11:00
Damien George
c88cfe165b py: Use size_t as len argument and return type of mp_get_index.
These values are used to compute memory addresses and so size_t is the
more appropriate type to use.
2017-03-23 16:17:40 +11:00
stijn
bf29fe2e13 py/objstr: Use better msg in bad implicit str/bytes conversion exception
Instead of always reporting some object cannot be implicitly be converted
to a 'str', even when it is a 'bytes' object, adjust the logic so that
when trying to convert str to bytes it is shown like that.
This will still report bad implicit conversion from e.g. 'int to bytes'
as 'int to str' but it will not result in the confusing
'can't convert 'str' object to str implicitly' anymore for calls like
b'somestring'.count('a').
2017-03-20 15:11:45 +11:00
Damien George
d279bcff8a py/objstr: Fix eager optimisation of str/bytes addition.
The RHS can only be returned if it is the same type as the LHS.
2017-03-16 14:30:04 +11:00
Krzysztof Blazewicz
7e480e8a30 py: Use mp_obj_get_array where sequence may be a tuple or a list. 2017-03-07 16:48:16 +11:00
Damien George
ae8d867586 py: Add iter_buf to getiter type method.
Allows to iterate over the following without allocating on the heap:
- tuple
- list
- string, bytes
- bytearray, array
- dict (not dict.keys, dict.values, dict.items)
- set, frozenset

Allows to call the following without heap memory:
- all, any, min, max, sum

TODO: still need to allocate stack memory in bytecode for iter_buf.
2017-02-16 18:38:06 +11:00
Damien George
c0d9500eee py/objstr: Convert mp_uint_t to size_t (and use int) where appropriate. 2017-02-16 16:51:16 +11:00
Damien George
90ab191b65 py/objstr: Convert some instances of mp_uint_t to size_t. 2017-02-03 13:04:56 +11:00
Damien George
7317e34383 py/objstr: Give correct behaviour when passing a dict to %-formatting.
This patch fixes two main things:
- dicts can be printed directly using '%s' % dict
- %-formatting should not crash when passed a non-dict to, eg, '%(foo)s'
2017-02-03 12:13:44 +11:00
Paul Sokolovsky
e2e663291d py/objstr: Optimize string concatenation with empty string.
In this, don't allocate copy, just return non-empty string. This helps
with a standard pattern of buffering data in case of short reads:

    buf = b""
    while ...:
        s = f.read(...)
        buf += s
        ...

For a typical case when single read returns all data needed, there won't
be extra allocation. This optimization helps uasyncio.
2017-01-27 00:49:39 +03:00
Damien George
897129a7ff py/objstr: Remove unreachable function used only for terse error msgs. 2016-09-27 15:45:42 +10:00
Damien George
5f3bda422a py: If str/bytes hash is 0 then explicitly compute it. 2016-09-02 14:49:50 +10:00
Damien George
2196799051 py/objstr: Use mp_raise_{Type,Value}Error instead of mp_raise_msg.
This patch does further refactoring using the new mp_raise_TypeError
and mp_raise_ValueError functions.
2016-08-14 16:51:54 +10:00
Paul Sokolovsky
c4a8004933 py: Get rid of assert() in method argument checking functions.
Checks for number of args removes where guaranteed by function descriptor,
self checking is replaced with mp_check_self(). In few cases, exception
is raised instead of assert.
2016-08-12 22:39:03 +03:00
Paul Sokolovsky
9e1b61dedd py/runtime: Factor out exception raising helpers.
Introduce mp_raise_msg(), mp_raise_ValueError(), mp_raise_TypeError()
instead of previous pattern nlr_raise(mp_obj_new_exception_msg(...)).
Save few bytes on each call, which are many.
2016-08-12 21:28:45 +03:00
Paul Sokolovsky
1563388001 py/objstr,objstrunicode: Fix inconistent #if indentation. 2016-08-07 15:24:57 +03:00
Paul Sokolovsky
56eb25f049 py/objstr: Make .partition()/.rpartition() methods configurable.
Default is disabled, enabled for unix port. Saves 600 bytes on x86.
2016-08-07 06:46:55 +03:00
Paul Sokolovsky
9dde6062cc py/objstr: Fix mix-signed comparison in str.center(). 2016-05-22 02:22:14 +03:00
Dave Hylands
6a60fb3cf4 py/objstr*: Properly ifdef str.center(). 2016-05-22 01:54:41 +03:00
Paul Sokolovsky
1b5abfcaae py/objstr: Implement str.center().
Disabled by default, enabled in unix port. Need for this method easily
pops up when working with text UI/reporting, and coding workalike
manually again and again counter-productive.
2016-05-22 00:13:44 +03:00
Damien George
cc80c4dd59 py/objstr: Make dedicated splitlines function, supporting diff newlines.
It now supports \n, \r and \r\n as newline separators.

Adds 56 bytes to stmhal and 80 bytes to unix x86-64.

Fixes issue #1689.
2016-05-13 12:21:32 +01:00
Paul Sokolovsky
40f0096ee7 Revert "py/objstr: .format(): Avoid call to vstr_null_terminated_str()."
This reverts commit 6de8dbb488. The change
was incorrect (correct change would require comparing with end pointer in
each if statement in the block).
2016-05-09 23:42:42 +03:00
Paul Sokolovsky
6de8dbb488 py/objstr: .format(): Avoid call to vstr_null_terminated_str().
By comparing with string end pointer instead of checking for NUL byte.
Should alleviate reallocations and fragmentation a tiny bit.
2016-05-09 21:55:09 +03:00
Damien George
12dd8df375 py/objstr: Binary type of str/bytes for buffer protocol is 'B'.
The type is an unsigned 8-bit value, since bytes objects are exactly
that.  And it's also sensible for unicode strings to return unsigned
values when accessed in a byte-wise manner (CPython does not allow this).
2016-05-07 21:18:17 +01:00
Damien George
a649d72606 py/makeqstrdata: Add special case to handle \n qstr. 2016-04-14 15:22:36 +01:00
Paul Sokolovsky
c38809e26b py/objarray: Implement "in" operator for bytearray. 2016-02-14 18:57:11 +02:00
Damien George
086d98cbde py/objstr: Make mp_obj_str_format_helper static. 2016-02-02 16:51:52 +00:00
Damien George
87e07ea943 py/objstr: For str.format, don't allocate on the heap for field name. 2016-02-02 16:26:21 +00:00
pohmelie
e3a29de1dc py/objstr: For str.format, add nested/computed fields support.
Eg: '{:{}}'.format(123, '>20')

@pohmelie was the original author of this patch, but @dpgeorge made
significant changes to reduce code size and improve efficiency.
2016-02-02 16:25:24 +00:00
Damien George
22d85ec5be py: Use new code pattern for parsing kw args with mp_arg_parse_all.
Makes code easier to read and more maintainable.
2016-01-13 15:47:56 +00:00
Damien George
5b3f0b7f39 py: Change first arg of type.make_new from mp_obj_t to mp_obj_type_t*.
The first argument to the type.make_new method is naturally a uPy type,
and all uses of this argument cast it directly to a pointer to a type
structure.  So it makes sense to just have it a pointer to a type from
the very beginning (and a const pointer at that).  This patch makes
such a change, and removes all unnecessary casting to/from mp_obj_t.
2016-01-11 00:49:27 +00:00
Damien George
4b72b3a133 py: Change type signature of builtin funs that take variable or kw args.
With this patch the n_args parameter is changed type from mp_uint_t to
size_t.
2016-01-11 00:49:27 +00:00
Damien George
a0c97814df py: Change type of .make_new and .call args: mp_uint_t becomes size_t.
This patch changes the type signature of .make_new and .call object method
slots to use size_t for n_args and n_kw (was mp_uint_t.  Makes code more
efficient when mp_uint_t is larger than a machine word.  Doesn't affect
ports when size_t and mp_uint_t have the same size.
2016-01-11 00:48:41 +00:00
Damien George
d4df8f4925 py/objstr: In str.format, handle case of no format spec for string arg.
Handles, eg, "{:>20}".format("foo"), where there is no explicit spec for
the type of the argument.
2016-01-04 13:13:39 +00:00