Commit Graph

387 Commits

Author SHA1 Message Date
Damien George a853fff838 py/obj.h: Fix mp_seq_replace_slice_no_grow to use memmove not memcpy.
Because the argument arrays may overlap, as show by the new tests in this
commit.

Also remove the debugging comments for these macros, add a new comment
about overlapping regions, and separate the macros by blank lines to make
them easier to read.

Fixes issue #6244.

Signed-off-by: Damien George <damien@micropython.org>
2020-07-21 01:05:59 +10:00
Damien George 332d83343f py: Rework mp_convert_member_lookup to properly handle built-ins.
This commit fixes lookups of class members to make it so that built-in
functions that are used as methods/functions of a class work correctly.

The mp_convert_member_lookup() function is pretty much completely changed
by this commit, but for the most part it's just reorganised and the
indenting changed.  The functional changes are:

- staticmethod and classmethod checks moved to later in the if-logic,
  because they are less common and so should be checked after the more
  common cases.

- The explicit mp_obj_is_type(member, &mp_type_type) check is removed
  because it's now subsumed by other, more general tests in this function.

- MP_TYPE_FLAG_BINDS_SELF and MP_TYPE_FLAG_BUILTIN_FUN type flags added to
  make the checks in this function much simpler (now they just test this
  bit in type->flags).

- An extra check is made for mp_obj_is_instance_type(type) to fix lookup of
  built-in functions.

Fixes #1326 and #6198.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-30 23:55:32 +10:00
Damien George d06ae1d2b1 py/obj.h: Make existing MP_TYPE_FLAG_xxx macros sequential.
There's no reason to have them non-sequential, this was likely a typo from
commit 9ec1caf42e.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-30 23:55:32 +10:00
Damien George 9f911d822e py/objcomplex: Add mp_obj_get_complex_maybe for use in complex bin-op.
This allows complex binary operations to fail gracefully with unsupported
operation rather than raising an exception, so that special methods work
correctly.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-27 01:03:10 +10:00
Damien George 456a3abe8d py/obj.h: Add public mp_obj_is_dict_or_ordereddict() helper macro.
And use it in py/objdict.c instead of mp_obj_is_dict_type.

Signed-off-by: Damien George <damien@micropython.org>
2020-06-24 12:05:00 +10:00
Damien George f3062b5cbd py/obj.h: Clarify comments about mp_map_t is_fixed and is_ordered.
Long ago, prior to 0ef01d0a75, fixed and
ordered maps were the same setting with the "table_is_fixed_array" member
of mp_map_t.  But these settings are actually independent, and it is
possible to have is_fixed=1, is_ordered=0 (although this can currently
only be done by tools/cc1).  So update the comments to reflect this.
2020-06-10 22:27:10 +10:00
Andrew Leech 28370c0450 py/objtype: Add __dict__ attribute for class objects.
The behavior mirrors the instance object dict attribute where a copy of the
local attributes are provided (unless the dict is read-only, then that dict
itself is returned, as an optimisation).  MicroPython does not support
modifying this dict because the changes will not be reflected in the class.

The feature is only enabled if MICROPY_CPYTHON_COMPAT is set, the same as
the instance version.
2020-06-10 21:58:13 +10:00
Jim Mussared 57fce3bdb2 py/objdict: Fix popitem for ordered dicts.
The popitem method wasn't implemented for ordered dicts and would result in
an invalid state.

Fixes issue #5956.
2020-04-27 23:53:17 +10:00
stijn 0ba68f8a1d all: Fix implicit floating point promotion.
Initially some of these were found building the unix coverage variant on
MacOS because that build uses clang and has -Wdouble-promotion enabled, and
clang performs more vigorous promotion checks than gcc.  Additionally the
codebase has been compiled with clang and msvc (the latter with warning
level 3), and with MICROPY_FLOAT_IMPL_FLOAT to find the rest of the
conversions.

Fixes are implemented either as explicit casts, or by using the correct
type, or by using one of the utility functions to handle floating point
casting; these have been moved from nativeglue.c to the public API.
2020-04-18 22:36:14 +10:00
Damien George e292296d52 py/objexcept: Remove optional TimeoutError exception.
TimeoutError was added back in 077812b2ab for
the cc3200 port. In f522849a4d the cc3200
port enabled use of it in the socket module aliased to socket.timeout.  So
it was never added to the builtins.  Then it was replaced by
OSError(ETIMEDOUT) in 047af9b10b.

The esp32 port enables this exception, since the very beginning of that
port, but it could never be accessed because it's not in builtins.

It's being removed: 1) to not encourage its use; 2) because there are a lot
of other OSError subclasses which are not defined at all, and having
TimeoutError is a bit inconsistent.

Note that ports can add anything to the builtins via MICROPY_PORT_BUILTINS.
And they can also define their own exceptions using the
MP_DEFINE_EXCEPTION() macro.
2020-04-09 16:09:38 +10:00
Jim Mussared 85858e72df py/objexcept: Allow compression of exception message text.
The decompression of error-strings is only done if the string is accessed
via printing or via er.args.  Tests are added for this feature to ensure
the decompression works.
2020-04-05 15:02:06 +10:00
Damien George 69661f3343 all: Reformat C and Python source code with tools/codeformat.py.
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28 10:33:03 +11:00
Damien George 9344e876bb py/objtype: Allow mp_instance_cast_to_native_base to take native obj.
And rename it to mp_obj_cast_to_native_base() to indicate this.  This
allows users of this function to easily support native and native-subclass
objects in the same way (by just passing the object through this function).
2020-02-21 13:20:12 +11:00
Damien George 1fccda049f py/objexcept: Rename mp_obj_new_exception_msg_varg2 to ..._vlist.
Follow up to recent commit ad7213d3c3, the
name "varg2" is misleading, vlist describes better that the argument is a
va_list.  This name also matches CircuitPython, which already has such
helper functions.
2020-02-18 21:00:42 +11:00
David Lechner f020eac6a8 py/obj.h: Remove TODO idea comment about truncated mp_map_t.
It was suggested to move this to a GitHub issue rather than keep it in the
code, which isn't really sustainable for all ideas.
2020-02-13 13:14:17 +11:00
Damien George ad7213d3c3 py: Add mp_raise_msg_varg helper and use it where appropriate.
This commit adds mp_raise_msg_varg(type, fmt, ...) as a helper for
nlr_raise(mp_obj_new_exception_msg_varg(type, fmt, ...)).  It makes the
C-level API for raising exceptions more consistent, and reduces code size
on most ports:

   bare-arm:   +28 +0.042%
minimal x86:  +100 +0.067%
   unix x64:   -56 -0.011%
unix nanbox:  -300 -0.068%
      stm32:  -204 -0.054% PYBV10
     cc3200:    +0 +0.000%
    esp8266:   -64 -0.010% GENERIC
      esp32:  -104 -0.007% GENERIC
        nrf:  -136 -0.094% pca10040
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2020-02-13 11:52:40 +11:00
Damien George 9ec1caf42e py: Expand type equality flags to 3 separate ones, fix bool/namedtuple.
Both bool and namedtuple will check against other types for equality; int,
float and complex for bool, and tuple for namedtuple.  So to make them work
after the recent commit 3aab54bf43 they would
need MP_TYPE_FLAG_NEEDS_FULL_EQ_TEST set.  But that makes all bool and
namedtuple equality checks less efficient because mp_obj_equal_not_equal()
could no longer short-cut x==x, and would need to try __ne__.  To improve
this, this commit splits the MP_TYPE_FLAG_NEEDS_FULL_EQ_TEST flags into 3
separate flags to give types more fine-grained control over how their
equality behaves.  These new flags are then used to fix bool and namedtuple
equality.

Fixes issue #5615 and #5620.
2020-02-11 11:06:00 +11:00
Nicko van Someren 3aab54bf43 py: Support non-boolean results for equality and inequality tests.
This commit implements a more complete replication of CPython's behaviour
for equality and inequality testing of objects.  This addresses the issues
discussed in #5382 and a few other inconsistencies.  Improvements over the
old code include:

- Support for returning non-boolean results from comparisons (as used by
  numpy and others).
- Support for non-reflexive equality tests.
- Preferential use of __ne__ methods and MP_BINARY_OP_NOT_EQUAL binary
  operators for inequality tests, when available.
- Fallback to op2 == op1 or op2 != op1 when op1 does not implement the
  (in)equality operators.

The scheme here makes use of a new flag, MP_TYPE_FLAG_NEEDS_FULL_EQ_TEST,
in the flags word of mp_obj_type_t to indicate if various shortcuts can or
cannot be used when performing equality and inequality tests.  Currently
four built-in classes have the flag set: float and complex are
non-reflexive (since nan != nan) while bytearray and frozenszet instances
can equal other builtin class instances (bytes and set respectively).  The
flag is also set for any new class defined by the user.

This commit also includes a more comprehensive set of tests for the
behaviour of (in)equality operators implemented in special methods.
2020-01-30 14:53:07 +11:00
Damien George c3450effd4 py/objtype: Make mp_obj_type_t.flags constants public, moved to obj.h. 2020-01-30 14:53:07 +11:00
Yonatan Goldschmidt d9433d3e94 py/obj.h: Add and use mp_obj_is_bool() helper.
Commit d96cfd13e3 introduced a regression in
testing for bool objects, that such objects were in some cases no longer
recognised and bools, eg when using mp_obj_is_type(o, &mp_type_bool), or
mp_obj_is_integer(o).

This commit fixes that problem by adding mp_obj_is_bool(o).  Builds with
MICROPY_OBJ_IMMEDIATE_OBJS enabled check if the object is any of the const
True or False objects.  Builds without it use the old method of ->type
checking, which compiles to smaller code (compared with the former
mentioned method).

Fixes #5538.
2020-01-24 10:53:45 +11:00
Yonatan Goldschmidt 176ab99180 py/objint: Add mp_obj_int_get_uint_checked() helper.
Can be used where mp_obj_int_get_checked() will overflow due to the
sign-bit solely.  This returns an mp_uint_t, so it also verifies the given
integer is not negative.

Currently implemented only for mpz configurations.
2020-01-14 23:35:22 +11:00
Damien George d96cfd13e3 py/obj: Add MICROPY_OBJ_IMMEDIATE_OBJS option to reduce code size.
This option (enabled by default for object representation A, B, C) makes
None/False/True objects immediate objects, ie they are no longer a concrete
object in ROM but are rather just values, eg None=0x6 for representation A.

Doing this saves a considerable amount of code size, due to these objects
being widely used:

   bare-arm:  -392 -0.591%
minimal x86:  -252 -0.170% [incl +52(data)]
   unix x64:  -624 -0.125% [incl -128(data)]
unix nanbox:    +0 +0.000%
      stm32: -1940 -0.510% PYBV10
     cc3200: -1216 -0.659%
    esp8266:  -404 -0.062% GENERIC
      esp32:  -732 -0.064% GENERIC[incl +48(data)]
        nrf:  -988 -0.675% pca10040
       samd:  -564 -0.556% ADAFRUIT_ITSYBITSY_M4_EXPRESS

Thanks go to @Jongy aka Yonatan Goldschmidt for the idea.
2020-01-13 01:01:45 +11:00
Damien George 6f0c83f6e1 py/obj.h: Redefine qstr object encoding to add immediate obj encoding.
This commit adjusts the definition of qstr encoding in all object
representations by taking a single bit from the qstr space and using it to
distinguish between qstrs and a new kind of literal object: immediate
objects.  In other words, the qstr space is divided in two pieces, one half
for qstrs and the other half for immediate objects.

There is still enough room for qstr values (29 bits in representation A on
a 32-bit architecture, and 19 bits in representation C) and the new
immediate objects can be used for things like None, False and True.
2020-01-13 01:01:45 +11:00
Damien George bfbd94401d py: Make mp_obj_get_type() return a const ptr to mp_obj_type_t.
Most types are in rodata/ROM, and mp_obj_base_t.type is a constant pointer,
so enforce this const-ness throughout the code base.  If a type ever needs
to be modified (eg a user type) then a simple cast can be used.
2020-01-09 11:25:26 +11:00
Nicko van Someren 10709846f3 py/objslice: Inline fetching of slice paramters in str_subscr().
To reduce code size.
2019-12-29 00:06:02 +11:00
Nicko van Someren 4c93955b7b py/objslice: Add support for indices() method on slice objects.
Instances of the slice class are passed to __getitem__() on objects when
the user indexes them with a slice.  In practice the majority of the time
(other than passing it on untouched) is to work out what the slice means in
the context of an array dimension of a particular length.  Since Python 2.3
there has been a method on the slice class, indices(), that takes a
dimension length and returns the real start, stop and step, accounting for
missing or negative values in the slice spec.  This commit implements such
a indices() method on the slice class.

It is configurable at compile-time via MICROPY_PY_BUILTINS_SLICE_INDICES,
disabled by default, enabled on unix, stm32 and esp32 ports.

This commit also adds new tests for slice indices and for slicing unicode
strings.
2019-12-28 23:55:15 +11:00
Damien George d56bc6e03d py/obj.h: Use 32-bit shift in MP_OBJ_NEW_QSTR calc for obj-repr D.
The qst value is always small enough to fit in 31-bits (even less) and
using a 32-bit shift rather than a 64-bit shift reduces code size by about
300 bytes.
2019-12-27 23:33:34 +11:00
Damien George d97b40bdaa py: Introduce MP_ROM_FALSE/MP_ROM_TRUE for ROM to refer to bool objects.
This helps to prevent mistakes, and allows easily changing the ROM value of
False/True if needed.
2019-12-27 22:54:20 +11:00
Damien George 09376f0e47 py: Introduce MP_ROM_NONE macro for ROM to refer to None object.
This helps to prevent mistakes, and allows easily changing the ROM value of
None if needed.
2019-12-27 22:51:17 +11:00
Damien George 5e431188db py/obj.h: Remove comments about additional mp_buffer_info_t entries.
These entries are unlikely to be needed, so remove them to clean up the
struct definition.
2019-12-20 23:36:17 +11:00
Damien George 01e5802ee3 py: Remove 3 obsolete commented-out lines from header files. 2019-11-26 21:36:41 +11:00
Andrew Leech cf22f4793c py: Allow registration of modules at their definition.
During make, makemoduledefs.py parses the current builds c files for
MP_REGISTER_MODULE(module_name, obj_module, enabled_define)

These are used to generate a header with the required entries for
"mp_rom_map_elem_t mp_builtin_module_table[]" in py/objmodule.c
2019-03-08 22:46:43 +11:00
Damien George 02cc288edb py: Add independent config for debugging sentinel object values.
The new compile-time option is MICROPY_DEBUG_MP_OBJ_SENTINELS, disabled by
default.  This is to allow finer control of whether this debugging feature
is enabled or not (because, for example, this setting must be the same for
mpy-cross and the MicroPython main code when using native code generation).
2019-03-08 15:53:04 +11:00
Damien George 7bc71f5446 py/objfun: Make fun_data arg of mp_obj_new_fun_asm() a const pointer. 2019-02-20 13:14:03 +11:00
Damien George bf352047de py/obj.h: Remove obsolete mp_obj_new_fun_viper() declaration. 2019-02-20 13:06:35 +11:00
Damien George 054dd33eba py: Downcase MP_xxx_SLOT_IS_FILLED inline functions. 2019-02-12 14:54:51 +11:00
Damien George eee1e8841a py: Downcase all MP_OBJ_IS_xxx macros to make a more consistent C API.
These macros could in principle be (inline) functions so it makes sense to
have them lower case, to match the other C API functions.

The remaining macros that are upper case are:
- MP_OBJ_TO_PTR, MP_OBJ_FROM_PTR
- MP_OBJ_NEW_SMALL_INT, MP_OBJ_SMALL_INT_VALUE
- MP_OBJ_NEW_QSTR, MP_OBJ_QSTR_VALUE
- MP_OBJ_FUN_MAKE_SIG
- MP_DECLARE_CONST_xxx
- MP_DEFINE_CONST_xxx

These must remain macros because they are used when defining const data (at
least, MP_OBJ_NEW_SMALL_INT is so it makes sense to have
MP_OBJ_SMALL_INT_VALUE also a macro).

For those macros that have been made lower case, compatibility macros are
provided for the old names so that users do not need to change their code
immediately.
2019-02-12 14:54:51 +11:00
Damien George 5089b3ffb6 py/obj.h: Explicitly cast args to uint32_t in MP_OBJ_FUN_MAKE_SIG.
For architectures where size_t is less than 32 bits (eg 16 bits) the args
must be casted to uint32_t so the left shift will work.  For architectures
where size_t is greater than 32 bits (eg 64 bits) this new casting will not
lose any bits because the end result must anyway fit in a uint32_t.
2019-01-25 16:03:05 +11:00
Damien George 69e7903904 py/obj.h: Use uint64_t instead of mp_int_t in repr-D MP_OBJ_IS_x macros.
This follows how it's already done in MP_OBJ_IS_OBJ: the objects are
considered 64-bit unsigned ints for the purpose of bitwise manipulation.
2018-10-01 16:36:46 +10:00
Damien George cc2bd63c57 py/emitnative: Implement yield and yield-from in native emitter.
This commit adds first class support for yield and yield-from in the native
emitter, including send and throw support, and yields enclosed in exception
handlers (which requires pulling down the NLR stack before yielding, then
rebuilding it when resuming).

This has been fully tested and is working on unix x86 and x86-64, and
stm32.  Also basic tests have been done with the esp8266 port.  Performance
of existing native code is unchanged.
2018-10-01 13:31:11 +10:00
stijn 57a7d5be9a py: Fix msvc C++ compiler warnings with MP_OBJ_FUN_MAKE_SIG macro.
When obj.h is compiled as C++ code, the cl compiler emits a warning about
possibly unsafe mixing of size_t and bool types in the or operation in
MP_OBJ_FUN_MAKE_SIG.  Similarly there's an implicit narrowing integer
conversion in runtime.h.  This commit fixes this by being explicit.
2018-09-26 15:34:59 +10:00
Damien George 9f241ef398 py: Optimise call to mp_arg_check_num by compressing fun signature.
With 5 arguments to mp_arg_check_num(), some architectures need to pass
values on the stack.  So compressing n_args_min, n_args_max, takes_kw into
a single word and passing only 3 arguments makes the call more efficient,
because almost all calls to this function pass in constant values.  Code
size is also reduced by a decent amount:

   bare-arm:  -116
minimal x86:   -64
   unix x64:  -256
unix nanbox:  -112
      stm32:  -324
     cc3200:  -192
    esp8266:  -192
      esp32:  -144
2018-09-14 13:39:17 +10:00
Damien George 4a1edd8382 py/obj.h: Give compile error if using obj repr D with single-prec float.
Object representation D only works with no floats, or double precision
floats.
2018-07-08 23:45:05 +10:00
Damien George 4cd853fbd2 py/objmodule: Make mp_obj_module_get_globals an inline function.
Because this function is simple it saves code size to have it inlined.
Being an auxiliary helper function (and only used in the py/ core) the
argument should always be an mp_obj_module_t*, so there's no need for the
assert (and having it would require including assert.h in obj.h).
2018-07-08 22:27:39 +10:00
Damien George d9cdb880ff py/objdict: Make mp_obj_dict_get_map an inline function.
It's a very simple function and saves code, and improves efficiency, by
being inline.  Note that this is an auxiliary helper function and so
doesn't need mp_check_self -- that's used for functions that can be
accessed directly from Python code (eg from a method table).
2018-07-08 22:27:05 +10:00
Nicko van Someren d66c33cbd6 py/obj.h: Fix broken build for object repr C when float disabled.
Fixes issue #3914.
2018-07-03 09:51:08 +10:00
Damien George b488a4a848 py/objgenerator: Eliminate need for mp_obj_gen_wrap wrapper instances.
For generating functions there is no need to wrap the bytecode function in
a generator wrapper instance.  Instead the type of the bytecode function
can be changed to mp_type_gen_wrap.  This reduces code size and saves a
block of GC heap RAM for each generator.
2018-07-02 15:30:57 +10:00
Damien George db5d8c97f1 py/obj.h: Introduce a "flags" entry in mp_obj_type_t. 2018-06-08 11:48:25 +10:00
Damien George 1427f8f593 py/stream: Move definition of mp_stream_p_t from obj.h to stream.h.
Since a long time now, mp_obj_type_t no longer refers explicitly to
mp_stream_p_t but rather to an abstract "const void *protocol".  So there's
no longer any need to define mp_stream_p_t in obj.h and it can go with all
its associated definitions in stream.h.  Pretty much all users of this type
will already include the stream header.
2018-06-04 16:53:17 +10:00
Damien George 6b4b6d388b py/obj.h: Fix math.e constant for nan-boxing builds.
Due to a typo, math.e was too small by around 6e-11.
2018-05-01 23:25:18 +10:00