djsundog/circuitpython
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

py/objtype: Optimise instance get/set/del by skipping special accessors.

Browse Source 
This patch is a code optimisation, trading text bytes for speed.  On
pyboard it's an increase of 0.06% in code size for a gain (in pystone
performance) of roughly 6.5%.

The patch optimises load/store/delete of attributes in user defined classes
by not looking up special accessors (@property, __get__, __delete__,
__set__, __setattr__ and __getattr_) if they are guaranteed not to exist in
the class.

Currently, if you do my_obj.foo() then the runtime has to do a few checks
to see if foo is a property or has __get__, and if so delegate the call.
And for stores things like my_obj.foo = 1 has to first check if foo is a
property or has __set__ defined on it.

Doing all those checks each and every time the attribute is accessed has a
performance penalty.  This patch eliminates all those checks for cases when
it's guaranteed that the checks will always fail, ie no attributes are
properties nor have any special accessor methods defined on them.

To make this guarantee it checks all attributes of a user-defined class
when it is first created.  If any of the attributes of the user class are
properties or have special accessors, or any of the base classes of the
user class have them, then it sets a flag in the class to indicate that
special accessors must be checked for.  Then in the load/store/delete code
it checks this flag to see if it can take the shortcut and optimise the
lookup.

It's an optimisation that's pretty widely applicable because it improves
lookup performance for all methods of user defined classes, and stores of
attributes, at least for those that don't have special accessors.  And, it
allows to enable descriptors with minimal additional runtime overhead if
they are not used for a particular user class.

There is one restriction on dynamic class creation that has been introduced
by this patch: a user-defined class cannot go from zero special accessors
to one special accessor (or more) after that class has been subclassed.  If
the script attempts this an AttributeError is raised (see addition to
tests/misc/non_compliant.py for an example of this case).

The cost in code space bytes for the optimisation in this patch is:

   unix x64:  +528
unix nanbox:  +508
      stm32:  +192
     cc3200:  +200
    esp8266:  +332
      esp32:  +244

Performance tests that were done:

- on unix x86-64, pystone improved by about 5%
- on pyboard, pystone improved by about 6.5%, from 1683 up to 1794
- on pyboard, bm_chaos (from CPython benchmark suite) improved by about 5%
- on esp32, pystone improved by about 30% (but there are caching effects)
- on esp32, bm_chaos improved by about 11%
This commit is contained in:
Damien George 2018-05-25 17:09:54 +10:00
parent bace1a16d0
commit 36c1052183
5 changed files with 161 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
py/mpconfig.h
View File

@ -706,14 +706,16 @@ typedef double mp_float_t;
#define MICROPY_PY_FUNCTION_ATTRS (0)
#endif
// Whether to support descriptors (__get__ and __set__)
// This costs some code size and makes all load attrs and store attrs slow
// Whether to support the descriptors __get__, __set__, __delete__
// This costs some code size and makes load/store/delete of instance
// attributes slower for the classes that use this feature
#ifndef MICROPY_PY_DESCRIPTORS
#define MICROPY_PY_DESCRIPTORS (0)
#endif
// Whether to support class __delattr__ and __setattr__ methods
// This costs some code size and makes all del attrs and store attrs slow
// This costs some code size and makes store/delete of instance
// attributes slower for the classes that use this feature
#ifndef MICROPY_PY_DELATTR_SETATTR
#define MICROPY_PY_DELATTR_SETATTR (0)
#endif

101
py/objtype.c
View File

@ -3,7 +3,7 @@
*
* The MIT License (MIT)
*
* Copyright (c) 2013, 2014 Damien P. George
* Copyright (c) 2013-2018 Damien P. George
* Copyright (c) 2014-2016 Paul Sokolovsky
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
@ -41,6 +41,12 @@
#define DEBUG_printf(...) (void)0
#endif
#define ENABLE_SPECIAL_ACCESSORS \
(MICROPY_PY_DESCRIPTORS || MICROPY_PY_DELATTR_SETATTR || MICROPY_PY_BUILTINS_PROPERTY)
#define TYPE_FLAG_IS_SUBCLASSED (0x0001)
#define TYPE_FLAG_HAS_SPECIAL_ACCESSORS (0x0002)
STATIC mp_obj_t static_class_method_make_new(const mp_obj_type_t *self_in, size_t n_args, size_t n_kw, const mp_obj_t *args);
/******************************************************************************/
@ -591,6 +597,11 @@ STATIC void mp_obj_instance_load_attr(mp_obj_t self_in, qstr attr, mp_obj_t *des
mp_obj_class_lookup(&lookup, self->base.type);
mp_obj_t member = dest[0];
if (member != MP_OBJ_NULL) {
if (!(self->base.type->flags & TYPE_FLAG_HAS_SPECIAL_ACCESSORS)) {
// Class doesn't have any special accessors to check so return straightaway
return;
}
#if MICROPY_PY_BUILTINS_PROPERTY
if (MP_OBJ_IS_TYPE(member, &mp_type_property)) {
// object member is a property; delegate the load to the property
@ -652,11 +663,15 @@ STATIC void mp_obj_instance_load_attr(mp_obj_t self_in, qstr attr, mp_obj_t *des
STATIC bool mp_obj_instance_store_attr(mp_obj_t self_in, qstr attr, mp_obj_t value) {
mp_obj_instance_t *self = MP_OBJ_TO_PTR(self_in);
if (!(self->base.type->flags & TYPE_FLAG_HAS_SPECIAL_ACCESSORS)) {
// Class doesn't have any special accessors so skip their checks
goto skip_special_accessors;
}
#if MICROPY_PY_BUILTINS_PROPERTY || MICROPY_PY_DESCRIPTORS
// With property and/or descriptors enabled we need to do a lookup
// first in the class dict for the attribute to see if the store should
// be delegated.
// Note: this makes all stores slow... how to fix?
mp_obj_t member[2] = {MP_OBJ_NULL};
struct class_lookup_data lookup = {
.obj = self,
@ -728,9 +743,9 @@ STATIC bool mp_obj_instance_store_attr(mp_obj_t self_in, qstr attr, mp_obj_t val
}
#endif
#if MICROPY_PY_DELATTR_SETATTR
if (value == MP_OBJ_NULL) {
// delete attribute
#if MICROPY_PY_DELATTR_SETATTR
// try __delattr__ first
mp_obj_t attr_delattr_method[3];
mp_load_method_maybe(self_in, MP_QSTR___delattr__, attr_delattr_method);
@ -740,13 +755,8 @@ STATIC bool mp_obj_instance_store_attr(mp_obj_t self_in, qstr attr, mp_obj_t val
mp_call_method_n_kw(1, 0, attr_delattr_method);
return true;
}
#endif
mp_map_elem_t *elem = mp_map_lookup(&self->members, MP_OBJ_NEW_QSTR(attr), MP_MAP_LOOKUP_REMOVE_IF_FOUND);
return elem != NULL;
} else {
// store attribute
#if MICROPY_PY_DELATTR_SETATTR
// try __setattr__ first
mp_obj_t attr_setattr_method[4];
mp_load_method_maybe(self_in, MP_QSTR___setattr__, attr_setattr_method);
@ -757,8 +767,17 @@ STATIC bool mp_obj_instance_store_attr(mp_obj_t self_in, qstr attr, mp_obj_t val
mp_call_method_n_kw(2, 0, attr_setattr_method);
return true;
}
#endif
}
#endif
skip_special_accessors:
if (value == MP_OBJ_NULL) {
// delete attribute
mp_map_elem_t *elem = mp_map_lookup(&self->members, MP_OBJ_NEW_QSTR(attr), MP_MAP_LOOKUP_REMOVE_IF_FOUND);
return elem != NULL;
} else {
// store attribute
mp_map_lookup(&self->members, MP_OBJ_NEW_QSTR(attr), MP_MAP_LOOKUP_ADD_IF_NOT_FOUND)->value = value;
return true;
}
@ -899,6 +918,34 @@ STATIC mp_int_t instance_get_buffer(mp_obj_t self_in, mp_buffer_info_t *bufinfo,
// - there is a constant mp_obj_type_t (called mp_type_type) for the 'type' object
// - creating a new class (a new type) creates a new mp_obj_type_t
#if ENABLE_SPECIAL_ACCESSORS
STATIC bool check_for_special_accessors(mp_obj_t key, mp_obj_t value) {
#if MICROPY_PY_DELATTR_SETATTR
if (key == MP_OBJ_NEW_QSTR(MP_QSTR___setattr__) || key == MP_OBJ_NEW_QSTR(MP_QSTR___delattr__)) {
return true;
}
#endif
#if MICROPY_PY_BUILTINS_PROPERTY
if (MP_OBJ_IS_TYPE(value, &mp_type_property)) {
return true;
}
#endif
#if MICROPY_PY_DESCRIPTORS
static const uint8_t to_check[] = {
MP_QSTR___get__, MP_QSTR___set__, MP_QSTR___delete__,
};
for (size_t i = 0; i < MP_ARRAY_SIZE(to_check); ++i) {
mp_obj_t dest_temp[2];
mp_load_method_protected(value, to_check[i], dest_temp, true);
if (dest_temp[0] != MP_OBJ_NULL) {
return true;
}
}
#endif
return false;
}
#endif
STATIC void type_print(const mp_print_t *print, mp_obj_t self_in, mp_print_kind_t kind) {
(void)kind;
mp_obj_type_t *self = MP_OBJ_TO_PTR(self_in);
@ -985,6 +1032,19 @@ STATIC void type_attr(mp_obj_t self_in, qstr attr, mp_obj_t *dest) {
dest[0] = MP_OBJ_NULL; // indicate success
}
} else {
#if ENABLE_SPECIAL_ACCESSORS
// Check if we add any special accessor methods with this store
if (!(self->flags & TYPE_FLAG_HAS_SPECIAL_ACCESSORS)) {
if (check_for_special_accessors(MP_OBJ_NEW_QSTR(attr), dest[1])) {
if (self->flags & TYPE_FLAG_IS_SUBCLASSED) {
// This class is already subclassed so can't have special accessors added
mp_raise_msg(&mp_type_AttributeError, "can't add special method to already-subclassed class");
}
self->flags |= TYPE_FLAG_HAS_SPECIAL_ACCESSORS;
}
}
#endif
// store attribute
mp_map_elem_t *elem = mp_map_lookup(locals_map, MP_OBJ_NEW_QSTR(attr), MP_MAP_LOOKUP_ADD_IF_NOT_FOUND);
elem->value = dest[1];
@ -1016,6 +1076,7 @@ mp_obj_t mp_obj_new_type(qstr name, mp_obj_t bases_tuple, mp_obj_t locals_dict)
// TODO might need to make a copy of locals_dict; at least that's how CPython does it
// Basic validation of base classes
uint16_t base_flags = 0;
size_t bases_len;
mp_obj_t *bases_items;
mp_obj_tuple_get(bases_tuple, &bases_len, &bases_items);
@ -1033,10 +1094,17 @@ mp_obj_t mp_obj_new_type(qstr name, mp_obj_t bases_tuple, mp_obj_t locals_dict)
"type '%q' is not an acceptable base type", t->name));
}
}
#if ENABLE_SPECIAL_ACCESSORS
if (mp_obj_is_instance_type(t)) {
t->flags |= TYPE_FLAG_IS_SUBCLASSED;
base_flags |= t->flags & TYPE_FLAG_HAS_SPECIAL_ACCESSORS;
}
#endif
}
mp_obj_type_t *o = m_new0(mp_obj_type_t, 1);
o->base.type = &mp_type_type;
o->flags = base_flags;
o->name = name;
o->print = instance_print;
o->make_new = mp_obj_instance_make_new;
@ -1069,6 +1137,21 @@ mp_obj_t mp_obj_new_type(qstr name, mp_obj_t bases_tuple, mp_obj_t locals_dict)
o->locals_dict = MP_OBJ_TO_PTR(locals_dict);
#if ENABLE_SPECIAL_ACCESSORS
// Check if the class has any special accessor methods
if (!(o->flags & TYPE_FLAG_HAS_SPECIAL_ACCESSORS)) {
for (size_t i = 0; i < o->locals_dict->map.alloc; i++) {
if (MP_MAP_SLOT_IS_FILLED(&o->locals_dict->map, i)) {
const mp_map_elem_t *elem = &o->locals_dict->map.table[i];
if (check_for_special_accessors(elem->key, elem->value)) {
o->flags |= TYPE_FLAG_HAS_SPECIAL_ACCESSORS;
break;
}
}
}
}
#endif
const mp_obj_type_t *native_base;
size_t num_native_bases = instance_count_native_bases(o, &native_base);
if (num_native_bases > 1) {

53
tests/basics/builtin_property_inherit.py Normal file
View File

@ -0,0 +1,53 @@
# test builtin property combined with inheritance
try:
property
except:
print("SKIP")
raise SystemExit
# test property in a base class works for derived classes
class A:
@property
def x(self):
print('A x')
return 123
class B(A):
pass
class C(B):
pass
class D:
pass
class E(C, D):
pass
print(A().x)
print(B().x)
print(C().x)
print(E().x)
# test that we can add a property to base class after creation
class F:
pass
F.foo = property(lambda self: print('foo get'))
class G(F):
pass
F().foo
G().foo
# should be able to add a property to already-subclassed class because it already has one
F.bar = property(lambda self: print('bar get'))
F().bar
G().bar
# test case where class (H here) is already subclassed before adding attributes
class H:
pass
class I(H):
pass
# should be able to add a normal member to already-subclassed class
H.val = 2
print(I().val)
# should be able to add a property to the derived class
I.baz = property(lambda self: print('baz get'))
I().baz

10
tests/misc/non_compliant.py
View File

@ -139,3 +139,13 @@ class A:
class B(object, A):
pass
B().foo()
# can't assign property (or other special accessors) to already-subclassed class
class A:
pass
class B(A):
pass
try:
A.bar = property()
except AttributeError:
print('AttributeError')

1
tests/misc/non_compliant.py.exp
View File

@ -20,3 +20,4 @@ NotImplementedError
AttributeError
TypeError
A.foo
AttributeError