tools/mpy-tool: Set sane initial dynamic qstr pool size with frozen mods

The first dynamic qstr pool is double the size of the 'alloc' field of
the last const qstr pool. The built in const qstr pool
(mp_qstr_const_pool) has a hardcoded alloc size of 10, meaning that the
first dynamic pool is allocated space for 20 entries. The alloc size
must be less than or equal to the actual number of qstrs in the pool
(the 'len' field) to ensure that the first dynamically created qstr
triggers the creation of a new pool.

When modules are frozen a second const pool is created (generally
mp_qstr_frozen_const_pool) and linked to the built in pool. However,
this second const pool had its 'alloc' field set to the number of qstrs
in the pool. When freezing a large quantity of modules this can result
in thousands of qstrs being in the pool. This means that the first
dynamically created qstr results in a massive allocation. This commit
sets the alloc size of the frozen qstr pool to 10 or less (if the number
of qstrs in the pool is less than 10). The result of this is that the
allocation behaviour when a dynamic qstr is created is identical with an
without frozen code.

Note that there is the potential for a slight memory inefficiency if the
frozen modules have less than 10 qstrs, as the first few dynamic
allocations will have quite a large overhead, but the geometric growth
soon deals with this.
This commit is contained in:
Rich Barlow 2018-07-19 12:42:26 +01:00 committed by Damien George
parent 5482d84673
commit 6e5a40cf3c

View File

@ -517,12 +517,15 @@ def freeze_mpy(base_qstrs, raw_codes):
print(' MP_QSTR_%s,' % new[i][1])
print('};')
# As in qstr.c, set so that the first dynamically allocated pool is twice this size; must be <= the len
qstr_pool_alloc = min(len(new), 10)
print()
print('extern const qstr_pool_t mp_qstr_const_pool;');
print('const qstr_pool_t mp_qstr_frozen_const_pool = {')
print(' (qstr_pool_t*)&mp_qstr_const_pool, // previous pool')
print(' MP_QSTRnumber_of, // previous pool size')
print(' %u, // allocated entries' % len(new))
print(' %u, // allocated entries' % qstr_pool_alloc)
print(' %u, // used entries' % len(new))
print(' {')
for _, _, qstr in new: