From e7fc765880ba6025ac5677db696669f43975483e Mon Sep 17 00:00:00 2001 From: Paul Sokolovsky Date: Sun, 10 Dec 2017 02:15:43 +0200 Subject: [PATCH] unix/mpconfigport: Disable uio.resource_stream(). This function was implemented as an experiment, and was enabled only in unix port. To remind, it allows to access arbitrary files frozen as source modules (vs bytecode). However, further experimentation showed that the same functionality can be implemented with frozen bytecode. The process requires more steps, but with suitable toolset it doesn't matter patch. This process is: 1. Convert binary files into "Python resource module" with tools/mpy_bin2res.py. 2. Freeze as the bytecode. 3. Use micropython-lib's pkg_resources.resource_stream() to access it. In other words, the extra step is using tools/mpy_bin2res.py (because there would be wrapper for uio.resource_stream() anyway). Going frozen bytecode route allows more flexibility, and same/additional efficiency: 1. Frozen source support can be disabled altogether for additional code savings. 2. Resources could be also accessed as a buffer, not just as a stream. There're few caveats too: 1. It wasn't actually profiled the overhead of storing a resource in "Python resource module" vs storing it directly, but it's assumed that overhead is small. 2. The "efficiency" claim above applies to the case when resource file is frozen as the bytecode. If it's not, it actually will take a lot of RAM on loading. But in this case, the resource file should not be used (i.e. generated) in the first place, and micropython-lib's pkg_resources.resource_stream() implementation has the appropriate fallback to read the raw files instead. This still poses some distribution issues, e.g. to deployable to baremetal ports (which almost certainly would require freezeing as the bytecode), a distribution package should include the resource module. But for non-freezing deployment, presense of resource module will lead to memory inefficiency. All the discussion above reminds why uio.resource_stream() was implemented in the first place - to address some of the issues above. However, since then, frozen bytecode approach seems to prevail, so, while there're still some issues to address with it, this change is being made. This change saves 488 bytes for the unix x86_64 port. --- ports/unix/mpconfigport.h | 1 - py/mpconfig.h | 6 +++++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/ports/unix/mpconfigport.h b/ports/unix/mpconfigport.h index a3d2bb7dbd..06ae1e2343 100644 --- a/ports/unix/mpconfigport.h +++ b/ports/unix/mpconfigport.h @@ -103,7 +103,6 @@ #endif #define MICROPY_PY_CMATH (1) #define MICROPY_PY_IO_FILEIO (1) -#define MICROPY_PY_IO_RESOURCE_STREAM (1) #define MICROPY_PY_GC_COLLECT_RETVAL (1) #define MICROPY_MODULE_FROZEN_STR (1) diff --git a/py/mpconfig.h b/py/mpconfig.h index 095b7ef7b9..7784367085 100644 --- a/py/mpconfig.h +++ b/py/mpconfig.h @@ -938,7 +938,11 @@ typedef double mp_float_t; // Whether to provide "uio.resource_stream()" function with // the semantics of CPython's pkg_resources.resource_stream() -// (allows to access resources in frozen packages). +// (allows to access binary resources in frozen source packages). +// Note that the same functionality can be achieved in "pure +// Python" by prepocessing binary resources into Python source +// and bytecode-freezing it (with a simple helper module available +// e.g. in micropython-lib). #ifndef MICROPY_PY_IO_RESOURCE_STREAM #define MICROPY_PY_IO_RESOURCE_STREAM (0) #endif