df9a949891
This commit adds a test runner and initial test scripts which run multiple Python/MicroPython instances (eg executables, target boards) in parallel. This is useful for testing, eg, network and Bluetooth functionality. Each test file has a set of functions called instanceX(), where X ranges from 0 up to the maximum number of instances that are needed, N-1. Then run-multitests.py will execute this script on N separate instances (eg micropython executables, or attached boards via pyboard.py) at the same time, synchronising their start in the right order, possibly passing IP address (or other address like bluetooth MAC) from the "server" instance to the "client" instances so they can connect to each other. It then runs them to completion, collects the output, and then tests against what CPython gives (or what's in a provided .py.exp file). The tests will be run using the standard unix executable for all instances by default, eg: $ ./run-multitests.py multi_net/*.py Or they can be run with a board and unix executable via: $ ./run-multitests.py --instance pyb:/dev/ttyACM0 --instance exec:micropython multi_net/*.py |
||
---|---|---|
.. | ||
basics | ||
cmdline | ||
cpydiff | ||
extmod | ||
feature_check | ||
float | ||
import | ||
inlineasm | ||
internal_bench | ||
io | ||
jni | ||
micropython | ||
misc | ||
net_hosted | ||
net_inet | ||
perf_bench | ||
pyb | ||
pybnative | ||
qemu-arm | ||
stress | ||
thread | ||
unicode | ||
unix | ||
wipy | ||
README | ||
run-internalbench.py | ||
run-multitests.py | ||
run-natmodtests.py | ||
run-perfbench.py | ||
run-tests | ||
run-tests-exp.py | ||
run-tests-exp.sh |
This directory contains tests for various functionality areas of MicroPython. To run all stable tests, run "run-tests" script in this directory. Tests of capabilities not supported on all platforms should be written to check for the capability being present. If it is not, the test should merely output 'SKIP' followed by the line terminator, and call sys.exit() to raise SystemExit, instead of attempting to test the missing capability. The testing framework (run-tests in this directory, test_main.c in qemu_arm) recognizes this as a skipped test. There are a few features for which this mechanism cannot be used to condition a test. The run-tests script uses small scripts in the feature_check directory to check whether each such feature is present, and skips the relevant tests if not. Tests are generally verified by running the test both in MicroPython and in CPython and comparing the outputs. If the output differs the test fails and the outputs are saved in a .out and a .exp file respectively. For tests that cannot be run in CPython, for example because they use the machine module, a .exp file can be provided next to the test's .py file. A convenient way to generate that is to run the test, let it fail (because CPython cannot run it) and then copy the .out file (but not before checking it manually!) When creating new tests, anything that relies on float support should go in the float/ subdirectory. Anything that relies on import x, where x is not a built-in module, should go in the import/ subdirectory.