djsundog/circuitpython
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

tests: Make run-tests help and README be more descriptive of behaviour.

Browse Source
This commit is contained in:
Thorsten von Eicken 2020-01-16 09:07:50 -08:00 committed by Damien George
parent 3032ae1155
commit 5c5f93c1b8
2 changed files with 18 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
tests/README
View File

@ -13,6 +13,15 @@ condition a test. The run-tests script uses small scripts in the
feature_check directory to check whether each such feature is present,
and skips the relevant tests if not.
Tests are generally verified by running the test both in MicroPython and
in CPython and comparing the outputs. If the output differs the test fails
and the outputs are saved in a .out and a .exp file respectively.
For tests that cannot be run in CPython, for example because they use
the machine module, a .exp file can be provided next to the test's .py
file. A convenient way to generate that is to run the test, let it fail
(because CPython cannot run it) and then copy the .out file (but not
before checking it manually!)
When creating new tests, anything that relies on float support should go in the
float/ subdirectory. Anything that relies on import x, where x is not a built-in
module, should go in the import/ subdirectory.

11
tests/run-tests
View File

@ -553,7 +553,14 @@ class append_filter(argparse.Action):
def main():
cmd_parser = argparse.ArgumentParser(
formatter_class=argparse.RawDescriptionHelpFormatter,
description='Run and manage tests for MicroPython.',
description='''Run and manage tests for MicroPython.
When running tests, run-tests compares the MicroPython output of the test with the output
produced by running the test through CPython unless a <test>.exp file is found, in which
case it is used as comparison.
If a test fails, run-tests produces a pair of <test>.out and <test>.exp files in the current
directory with the MicroPython output and the expectations, respectively.
''',
epilog='''\
Options -i and -e can be multiple and processed in the order given. Regex
"search" (vs "match") operation is used. An action (include/exclude) of
@ -570,7 +577,7 @@ the last matching regex is used:
cmd_parser.add_argument('-d', '--test-dirs', nargs='*', help='input test directories (if no files given)')
cmd_parser.add_argument('-e', '--exclude', action=append_filter, metavar='REGEX', dest='filters', help='exclude test by regex on path/name.py')
cmd_parser.add_argument('-i', '--include', action=append_filter, metavar='REGEX', dest='filters', help='include test by regex on path/name.py')
cmd_parser.add_argument('--write-exp', action='store_true', help='save .exp files to run tests w/o CPython')
cmd_parser.add_argument('--write-exp', action='store_true', help='use CPython to generate .exp files to run tests w/o CPython')
cmd_parser.add_argument('--list-tests', action='store_true', help='list tests instead of running them')
cmd_parser.add_argument('--emit', default='bytecode', help='MicroPython emitter to use (bytecode or native)')
cmd_parser.add_argument('--heapsize', help='heapsize to use (use default if not specified)')