For builds without the compiler enabled (and hence without eval/exec) it is useful to still be able to run as many tests as possible.