Commit Graph

7 Commits

Author SHA1 Message Date
Scott Shawcroft b057fb8a4b
codeformat 2021-04-19 22:22:44 -07:00
Damien George 9849209ad8 tests/float/float_parse.py: Add tests for accuracy of small decimals. 2018-09-20 22:26:53 +10:00
Damien George 1ad0013dec tests: Add some tests for bigint hash, float hash and float parsing.
Following outcome of recent fuzz testing and sanitizing by @jepler.
2018-05-21 13:05:40 +10:00
Damien George d2c1db1e5c tests/float/float_parse: Allow test to run on 32-bit archs.
Printing of uPy floats can differ by the floating-point precision on
different architectures (eg 64-bit vs 32-bit x86), so it's not possible to
using printing of floats in some parts of this test.  Instead we can just
check for equivalence with what is known to be the correct answer.
2018-05-11 13:51:18 +10:00
Damien George 6dad088569 tests/float: Adjust float-parsing tests to pass with only a small error.
Float parsing (both single and double precision) may have a relative error
of order the floating point precision, so adjust tests to take this into
account by not printing all of the digits of the answer.
2018-02-26 15:54:03 +11:00
Damien George b75cb8392b py/parsenum: Fix parsing of floats that are close to subnormal.
Prior to this patch, a float literal that was close to subnormal would
have a loss of precision when parsed.  The worst case was something like
float('10000000000000000000e-326') which returned 0.0.
2018-02-08 14:02:50 +11:00
Damien George 84895f1a21 py/parsenum: Improve parsing of floating point numbers.
This patch improves parsing of floating point numbers by converting all the
digits (integer and fractional) together into a number 1 or greater, and
then applying the correct power of 10 at the very end.  In particular the
multiple "multiply by 0.1" operations to build a fraction are now combined
together and applied at the same time as the exponent, at the very end.

This helps to retain precision during parsing of floats, and also includes
a check that the number doesn't overflow during the parsing.  One benefit
is that a float will have the same value no matter where the decimal point
is located, eg 1.23 == 123e-2.
2017-11-27 12:51:52 +11:00