Commit Graph

4 Commits

Author SHA1 Message Date
microDev
a52eb88031
run code formatting script 2021-03-15 19:27:36 +05:30
Damien George
6dad088569 tests/float: Adjust float-parsing tests to pass with only a small error.
Float parsing (both single and double precision) may have a relative error
of order the floating point precision, so adjust tests to take this into
account by not printing all of the digits of the answer.
2018-02-26 15:54:03 +11:00
Damien George
b75cb8392b py/parsenum: Fix parsing of floats that are close to subnormal.
Prior to this patch, a float literal that was close to subnormal would
have a loss of precision when parsed.  The worst case was something like
float('10000000000000000000e-326') which returned 0.0.
2018-02-08 14:02:50 +11:00
Damien George
84895f1a21 py/parsenum: Improve parsing of floating point numbers.
This patch improves parsing of floating point numbers by converting all the
digits (integer and fractional) together into a number 1 or greater, and
then applying the correct power of 10 at the very end.  In particular the
multiple "multiply by 0.1" operations to build a fraction are now combined
together and applied at the same time as the exponent, at the very end.

This helps to retain precision during parsing of floats, and also includes
a check that the number doesn't overflow during the parsing.  One benefit
is that a float will have the same value no matter where the decimal point
is located, eg 1.23 == 123e-2.
2017-11-27 12:51:52 +11:00