djsundog
/
circuitpython
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

py/formatfloat: Fix case of float format where leading digit was "10". 

When taking the logarithm of the float to determine the exponent, there
are some edge cases that finish the log loop too large.  Eg for an
input value of 1e32-epsilon, this is actually less than 1e32 from the
log-loop table and finishes as 10.0e31 when it should be 1.0e32.  It
is thus rendered as :e32 (: comes after 9 in ascii).

There was the same problem with numbers less than 1.
This commit is contained in:
Damien George 2016-03-29 22:03:13 +01:00
parent d88250c06e
commit 03b8bb7ec9
1 changed files with 7 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
py/formatfloat.c
View File

@ -198,7 +198,7 @@ int mp_format_float(FPTYPE f, char *buf, size_t buf_size, char fmt, int prec, ch
if (e == 0) {
e_sign_char = '+';
}
} else {
} else if (fp_isless1(f)) {
e++;
f *= FPCONST(10.0);
}
@ -250,6 +250,12 @@ int mp_format_float(FPTYPE f, char *buf, size_t buf_size, char fmt, int prec, ch
}
}
// It can be that f was right on the edge of an entry in pos_pow needs to be reduced
if (f >= FPCONST(10.0)) {
e += 1;
f *= FPCONST(0.1);
}
// If the user specified fixed format (fmt == 'f') and e makes the
// number too big to fit into the available buffer, then we'll
// switch to the 'e' format.