djsundog/circuitpython
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

py/formatfloat: Fix case where floats could render with a ":" character.

Browse Source 
Prior to this patch, some architectures (eg unix x86) could render floats
with a ":" character in them, eg 1e+39 would come out as ":e+38" (":" is
just after "9" in ASCII so this is like 10e+38).  This patch fixes some of
these cases.
This commit is contained in:
Damien George 2018-03-01 16:02:59 +11:00
parent bc12eca461
commit 7b050fa76c
2 changed files with 5 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
py/formatfloat.c
View File

@ -258,7 +258,7 @@ int mp_format_float(FPTYPE f, char *buf, size_t buf_size, char fmt, int prec, ch
} }
// It can be that f was right on the edge of an entry in pos_pow needs to be reduced // It can be that f was right on the edge of an entry in pos_pow needs to be reduced
if (f >= FPCONST(10.0)) { if ((int)f >= 10) {
e += 1; e += 1;
f *= FPCONST(0.1); f *= FPCONST(0.1);
} }

4
tests/float/float_format.py
View File

@ -9,3 +9,7 @@ for val in (116, 1111, 1234, 5010, 11111):
# make sure rounding is done at the correct precision # make sure rounding is done at the correct precision
for prec in range(8): for prec in range(8):
print(('%%.%df' % prec) % 6e-5) print(('%%.%df' % prec) % 6e-5)
# check certain cases that had a digit value of 10 render as a ":" character
print('%.2e' % float('9' * 51 + 'e-39'))
print('%.2e' % float('9' * 40 + 'e-21'))