How can I prevent csv.DictWriter() or writerow() rounding my floats?

Question

I have a dictionary that I want to write to a csv file, but the floats in the dictionary are rounded off when I write them to the file. I want to keep the maximum precision

Answer 1

It's a known bug^H^H^Hfeature. According to the docs:

"""... the value None is written as the empty string. [snip] All other non-string data are stringified with str() before being written."""

Don't rely on the default conversions. Use repr() for floats. unicode objects need special handling; see the manual. Check whether the consumer of the file will accept the default format of datetime.x objects for x in (datetime, date, time, timedelta).

Update:

For float objects, "%f" % value is not a good substitute for repr(value). The criterion is whether the consumer of the file can reproduce the original float object. repr(value) guarantees this. "%f" % value doesn't.

# Python 2.6.6
>>> nums = [1323494016.855676, 1323493818.004238, 198.8514380455017, 1.0 / 3]
>>> for v in nums:
...     rv = repr(v)
...     fv = "%f" % v
...     sv = str(v)
...     print rv, float(rv) == v, fv, float(fv) == v, sv, float(sv) == v
...
1323494016.8556759 True 1323494016.855676 True 1323494016.86 False
1323493818.0042379 True 1323493818.004238 True 1323493818.0 False
198.85143804550171 True 198.851438 False 198.851438046 False
0.33333333333333331 True 0.333333 False 0.333333333333 False

Notice that in the above, it appears by inspection of the strings produced that none of the %f cases worked. Before 2.7, Python's repr always used 17 significant decimal digits. In 2.7, this was changed to using the minimum number of digits that still guaranteed float(repr(v)) == v. The difference is not a rounding error.

# Python 2.7 output
1323494016.855676 True 1323494016.855676 True 1323494016.86 False
1323493818.004238 True 1323493818.004238 True 1323493818.0 False
198.8514380455017 True 198.851438 False 198.851438046 False
0.3333333333333333 True 0.333333 False 0.333333333333 False

Note the improved repr() results in the first column above.

Update 2 in response to comment """And thanks for the info on Python 2.7. Unfortunately, I'm limited to 2.6.2 (running on the destination machine which can't be upgraded). But I'll keep this in mind for future scripts. """

It doesn't matter. float('0.3333333333333333') == float('0.33333333333333331') produces True on all versions of Python. This means that you could write your file on 2.7 and it would read the same on 2.6, or vice versa. There is no change in the accuracy of what repr(a_float_object) produces.