Python arithmetic with small numbers

问题

I am getting the following unexpected result when I do arithmetic with small numbers in Python:

>>> sys.float_info
sys.float_info(max=1.7976931348623157e+308, max_exp=1024, max_10_exp=308, min=2.2250738585072014e-308, min_exp=-1021, min_10_exp=-307, dig=15, mant_dig=53, epsilon=2.220446049250313e-16, radix=2, rounds=1)
>>> (1. - (1.e-17) ) < 1.
False

I know that floating point numbers do not have infinite precision, but it should be able to handle "large" small numbers like 1e-17, shouldn't it?

回答1:

>>> import numpy as np
>>> np.nextafter(1., 0.)
0.99999999999999989

This is the next float after 1., in the direction of 0.

I guess 1. - 1.e-17 is simply closer to 1. than it is to numpy.nextafter(1., 0.), so when 1. - 1.e-17 is evaluated it gives you 1 exactly back. It wouldn't make sense to use some other float which was further away.

Related question -> Increment a python floating point value by the smallest possible amount

回答2:

First, let's review what epsilon really is in the return value of sys.float_info.

Epsilon (or 𝟄) is the smallest number such that 0.5 + 𝟄 ≠ 0.5 AND 0.5 - 𝟄 ≠ 0.5

Python is telling you that the smallest number that will cause 0.5 to increment or decrement repeatably is epsilon=2.220446049250313e-16 -- but this is only for the value 0.5. You are attempting to increment 1.0 by 1.0e-17. This is a larger value (1.0 vs 0.5) being incremented by a smaller number than the 𝟄 for 0.5 (1.0e-17 vs 2.2e-16). You are off by an order of magnitude roughly, since the increment value of 1.0e-17 is an order of magnitude smaller than the relative epsilon for 1.0.

You can see this here:

These change the value of 0.5

>>> 0.5+sys.float_info.epsilon
0.5000000000000002
>>> 0.5-sys.float_info.epsilon
0.4999999999999998

These values do not:

>>> 0.5+sys.float_info.epsilon/10.0
0.5
>>> 0.5-sys.float_info.epsilon/10.0
0.5
>>> 5.0+sys.float_info.epsilon
5.0
>>> 5.0-sys.float_info.epsilon
5.0

Explanation:

IEEE 754 defines the floating point format in use today on most standard computers (specialty computers or libraries may use a different format.) The 64 bit format of IEEE 754 uses 53 bits of precision to calculate and 52 to store to the mantissa of a floating point value. Since you have a fixed 52/53 bits to work with, the magnitude and accuracy of the mantissa changes for larger / smaller values. So then the 𝟄 changes as the relative magnitude of a floating point number changes. The value of 𝟄 for 0.5 is different that the value for 1.0 and for 100.0.

For a variety of very good and platform-specific reasons (storage and representation, rounding, etc), even though you could use a smaller number, epsilon is defined as using 52 bits of precision for the 64 bit float format. Since most Python implementations use a C double float for float, this can be demonstrated:

>>> 2**-52==sys.float_info.epsilon
True

See how many bits your platform will do:

>>> 0.5 + 2.0**-53
0.5000000000000001
>>> 0.5 - 2.0**-53
0.4999999999999999
>>> 0.5 + 2.0**-54
0.5                           # fail for 0.5 + 54 bits...
>>> 0.5 - 2.0**-54
0.49999999999999994           # OK for minus
>>> 0.5 - 2.0**-55
0.5                           # fail  for 0.5 minus 55 bits...

There are several work arounds for your issue:

1. You can use the C99 concept of nextafter to calculate the value appropriate epsilon. For Python, either use numpy or the Decimal class to calculate nextafter. More on nextafter in my previous answer HERE
2. Use integers. A 64 bit integer will clearly handle an epsilon value in the 17th order of magnitude without rounding.
3. Use an arbitrary precision math library. Decimal is in the standard Python distribution.

The important concept is that the value of 𝟄 is relative to value (and if you are incrementing or decrementing).

This can be seen here:

>>> numpy.nextafter(0.0,1.0)-0.0
4.9406564584124654e-324                 # a relative epsilon value of 4.94e-324
>>> numpy.nextafter(0.01,1.0)-0.01
1.7347234759768071e-18                  # 1e-17 would still work...
>>> numpy.nextafter(0.1,1.0)-0.1
1.3877787807814457e-17                  # 1e-17 would >>barely<< work...
>>> numpy.nextafter(0.5,1.0)-0.5
1.1102230246251565e-16                  # a relative epsilon value of 1.1e-16
>>> numpy.nextafter(500.0,501.0)-500.0
5.6843418860808015e-14                  # relative epsilon of 5.6e-14
>>> numpy.nextafter(1e17,1e18)-1e17
16.0                                    # the other end of the spectrum...

So you can see that 1e-17 will work handily to increment values between 0.0 and 0.1 but not many values greater than that. As you can see above, the relative 𝟄 for 1e17 is 16.

回答3:

it should be able to handle "large" small numbers like 1e-17, shouldn't it?

Not necessarily (it depends on the numbers). A float cannot exactly represent either 1e-17 or 1-(1e-17). In the case of the latter, the nearest number that it can represent is 1.

I suggest you read What Every Computer Scientist Should Know About Floating-Point Arithmetic.

回答4:

If you need this level of precision, consider the Decimal module

>>> decimal.Decimal(1.0)-decimal.Decimal('1.0e-17')
Decimal('0.999999999999999990')
>>> decimal.Decimal(1.0)-decimal.Decimal('1.0e-17')<decimal.Decimal(1.0)
True

And:

>>> decimal.Decimal(1.0)-decimal.Decimal('1.0e-17')<1.0
True

Careful with the last one though because you can get conversion errors.

Others have suggested What Every Computer Scientist Should Know About Floating-Point Arithmetic. and I also recommend Don’t Store That in a Float

回答5:

you can handle those. note that

>>> 1.e-17 == 0
False

and

>>> 1.e-17 + 1.e-18
1.1e-17

you simply cannot handle 1-1e-17, because the mantissa won't fit in the finite precision

回答6:

>>> from decimal import Decimal
>>> Decimal('1')-Decimal(str(10**-17)) < Decimal('1')
True

Use the decimal module, for that kind of accuracy!

回答7:

You have epsilon=2.220446049250313e-16 so it is normal that (1. - (1.e-17) ) = 1 because 1.e-17 < epsilon.

来源：https://stackoverflow.com/questions/10555659/python-arithmetic-with-small-numbers