What exactly is the “resolution” parameter of numpy float

Question

I am seeking some more understanding about the \"resolution\" parameter of a numpy float (I guess any computer defined float for that matter).

Consider the followin

Answer 1

The short answer is "dont' confuse numpy.finfo with numpy.spacing".

finfo operates on the dtype, while spacing operates on the value.

Background Information

First, though, some general explanation:

---

The key part to understand is that floating point numbers are similar to scientific notation. Just like you'd write 0.000001 as 1.0 x 10^-6, floats are similar to c x 2^q. In other words, they have two separate parts - a coefficient (c, a.k.a. "significand") and an exponent (q). These two values are stored as integers.

Therefore, how closely a value can be represented (let's think of this as the degree of discretization) is a function of both parts, and depends on the magnitude of the value.

However, the "precision" (as referred to by np.finfo) is essentially the number of significant digits if the number were written in base-10 scientific notation. The "resolution" is the resolution of the coefficient (part in front) if the value were written in the same base-10 scientific notation (i.e. 10^-precision). In other words, both are only a function of the coefficient.

Numpy-specific

For numpy.finfo, "precision" and "resolution" are simply the inverse of each other. Neither one tells you how closely a particular number is being represented. They're purely a function of the dtype.

Instead, if you're worried about the absolute degree of discretization, use numpy.spacing(your_float). This will return the difference in the next largest value in that particular format (e.g. it's different for a float32 than a float64).

Examples

As an example:

In [1]: import numpy as np

In [2]: np.spacing(10.1)
Out[2]: 1.7763568394002505e-15

In [3]: np.spacing(10000000000.1)
Out[3]: 1.9073486328125e-06

In [4]: np.spacing(1000000000000.1)
Out[4]: 0.0001220703125

In [5]: np.spacing(100000000000000.1)
Out[5]: 0.015625

In [6]: np.spacing(10000000000000000.1)
Out[6]: 2.0

But the precision and resolution don't change:

In [7]: np.finfo(10.1).precision
Out[7]: 15

In [8]: np.finfo(10000000000000000.1).precision
Out[8]: 15

In [9]: np.finfo(10.1).resolution
Out[9]: 1.0000000000000001e-15

In [10]: np.finfo(10000000000000000000.1).resolution
Out[10]: 1.0000000000000001e-15

Also note that all of these depend on the data type that you're using:

In [11]: np.spacing(np.float32(10.1))
Out[11]: 9.5367432e-07

In [12]: np.spacing(np.float32(10000000000000.1))
Out[12]: 1048576.0

In [13]: np.finfo(np.float32).precision
Out[13]: 6

In [14]: np.finfo(np.float32).resolution
Out[14]: 1e-06

In [15]: np.spacing(np.float128(10.1))
Out[15]: 8.6736173798840354721e-19

In [16]: np.spacing(np.float128(10000000000000.1))
Out[16]: 9.5367431640625e-07

In [17]: np.finfo(np.float128).precision
Out[17]: 18

In [18]: np.finfo(np.float128).resolution
Out[18]: 1.0000000000000000007e-18

---

Specific Questions

Now on to your specific questions:

But practically, does it mean that I should expect results to be erroneous if I preform operations using numbers less than the resolution?

No, because the precision/resolution (in numpy.finfo terms) is only a function of the coefficient, and doesn't take into account the exponent. Very small and very large numbers have the same "precision", but that's not an absolute "error".

As a rule of thumb, when using the "resolution" or "precision" terms from finfo, think of scientific notation. If we're operating on small numbers with similar magnitudes, we don't need to worry about much.

Let's take the decimal math case with 6 significant digits (somewhat similar to a float32):

1.20000 x 10^-19 + 2.50000 x 10^-20 => 1.45000 x 10^19

However, if we operate on numbers with wildly different magnitudes but limited precision (again, 6 significant digits):

1.20000 x 10^6 + 2.50000 x 10^-5 => 1.20000

We'll start to see the effects quite clearly.

How can I quantify the error, for say addition, of two floating point numbers given their precision?

Use np.spacing(result).

If the resolution is as "large" as 1e-15, why would the smallest allowable number be on the order of 1e-308?

Again, the "resolution" in this case doesn't take into account the exponent, just the part in front.

---

Hopefully that helps clarify things somewhat. All of this is a bit confusing, and everyone gets bitten by it at some point. It's good to try to build up a bit of intuition about it and to know what functions to call to find out exactly in your platform-of-choice!