Is it 52 or 53 bits of floating point precision?

Question

I keep on seeing this nonsense about 53 bits of precision in 64-bit IEEE floating point representation. Would someone please explain to me how in the world a bit that is st

Answer 1

The key concept here is "normalization". In general scientific notation, every value has many representations. That makes arithmetic, especially comparisons, more difficult than necessary. The common solution is to require the most significant digit of the significand to be non-zero. For example, the first floating point system I worked with was base 16, and the leading digit of the significand was in the range 1 through F.

That has a special effect for binary floating point. The most significant bit of the significand is a non-zero bit. There is no point wasting one of the limited number of bits in the physical representation on a bit that is known to be non-zero.

Normal numbers in IEEE 754 64-bit binary have a 53 bit significand whose implicit leading bit is known to be 1, and with the remaining 52 bits stored in the physical representation.

There being no such thing as a free lunch, there is a cost to this. The cost is a limitation on how small a number can be stored with a given exponent. For most exponents that does not matter - the number just gets stored with a smaller exponent, and still with a leading one bit that does not need to be stored.

It would be a real limitation for zero exponent, because there is no smaller exponent to use. IEEE 754 binary floating point solves that by storing very small magnitude numbers, with a zero exponent, differently. They have at most 52 significant bits, all stored, with leading zeros permitted. That allows very small magnitude numbers to be represented as non-zero numbers, at a cost of reduced precision.

Infinities and NaNs are stored differently, with the all ones exponent.