Decimal place issues with floats and decimal.Decimal

前端 未结 6 1567
悲哀的现实
悲哀的现实 2020-11-29 10:47

I seem to be losing a lot of precision with floats.

For example I need to solve a matrix:

4.0x -2.0y 1.0z =11.0
1.0x +5.0y -3.0z =-6.0
2.0x +2.0y +5.         


        
6条回答
  •  孤街浪徒
    2020-11-29 11:36

    IEEE floating point is binary, not decimal. There is no fixed length binary fraction that is exactly 0.1, or any multiple thereof. It is a repeating fraction, like 1/3 in decimal.

    Please read What Every Computer Scientist Should Know About Floating-Point Arithmetic

    Other options besides a Decimal class are

    • using Common Lisp or Python 2.6 or another language with exact rationals

    • converting the doubles to close rationals using, e.g., frap

提交回复
热议问题