Inconsistency rounding real numbers in C #

前端 未结 3 1709
梦谈多话
梦谈多话 2021-01-14 03:38

I have this test code:

class Test
{
    static void Main()
    {
        decimal m = 1M / 6M;
        double d = 1.0 / 6.0;

        decimal notQuiteWholeM =         


        
3条回答
  •  予麋鹿
    予麋鹿 (楼主)
    2021-01-14 04:22

    Not quite reading your question in the same way as the other two answers. The gist of it: Does the formatted string representation of a double "round" in C#?

    Yes.

    Internally double is represented with full IEEE-754 decimal digit precision (15-17 digits), which is why:

    notQuiteWholeD < 1.0 == true    // because notQuiteWholeD = 0.99999999999999989
    

    However, when formatting it as a string, by default it will use 15 digit precision - equivalent to:

    String.Format("{0:G15}", notQuiteWholeD)   // outputs "1"
    

    To get all the digits of the full internal representation, you can use:

    Console.WriteLine("{0:G17}", notQuiteWholeD);
    

    Or:

    Console.WriteLine("{0:R}", notQuiteWholeD);
    

    Both, in this case, will output "0,99999999999999989".

    The former will always use 17 digit precision. The latter ("roundtrip precision") will use 15 digits if that's enough precision for the following to be true, otherwise it will use 17:

    Double.Parse(String.Format("{0:G15}", notQuiteWholeD)) == notQuiteWholeD
    

    Bonus Example: ... of when G17 and R differ:

    Console.WriteLine("{0:G17}", 1.0000000000000699); // outputs "1.0000000000000699"
    Console.WriteLine("{0:R}",   1.0000000000000699); // outputs "1.00000000000007"
    

    1.0000000000000699 (17 significant digits) can be represented accurately enough for a roundtrip using only 15 significant digits. In other words, the double representation of 1.00...07 is the same as for 1.00...0699.

    So 1.00...07 (15 digits) is a shorter input to get the exact same internal (17 digit) representation. That means R will round it to 15 digits, while G17 will keep all the digits of the internal representation.

    Maybe it's clearer when realizing that this:

    Console.WriteLine("{0:G17}", 1.00000000000007); // outputs "1.0000000000000699"
    Console.WriteLine("{0:R}",   1.00000000000007); // outputs "1.00000000000007"
    

    ... gives the exact same results.

提交回复
热议问题