Why precision is decreasing when multiply sum to other number

前端 未结 3 573
挽巷
挽巷 2020-12-11 01:51

I have encountered with following bug (or feature) in SQL Server.

When I use SUM (*column*) where column has a numeric(18, 8)

相关标签:
3条回答
  • 2020-12-11 02:09

    Aggregating a numeric(18, 8) with SUM results in the datatype numeric(38, 8).

    How the resulting datatype is calculated when multiplying something with numeric can be found here: Precision, Scale, and Length (Transact-SQL)

    The datatype for your constant -1 is numeric(1, 0)

    Precision is p1 + p2 + 1 = 40
    Scale is s1 + s2 = 8

    Max precision is 38 and that leaves you with numeric(38, 6).

    Read more about why it is numeric(38, 6) here: Multiplication and Division with Numerics

    0 讨论(0)
  • 2020-12-11 02:10

    If you read SUM's reference page, you'll see that on a decimal column it yields a type of NUMERIC(38,6). You need to cast the result of the SUM to NUMERIC(18,8) for it to work the way you want.

    Executing SELECT CAST(SUM(Qnty) as numeric(18,8)) * 2.234 FROM #temp yields 0.00000013404 as you'd expect.

    0 讨论(0)
  • 2020-12-11 02:18

    a technical explanation can be found at http://social.msdn.microsoft.com/Forums/en/transactsql/thread/233f7380-3f19-4836-b224-9f665b852406

    0 讨论(0)
提交回复
热议问题