How scale is defined when decimal and bigint are divided?

百般思念 提交于 2019-12-06 07:53:07
Lukasz Szozda

Argument 1: 3 AS DECIMAL(19, 8)

Argument 2: 27 AS DECIMAL (18, 0) -- default precision is 18, default scale is 0 (BIGINT was converted to DECIMAL due to type precedence)

p1 = 19
p2 = 18
s1 = 8
s2 = 0

max precision = (p1 - s1 + s2) + MAX(6, s1 + p2 + 1) -- up to 38

max scale = MAX(6, s1 + p2 + 1)

Let's calculate for example 1:

precision: (19 - 8 + 0) + MAX(6, 8 + 18 + 1) = 38
scale:     MAX(6, 8 + 18 + 1) = 27

For all your examples you will get always max 27 scale.

 0.111111111111111111111111111 (27)
11.111111111111111111111111111 (27)
 0.005488934750153684025643277 (27)

The whole part takes only necessary digits (1), (2), (1).

For me everything is perfectly valid.

This answer is based on work of @Paul White from Decimal Truncation In division.

This is call Data Type Precedence.

When a query do something between different but yet compatible types, one of them has to be casted to the other type, eitheir with an explicit or implicit conversion.

If you look at Data Type Conversion (Database Engine) , you will see that there is an implicit conversion between Decimal and Bigint.

Therefore you query does not requiere an explicit cast.

If you look at Data Type Precedence (Transact-SQL) on MSDN, you will see:

    1. decimal
    1. bigint

This means that decimal has a higher precedence than bigint and the bigint value will converted to decimal.

In the end, you calculation will be:

  • 3,000... / 27,000...
  • 300,000... / 27,000...
  • 75003,000... / 27,000...

If you want it to be 3 / 27, you must do an explicit cast on the Decimal value.

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!