decimal

问题 Why when I do: select CAST(1 AS DECIMAL(38, 28))/CAST(1625625 AS DECIMAL(38, 28)) Do I get 0, but when I do select CAST(1 AS DECIMAL(20, 10))/CAST(1625625 AS DECIMAL(20, 10)) I get a value? I would have expected the higher precision division to return a higher precision result, why is this not the case? 回答1: This happens because for math operations the resulting precision and scale is calculated with the rules found in http://msdn.microsoft.com/en-us/library/ms190476.aspx#division operation

问题 I run the following code, but did not expect that result.. public class SampleDemo { public static void main(String args[]) { System.out.println(10.00 - 9.10); } } I am getting o/p as 0.9000000000000004 Why is it so? 回答1: This is because decimal values can’t be represented exactly by float or double. One suggestion : Avoid float and double where exact answers are required. Use BigDecimal, int, or long instead Using int : public class SampleDemo { public static void main(String args[]) {

问题 I am trying to construct a query in MySQL that concatenates a bunch of binary fields and then gives me the result as DECIMAL. e.g: SELECT CONCAT (setting1, setting2, setting3) AS settings; may gave me: 101 111 110 I want to pass this value to a function (convert? cast?) that will give me the corresponding DECIMAL value: 5 7 6 I've already tried a few combinations of cast() and convert() but haven't cracked it yet. 回答1: CONV(BINARY(CONCAT(setting1, setting2)), 2, 10) 回答2: Didn't try, but try

问题 I need to implement a decimal to chars converter. I have 26 chars available, so it's about converting an integer to base 26 system and then, changing each number to it's alphabet counterpart. I don't want to use the characters 0-9 in the final result string . I can use to_s() method and this is how it goes: 82.to_s(26) #=> "34" / which gives me "de" 120.to_s(26) #=> "4g" / which should give me "aep", but it's not Ruby to_s() method returns a value in a format that is not helpful. Number 82 is

问题 I'm new to mysql and need some basic things. I need to round the decimals like : 21,4758 should be 21,48 0,2250 should be 0,22 23,0850 should be 23,08 22,9950 should be 22,99 I tried lots of thing but couldn't make it. Thanks. 回答1: DECLARE @old decimal(38, 10) DECLARE @p decimal(38, 10) SET @p = 21.4758 SET @p = @p * 100 SET @p = @p - 0.01 SET @p = ROUND(@p, 0) SET @p = @p / 100.0 SELECT @p 回答2: Try this: // round the value to two decimal places SELECT ROUND(<YOUR_FIELD>, 2) field FROM <YOUR

问题 I looked at decimal in C# but I wasnt 100% sure what it did. Is it lossy? in C# writing 1.0000000000001f+1.0000000000001f results in 2 when using float ( double gets you 2.0000000000002 which is correct) is it possible to add two things with decimal and not get the correct answer? How many decimal places can I use? I see the MaxValue is 79228162514264337593543950335 but if i subtract 1 how many decimal places can I use? Are there quirks I should know of? In C# its 128bits, in other language

问题 I'm trying to find the number less than 1000 that produces the longest string of repeated numbers when it divides 1. I have a list of decimal numbers and have to find the ones which have the longest repeated sequence. Here's what I have so far numbers = [*2..999] decimal_representations = numbers.map { |number| 1.to_f/number } decimal_representations.map!(&:to_s) I can produce a three dimensional array by using regex. Regex /(.+)\1+/ produces an array of repeated substrings. I want to find

问题 I have a method like this: Prefix GetPrefix(decimal value) { if(value > 11000000000000000000) return Prefix.CosmicBig; if(value > 1000000000000000) return Prefix.ReallyBig; if(value > 3000000000000) return Prefix.Big; if(value > 50000000) return Prefix.Okay; if(value > 750000) return Prefix.MostlyNormal; if(value > 750000) return Prefix.SoSo; if(value > 750) return Prefix.Small; return Prefix.MiserablySmall; } The exact values are not important. What matters is that they are sometimes changed

问题 Can someone explain why the following three examples are not all equal ? ipdb> Decimal(71.60) == Decimal(71.60) True ipdb> Decimal('71.60') == Decimal('71.60') True ipdb> Decimal(71.60) == Decimal('71.60') False Is there a general 'correct' way to create Decimal objects in Python? (ie, as strings or as floats) 回答1: Floating point numbers, what are used by default, are in base 2. 71.6 can't be accurately represented in base 2. (Think of numbers like 1/3 in base 10). Because of this, they will

问题 The following program will print the fields and whether the are constant or not using IsLiteral public static class Program { public static void Main(string[] args) { foreach (var field in typeof(Program).GetFields()) { System.Console.WriteLine(field.Name + " IsLiteral: " + field.IsLiteral); } System.Console.ReadLine(); } public const decimal DecimalConstant = 99M; public const string StringConstant = "StringConstant"; public const int IntConstant = 1; public const double DoubleConstant = 1D;