decimal

问题 The main question: How many digits? Let me explain. I have a number in binary system: 11000000 and in decimal is 192. After converting to decimal, how many digits it will have (in dicimal)? In my example, it's 3 digits. But, it isn't a problem. I've searched over internet and found one algorithm for integral part and one for fractional part. I'm not quite understand them, but (I think) they works. When converting from binary to octal, it's more easy: each 3 bits give you 1 digit in octal.

问题 How you can validate against the scale of a decimal? For example, let's say we want to store ratings for a hotel with maximum 2 decimal points after the decimal. 4.34. 3.76. etc I've read online that sqlite with truncate based on the precision/scale you've tied to the column. So if you have a precision 3, scale 2, and enter 1.34567 1.35 will be stored. However, I'm using postgres and this is not the case. I enter this, and the DB stores the full thing somehow despite my precision 3 scale 2. t

问题 In MySQL I have a DECIMAL field set to 10,10. However, whenever I enter a new value, it shows 0.9999999999. What would I need to change to accept any number with an accuracy of 10 digits after the comma? It does work with 10,6 for some reason. PS: I want to insert exchange rates. 回答1: The first number is the total number of digits to store, the second one is the number of digits on the fractional part. So DECIMAL (10, 10) is only for 10 fractional digits. You can use something like DECIMAL

问题 We have an interesting problem were we need to determine the decimal precision of a users input (textbox). Essentially we need to know the number of decimal places entered and then return a precision number, this is best illustrated with examples: 4500 entered will yield a result 1 4500.1 entered will yield a result 0.1 4500.00 entered will yield a result 0.01 4500.450 entered will yield a result 0.001 We are thinking to work with the string, finding the decimal separator and then calculating

问题 I've been participating in a programming contest and one of the problems' input data included a fractional number in a decimal format: 0.75 is one example. Parsing that into Double is trivial (I can use read for that), but the loss of precision is painful. One needs to be very careful with Double comparisons (I wasn't), which seems redundant since one has Rational data type in Haskell. When trying to use that, I've discovered that to read a Rational one has to provide a string in the

问题 This should be easy, but I can’t find a built in method for it, the .net framework must have a method to do this! private decimal RoundDownTo2DecimalPlaces(decimal input) { if (input < 0) { throw new Exception("not tested with negitive numbers"); } // There must be a better way! return Math.Truncate(input * 100) / 100; } 回答1: If you are rounding down then you need: Math.Floor(number * 100) / 100; if you are looking for something called 'bankers rounding' (probably not if it's for output and

问题 I want to format a number so that it always have at least two decimal places. Samples: 1 2.1 123.456 234.45 Output: 1.00 2.10 123.456 234.45 回答1: You could fix to 2 or the count of current places; var result = num.toFixed(Math.max(2, (num.toString().split('.')[1] || []).length)); 回答2: How about using Intl : Intl.NumberFormat(navigator.language, { minimumFractionDigits: 2, maximumFractionDigits: 10, }).format(num) 回答3: Try this: var num = 1.2; function decimalPlaces(num) { var match = (''+num)

问题 I'm wondering whether this is a bug or if I'm going something wrong. I'm loading values with a SqlDataReader from a SQL Server 2008 database but under certain circumstances, it fails to convert the SQL values into .net values. (.NET 4.0) I have traced it down to an test-case which demonstrates the actual problem: Working example: "select convert(decimal(38, 19), 260000 ) as test" rs.GetValue(1); --> returns 260000 (decimal) Not working exmaple: "select convert(decimal(36, 26), 260000 ) as

问题 In MS SQL, I need a approach to determine the largest scale being used by the rows for a certain decimal column. For example Col1 Decimal(19,8) has a scale of 8, but I need to know if all 8 are actually being used, or if only 5, 6, or 7 are being used. Sample Data: 123.12345000 321.43210000 5255.12340000 5244.12345000 For the data above, I'd need the query to either return 5, or 123.12345000 or 5244.12345000. I'm not concerned about performance, I'm sure a full table scan will be in order, I

问题 The number is bigger than int & long but can be accomodated in Decimal . But the normal ToString or Convert methods don't work on Decimal . 回答1: I believe this will produce the right results where it returns anything, but may reject valid integers. I dare say that can be worked around with a bit of effort though... (Oh, and it will also fail for negative numbers at the moment.) static string ConvertToHex(decimal d) { int[] bits = decimal.GetBits(d); if (bits[3] != 0) // Sign and exponent {