rounding

问题 I've been struggling with precision nightmare in Java and SQL Server up to the point when I don't know anymore. Personally, I understand the issue and the underlying reason for it, but explaining that to the client half way across the globe is something unfeasible (at least for me). The situation is this. I have two columns in SQL Server - Qty INT and Price FLOAT. The values for these are - 1250 and 10.8601 - so in order to get the total value its Qty * Price and result is 13575.124999999998

问题 I recently got into using MonoGame, and I love the library. However, I seem to be having some issues with drawing bezier curves The result that my code produces looks something like this Look bad, no? The lines aren't smooth at all. Let me show you some of the code: //This is what I call to get all points between which to draw. public static List<Point> computeCurvePoints(int steps) { List<Point> curvePoints = new List<Point>(); for (float x = 0; x < 1; x += 1 / (float)steps) { curvePoints

问题 Can anybody explain or propose a fix for why when I round a decimal in Python 3 with the context set to round half up, it rounds 2.5 to 2, whereas in Python 2 it rounds correctly to 3: Python 3.4.3 and 3.5.2: >>> import decimal >>> context = decimal.getcontext() >>> context.rounding = decimal.ROUND_HALF_UP >>> round(decimal.Decimal('2.5')) 2 >>> decimal.Decimal('2.5').__round__() 2 >>> decimal.Decimal('2.5').quantize(decimal.Decimal('1'), rounding=decimal.ROUND_HALF_UP) Decimal('3') Python 2

问题 I've searched all over the net, but I could not find a solution to my problem. I simply want a function that rounds double values like MS Excel does. Here is my code: #include <iostream> #include "math.h" using namespace std; double Round(double value, int precision) { return floor(((value * pow(10.0, precision)) + 0.5)) / pow(10.0, precision); } int main(int argc, char *argv[]) { /* The way MS Excel does it: 1.27815 1.27840 -> 1.27828 1.27813 1.27840 -> 1.27827 1.27819 1.27843 -> 1.27831 1

问题 I've searched all over the net, but I could not find a solution to my problem. I simply want a function that rounds double values like MS Excel does. Here is my code: #include <iostream> #include "math.h" using namespace std; double Round(double value, int precision) { return floor(((value * pow(10.0, precision)) + 0.5)) / pow(10.0, precision); } int main(int argc, char *argv[]) { /* The way MS Excel does it: 1.27815 1.27840 -> 1.27828 1.27813 1.27840 -> 1.27827 1.27819 1.27843 -> 1.27831 1

问题 I was wondering whether, under specific conditions, it is possible to remove floating point errors without resorting to arbitrary-precision datatypes. The problem is the usual one. The language is Ruby, but it holds in any language: f = 1829.82 => 1829.82 f / 12.0 => 152.485 (f / 12.0).round(2) => 152.48 Why not 152.49? Because due to the finite precision of floating points: format("%18.14f", f) => "1829.81999999999994" format("%18.14f", f / 12.0) => "152.48499999999999" So the rounding is

问题 every time I want to round to 2 decimal places when it comes to zeros it doesnt want to round it... how could I round it to 2 decimal places with zeros at the end which would give me 92.00 instead of just 92 ??? SELECT ROUND(COLUMN_NAME,2) FROM .... it's giving me COLUMN_NAME 92 but I want COLUMN_NAME 92.00 I used TO_CHAR and it worked ROUND(TO_CHAR(COLUMN_NAME),2) thanks guys! 回答1: you may try the TO_CHAR function to convert the result e.g. SELECT TO_CHAR(92, '99.99') AS RES FROM DUAL SELECT

问题 I would like to be able to round a number to n significant figures in SQL. So: 123.456 rounded to 2sf would give 120 0.00123 rounded to 2sf would give 0.0012 I am aware of the ROUND() function, which rounds to n decimal places rather than significant figures. 回答1: select round(@number,@sf-1- floor(log10(abs(@number)))) should do the trick ! Successfully tested on your two examples. Edit : Calling this function on @number=0 won't work. You should add a test for this before using this code.

问题 Python has default round() function, but I was programming with cython and want to replace pythonic code with numpy function. However, I got the following results when experimenting it in terminal. >>> np.around(1.23456789) 1.0 >>> np.around(1.23456789, decimals=0) 1.0 >>> np.around(1.23456789, decimals=1) 1.2 >>> np.around(1.23456789, decimals=2) 1.23 >>> np.around(1.23456789, decimals=3) 1.2350000000000001 >>> np.around(1.23456789, decimals=4) 1.2345999999999999 Which is kind of strange,

问题 I am curious about the existence of any "rounding" standards" when it comes to the calculation of financial data. My initial thoughts are to perform rounding only when the data is being presented to the user (presentation layer). If "rounded" data is then used for further calculations, should be use the "rounded" figure or the "raw" figure? Does anyone have any advice? Please note that I am aware of different rounding methods, i.e. Bankers Rounding etc. 回答1: The first and most important rule: