rounding

NSDecimalNumber(x).intValue returns -2, 0, 15 and 199, depending on the amount of decimals in x (x = 199.999…5)

谁都会走 提交于 2020-06-26 04:26:06
问题 We found an interesting case in our business logic that totally breaks our logic and we don't understand why NSDecimalNumber and Decimal behaves the way it does. My playground for the cases is as follows: import Foundation let pQuantity = Decimal(string: "0.2857142857142857")! let pPrice = Decimal(string: "7.00000000000000035")! let calced = NSDecimalNumber(decimal: pQuantity * pPrice * Decimal(integerLiteral: 100)) // 200 let decimal = calced.decimalValue // 199

Rounding Mechanism to nearest 0.05

无人久伴 提交于 2020-06-22 11:37:46
问题 I would like to solve rounding mechanism by using php4,5.2 and below (not 5.3) Currently I am doing 0.05 rounding, something like this page: http://www.bnm.gov.my/index.php?ch=209&pg=657&ac=568 before rounding | after rounding 89.90 | 89.90 89.91 | 89.90 89.92 | 89.90 89.93 | 89.95 89.94 | 89.95 89.95 | 89.95 89.96 | 89.95 89.97 | 89.95 89.98 | 90.00 89.99 | 90.00 I try to use string to split it out and manually do adding, but not really a good solution, hoping here can find someone to solve

Round a decimal number to the first decimal position that is not zero

泄露秘密 提交于 2020-06-08 15:47:03
问题 I want to shorten a number to the first significant digit that is not 0. The digits behind should be rounded. Examples: 0.001 -> 0.001 0.00367 -> 0.004 0.00337 -> 0.003 0.000000564 -> 0.0000006 0.00000432907543029 -> 0.000004 Currently I have the following procedure: if (value < (decimal) 0.01) { value = Math.Round(value, 4); } Note: numbers will always be positive the number of significant digits will always be 1 values larger 0.01 will always be rounded to two decimal places, hence the if <

jq: error: round/0 is not defined at <top-level>

杀马特。学长 韩版系。学妹 提交于 2020-05-30 07:37:45
问题 round function in jq doesn't work. $ jq '10.01 | round' jq: error: round/0 is not defined at <top-level>, line 1: 10.01 | round jq: 1 compile error $ jq --help jq - commandline JSON processor [version 1.5-1-a5b5cbe] What I need to do? 回答1: Seems like round is unavailable in your build. Either upgrade jq or implement round using floor : def round: . + 0.5 | floor; Usage example: $ jq -n 'def round: . + 0.5 | floor; 10.01 | round' 10 来源: https://stackoverflow.com/questions/56593531/jq-error

How to round float down to a given precision?

谁说我不能喝 提交于 2020-05-27 04:56:21
问题 I need a way to round a float to a given number of decimal places, but I want to always round down. For example, instead of >>> round(2.667, 2) 2.67 I would rather have >>> round_down(2.667, 2) 2.66 回答1: Something like this should work for whatever number of digits you want to do: >>> import math >>> def round_down(num,digits): factor = 10.0 ** digits return math.floor(num * factor) / factor >>> round_down(2.667,2) 2.66 回答2: You've got a friend in quantize and ROUND_FLOOR >>> from decimal

Adding Decimal Places (3 Decimal Places) in Java

寵の児 提交于 2020-05-17 16:45:10
问题 I was wondering how to make avg in the following program round to 3 decimal places: public class BaseballCalculator { public static void main(String args[]) { Scanner myScanner = new Scanner(System.in); double atBats; double baseHits; double onBase; double avg; double onBasePercentage; System.out.println("How many atbats did you have? "); atBats = myScanner.nextDouble(); System.out.println("How many base hits and homeruns did you have? "); baseHits = myScanner.nextDouble(); System.out.println

Adding Decimal Places (3 Decimal Places) in Java

こ雲淡風輕ζ 提交于 2020-05-17 16:40:48
问题 I was wondering how to make avg in the following program round to 3 decimal places: public class BaseballCalculator { public static void main(String args[]) { Scanner myScanner = new Scanner(System.in); double atBats; double baseHits; double onBase; double avg; double onBasePercentage; System.out.println("How many atbats did you have? "); atBats = myScanner.nextDouble(); System.out.println("How many base hits and homeruns did you have? "); baseHits = myScanner.nextDouble(); System.out.println

What is the correct/standard way to check if difference is smaller than machine precision?

别等时光非礼了梦想. 提交于 2020-05-09 18:27:46
问题 I often end up in situations where it is necessary to check if the obtained difference is above machine precision. Seems like for this purpose R has a handy variable: .Machine$double.eps . However when I turn to R source code for guidelines about using this value I see multiple different patterns. Examples Here are a few examples from stats library: t.test.R if(stderr < 10 *.Machine$double.eps * abs(mx)) chisq.test.R if(abs(sum(p)-1) > sqrt(.Machine$double.eps)) integrate.R rel.tol < max(50*

What is the correct/standard way to check if difference is smaller than machine precision?

为君一笑 提交于 2020-05-09 18:27:38
问题 I often end up in situations where it is necessary to check if the obtained difference is above machine precision. Seems like for this purpose R has a handy variable: .Machine$double.eps . However when I turn to R source code for guidelines about using this value I see multiple different patterns. Examples Here are a few examples from stats library: t.test.R if(stderr < 10 *.Machine$double.eps * abs(mx)) chisq.test.R if(abs(sum(p)-1) > sqrt(.Machine$double.eps)) integrate.R rel.tol < max(50*

Round to 25, 50, 75, 100

故事扮演 提交于 2020-05-09 16:30:10
问题 I'm not a Math person so I'm having a hard time to come up with a calculation to round the decimals to 25, 50, 75 and 100. And this will not be the typical round off because the decimals will not be decreased but only increased. Example: if 11.12, round to 11.25 if 11.34, round to 11.50 if 11.52, round to 11.75 if 11.76, round to 12.00 Here's my starting method: public float RoundNearestCents(String price) { float srp; return srp; } 回答1: public float RoundNearestCents(double d) { return