rounding

LINQ to SQL Math.Round Problem

微笑、不失礼 提交于 2020-01-04 13:39:14
问题 I'm having an issue with a query I wrote where for some reason the variable that I'm using to store a decimal value in returns 6 values after the decimal place (they're mostly 0). I have tried the following (and different combinations using Math.Round) with no luck. Sales = (from invhed in INVHEAD ... // Joins here orderby cust.State ascending select new Sale { InvoiceLine = inv.InvoiceLine, InvoiceNum = inv.InvoiceNum, ... NetPrice = Math.Round((inv.ExtPrice - inv.Discount) * (Decimal)

NSTimeInterval to readable NSNumber

帅比萌擦擦* 提交于 2020-01-04 01:52:27
问题 NSTimeInterval == double; (e.g. 169.12345666663) How can I round up this double so that there are only 2 digits left after the "dot"? It would be very good if the result is a NSNumber. 回答1: If this is for display purposes, take a look at NSNumberFormatter. If you really want to round the double in your calculations for some reason, you can use the standard C round() function. 回答2: A NSDecimal can be rounded to a specified number of digits with NSDecimalRound(). double d = [[NSDate date]

Round up to nearest power of 10

和自甴很熟 提交于 2020-01-04 01:49:12
问题 I'm trying to figure out how to round up a number (greater than 0) to the nearest power of 10. Examples: roundUp(23.4) = 100 roundUp(2.34) = 10 roundUp(.234) = 1 roundUp(0.0234) = 0.1 roundUp(0.00234) = 0.01 For numbers greater than 1, I believe this works: 10^(ceil(log10(x))) But for numbers between 0 and 1, I'm not sure how to arrive at the answer. 回答1: Oops. I didn't realize the function actually DOES work for numbers between 0 & 1. It was a brain fart that I saw a negative number for

Printing to specific (runtime-determined) precision in Julia

≯℡__Kan透↙ 提交于 2020-01-03 19:18:42
问题 Given a value x and an integer n (assigned at runtime), I want to print x to exactly n digits after the decimal (after rounding if needed). print(round(x, n)) works fine for (x,n)=(3.141592, 3) but for (x,n)=(2.5,5) , it prints just 2.5 , not 2.50000 (5 digits after decimal point). If I knew n at runtime, say 5, I could do @printf("%.5f", x) But @printf being a macro needs n to be known at compile time. Is this possible using some show magic or something else? 回答1: Using the fresh new Format

Printing to specific (runtime-determined) precision in Julia

◇◆丶佛笑我妖孽 提交于 2020-01-03 19:18:26
问题 Given a value x and an integer n (assigned at runtime), I want to print x to exactly n digits after the decimal (after rounding if needed). print(round(x, n)) works fine for (x,n)=(3.141592, 3) but for (x,n)=(2.5,5) , it prints just 2.5 , not 2.50000 (5 digits after decimal point). If I knew n at runtime, say 5, I could do @printf("%.5f", x) But @printf being a macro needs n to be known at compile time. Is this possible using some show magic or something else? 回答1: Using the fresh new Format

MS Access Rounding Precision With Group By

两盒软妹~` 提交于 2020-01-03 15:56:15
问题 Why doesn't the average of the score of an employee of each month, when summed, equal the average of the employees score (ever)? Average SELECT Avg(r.score) AS rawScore FROM (ET INNER JOIN Employee AS e ON ET.employeeId = e.id) INNER JOIN (Employee AS a INNER JOIN Review AS r ON a.id = r.employeeId) ON ET.id = r.ETId WHERE (((e.id)=@employeeId)) Returns 80.737 Average By Month SELECT Avg(r.score) AS rawScore, Format(submitDate, 'mmm yy') AS MonthText, month(r.submitDate) as mm, year

Default rounding mode in python, and how to specify it to another one?

旧时模样 提交于 2020-01-03 09:06:14
问题 What is the default rounding mode (rounding to nearest, etc) in Python? And how can we specify it? 回答1: With IEEE754-based platform (as most modern ones do, including x86, ARM, MIPS...), it's default mode "round to nearest, ties to even" is the only mode available in Python standard library. That is "provided" by standardized defaults and absense of library methods to change it. There are more languages that doesn't allow to change rounding mode - e.g. Java - so this isn't an isolated Python

Round each number in a Python pandas data frame by 2 decimals

强颜欢笑 提交于 2020-01-03 08:14:12
问题 This works p_table.apply(pd.Series.round) however it has no decimal places Documentation says import pandas as pd Series.round(decimals=0, out=None) i tried this p_table.apply(pd.Series.round(2)) but get this error: unbound method round() must be called with Series instance as first argument (got int instance instead) How do I round all elements in the data frame to two decimal places? [EDIT] Figured it out. import numpy as np np.round(p_table, decimals=2) 回答1: import numpy as np np.round(p

Rounding up float point numbers bash

房东的猫 提交于 2020-01-03 06:48:11
问题 Ok, so I'm trying to round up an input of 17.92857 , so that it gets an input of 17.929 in bash. My code so far is: read input echo "scale = 3; $input" | bc -l However, when I use this, it doesn't round up, it returns 17.928 . Does anyone know any solutions to this? 回答1: In case input contains a number, there is no need for an external command like bc . You can just use printf : printf "%.3f\n" "$input" Edit: In case the input is a formula, you should however use bc as in one of the following

Convert between Degree and Milliseconds

只谈情不闲聊 提交于 2020-01-03 02:21:14
问题 I know the formular for conversion from Degree to Milliseconds and vice-versa. It can be implemented like that: protected function decimal_to_milisecond($dec) { if (!empty($dec)) { $vars = explode(".",$dec); if (count($vars) == 2) { $deg = $vars[0]; $tempma = "0.".$vars[1]; $tempma = $tempma * 3600; $min = floor($tempma / 60); $sec = $tempma - ($min*60); return round((((($deg * 60) + $min) * 60 + $sec) * 1000)); } else return false; } else return false; } function milisecond_to_decimal($sec)