division

Incorrect division results

泄露秘密 提交于 2020-01-04 02:44:26
问题 I've got a time calculator that has worked reasonably well for a number of years. One thing that always bothered me, though, was that if one used fractional seconds, the results would fall victim to floating-point "errors". So, I recently switched to using this BigDecimal library. Now, I'm getting math errors. Here's a simplified test case from a bug report I received today: 27436 / 30418 is returning 1 instead of the expected 0.9019659412190151 . To illustrate my problem, here's a Javascript

Binary division issue: bad examples on internet or what am I missing?

寵の児 提交于 2020-01-04 01:57:16
问题 I would like to create a 16 bit CRC. Actually I am entirely ready whit that, so a couple of hours ago I tested it, but did not work properly. But what I discovered is that the examples on the internet might be wrong when its about binary division. I link just two of them (from a lot!): http://www.ross.net/crc/download/crc_v3.txt , http://en.wikipedia.org/wiki/Cyclic_redundancy_check So what I am doing with the binary division is that I use BigIntegers. Just like this: BigInteger divisor = new

Division to percentage in Ruby on Rails

蓝咒 提交于 2020-01-03 08:38:08
问题 If @prescribed_wod_count = @user.workouts.rx_workouts.count returns 4 and @user_workout_count = @user.workouts.count returns 26 how come <%= number_to_percentage(@prescribed_wod_count / @user_workout_count) %> returns 0.000% and not 15% ? 回答1: It does integer division, before you call number_to_percentage. You want <%= number_to_percentage(@prescribed_wod_count.to_f / @user_workout_count) %> to force it to do floating-point 来源: https://stackoverflow.com/questions/4311190/division-to

Complexity - determining the order of growth

空扰寡人 提交于 2020-01-03 05:52:09
问题 I understand how to calculate a function's complexity for the most part. The same goes for determining the order of growth for a mathematical function. [I probably don't understand it as much as I think I do, which is why I'm probably asking this.] For instance: an^3 + bn^2 + cn + d could be written as O(n^3) in big-o notation, since for large enough n the values of the term bn^2 + cn + d are insignificant in comparison to an^3 (the constant coefficients a, b, c and d are left out as well, as

How scale is defined when decimal and bigint are divided?

馋奶兔 提交于 2020-01-02 12:49:32
问题 I have value A of type DECIMAL(19,8) - the scale is 8 , so the number of decimal digits that will be stored to the right of the decimal point is 8 . Now, I am dividing A on B , where B is BIGINT . For, example: SELECT CAST(3 AS DECIMAL(19, 8)) / CAST(27 AS BIGINT) -- 0.111111111111111111111111111 ,CAST(300 AS DECIMAL(19, 8)) / CAST(27 AS BIGINT) -- 11.111111111111111111111111111 ,CAST(75003 AS DECIMAL(19, 8)) / CAST(13664400 AS BIGINT) -- 0.005488934750153684025643277 the output values are

integer division properties

时光总嘲笑我的痴心妄想 提交于 2020-01-02 02:54:32
问题 does the following integer arithmetic property hold? (m/n)/l == m/(n*l) At first I thought I knew answer (does not hold), but now am not sure. Does it hold for all numbers or only for certain conditions, i.e. n > l ? the question pertains to computer arithmetic, namely q = n/m, q*m != n , ignoring overflow. 回答1: Case1 assume m = kn+b (b<n), left = (m/n)/l = ((kn+b)/n)/l = (k+b/n)/l = k/l (b/n=0, because b<n) right = (kn+b)/(n*l) = k/l + b/(n*l) = k/l (b/(n*l)=0, because b<n) => left = right

efficient way to divide a very large number stored in 2 registers by a constant

情到浓时终转凉″ 提交于 2019-12-31 02:49:12
问题 Let's say I want to calculate the following: A/Z Where A is of length 128 bit and Z is 64 bit long. A is stored in 2 64 bit registers since the registers of the system can store up to 64 bits. What would be an efficient way to calculate the result? P.S: I've solved similar multiplication problems by using CSD representations. However, this would require calculating 1/Z first. 回答1: The right way to solve such a problem, is by returning to the basics: divide the most significant register by the

Generic Numeric division

こ雲淡風輕ζ 提交于 2019-12-29 06:59:09
问题 As a general rule, we can take any value of any number type, and divide it by any non-zero value of any number type, and get a reasonable result. 212.7 / 6 // Double = 35.449999999999996 77L / 2.1F // Float = 36.666668 The one exception, that I've found, is that we can't mix a BigInt with a fractional type ( Float or Double ). In the realm of generics, however, there's this interesting distinction between Integral and Fractional types. // can do this def divideI[I](a: I, b: I)(implicit ev:

Make division by zero equal to zero

别等时光非礼了梦想. 提交于 2019-12-28 13:39:51
问题 How can I ignore ZeroDivisionError and make n / 0 == 0 ? 回答1: Check if the denominator is zero before dividing. This avoids the overhead of catching the exception, which may be more efficient if you expect to be dividing by zero a lot. def weird_division(n, d): return n / d if d else 0 回答2: You can use a try / except block for this. def foo(x,y): try: return x/y except ZeroDivisionError: return 0 >>> foo(5,0) 0 >>> foo(6,2) 3.0 回答3: I think try except (as in Cyber's answer) is usually the

Why does this division result in zero?

浪尽此生 提交于 2019-12-28 07:06:12
问题 I was writing this code in C when I encountered the following problem. #include <stdio.h> int main() { int i=2; int j=3; int k,l; float a,b; k=i/j*j; l=j/i*i; a=i/j*j; b=j/i*i; printf("%d %d %f %f\n",k,l,a,b); return 0; } Can anyone tell me why the code is returning zero for the first and third variables ( k and a )? 回答1: What I think you are experiencing is integer arithmetic . You correctly suppose l and b to be 2, but incorrectly assume that k and a will be 3 because it's the same