floating-accuracy

Handle decimal numbers in mysqli

ε祈祈猫儿з 提交于 2020-07-03 05:52:09
问题 I have to put the price of some items inside a mysql table. When creating the table I'm using DECIMAL(10,2) as I don't need more than 2 digits after the comma (for example: 123,45 would be accepted as an input but 123,456 would be rounded to 123,45 with PHP). First question: using DECIMAL(10,2) , how do I know how many numbers may be stored before the comma? I know it is not 10, as 10 is just the precision Mysql uses when doing math with those numbers: so where the length of the number itself

How to deal with inexact floating point arithmetic results in Rust? [closed]

牧云@^-^@ 提交于 2020-05-14 13:52:08
问题 Closed . This question needs to be more focused. It is not currently accepting answers. Want to improve this question? Update the question so it focuses on one problem only by editing this post. Closed 2 years ago . How to deal with floating point arithmetic in Rust? For example: fn main() { let vector = vec![1.01_f64, 1.02, 1.03, 1.01, 1.05]; let difference: Vec<f64> = vector.windows(2).map(|slice| slice[0] - slice[1]).collect(); println!("{:?}", difference); } Returns: [-0

Do FP operations give EXACTLY the same result on various x86 CPUs?

*爱你&永不变心* 提交于 2020-05-11 05:05:54
问题 Do different x86 CPUs (with build-in FPUs and reasonably recent, say launched this millenium) produce exactly the same result for their Floating Point primitives, assuming the same instruction is available on the CPUs being compared, same input and same operating parameters such as rounding mode? I'm not interested in differences in timing, nor in the Pentium FDIV bug (which does not qualify only because that incident is ancient). I guess the answer is yes for addition, subtraction, negation,

Do FP operations give EXACTLY the same result on various x86 CPUs?

坚强是说给别人听的谎言 提交于 2020-05-11 05:02:07
问题 Do different x86 CPUs (with build-in FPUs and reasonably recent, say launched this millenium) produce exactly the same result for their Floating Point primitives, assuming the same instruction is available on the CPUs being compared, same input and same operating parameters such as rounding mode? I'm not interested in differences in timing, nor in the Pentium FDIV bug (which does not qualify only because that incident is ancient). I guess the answer is yes for addition, subtraction, negation,