rounding

问题 >>> sum([0.3, 0.1, 0.2]) 0.6000000000000001 >>> sum([0.3, 0.1, 0.2]) == 0.6 False What can I do to make the result be exactly 0.6? I don't want to round the result to a certain number of decimal digits because then I could lose precision for other list instances. 回答1: A float is inherently imprecise in pretty much every language because it cannot be represented precisely in binary. If you need exact precision use the Decimal class : from decimal import Decimal num1 = Decimal("0.3") num2 =

问题 >>> sum([0.3, 0.1, 0.2]) 0.6000000000000001 >>> sum([0.3, 0.1, 0.2]) == 0.6 False What can I do to make the result be exactly 0.6? I don't want to round the result to a certain number of decimal digits because then I could lose precision for other list instances. 回答1: A float is inherently imprecise in pretty much every language because it cannot be represented precisely in binary. If you need exact precision use the Decimal class : from decimal import Decimal num1 = Decimal("0.3") num2 =

问题 Is there any way to get correct rounding with the i387 fsqrt instruction?... ... aside from changing the precision mode in the x87 control word - I know that's possible, but it's not a reasonable solution because it has nasty reentrancy-type issues where the precision mode will be wrong if the sqrt operation is interrupted. The issue I'm dealing with is as follows: the x87 fsqrt opcode performs a correctly-rounded (per IEEE 754) square root operation in the precision of the fpu registers,

问题 Is there any way to get correct rounding with the i387 fsqrt instruction?... ... aside from changing the precision mode in the x87 control word - I know that's possible, but it's not a reasonable solution because it has nasty reentrancy-type issues where the precision mode will be wrong if the sqrt operation is interrupted. The issue I'm dealing with is as follows: the x87 fsqrt opcode performs a correctly-rounded (per IEEE 754) square root operation in the precision of the fpu registers,

问题 I know svg has an in built function to do rounded corners, but I need to do rounded corners on only 2 of the four corners. I know I can draw multiple rectangles on top of each other to imitate that, but that seems kind of cheesy. Any way to do it using clipping or any d3.js method? Right now I have a horizontal bar graph that has rects like: rects.enter().append("rect") .attr("x",function(d,i) { return x(0); }) .attr("width",function(d) { return x(d.value) - x(0); }) .attr("height",y

问题 One of the purposes of the C1 bit in the x87 FPU status word is to show whether or not an inexact result was rounded up. Does SSE/AVX provide any such indication for scalar operations? I did not see a similar bit in the MXCSR register. Am I forced to use x87 instructions if I want this information? 回答1: SSE/AVX do not provide hardware support for detecting this, even for scalar instructions like addss . SSE was designed for SIMD, with 4 floats per XMM vector, and presumably Intel didn't want

问题 I do not know how to round a number in MongoDB. I only find how to do it with 2 decimals but not with more decimals. "location" : { "type" : "Point", "coordinates" : [ -74.00568, 40.70511 ] } These is an example of a coordinate that I need to round with 3 numbers after the dot. Thank you 回答1: For 3 decimal rounding, you can use this formula. $divide: [ {$trunc: { $multiply: [ "$$coordinate" , 1000 ] } }, 1000 ] For example, with your sample data, and using this aggregation: db.getCollection(

问题 I am attempting to do some calculations in Excel on numbers that include long decimals. However, Excel doesn't seem to let me populate the cells with the numbers I would like to use. Example: If I enter 600000.00000000030000000000 into a standard cell on a new spreadsheet, it gets converted to 600000. This makes sense as the cell format is set to General. If I set the cell format to Number and set the decimals places to 20, I would expect to be able to enter the number properly. Instead, the

问题 I am attempting to do some calculations in Excel on numbers that include long decimals. However, Excel doesn't seem to let me populate the cells with the numbers I would like to use. Example: If I enter 600000.00000000030000000000 into a standard cell on a new spreadsheet, it gets converted to 600000. This makes sense as the cell format is set to General. If I set the cell format to Number and set the decimals places to 20, I would expect to be able to enter the number properly. Instead, the

问题 I am attempting to do some calculations in Excel on numbers that include long decimals. However, Excel doesn't seem to let me populate the cells with the numbers I would like to use. Example: If I enter 600000.00000000030000000000 into a standard cell on a new spreadsheet, it gets converted to 600000. This makes sense as the cell format is set to General. If I set the cell format to Number and set the decimals places to 20, I would expect to be able to enter the number properly. Instead, the