decimal

Decimal to Octal Conversion [duplicate]

♀尐吖头ヾ 提交于 2019-12-02 23:13:17
问题 This question already has answers here : Closed 7 years ago . Possible Duplicate: Decimal Conversion error I am writing a program for a class and is having trouble figuring how to convert an octal number to an decimal number. Here is what I been trying to do so far: import java.util.Scanner; public class test { public static void main ( String args[]) { Scanner input = new Scanner(System.in); System.out.print("Enter number: "); int oct = input.nextInt(); int d2count = 0; int result=0; int

How to convert byte array (containing hex values) to decimal

三世轮回 提交于 2019-12-02 23:06:10
问题 I am writing some code for an Atmel micro-controller. I am getting some data via Uart, and I store these hex values into an array. Suppose the elements of the array are: 1F, 29, and 3C. I want to have one hex number like 0x1F293C, and convert it to a decimal number. So, I want to get “2042172” at the end. The array could have n elements, so I need a general solution. Thank you. 回答1: If you have a array of characters like "0x1F293C" and want to convert it to int try this code: char

Problems with Rounding Decimals (python)

时光怂恿深爱的人放手 提交于 2019-12-02 22:15:13
In my program, decimal accuracy is very important. A lot of my calculations must be accurate to many decimal places (such as 50). Because I am using python, I have been using the decimal module (with context().prec = 99. ie; Set to have 99 decimal places of accuracy when instantiating a decimal object ) as pythonic floats don't allow anywhere near such accuracy. Since I wish for the User to specify the decimal places of accuracy of the calculations, I've had to implement several round() functions in my code. Unfortuneately, the inbuilt round function and the decimal object do not interact well

Precision of decimals in Python

纵饮孤独 提交于 2019-12-02 21:56:32
问题 I don't mean precision as in how many numbers are displayed after the decimal. I mean precision as in the decimal I am trying to use in this pictograph function keeps coming up one tenth shy of what it should be. I have tried using multiple different strategies including importing the decimal module. Here is the function I am trying to use. values = [('tens', 10), ('fives', 5), ('ones', 1), ('tenths', 0.1)] def get_digits(num): num = int(num * 10) num = float(num) / 10 output_dict = {} for

What is the purpose of Decimal.One, Decimal.Zero, Decimal.MinusOne in .Net

余生颓废 提交于 2019-12-02 21:33:57
Simple question - why does the Decimal type define these constants? Why bother? I'm looking for a reason why this is defined by the language, not possible uses or effects on the compiler. Why put this in there in the first place? The compiler can just as easily in-line 0m as it could Decimal.Zero, so I'm not buying it as a compiler shortcut. JaredPar Small clarification. They are actually static readonly values and not constants. That has a distinct difference in .Net because constant values are inlined by the various compilers and hence it's impossible to track their usage in a compiled

Output in MatLab has a capped amount of decimal points [duplicate]

試著忘記壹切 提交于 2019-12-02 21:07:33
问题 This question already has answers here : Is it possible in matlab to explicitly format the output numbers? (7 answers) Closed 4 years ago . I have modified some code in MatLab so that it will give me the root of the function cos(x) - 3*x. When I run the code and ask for it to return the value of xnew (as xnew should equal the root of the function) it returns xnew to only 4 decimal points. I would like for it to be more than this. Does anyone know why it capping this value? x = 0; N = 100000;

Decimal TryParse in VBA

老子叫甜甜 提交于 2019-12-02 19:57:30
问题 I am attempting to tryparse using decimal; however, I keep getting an "Object Required" run-time error. I'm not certain what I'm doing wrong. I'm used to doing a tryparse in C#. This is VBA, so the language translation is not clicking just yet. Any help appreciated. Sub try() Dim val As Variant Dim res As Boolean res = Decimal.TryParse("2.5", val) MsgBox (res & ":" & val) End Sub 回答1: You can try CInt and check for a specific error using On Error Goto. 回答2: res = cBool(Val("2.5")) should do

Get number of digits before decimal point

我的梦境 提交于 2019-12-02 19:54:39
I have a variable of decimal type and I want to check the number of digits before decimal point in it. What should I do? For example, 467.45 should return 3 . Solution without converting to string (which can be dangerous in case of exotic cultures): static int GetNumberOfDigits(decimal d) { decimal abs = Math.Abs(d); return abs < 1 ? 0 : (int)(Math.Log10(decimal.ToDouble(abs)) + 1); } Note, that this solution is valid for all decimal values UPDATE In fact this solution does not work with some big values, for example: 999999999999998 , 999999999999999 , 9999999999999939 ... Obviously, the

MySQL & PHP decimal precision wrong

戏子无情 提交于 2019-12-02 19:22:36
问题 24151.40 - 31891.10 = -7739.699999999997 I grab these two numbers from a MySQL table with the type as decimal(14,2) 24151.40 31891.10 It is saved exactly as stated above and it echos exactly like that in PHP. But the minute I subtract the second value from the first value, I get a number -7739.699999999997 instead of -7,739.7. Why the extra precision? And where is it coming from? 回答1: From an article I wrote for Authorize.Net: One plus one equals two, right? How about .2 plus 1.4 times 10?

Decimal group seperator for the fractional part

本小妞迷上赌 提交于 2019-12-02 19:19:34
问题 I wonder what would be the best way to format numbers so that the NumberGroupSeparator would work not only on the integer part to the left of the comma, but also on the fractional part, on the right of the comma. Math.PI.ToString("###,###,##0.0##,###,###,###") // As documented .. // ..this doesn't work 3.14159265358979 // result 3.141,592,653,589,79 // desired result As documented on MSDN the NumberGroupSeparator works only to the left of the comma. I wonder why?? 回答1: A little clunky, and it