Alex Rivera | Logout

Questions tagged floating-accuracy

19 questions

Sort by: Newest Votes Active
8
votes
1
answers

Accuracy of floating point arithmetic

c floating-point floating-accuracy ieee-754
asked 2013-03-15T19:59:53.103
13
votes
2
answers

Do FP operations give EXACTLY the same result on various x86 CPUs?

x86 floating-accuracy
asked 2012-10-27T16:53:14.970
10
votes
7
answers

In C and Objective-C, what really is the right way to truncate a float or double to an integer?

objective-c c floating-point floating-accuracy
asked 2012-06-28T12:31:52.173
10
votes
4
answers

Math.Pow taking an integer value

c# floating-accuracy pow
asked 2012-06-25T20:17:53.600
9
votes
6
answers

Math "pow" in Java and C# return slightly different results?

c# java math floating-accuracy
asked 2012-04-12T20:07:16.097
9
votes
4
answers

lossless conversion of float to string and back: is it possible?

string floating-point floating-accuracy
asked 2012-01-24T16:32:18.290
75
votes
3
answers

Why 0.1 + 0.2 == 0.3 in D?

floating-point floating-accuracy d constantfolding
asked 2011-07-29T14:03:59.403
19
votes
5
answers

Why do I need 17 significant digits (and not 16) to represent a double?

c floating-point precision ieee-754 floating-accuracy
asked 2011-05-25T00:11:12.563
9
votes
5
answers

Why does CLng produce different results?

vba casting rounding floating-accuracy
asked 2011-02-14T08:03:10.680
134
votes
33
answers

Truncate (not round off) decimal numbers in JavaScript

javascript floating-point floating-accuracy
asked 2011-02-06T10:42:08.843
12
votes
4
answers

Detecting precision loss when converting from double to float

c++ floating-point rounding floating-accuracy rounding-error
asked 2010-07-13T17:53:18.260
58
votes
1
answers

What is the standard solution in JavaScript for handling big numbers (BigNum)?

javascript floating-accuracy bignum
asked 2010-06-18T18:47:20.503
10
votes
9
answers

Manipulating and comparing floating points in java

java floating-point floating-accuracy
asked 2010-05-24T09:42:53.873
18
votes
7
answers

Why does (int)(33.46639 * 1000000) return 33466389?

c# floating-point floating-accuracy multiplication
asked 2010-03-11T01:52:17.107
29
votes
7
answers

Floating point inaccuracy examples

floating-point floating-accuracy
asked 2010-01-20T10:10:30.243
9
votes
7
answers

Why does ghci say that 1.1 + 1.1 + 1.1 > 3.3 is True?

haskell floating-accuracy ghci
asked 2010-01-10T21:14:18.423
11
votes
4
answers

Python float - str - float weirdness

python string floating-point floating-accuracy ieee-754
asked 2009-11-22T10:28:18.340
19
votes
2
answers

PHP/MySQL: Best money operations/storing practices?

php mysql decimal currency floating-accuracy
asked 2009-07-04T10:51:33.233
32
votes
9
answers

Rounding Errors?

java memory floating-accuracy
asked 2009-06-06T16:41:56.020