Questions tagged [floating-point]

12

votes

3

answers

Converting String "2½" (two and a half) into 2.5

asked 2012-01-29T07:07:11.973

8

votes

2

answers

Why is there no warning like C4738 for double?

c++ visual-c++ floating-point compiler-warnings

asked 2012-01-26T14:59:22.367

9

votes

4

answers

lossless conversion of float to string and back: is it possible?

string floating-point floating-accuracy

asked 2012-01-24T16:32:18.290

9

votes

1

answers

Why does the IEEE 754 standard use a 127 bias?

floating-point ieee-754

asked 2012-01-18T12:07:39.640

19

votes

2

answers

Is catastrophic cancellation an issue when calculating dot products of floating point vectors? And if so, how is it typically addressed?

c++ math floating-point physics

asked 2012-01-15T10:56:59.647

162

votes

10

answers

Why does Double.NaN==Double.NaN return false?

java floating-point nan scjp ocpjp

asked 2012-01-11T13:02:30.917

17

votes

4

answers

How to convert packed integer (16.16) fixed-point to float?

floating-point fixed-point

asked 2011-12-26T20:04:48.397

40

votes

6

answers

Is using increment (operator++) on floats bad?

c++ floating-point increment

asked 2011-12-19T12:29:31.567

21

votes

5

answers

Formatting a double and not rounding off

java floating-point double

asked 2011-12-19T11:28:46.583

10

votes

6

answers

How do I compute the logarithm of 1 minus the exponent of a given small number in python

python math numpy floating-point scipy

asked 2011-11-29T01:54:07.950

30

votes

3

answers

TypeError: 'float' object not iterable

python python-3.x for-loop floating-point typeerror

asked 2011-11-14T10:10:56.420

32

votes

6

answers

what does NaN mean for doubles?

c# floating-point nan

asked 2011-11-13T15:18:02.550

8

votes

3

answers

Implicit type conversion in C

c floating-point type-conversion

asked 2011-11-09T23:14:50.397

14

votes

2

answers

Can we rely on op== to binary-compare floating-point values?

c++ floating-point

asked 2011-11-08T01:15:11.950

14

votes

3

answers

C++ gamedev: truncating float to int

c++ floating-point rounding

asked 2011-10-31T01:49:31.463

12

votes

10

answers

Can I guarantee the C++ compiler will not reorder my calculations?

c++ optimization floating-point ieee-754 double-double-arithmetic

asked 2011-10-05T02:29:51.543

12

votes

3

answers

Ruby: Why is 1.025.round(2) rounded to 1.02?

ruby floating-point decimal rounding

asked 2011-09-29T08:38:59.280

25

votes

4

answers

Can float (or double) be set to NaN?

java floating-point double nan

asked 2011-09-14T15:50:59.500

26

votes

6

answers

How can floating point calculations be made deterministic?

c floating-point deterministic

asked 2011-09-09T18:16:23.863

49

votes

2

answers

Haskell ranges and floats

haskell floating-point range-notation

asked 2011-09-03T00:44:06.347

21

votes

3

answers

Cortex A9 NEON vs VFP usage confusion

c++ c floating-point arm neon

asked 2011-09-01T11:59:02.377

9

votes

3

answers

Problems with Rounding Floats in Python

python numpy types floating-point rounding

asked 2011-08-31T04:35:10.703

17

votes

4

answers

Comparing floating point numbers in C

c comparison floating-point zero

asked 2011-08-10T09:36:03.600

14

votes

2

answers

Python vs Javascript floating point arithmetic giving very different answers. What am I doing wrong?

javascript python floating-point

asked 2011-08-07T17:56:14.820

25

votes

12

answers

How does Excel successfully round floating point numbers even though they are imprecise?

c++ excel math floating-point rounding

asked 2011-08-03T17:43:47.300

75

votes

3

answers

Why 0.1 + 0.2 == 0.3 in D?

floating-point floating-accuracy d constantfolding

asked 2011-07-29T14:03:59.403

47

votes

2

answers

Change floating point rounding mode

c++ c floating-point x86

asked 2011-07-29T01:26:12.120

15

votes

1

answers

Rounding problem with rspec tests when comparing float arrays

ruby-on-rails arrays testing rspec floating-point

asked 2011-07-28T08:19:45.457

37

votes

5

answers

Floating point keys in std:map

c++ stl floating-point

asked 2011-07-13T19:38:03.893

77

votes

6

answers

Python float to int conversion

python floating-point int

asked 2011-07-04T09:25:19.463

45

votes

6

answers

Using "double" as counter variables in loops

c# c++ loops floating-point counter

asked 2011-06-30T14:24:06.593

9

votes

4

answers

Difference in accuracy with floating point division vs multiplication

algorithm language-agnostic numbers floating-point

asked 2011-06-26T07:18:30.270

12

votes

5

answers

Converting 8 bytes of little-endian binary into a double precision float

java parsing floating-point precision endianness

asked 2011-06-24T16:58:55.070

2326

votes

12

answers

Why doesn't GCC optimize aaaaaa to (aaa)(aaa)?

gcc assembly floating-point compiler-optimization fast-math

asked 2011-06-21T18:49:55.103

58

votes

8

answers

Is there any advantage to using pow(x,2) instead of x*x, with x double?

c++ math floating-point

asked 2011-06-12T09:18:15.680

27

votes

4

answers

Reading 32 bit signed ieee 754 floating points from a binary file with python?

python parsing floating-point binaryfiles ieee-754

asked 2011-06-08T22:25:59.800

12

votes

2

answers

How to convert a string to float?

objective-c ios cocoa-touch nsstring floating-point

asked 2011-06-05T02:09:52.647

17

votes

7

answers

Problem Storing Latitude and Longitude values in MySQL database

php mysql types floating-point

asked 2011-05-31T13:17:39.703

1315

votes

39

answers

Format number to always show 2 decimal places

javascript floating-point number-formatting

asked 2011-05-26T05:22:51.133

19

votes

5

answers

Why do I need 17 significant digits (and not 16) to represent a double?

c floating-point precision ieee-754 floating-accuracy

asked 2011-05-25T00:11:12.563

22

votes

6

answers

Java, impossible cast Object to Float.....why?

java casting floating-point

asked 2011-05-12T16:05:33.050

18

votes

5

answers

losing precision converting from java BigDecimal to double

java floating-point double precision bigdecimal

asked 2011-04-21T20:45:07.690

14

votes

3

answers

MATLAB precision

matlab floating-point precision

asked 2011-04-06T15:29:02.960

40

votes

2

answers

How do I pad a printf to take account of negative signs and variable length numbers?

c formatting floating-point printf

asked 2011-03-16T12:20:20.847

20

votes

6

answers

how to change 39.54484700000000 to 39.54 and using python

python floating-point

asked 2011-03-05T06:51:56.867

34

votes

5

answers

How can I make sure a float will always be rounded up with PHP?

php floating-point rounding precision ceil

asked 2011-02-23T16:18:13.787

134

votes

33

answers

Truncate (not round off) decimal numbers in JavaScript

javascript floating-point floating-accuracy

asked 2011-02-06T10:42:08.843

11

votes

3

answers

Coercing float into unsigned char on ARM vs. Intel

iphone floating-point arm intel coercion

asked 2011-01-20T20:39:36.510

36

votes

8

answers

Printing double without losing precision

c++ floating-point iostream

asked 2011-01-19T17:49:40.117

13

votes

2

answers

iphone / Objective C - Comparing doubles not working

iphone objective-c ios floating-point

asked 2011-01-19T07:13:32.970

Questions tagged floating-point