How deterministic is floating point inaccuracy?

Asked 2008-11-30T08:11:54.757

I understand that floating point calculations have accuracy issues and there are plenty of questions explaining why. My question is if I run the same calculation twice, can I always rely on it to produce the same result? What factors might affect this?

Time between calculations?
Current state of the CPU?
Different hardware?
Language / platform / OS?
Solar flares?

I have a simple physics simulation and would like to record sessions so that they can be replayed. If the calculations can be relied on then I should only need to record the initial state plus any user input and I should always be able to reproduce the final state exactly. If the calculations are not accurate errors at the start may have huge implications by the end of the simulation.

I am currently working in Silverlight though would be interested to know if this question can be answered in general.

Update: The initial answers indicate yes, but apparently this isn't entirely clear cut as discussed in the comments for the selected answer. It looks like I will have to do some tests and see what happens.

Edit Report

3 Answers

The short answer is that FP calculations are entirely deterministic, as per the IEEE Floating Point Standard, but that doesn't mean they're entirely reproducible across machines, compilers, OS's, etc.

The long answer to these questions and more can be found in what is probably the best reference on floating point, David Goldberg's What Every Computer Scientist Should Know About Floating Point Arithmetic. Skip to the section on the IEEE standard for the key details.

To answer your bullet points briefly:

Time between calculations and state of the CPU have little to do with this.
Hardware can affect things (e.g. some GPUs are not IEEE floating point compliant).
Language, platform, and OS can also affect things. For a better description of this than I can offer, see Jason Watkins's answer. If you are using Java, take a look at Kahan's rant on Java's floating point inadequacies.
Solar flares might matter, hopefully infrequently. I wouldn't worry too much, because if they do matter, then everything else is screwed up too. I would put this in the same category as worrying about EMP.

Finally, if you are doing the same sequence of floating point calculations on the same initial inputs, then things should be replayable exactly just fine. The exact sequence can change depending on your compiler/os/standard library, so you might get some small errors this way.

Where you usually run into problems in floating point is if you have a numerically unstable method and you start with FP inputs that are approximately

answered 2008-11-30T08:33:04.713

1

Sorry, but I can't help thinking that everybody is missing the point.

If the inaccuracy is significant to what you are doing then you should look for a different algorithm.

You say that if the calculations are not accurate, errors at the start may have huge implications by the end of the simulation.

That my friend is not a simulation. If you are getting hugely different results due to tiny differences due to rounding and precision then the chances are that none of the results has any validity. Just because you can repeat the result does not make it any more valid.

On any non-trivial real world problem that includes measurements or non-integer calculation, it is always a good idea to introduce minor errors to test how stable your algorithm is.

answered 2008-11-30T09:12:07.973

-4

Very few FPUs meet the IEEE standard (despite their claims). So running the same program in different hardware will indeed give you different results. The results are likely to be in corner cases that you should already avoid as part of using an FPU in your software.

IEEE bugs are often patched in software and are you sure that the operating system you are running today includes the proper traps and patches from the manufacturer? What about before or after the OS has an update? Are all bugs removed and bug fixes added? Is the C compiler in sync with all of this and is the C compiler producing the proper code?

Testing this may prove futile. You wont see the problem until you deliver the product.

Observe FP rule number 1: Never use an if(something==something) comparison. And rule number two IMO would have to do with ascii to fp or fp to ascii (printf, scanf, etc). There are more accuracy and bug problems there than in the hardware.

With each new generation of hardware (density) the affects from the sun are more apparent. We already have problems with SEU's on the planets surface, so independent of floating point calculations, you will have problems (few vendors have bothered to care, so expect crashes more often with new hardware).

By consuming enormous amounts of logic the fpu is likely to be a very fast (a single clock cycle). Not any slower than an integer alu. Do not confuse this with modern fpus being as simple as alus, fpus are expensive. (alus likewise consume more logic for multiply and divide to get that down to one clock cycle but its not nearly as big as the fpu).

Keep to the simple rules above, study floating point a bit more, understand the warts and traps that go with it. You may want to check for infinity or nans periodically. Your problems are more likely to be found in the compiler and operating system than the hardware (in general not just fp math). Modern hardware (and software) is,

answered 2008-11-30T15:59:46.497

Your Answer