C#: Is this benchmarking class accurate?

I created a simple class to benchmark some methods of mine. But is it accurate? I am kind of new to benchmarking, timing, et cetera, so thought I could ask for some feedback here. Also, if it is good, maybe somebody else can make use of it as well :)

public static class Benchmark
{
    public static IEnumerable<long> This(Action subject)
    {
        var watch = new Stopwatch();
        while (true)
        {
            watch.Reset();
            watch.Start();
            subject();
            watch.Stop();
            yield return watch.ElapsedTicks;
        }
    }
}

You can use it like this:

var avg = Benchmark.This(() => SomeMethod()).Take(500).Average();

Any feedback? Does it look to be pretty stable and accurate, or have I missed something?

c# benchmarking feedback

Edit Report

It is about as accurate as you can get for a simple benchmark. But there are some factors not under your control:

load on the system from other processes
state of the heap before/during the benchmark

You could do something about that last point, a benchmark is one of the rare situations where calling GC.Collect can be defended. And you might call subject once beforehand to eliminate any JIT issues. But that requires calls to subject to be independent.

public static IEnumerable<TimeSpan> This(Action subject)
{
    subject();     // warm up
    GC.Collect();  // compact Heap
    GC.WaitForPendingFinalizers(); // and wait for the finalizer queue to empty

    var watch = new Stopwatch();
    while (true)
    {
        watch.Reset();
        watch.Start();
        subject();
        watch.Stop();
        yield return watch.Elapsed;  // TimeSpan
    }
}

For bonus, your class should check the System.Diagnostics.Stopwatch.IsHighResolution field. If it is off, you only have a very coarse (20 ms) resolution.

But on an ordinary PC, with many services running in the background, it is never going to be very accurate.

answered 2009-10-02T09:35:37.570

C#: Is this benchmarking class accurate?

1 Answer

Your Answer