Alex Rivera | Logout

Is there a better way to benchmark a C program than timing?

Asked 2011-09-17T16:17:39.697
13

I'm coding a little program that has to sort a large array (up to 4 million text strings). Seems like I'm doing quite well at it, since a combination of radixsort and mergesort already cut the original q(uick)sort execution time in less than half.

Execution time being the main point, since this is what I'm using to benchmark my piece of code.

My question is:

Is there a better (i. e. more reliable) way of benchmarking a program than just time the execution? It kinda works, but the same program (with the same background processes running) usually has slightly different execution times if run twice.

This kinda defeats the purpose of detecting small improvements. And several small improvements could add up to a big one...

Thanks in advance for any input!

Results:

I managed to get gprof to work under Windows (using gcc and MinGW). gcc behaves poorly (considering execution time) compared to my normal compiler (tcc), but it gave me quite some insight.

c sorting benchmarking
Edit
Report

1 Answer

1

Call your routine from a test harness, whereby it executes N + 1 times. Ignore the timing for the first iteration and then take the average of iterations 1..N. The reason for ignoring the first time is that is is often slightly inflated due to various effects, e.g. virtual memory, code being paged in, etc. The reason for averaging N iterations is that you get rid of artefacts caused by other processes, the scheduler, etc.

If you're running on Linux or similar You might also want to use taskset to pin your code to a specific CPU core (assuming it's single-threaded), ideally not core 0, since this tends to handle all interrupts.

answered 2011-09-17T16:26:13.617

Your Answer