I'm debugging an application that is running quite a bit slower when built as a 64-bit Linux ELF executable than as a 32-bit Linux ELF executable. Using Rational (IBM) Quantify, I tracked much of the performance difference down to (drum roll...) memset. Oddly, memset is taking a lot longer in the 64-bit executable.

I am even able to see this with a small, simple application:

#include <stdlib.h>
#include <string.h>

#define BUFFER_LENGTH 8000000

int main()
{
  unsigned char* buffer = malloc(BUFFER_LENGTH * sizeof(unsigned char));
  for(int i = 0; i < 10000; i++)
    memset(buffer, 0, BUFFER_LENGTH * sizeof(unsigned char));
}

I build like this:
$ gcc -m32 -std=gnu99 -g -O3 ms.c
and
$ gcc -m64 -std=gnu99 -g -O3 ms.c

The wall-clock time as reported by time is longer for the -m64 build and Quantify confirms that the extra time is being spent in memset.

So far I've tested in VirtualBox and VMWare (but not bare-metal Linux; I realize I need to do that next). The amount of extra time spent seems to vary a bit from one system to the next.

What's going on here? Is there a well-known issue that my Google-foo is not able to uncover?

EDIT: The disassembly (gcc ... -S) on my system shows that memset is being invoked as an external function:

32-bit:

.LBB2:
    .loc 1 14 0
    movl    $8000000, 8(%esp)
    .loc 1 12 0
    addl    $1, %ebx
    .loc 1 14 0
    movl    $0, 4(%esp)
    movl    %esi, (%esp)
    call    memset

64-bit:

.LBB2:
    .loc 1 14 0
    xorl    %esi, %esi
    movl    $8000000, %edx
    movq    %rbp, %rdi
.LVL1:
    .loc 1 12 0
    addl    $1, %ebx
    .loc 1 14 0
    call    memset

System:

  • CentOS 5.7 2.6.18-274.17.1
c linux performance 32bit-64bit
Edit
Report