Alex Rivera | Logout

Fastest way to see how many bytes are equal between fixed length arrays

Asked 2008-09-22T18:09:23.010
12

I have 2 arrays of 16 elements (chars) that I need to "compare" and see how many elements are equal between the two.

This routine is going to be used millions of times (a usual run is about 60 or 70 million times), so I need it to be as fast as possible. I'm working on C++ (C++Builder 2007, for the record)

Right now, I have a simple:

matches += array1[0] == array2[0];

repeated 16 times (as profiling it appears to be 30% faster than doing it with a for loop)

Is there any other way that could work faster?

Some data about the environment and the data itself:

  • I'm using C++Builder, which doesn't have any speed optimizations to take into account. I will try eventually with another compiler, but right now I'm stuck with this one.
  • The data will be different most of the times. 100% equal data is usually very very rare (maybe less than 1%)
Edit
Report

3 Answers

17

UPDATE: This answer has been modified to make my comments match the source code provided below.

There is an optimization available if you have the capability to use SSE2 and popcnt instructions.

16 bytes happens to fit nicely in an SSE register. Using c++ and assembly/intrinsics, load the two 16 byte arrays into xmm registers, and cmp them. This generates a bitmask representing the true/false condition of the compare. You then use a movmsk instruction to load a bit representation of the bitmask into an x86 register; this then becomes a bit field where you can count all the 1's to determine how many true values you had. A hardware popcnt instruction can be a fast way to count all the 1's in a register.

This requires knowledge of assembly/intrinsics and SSE in particular. You should be able to find web resources for both.

If you run this code on a machine that does not support either SSE2 or popcnt, you must then iterate through the arrays and count the differences with your unrolled loop approach.

Good luck

Edit: Since you indicated you did not know assembly, here's some sample code to illustrate my answer:

#include "stdafx.h"
#include <iostream>
#include "intrin.h"

inline unsigned cmpArray16( char (&arr1)[16], char (&arr2)[16] )
{
    __m128i first = _mm_loadu_si128( reinterpret_cast<__m128i*>( &arr1 ) );
    __m128i second = _mm_loadu_si128( reinterpret_cast<__m128i*>( &arr2 ) );

    return _mm_movemask_epi8( _mm_cmpeq_epi8( first, second ) );
}

int _tmain( int argc, _TCHAR* argv[] )
{
    unsigned count = 0;
    char    arr1[16] = { 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0 };
    char    arr2[16] = { 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0 };

    count = __popcnt( cmpArray16( arr1, arr2 ) );

    std::cout << "The number of equivalent bytes = " << count << std::endl;

    return 0;
}

Some notes: This funct

answered 2008-09-22T18:30:55.593
0

Is it faster as one statement?

matches += (array1[0] == array2[0]) + (array1[1] == array2[1]) + ...;
answered 2008-09-22T18:12:05.563
0

There's always the good old x86 REPNE CMPS instruction.

answered 2008-09-22T18:26:40.877

Your Answer