Anyone know a good and effective way to search/match for a byte pattern in an byte[] array and then return the positions.
For example
byte[] pattern = new byte[] {12,3,5,76,8,0,6,125};
byte[] toBeSearched = new byte[] {23,36,43,76,125,56,34,234,12,3,5,76,8,0,6,125,234,56,211,122,22,4,7,89,76,64,12,3,5,76,8,0,6,125}
Use the efficient Boyer-Moore algorithm.
It's designed to find strings withing strings but you need little imagination to project this to byte arrays.
In general the best answer is: use any string searching algorithm that you like :).
I'm a little late to the party How about using Boyer Moore algorithm but search for bytes instead of strings. c# code below.
EyeCode Inc.
class Program {
static void Main(string[] args) {
byte[] text = new byte[] {12,3,5,76,8,0,6,125,23,36,43,76,125,56,34,234,12,4,5,76,8,0,6,125,234,56,211,122,22,4,7,89,76,64,12,3,5,76,8,0,6,123};
byte[] pattern = new byte[] {12,3,5,76,8,0,6,125};
BoyerMoore tmpSearch = new BoyerMoore(pattern,text);
Console.WriteLine(tmpSearch.Match());
Console.ReadKey();
}
public class BoyerMoore {
private static int ALPHABET_SIZE = 256;
private byte[] text;
private byte[] pattern;
private int[] last;
private int[] match;
private int[] suffix;
public BoyerMoore(byte[] pattern, byte[] text) {
this.text = text;
this.pattern = pattern;
last = new int[ALPHABET_SIZE];
match = new int[pattern.Length];
suffix = new int[pattern.Length];
}
/**
* Searches the pattern in the text.
* returns the position of the first occurrence, if found and -1 otherwise.
*/
public int Match() {
// Preprocessing
ComputeLast();
ComputeMatch();
// Searching
int i = pattern.Length - 1;
int j = pattern.Length - 1;
while (i < text.Length) {
if (pattern[j] == text[i]) {
if (j == 0) {
return i;
}
j--;
i--;
}
else {
i += pattern.Length - j - 1 + Math.Max(j - last[text[i]], match[j]);
j = pattern.Length - 1;
}
}
return -1;
}
/**
* Computes the function last and stores its valueJb Evain's answer has:
for (int i = 0; i < self.Length; i++) {
if (!IsMatch (self, i, candidate))
continue;
list.Add (i);
}
and then the IsMatch function first checks whether candidate goes beyond the length of the array being searched.
This would be more efficient if the for loop were coded:
for (int i = 0, n = self.Length - candidate.Length + 1; i < n; ++i) {
if (!IsMatch (self, i, candidate))
continue;
list.Add (i);
}
at this point one could also eliminate the test from the start of IsMatch, so long as you contract via pre-conditions never to call it with "illegal' parameters.
Note: Fixed an off-by-one bug in 2019.
I've created a new function using the tips from my answer and the answer by Alnitak.
public static List<Int32> LocateSubset(Byte[] superSet, Byte[] subSet)
{
if ((superSet == null) || (subSet == null))
{
throw new ArgumentNullException();
}
if ((superSet.Length < subSet.Length) || (superSet.Length == 0) || (subSet.Length == 0))
{
return new List<Int32>();
}
var result = new List<Int32>();
Int32 currentIndex = 0;
Int32 maxIndex = superSet.Length - subSet.Length;
while (currentIndex < maxIndex)
{
Int32 matchCount = CountMatches(superSet, currentIndex, subSet);
if (matchCount == subSet.Length)
{
result.Add(currentIndex);
}
currentIndex++;
if (matchCount > 0)
{
currentIndex += matchCount - 1;
}
}
return result;
}
private static Int32 CountMatches(Byte[] superSet, int startIndex, Byte[] subSet)
{
Int32 currentOffset = 0;
while (currentOffset < subSet.Length)
{
if (superSet[startIndex + currentOffset] != subSet[currentOffset])
{
break;
}
currentOffset++;
}
return currentOffset;
}
the only part I'm not so happy about is the
currentIndex++;
if (matchCount > 0)
{
currentIndex += matchCount - 1;
}
part... I would have like to use a if else to avoid the -1, but this results in a better branch prediction (although I'm not that sure it will matter that much)..
I tried various solutions and ended up modifying the SearchBytePattern one. I tested on a 30k sequence and it is, fast :)
static public int SearchBytePattern(byte[] pattern, byte[] bytes)
{
int matches = 0;
for (int i = 0; i < bytes.Length; i++)
{
if (pattern[0] == bytes[i] && bytes.Length - i >= pattern.Length)
{
bool ismatch = true;
for (int j = 1; j < pattern.Length && ismatch == true; j++)
{
if (bytes[i + j] != pattern[j])
ismatch = false;
}
if (ismatch)
{
matches++;
i += pattern.Length - 1;
}
}
}
return matches;
}
Let me know your thoughts.
toBeSearched.Except(pattern) will return you differences toBeSearched.Intersect(pattern) will produce set of intersections Generally, you should look into extended methods within Linq extensions