How can I make reverse scanning of a binary file faster?

I have a binary file specification that describes a packetized data structure. Each data packet has a two-byte sync pattern, so scanning for the beginning of a packet is possible, using a BinaryReader and FileStream combination:

while(!reader.EndOfFile)
{
    // Check for sync pattern.
    if (reader.ReadUInt16() != 0xEB25)
    {
        // Move to next byte.
        reader.BaseStream.Seek(-1, SeekOrigin.Current);
        continue;
    }

    // If we got here, a sync pattern was found.
}

This process works perfectly fine in the forward direction, but similar code scanning in the reverse direction is at least two orders of magnitude slower:

while(!reader.BeginningOfFile)
{
    // Check for sync pattern.
    if (reader.ReadUInt16() != 0xEB25)
    {
        // Move to previous byte.
        reader.BaseStream.Seek(-3, SeekOrigin.Current);
        continue;
    }

    // If we got here, a sync pattern was found.
}

I've tried a few workarounds, like moving back by an arbitrary amount (currently 1 megabyte) and scanning forward, but it's becoming clear that what I really need is a BinaryReader or FileStream that is modified to have adequate performance characteristics when reading in both forward and reverse directions.

I already have a FastFileStream which improves forward read performance by subclassing an ordinary FileStream and caching the Position and Length properties (it also provides the BeginningOfFile and EndOfFile properties). That's what drives the reader variable in the code above.

Is there something similar I could do to improve reverse reading performance, perhaps by incorporating a MemoryStream as a buffer?

Edit Report

1 Answer

EDIT: Okay, I've got some code. Well, quite a lot of code. It allows you to scan forwards and backwards for packet headers.

I make no guarantee that it has no bugs, and you definitely want to tweak the buffer size to see how it performs... but given the same file you sent me, it at least shows the same packet header locations when scanning forwards and backwards :)

Before the code, I would still suggest that if you possibly can, scanning through the file once and saving an index of packet information for later use would probably be a better approach.

Anyway, here's the code (complete with no tests other than the sample program):

PacketHeader.cs:

using System;

namespace Chapter10Reader
{
    public sealed class PacketHeader
    {
        private readonly long filePosition;
        private readonly ushort channelId;
        private readonly uint packetLength;
        private readonly uint dataLength;
        private readonly byte dataTypeVersion;
        private readonly byte sequenceNumber;
        private readonly byte packetFlags;
        private readonly byte dataType;
        private readonly ulong relativeTimeCounter;

        public long FilePosition { get { return filePosition; } }
        public ushort ChannelId { get { return channelId; } }
        public uint PacketLength { get { return packetLength; } }
        public uint DataLength { get { return dataLength; } }
        public byte DataTypeVersion { get { return dataTypeVersion; } }
        public byte SequenceNumber { get { return sequenceNumber; } }
        public byte PacketFlags { get { return packetFlags; } }
        public byte DataType { get { return dataType; } }
        public ulong RelativeTimeCounter { get { return relativeTimeCounter; } }

        public PacketHeader(ushort channelId, uint packetLength, uint dataLength, byte dataTypeVersion,
            byte sequenceNumber, byte packetFlags, byte dataType, ulong relativeTim


                        
                        
                            answered 2012-03-08T17:48:29.780

How can I make reverse scanning of a binary file faster?

1 Answer

Your Answer