C# Dictionary and Efficient Memory Usage

I have a tool to compare 2 csv files and then bucket each cell into one of the 6 buckets. Basically, it reads in the csv files (using fast csv reader, credit: http://www.codeproject.com/KB/database/CsvReader.aspx) and then creates a dictionary pertaining to each file based on the keys provided by the user. I then iterate through th dictionaries comparing the values and writing a result csv file.

While it is blazing fast, it is very inefficient in terms of memory usage. I cannot compare more than 150 MB files on my box with 3 GB physical memory.

Here is a code snippet to read the expected file. At the end of this piece, the memory usage is close to 500 MB from the task manager.

// Read Expected
long rowNumExp;
System.IO.StreamReader readerStreamExp = new System.IO.StreamReader(@expFile);
SortedDictionary<string, string[]> dictExp = new SortedDictionary<string, string[]>();
List<string[]> listDupExp = new List<string[]>();
using (CsvReader readerCSVExp = new CsvReader(readerStreamExp, hasHeaders, 4096))
{
    readerCSVExp.SkipEmptyLines = false;
    readerCSVExp.DefaultParseErrorAction = ParseErrorAction.ThrowException;
    readerCSVExp.MissingFieldAction = MissingFieldAction.ParseError;
    fieldCountExp = readerCSVExp.FieldCount;                
    string keyExp;
    string[] rowExp = null;
    while (readerCSVExp.ReadNextRecord())
    {
        if (hasHeaders == true)
        {
            rowNumExp = readerCSVExp.CurrentRecordIndex + 2;
        }
        else
        {
            rowNumExp = readerCSVExp.CurrentRecordIndex + 1;
        }
        try
        {
            rowExp = new string[fieldCount + 1];                    
        }
        catch (Exception exExpOutOfMemory)
        {
            MessageBox.Show(exExpOutOfMemory.Message);
            Environment.Exit(1);
        }                
        key