I am currently writing a program in C++ which includes reading lots of large text files. Each has around 400.000 lines with in extreme cases 4000 or more characters per line. Just for testing, I read one of the files using ifstream and an implementation I found on cplusplus.com. On my machine, it took around 60 seconds, which is quite long considering my workflow. Is there a straightforward way to improve reading speed?

The code I am using is more or less this:

string tmpString;
ifstream txtFile(path);
if(txtFile.is_open())
{
    while(txtFile.good())
    {
        m_numLines++;
        getline(txtFile, tmpString);
    }
    txtFile.close();
}

Notes:

  • The file I read is only ~80 MB big. I mentioned the maximum charcount of 4000 per line because I thought it might be necessary to know in order to do buffering
  • I have to use readline, since I need to count the number of lines
  • Instantiating the ifstream as binary didn't make reading any faster
  • Parallelizing the task will speed it up, but isn't straight-forward and has a good chance of introducing bugs

After lots of back and forth discussion, a big thank you to sehe for putting so much time into this, I appreciate it a lot!

c++ performance io ifstream
Edit
Report