I have an ... odd issue with a bash shell script that I was hoping to get some insight on.
My team is working on a script that iterates through lines in a file and checks for content in each one. We had a bug where, when run via the automated process that sequences different scripts together, the last line wasn't being seen.
The code used to iterate over the lines in the file (name stored in DATAFILE was
cat "$DATAFILE" | while read line
We could run the script from the command line and it would see every line in the file, including the last one, just fine. However, when run by the automated process (which runs the script that generates the DATAFILE just prior to the script in question), the last line is never seen.
We updated the code to use the following to iterate over the lines, and the problem cleared up:
for line in `cat "$DATAFILE"`
Note: DATAFILE has no newline ever written at the end of the file.
My question is two part... Why would the last line not be seen by the original code, and why this would change make a difference?
I only thought I could come up with as to why the last line would not be seen was:
- The previous process, which writes the file, was relying on the process to end to close the file descriptor.
- The problem script was starting up and opening the file prior fast enough that, while the previous process had "ended", it hadn't "shut down/cleaned up" enough for the system to close the file descriptor automatically for it.
That being said, it seems like, if you have 2 commands in a shell script, the first one should be completely shut down by the time the script runs the second one.
Any insight into the questions, especially the first one, would be very much appreciated.