How to use `while read` (Bash) to read the last line in a file if there’s no newline at the end of the file?

Question

Let’s say I have the following Bash script:

while read SCRIPT_SOURCE_LINE; do
  echo "$SCRIPT_SOURCE_LINE"
done

I noticed that for files without a newline at the end, this will effectively skip the last line.

I’ve searched around for a solution and found this:

When read reaches end-of-file instead of end-of-line, it does read in the data and assign it to the variables, but it exits with a non-zero status. If your loop is constructed "while read ;do stuff ;done

So instead of testing the read exit status directly, test a flag, and have the read command set that flag from within the loop body. That way regardless of reads exit status, the entire loop body runs, because read was just one of the list of commands in the loop like any other, not a deciding factor of if the loop will get run at all.

DONE=false
until $DONE ;do
read || DONE=true
# process $REPLY here
done < /path/to/file.in

How can I rewrite this solution to make it behave exactly the same as the while loop I was having earlier, i.e. without hardcoding the location of the input file?

Answer 1

In your first example, I'm assuming you are reading from stdin. To do the same with the second code block, you just have to remove the redirection and echo $REPLY:

DONE=false
until $DONE ;do
read || DONE=true
echo $REPLY
done

Answer 2

I use the following construct:

while IFS= read -r LINE || [[ -n "$LINE" ]]; do
    echo "$LINE"
done

It works with pretty much anything except null characters in the input:

• Files that start or end with blank lines
• Lines that start or end with whitespace
• Files that don't have a terminating newline

Answer 3

Using grep with while loop:

while IFS= read -r line; do
  echo "$line"
done < <(grep "" file)

Using grep . instead of grep "" will skip the empty lines.

Note:

1. Using IFS= keeps any line indentation intact.

2. You should almost always use the -r option with read.

3. File without a newline at the end isn't a standard unix text file.

Answer 4

Instead of read, try to use GNU Coreutils like tee, cat, etc.

from stdin

readvalue=$(tee)
echo $readvalue

from file

readvalue=$(cat filename)
echo $readvalue

Answer 5

The basic issue here is that read will return errorlevel 1 when it encounters EOF, even if it'll still correctly feed the variable.

So you can use errorlevel of read right away in your loop, otherwize, the last data won't be parsed. But you could do this:

eof=
while [ -z "$eof" ]; do
    read SCRIPT_SOURCE_LINE || eof=true   ## detect eof, but have a last round
    echo "$SCRIPT_SOURCE_LINE"
done

If you want a very solid way to parse your lines, you should use:

IFS='' read -r LINE

Remember that:

• NUL character will be ignored
• if you stick to using echo to mimick the behavior of cat you'll need to force an echo -n upon EOF detected (you can use the condition [ "$eof" == true ])

Answer 6

This is the pattern I've been using:

while read -r; do
  echo "${REPLY}"
done
[[ ${REPLY} ]] && echo "${REPLY}"

Which works because even tho' the while loop ends as the "test" from the read exits with a non-zero code, read still populates the inbuilt variable $REPLY (or whatever variables you choose to assign with read).

Answer 7

@Netcoder's answer is good, this optimisation eliminates spurious blank lines, also allows for the last line not to have a newline, if that's how the original was.

DONE=false
NL=
until $DONE ;do
if ! read ; then DONE=true ; NL='-n ';fi
echo $NL$REPLY
done

I used a variant of this to create 2 functions to allow piping of text that includes a '[' to keep grep happy. (you can add other translations)

function grepfix(){
    local x="$@";
    if [[ "$x" == '-' ]]; then
      local DONE=false
      local xx=
      until $DONE ;do
         if ! IFS= read ; then DONE=true ; xx="-n "; fi
         echo ${xx}${REPLY//\[/\\\[}
      done
    else
      echo "${x//\[/\\\[}"
    fi
 }


 function grepunfix(){
    local x="$@";
    if [[ "$x" == '-' ]]; then
      local DONE=false
      local xx=
      until $DONE ;do
         if ! IFS= read ; then DONE=true ; xx="-n "; fi
         echo ${xx}${REPLY//\\\[/\[}
      done
    else
      echo "${x//\\\[/\[}"
    fi
 }

(passing - as $1 enables pipe otherwise just translates arguments)