问题
Here is my sample log file.http://pastebin.com/DwWeFhJk.
When I am doing
tail -f log | awk '{if (NF>3) {print $1}; }'
the result I am getting is correct
64.242.88.10
64.242.88.10
64.242.88.10
64.242.88.10
64.242.88.10
64.242.88.10
64.242.88.10
64.242.88.10
64.242.88.10
64.242.88.10
But when I am doing:
tail -f log |
awk '{if (NF>3) {print $1}; }' |
awk '{print $1}'
I am not getting any output. Even no output in case of
tail -f log | awk '{if (NF>3) {print $1}; }' | grep "64"
I am not getting the reason why the output of the first awk is not getting passed as the input of the second awk/grep after the pipe.
回答1:
When the output of the first awk is going to the terminal, the output is line-buffered, so each line is printed as it is produced. When the output is going to the second awk or the grep, it is fully buffered. The output won't be sent until the buffer is full. When enough extra records are appended to the log, then the second awk will a buffer full of data to process. Until then, nothing will happen.
回答2:
You start the command with tail -f, that keeps the output open and therefore does not send a needed newline to the other commands.
This works perfectly fine:
cat log | awk '{if (NF>3) {print $1}; }' | grep 64
So, the problem is buffering. The middle awk is doing normal buffering instead of interactive buffering. This works (non-portably) with mawk:
tail -f log | mawk -W interactive '{if (NF>3) {print $1}; }' | awk '{print}'
You could read GNU description of the issue.
In any case, just check that the awk used in the middle could be told to buffer interactively.
Added:
The command system("") seems to unblock the buffering. It is POSIX, but does not work with mawk.
tail -f log | awk '{if (NF>3) {print $1}; system("")}' | awk '{print}'
回答3:
search for "parallel --pipe" in the link to avoid the buffering
https://www.gnu.org/software/parallel/parallel_tutorial.html
来源：https://stackoverflow.com/questions/33448231/awk-pipe-and-tail-f-giving-unexpected-behavior