Buffering problem when piping output between CLI programs

二次信任 提交于 2019-12-11 07:36:26

问题


I'm trying to tail apache error logs through a few filters.

This works perfectly:

tail -fn0 /var/log/apache2/error.log | egrep -v "PHP Notice|File does not exist"

but there are some literal "\n" in the output which I want to replace with an actual new line so I pipe into perl:

tail -fn0 /var/log/apache2/error.log | egrep -v "PHP Notice|File does not exist" | perl -ne 's/\\n/\n/g; print"$_"'

This seems to have some caching issue (first page hit produces nothing, second page hit and two loads of debugging info comes out), It also seems a bit tempramental.

So I tried sed:

tail -fn0 /var/log/apache2/error.log | egrep -v "PHP Notice|File does not exist" | sed 's/\\n/\n/g'

which seems to suffer the same problem.


回答1:


Correct, when you use most programs to file or pipe they buffer output. You can control this in some cases: the GNU grep family accepts the --line-buffered option, specifically for use in pipelines like this. Also, in Perl you can use $| = 1; for the same effect. (sed doesn't have any such option that I'm aware of.)

It's the stuff at the beginning or middle of the pipeline that will be buffering, not the end (which is talking to your terminal so it will be line buffered) so you want to use egrep --line-buffered.




回答2:


Looks like you can use -u for sed as in:

tail -f myLog | sed -u "s/\(joelog\)/^[[46;1m\1^[[0m/g" | sed -u 's/\\n/\n/g'

which tails the log, highlights 'joelog', and then adds linebreaks where there are '\n'

source: http://www-01.ibm.com/support/docview.wss?uid=isg1IZ42070



来源:https://stackoverflow.com/questions/5503084/buffering-problem-when-piping-output-between-cli-programs

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!