remove block of text between two lines based on content

Question

I need to remove/filter a very large log file i managed to bring the log-file into blocks of text starting with a line containing <-- or -->

Answer 1

If I get what you need correctly, you want to filter out the block, that is this only print the block:

tail -f logfile | sed -n '/\(<--\|-->\)/,/Content-Length:/ p' 

If you want to delete it:

tail -f logfile | sed '/\(<--\|-->\)/,/Content-Length:/ d'

Answer 2

Try this:

awk '/<--|-->/{rec=""; f=1} f{rec = rec $0 ORS} /Content-Length:/{ if (f && (rec !~ "REGISTER")) printf "%s",rec; f=0}' file

If it doesn't do what you want, provide more info on what you want along with sample input and output.

To break down the above, here's each statement on separate lines with some comments:

awk '
   /<--|-->/ {rec=""; f=1} # find the start of the record, reset the string to hold it and set a flag to indicate we've started processing a record
   f {rec = rec $0 ORS}    # append to the end of the string containing the current record
   /Content-Length:/{      # find the end of the record
      if (f && (rec !~ "REGISTER")) # print the record if it doesn't contain "REGISTER"
         printf "%s",rec
      f=0                  # clear the "found record" indicator
   }
' file

and if you have text between your records that you'd want printed, just add a test for the "found" flag not being set and invoke the default action of printing the current record (!f;)

awk '/<--|-->/{rec=""; f=1} f{rec = rec $0 ORS} !f; /Content-Length:/{ if (f && (rec !~ "REGISTER")) printf "%s",rec; f=0}' file

Answer 3

This might work for you (GNU sed);

sed '/<--\|-->/!b;:a;/Content-Length/!{$!{N;ba}};//{/REGISTER/d}' file

• /<--\|-->/!b if a line does not contain <-- or --> print it
• :a;/Content-Length/!{$!{N;ba}} keep appending lines until the string Content-Length or the end of file is encountered.
• //{/REGISTER/d} if the line(s) read in contains Content-Length and REGISTER delete it/them else print it/them as normal.