how can I count number of lines after a string match until next especific match occurs

问题

I have a file with the following structure (see below), I need help to find the way to match every ">Cluster" string, and for every case count the number of lines until the next ">cluster" and so on until the end of the file.

>Cluster 0
0       10565nt, >CL9602.Contig1_All... *
1       1331nt, >CL9602.Contig2_All... at -/98.05%
>Cluster 1
0       3798nt, >CL3196.Contig1_All... at +/97.63%
1       9084nt, >CL3196.Contig3_All... *
>Cluster 2
0       8710nt, >Unigene21841_All... *
>Cluster 3
0       8457nt, >Unigene10299_All... *

The desired Output should look like below:

Cluster 0  2 
Cluster 1  2
Cluster 2  1
Cluster 3  1

I tried with awk as below, but it gives me only the line numbers.

awk '{print FNR "\t" $0}' All-Unigene_Clustered.fa.clstr | head - 20
==> standard input <==
1       >Cluster 0
2       0       10565nt, >CL9602.Contig1_All... *
3       1       1331nt, >CL9602.Contig2_All... at -/98.05%
4       >Cluster 1
5       0       3798nt, >CL3196.Contig1_All... at +/97.63%
6       1       9084nt, >CL3196.Contig3_All... *
7       >Cluster 2
8       0       8710nt, >Unigene21841_All... *
9       >Cluster 3
10      0       8457nt, >Unigene10299_All... *

I also tried with sed, but it only prints the lines while even ommiting some lines.

sed -n -e '/>Cluster/,/>Cluster/ p' All-Unigene_Clustered.fa.clstr | head             
>Cluster 0
0       10565nt, >CL9602.Contig1_All... *
1       1331nt, >CL9602.Contig2_All... at -/98.05%
>Cluster 1
>Cluster 2
0       8710nt, >Unigene21841_All... *
>Cluster 3
>Cluster 4
0       1518nt, >CL2313.Contig1_All... at -/95.13%
1       8323nt, >CL2313.Contig8_All... *

In addition I tried awk and sed in combination with 'wc' but it gives me only the total count of occurrencies for the string match.

I thought subtracting the lines not matching the string '>cluster' using the -v option of grep, then substracting every line matching the string '>Cluster' and adding both to a new file, e.g

grep -vw '>Cluster' All-Unigene_Clustered.fa.clstr | head
0       10565nt, >CL9602.Contig1_All... *
1       1331nt, >CL9602.Contig2_All... at -/98.05%
0       3798nt, >CL3196.Contig1_All... at +/97.63%
1       9084nt, >CL3196.Contig3_All... *
0       8710nt, >Unigene21841_All... *
0       8457nt, >Unigene10299_All... *
0       1518nt, >CL2313.Contig1_All... at -/95.13%

grep -w '>Cluster' All-Unigene_Clustered.fa.clstr | head
>Cluster 0
>Cluster 1
>Cluster 2
>Cluster 3
>Cluster 4

but the problem is that the number of lines following each '>Cluster' isn't constant, each '>Cluster' string is followed by 1, 2, 3 or more lines until the next string occurs.

I have decided to post my question after extensively searching for help within previously ansewred questions but I could't find any helpful answer.

Thanks

回答1:

Could you please try following.

awk '
/^>Cluster/{
  if(count){
    print prev,count
  }
  sub(/^>/,"")
  prev=$0
  count=""
  next
}
{
  count++
}
END{
  if(count && prev){
    print prev,count
  }
}
' Input_file

Explanation: Adding explanation for above code.

awk '                      ##Starting awk program from here.
/^>Cluster/{               ##Checking condition if a line is having string Cluster then do following.
  if(count){               ##Checking condition if variable count is NOT NULL then do following.
    print prev,count       ##Printing prev and count variable here.
  }                        ##Closing BLOCK for if condition here.
  sub(/^>/,"")             ##Using sub for substitution of starting > with NULL in current line.
  prev=$0                  ##Creating a variable named prev whose value is current line.
  count=""                 ##Nullifying count variable here.
  next                     ##next will skip all further statements from here.
}                          ##Closing BLOCK for Cluster condition here.
{
  count++                  ##Doing increment of variable count each time cursor comes here.
}
END{                       ##Mentioning END BLOCK for this program.
  if(count && prev){       ##Checking condition if variable count and prev are NOT NULL then do following.
    print prev,count       ##Printing prev and count variable here.
  }                        ##Closing BLOCK for if condition here.
}                          ##Closing BLOCK for END BLOCK of this program.
' Input_file               ##Mentioning Input_file name here.

Output will be as follows.

Cluster 0 2
Cluster 1 2
Cluster 2 1
Cluster 3 1

回答2:

With GNU awk for multi-char RS:

$ awk -v RS='(^|\n)(>|$)' -F'\n' 'NR>1{print $1, NF-1}' file
Cluster 0 2
Cluster 1 2
Cluster 2 1
Cluster 3 1

The above just separates the input into records that start with > at the start of a line and then prints the number of lines in each record (subtracting 1 for the >Cluster... line).

回答3:

Here's a, allbeit quite verbose one liner in Perl. I'm really not good at this golfing stuff.

perl -n -e "if ( /^>(.+)/ ) { print qq($last, $count\n) if $count; $last = $1; $count = 0; } else { $count++ } END { print qq($last, $count) }" All-Unigene_Clustered.fa.clstr

This is for Windows. For a unix shell you probably need to change the double to single quotes.

回答4:

In perl the code can be in following form

use strict;
use warnings;

my $cluster;
my $count;

while( <DATA> ) {
    chomp;
    if( /Cluster \d+/ ) {
        print "$cluster $count\n" if defined $cluster;
        s/>//;
        $cluster = $_;
        $count   = 0;
    } else {
        $count++;
    }
}

print "$cluster $count\n" if defined $store;

__DATA__
>Cluster 0
0       10565nt, >CL9602.Contig1_All... *
1       1331nt, >CL9602.Contig2_All... at -/98.05%
>Cluster 1
0       3798nt, >CL3196.Contig1_All... at +/97.63%
1       9084nt, >CL3196.Contig3_All... *
>Cluster 2
0       8710nt, >Unigene21841_All... *
>Cluster 3
0       8457nt, >Unigene10299_All... *

output

Cluster 0 2
Cluster 1 2
Cluster 2 1
Cluster 3 1

来源：https://stackoverflow.com/questions/59179636/how-can-i-count-number-of-lines-after-a-string-match-until-next-especific-match