grep two files (a.txt, b.txt) - how many lines in b.txt starts (or ends) with the words from a.txt - output: 2 files with the results

问题

I know I ask too much, but maybe you can help with this problem too.

a.txt contains words, b.txt contains strings.

I would like to know how many string from b.txt ends with the words from a.txt

Example: a.txt

apple
peach
potato

b.txt

greenapple
bigapple
rottenapple
pinkpeach
xxlpotatoxxx

Output

3 apple greenapple bigapple rottenapple
1 peach pinkpeach

I would like to have a solution with grep, since it is way more faster than awk.

Can you guys please help me?

回答1:

Here is an awk solution

awk 'FNR==NR{a[$1]++;next} {for (i in a) {if ($0~i"$") {b[i]++;w[i]=w[i]?w[i] FS $0:$0}}} END {for (j in b) print b[j],j,w[j]}' a.txt b.txt
3 apple greenapple bigapple rottenapple
1 peach pinkpeach

It will not be simple or not possible at all to do this with grep

How does it work (it's not so complicated)?

awk '
FNR==NR{                        # Run this part for first file (a.txt) only
  a[$1]++                       # Store it in an array a
  next}                         # Skip to next record
  {                             # Run this part for file b.txt
  for (i in a) {                # Loop trough all data in array a
    if ($0~i"$") {              # Does b.txt have some from array a at the end of it?
      b[i]++                    # Yes , count it
      w[i]=w[i]?w[i] FS $0:$0   # and store the record it found it in in array w
      }
    }
  } 
END {                           # When both file has been read do the END part
  for (j in b)                  # Loop trough all element in array b and
    print b[j],j,w[j]}          # Print array b, index and array w
  ' a.txt b.txt                 # Read the two files

回答2:

Here is a solution that relies on bash and grep only. IMHO it is a little bit easier to understand than an awk-only approach:

#!/bin/bash

# Set input parameters (usually a good idea than hardcoding them)
WORDFILE=a.txt
SEARCHFILE=b.txt

# Read 'a.txt' word by word (i.e. line by line)
while read word; do
  # Get numbers of hits
  num=`grep "$word\$" $SEARCHFILE | wc -l`

  # If no line matches in 'b.txt', skip this word
  if [ $num -eq 0 ]; then
    continue
  fi

  # Print number of hits and search word
  printf "%d $word" $num

  # Print all lines that match from file 'b.txt'
  for found in `grep "$word\$" $SEARCHFILE`; do
    printf " $found"
  done

  # Print newline
  printf "\n"
done < $WORDFILE

EDIT

If you want to store the results in a file, you can redirect the output of the above script the usual way, e.g.

./find_matching_ends.sh > matching_ends.txt

If you want to search for lines that start with the word, you need to change the grep pattern from "$word\$" to "^$word". If you want this search to happen simultaneously to the search for matching ends, you need to move the redirection above inside your script, e.g.

...
printf "%d $word" $num > matching_ends.txt
...

when you are searching for matching ends, and

...
printf "%d $word" $num > matching_starts.txt
...

when your are looking for lines that start with the search word.

回答3:

I'd like to come up with a solution based on Bash that avoids grep. Instead it uses for-loops and arrays:

#!/usr/bin/env bash

# Set mode: start | end
mode="end"

# Read contents of input files into arrays - line by line
IFS=$'\n' read -d -r -a patterns < "$1"
IFS=$'\n' read -d -r -a targets < "$2"

# Bash 4 can use readarray
#readarray -t patterns < "$1"
#readarray -t targets < "$2"

# Alternatively use cat to get the contents into arrays (slower)
#patterns=($(cat $1))
#targets=($(cat $2))


# Iterate over both arrays to compare the strings with each other
for pattern in "${patterns[@]}"; do

    # Setup a variable that counts the hits for each pattern
    hits_counter=0

    # Setup a variable that takes the matched strings for each pattern
    hits_match=""

    # Setup a regex pattern according to the user defined mode
    if [[ "$mode" == "start" ]]; then
        regex="^${pattern}"
    elif [[ "$mode" == "end" ]]; then
        regex="${pattern}$"
    fi

    for target in "${targets[@]}"; do

        # Use regex pattern matching
        if [[ "$target" =~ $regex ]]; then

            # If we detect a match increase the counter by 1
            (( hits_counter++ ))

            # If we detect a match write it to our hits_match variable and append a space
            hits_match+="${target} "
        fi
    done

    # Print a result for each pattern if we have at least one match
    if (( hits_counter > 0 )); then
        printf "%i %s %s\n" "$hits_counter" "$pattern" "$hits_match"
    fi
done

This gives the follwing result:

./filter a.txt b.txt
3 apple greenapple bigapple rottenapple
1 peach pinkpeach

来源：https://stackoverflow.com/questions/22421188/grep-two-files-a-txt-b-txt-how-many-lines-in-b-txt-starts-or-ends-with-th

标签

string

bash

grep

comparison