How to Extract Parts of String in Shell Script into Variables

Question

I am trying to do the following in sh.

Here's my file:

foo
bar
Tests run: 729, Failures: 0, Errors: 253, Skipped: 0
baz

How can I pull the 4 numbers into 4 different variables? I've spent about an hour now on sed and awk man pages and I'm spinning my wheels.

Answer 1

Adopting my prior answer to use the heredoc approach suggested by @chepner:

read run failures errors skipped <<EOF
$(grep -E '^Tests run: ' <file.in | tr -d -C '[:digit:][:space:]')
EOF

echo "Tests run: $run"
echo "Failures: $failures"
echo "Errors: $errors"
echo "Skipped: $skipped"

---

Alternately (put this into a shell function to avoid overriding "$@" for the duration of the script):

unset IFS # assert default values
set -- $(grep -E '^Tests run: ' <in.file | tr -d -C '[:digit:][:space:]')
run=$1; failures=$2; errors=$3; skipped=$4

Note that this is only safe because no glob characters can be present in the output of tr when run in this way; set -- $(something) usually a practice better avoided.

---

Now, if you were writing for bash rather than POSIX sh, you could perform regex matching internal to the shell (assuming in the below that your input file is relatively short):

#!/bin/bash
re='Tests run: ([[:digit:]]+), Failures: ([[:digit:]]+), Errors: ([[:digit:]]+), Skipped: ([[:digit:]]+)'
while IFS= read -r line; do
  if [[ $line =~ $re ]]; then
    run=${BASH_REMATCH[1]}
    failed=${BASH_REMATCH[2]}
    errors=${BASH_REMATCH[3]}
    skipped=${BASH_REMATCH[4]}
  fi
done <file.in

If your input file is not short, it may be more efficient to have it pre-filtered by grep, thus changing the last line to:

done < <(egrep -E '^Tests run: ' <file.in)

Answer 2

Given the format of the input file, you can capture the output of grep in a here document, then split it with read into four parts to be post-processed.

IFS=, read part1 part2 part3 part4 <<EOF
$(grep '^Tests run' input.txt)
EOF

Then just strip the unwanted prefix from each part.

run=${part1#*: }
failures=${part2#*: }
errors=${part3#*: }
skipped=${part4#*: }

Answer 3

assuming there is only one line starting with Tests run: in your file, and that the file is named foo.txt, the following command will create 4 shell variable that you can work with:

eval $(awk 'BEGIN{ FS="(: |,)" }; /^Tests run/{ print "TOTAL=" $2 "\nFAIL=" $4 "\nERROR=" $6 "\nSKIP=" $8 }' foo.txt); echo $TOTAL; echo $SKIP; echo $ERROR; echo $FAIL

---

echo $TOTAL; echo $SKIP; echo $ERROR; echo $FAIL is just to demonstrate that the environment variable exists and can be used.

The awk script in a more readable manner is:

BEGIN { FS = "(: |,)" }

/^Tests run/ {
    print "TOTAL=" $2 "\nFAIL=" $4 "\nERROR=" $6 "\nSKIP=" $8
}

FS = "(: |,)" tells awk to consider ":" or "," as field separators.

Then the eval command will read as a command the result of the awk script and as such create the 4 environment variables.

---

NOTE: due to the use of eval, you must trust the content of the foo.txt file as one could forge a line starting with Tests run: which could have commands thereafter.

You could improve that bit by having a more restrictive regex in the awk script: /^Tests run: \d+, Failures: \d+, Errors: \d+, Skipped: \d+$/

The full command would then be:

eval $(awk 'BEGIN{ FS="(: |,)" }; /^Tests run: \d+, Failures: \d+, Errors: \d+, Skipped: \d+$/{ print "TOTAL=" $2 "\nFAIL=" $4 "\nERROR=" $6 "\nSKIP=" $8 }' foo.txt); echo $TOTAL; echo $SKIP; echo $ERROR; echo $FAIL

Answer 4

There are shorter versions, but this one "shows" each step.

#!/bin/bash
declare -a arr=`grep 'Tests ' a | awk -F',' '{print $1 "\n" $2 "\n" $3 "\n" $4}' | sed 's/ //g' | awk -F':' '{print $2}'`
echo $arr
for var in $arr
do
    echo $var
done