How to use awk for a compressed file

Question

How can I change the following command for a compressed file?

awk \'FNR==NR { array[$1,$2]=$8; next } ($1,$2) in array { print $0 \";\" array[$1,$2] }\' inpu         


        

Answer 1

bzip2 -dc input1.vcf.bz2 input2.vcf.bz2 | awk 'FNR==NR { array[$1,$2]=$8; next } ($1,$2) in array { print $0 ";" array[$1,$2] }'

or

gzip -dc input1.vcf.gz input2.vcf.gz | awk 'FNR==NR { array[$1,$2]=$8; next } ($1,$2) in array { print $0 ";" array[$1,$2] }'

EDIT:

To write compressed output just append

| bzip2 >output.vcf.bz2

or

| gzip >output.vcf.gz

This will work with any program that prints results to standard output.

BTW: Editing such large command lines gets tedious very quickly. You should consider writing a small shell script to do the job. This has the additional benefit that you don't have to remember the entire thing and can easily repeat the command or modify it if necessary.

A good starting point for Linux shell programming is the Bash Programming Inroduction by Mike G.