Can't close after awk command inside for loop to read and write multiple files

问题

I have a set of csv files, one of them has a timestamp, all the other ones just have data, but they all have the same number of rows. The timestamp is in the 2nd column of the csv.

I would like to append the timestamp to the first column of all csv files. This is currently working but when I try to close the files, I get an error. There can be 50-500 csv files, and each can have thousands of rows, so that is why I wonder if the close() is required.

Also, can anyone suggest any ways of improving this script either for performance or for reliability and check for any errors?

Sample input blah_blah_blah_timestamp.csv

name, time
1,121
2,122
3,123

data1.csv

name,X1
A1,11
A2,12
A3,15

data2.csv

name,Y1,Y2,Y3
B1,1,1,2
B2,2,1,3
B3,3,2,4

data3.csv

name,Z1,Z2,Z3
C1,1,9,5
C2,2,8,4
C3,3,7,3

out/data1.csv

time,name,X1
121,A1,11
122,A2,12
123,A3,15

out/data2.csv

time,name,Y1,Y2,Y3
121,B1,1,1,2
122,B2,2,1,3
123,B3,3,2,4

out/data3.csv

time,name,Z1,Z2,Z3
121,C1,1,9,5
122,C2,2,8,4
123,C3,3,7,3

current script

#!/bin/bash
mkdir -p out
ts='blah_blah_blah_timestamp.csv'
for sfile in *.csv;
do
   awk -F, -v afile="$sfile" '{getline f1 < afile ;print $2, f1}' OFS=, $ts >  out/"$sfile"
   close('sfile')
done

回答1:

This is probably what you want:

awk '
BEGIN { FS=OFS="," }
NR==FNR {
    map[FNR] = $2
    next
}
FNR==1 {
    close(out)
    out = "out/" FILENAME
}
{ print map[FNR], $0 > out }
' blah_blah_blah_timestamp.csv data*.csv

See https://stackoverflow.com/a/65814521/1745001 for an explanation for why/how I'm using close().

By the way, if you're ever considering using getline in future - it's rarely the right approach and it's hard to use correctly, see http://awk.freeshell.org/AllAboutGetline.

来源：https://stackoverflow.com/questions/65795640/cant-close-after-awk-command-inside-for-loop-to-read-and-write-multiple-files

标签

bash

csv

awk