awk and md5: replace a column

问题

Starting from Awk replace a column with its hash value, I tried to hash(md5) a list of numbers:

$ cat -n file
 1  40755462755
 2  40751685373
 3  40730094339
 4  40722740446
 5  40722740446
 6  40743802204
 7  40730094339
 8  40745188886
 9  40740593352
10  40745561530

If I run:

cat file | awk '{cmd="echo -n " $1 " | md5sum|cut -d\" \" -f1"; cmd|getline md5; $1=md5;print;}' | cat -n
 1  29ece26ce4633b6e9480255db194cc40
 2  120148eca0891d0fc645413d0f26b66b
 3  cafc48d392a004f75b669f9d1d7bf894
 4  7b4367e8f58835c0827dd6a2f61b7258
 5  7b4367e8f58835c0827dd6a2f61b7258
 6  49b12d1f3305ab93b33b330e8b1d3165
 7  49b12d1f3305ab93b33b330e8b1d3165
 8  bee44c89ac9d4e8e4e1f1c5c63088c71
 9  f07262ac8f53755232c5abbf062364d0
10  2ac7c22170c00a3527eb99a2bfde2c2c

I don't know why the line 7 get the same md5 as line 6 because if I run them separately they are different:

$ echo -n 40743802204 | md5sum|cut -d" " -f1
49b12d1f3305ab93b33b330e8b1d3165
$ echo -n 40730094339 | md5sum|cut -d" " -f1
cafc48d392a004f75b669f9d1d7bf894

I tried some prints:

cat file| awk '{print $0,NF,NR;cmd="echo -n " $1 " | md5sum|cut -d\" \" -f1"; cmd|getline md5; $1=md5"---"cmd"---"$1;print;}' | cat -n

but with no success to find what's going wrong.

EDIT: As the title says, I try to replace a column in a file(a file with hundred fields). So, $1 would be $24 and NF would be 120 for a file and 233 for another file.

回答1:

Ok, I found the issue. The pipes in awk should be closed. So, I needed a close(cmd);

I found the solution here

回答2:

I wouldn't use getline in awk like that. You can do:

while read -r num; do 
    echo -n $num | md5sum | cut -d ' ' -f1; 
done < file
29ece26ce4633b6e9480255db194cc40
120148eca0891d0fc645413d0f26b66b
cafc48d392a004f75b669f9d1d7bf894
7b4367e8f58835c0827dd6a2f61b7258
7b4367e8f58835c0827dd6a2f61b7258
49b12d1f3305ab93b33b330e8b1d3165
cafc48d392a004f75b669f9d1d7bf894
bee44c89ac9d4e8e4e1f1c5c63088c71
f07262ac8f53755232c5abbf062364d0
2ac7c22170c00a3527eb99a2bfde2c2c

回答3:

I would GUESS, but can't tell since you aren't testing it's return code, that it's because your getline is failing at line 7 so md5 has the same value it did for the previous line. Use of getline is fraught with caveats and not for use by beginners, see http://awk.info/?tip/getline.

What value are you getting out of using awk for this anyway as opposed to just staying in shell?

回答4:

It's a bit awkward with all the quoting - I'm not sure why would it fail to be honest. But here's something that uses less awk and works just fine:

< tmp | while read num ; do echo -n $num | md5sum | cut -f1 -d' '; done | cat -n

来源：https://stackoverflow.com/questions/23696722/awk-and-md5-replace-a-column

标签

bash

awk

md5