Pig Performance Measurement

问题

I wrote a Pig script and want to execute it on Hadoop cluster. How could I measure the total processing time? Is there any command that I could get the processing time from start to end?

回答1:

EDIT: Added the time alternative.

To know how long it takes (in seconds):

time pig <options>

Another way to do it:

d1=$(date +%s)
pig <options>
d2=$(date +%s)
echo "$d2 - $d1" | bc

Or, in a single line:

d1=$(date +%s) ; pig <options> ; d2=$(date +%s) ; echo "$d2 - $d1" | bc

You can also just take a look at the output of pig. When you run a pig script in the command line, towards the end of the output you'll see:

HadoopVersion   PigVersion  UserId  StartedAt   FinishedAt  Features
...

You can then subtract FinishedAt - StartedAt.

来源：https://stackoverflow.com/questions/19434894/pig-performance-measurement