问题
After getting help from orangeoctopus with this question, I now need to suppress the message "Output Location Validation Failed" "Output directory ... already exists". I know the directory exists, I want it that way. I am pretty sure this will be a matter of overriding something in my Storage UDF, but I am having trouble figuring out what. Totally new to Java so bear with me. Thanks in advance.
回答1:
as far as i know, you cannot reuse a direct output directory. Hadoop prevents it. if i understand correctly, you're dealing with daily logs, therefore, i suggest you set a parent output directory say called output, and set your output directory in the script to be output/daily_date.
回答2:
Delete your output directory before the store operation:
rmf $outpath;
来源:https://stackoverflow.com/questions/10441755/apache-pig-suppress-output-location-validation-failed-output-directory-a