Encountered IOException while registering python UDF in pig. File helloworld.py does not exist

问题

Pytjon UDF :

@outputSchema("word:chararray")
def helloworld():
  return 'Hello, World'


register '/user/hdfs/helloworld.py' using jython as myfunc;

Error:

grunt> REGISTER 'helloworld.py' USING org.apache.pig.scripting.jython.JythonScriptEngine as myfuncs;
2016-05-16 12:08:04,909 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2997: Encountered IOException. File helloworld.py does not exist
2016-05-16 12:08:04,909 [main] WARN  org.apache.pig.tools.grunt.Grunt - There is no log file to write to.
2016-05-16 12:08:04,909 [main] ERROR org.apache.pig.tools.grunt.Grunt - java.io.FileNotFoundException: File helloworld.py does not exist
        at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:611)
        at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:824)
        at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:601)
        at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:421)
        at org.apache.pig.impl.io.FileLocalizer.fetchFilesInternal(FileLocalizer.java:818)
        at org.apache.pig.impl.io.FileLocalizer.fetchFile(FileLocalizer.java:756)
        at org.apache.pig.PigServer.registerCode(PigServer.java:593)
        at org.apache.pig.tools.grunt.GruntParser.processRegister(GruntParser.java:467)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:546)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:230)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)
        at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:66)
        at org.apache.pig.Main.run(Main.java:558)
        at org.apache.pig.Main.main(Main.java:170)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

回答1:

can you do a "ls /user/hdfs/helloworld.py" and check whether the file exists

来源：https://stackoverflow.com/questions/37248294/encountered-ioexception-while-registering-python-udf-in-pig-file-helloworld-py

标签

python

Hadoop

apache-pig