Running shell script from oozie through Hue

问题

I am invoking a bash shell script using oozie editor in Hue.

I used the shell action in the workflow and tried below different options in shell command:

1. Uploaded the shell script using 'choose a file'
2. Gave local directory path where shell script is present
3. Gave HDFS path where shell script is present

But all these options gave following error:

Cannot run program "sec_test_oozie.sh" (in directory "/data/hadoop/yarn/local/usercache/user/appcache/application_1399542362142_0086/container_1399542362142_0086_01_000002"): java.io.IOException: error=2, No such file or directory

How should I give the shell script execution command? Where the shell script file should be residing?

回答1:

You need add file "sec_test_oozie.sh" in oozie shell step. In add files

回答2:

I think you are creating the file from windows machine which is adding extra line break characters.You need to convert the shell script file to Unix format.I also faced the same issue.Then I created the file from a Linux system and it started working.The error is misguiding.

回答3:

I want to extend the @SergioRG answer. Oozie, at least with Cloudera's Hue interface is very counterintuitive.

To run a script file, three conditions should be met:

1. the file is on the HDFS file system, in a folder accessible by Oozie
2. the file should be indicated in the shell command field
3. the file should be added with any other dependent file in the "Files+" part of the task card.

I wonder why they didn't add by default the script file you are calling.

Edit: please also check in advanced options (the gear in the left upper corner) if you need to set the path variable (eg. PATH=/usr/local/bin:/usr/bin).

回答4:

Did you edit sec_test_oozie.sh with the Hue File Browser? Depending on your Hue version it might have corrupted it: hue-list

回答5:

I encountered the same problem, and the problem was that the script echoed some irrelevant line while the workflow tried to parse it as a property line. Oozie gave a very irrelevant error message of java.io.IOException: error=2, No such file or directory which only added confusion.

1. You will need to use <file> to add your script.
2. If you used <capture-output/> then you must make sure that your script prints only "key=value" lines, like java properties, otherwise you will get the error you see java.io.IOException: error=2, No such file or directory with some path pointing to .../yarn/local/usercache/...

回答6:

We had this issue on a test script, basically if you use an editor that adds wierd characters or line ending to the file, it'll throw this error because the script cannot be used in the container.

Try using nano file.sh to see if any strange characters appear. Then push it back to hdfs with hdfs dfs -put file.sh /path/you/need

回答7:

Removing the #!/bin/bash from my shell script helped me

回答8:

"No such a file or directory" oozie cannot locate the file. Please check the AddPath setting in the command.

In the edit node seciton, get the oozie application hdfs path.

Upload the shell script in hdfs oozie application path.

In the oozie edit node step, Shell command - specify the shell script name which is uploaded.

Below that there would be option to AddPath, then add files, add the shell script which was uploaded in the hdfs path.

来源：https://stackoverflow.com/questions/23628129/running-shell-script-from-oozie-through-hue