Oozie

问题 When run from the command line using spark2-submit its running under Spark version 2.2.0. But when i use a oozie spark action its running under Spark version 1.6.0 and failing with error TypeError: 'JavaPackage' object is not callable My oozie spark action below <!-- Spark action first --> <action name="foundationorder" cred="hcat"> <spark xmlns="uri:oozie:spark-action:0.1"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <job-xml>${hiveConfig}</job-xml> <master

问题 Our oozie jobs are failing with java.lang.ClassNotFoundException. Please find complete log attached. 2019-11-26 12:41:31,690 INFO [main] org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state INITED; cause: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.OozieLauncherOutputCommitter not found java.lang.RuntimeException: java.lang.RuntimeException: java.lang

问题 I am using the workflow editor in Hue to develop an Oozie workflow. There are a few action that should be executed in parallel. Is it possible to execute two or more actions concurrently? How can I set it up in Hue? 回答1: Yes, it is possible. Among various Oozie workflow nodes, there are two control nodes fork and join : A fork node splits one path of execution into multiple concurrent paths of execution. A join node waits until every concurrent execution path of a previous fork node arrives

问题 I am attempting to run a sqoop command with a free-form query, because I need to perform an aggregation. It's being submitted via the Hue interface, as an Oozie workflow. The following is a scaled-down version of the command and query. When the command is processed, the "--query" statement (enclosed in quotes) results in each portion of the query to be interpreted as unrecognized arguments, as shown in the error following the command. In addition, the target directory is being misinterpreted.

问题 I am attempting to run a sqoop command with a free-form query, because I need to perform an aggregation. It's being submitted via the Hue interface, as an Oozie workflow. The following is a scaled-down version of the command and query. When the command is processed, the "--query" statement (enclosed in quotes) results in each portion of the query to be interpreted as unrecognized arguments, as shown in the error following the command. In addition, the target directory is being misinterpreted.

问题 Can there be two oozie workflow.xml files in one directory? If so how can I instruct oozie runner which one to run? 回答1: You can have two workflow files (just give them unique names), then you can select which one to call by setting the oozie.wf.application.path value in your config file: oozie.wf.application.path=hdfs://namenode:9000/path/to/job/wf-1.xml #oozie.wf.application.path=hdfs://namenode:9000/path/to/job/wf-2.xml 回答2: Use 2 different directories. But if you need to call the second

问题 Oozie SSH Action Issue: Issue: We are trying to run few commands on a particular host machine of our cluster. We chose SSH Action for the same. We have been facing this SSH issue for some time now. What might be the real issue here? Please point me towards the solution. logs: AUTH_FAILED: Not able to perform operation [ssh -o PasswordAuthentication=no -o KbdInteractiveDevices=no -o StrictHostKeyChecking=no -o ConnectTimeout=20 USER@1.2.3.4 mkdir -p oozie-oozi/0000000-131008185935754-oozie

问题 I am working on automating Pig jobs using oozie in hadoop cluster. I was able to run a sample pig script from oozie but my next requirement is to run a pig job where the pig script recieves it's input parameters from a shell script. Please share your thoughts 回答1: UPDATE: OK make the original question clear, how can you pass a parameter form a shell script output. Here's the working example: WORKFLOW.XML <workflow-app xmlns='uri:oozie:workflow:0.3' name='shell-wf'> <start to='shell1' />

问题 I have three different type of jobs running on the data in HDFS. These three jobs have to be run separately in the current scenario. Now, we want to run the three jobs together by piping the OUTPUT data of one job to the other job without writing the data in HDFS to improve the architecture and overall performance. Any suggestions are welcome for this scenario. PS : Oozie is not fitting for the workflow.Cascading framework is also ruled out because of Scalability issues. Thanks 回答1: Hadoop