Oozie

问题 Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], main() threw exception, begin > end in range (begin, end): (1464070940802, 1464070938036) java.lang.IllegalArgumentException: begin > end in range (begin, end): (1464070940802, 1464070938036) at org.apache.hadoop.yarn.api.protocolrecords.impl.pb.GetApplicationsRequestPBImpl.setStartRange(GetApplicationsRequestPBImpl.java:340) at org.apache.oozie.action.hadoop.LauncherMainHadoopUtils.getChildYarnJobs

问题 I am able to execute the following SQOOP command in CLI perfectly. sqoop list-tables --connect 'jdbc:sqlserver://xx.xx.xx.xx\MSSQLSERVER2012:1433;username=usr;password=xxx;database=db' --connection-manager org.apache.sqoop.manager.SQLServerManager --driver com.microsoft.sqlserver.jdbc.SQLServerDriver -- --schema schma But getting errors while trying out the same in OOZIE (HUE) 2055 [main] ERROR org.apache.sqoop.manager.CatalogQueryManager - Failed to list tables java.sql.SQLException: No

问题 2.2.0 and oozie-4.0.0 in ubuntu. I am cant able to execute mapreduce programs in oozie. i am uisng resource manager port number for jobtracker 8032 in oozie. while scheduling in oozie to goes to running state and running in yarn also after some time i am getting error like this(below) in hadoop logs and still running in oozie logs Error: 2014-05-30 10:38:14,322 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application

问题 I have installed Apache Amabari and configured the Hue . I want to export hbase table data into csv file using pig script but I am getting following error. 2017-06-03 10:27:45,518 [ATS Logger 0] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Exception caught by TimelineClientConnectionRetry, will try 30 more time(s). Message: java.net.ConnectException: Connection refused 2017-06-03 10:27:45,703 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is

问题 I am currently using Oozie to call Shell script. On base of exit value of the called shell script, Oozie can easily choose OK or ERROR node, but what if Shell exits with other values than 0 or 1? can Oozie use the decision node to switch to different nodes on base of the exit value of the called Shell script? I am thinking about using EL function, but do not know whyc EL function get the exit code of previously called Shell script 回答1: make sure that the shell exits with non-zero for critical

问题 In one of our pipelines we are doing aggregation using spark(java) and it is orchestrated using oozie. This pipelines writes the aggregated data to an ORC file using the following lines. HiveContext hc = new HiveContext(sc); DataFrame modifiedFrame = hc.createDataFrame(aggregateddatainrdd, schema); modifiedFrame.write().format("org.apache.spark.sql.hive.orc").partitionBy("partition_column_name").save(output); When the spark action in the oozie job gets triggered it throws the following

问题 I am unable to add/delete any files or directories on HDFS from a shell script which I am executing from Oozie workflow. The username is "scitest" and the hdfs path I am trying to edit/add/delete is /user/scitest/. In the shell script I am trying to delete a folder named test123456 from the path /user/scitest/. ---------------Error from oozie log------------------ 429737-oozie-oozi-W@shell-node] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1] 2016

问题 In Oozie, I have used Hive action in Hue, same action I used parameter options to supply date parameter. Here I want to provide dynamic date parameter such as yesterday date and day before yesterday. How can I generate those date? and how can I pass as parameter. My HQL is : CREATE TABLE IF NOT EXISTS tmp_table as select * from emptable where day>=${fromdate} and day<=${todate} My HiveServer Action contains: a. HQL script b. Two parameters options one for each dates like as fromdate = ,

问题 I have a sqoop action which pulls data from postgres database and then imports into a hive table. When I execute the oozie workflow, scoop pulls the data from postgres into HDFS. But it fails to import data into the hive table. The logs are no way useful as I just get Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1] from the oozie web console UI. Can we actually do a hive import inside sqoop action? Or do I have to perform a Hive action separately after sqoop does an

问题 I am running Hadoop 2.4.0, Oozie 4.0.0, Hive 0.13.0 from Hortonworks distro. I have multiple Oozie coordinator jobs that can potentially launch workflows all around the same time. The coordinator jobs each watch different directories and when the _SUCCESS files show up in those directories, the workflow would be launched. The workflow runs a Hive action that reads from external directory and copy stuff. SET hive.exec.dynamic.partition=true; SET hive.exec.dynamic.partition.mode=nonstrict; DROP