what difference between execute a map-reduce job using hadoop and java command

百般思念 提交于 2019-12-29 07:34:09

问题


Find many option for run a map-reduce program. Can any one explain difference between theses below commands. And what impact on Map-reduce job if any.

     java -jar MyMapReduce.jar [args] 

     hadoop jar MyMapReduce.jar [args] 

     yarn jar MyMapReduce.jar [args] 

In these command which one best or other in any?

Can make configuration like display all information about job using Yarn and Job History (like as display Hadoop and yarn command) on Web service normal using port for web service 8088(YARN) on below command?

     java -jar MyMapReduce.jar [args] 

回答1:


None of them is better than the other. When you execute the java -jar command, it is just the same as executing a non Hadoop application. If you use hadoop jar or yarn jar, it will use the /usr/bin/hadoop and /usr/bin/yarn scripts to set up the environment.

Unless you have modified any of the scripts to set up additional variables, the three of them should work the same.




回答2:


The options have its own specifications :

   java -jar MyMapReduce.jar [args] 

The above expects that all the hadoop jars are defined in class path of the jar .

in case of

   hadoop jar MyMapReduce.jar [args]  and 
   yarn jar MyMapReduce.jar [args] 

The above jars will run by fetching those jars predefined in $HADOOP_CLASSPATH.



来源:https://stackoverflow.com/questions/25180142/what-difference-between-execute-a-map-reduce-job-using-hadoop-and-java-command

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!