Running MapReduce jobs on AWS-EMR from Eclipse

a 夏天 提交于 2019-12-07 13:30:24

问题


I have the WordCount MapReduce example in Eclipse. I exported it to Jar, and copied it to S3. I then ran it on AWS-EMR. Successfully.

Then, I read this article - http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-common-programming-sample.html It shows how to use AWS-EMR Api to run MapReduce jobs. It still assumes your MapReduce code is packaged in a Jar.

I would like to know if there is a way to run MapReduce code from Eclipse directly on AWS-EMR, without having to export it to a Jar.


回答1:


I haven't found a way to do this (for mapreduce jobs written in Java). I believe there is no Eclipse utility or plugin that does this. You could maybe write a script of some kind and incorporate it in your build process to push out the Jar to your EMR cluster. But there is no single utility/tool that will compile, bundle, deploy and execute your mapreduce job to EMR.



来源:https://stackoverflow.com/questions/24159980/running-mapreduce-jobs-on-aws-emr-from-eclipse

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!