Running MapReduce jobs on AWS-EMR from Eclipse

Question

I have the WordCount MapReduce example in Eclipse. I exported it to Jar, and copied it to S3. I then ran it on AWS-EMR. Successfully.

Then, I read this article - http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-common-programming-sample.html It shows how to use AWS-EMR Api to run MapReduce jobs. It still assumes your MapReduce code is packaged in a Jar.

I would like to know if there is a way to run MapReduce code from Eclipse directly on AWS-EMR, without having to export it to a Jar.

Answer 1

I haven't found a way to do this (for mapreduce jobs written in Java). I believe there is no Eclipse utility or plugin that does this. You could maybe write a script of some kind and incorporate it in your build process to push out the Jar to your EMR cluster. But there is no single utility/tool that will compile, bundle, deploy and execute your mapreduce job to EMR.