Hadoop/Elastic Map Reduce with binary executable?

╄→гoц情女王★ 提交于 2019-12-08 06:50:58

问题


I am writing and distributed image processing application using hadoop streaming, python, matlab, and elastic map reduce. I have compiled a binary executable of my matlab code using the matlab compiler. I am wondering how I can incorporate this into my workflow so the binary is part of the processing on Amazon's elastic map reduce?

It looks like I have to use the Hadoop Distributed Cache?

The code is very complicated (and not written by me) so porting it to another language is not possible right now.

THanks


回答1:


The following is not exactly an answer to your Hadoop question, but I couldn't resist not asking why you don't execute your processing jobs on the Grid resources? There are proven solutions for executing compute intensive workflows on the Grid. And as far as I know matlab runtime environment is usually available on these resources. You may also consider using the Grid especially if you are in academia.

Good luck




回答2:


Joseph,

I've just asked a similar, but more general version of your question. Hopefully we'll get some answers. ;)

Hadoop Streaming: Mapper 'wrapping' a binary executable

  • Nick


来源:https://stackoverflow.com/questions/4101815/hadoop-elastic-map-reduce-with-binary-executable

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!