Java framework/tool for simple distributed computing problem

北城以北 提交于 2021-02-05 20:12:35

问题


We generate pdf files with data regarding monthly financial balance of tens of thousands of clients. At it's peak (100.000 files at the end of year), the process may take as long as 5 days to complete using distribute the load between 5 servers. The distribution of workload is a manual process (eg. server 1 generates pdf for clients 1 to 20.000, server 2 from 20.001 to 40.000, and so on).

We use Java, so we would like to use a Java tool or framework in a fashion similar to BOINC (BOINC is not coded in Java), which would allow us to have:

  • a server automatically distributing data and code to workers, and managing them
  • workers (plain dual core desktops using free (or not so) cpu cycles) receiving code sent by server and executing it using the data as input, giving some status feedback, and finally doing something with the output, eg. sending pdf generated files to network storage.

For now, we don't want to try heavyweight all-powerful toolkits such as Globus Toolkit.

¿Do you know any java framework/tools suitable to our needs? Open source would be nice, but we don't discard commercial solutions.


回答1:


There are many:

  • http://www.jppf.org/
  • https://jxta.dev.java.net/
  • https://java.net/projects/cajo/pages/Home
  • http://www.dmoz.org/Computers/Programming/Languages/Java/Class_Libraries/Networking_and_Distributed_Computing/

Google is your best friend here.




回答2:


  • http://hadoop.apache.org/
  • http://www.gridgain.com/
  • http://www.oracle.com/technology/products/coherence/index.html



回答3:


Check out Hazelcast distributed executor service. It uses Java's standard ExecutorService API.




回答4:


Another one you can look at is Terracotta.




回答5:


Google says there is a Java port of BOINC here: http://boincoid.sf.net

It's mostly targeting about android :-/ but apparently all but the GUI can run on a regular VM.



来源:https://stackoverflow.com/questions/1338127/java-framework-tool-for-simple-distributed-computing-problem

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!