cluster-computing

I'm thinking about using hadoop to process large text files on my existing windows 2003 servers (about 10 quad core machines with 16gb of RAM) The questions are: Is there any good tutorial on how to configure an hadoop cluster on windows? What are the requirements? java + cygwin + sshd ? Anything else? HDFS, does it play nice on windows? I'd like to use hadoop in streaming mode. Any advice, tool or trick to develop my own mapper / reducers in c#? What do you use for submitting and monitoring the jobs? Thanks From the Hadoop documentation : Win32 is supported as a development platform .

I have been playing with Cloudera and I define the number of clusters before I start my job then use the cloudera manager to make sure everything is running. I’m working on a new project that instead of using hadoop is using message queues to distribute the work but the results of the work are stored in HBase. I might launch 10 servers to process the job and store to Hbase but I’m wondering if I later decided to add a few more worker nodes can I easily (read: programmable) make them automatically connect to the running cluster so they can locally add to clusters HBase/HDFS? Is this possible

I'm new to weblogic. I've read http://download.oracle.com/docs/cd/E11035_01/wls100/cluster/overview.html and searched this topic on the internet but still had a hard time understanding some of weblogic's clustering concepts. Can anybody confirm/correct my understandings below? a cluster contains one or more logical servers which can reside on one or many physical servers when deploying a j2ee app to a cluster, it is tied to one server in that cluster external users of the deployed app aren't aware of clustering the log file of that app is located on the server it's deployed if the server

In one of the answers to Broadcast like UDP with the Reliability of TCP , a user mentions the Spread messaging API. I've also run across one called ØMQ . I also have some familiarity with MPI . So, my main question is: why would I choose one over the other? More specifically, why would I choose to use Spread or ØMQ when there are mature implementations of MPI to be had? MPI was deisgned tightly-coupled compute clusters with fast, reliable networks. Spread and ØMQ are designed for large distributed systems. If you're designing a parallel scientific application, go with MPI, but if you are

— I built a simple app that pulls in data (50 items) from a Redis DB and throws it up at localhost. I did an ApacheBench (c = 100, n = 50000) and I'm getting a semi-decent 150 requests/sec on a dual-core T2080 @ 1.73GHz (my 6 y.o laptop), but the proc usage is very disappointing as shown: Only one core is used, which is as per design in Node, but I think I can nearly double my requests/sec to ~300, maybe even more, if I can use Node.js clusters. I fiddled around quite a bit but I haven't been able to figure out how to put the code given here for use with my app which is listed below: var