cluster-computing

I would like to submit MapReduce jobs from a java web application to a remote Hadoop cluster but am unable to specify which user the job should be submitted for. I would like to configure and use a system user which should be used for all MapReduce jobs. Currently I am unable to specify any user and no matter what the hadoop job runs under the username of the currently logged in user of the client system. This causes an error with the message Permission denied: user=alice, access=WRITE, inode="staging":hduser:supergroup:rwxr-xr-x ... where "alice" is the local, logged in user on the client

I am having trouble understanding the concept of blocking communication and non-blocking communication in MPI. What are the differences between the two? What are the advantages and disadvantages? user1202136 Blocking communication is done using MPI_Send() and MPI_Recv() . These functions do not return (i.e., they block) until the communication is finished. Simplifying somewhat, this means that the buffer passed to MPI_Send() can be reused, either because MPI saved it somewhere, or because it has been received by the destination. Similarly, MPI_Recv() returns when the receive buffer has been

In Submitting Applications in the Spark docs, as of 1.6.0 and earlier , it's not clear how to specify the --jars argument, as it's apparently not a colon-separated classpath not a directory expansion. The docs say "Path to a bundled jar including your application and all dependencies. The URL must be globally visible inside of your cluster, for instance, an hdfs:// path or a file:// path that is present on all nodes." Question: What are all the options for submitting a classpath with --jars in the spark-submit script in $SPARK_HOME/bin? Anything undocumented that could be submitted as an

At the startup I'm working at we are now considering scaling solutions for our database. Things get somewhat confusing (for me at least) with MySQL, which has the MySQL cluster , replication and MySQL cluster replication (from ver. 5.1.6), which is an asynchronous version of the MySQL cluster. The MySQL manual explains some of the differences in its cluster FAQ , but it is hard to ascertain from it when to use one or the other. I would appreciate any advice from people who are familiar with the differences between those solutions and what are the pros and cons, and when do you recommend to use