cluster-computing

I've set up a 3 node Apache Hadoop cluster. On master node, I can see [hadoop-conf]$ jps 16856 DataNode 17051 SecondaryNameNode 16701 NameNode 21601 ResourceManager 21742 NodeManager 18335 JobHistoryServer and on slave nodes, I see [fedora20-template dfs]$ jps 28677 Jps 28510 NodeManager 27449 DataNode I can see three live nodes from master:50070. However, in the ResourceManager Web UI ( http://master:8088/cluster/nodes ), I can see only master node. Why are the two slave nodes not in the resource manager? Does that mean they are not part of resources that are used to do mapreduce? pythonician

I'm trying to build nodejs application which will take advantage of multicore machines ( a.k.a. clustering ) and I got a question about sessions. My code looks like this: var cluster = exports.cluster = require('cluster'); var numCPUs = require('os').cpus().length; if (cluster.isMaster) { for (var i = 0; i < numCPUs; i++) { cluster.fork(); } cluster.on('exit', function(worker, code, signal) { console.log('worker ' + worker.process.pid + ' died. Trying to respawn...'); cluster.fork(); }); } else { //spawn express etc } My question is: Everytime a single user hits random node instance or for

I've looked through many examples and I'm still confused. I've compiled a simple latency check program from here , and it runs perfectly on one host, but when I try to run it on two hosts it hangs. However, running something like hostname runs fine: [hamiltont@4 latency]$ mpirun --report-bindings --hostfile hostfile --rankfile rankfile -np 2 hostname [4:16622] [[5908,0],0] odls:default:fork binding child [[5908,1],0] to slot_list 0 4 [5:12661] [[5908,0],1] odls:default:fork binding child [[5908,1],1] to slot_list 0 5 But here is the compiled latency program: [hamiltont@4 latency]$ mpirun -

I have a kubernetes cluster running on 2 machines (master-minion node and minion node). I want to add a new minion node without disrupting the current set up, is there a way to do it? I have seen that when I try to add the new node, the services on the other nodes stops it, due to which I have to stop the services before deploying the new node to the existing cluster. To do this in the latest version (tested on 1.10.0) you can issue following command on the masternode: kubeadm token create --print-join-command It will then print out a new join command (like the one you got after kubeadmn init

From within an Akka actor, how can you find out the nodes of the cluster? That is, the nodes which the local node considers to be currently accessible. Thanks, - Daniel You actually do not need to subscribe to ClusterDomainEvent or MemberEvent . You can just access the state member of the Cluster extension, e.g., val cluster = akka.cluster.Cluster(context.system) val members = cluster.state.members.filter(_.status == MemberStatus.Up) Tweaked sample from Typesafe Activator tutorial : case object GetNodes class MemberListener extends Actor { val cluster = Cluster(context.system) override def

I am using the quartz plugin with Grails 1.3.7. I have a need to load balance/cluster a server app that uses quartz jobs. Apparently this is supported but I am finding that all the google search results and links within documents are broken. I've found some raw Java examples but I would assume Grails has a more grailsy way to do this. All I need is a simple example to use as a template. I understand I need to somehow enable quartz to use JDBC to store the jobs and manage locking. I think a link to a single sample would do it. But literally every time I've found something that looks promising

I am running the termstrc yield curve analysis package in R across 10 years of daily bond price data for 5 different countries. This is highly compute intensive, it takes 3200 seconds per country on a standard lapply, and if I use foreach and %dopar% (with doSNOW) on my 2009 i7 mac, using all 4 cores (8 with hyperthreading) I get this down to 850 seconds. I need to re-run this analysis every time I add a country (to compute inter-country spreads), and I have 19 countries to go, with many more credit yield curves to come in the future. The time taken is starting to look like a major issue. By