cluster-computing

问题 I am using the quartz plugin with Grails 1.3.7. I have a need to load balance/cluster a server app that uses quartz jobs. Apparently this is supported but I am finding that all the google search results and links within documents are broken. I've found some raw Java examples but I would assume Grails has a more grailsy way to do this. All I need is a simple example to use as a template. I understand I need to somehow enable quartz to use JDBC to store the jobs and manage locking. I think a

问题 I would like to get the docker container name from within a container. I need this information because I am running a script from the Dockerfile as an ENTRYPOINT that needs to know the node number. I am using the --scale with docker-compose to create a cluster of my image. I am able to get the hostname of the container which is the container ID. But is there a way to get the container name from inside the container? 回答1: Docker remote apis should work for you. Depending on the version of your

问题 I am trying to fix an issue with running out of memory, and I want to know whether I need to change these settings in the default configurations file ( spark-defaults.conf ) in the spark home folder. Or, if I can set them in the code. I saw this question PySpark: java.lang.OutofMemoryError: Java heap space and it says that it depends on if I'm running in client mode. I'm running spark on a cluster and monitoring it using standalone. But, how do I figure out if I'm running spark in client mode

问题 Imagine you have many clustered servers, across many hosts, in a heterogeneous network environment, such that the connections between servers may have wildly varying latencies and bandwidth. You want to build a map of the connections between servers by transferring data between them. Of course, this map may become stale over time as the network topology changes - but lets ignore those complexities for now and assume the network is relatively static. Given the latencies between nodes in this

问题 I need to submit spark apps/jobs onto a remote spark cluster. I have currently spark on my machine and the IP address of the master node as yarn-client. Btw my machine is not in the cluster. I submit my job with this command ./spark-submit --class SparkTest --deploy-mode client /home/vm/app.jar I have the address of my master hardcoded into my app in the form val spark_master = spark://IP:7077 And yet all I get is the error 16/06/06 03:04:34 INFO AppClient$ClientEndpoint: Connecting to master

问题 I want the following During startup, the master process loads a large table from file and saves it into a shared variable. The table has 9 columns and 12 million rows, 432MB in size. The worker processes run HTTP server, accepting real-time queries against the large table. Here is my code, which obviously does not achieve my goal. var my_shared_var; var cluster = require('cluster'); var numCPUs = require('os').cpus().length; if (cluster.isMaster) { // Load a large table from file and save it

问题 What is the best strategy to refactor a Singleton object to a cluster environment? We use Singleton to cache some custom information from Database. Its mostly read-only but gets refreshed when some particular event occurs. Now our application needs to be deployed in a Clustered environment. By definition, each JVM will have its own Singleton instance. So the cache may be out-of-sync between the JVM's when a refresh event occurs on a single node and its cache is refreshed. What is the best way

问题 I have working (stock) script from node var cluster = require('cluster'); var http = require('http'); var numReqs = 0; if (cluster.isMaster) { // Fork workers. for (var i = 0; i < 2; i++) { var worker = cluster.fork(); worker.on('message', function(msg) { if (msg.cmd && msg.cmd == 'notifyRequest') { numReqs++; } }); } setInterval(function() { console.log("numReqs =", numReqs); }, 1000); } else { // Worker processes have a http server. http.Server(function(req, res) { res.writeHead(200); res

问题 I have working (stock) script from node var cluster = require('cluster'); var http = require('http'); var numReqs = 0; if (cluster.isMaster) { // Fork workers. for (var i = 0; i < 2; i++) { var worker = cluster.fork(); worker.on('message', function(msg) { if (msg.cmd && msg.cmd == 'notifyRequest') { numReqs++; } }); } setInterval(function() { console.log("numReqs =", numReqs); }, 1000); } else { // Worker processes have a http server. http.Server(function(req, res) { res.writeHead(200); res

问题 I am looking for a set of instructions to create a local multi-node Cassandra 2.x cluster on a Window 7 64 bit PC. It should preferably use CCM “Cassandra Cluster Manager” and allow management using DataStax OpsCenter I followed the instructions in “Getting Started with Apache Cassandra on Windows the Easy Way” but they are for a single node cluster. EDIT: I got stuck on deploying OpsCenter agents on each node using CCM, any ideas? 回答1: Articles used for this tutorial: CCM 2.0 and Windows