cluster-computing

How fast can one submit consecutive and independent jobs with qsub?

*爱你&永不变心* 提交于 2019-12-03 08:36:21
This question is related to pbs job no output when busy . i.e Some of the jobs I submit produce no output when PBS/Torque is 'busy'. I imagine that it is busier when many jobs are being submitted one after another, and as it so happens, of the jobs submitted in this fashion, I often get some that do not produce any output. Here're some codes. Suppose I have a python script called "x_analyse.py" that takes as its input a file containing some data, and analyses the data stored in the file: ./x_analyse.py data_1.pkl Now, suppose I need to: (1) Prepare N such data files: data_1.pkl, data_2.pkl, ..

Clustered EJBs not being balanced in JBoss AS 7

萝らか妹 提交于 2019-12-03 08:31:00
I've successfully setup a cluster of 2 JBoss AS 7 instances, and deployed the following SLSB: @Stateless @Remote(TestEJBRemote.class) @Clustered public class TestEJB implements TestEJBRemote { private static final long serialVersionUID = 1L; private static final Logger logger = Logger.getLogger(...); @Override public void test() { String nodeName = System.getProperty("jboss.node.name"); logger.info(nodeName); } } From the log files I can see that the bean is correctly deployed on the cluster. On the client side I then create a number of threads that lookup and invoke an instance of TestEJB .

Apache Spark: “failed to launch org.apache.spark.deploy.worker.Worker” or Master

。_饼干妹妹 提交于 2019-12-03 08:26:51
问题 I have created a Spark cluster on Openstack running on Ubuntu14.04 with 8gb of ram. I created two virtual machines with 3gb each (keeping 2 gb for the parent OS). Further, i create a master and 2 workers from first virtual machine and 3 workers from second machine. The spark-env.sh file has basic setting with export SPARK_MASTER_IP=10.0.0.30 export SPARK_WORKER_INSTANCES=2 export SPARK_WORKER_MEMORY=1g export SPARK_WORKER_CORES=1 Whenever i deploy the cluster with start-all.sh, i get "failed

How to create a Linux cluster for running physics simulations in java?

大憨熊 提交于 2019-12-03 07:52:35
问题 I am developing a scientific application used to perform physical simulations. The algorithms used are O(n3), so for a large set of data it takes a very long time to process. The application runs a simulation in around 17 minutes, and I have to run around 25,000 simulations. That is around one year of processing time. The good news is that the simulations are completely independent from each other, so I can easily change the program to distribute the work among multiple computers. There are

Python compute cluster

随声附和 提交于 2019-12-03 07:49:54
问题 Would it be possible to make a python cluster, by writing a telnet server, then telnet-ing the commands and output back-and-forth? Has anyone got a better idea for a python compute cluster? PS. Preferably for python 3.x, if anyone knows how. 回答1: The Python wiki hosts a very comprehensive list of Python cluster computing libraries and tools. You might be especially interested in Parallel Python. Edit: There is a new library that is IMHO especially good at clustering: execnet. It is small and

Sharing a Java synchronized block across a cluster, or using a global lock?

三世轮回 提交于 2019-12-03 07:09:51
问题 I have some code that I want to only allow access to by one thread. I know how to accomplish this using either synchronized blocks or methods, but will this work in a clustered environment? The target environment is WebSphere 6.0, with 2 nodes in the cluster. I have a feeling that synchronized won't work, since each instance of the application on each node will have its own JVM, right? What I am trying to do here is perform some updates to database records when the system is booted. It will

How to run a Scheduled Method in a Cluster for one Node and for All?

有些话、适合烂在心里 提交于 2019-12-03 06:59:20
I have a Glassfish 3.1.2 that runs in a cluster and a EJB 3.1 application. And I need two kind of Scheduled Methods in my application: one kind that runs only once (on a single node) once a day and other kind that runs on all nodes (every 1-2 Minutes) -- They do NOT need to run a syncron way! The requirement is just that this kind run on every node. I have no clue how to start with this cluster - problem, is this possible with @Schedule (and how) or do I need something else? I faced the exact same issue (need one cluster timer and a per node timer) and came across this question. So, for anyone

Difference between pool and cluster

泪湿孤枕 提交于 2019-12-03 06:29:08
From a purest perspective, they kind of feel like identical concepts. Both manage sets of reosurces/nodes and control their access from or by external components. With a pool, you borrow and return these resources/nodes to and from the pool. With a cluster, you have a load balancer sitting in front of the resources/nodes and you hit the load balancer with a request. In both cases you have absolutely no control over which resource/node your request/borrow gets mapped to. So I pose the question: what's the fundamental difference between the "pool" pattern and a load-balanced cluster? A pool is

How to run Cron Job in Node.js application that uses cluster module?

随声附和 提交于 2019-12-03 05:46:05
问题 I'm using node-cron module for scheduling tasks in Node.js application. I also want run the application in several processes using core cluster module. Running application in several processes ends up in scheduled tasks execution in each process (e.g. if task was to send an email the email would be sent multiple times). What are the best practices/possible ways of running cron job along with cluster module? Should I create some separate process which will handle only cron job and do not

What are the scenarios for using mirroring, log shipping, replication and clustering in SQL Server

爱⌒轻易说出口 提交于 2019-12-03 05:41:32
问题 As far as i know SQL Server provides 4 techniques for better availability. I think these are the primary usage scenarios, in summary :- 1) Replication would be primarily suited for online-offline data synchronization scenarios (laptop , mobile devices, remote servers). 2) Log shipping could be used to have a failover server with manual switching, whereas 3) Database Mirroring is an automatic failover technique 4) Failover Clustering is an advanced type of database mirroring. Am i right ?