distributed

Anatomy of a Distributed System in PHP

我只是一个虾纸丫 提交于 2019-11-28 15:49:11
I've a problem which is giving me some hard time trying to figure it out the ideal solution and, to better explain it, I'm going to expose my scenario here. I've a server that will receive orders from several clients. Each client will submit a set of recurring tasks that should be executed at some specified intervals, eg.: client A submits task AA that should be executed every minute between 2009-12-31 and 2010-12-31 ; so if my math is right that's about 525 600 operations in a year, given more clients and tasks it would be infeasible to let the server process all these tasks so I came up with

What are some scenarios for which MPI is a better fit than MapReduce?

孤人 提交于 2019-11-28 13:51:07
问题 As far as I understand, MPI gives me much more control over how exactly different nodes in the cluster will communicate. In MapReduce/Hadoop, each node does some computation, exchanges data with other nodes, and then collates its partition of results. Seems simple, but since you can iterate the process, even algorithms like K-means or PageRank fit the model quite well. On a distributed file system with locality of scheduling, the performance is apparently good. In comparison, MPI gives me

How to copy a local Git branch to a remote repo

删除回忆录丶 提交于 2019-11-28 13:47:03
问题 I've taken the following steps so far: Cloned a remote Git repo Branched the master branch to an experimental edited/tested/committed code in the experimental branch Now, I'm not ready to merge experimental into master. I do however want to push it back to the remote repo as that's the repository I share with a few colleagues. I'd like for them to see what I've done in the experimental branch. I typically just access the remote repo via SSH. How do I share my local branch on the remote repo,

how to run tensorflow distributed mnist example

不羁岁月 提交于 2019-11-28 08:31:11
I am new to distributed tensorflow. I found this distributed mnist test in here: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/dist_test/python/mnist_replica.py But I don't know how to make it run. I used the following script: python distributed_mnist.py --num_workers=3 --num_parameter_servers=1 --worker_index=0 --worker_grpc_url="grpc://tf-worker0:2222"\ & python distributed_mnist.py --num_workers=3 --num_parameter_servers=1 --worker_index=1 --worker_grpc_url="grpc://tf-worker1:2222"\ & python distributed_mnist.py --num_workers=3 --num_parameter_servers=1 --worker

calculate object delta

半腔热情 提交于 2019-11-28 05:11:27
问题 I am working on an application where client and server share an object model, and the object graphs can become rather big. To save an object from client to server, ideally i would like to send only the difference over the wire, to minimize network traffic. I can pull the original object graph on the server and apply the delta to it Wondering if there are any tools or projects out there or if anyone has had any experience with doing such a thing .. many thanks 回答1: At a previous job, we had

Sync nightmare - is it possible to use Merge Replication (or RDA) between 2 SQL CE instances without IIS?

瘦欲@ 提交于 2019-11-28 04:19:52
问题 We are faced with the following problem that involves keeping the following in sync: 1 Centralised Server (IIS / MSSQL 2005) Many Desktop WPF clients distributed by ClickOnce Many Mobile clients - (Windows CE) alt text http://img502.imageshack.us/img502/8246/deployment.png With these thorny constraints: all sync relationships are bi-directional the desktop & mobile nodes require offline mode the mobile nodes cannot sync with the central server but are to sync with the desktop nodes over USB.

In Apache Kafka why can't there be more consumer instances than partitions?

戏子无情 提交于 2019-11-28 03:47:28
I'm learning about Kafka, reading the introduction section here https://kafka.apache.org/documentation.html#introduction specifically the portion about Consumers. In the second to last paragraph in the Introduction it reads Kafka does it better. By having a notion of parallelism—the partition—within the topics, Kafka is able to provide both ordering guarantees and load balancing over a pool of consumer processes. This is achieved by assigning the partitions in the topic to the consumers in the consumer group so that each partition is consumed by exactly one consumer in the group. By doing this

Implementation of model parallelism in tensorflow

随声附和 提交于 2019-11-27 21:53:28
问题 I am a beginner to tensorflow. I'm currently working on a system with 2 GPUs each of 12GB. I want to implement model parallelism across the two GPUs to train large models. I have been looking through all over the internet, SO, tensorflow documentation, etc, i was able to find the explanations of model parallelism and its results but nowhere did i find a small tutorial or small code snippets on how to implement it using tensorflow. I mean we have to exchange activations after every layer right

Best way to aggregate multiple log files from several servers [closed]

a 夏天 提交于 2019-11-27 17:42:15
I need a simple way to monitor multiple text log files distributed over a number of HP-UX servers. They are a mix of text and XML log files from several distributed legacy systems. Currently we just ssh to the servers and use tail -f and grep , but that doesn't scale when you have many logs to keep track of. Since the logs are in different formats and just files in folders (automatically rotated when they reach a certain size) I need to both collect them remotely and parse each one differently. My initial thought was to make a simple daemon process that I can run on each server using a custom

Web Services vs EJB vs RMI, advantages and disadvantages?

牧云@^-^@ 提交于 2019-11-27 16:45:15
My web server would be overloaded quickly if all the work were done there. I'm going to stand up a second server behind it, to process data. What's the advantage of EJB over RMI, or vice versa? What about web services (SOAP, REST)? duffymo EJBs are built on top of RMI. Both imply Java clients and beans. If your clients need to be written in something else (e.g., .NET, PHP, etc.) go with web services or something else that speaks a platform-agnostic wire protocol, like HTTP or XML over HTTP or SOAP. If you choose RMI, you don't need a Java EE EJB app server. You have to keep client and server