zeromq

Cross-Platform Protobuf Serialization

阅读更多关于 Cross-Platform Protobuf Serialization

问题 I have three applications that are communicating via ZeroMQ all performing different operations. The different applications are as follows: The first is a C++ application which is an engine for "hard work", this takes a Protobuf message as a request from a client, does some work, and returns a Protobuf message back to that client (or whoever is connected, Request/Reply Pattern). This uses 0MQ version 4.0.4 and using protobuf-2.6.0 where we have built the required header files ourselves, the

The proper way to scale python tornado application

阅读更多关于 The proper way to scale python tornado application

问题 I am searching for some way to scale one instance of tornado application to many. I have 5 servers and want to run at each 4 instances of application. The main issue I don't know how to resolve - is to make communication between instances in right way. I see next approaches to make it: Use memcached for sharing data. I don't think this approach is good, because much traffic would go to server with memcached. Therefore in the future there can be trafic-related issues. Open sockets between each

Majordomo broker: handling large number of connections

阅读更多关于 Majordomo broker: handling large number of connections

问题 I am using the majordomo code found here (https://github.com/zeromq/majordomo) in the following manner: Instead of using a single broker to process the requests and replies, I start two brokers such that one of them handles all the requests, and the other handles all the replies. I did some testing to see how many connections the majordomo broker can handle: num of reqs per client num of requests handled without pkt loss 1 614 (614 clients) 10 6000 (600 clients) 100 35500 (355 clients) 1000

ZeroMQ: Address in use error when re-binding socket

阅读更多关于 ZeroMQ: Address in use error when re-binding socket

问题 After binding a ZeroMQ socket to an endpoint and closing the socket, binding another socket to the same endpoint requires several attempts. The previous calls to zmq_bind up until the successful one fail with the error "Address in use" ( EADDRINUSE ). The following code demonstrates the problem: #include <cassert> #include <iostream> #include "zmq.h" int main() { void *ctx = zmq_ctx_new(); assert( ctx ); void *skt; skt = zmq_socket( ctx, ZMQ_REP ); assert( skt ); assert( zmq_bind( skt, "tcp:/

ZeroMQ: HWM on PUSH does not work

阅读更多关于 ZeroMQ: HWM on PUSH does not work

问题 I am trying to write a server/client script with a server that vents the tasks, and multiple workers that execute it. The problem is that my ventilator has so many tasks that it would fill up the memory in a heartbeat. I tried to set the HWM before it binds, but with no success. It just keeps on sending messages as soon as a worker connects, completely disregarding the HWM that was set. I also have a sink that keeps record of the tasks that were done. server.py import zmq def ventilate():

pyzmq recv_json can't decode message sent by send_json

阅读更多关于 pyzmq recv_json can't decode message sent by send_json

问题 Here is my code with the extraneous stuff stripped out: coordinator.py context = zmq.Context() socket = context.socket(zmq.ROUTER) port = socket.bind_to_random_port(ZMQ_ADDRESS) poller = zmq.Poller() poller.register(socket, zmq.POLLIN) while True: event = poller.poll(1) if not event: continue process_id, val = socket.recv_json() worker.py context = zmq.Context() socket = context.socket(zmq.DEALER) socket.connect('%s:%s' % (ZMQ_ADDRESS, kwargs['zmq_port'])) socket.send_json( (os.getpid(), True

Identifying the origin of ZMQ messages?

阅读更多关于 Identifying the origin of ZMQ messages?

问题 If I receive a message via the recv() method on a ZeroMQ (0MQ) socket... data = s.recv() ...is there any way for me to get at the value of getpeername() for the underlying socket? My goal is to identify the origin of the message in a way that does not rely on the sender to provide accurate information. I'm using ZMQ (via Python) to collect host metrics, and the address of the sender from the perspective of the receiver is a useful identifier. Or is this just a Bad Idea? 回答1: No, you cannot

How to build a project (say zeromq) as static library and linked it into my project

阅读更多关于 How to build a project (say zeromq) as static library and linked it into my project

问题 I want to use the zeroMQ in my project and I run the configure as below to build the libaray into my home folder ./configure --enable-static --disable-shared --prefix=/home/xx/out then I link my project by gcc -o myproject x.c y.c /home/xx/out/libzmq.a but there still a lot of link error like below: ../zmq/lib/libzmq.a(libzmq_la-ip.o): In function zmq::resolve_ip_interface(sockaddr_storage*, unsigned int*, char const*)': /home/sureone/share/zeromq-2.2.0/src/ip.cpp:221: undefined reference to

How to configure ZeroRPC and timeouts

阅读更多关于 How to configure ZeroRPC and timeouts

问题 I'm trying to do some simple load-testing with a ZeroRPC python server and node.js client. What I notice is that if the request takes longer than 10 seconds, I get no data back. I tried to configure no heartbeat in the python code: s = zerorpc.Server(Test(), heartbeat=None) as well as trying to configure the node.js client: new zerorpc.Client({ timeout: 60, heartbeatInterval: 60000 }), but still see the same behavior. How can I get requests taking longer than 10 seconds to return results? 回答1

阅读更多关于 ZeroMQ

ZeroMQ 建议也了解一下原作者新重新实现的nanomsg 对比 libevent libevent 代码深度剖析 libev libuv boost-asio 1 ZeroMQ概述 ZeroMQ是一种基于消息队列的多线程网络库，其对套接字类型、连接处理、帧、甚至路由的底层细节进行抽象，提供跨越多种传输协议的套接字。ZeroMQ是网络通信中新的一层，介于应用层和传输层之间（按照TCP/IP划分），其是一个可伸缩层，可并行运行，分散在分布式系统间。 2 系统架构 2.1总体架构 ZeroMQ几乎所有的I/O操作都是异步的，主线程不会被阻塞。ZeroMQ会根据用户调用zmq_init函数时传入的接口参数，创建对应数量的I/O Thread。每个I/O Thread都有与之绑定的Poller，Poller采用经典的Reactor模式实现，Poller根据不同操作系统平台使用不同的网络I/O模型（select、poll、epoll、devpoll、kequeue等）。主线程与I/O线程通过Mail Box传递消息来进行通信。Server开始监听或者Client发起连接时，在主线程中创建zmq_connecter或zmq_listener，通过Mail Box发消息的形式将其绑定到I/O线程，I/O线程会把zmq_connecter或zmq_listener添加到Poller中用以侦听读