ZeroMQ - Emulating standard socket for multiple clients to one server

问题

I am hoping to utilise ZeroMQ in order to handle queuing of a large number of requests coming in at the same time to an attestation server.

I am in an HPC environment and so have a large number of compute nodes all running the same launcher program that attests with a server. This server code runs on a front-end node and after successfully attesting the client, will then release a key in order for the client to decrypt job data.

Currently standard sockets are used. When a client initially sends something to the server a new socket is spawned using accept() from sys/socket.h. This allows the client and server to send multiple messages between each other in the attestation process (checksums etc) before finally returning the key if successful.

The issue with ZeroMQ is that the attach command is not necessary and so a secondary socket for that specific attestation is not created. All the messages are dealt with from all the clients in whatever order they come in, leading to the multi-part attestation process not working. I have spent ages going through the guide and googling to try and find a similar solution but have not had any luck so far.

Is there a way I can utilise ZeroMQ to give the same behaviour in this application as a standard socket?

回答1:

Q : Is there a way I can utilise ZeroMQ to give the same behaviour in this application as a standard socket?

ZeroMQ is a very smart and rather behaviour-oriented signaling / messaging-platform for distributed-systems, not a socket.

Given your intentions are to be worth for HPC-ecosystem, a solution postulated / directed to use some tool and a must to bend it as much as possible, so as it will become close to resemble a behaviour that is native, but for other tool, does not seem to be a typical HPC-grade approach.

HPC-code is typically very well-crafted so as to become computing-costs-efficient ( and bless the Boss, CFO & gov/mil-funding for all those, who are today permitted not to design the HPC-code for the ultimate performance and hardware-resources' use efficiency :o) ) - here, if one pays expenses on ZeroMQ instantiations, there seems no benefit to come from these non-zero costs of instantiations and getting "just"-a-socket-alike behaviour, at cost, has negative performance yield, without any adjustments in some future benefits from smart, cluster-wide ZeroMQ services ( be it an N+1 or N+M redundancy, low-latency smart inter-node cluster signaling, cryptography, cheap security-motivated white-listing, or anything that may represent any additional HPC-grade Project's benefit, that may justify the costs of the initial ZeroMQ instantiation ).

---

Defined archetype of ZMQ_STREAM may provide some tools, yet, ref. above

A socket of type ZMQ_STREAM is used to send and receive TCP data from a non-ØMQ peer, when using the tcp:// transport. A ZMQ_STREAM socket can act as client and/or server, sending and/or receiving TCP data asynchronously.

When receiving TCP data, a ZMQ_STREAM socket shall prepend a message part containing the identity of the originating peer to the message before passing it to the application. Messages received are fair-queued from among all connected peers.

When sending TCP data, a ZMQ_STREAM socket shall remove the first part of the message and use it to determine the identity of the peer the message shall be routed to, and unroutable messages shall cause an EHOSTUNREACH or EAGAIN error.

To open a connection to a server, use the zmq_connect call, and then fetch the socket identity using the ZMQ_IDENTITY zmq_getsockopt call.

To close a specific connection, send the identity frame followed by a zero-length message (see EXAMPLE section).

When a connection is made, a zero-length message will be received by the application. Similarly, when the peer disconnects (or the connection is lost), a zero-length message will be received by the application.

You must send one identity frame followed by one data frame. The ZMQ_SNDMORE flag is required for identity frames but is ignored on data frames.

ZMQ_STREAM Example:

void *ctx = zmq_ctx_new ();                          assert (ctx     && "Context Instantiation Failed..." );
void *socket = zmq_socket (ctx, ZMQ_STREAM);         assert (socket  && "socket Instantiation Failed..." );
int rc = zmq_bind (socket, "tcp://*:8080");          assert (rc == 0 && "socket.bind() Failed..." );
uint8_t id [256];                                 /* Data structure to hold the ZMQ_STREAM ID */
size_t id_size = 256;
uint8_t raw [256];                                /* Data structure to hold the ZMQ_STREAM received data */
size_t raw_size = 256;
while (1) {
   id_size = zmq_recv (socket, id, 256, 0);          assert (id_size > 0 && "Get HTTP request; ID frame and then request; Failed..." )
   do {
        raw_size = zmq_recv (socket, raw, 256, 0);   assert (raw_size >= 0 && "socket.recv() Failed..." );
   } while (raw_size == 256);

   char http_response [] =                        /* Prepares the response */
            "HTTP/1.0 200 OK\r\n"
            "Content-Type: text/plain\r\n"
            "\r\n"
            "Hello, World!";
   zmq_send (socket, id, id_size, ZMQ_SNDMORE);   /* Sends the ID frame followed by the response */
   zmq_send (socket, http_response, strlen (http_response), 0);
   zmq_send (socket, id, id_size, ZMQ_SNDMORE);   /* Closes the connection by sending the ID frame followed by a zero response */
   zmq_send (socket, 0, 0, 0);
}
zmq_close (socket);
zmq_ctx_destroy (ctx);

---

ZeroMQ zmq_getsockopt() can deliver a POSIX/SOCKET descriptor, for low-level tricks

The ZMQ_FD option shall retrieve the file descriptor associated with the specified socket. The returned file descriptor can be used to integrate the socket into an existing event loop; the ØMQ library shall signal any pending events on the socket in an edge-triggered fashion by making the file descriptor become ready for reading.

The ability to read from the returned file descriptor does not necessarily indicate that messages are available to be read from, or can be written to, the underlying socket; applications must retrieve the actual event state with a subsequent retrieval of the ZMQ_EVENTS option.

The returned file descriptor is also used internally by the zmq_send and zmq_recv functions. As the descriptor is edge triggered, applications must update the state of ZMQ_EVENTS after each invocation of zmq_send or zmq_recv.

To be more explicit: after calling zmq_send the socket may become readable (and vice versa) without triggering a read event on the file descriptor.

The returned file descriptor is intended for use with a poll or similar system call only. Applications must never attempt to read or write data to it directly, neither should they try to close it.

Option value type: int on POSIX systems, SOCKET on Windows

---

For more details on ZeroMQ tricks one may enjoy to read solutions, performance benchmarks, latency-shaving details and other problem-solving tricks that have already been discussed here.

来源：https://stackoverflow.com/questions/57588005/zeromq-emulating-standard-socket-for-multiple-clients-to-one-server