IoT request response protocol

问题

We need to build a server that can communicate with some embedded devices running a variant of Android. We need to be able to send commands to the device, and receive a response. A simple command might be asking the device for it's status. We won't have HTTP, so we need to have the client/device establish a connection with the server.

We were considering using MQTT as it has a lot of nice properties (QoS, lightweight, built for IoT), but it doesn't natively support a request response workflow.

We have considered building RPC on top of MQTT, but before we do I just wanted peoples thoughts on the matter. Would Websockets, WAMP, ZeroMQ be a better approach?

Edit:

Q1: Do we even need RPC?

Q2: Is there an approach to building systems where I always send async type messages and still provide a good user experience?

Q3: Any examples?

Looking for implementation examples and hands on experience of building an IoT communication system beyond a toy example with a single device.

回答1:

^{"one-size-fits-all" may sound as a "smart" slogan for T-shirts
but causes nightmare for ex-post attempts
to fix poorly designed architectures
once real-world implementations scale}
_{"right-sizing" and "Minimum-Viable-Product" strategies for just-enough designs have much better chance to survive IoT scales and to keep costs-of-adaptation acceptable ( take just the scales of the recent VW global device firmware update, expected to have about -2.5% to -3.0% GDP adverse impacts on Germany and automotive supply chains in Hungary and former Czechoslovakia regions - Yes, co$t$ matter in IoT domain more than just the trivial count$.)}

A smart-fit tool for IoT domain-specific architecture is a must

A first thing that ought to be born in mind is the fact, that IoT domain is by several orders of magnitude different from scales of the classical legacy computing architectures. Minimised local-resources ( by design, also mentioned above ), massive scales/counts with uncontrolled concurrency, immense synchronisation complications for true parallelism ( if such system design is needed ), ref.: a PARALLEL v/s CONCURRENT SEQUENTIAL Disambiguation Link.

Thus a proper selection of tools is needed in context with this given state.

While AMQP and other power-MQ tools are great for broker-based ( if well designed, the central MQ-broker need not be a single-point of failure & remains "just" a performance bottleneck ) the overheads for architectures with IoT-devices are to be carefully validated, whether feasible.

Broker-less ZeroCopy, ZeroSharing, ZeroBlocking, ZeroLatency_(...almost)

While AMQP has opened doors for the broker-less powers of well known ZeroMQ, the same happened another step further when Martin SUSTRIK redefined the rules and came with nanomsg.

nanomsg, besides it's portability and light-weight-ness or a just enough right-weight-ness sets itself a good candidate ready for IoT models of co-operation, giving your Project much more than the asked REQ/REP _{where needed} -- more advanced behaviours, alike SURVEY _{one asks, all vote}
BUS_{decentralised routing}
or PIPE _{a directed, one-way pipe} are particularly attractive in distributed process compositions in massive sensoric networks and a lovely example

Answers for ad-hoc added Questions:

A1: Yes, if design architecture requires, RPC might be using the same uniform signalling framework ( not reinventing wheel or adding just-another-distributed layer just for Remote Proceducer Call

A2: Yes, ZeroMQ and similar broker-less almost Zero-Latency nanomsg framework from Martin SUSTRIK are a good fit for inter-process messaging/signalling services. Your top-level design decides, whether these powers get harnessed anywhere near to their ^{(awfully magnific)} full potential or wasted into underperforming usage-patterns. To have an idea of their limits, FOREX event-streams execute spurious blasts of event with less than microsecond resolution time-stamping. There you really need a framework, that is robust _{( to handle such blasts )}, fast ( not to add unnecessary delays ), elastically linear-scaleable ( with inner abilities to handle load-balancing on demand in many-folds ). After hands-on experience I can confirm that my own team's creativity ( while highly appreciated and field-tested with many decades of successfull project achievements on the list ) is the very limiting factor for user-experience, not the ZeroMQ / nanomsg smart-frameworks.

A3: Yes, for a few years already using ZeroMQ ( DLL/LIB-adaptations are currently in progress for a nanomsg port ) for remote (load-balanced) central logging ( soft-realtime minimum latency-motivated, off-loading of distributed agents' capabilities ). Unless your system span grows into space ( where round-trip latencies are easily in minutes-hours ) this modus operandi is both smart & close to "just-enough"-design ideals.

回答2:

Based on your requirement of a light weight request/response protocol for IoT, CoAP (http://coap.technology/), an IETF standard, might be useful. It's light weight, and you can build RESTful services on top of it.

The other thing worth to consider is the "data model" and "service interfaces" for your server. Choosing a standard-based communication protocol, such as HTTP, MQTT, CoAP, is important, but it might be equally important to choose standard-based interoperable sensor data model and interfaces, so that your application can be interoperable and don't need to worry it becomes obsolete soon. Open Geospatial Consortium (OGC) SensorThings API (http://ogc-iot.github.io/ogc-iot-api/) might be an option to consider. It is an open standard, and it's data model is based on ISO 19156 Observation and Measurement.

回答3:

I could suggest to use AMQP if one of your requirements is request/response pattern. The AMQP protocol supports this pattern natively with a "correlation" mechanism between the request end the response. In your environment you could try to use the Apache Qpid Proton in C of eventually all the available language bindings like Java (for you Android based system).

回答4:

For those already using MQTT communications and want to have request/response over their service you can try replyer (https://github.com/netbeast/replyer), which is a strategy over MQTT packet structure and protocol, rather than a new one.

回答5:

I'd advice not to create your own protocol, but use LoraWAN protocol, which already contains those join/accept (the same as request/response) protocols.

Here's spec of LoraWAN protocol - page 47 describes join/accept.

回答6:

Basically, rpc and message passing are functionally equivalent as I believe was formally proved by Prof Needham in Cambridge back in the 70's. As you say, MQTT has some nice transport properties designed to help with small footprint, intermittently connected devices.

The point about RPC is that is enables a synchronous, single thread style of programming. However, if you are using Android, it's kind of unlikely that you will really be prepared for a UI to synchronously wait for an RPC to complete. Therefore, my personal opinion is that I find it easier to use a straight messaging system, such as MQTT, and track the state of the transaction however you want, (state machine, state variable, whatever).

As far as non-toy examples of MQTT based UI, you could checkout our platform http://www.thingstud.io. With MQTT multiple devices are a non-issue, as the UI is not even aware if it is talking to one device or many.

Mike

回答7:

Can't speak to the other protocols but MQTT does have some features that you may want to look into:

If you are just trying to figure out whether a device is connected or not, you can use a feature called 'last will' to send a pre-determined message on timeout or disconnect. Using that and Quality-of-service levels you should be able to keep track of the device state enough to know whether your messages are being received or not, and then monitor the publishing channels from the devices to process the responses.