Pub/Sub and Content-Based Subscriptions

自作多情 提交于 2019-12-10 10:49:45

问题


I am in the beginning phases of designing a system that will be event-driven architecture (basic pub/sub). The actual tools and frameworks have not been selected yet, so this question is more conceptual than technology specific (though it will be done in .Net).

A nearly perfect analogy for the situation is a financial trading system. Imagine a server that is constantly receiving market data (in real time or at intervals--it doesn't matter). The server will publish pricing updates for specific securities. For the purpose of this question, let's say there are 1000 securities that the server is "watching" and publishing.

There are 100 client applications that are subscribers to this service. Each client would only be interested in a small subset of the securities that are being published.

If the client receives messages for all 1000 securities, it then has to parse them, which would obviously create more network traffic and more load on the client.

On the other hand, having to parse the message content on the server side would seem to add load when publishing/routing.

How is this normally handled in this type of architecture?


回答1:


Take a look at some of the tutorials offered by RabbitMQ

Link

Notice specifically that you can use routing and topics to deliver messages to applicable clients. The "Server" "here could simply drop the messages on the queue and the clients could simply be listening for certain messages based on the routing / topics they have subscribed to. There are probably a lot of other options, but this appears to be a good route to go. RabbitMQ has a few .Net clients available and I have been very happy with their product.

I have also used IBM MQ and MSMQ, but I like Rabbit a lot better. Mainly because it is based on AMQP which is an open protocol and lets me use .Net, Python, Java, etc to connect instead of proprietary driver or trying to wrap everything in JMS. By no means an expert so maybe some others have some better insight into your problem.

Good Luck.




回答2:


Generally Servers are more powerful machines capable of doing heavy tasks. As the client only need the subset of information, in my opinion Server should parse the information and only send to client information they need.




回答3:


Distributing stock quotes is exactly the use-case that led to patenting of publish/subscribe addressing which we now know and love as "topics" in JMS and other messaging technologies (patent EP0412232).

In this case the server would publish stock quotes ("ticks") on a subject containing the ticker symbol...generally you'd use a subject namespace which gave some hierarchical information about the symbol - e.g. NASDAQ.TECHNOLOGY.HARDWARE.AAPL.

Clients would subscribe to a symbol or group of symbols using subject wildcards. The pub/sub mechanism would ensure distribution to only the right subscribers. Use of multicast networking can make this very efficient.

So to answer your question, in this case the publisher (server) is performing whatever logic is required to categorise the message and then publish on the relevant topic(s). The subscriber doesn't have to do much work in order to pick up the relevant messages.




回答4:


From the short description you give, your scenario looks like a good fit for an OMG DDS (Data Distribution Service) implementation. This is a language neutral specification, and there are several products available that offer a C# API.

See this Wikipedia entry for a very brief introduction and a list of references.

DDS supports many advanced data management features like a strong-typed and content aware databus, distributed state management and historical data access. Its rich set of Quality of Service settings allows to off-load a lot of the complexity from your applications to the middleware.

In particular, the content filtering requirement that you describe is an important feature. Some of the available products have implemented sophisticated filtering mechanisms that scale well and have smart trade-offs between the amount of traffic on the network versus the CPU load required to evaluate the filters. Those filters are expressed in SQL syntax, like the WHERE clause of a SELECT.

DDS implementations typically are highly scalable and decentralized in nature. Components participating in a DDS infrastructure are decoupled, both in space and in time. Some DDS products are deployed in numerous mission- and business-critical systems.

If you are interested, then I can give you much more information. I have specialized in DDS for more than 10 years, I still like it and I think it is one of the most useful technologies around.



来源:https://stackoverflow.com/questions/11803542/pub-sub-and-content-based-subscriptions

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!