I wanted a feedback on whether my evaluations and concerns are correct.

I have been reseaching the three, Data Distribution Service, AMQP and ZeroMQ for sometime now for building a data transport layer in a datacenter. All the three look promising, yet i encountered some blocking issues in few.

To give a context, my requirements are:

  1. Scale upto 500+ physical nodes, 1000+ publishers and subscribers.
  2. Support durable delivery of messages to take care of failing subscribers.
  3. The aggregated throughput should be north of 1 Million messages / sec

Problems with AMQP:

  1. The Broker architecture seems to be a bottleneck and central point of failure in the whole deployment setup. I can complicate my deployment by putting a federation and cluster to improve performance and availability of pending messages, but they still do not seem to be failproof.
  2. The performance for durable queues seems to be very less. My sample application could only clock 6-7K messages / core / queue/ application.

Problems with ZeroMQ:

  1. The Documentation seems to be a little wanting in depth.
  2. The behavior of the system for pending messages seems to invite trouble in PUB/SUB model of communication. Please refer to : How zeromq handles slow consumers with PUB/SUB mode

OpenSplice DDS: I didnt find anything lacking in the DDS protocol except for adoption in the industry. Will like to know a first hand review on this product in terms of stability, performance or limitations.

amqp zeromq data-distribution-service
Edit
Report