Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
GPT-4o
Gemini 2.5 Pro Pro
o3 Pro
GPT-4.1 Pro
DeepSeek R1 via Azure Pro
2000 character limit reached

Towards Message Brokers for Generative AI: Survey, Challenges, and Opportunities (2312.14647v2)

Published 22 Dec 2023 in cs.DC, cs.AI, cs.LG, and cs.NI

Abstract: In today's digital world, Generative Artificial Intelligence (GenAI) such as LLMs is becoming increasingly prevalent, extending its reach across diverse applications. This surge in adoption has sparked a significant increase in demand for data-centric GenAI models, highlighting the necessity for robust data communication infrastructures. Central to this need are message brokers, which serve as essential channels for data transfer within various system components. This survey aims to delve into a comprehensive analysis of traditional and modern message brokers, offering a comparative study of prevalent platforms. Our study considers numerous criteria including, but not limited to, open-source availability, integrated monitoring tools, message prioritization mechanisms, capabilities for parallel processing, reliability, distribution and clustering functionalities, authentication processes, data persistence strategies, fault tolerance, and scalability. Furthermore, we explore the intrinsic constraints that the design and operation of each message broker might impose, recognizing that these limitations are crucial in understanding their real-world applicability. Finally, this study examines the enhancement of message broker mechanisms specifically for GenAI contexts, emphasizing the criticality of developing a versatile message broker framework. Such a framework would be poised for quick adaptation, catering to the dynamic and growing demands of GenAI in the foreseeable future. Through this dual-pronged approach, we intend to contribute a foundational compendium that can guide future innovations and infrastructural advancements in the realm of GenAI data communication.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (277)
  1. L. Bariah, Q. Zhao, H. Zou, Y. Tian, F. Bader, and M. Debbah, “Large language models for telecom: The next big thing?” arXiv preprint arXiv:2306.10249, 2023.
  2. H. Kokkonen, L. Lovén, N. H. Motlagh, A. Kumar, J. Partala, T. Nguyen, V. C. Pujol, P. Kostakos, T. Leppänen, A. González-Gil et al., “Autonomy and intelligence in the computing continuum: Challenges, enablers, and future directions for orchestration,” arXiv preprint arXiv:2205.01423, 2022.
  3. N. H. Motlagh, L. Lovén, J. Cao, X. Liu, P. Nurmi, S. Dustdar, S. Tarkoma, and X. Su, “Edge computing: The computing infrastructure for the smart megacities of the future,” Computer, vol. 55, no. 12, pp. 54–64, 2022.
  4. Y.-C. Wang, J. Xue, C. Wei, and C.-C. J. Kuo, “An overview on generative ai at scale with edge-cloud computing,” 2023.
  5. S. Dustdar, V. C. Pujol, and P. K. Donta, “On distributed computing continuum systems,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 4, pp. 4092–4105, 2022.
  6. P. K. Donta, I. Murturi, V. Casamayor Pujol, B. Sedlak, and S. Dustdar, “Exploring the potential of distributed computing continuum systems,” Computers, vol. 12, no. 10, 2023. [Online]. Available: https://www.mdpi.com/2073-431X/12/10/198
  7. P. T. Eugster, P. A. Felber, R. Guerraoui, and A.-M. Kermarrec, “The many faces of publish/subscribe,” ACM computing surveys (CSUR), vol. 35, no. 2, pp. 114–131, 2003.
  8. P. K. Donta, S. N. Srirama, T. Amgoth, and C. S. R. Annavarapu, “Survey on recent advances in iot application layer protocols and machine learning scope for research directions,” Digital Communications and Networks, vol. 8, no. 5, pp. 727–744, 2022.
  9. J. Hasenburg, F. Stanek, F. Tschorsch, and D. Bermbach, “Managing latency and excess data dissemination in fog-based publish/subscribe systems,” in 2020 IEEE international conference on fog computing (ICFC).   IEEE, 2020, pp. 9–16.
  10. L. Lovén, R. Morabito, A. Kumar, S. Pirttikangas, J. Riekki, and S. Tarkoma, “How can ai be distributed in the computing continuum? introducing the neural pub/sub paradigm,” arXiv preprint arXiv:2309.02058, 2023.
  11. F. Z. Chafi, Y. Fakhri, and F. Z. A. H. Aadi, “Introduction to internet of things’ communication protocols,” in Advanced Intelligent Systems for Sustainable Development (AI2SD’2020) Volume 2.   Springer, 2022, pp. 142–150.
  12. V. Maniezzo, M. A. Boschetti, and P. Manzoni, “Self-adaptive publish/subscribe network design,” in Metaheuristics: 14th International Conference, MIC 2022, Syracuse, Italy, July 11–14, 2022, Proceedings.   Springer, 2023, pp. 478–484.
  13. F. Pedrosa and L. Rodrigues, “Reducing the subscription latency in reliable causal publish-subscribe systems,” in Proceedings of the 36th Annual ACM Symposium on Applied Computing, 2021, pp. 203–212.
  14. V. John and X. Liu, “A survey of distributed message broker queues,” arXiv preprint arXiv:1704.00411, 2017.
  15. O. Iakushkin and V. Grishkin, “Messaging middleware for cloud applications: Extending brokerless approach,” in 2014 2nd 2014 2nd International Conference on Emission Electronics (ICEE).   IEEE, 2014, pp. 1–4.
  16. T. A. S. Foundation, “Apache activemq,” https://activemq.apache.org/, last accessed: December 22, 2023.
  17. R. Hat, “Fuse message broker,” https://access.redhat.com/taxonomy/products/fuse-message-broker, last accessed: December 22, 2023.
  18. A. S. Foundation, “Apache qpid,” https://qpid.apache.org/, last accessed: December 22, 2023.
  19. R. Technologies, “Rabbitmq,” https://www.rabbitmq.com/, last accessed: December 22, 2023.
  20. R. Hat, “Hornetq,” https://hornetq.jboss.org/, last accessed: December 22, 2023.
  21. I. Red Hat, “Red hat amq,” https://www.redhat.com/en/technologies/jboss-middleware/amq, last accessed: December 22, 2023.
  22. C. software, “Celery,” https://docs.celeryq.dev/, last accessed: December 22, 2023.
  23. R. Hat, “Jboss messaging,” https://jbossmessaging.jboss.org/, last accessed: December 22, 2023.
  24. Oracle, “Openmq,” https://javaee.github.io/openmq/, last accessed: December 22, 2023.
  25. I. Philotic, “Beanstalk,” https://beanstalkd.github.io/, last accessed: December 22, 2023.
  26. Gearman, “Gearman,” http://gearman.org/, last accessed: December 22, 2023.
  27. Mavimax, “Enduro/x,” https://www.endurox.org/, last accessed: December 22, 2023.
  28. P. Fremantle, “Wso2,” https://wso2.com/, last accessed: December 22, 2023.
  29. HiveMQ, “Hivemq,” https://www.hivemq.com/, last accessed: December 22, 2023.
  30. R. Labs, “Redis,” https://redis.io/, last accessed: December 22, 2023.
  31. E. Technologies, “Emqx,” https://www.emqx.io/, last accessed: December 22, 2023.
  32. T. A. S. Foundation, “Apache pulsar,” https://pulsar.apache.org/, last accessed: December 22, 2023.
  33. A. S. Foundation, “Apache kafka,” https://kafka.apache.org/, last accessed: December 22, 2023.
  34. A. Group, “Apache rocketmq,” https://rocketmq.apache.org/, last accessed: December 22, 2023.
  35. Eclipse, “Eclipse mosquitto,” https://mosquitto.org/, last accessed: December 22, 2023.
  36. iMatix, “Zero mq,” https://zeromq.org/, last accessed: December 22, 2023.
  37. I. Onyara, “Apache nifi,” https://nifi.apache.org/, last accessed: December 22, 2023.
  38. A. R. LTD, “Ably realtime,” https://ably.com/, last accessed: December 22, 2023.
  39. A. S. Foundation, “Apache samza,” https://samza.apache.org/, last accessed: December 22, 2023.
  40. VerneMQ, “Vernemq,” https://vernemq.com/, last accessed: December 22, 2023.
  41. P. Software, “Nservicebus,” https://particular.net/nservicebus, last accessed: December 22, 2023.
  42. Twitter, “kestrel,” https://github.com/twitter-archive/kestrel, last accessed: December 22, 2023.
  43. Bitly, “Nsq,” https://nsq.io/, last accessed: December 22, 2023.
  44. S. Communications, “Nats,” https://nats.io/, last accessed: December 22, 2023.
  45. KubeMQ, “Kubemq,” https://kubemq.io/, last accessed: December 22, 2023.
  46. IBM, “Ibm mq,” https://www.ibm.com/docs/en/ibm-mq, last accessed: December 22, 2023.
  47. A. W. S. (AWS), “Amazon sqs,” https://aws.amazon.com/sqs/, last accessed: December 22, 2023.
  48. Microsoft, “Msmq,” https://learn.microsoft.com/en-us/previous-versions/windows/desktop/msmq/, last accessed: December 22, 2023.
  49. Oracle, “Oracle message broker,” https://docs.oracle.com/cd/E26576_01/doc.312/e24948.pdf, last accessed: December 22, 2023.
  50. TIBCO, “Tibco rendezvous,” https://www.tibco.com/products/tibco-rendezvous, last accessed: December 22, 2023.
  51. ——, “Tibco enterprise message service™,” https://www.tibco.com/products/tibco-enterprise-message-service, last accessed: December 22, 2023.
  52. MuleSoft, “Anypoint mq,” https://docs.mulesoft.com/mq/, last accessed: December 22, 2023.
  53. Microsoft, “Azure service bus,” https://learn.microsoft.com/en-us/azure/service-bus-messaging/, last accessed: December 22, 2023.
  54. S. AG, “Sap netweaver process integration,” https://help.sap.com/docs/, last accessed: December 22, 2023.
  55. C. Betts, “Solace,” https://solace.com/, last accessed: December 22, 2023.
  56. Google, “Google cloud pub/sub,” https://cloud.google.com/pubsub, last accessed: December 22, 2023.
  57. A. W. S. (AWS), “Amazon mq,” https://aws.amazon.com/amazon-mq/, last accessed: December 22, 2023.
  58. Intel, “Intel mpi library,” https://www.intel.com/content/www/us/en/developer/tools/oneapi/mpi-library.html, last accessed: December 22, 2023.
  59. Amazon, “Kinesis,” https://aws.amazon.com/kinesis/, last accessed: December 22, 2023.
  60. Microsoft, “Azure storage queue,” https://learn.microsoft.com/en-us/azure/storage/queues/storage-queues-introduction, last accessed: December 22, 2023.
  61. Iron.io, “Ironmq,” http://www.iron.io/mq, last accessed: December 22, 2023.
  62. T. A. S. Foundation, “Apache activemq per destination policies,” https://activemq.apache.org/clustering, last accessed: December 22, 2023.
  63. Oracle, “Open message queue, administration guide, release 5.0,” https://javaee.github.io/glassfish/doc/4.0/mq-admin-guide.pdf, 2013.
  64. IBM, “Ibm mq,” https://www.ibm.com/topics/message-brokers, last accessed: December 22, 2023.
  65. A. S. Foundation, “Apache kafka,” https://kafka.apache.org/documentation/#design_pull, last accessed: December 22, 2023.
  66. Oracle, “Oracle message broker,” https://docs.oracle.com/cd/E19316-01/820-6424/aerbz/index.html, last accessed: December 22, 2023.
  67. ——, “Oracle message broker,” https://docs.oracle.com/cd/E19879-01/821-0028/aercs/index.html, last accessed: December 22, 2023.
  68. ——, “Oracle message broker,” https://docs.oracle.com/cd/E19879-01/821-0028/aercv/index.html, last accessed: December 22, 2023.
  69. I. Red Hat, “Red hat amq,” https://access.redhat.com/documentation/en-us/red_hat_amq/6.1/html/product_introduction/fmbscalable, last accessed: December 22, 2023.
  70. OpenAI, “Gpt-4 technical report,” 2023.
  71. F. García-Peñalvo and A. Vázquez-Ingelmo, “What do we mean by genai? a systematic mapping of the evolution, trends, and techniques involved in generative ai,” 2023.
  72. M. Gupta, C. Akiri, K. Aryal, E. Parker, and L. Praharaj, “From chatgpt to threatgpt: Impact of generative ai in cybersecurity and privacy,” IEEE Access, vol. 11, pp. 80 218–80 245, 2023.
  73. R. Stuart and N. Peter, “Artificial intelligence a modern approach third edition,” 2010.
  74. Z. Xi, W. Chen, X. Guo, W. He, Y. Ding, B. Hong, M. Zhang, J. Wang, S. Jin, E. Zhou et al., “The rise and potential of large language model based agents: A survey,” arXiv preprint arXiv:2309.07864, 2023.
  75. T. Gong, C. Lyu, S. Zhang, Y. Wang, M. Zheng, Q. Zhao, K. Liu, W. Zhang, P. Luo, and K. Chen, “Multimodal-gpt: A vision and language model for dialogue with humans,” arXiv preprint arXiv:2305.04790, 2023.
  76. J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds et al., “Flamingo: a visual language model for few-shot learning,” Advances in Neural Information Processing Systems, vol. 35, pp. 23 716–23 736, 2022.
  77. Y. Shen, K. Song, X. Tan, D. Li, W. Lu, and Y. Zhuang, “Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface,” arXiv preprint arXiv:2303.17580, 2023.
  78. R. Huang, M. Li, D. Yang, J. Shi, X. Chang, Z. Ye, Y. Wu, Z. Hong, J. Huang, J. Liu et al., “Audiogpt: Understanding and generating speech, music, sound, and talking head,” arXiv preprint arXiv:2304.12995, 2023.
  79. C. Wu, S. Yin, W. Qi, X. Wang, Z. Tang, and N. Duan, “Visual chatgpt: Talking, drawing and editing with visual foundation models,” arXiv preprint arXiv:2303.04671, 2023.
  80. L. Wang, C. Ma, X. Feng, Z. Zhang, H. Yang, J. Zhang, Z. Chen, J. Tang, X. Chen, Y. Lin et al., “A survey on large language model based autonomous agents,” arXiv preprint arXiv:2308.11432, 2023.
  81. J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou et al., “Chain-of-thought prompting elicits reasoning in large language models,” Advances in Neural Information Processing Systems, vol. 35, pp. 24 824–24 837, 2022.
  82. S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, “React: Synergizing reasoning and acting in language models,” arXiv preprint arXiv:2210.03629, 2022.
  83. N. Shinn, F. Cassano, A. Gopinath, K. R. Narasimhan, and S. Yao, “Reflexion: Language agents with verbal reinforcement learning,” in Thirty-seventh Conference on Neural Information Processing Systems, 2023.
  84. openai, “Chatgpt,” https://chat.openai.com/, last accessed: December 22, 2023.
  85. D. Shah, B. Osiński, S. Levine et al., “Lm-nav: Robotic navigation with large pre-trained models of language, vision, and action,” in Conference on Robot Learning.   PMLR, 2023, pp. 492–504.
  86. Y. Mu, Q. Zhang, M. Hu, W. Wang, M. Ding, J. Jin, B. Wang, J. Dai, Y. Qiao, and P. Luo, “Embodiedgpt: Vision-language pre-training via embodied chain of thought,” arXiv preprint arXiv:2305.15021, 2023.
  87. C. Qian, C. Han, Y. R. Fung, Y. Qin, Z. Liu, and H. Ji, “Creator: Disentangling abstract and concrete reasonings of large language models through tool creation,” arXiv preprint arXiv:2305.14318, 2023.
  88. X. Chen, M. Lin, N. Schärli, and D. Zhou, “Teaching large language models to self-debug,” arXiv preprint arXiv:2304.05128, 2023.
  89. J. Park, S. Samarakoon, M. Bennis, and M. Debbah, “Wireless network intelligence at the edge,” Proceedings of the IEEE, vol. 107, no. 11, pp. 2204–2239, 2019.
  90. L. Lovén, T. Leppänen, E. Peltonen, J. Partala, E. Harjula, P. Porambage, M. Ylianttila, and J. Riekki, “Edgeai: A vision for distributed, edge-native artificial intelligence in future 6g networks,” 6G Wireless Summit, March 24-26, 2019 Levi, Finland, 2019.
  91. S. Deng, H. Zhao, W. Fang, J. Yin, S. Dustdar, and A. Y. Zomaya, “Edge intelligence: The confluence of edge computing and artificial intelligence,” IEEE Internet of Things Journal, vol. 7, no. 8, pp. 7457–7469, 2020.
  92. P. Dhoni, “Exploring the synergy between generative ai, data and analytics in the modern age,” Authorea Preprints, 2023.
  93. OpenAI, “Gpt-4 system card,” https://cdn.openai.com/papers/gpt-4-system-card.pdf, last accessed: December 22, 2023.
  94. Oracle, “Oracle message broker,” https://docs.oracle.com/cd/E19879-01/821-0028/aercs/index.html, last accessed: December 22, 2023.
  95. L. Xia, Y. Sun, C. Liang, L. Zhang, M. A. Imran, and D. Niyato, “Generative ai for semantic communication: Architecture, challenges, and outlook,” arXiv preprint arXiv:2308.15483, 2023.
  96. P. K. Donta, B. Sedlak, V. Casamayor Pujol, and S. Dustdar, “Governance and sustainability of distributed continuum systems: a big data approach,” Journal of Big Data, vol. 10, no. 1, pp. 1–31, 2023.
  97. D. Oniani, J. Hilsman, Y. Peng, R. K. Poropatich, C. Pamplin, L. Legault, Y. Wang et al., “From military to healthcare: Adopting and expanding ethical principles for generative artificial intelligence,” arXiv preprint arXiv:2308.02448, 2023.
  98. A. Group, “Apache rocketmq,” https://rocketmq.apache.org/docs/4.x/, last accessed: December 22, 2023.
  99. MuleSoft, “Anypoint mq,” https://docs.mulesoft.com/mq/mq-routing-rules, last accessed: December 22, 2023.
  100. L. Zhang, Y. Zhang, K. Ren, D. Li, and Y. Yang, “Mlcopilot: Unleashing the power of large language models in solving machine learning tasks,” arXiv preprint arXiv:2304.14979, 2023.
  101. Y. Shen, J. Shao, X. Zhang, Z. Lin, H. Pan, D. Li, J. Zhang, and K. B. Letaief, “Large language models empowered autonomous edge ai for connected intelligence,” arXiv preprint arXiv:2307.02779, 2023.
  102. Google, “Google cloud pub/sub,” https://cloud.google.com/pubsub/docs/encryption, last accessed: December 22, 2023.
  103. Amazon, “Kinesis,” https://aws.amazon.com/kinesis/data-streams/faqs/, last accessed: December 22, 2023.
  104. ——, “Kinesis,” https://docs.aws.amazon.com/kinesisanalytics/latest/dev/error-handling.html, last accessed: December 22, 2023.
  105. S. Li, H. Liu, Z. Bian, J. Fang, H. Huang, Y. Liu, B. Wang, and Y. You, “Colossal-ai: A unified deep learning system for large-scale parallel training,” in Proceedings of the 52nd International Conference on Parallel Processing, 2023, pp. 766–775.
  106. S. Li, F. Xue, C. Baranwal, Y. Li, and Y. You, “Sequence parallelism: Long sequence training from system perspective,” arXiv preprint arXiv:2105.13120, 2021.
  107. V. A. Korthikanti, J. Casper, S. Lym, L. McAfee, M. Andersch, M. Shoeybi, and B. Catanzaro, “Reducing activation recomputation in large transformer models,” Proceedings of Machine Learning and Systems, vol. 5, 2023.
  108. A. Radford, K. Narasimhan, T. Salimans, I. Sutskever et al., “Improving language understanding by generative pre-training,” 2018.
  109. P. Micikevicius, S. Narang, J. Alben, G. Diamos, E. Elsen, D. Garcia, B. Ginsburg, M. Houston, O. Kuchaiev, G. Venkatesh, and H. Wu, “Mixed precision training,” 2018.
  110. Y. Jiang, S. Wang, V. Valls, B. J. Ko, W.-H. Lee, K. K. Leung, and L. Tassiulas, “Model pruning enables efficient federated learning on edge devices,” IEEE Transactions on Neural Networks and Learning Systems, 2022.
  111. X. Zhu, J. Li, Y. Liu, C. Ma, and W. Wang, “A survey on model compression for large language models,” arXiv preprint arXiv:2308.07633, 2023.
  112. A. Polino, R. Pascanu, and D. Alistarh, “Model compression via distillation and quantization,” arXiv preprint arXiv:1802.05668, 2018.
  113. Y.-C. Hsu, T. Hua, S. Chang, Q. Lou, Y. Shen, and H. Jin, “Language model compression with weighted low-rank factorization,” arXiv preprint arXiv:2207.00112, 2022.
  114. C. software, “Celery,” https://docs.celeryq.dev/en/stable/, last accessed: December 22, 2023.
  115. ——, “Celery,” https://docs.celeryq.dev/en/stable/getting-started/introduction.html, last accessed: December 22, 2023.
  116. A. W. S. (AWS), “Amazon sqs,” https://aws.amazon.com/sqs/features/, last accessed: December 22, 2023.
  117. A. S. Foundation, “Apache kafka,” https://kafka.apache.org/20/documentation.html, last accessed: December 22, 2023.
  118. Microsoft, “Azure service bus,” https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus. servicebussender.schedulemessageasync?view=azure-dotnet, last accessed: December 22, 2023.
  119. ——, “Azure service bus,” https://azure.microsoft.com/en-us/products/service-bus, last accessed: December 22, 2023.
  120. D. Kreuzberger, N. Kühl, and S. Hirschl, “Machine learning operations (mlops): Overview, definition, and architecture,” IEEE Access, vol. 11, pp. 31 866–31 879, 2023.
  121. D. Praveen Kumar, M. Ilir, C. P. Victor, S. Boris, and D. Schahram, “Exploring the potential of distributed computing continuum systems,” Computers, 2023.
  122. KubeMQ, “Kubemq,” https://kubemq.io/product-use-cases/, last accessed: December 22, 2023.
  123. IBM, “Ibm mq,” https://www.ibm.com/docs/en/ibm-mq/8.0?topic=network-message-monitoring, last accessed: December 22, 2023.
  124. C. Betts, “Solace,” https://www.solace.dev/, last accessed: December 22, 2023.
  125. A. E. C. Redondi, A. Arcia-Moret, and P. Manzoni, “Towards a scaled iot pub/sub architecture for 5g networks: the case of multiaccess edge computing,” in 2019 IEEE 5th World Forum on Internet of Things (WF-IoT), 2019, pp. 436–441.
  126. E. Peltonen, M. Bennis, M. Capobianco, M. Debbah, A. Ding, F. Gil-Castiñeira, M. Jurmu, T. Karvonen, M. Kelanti, A. Kliks et al., “6g white paper on edge intelligence,” arXiv preprint arXiv:2004.14850, 2020.
  127. E. Peltonen, I. Ahmad, A. Aral, M. Capobianco, A. Y. Ding, F. Gil-Castineira, E. Gilman, E. Harjula, M. Jurmu, T. Karvonen et al., “The many faces of edge intelligence,” IEEE Access, vol. 10, pp. 104 769–104 782, 2022.
  128. P. Porambage, T. Kumar, M. Liyanage, J. Partala, L. Lovén, M. Ylianttila, and T. Seppänen, “Sec-edgeai: Ai for edge security vs security for edge ai,” The 1st 6G Wireless Summit,(Levi, Finland), 2019.
  129. T. A. S. Foundation, “Apache activemq artemis user manual,” https://activemq.apache.org/components/artemis/documentation/2.6.0/, last accessed: December 22, 2023.
  130. ——, “Apache activemq per destination policies,” https://activemq.apache.org/per-destination-policies, last accessed: December 22, 2023.
  131. R. Hat, “Fuse message broker,” https://docs.huihoo.com/fuse/getting_started.pdf, last accessed: December 22, 2023.
  132. A. S. Foundation, “Apache qpid,” https://qpid.apache.org/releases/qpid-cpp-1.39.0/cpp-broker/book/AMQP-Compatibility.html, last accessed: December 22, 2023.
  133. ——, “Apache qpid,” https://qpid.apache.org/cves/CVE-2018-8030.html, last accessed: December 22, 2023.
  134. ——, “Apache qpid,” https://people.apache.org/~jonathan/Programming-In-Apache-Qpid.html, last accessed: December 22, 2023.
  135. ——, “Apache qpid,” https://qpid.apache.org/releases/qpid-jms-amqp-0-x-6.4.0/jms-amqp-0-8-book/, last accessed: December 22, 2023.
  136. ——, “Apache qpid,” https://qpid.apache.org/releases/qpid-cpp-1.39.0/cpp-broker/book/pr01.html, last accessed: December 22, 2023.
  137. ——, “Apache qpid,” https://qpid.apache.org/releases/qpid-broker-j-8.0.1/book/Java-Broker-Security.html, last accessed: December 22, 2023.
  138. ——, “Apache qpid,” https://qpid.apache.org/releases/qpid-broker-j-7.1.10/book/index.html, last accessed: December 22, 2023.
  139. R. Technologies, “Rabbitmq,” https://www.rabbitmq.com/erlang-client-user-guide.html, last accessed: December 22, 2023.
  140. ——, “Rabbitmq,” https://www.rabbitmq.com/memory-use.html, last accessed: December 22, 2023.
  141. ——, “Rabbitmq,” https://www.rabbitmq.com/production-checklist.html, last accessed: December 22, 2023.
  142. ——, “Rabbitmq,” https://www.rabbitmq.com/configure.html, last accessed: December 22, 2023.
  143. ——, “Rabbitmq,” https://www.rabbitmq.com/platforms.html, last accessed: December 22, 2023.
  144. ——, “Rabbitmq,” https://www.rabbitmq.com/documentation.html, last accessed: December 22, 2023.
  145. ——, “Rabbitmq,” https://www.rabbitmq.com/devtools.html, last accessed: December 22, 2023.
  146. ——, “Rabbitmq,” https://blog.rabbitmq.com/posts/2020/07/disaster-recovery-and-high-availability-101/, last accessed: December 22, 2023.
  147. ——, “Rabbitmq,” https://www.rabbitmq.com/protocols.html, last accessed: December 22, 2023.
  148. ——, “Rabbitmq,” https://www.rabbitmq.com/features.html, last accessed: December 22, 2023.
  149. ——, “Rabbitmq,” https://www.rabbitmq.com/flow-control.html, last accessed: December 22, 2023.
  150. R. Hat, “Hornetq,” https://hornetq.sourceforge.net/docs/hornetq-2.1.2.Final/user-manual/en/html_single/index.html, last accessed: December 22, 2023.
  151. ——, “Hornetq,” https://docs.jboss.org/hornetq/2.4.0.Final/docs/user-manual/html/interoperability.html, last accessed: December 22, 2023.
  152. I. Red Hat, “Red hat amq,” https://access.redhat.com/documentation/en-us/red_hat_amq/2020.q4/html/release_notes_for_red_hat_amq_broker_7.8/known, last accessed: December 22, 2023.
  153. ——, “Red hat amq,” https://access.redhat.com/documentation/en-us/red_hat_amq/7.2/html/amq_broker_7.2_release_notes/known, last accessed: December 22, 2023.
  154. C. software, “Celery,” https://docs.celeryq.dev/en/3.1/getting-started/brokers/index.html, last accessed: December 22, 2023.
  155. ——, “Celery,” https://docs.celeryq.dev/en/stable/userguide/optimizing.html, last accessed: December 22, 2023.
  156. ——, “Celery,” https://docs.celeryq.dev/en/stable/userguide/tasks.html, last accessed: December 22, 2023.
  157. R. Hat, “Jboss messaging,” https://docs.jboss.org/jbossmessaging/docs/usermanual-2.0.0.beta1/html, last accessed: December 22, 2023.
  158. ——, “Jboss messaging,” https://docs.jboss.org/jbossmessaging/docs/usermanual-2.0.0.beta1/html/messaging-concepts.html#d0e148, last accessed: December 22, 2023.
  159. ——, “Jboss messaging,” https://docs.jboss.org/jbossmessaging/docs/usermanual-2.0.0.beta1/html/messaging-concepts.html#d0e138, last accessed: December 22, 2023.
  160. ——, “Jboss messaging,” https://docs.jboss.org/jbossmessaging/docs/usermanual-2.0.0.beta1/html/paging.html#paging.main.config, last accessed: December 22, 2023.
  161. ——, “Jboss messaging,” https://docs.jboss.org/jbossmessaging/docs/usermanual-2.0.0.beta1/html/appserver-integration.html, last accessed: December 22, 2023.
  162. Oracle, “Openmq,” https://javaee.github.io/openmq/Overview.html, last accessed: December 22, 2023.
  163. I. Philotic, “Beanstalk,” https://raw.githubusercontent.com/beanstalkd/beanstalkd/master/doc/protocol.txt, last accessed: December 22, 2023.
  164. ——, “Beanstalk,” https://github.com/beanstalkd/beanstalkd/wiki/Client-Libraries, last accessed: December 22, 2023.
  165. Gearman, “Gearman,” http://gearman.org/documentation/, last accessed: December 22, 2023.
  166. ——, “Gearman,” http://gearman.org/protocol/, last accessed: December 22, 2023.
  167. ——, “Gearman,” http://gearman.org/manual/introduction, last accessed: December 22, 2023.
  168. Mavimax, “Enduro/x,” https://github.com/endurox-dev/endurox/blob/master/doc/ex_adminman.adoc, last accessed: December 22, 2023.
  169. ——, “Enduro/x,” https://github.com/endurox-dev/endurox/blob/master/doc/ex_devguide.adoc, last accessed: December 22, 2023.
  170. L. Mavimax, “Enduro/x middleware data sheet,” 2022.
  171. Mavimax, “Enduro/x,” https://www.endurox.org/projects/endurox/wiki/Version_matrix, last accessed: December 22, 2023.
  172. P. Fremantle, “Wso2,” https://ei.docs.wso2.com/en/latest/micro-integrator/setup/performance_tuning/tuning_jvm_performance/, last accessed: December 22, 2023.
  173. ——, “Wso2,” https://apim.docs.wso2.com/en/4.1.0/install-and-setup/setup/mi-setup/transport_configurations/configuring-transports/, last accessed: December 22, 2023.
  174. ——, “Wso2,” https://ei.docs.wso2.com/en/latest/micro-integrator/references/mediators/filter-Mediator/, last accessed: December 22, 2023.
  175. HiveMQ, “Hivemq,” https://docs.hivemq.com/hivemq/4.21/user-guide/restrictions, last accessed: December 22, 2023.
  176. ——, “Hivemq,” https://docs.hivemq.com/hivemq/3.4/user-guide/monitoring.html, last accessed: December 22, 2023.
  177. ——, “Hivemq,” https://www.hivemq.com/products/mqtt-broker/, last accessed: December 22, 2023.
  178. R. Labs, “Redis,” https://redis.io/docs/management/persistence/, last accessed: December 22, 2023.
  179. ——, “Redis,” https://docs.redis.com/latest/rs/security/, last accessed: December 22, 2023.
  180. E. Technologies, “Emqx,” https://www.emqx.com/en/blog/emqx-vs-mosquitto-2023-mqtt-broker-comparison, last accessed: December 22, 2023.
  181. ——, “Emqx,” https://www.emqx.io/docs/en/latest/data-integration/data-bridge-mqtt.html, last accessed: December 22, 2023.
  182. ——, “Emqx,” https://www.emqx.io/docs/en/latest/data-integration/data-bridges.html, last accessed: December 22, 2023.
  183. T. A. S. Foundation, “Apache pulsar,” https://pulsar.apache.org/docs/2.10.x/reference-configuration/, last accessed: December 22, 2023.
  184. ——, “Apache pulsar,” https://pulsar.apache.org/docs/3.1.x/concepts-architecture-overview/, last accessed: December 22, 2023.
  185. A. Group, “Apache rocketmq,” https://rocketmq.apache.org/docs/introduction/03limits/, last accessed: December 22, 2023.
  186. ——, “Apache rocketmq,” https://rocketmq.apache.org/docs/, last accessed: December 22, 2023.
  187. ——, “Apache rocketmq,” https://rocketmq.apache.org/docs/sdk/01overview/, last accessed: December 22, 2023.
  188. Eclipse, “Eclipse mosquitto,” https://mosquitto.org/security/, last accessed: December 22, 2023.
  189. ——, “Eclipse mosquitto,” https://www.emqx.com/en/blog/mosquitto-mqtt-broker-pros-cons-tutorial-and-modern-alternatives, last accessed: December 22, 2023.
  190. ——, “Eclipse mosquitto,” https://projects.eclipse.org/projects/iot.mosquitto, last accessed: December 22, 2023.
  191. ——, “Eclipse mosquitto,” https://mosquitto.org/man/mosquitto-conf-5.html, last accessed: December 22, 2023.
  192. ——, “Eclipse mosquitto,” https://mosquitto.org/man/mosquitto_sub-1.html, last accessed: December 22, 2023.
  193. iMatix, “Zero mq,” https://www.hivemq.com/article/mqtt-vs-zeromq-for-iot/, last accessed: December 22, 2023.
  194. ——, “Zero mq,” http://wiki.zeromq.org/area:faq, last accessed: December 22, 2023.
  195. ——, “Zero mq,” https://zeromq.org/get-started/, last accessed: December 22, 2023.
  196. I. Onyara, “Apache nifi,” https://nifi.apache.org/docs/nifi-docs/html/user-guide.html, last accessed: December 22, 2023.
  197. ——, “Apache nifi,” https://nifi.apache.org/docs/nifi-docs/html/nifi-in-depth.html, last accessed: December 22, 2023.
  198. ——, “Apache nifi,” https://nifi.apache.org/docs/nifi-docs/html/overview.html, last accessed: December 22, 2023.
  199. A. R. LTD, “Ably realtime,” https://ably.com/docs/general/limits, last accessed: December 22, 2023.
  200. ——, “Ably realtime,” https://ably.com/platform, last accessed: December 22, 2023.
  201. A. S. Foundation, “Apache samza,” https://samza.incubator.apache.org/learn/documentation/0.7.0/comparisons/storm.html, last accessed: December 22, 2023.
  202. M. Kleppmann, “Apache samza.” 2019.
  203. A. S. Foundation, “Apache samza,” https://samza.apache.org/learn/documentation/latest/core-concepts/core-concepts.html, last accessed: December 22, 2023.
  204. E. Technologies, “Emqx,” https://www.emqx.com/en/blog/emqx-vs-vernemq-2023-mqtt-broker-comparison, last accessed: December 22, 2023.
  205. VerneMQ, “Vernemq,” https://vernemq.com/intro/, last accessed: December 22, 2023.
  206. P. Software, “Nservicebus,” https://docs.particular.net/nservicebus/scaling, last accessed: December 22, 2023.
  207. ——, “Nservicebus,” https://docs.particular.net/tutorials/monitoring-setup/, last accessed: December 22, 2023.
  208. ——, “Nservicebus,” https://docs.particular.net/nservicebus/best-practices, last accessed: December 22, 2023.
  209. ——, “Nservicebus,” https://docs.particular.net/nservicebus/recoverability/, last accessed: December 22, 2023.
  210. ——, “Nservicebus,” https://docs.particular.net/nservicebus/recoverability/configure-error-handling, last accessed: December 22, 2023.
  211. Twitter, “kestrel,” https://github.com/twitter-archive/kestrel/blob/master/docs/guide.md, last accessed: December 22, 2023.
  212. ——, “kestrel,” https://github.com/memcached/memcached/blob/master/doc/protocol.txt, last accessed: December 22, 2023.
  213. Bitly, “Nsq,” https://nsq.io/overview/features_and_guarantees.html, last accessed: December 22, 2023.
  214. ——, “Nsq,” https://nsq.io/overview/design.html, last accessed: December 22, 2023.
  215. S. Communications, “Nats,” https://docs.nats.io/reference/faq, last accessed: December 22, 2023.
  216. ——, “Nats,” https://docs.nats.io/nats-concepts/what-is-nats, last accessed: December 22, 2023.
  217. B. Ayaz, N. Slamnik-Kriještorac, and J. Marquez-Barja, “Data management platform for smart orchestration of decentralized and heterogeneous vehicular edge networks,” in Proceedings of the 2022 ACM Conference on Information Technology for Social Good, 2022, pp. 118–124.
  218. KubeMQ, “Kubemq,” https://kubemq.io/kubemq-commercially-available/, last accessed: December 22, 2023.
  219. IBM, “Ibm mq,” https://cloud.ibm.com/catalog/services/mq, last accessed: December 22, 2023.
  220. Iron.io, “Ironmq,” https://blog.iron.io/ibm-mq-vs-ironmq-pros-cons-and-choosing-an-mq/#4, last accessed: December 22, 2023.
  221. IBM, “Ibm mq,” https://www.ibm.com/docs/en/integration-bus/10.0?topic=bus-quality-service-connection-management, last accessed: December 22, 2023.
  222. ——, “Ibm mq,” https://www.ibm.com/docs/en/ibm-mq/7.5?topic=management-what-happens-when-message-cannot-be-delivered, last accessed: December 22, 2023.
  223. ——, “Ibm mq,” https://www.ibm.com/docs/en/integration-bus/9.0.0?topic=messages-application-programming-interfaces, last accessed: December 22, 2023.
  224. ——, “Ibm mq,” https://www.ibm.com/docs/en/ibm-mq/9.2?topic=mq-introduction, last accessed: December 22, 2023.
  225. A. W. S. (AWS), “Amazon sqs,” https://aws.amazon.com/sqs/pricing/, last accessed: December 22, 2023.
  226. ——, “Amazon sqs,” https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/quotas-messages.html, last accessed: December 22, 2023.
  227. ——, “Amazon sqs,” https://aws.amazon.com/sqs/faqs/, last accessed: December 22, 2023.
  228. Microsoft, “Msmq,” https://learn.microsoft.com/en-us/previous-versions/windows/desktop/msmq/ms711436(v=vs.85), last accessed: December 22, 2023.
  229. ——, “Msmq,” https://techcommunity.microsoft.com/t5/skype-for-business-blog/troubleshooting-microsoft-message-queuing-issues-on-microsoft/ba-p/619639, last accessed: December 22, 2023.
  230. ——, “Msmq,” https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-mqmr/5fd22730-7423-4be0-89e2-688212e00e3a, last accessed: December 22, 2023.
  231. ——, “Msmq,” https://learn.microsoft.com/en-us/previous-versions/windows/desktop/msmq/ms703188(v=vs.85), last accessed: December 22, 2023.
  232. ——, “Msmq,” https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-mqmq/87486cd1-6ef0-4b58-93f9-f9b97b2279c4, last accessed: December 22, 2023.
  233. ——, “Msmq,” https://learn.microsoft.com/en-us/previous-versions/windows/desktop/msmq/ms711472(v=vs.85), last accessed: December 22, 2023.
  234. ——, “Msmq,” https://learn.microsoft.com/en-us/previous-versions/windows/desktop/msmq/ms699870(v=vs.85), last accessed: December 22, 2023.
  235. Oracle, “Oracle message broker,” https://docs.oracle.com/cd/E18930_01/html/821-2438/aeojm.html, last accessed: December 22, 2023.
  236. TIBCO, “Tibco enterprise message service™,” https://docs.tibco.com/pub/ems/10.1.0/TIB_ems_10.1.0_relnotes.pdf?id=1, last accessed: December 22, 2023.
  237. ——, “Tibco enterprise message service™,” https://support.tibco.com/s/article/Tibco-KnowledgeArticle-Article-22588, last accessed: December 22, 2023.
  238. ——, “Tibco enterprise message service™,” https://docs.tibco.com/pub/ems/8.6.0/doc/pdf/TIB_ems_8.6.0_users_guide.pdf?id=3, last accessed: December 22, 2023.
  239. ——, “Tibco rendezvous,” https://www.tibco.com/products/tibco-cloud-events/pricing-plans, last accessed: December 22, 2023.
  240. ——, “Tibco rendezvous,” https://docs.tibco.com/pub/ems/8.6.0/doc/html/GUID-66774B42-2A5F-4221-864E-3331622E1091.html, last accessed: December 22, 2023.
  241. ——, “Tibco rendezvous,” https://docs.tibco.com/pub/rendezvous/8.6.0/doc/pdf/TIB_rv_8.6.0_concepts.pdf, last accessed: December 22, 2023.
  242. ——, “Tibco rendezvous,” https://docs.tibco.com/pub/rendezvous/8.5.2/doc/pdf/TIB_rv_8.5_configuration_tools.pdf?id=7, last accessed: December 22, 2023.
  243. ——, “Tibco rendezvous,” https://docs.tibco.com/pub/rendezvous/8.5.2/doc/pdf/TIB_rv_8.5_concepts.pdf?id=6, last accessed: December 22, 2023.
  244. MuleSoft, “Anypoint mq,” https://www.mulesoft.com/anypoint-pricing#, last accessed: December 22, 2023.
  245. ——, “Anypoint mq,” https://docs.mulesoft.com/mq/mq-faq, last accessed: December 22, 2023.
  246. ——, “Anypoint mq,” https://docs.mulesoft.com/mq/mq-faq#in-what-order-are-messages-in-a-standard-non-fifo-queue, last accessed: December 22, 2023.
  247. ——, “Anypoint mq,” https://www.mulesoft.com/integration-solutions/dataweave-integration, last accessed: December 22, 2023.
  248. Microsoft, “Azure service bus,” https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-faq, last accessed: December 22, 2023.
  249. ——, “Azure storage queue,” https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-azure-and-service-bus-queues-compared-contrasted, last accessed: December 22, 2023.
  250. ——, “Azure service bus,” https://learn.microsoft.com/en-us/azure/service-bus-messaging/enable-duplicate-detection, last accessed: December 22, 2023.
  251. ——, “Azure service bus,” https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-messaging-overview, last accessed: December 22, 2023.
  252. S. AG, “Sap netweaver process integration,” https://blogs.sap.com/2014/04/02/message-size-as-source-of-performance-bottleneck/, last accessed: December 22, 2023.
  253. ——, “Sap netweaver process integration,” https://help.sap.com/docs/portfolio-category/INTEGRATION, last accessed: December 22, 2023.
  254. ——, “Sap netweaver process integration,” https://help.sap.com/docs/SAP_NETWEAVER_731_BW_ABAP/, last accessed: December 22, 2023.
  255. C. Betts, “Solace,” https://docs.solace.com/API/API-Developer-Guide/Adding-Data-Payloads.htm, last accessed: December 22, 2023.
  256. ——, “Solace,” https://docs.solace.com/Software-Broker/System-Scaling-Parameters.htm#max-queue-messages, last accessed: December 22, 2023.
  257. ——, “Solace,” https://docs.solace.com/API/IPC/Limitations.htm, last accessed: December 22, 2023.
  258. ——, “Solace,” https://solace.com/products/event-broker/distributed-tracing/, last accessed: December 22, 2023.
  259. Google, “Google cloud pub/sub,” https://cloud.google.com/pubsub/quotas, last accessed: December 22, 2023.
  260. ——, “Google cloud pub/sub,” https://cloudplatform.googleblog.com/2015/04/big-data-cloud-way.html, last accessed: December 22, 2023.
  261. ——, “Google cloud pub/sub,” https://cloud.google.com/pubsub/architecture, last accessed: December 22, 2023.
  262. ——, “Google cloud pub/sub,” https://cloud.google.com/pubsub/docs/ordering, last accessed: December 22, 2023.
  263. ——, “Google cloud pub/sub,” https://cloud.google.com/pubsub/docs/handling-failures, last accessed: December 22, 2023.
  264. Microsoft, “Azure storage queue,” https://learn.microsoft.com/en-us/azure/storage/queues/monitor-queue-storage?tabs=azure-portal, last accessed: December 22, 2023.
  265. A. W. S. (AWS), “Amazon mq,” https://docs.aws.amazon.com/amazon-mq/latest/migration-guide/key-concepts.html, last accessed: December 22, 2023.
  266. ——, “Amazon mq,” https://docs.aws.amazon.com/amazon-mq/latest/developer-guide/maintaining-brokers.html, last accessed: December 22, 2023.
  267. ——, “Amazon mq,” https://aws.amazon.com/amazon-mq/features/, last accessed: December 22, 2023.
  268. Intel, “Intel mpi library,” https://texas.gs.shi.com/product/32703496/Intel-MPI-Library-for-Windows, last accessed: December 22, 2023.
  269. ——, “Intel mpi library,” https://www.intel.com/content/www/us/en/developer/articles/technical/improve-performance-and-stability-with-intel-mpi-library-on-infiniband.html, last accessed: December 22, 2023.
  270. Amazon, “Kinesis,” https://repost.aws/knowledge-center/troubleshoot-kinesis-agent-linux, last accessed: December 22, 2023.
  271. ——, “Kinesis,” https://aws.amazon.com/kinesis/data-streams/pricing/, last accessed: December 22, 2023.
  272. ——, “Kinesis,” https://docs.aws.amazon.com/streams/latest/dev/service-sizes-and-limits.html, last accessed: December 22, 2023.
  273. ——, “Kinesis,” https://docs.aws.amazon.com/kinesis/latest/APIReference/API_DescribeLimits.html, last accessed: December 22, 2023.
  274. ——, “Kinesis,” https://docs.aws.amazon.com/streams/latest/dev/kinesis-producer-adv-retries-rate-limiting.html, last accessed: December 22, 2023.
  275. ——, “Kinesis,” https://aws.amazon.com/kinesis/data-streams/resources/, last accessed: December 22, 2023.
  276. Iron.io, “Ironmq,” https://try.iron.io/pricing-worker-monthly/, last accessed: December 22, 2023.
  277. ——, “Ironmq,” https://blog.iron.io/apache-kafka-vs-ironmq-whats-best-for-your-business/, last accessed: December 22, 2023.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now