Editor’s Recommendation
In traditional enterprise architecture, high-performance message queues (MQ) and messaging systems have always been the cornerstone of Event-Driven Architecture (EDA). They serve as the “glue” that allows distributed systems to maintain high throughput and low latency while providing better fault tolerance and retry capabilities in failure scenarios.
So, in today’s era of AI agents, does such traditional infrastructure still have a place? Perhaps with the future development of multi-agent collaboration and distributed agents, the message-driven model is ushering in new opportunities:
-
Collaboration and state transfer between multiple agents through messaging
-
Providing more flexible retry and load balancing mechanisms
-
Helping enterprises integrate new AI capabilities in a more loosely coupled manner
In our previous article, we introduced Microsoft’s new version of AutoGen, which is a message-driven multi-agent framework. Its “runtime” essentially serves as the messaging facility for agent communication, and publish/subscribe is one of the important communication modes it supports. Although Google’s advocated A2A protocol currently uses HTTP, it does not rule out the possibility of compatibility with messaging architectures in the future.
This article will take you through the essence of EDA, reflecting on how it re-emerges in the wave of AI agents; and using Apache’s open-source messaging platform Pulsar as an example, we will explore more possibilities of its integration with AI.
Open-source messaging queue technology has developed over 20 years, evolving from single-machine architecture to cloud-native distributed systems, always playing the role of the “central nervous system” of distributed systems. As AI agents become the core paradigm of the next generation of intelligent applications, the integration of messaging queues and AI technology is opening a new chapter.
1
History and Trends: How messaging queue technology has evolved alongside hardware revolutions and scenario changes, ultimately becoming a key infrastructure in the cloud-native era.
2
The Necessity of AI Agents: Why multi-agent collaborative systems (MAS) will become the core architecture for AI implementation, and the essential similarities between this architecture and distributed systems.
3
Key Integration Points: How messaging queues provide core capabilities such as decoupled collaboration, reliable communication, and asynchronous orchestration for AI agent systems, becoming the “lifeline” of the agent ecosystem.
4
Pulsar’s Empowerment: How Apache Pulsar, as a next-generation messaging streaming platform, meets the special needs of AI agents in complex message processing, data isolation, and elastic scaling through its unique architecture.
5
Bidirectional Evolution: Exploring how messaging queues empower AI systems (“middleware for AI”) and envisioning how AI technology can enhance the intelligence of messaging queues (“AI for middleware”).
1
The History of Messaging Queues: A Duet of Hardware Revolution and Scenario Transformation Driving Generational Leap in Architecture
1980-2000 Commercial Closed Source Era
Closed-source systems like IBM MQ laid the foundation for enterprise-level message transmission, ensuring high reliability for core businesses in finance and telecommunications, but were limited by ecological closure and high costs.
2000-2007 Open Source Breakthrough: Single-Machine Architecture Era
Apache ActiveMQ and RabbitMQ sparked the open-source revolution, with lightweight and flexible protocols (like AMQP) accelerating the democratization of messaging technology, supporting the rise of early e-commerce and instant messaging.
2010-2017 Golden Age of Distributed Architecture
The mobile internet wave ignited the demand for real-time interaction among hundreds of millions of terminals, leading to explosive growth in scenarios like e-commerce (flash sales), social platforms, and financial payments. Kafka reshaped data pipelines with its high throughput and distributed log architecture, helping Netflix achieve real-time streaming to millions of devices. Alibaba relied on RocketMQ to support the peak of hundreds of billions in transactions during Double Eleven.
2017-2023 Cloud-Native Leap
The maturity of the Docker/Kubernetes ecosystem has driven messaging queues to upgrade to layered storage and compute separation architectures. Apache Pulsar, with its multi-tenant isolation and stateless broker design, has become a benchmark for infrastructure in the cloud-native era, supporting million-level topics with second-level scheduling.
2023+ AI Era: The “Neural Synapse” of Intelligent Systems?
2
AI Agents: The Inevitable Form of Future Intelligent Agents
AI agents are not merely simple chatbots; they are intelligent entities with perception, planning, action, learning, and collaboration capabilities. They can proactively understand goals, decompose tasks, invoke tools (APIs), interact with the environment, and continuously optimize. Whether as personal assistants, automation process executors, participants in complex decision-making, or building a complex ecosystem filled with autonomous agents, AI agents are widely regarded as the next important form and inevitable trend in AI development.
For example, in making Kung Pao Chicken, simulating the division of labor in a real restaurant, there are specialists for buying ingredients, washing vegetables, cutting vegetables, and cooking, with each role being an expert in its respective field. A multi-agent system is like a professional kitchen: procurement, washing, cutting, and cooking expert agents each perform their duties, efficiently and autonomously completing complex cooking tasks through the MCP protocol.
From the above example, it can be seen that multi-agent collaborative systems (Multi-Agent Systems, MAS) have become a more practical and powerful architectural paradigm for building usable AI agent systems, as they need to achieve complex capabilities such as task decomposition, concurrent processing, role division, and collaborative reasoning. This essentially constructs a distributed system.Just as the tightly coupled communication methods (like synchronous HTTP API calls) between services in the evolution of microservices architecture have proven difficult to support large-scale, high-concurrency, elastic, and reliable demands, the simple point-to-point communication (like direct RPC calls) between multiple agents in MAS will also quickly encounter bottlenecks such as communication blocking, fault propagation, and limited scalability.
Therefore,introducing message queues (MQ) as the core asynchronous communication infrastructure has become an inevitable choice for engineering implementation. It provides efficient, decoupled, reliable, buffered, and scalable messaging capabilities, which are key to solving the efficient collaboration issues among distributed agents.
3
Key Integration Point: Messaging Queues Become the “Lifeline” of the Agent Ecosystem
3.1 Decoupling and Collaboration: The Lifeline of the Agent Ecosystem
Complexity Multiplication: The AI agent world is a highly heterogeneous environment. Agents with different functions, architectures, and owners (tool agents, decision agents, execution agents, orchestration agents) need to collaborate efficiently.
The Role of MQ: As a standard “message bus”, messaging queues naturally become the universal protocol for communication between different agents, between agents and backend services/data sources, and even across enterprise agents. It completely decouples the interaction logic of agents, allowing each agent to focus on its core capabilities. The publish/subscribe model is particularly suitable for perceiving agent state changes, task completion notifications, or global events, driving the collaborative operation of the entire agent ecosystem.
3.2 Reliability and Robustness: Ensuring Determinism and Traceability of AI Actions
Actions Cannot Be Lost: The execution instructions, task results, and status updates of agents are crucial. The persistence, acknowledgment mechanisms, and retry strategies of messaging queues ensure that messages can be reliably transmitted even in the event of node failures or network fluctuations, avoiding “ghost actions” or inconsistent states.
Traceability: Messaging queues inherently have logging properties. By utilizing message traces and message content, every step of agent interaction can be completely recorded, providing key evidence for debugging, auditing, and explaining AI decision logic.
3.3 Asynchronous Orchestration: The Symphony of Agent Workflows
Long Process Orchestration: Completing a complex task (like booking a trip) may involve invoking multiple sub-services, waiting for external responses, and making multiple rounds of decisions.
The Role of MQ: Messaging queues are the ideal medium for implementing asynchronous, loosely coupled workflow orchestration. The main agent can decompose tasks into multiple steps, triggering the corresponding services or agents to execute by publishing messages to different queues and waiting for asynchronous completion notifications. This prevents the system from being blocked for long periods and can gracefully handle delays and failures.
3.4 Peak Shaving and Elastic Scaling: Responding to the “Intelligent Storm” of the Agent World
Dynamic Load: The interactions and triggers of AI agents may be sudden (like an event triggering a large number of agents to start) or periodically fluctuating.
The Buffer Pool of MQ: Messaging queues serve as a buffer layer, smoothing out the load peaks generated upstream, allowing backend computing resources (servers/containers running agents) to scale elastically and respond calmly. When countless agents are activated simultaneously, MQ becomes a key component in ensuring system stability.
3.5 Event-Driven Architecture: The Perception and Action Hub of Agents
The Dual Event Role of Agents: Agents respond to specific events (like “task planning” or “alarm analysis”) as consumers while also producing new events that drive workflows, forming a collaborative chain.
The Connector of MQ: Messaging queues are core components of modern event-driven architectures. They can efficiently and in real-time distribute events generated by sources (database change logs, IoT sensor data, user operations, other agent state changes) to agents that subscribe to these events. Agents can make more timely and relevant perceptions, decisions, and actions based on real-time event streams, making the system more responsive.
4
Empowerment and Embrace: Pulsar x AI
Apache Pulsar is a top-level project of the Apache Software Foundation, a next-generation cloud-native distributed messaging streaming platform that integrates messaging, storage, and lightweight functional computing, adopting a compute and storage separation architecture design, supporting multi-tenancy, persistent storage, cross-region data replication, and featuring strong consistency, high throughput, low latency, and high scalability for streaming data storage.
Using the example of making Kung Pao Chicken, the simple interaction process between AI agents and Pulsar is as follows:
4.1 Empowerment
4.1.1 Support for Complex Message Formats:
Structured Data: The information semantics exchanged by AI agents are richer. In addition to traditional text/JSON, there is a need to better support schema-based data (like Protobuf, Avro) to accommodate complex agent instructions, states, reasoning contexts, etc.
Pulsar’s Empowerment:Pulsar not only supports conventional message formats (like Bytes, String) but also adapts to the complex data needs of multi-agent systems through its native schema registry. It also supports binary multimodal data such as audio or video.
4.1.2 Built-in Powerful Messaging Patterns:
Rich Protocol Support: Provides native support for complex interaction patterns, such as delayed/timed messages (scheduled tasks), retry messages (automatic task retries), and ordered messages (strictly following user prompt input order for personal preference memory updates), etc.
Pulsar’s Empowerment:Pulsar enables technical empowerment by supporting various consumption patterns, covering both messaging and stream processing scenarios, while providing a unified API to help businesses achieve rapid development.
4.1.3 High Consistency, High Reliability, High Availability, High Performance, and Low Latency:
Core Capabilities: In real-time intelligent decision-making scenarios, messaging queues need to meet critical requirements for high consistency, high reliability, high availability, high performance, and low latency to ensure business continuity and user experience.
Pulsar’s Empowerment:Pulsar adopts a compute-storage separation architecture, effectively addressing the challenges of high concurrency, high availability, high performance, and high scalability (the “four highs”) in distributed systems, while providing stable end-to-end low-latency services.
4.1.4 Data Isolation Capabilities:
Data Isolation and Privacy: Data isolation capability is a core requirement for AI agents in commercial deployment, ensuring both security compliance and improved resource utilization, making it an essential feature for multi-tenancy and high-sensitivity scenarios.
Pulsar’s Empowerment:Pulsar achieves data isolation through its native multi-tenant architecture and physical separation of compute/storage resources, combined with transport layer encryption (TLS) and message content encryption (end-to-end), providing multiple guarantees for data security.
4.1.5 High Scalability:
Future Development: Although AI applications are still in the exploratory stage, high scalability enables AI agents to flexibly respond to business growth, traffic fluctuations, and global deployment needs, making it a core capability for building production-grade AI systems.
Pulsar’s Empowerment:Pulsar provides critical infrastructure support for the impending explosion of AI business scenarios through its second-level elastic scaling capabilities and native cross-region data synchronization mechanisms.
4.2 Embrace
Initiated by the China Academy of Information and Communications Technology, the “Middleware + AI” Seminar is ongoing, gathering experts from various middleware vendors. The seminar focuses on two core directions:
Middleware for AI: Middleware used in AI systems, such as message middleware, cache middleware, etc., applied in data collection and preprocessing, model training, and inference in AI systems.
AI for Middleware: Middleware gains AI attributes. While middleware provides software infrastructure support for AI systems, the development of AI technology also drives the intelligent upgrade of middleware, allowing middleware to develop AI attributes.
The previous sections have discussed how Pulsar, as a message middleware, empowers AI from the perspective of Middleware for AI. Below, we will briefly discuss what changes Pulsar can make from the perspective of AI for Middleware.
Operational Intelligence
Intelligent Resource Management: For example, in the operation of an AI agent (like the Kung Pao Chicken recipe coordination agent), the traditional model requires manual pre-creation of topics and resource allocation in the Pulsar console. To completely eliminate manual intervention, Pulsar needs to have situational awareness capabilities: automatically creating topics based on traffic predictions, dynamically adjusting the number of partitions, and intelligently routing topics. Technically, this can be achieved by building a Pulsar MCP Server (Management & Control Plane Server) to realize the core capability loop.
Smart Dead Letter Queue (DLQ): Not just storing failed messages, it can automatically analyze the reasons for DLQ failures, attempt repairs, notify, or trigger specific handling agents.
Intelligent Dynamic Load Balancing: The pain points of traditional load balancing rely on complex configurations and static scheduling strategies preset by humans, needing to handle numerous boundary conditions (like network topology awareness, service instance status thresholds), leading to high maintenance costs and difficulty in flexibly responding to business fluctuations. The ultimate AI-driven solution dynamically generates adaptive scheduling strategies through reinforcement learning (like the DDPG algorithm).
Operational Intelligence
Observational Intelligence: Upgrading from “seeing data” to “understanding system status”, intelligently identifying and aggregating key operational indicators, establishing causal chains between Broker/Bookie/ZK metrics and logs; integrating metrics, logs, and message traces into multimodal data.
Diagnostic Intelligence: Upgrading from “alarm storms” to “precise surgical repairs”, constructing fault decision trees based on historical work orders (like “production delays significantly increased”), generating interpretable diagnostic reports. For example, detecting that the memory of Broker-10 node is exhausted (99% usage), leading to increased message persistence delays; recommended action: immediately expand node memory and migrate topics to lightly loaded brokers.
Elastic Intelligence: Upgrading from “passive scaling” to “predictive adaptive tuning”, cloud-native architectures already possess native elastic scaling capabilities, and the key is to intelligently determine resource adjustments based on the current system’s operational status, balancing multiple objectives of cost/performance/compliance in real-time.
In the field of AI for middleware, the industry is still in the exploratory stage, and this article only lists some application directions. With the rapid development of AI technology, we need to continuously pay attention to the latest progress and actively apply innovative results.
5
Conclusion: The Bridge of Data Flow to an Intelligent Future
When the messaging queue that has developed for twenty years meets the rising AI agents, it is not a confrontation between new and old technologies, but an exciting fusion evolution and mutual empowerment. The core advantages of messaging queues—reliability, decoupling, asynchrony, and buffering—are not only undiminished in the distributed, heterogeneous, high-concurrency, and long-process ecosystem of AI agents but are significantly amplified, becoming the lifeline that connects intelligent agents.
The future messaging queue will not only be a data mover but also the intelligent information infrastructure of the AI agent world, the cornerstone of efficient collaboration, the guarantee of reliable actions, and the sensory extension of event perception. Actively embracing the demand for agent paradigms, continuously innovating in protocol support, performance, observability, security, and platform integration, the messaging queue, which has flowed for twenty years, will build a solid bridge to a vast intelligent future. For developers building, deploying, managing, and optimizing AI agent systems, choosing and effectively utilizing advanced messaging queue technology will be a key step towards the success of future intelligent applications.
References
[1] Apache Pulsar
https://pulsar.apache.org/
[2] 2024 AI Top Ten Trends Report
https://jkhbjkhb.feishu.cn/wiki/W5D7wuDcbiPXDLkaRLQcAJpOn8f
[3] MCP
https://modelcontextprotocol.io/introduction
[4] The Future of AI Agents is Event-Driven
https://mp.weixin.qq.com/s/C2x0OOKFXuXDn07hEcKtew
[5] Cloud Integration Intelligence, Smart Chain All Domain | The China Academy of Information and Communications Technology officially launched the “Middleware + AI” system research work!
https://mp.weixin.qq.com/s/ptPxo5KxIa2JoHiW4rkRzA
[6] Implementing a Simple Pulsar MCP Server Using Spring AI
https://mp.weixin.qq.com/s/0E3wmLdo1TCkS0g4gP_t8Q
[7] DDPG
https://arxiv.org/pdf/1509.02971
This article was first published in the “AI Middleware Column” of the “Cloud Native Industry Alliance” WeChat public account, written by Liudezhi, Technical Partner of Shanghai Anyou Technology Co., Ltd.
Apache Pulsar Project:
https://github.com/apache/pulsar
Apache Pulsar WeChat Public Account:
Scan the following business card
Join the WeChat public account group (please specify your intention)
