Since the era of ChatGPT, there has been a growing consensus: the core of large model applications is not an algorithmic issue, but an engineering issue. The large model itself is already in place as infrastructure; the key lies in how to solve practical problems such as memory storage, context management, tool invocation, and prompt optimization through engineering means.
Recently, I read about Anthropic’s engineering practices in multi-agent systems and found it very interesting, so I am sharing it here. This article will analyze:
- 1. Architecture Design of Multi-Agent Systems (Orchestrator-Worker Model)
- 2. Advantages and Disadvantages of Multi-Agent Systems
- 3. Prompt Engineering Practices (8 Core Experiences from Anthropic)
The next article will cover Anthropic’s agent evaluation, production reliability, and engineering challenges.
Architecture of Multi-Agent Systems
Analysis of the Orchestrator-Worker Model
Anthropic adopts the orchestrator-worker architectural model.
The term “orchestrate” is often seen in AI papers. The literal meaning of an orchestrator is a composer for an orchestra, someone who arranges a piece of music for orchestral performance and assigns appropriate parts and melodies to each instrument.
Here, the orchestrator is extended to mean “the component responsible for overall scheduling, coordination, and orchestration of all subtasks, also known as the lead agent.” The worker is the agent that executes subtasks, also known as subagents.
The lead agent serves as the “command center” of the system, with core responsibilities including:
- • Task decomposition and assignment
- • Resource coordination and scheduling
- • Result integration and decision-making
The subagents focus on executing specific subtasks, working in parallel to enhance the overall efficiency of the system through division of labor.
Architecture Diagram
Below is the architecture diagram of the research system.
(Image source: Anthropic official website)
As shown in the diagram, when a user submits a query, the lead agent analyzes it, formulates a strategy, and iteratively uses subagents to collect information through search tools, then returns the information to the lead agent, which finally compiles it into a final answer.
The difference between multi-agent systems and RAG:
Retrieval-Augmented Generation (RAG) is a static retrieval, which retrieves a set of text blocks most similar to the input query and uses these blocks to generate a response. In contrast, the multi-agent architecture employs multi-step searches to dynamically find relevant information, adapt to new discoveries, and analyze results to generate high-quality answers.
Interaction Process
Below is the interaction process of the multi-agent system, where we can see not only different agents but also a Memory module and a Citation Agent.
(Image source: Anthropic official website)
The steps are as follows:
- 1. When a user submits a query, the system creates a LeadResearcher (lead research agent), which enters an iterative research loop.
- 2. The lead researcher first considers the overall plan and writes it into Memory (the memory module) to ensure that the context is persisted—because once the context window exceeds 200,000 tokens, it will be truncated, making it crucial to retain the plan.
- 3. The lead researcher creates several specialized Subagents (sub-research agents, with 2 shown in the diagram, but the actual number can vary), each taking on specific subtasks.
- 4. Each sub-research agent independently performs web searches and evaluates the results returned by tools using an “Interleaved thinking” approach, then returns their findings to the lead researcher.
- 5. The lead researcher compiles these results and determines whether further research is needed;
- 6. If further research is required, it can create new sub-research agents or adjust strategies. Once sufficient information is gathered, the system exits the research loop and hands all findings to the Citation Agent.
- 7. The citation agent processes documents and research reports, locating specific sources for each citation needed, ensuring that all conclusions can be traced back to their origins. Finally, the research results with complete citations are returned to the user.
It is worth mentioning that Interleaved thinking refers to interleaved reasoning.
The Claude 4 model, when invoking tools (tool calls), does not package the “thinking process” all at once; instead, after each tool returns a result, it inserts a new reasoning segment, then decides whether to continue invoking tools and which tool to invoke next.
The entire process of “thinking—invoking tools—re-thinking—re-invoking…” interleaves like gears, rather than the traditional approach of “thinking through all steps at once, then executing them consecutively.”
Advantages and Disadvantages of Multi-Agent Systems
Why is research work suitable for Multi-Agent Systems?
Because research work mainly involves open-ended questions, researchers continuously update their research methods based on clues that arise during the research process.
The uncertainty of research work aligns perfectly with the characteristics of AI Agents.AI Agents can autonomously run multiple rounds and decide subsequent directions based on intermediate discoveries.
This complexity means that linear, one-time Single Agents cannot handle it; it must be Multi-Agent collaboration.
What is the strength of the Multi-Agent Research System?
The multi-agent research system is particularly adept at handling breadth-first queries involving multiple independent directions simultaneously.
The lead agent issues tasks, and subagents run in parallel with their respective context windows, exploring different aspects of the problem and distilling the most important information from each part.
Each subagent has different focuses, tools, prompts, and exploration trajectories, thereby reducing path dependency and supporting thorough, independent research.
What are the disadvantages of the Multi-Agent Research System?
The downside is that tokens are consumed very quickly.The number of tokens used by agents is 4 times that of chat, and the number of tokens used by Multi-Agent systems is 15 times that of chat.
Therefore, multi-agent systems are suitable for high-value tasks, suitable for parallel tasks, but not suitable for tasks with process orchestration due to their high costs. Compared to research work, there are relatively fewer parallel tasks in coding work.
What determines the performance of Multi-Agent Systems?
The system’s performance mainly depends on three key factors:
- 1. Whether sufficient tokens are used
- 2. The number of tool invocations
- 3. The capability of the base model: the performance of the underlying large model determines the system’s upper limit
The above conclusions were derived from Anthropic’s evaluation using the BrowseComp benchmark proposed by OpenAI.BrowseComp is a benchmark used to assess the ability of AI Agents to find difficult information.
Prompt Engineering Practices for Multi-Agent Systems
The key difference between multi-agent systems and single-agent systems is the rapid increase in coordination complexity.
Early agents made some mistakes:
- 1. Generating 50 subagents for a simple query
- 2. Searching endlessly online for non-existent information sources
- 3. Over-updating leading to deviation from the main direction
Note that all of the above issues can be solved purely through prompt engineering optimization.Prompt engineering is far more important than one might think.
In Anthropic’s multi-agent system, each agent has a guiding prompt.
Anthropic summarized the following 8 experiences:
Think from the agent’s perspective. Think like your agents.
**To iterate prompts, one must understand their effects.** Effective prompt engineering depends on whether the user can establish an “accurate expectation model” of the Agent in their mind; once expectations are set, it often becomes clear what needs to be changed and how to change it.
Teach the orchestrator how to delegate. Teach the orchestrator how to delegate.
In a multi-agent system, the lead agent decomposes queries into subtasks and describes them to subagents.
The task specification written by the lead agent must include four components:• objective—what conclusion or data is to be produced• output format—what format to submit (table, paragraph, JSON, etc.)• tools & sources—what tools can be used, which libraries/websites to check• task boundaries—responsibility boundaries to prevent “no man’s land” or “territorial disputes”
If the specification is too simplistic, three types of problems may arise:• Duplicate work• Gaps in information• Failure to find necessary information
Anthropic found that simple, brief instructions do not work, such as giving the instruction “research the semiconductor shortage issue,” which is so vague that subagents may misunderstand the task or perform searches identical to those of other agents.
Scale effort to query complexity. Scale effort to query complexity.
Agents struggle to determine the appropriate workload for different tasks, so Anthropic writes scaling rules in the prompts.
- • Simple fact-finding requires only 1 agent to invoke 3-10 tools;
- • Direct comparisons may require 2-4 subagents, each invoking 10-15 tools;
- • Complex research may require more than 10 subagents with clearly defined responsibilities.
This scaling rule has two benefits: it allows the lead agent to allocate resources efficiently; and it prevents excessive resource allocation on simple queries.
Tool design and selection are critical. Tool design and selection are critical.
Many people often overlook two points when designing tools: first, the “interface experience” between the agent and the tool, which is akin to the UI/UX of software for humans. If a button is not understandable to a person, they may click incorrectly; for agents, unclear schemas and vague descriptions can lead to incorrect calls. Choosing the wrong tool is not only inefficient but often means the task fails fundamentally.
Second, the definitions of tools may be unclear. For example, an MCP (Model-Context-Protocol) server may dump dozens of external tools into the agent’s view at once, many of which overlap or have varying quality of descriptions.
The design of the tools themselves (functional boundaries, input/output formats) and the decision of “which tool to choose” directly determine whether the agent can complete the task.
In this regard, Anthropic’s strategy is:• First, scan all available tools to have an overall understanding• Match tool capabilities with user intent (intent → tool mapping)• Use general web search only when extensive external information is needed• Use specialized tools instead of general ones (specialized > generic)
Anthropic has established three criteria for tool descriptions:• Unique positioning: functionality does not overlap with others• Clear description: a single sentence should make it clear to the agent “when to use it”• Clear boundaries: what inputs are needed, what outputs are given, and what cannot be done
Let agents improve themselves. Let agents improve themselves.
Allow agents to discover problems themselves → propose improvements → verify effectiveness.
Anthropic created a Tool-testing agent for this purpose, which works as follows:
- • Input: a newly written MCP tool (schema + description often has defects).
- • Actions:① Repeatedly invoke the tool (dozens of times), recording all failures, boundary cases, and misuse scenarios.② Automatically rewrite the tool description: supplement required field explanations, add examples, highlight pitfalls, and modify formats.
- • Output: a better-performing version of the tool description.
This tool has led to performance improvements: after the new tool description was launched, the task completion time for subsequent agents using that tool decreased by 40%; because common pitfalls were preemptively explained, agents no longer fell into traps.
Start wide, then narrow down. Start wide, then narrow down.
The agent’s problem: queries are composed of long and specialized terms, resulting in limited information found.Improved approach:
- • First, perform a “broad search” using core keywords;
- • Browse the returned results, gradually adding qualifiers (time, region, metrics, etc.) to make the target more focused and conduct a deep search.
This approach mimics human behavior: human experts always look at “what exists in this field” before deciding “which point is worth digging deeper”; agents should do the same.
Guide the thinking process. Guide the thinking process.
Expand thinking and interleaved reasoning.
Parallel tool calling transforms speed and performance. Parallel tool calling transforms speed and performance.
Complex research tasks naturally require exploring numerous sources, but serial execution can make tasks very lengthy.
To improve speed, Anthropic introduced two forms of parallelization: (1) the lead agent simultaneously initiates 3-5 subagents; (2) subagents use 3 or more tools in parallel. These improvements reduced the research time for complex queries by up to 90%.
Conclusion
Anthropic’s set of prompts is heuristic, meaning it is derived from human experience rather than fixed rules.
I believe their engineering is very detailed. They studied how experts handle research tasks and incorporated these strategies into the prompts—such as adjusting search methods based on new information and recognizing when to focus on depth (detailed study of a topic) rather than breadth (parallel exploration of multiple topics).
They also iterated through the Tool-testing agent while setting prompts for each agent, establishing standards to prevent agents from going out of control. However, considering the massive consumption of tokens, continuous improvements in cost control, error prevention, and process optimization are still needed.