Research: Multi-Agent Systems, Editor: PaperAgent
In multi-agent systems (MAS: multi-agent systems), designing effective prompts and topologies poses challenges, as individual agents may be sensitive to prompts, and manually designing topologies requires extensive experimentation.
Paper link: https://arxiv.org/pdf/2502.02533
Paper Title: Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies
To automate the entire design process, Google & Cambridge University first conducted an in-depth analysis of the design space to understand the factors that contribute to building effective MAS. It was found that: prompt design has a significant impact on downstream performance, while effective topologies only represent a small portion of the entire search space.In mathematical problems, Gemini 1.5 Pro compared agents that only used self-consistency (SC), self-refinement (reflect), and multi-agent debate (debate) for expansion, demonstrating the relationship between the accuracy of prompt-optimized agents and the total number of labels for each problem. The error bars represent one standard deviation. We show that higher accuracy can be achieved by utilizing more computational resources through more effective prompts.
Performance of different topologies using Gemini 1.5 Pro was optimized through APO compared to the base agents, where “Sum.” (Summary) and “Exe.” (Executor) are task-specific topologies as shown in Figure 4. We observed that not all topologies positively impact multi-agent system (MAS) design.
Based on these findings, Google & Cambridge University proposed the Mass framework, which optimizes MAS through three stages:
-
Block-level (local) prompt optimization: Optimizing prompts for agents within each topology block.
-
Workflow topology optimization: Optimizing workflow topologies within the pruned topology space.
-
Workflow-level (global) prompt optimization: Conducting global prompt optimization on the identified optimal topology.
The proposed Multi-Agent System Search (Mass) framework interleaves prompt optimization and topology optimization within a customizable multi-agent design space, discovering effective multi-agent system designs (the optimized topologies and prompts are shown on the right, with key components as shown on the left).
The experiments utilized Gemini 1.5 Pro and Flash models and were compared with various existing methods, including Chain of Thought (CoT), Self-Consistency (SC), Self-Refine, Multi-Agent Debate, ADAS, and AFlow.
-
Performance Improvement: Mass significantly outperformed existing methods on multiple tasks, with an average performance improvement of over 10%.
-
Importance of Optimization Stages: Through phased optimization, Mass achieved performance improvements at each stage, demonstrating the necessity of optimization from local to global.
-
Co-optimization of Prompts and Topologies: Mass achieved better performance by simultaneously optimizing prompts and topologies compared to optimizing them separately.
-
Cost-effectiveness: Mass exhibited stable and effective performance improvements during optimization, showing higher sample efficiency and cost-effectiveness compared to existing automated design methods.