Datawhale shares
Research: Multi-Agent Systems, Editor: PaperAgent
In multi-agent systems (MAS: multi-agent systems), designing effective prompts and topologies is challenging, as individual agents may be sensitive to prompts, and manually designing topologies requires extensive experimentation.
Paper link: https://arxiv.org/pdf/2502.02533
Paper title: Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies
To automate the entire design process, Google & Cambridge University first conducted an in-depth analysis of the design space to understand the factors that contribute to building effective MAS. It was found that: prompt design has a significant impact on downstream performance, while effective topologies only represent a small portion of the entire search space.In mathematical problems, Gemini 1.5 Pro compared agents that only used self-consistency (SC), self-refinement (reflect), and multi-agent debate (debate) for expansion, demonstrating the relationship between the accuracy of prompt-optimized agents and the total number of labels for each problem. The error bars represent one standard deviation. We show that by using more effective prompts, higher accuracy can be achieved with more computational resources.
Performance of different topologies using Gemini 1.5 Pro was optimized through APO compared to the base agents, where “Sum.” (summary) and “Exe.” (executor) are task-specific topologies as shown in Figure 4. We observed that not all topologies positively influence the design of multi-agent systems (MAS).
Based on these findings, Google & Cambridge University proposed the Mass framework, which optimizes MAS through three stages:
-
Block-level (local) prompt optimization: Optimizing prompts for agents within each topology block.
-
Workflow topology optimization: Optimizing workflow topologies within the pruned topology space.
-
Workflow-level (global) prompt optimization: Conducting global prompt optimization on the identified best topology.
The proposed multi-agent system search (Mass) framework discovers effective multi-agent system designs by interleaving prompt optimization and topology optimization within a customizable multi-agent design space (the right side shows the optimized topologies and optimized prompts, with key components shown on the left)
The experiments used Gemini 1.5 Pro and Flash models and were compared with various existing methods, including Chain of Thought (CoT), Self-Consistency (SC), Self-Refine, Multi-Agent Debate, ADAS, and AFlow.
-
Performance Improvement: Mass significantly outperformed existing methods on multiple tasks, with an average performance improvement of over 10%.
-
Importance of Optimization Stages: Through phased optimization, Mass achieved performance improvements at each stage, demonstrating the necessity of optimization from local to global.
-
Co-optimization of Prompts and Topologies: Mass achieved better performance by simultaneously optimizing prompts and topologies compared to optimizing them separately.
-
Cost-effectiveness: Mass demonstrated stable and effective performance improvements during optimization, with higher sample efficiency and cost-effectiveness compared to existing automated design methods.