The “vision” and “intelligence” of soccer AI are insufficient!
Soccer is known as the “most complex team sport,” but existing research is like a “fan who can only watch the game”—either only recognizing actions (like tackles and shots) or unable to answer questions that require background knowledge (like “how many goals did a player score last season?”). 
Even more embarrassingly, existing models are like “students who only excel in one subject”: some are good at recognizing jersey numbers, some can generate commentary, but they cannot work together. This is like having 11 forwards play a match, resulting in chaos. 
Paper: Multi-Agent System for Comprehensive Soccer Understanding Link: https://arxiv.org/pdf/2505.03735
Three Contributions: Knowledge Base + Evaluation Benchmark + Agent System
The Shanghai Jiao Tong University team has accomplished three things, which can be called the new infrastructure for intelligent soccer QA:
- SoccerWiki Knowledge Base: Contains detailed data on 9,471 players and 266 teams, equivalent to a soccer version of “Wikipedia + Excel sheet.”
- SoccerBench Evaluation Benchmark: Contains 13,000 “soccer questions” covering 13 types of tasks (such as recognizing jersey colors and judging foul perspectives), with question types including text, images, and videos.
- SoccerAgent Multi-Agent System: Like a well-organized coaching team, it utilizes 18 tools (17 of which are open-source!) to break down complex problems. For example, to answer “When did the substitute player with brown hair make his national team debut in the video?” it will first capture the key frame → perform facial recognition → query the database → generate the answer.
How SoccerAgent Works: Multi-Tool Collaboration
The core logic of this system is: Problem Decomposition → Tool Invocation → Answer Synthesis. 
For example 🌰: The question “When did the substitute player with dark brown hair make his national team debut in the video?”
- Step 1: Use the “frame selection tool” to find the player’s image;
- Step 2: Use the “facial recognition tool” to match the database and identify the player (e.g., Cedric Itten);
- Step 3: Invoke the “match history retrieval tool” to pull debut records from SoccerWiki.
The entire process is like a coaching staff working together: data analysts find video clips, scouts identify players, and assistants check archives.
Experimental Results: Outperforming GPT-4, but Still Has Blind Spots
In the SoccerBench tests, SoccerAgent achieved a comprehensive accuracy of 60.9%, far exceeding GPT-4 (57.5%) and Gemini (54%). The advantage is particularly evident in questions requiring background knowledge (such as player career data). 
However, the system also has its moments of failure: for instance, when it cannot recognize a face in the video, it may incorrectly invoke a tool. Nevertheless, it can self-correct, similar to a referee changing a call after reviewing VAR footage. 
Significance
This research not only enables AI to better understand soccer but may also change the soccer industry:
- Fans: Real-time access to in-depth data, such as “Messi’s running distance in this match vs. career average”;
- Coaches: Automatically generate tactical reports and identify opponent weaknesses;
- Broadcasters: AI commentary will no longer read scripts but will tell stories based on real-time footage and databases.
All data and code will be open-sourced soon, so those interested in intelligent sports can stay tuned.
Note:Nickname – School/Company – Direction/Conference (e.g., ACL), join the technical/submission group
ID: DLNLPer, remember to note it