Soccer is considered one of the most popular sports worldwide, mainly because soccer matches are often intense and showcase players’ outstanding physical abilities and skills, thrilling the audience. Therefore, it is natural for some to strive to impart the essence of soccer games to robots, including how to steal the ball, aim at the goal, pass the ball, and score.
In fact, there is a competition specifically dedicated to this idea. The RoboCup Small Size League (SSL) Visual Occlusion Technology Challenge encourages participating teams to “explore local sensing and processing methods rather than using common off-board computers and global camera perception of the environment.” João Guilherme, a student from the Federal University of Pernambuco in Recife, Brazil, along with his advisor Edna Barros and other SSL teammates, created an omnidirectional mobile robot powered by the NVIDIA Jetson Nano Developer Kit, which can autonomously perform soccer tasks.
The omnidirectional mobile robot built by the team is equipped with a monocular camera and can autonomously perform the following tasks:
-
Localization
-
Finding and stealing the ball
-
Coordinate calculation
-
Passing the ball to other robot teammates
-
Shooting to score in an empty goal
The team used an AI software workflow with an average processing speed of 30 FPS and a hardware power consumption of around 10.8 W while building this robot.
The front of the robot features a kicking device and is a four-wheeled omnidirectional mobile robot. Figure 1 shows the geometry of the robot.
Figure 1. Powered by the NVIDIA Jetson Nano Developer Kit
The motion capabilities of the omnidirectional mobile robot allow it to autonomously perform soccer tasks
The team explained in their paper “Building Autonomous Robots for RoboCup Small Size League”: “We evaluated the system on three soccer tasks: stealing the ball, scoring, and passing, achieving success rates of 80%, 80%, and 46.7%, respectively.”
During the competition, teams will use off-board computers to perform most of the calculations, receiving the ball’s position and collecting geometric information about the field and referee instructions. The competing teams are divided into 6 robot groups (Group B) and 11 robot groups (Group A), with robots receiving navigation instructions through low-bandwidth radio frequency communication. The diameter and height limits for robots are 180 mm (Group B) and 150 mm (Group A), hence the name “Small Size League.”
The RoboCup Small Size League consists of four stages:
-
Catch a stationary ball somewhere on the field
-
Score by shooting into an empty goal
-
Move the robot to specific coordinates
-
Indirectly score (requires two robots)
Additionally, the challenge requires robots to detect objects on the field, estimate their positions, calculate navigation paths, and record the trajectories they traverse.
Guilherme and his teammates stated in their paper “Building Autonomous Robots for RoboCup Small Size League”: “The SSL competition is a highly dynamic environment with extremely limited robotic resources, requiring solutions to consider trade-offs between size, power consumption, accuracy, and processing speed. We proposed an architecture in our research that enables these robots to autonomously execute basic soccer tasks without receiving any external information.”
Project Hardware
The team used the following hardware in the project:
-
A Jetson Nano Developer Kit for embedded vision and decision-making
-
An omnidirectional mobile robot
-
A Logitech C922 camera for providing monocular vision
-
An inertial sensor for estimating mileage
-
An STM32F767ZI microcontroller unit (MCU) for receiving the target relative position and navigation markers from the Nano, executing low-level control and trajectory estimation using inertial odometry
Figure 2. AI detection process and motion planning of the soccer robot
For more information about the hardware used, please refer to the RobôCIn 2020 team documentation: https://ssl.robocup.org/wp-content/uploads/2020/03/2020_TDP_RoboCIn.pdf
Technical Challenges
In this competition’s visual occlusion challenge, the winning robot must be able to perform various soccer skills, including catching stationary balls, scoring in an empty goal, moving to specific coordinates, and indirectly scoring (passing to another robot).
The robot must be able to complete these skills using only embedded sensing and processing technology. There is no height restriction for this challenge, so the team added an onboard camera, Jetson Nano, and a power board on top of a regular robot.
Figure 3. The modified soccer robot for the visual occlusion challenge (left)
and their original robot (right)
Furthermore, this challenge also requires robots to detect objects on the field, estimate their positions, calculate navigation paths, and record the trajectories they traverse. The SSL soccer competition utilizes external cameras and off-board computers to perceive the environment and send instructions to the robots.
The researchers indicated that the SSL visual architecture “has certain limitations, such as camera field of view, color segmentation, software latency, and communication interruptions, which forced the teams to develop solutions capable of dealing with complex conditions. For example, ball occlusion is a common issue in competitions, where the robot’s projection on the camera image overlaps with the ball. Additionally, the positions of the ball and the robot may misalign, occasionally resulting in undetected or misidentified situations.”
In the SSL competition, the speeds of the robots and the ball can reach 3.7 m/s and 6.5 m/s, respectively, making the competition very fast-paced, thus requiring high-throughput solutions. Moreover, due to size constraints and the use of batteries as power sources, the solutions must have low power consumption. During the competition, the robots also need to make precise long shots and passes, necessitating accurate position estimation.
The team also noted the importance of precise motor control so that the robot can move on the soccer field and maintain accuracy in measuring its position. They needed a method to reduce the discrepancy between the robot’s internal understanding of its position and the actual physical location.
Figure 4. The robot’s camera helps detect objects and provide a view,
allowing the robot to make decisions and plan paths
Project Software and AI
The research team used OpenCV2 and calibration and pose estimation techniques to extract the “intrinsic and extrinsic parameters” of the monocular camera (mounted on the robot). They used SSD MobileNet v2 to detect 2D bounding boxes of objects on the camera frames, and also employed a program to perform linear regression on the bounding box coordinates created by SSD MobileNet to estimate the pre-calibrated camera parameters. This will assign corresponding points at the bottom center of the detected objects (representing the relative positions of the objects to the camera), including points corresponding to the robot.
Results
The team was very satisfied with their robot’s performance in this year’s challenge, mainly including:
-
Catching stationary balls: Out of 15 attempts, the robot was able to stop the ball 12 times when the ball made contact with the dribbler, achieving a success rate of 80%.
-
Scoring: Out of 15 runs, there were 12 goals.
-
Passing: Out of 15 attempts, 7 successful passes were made, with a success rate of 46.7%.
The team has been participating in the RoboCup Small Size League since 2019 and won their first world championship (Group B) in 2022. They are currently three-time Latin American champions. The “Expanded Team Introduction Document for RoboCup 2023 RobôCIn Small Size League” (https://arxiv.org/abs/2307.10018) details the improvements made by the team to win the RoboCup 2023 Small Size League (SSL) Group B championship held in Bordeaux, France, at the end of July. Ultimately, they achieved their goal of winning.
Figure 5. The robot catches a stationary ball (top) and scores (bottom)
Future Plans
Guilherme elaborated on the challenges the team faced during the competition and improvements that could be made for future events. He pointed out that most failures were due to incorrectly detecting objects outside the field, “We are researching a solution to detect the field boundaries and use masks to obscure those objects.”
The team needs faster object detection solutions. Guilherme stated, “Although we can currently perform basic skills, a processing speed of 30 FPS is still low for the SSL environment. In major competitions, the camera’s operating speed is usually 70 FPS.”
The robot executes skills entirely based on the relative positions of detected objects, meaning it does not know its position on the field. Guilherme said, “We believe this information can help optimize performance in soccer tasks while allowing us to avoid penalties, such as preventing the robot from entering the goalkeeper area. We are researching a self-localization algorithm based on Monte Carlo localization (MCL) and will release relevant information in the coming months.”
The team plans to add more features to the robot’s system in the future (such as field line detection, localization algorithms, and path planning), and they will strive to optimize every part of the system to meet these needs.
Additionally, the team continues to research solutions for detecting field boundaries and field lines, as well as estimating the robot’s self-position. They also plan to replace the Jetson Orin Nano with the Jetson Nano to improve the robot’s processing speed. This upgrade will help enhance the team’s competitiveness in the league.
Please visit Developer Forum and GitHub for more information about the team’s original project.
Developer Forum:https://forums.developer.nvidia.com/t/ssl-detector-objects-detection-and-position-estimation-at-the-robocup-small-size-league-ssl/221385
GitHub:https://github.com/jgocm/ssl-detector
You can also explore Jetson Community Projects for more ideas and inspiration from other robot developers: https://developer.nvidia.com/embedded/community/jetson-projects
SIGGRAPH 2023
NVIDIA Exciting Release
SIGGRAPH 2023 | NVIDIA Keynote Highlights Review, Explore the Infinite Future of AI!
Stay tuned…
SIGGRAPH 2023 NVIDIA Keynote Chinese Subtitled Version is Now Live! Scan the QR code on the poster below or click “Read Original” to watch and explore the future of AI with NVIDIA founder and CEO Jensen Huang!