In the wild, being able to identify bird species by their appearance is already quite impressive, but being able to recognize them solely by their sounds is even more remarkable. Unless you are a seasoned bird enthusiast, identifying birds by sound alone can be quite challenging.
A group of researchers from the University of Marburg in Germany has designed a method for quickly identifying bird species and monitoring local biodiversity. They utilized a portable device equipped with the NVIDIA Jetson Nano Developer Kit to capture audio.
According to the researchers, the Bird@Edge project is an edge AI system that operates on embedded edge devices running in distributed systems, enabling efficient and continuous assessment of soundscapes recorded in forests.
About the Bird@Edge Project
The researchers used multiple ESP32-based microphones (Bird@Edge microphones) to transmit bird sounds to local Bird@Edge sites for species identification. Each microphone has a Wi-Fi transmission range of 50 meters.
This device is deployed in local forests, where researchers carefully considered the Wi-Fi transmission range of the audio streams and the performance of the GPU, ensuring that each site could connect up to 10 microphones. The audio results obtained at the sites are transmitted to a backend cloud for further analysis by biodiversity researchers in the team.
Researcher Jonas Höchst stated, “There is still some additional processing power available on the Jetson Nano GPU running the model. We can also use a custom low-power profile for the Jetson Nano, indicating that the GPU itself still has processing capacity available.”
The Bird@Edge project can quickly identify bird species solely through sound. The researchers noted, “From the moment the microphone records the bird calls to visualizing the bird calls takes only a few seconds, whereas previous methods required several days.”
Figure 1. The Bird@Edge system captures bird sounds in the forest and transmits the information to an edge server for analysis.
Bird@Edge System Software and AI
To identify and classify bird species based on recordings, the team developed a deep neural network (DNN) based on the EfficientNet-B3 architecture and trained it using TensorFlow, then optimized and deployed the model using NVIDIA TensorRT and NVIDIA DeepStream SDK.
This approach enables the team to support multiple real-time streaming AI applications built on the Jetson Nano GPU. After training, the algorithm can recognize any of the 82 bird species found in the University of Marburg forest and can be expanded to include more species as needed.
The Bird@Edge virtual drive program running on the Jetson Nano is responsible for monitoring the microphones and instantiating and running the AI processing workflow. The monitoring results are transmitted to the Bird@Edge server via cellular networks and stored in an InfluxDB database. By connecting to Grafana, researchers can visualize the data on the web.
The team demonstrated that their DNN “outperformed the state-of-the-art BirdNET neural network on several datasets, achieving an average recognition accuracy of 95.2% for soundscape recordings.”
The system can also identify less common bird species. Höchst mentioned, “We are currently collaborating with a volunteer ornithologist who is collecting calls from birds that are less frequently detected due to their rarity, which will further improve our work.”
Bird@Edge System Hardware
The Bird@Edge tools are based on embedded edge devices running in distributed systems, including:
-
Bird@Edge Microphone: A Knowles microphone compatible with ESP32 SOC, capable of communicating via Bluetooth or Wi-Fi, along with a low-cost rechargeable battery.
-
Bird@Edge Site: Captures recordings from incoming data streams and performs inference on each deployed microphone. The site is a small portable case containing a Jetson Nano, Wi-Fi, modem, voltage converter, and a small solar charger.
-
Bird@Edge Server: Recordings are transmitted via Grafana and dynamically presented in a visual format to generate insights.
Given that Bird@Edge sites are set up in forests, the team needed to ensure that each site could operate efficiently without frequent battery recharges, improving energy efficiency by coordinating the hardware of the microphones and edge sites.
The team created a high-efficiency energy profile with a power consumption of only 3.16 watts, allowing the site to run for nearly two weeks without recharging the battery. The site relies on connected solar panels for continuous operation. The research group found that even with an increased number of connected microphones, the power consumption of the site did not change significantly.
Video 1. Researchers discuss the Bird@Edge project
Tracking Regional Biodiversity
The research team believes this project is a great way to more easily track regional biodiversity. Previously, if researchers wanted to understand the health of a habitat, completing the tedious task of manually transcribing recordings could take months, along with additional months to collect, analyze, and present their results.
Thanks to the work of Höchst and his team, environmental scientists worldwide can now see which bird species are present locally in seconds and gain real-time insights into the state of the ecosystem.
The researchers stated, “Birds are important to many ecosystems as they connect habitats, resources, and biological processes, making them important early warning bioindicators of ecosystem health.”
Conclusion
Can Bird@Edge be used for broader applications? Höchst mentioned that the team is considering how to commercialize their system: “The challenge we face is to transform a manually assembled case that performs well in a controlled environment into a product that can be used at scale without regular maintenance.”
“We have gained experience in designing and producing hardware for automatic detection of VHF telemetry signals and are developing stable and reliable hardware designs that are easy to build in preparation for upcoming large-scale studies.”
Additionally, since the initial release of their paper, the team has been developing a web service that allows a wider audience to use the recording system.
Höchst explained, “On one hand, this allows users to utilize their available audio recording devices and upload files to our cloud-based web service. At the same time, users can receive direct feedback and view spectrograms of the uploaded files, manually verify results, or report classification errors, thereby improving the underlying machine learning model.”
All software and hardware components used in the Bird@Edge project are open source and can be found at BirdEdge on GitHub (https://github.com/umr-ds/BirdEdge).
Scan the poster QR code, or click “Read the original text” to watch the replay of NVIDIA founder and CEO Jensen Huang’s keynote speech at GTC23!
