Birdsong Recognition and Analysis using AI

Jun 28, 2022, Luke Horner

Catherine George completed an M.Sc. Data Science student at University of Salford on a hybrid internship for Internet of Things. Her course was part of the Nurture programme which accelerates the graduate journey into the world of data science; Wilder Sensing project is a programme partner. The collaboration allowed students to take advantage of academic support while exploring valuable data and ideas with their commercial partners.

Catherine’s Story

Catherine recounts her experience of joining the project and working on her own solution to the problem of bird call classification.

“From the interview to submitting the dissertation, the entire journey was a great experience. When Lucy Lynch from Eden Smith mentioned Ethos Wilder I was excited because they stood for environmental conservation. The project assigned was to classify birds from their respective sounds. Wilder Sensing co-founder Geoff Carss highlighted a sensor which would record bird sounds in a landscape.

We were assigned to create a Machine Learning model that classifies birds according to their sounds. The data for training and testing the model was obtained from xeno-canto database Xeno Canto, a massive database with a huge collection of bird sounds.

If we take a single recording of a particular bird which spans for a minute, we get a single data point. But audio data holds a lot of information and is multi-dimensional. If we try to chop this one minute recording into small windows, we will have many data points, say 100 data points. This process is called windowing. Now if the windows overlap with each other, this would help retain the bird song in any two data points. So what do we do with all this data now?

Consider we have gathered 100 recordings for each of three different birds; we have almost 30,000 data points for analysis. We did various visualisation techniques (like Principle Component Analysis (PCA) and t-SNE) on this data. This was done and we were able to get three different clusters for the three birds.

Spectograms

We tried a bunch of classification algorithms using these data points. But the accuracy was less than the existing commercially available models. Then we came across the app developed by Cornell University, which converted the audio recordings into an image domain. The audio recordings were converted into spectrograms. Ethos had created Amazon Web Service (AWS) accounts with 2000 credits for the M.Sc. students working on the project. Geoff arranged an Immersion day class with the AWS team and we were introduced to the AWS team along with our project. They guided on the different approaches that could be taken and shared the courses and links for learning Amazon SageMaker. This session with the AWS team was very informative and equipped us to use SageMaker to implement the model. One challenge we initially faced was to convert the mp3 recordings from xeno-canto into a spectrogram. A spectrogram is an image which shows the variation of frequency as a function of time.

Spectrograms were created for each recording using librosa library, a python package for music and audio analysis. Thus an audio classification problem was changed to an image classification problem. AWS has a feature called Amazon Rekognition which could automatically assign labels to the spectrograms based on how they are stored in S3 bucket. A model was built for ten birds using Rekognition. This model could be applied to identify birds in a particular region and would help in rewilding the environment. The birds considered for the model were birds from the UK only, this could however be extended to different birds.”

Wilder Sensing and Academia

At Wilder Sensing we strongly believe in the benefits of working in partnership with academia. Throughout his professional career, Dr L. Trojan, CTO of Wilder Sensing, has consistently worked to establish a strong connection between the world of Private Enterprises and Academia; he believes the two are not separate entities, they must find a way to work together in order to face the tough ecological and environmental challenges ahead of us. Dr L Trojan commented:

“At the end of the day, the whole point of Wilder Sensing is to let the natural world speak to us, so that we can learn how to live in harmony with it.

And the only way for us to achieve our ambitious goals is to forge and nourish a positive bond with academic research, as well as the vibrant ecosystem of other small businesses around us.”

In regards to Catherine outstanding performance and the value of working with academia, Wilder Sensing CEO Geoff Carss has commented:

” We are working with 2 more groups of M.Sc. students, exploring options for other aspects of Wilder Sensing – when you have millions of records for birds, animals, insects etc over 3, 4 or even more years, then you can really start asking interesting questions that were never possible before”

We are very keen in talking to both PhD candidates or Academic professors in any relevant subjects. Our academic and research projects have included a very diverse range of areas including: ecology, biology, bioacoustics and audio engineering, machine learning, Big Data Engineering and Analytics.

If you are interested in the research we conduct at Wilder Sensing and want to find out more, feel free to reach out to us: info@wildersensing.com