Today, we’re introducing NatureLM-audio: the first large audio-language model tailored for understanding animal sounds. https://arxiv.org/abs/2411.07186 🧵👇
Comments
Log in with your Bluesky account to leave a comment
1/ Traditional ML methods in bioacoustics struggle with species-specific data, while general-purpose audio models lack deep understanding of animal vocalizations. NatureLM-audio is trained to solve a wide range of bioacoustic tasks across species—all with natural language prompts
2/ Built from bioacoustic archives & enriched with speech and music data, NatureLM-audio enables zero-shot classification of animal vocalizations. Without any fine-tuning, it can classify sounds of thousands of species from birds to whales. 🌎🎶
4/ NatureLM-audio can even predict species it’s never “heard” before. The model correctly identified new species 20% of the time—a huge step forward from the random rate of 0.5%.
6/ With the development of NatureLM-audio, we aim to address some of the persistent challenges in using ML in bioacoustics. Looking ahead, we'll add new data types to support multi-modal analysis for an even richer understanding of animal communication.
i kept on bothering felon the immigrant elephant tusk about space travels should not be allowed before we understand all animals on the Earth,to in unison decide if we should stay on this planet or go to space. we need to understand what Animals have to say about that https://www.youtube.com/watch?v=o8GA2w-qrcg
Comments
- Predicting life stages in birds (chicks, juveniles, nestlings) 🐣
- Distinguishing bird call types 🐦
- Captioning bioacoustic audio 🎙️
- Counting zebra finch individuals in a recording 🪶
Koko the gorilla - Message for Humans
https://www.youtube.com/watch?v=cfj1o9kYgzw