Second paper out this week 😱😱 We present YOLO-Behaviour, a flexible, easy to implement, robust framework for automated behavioural annotation from videos, published in @methodsinecoevol.bsky.social!! [paper] doi.org/10.1111/2041... [documentation] alexhang212.github.io/YOLO_Behavio... - ThreadSky

alexhhchan.bsky.social • 15 days ago

Second paper out this week 😱😱

We present YOLO-Behaviour, a flexible, easy to implement, robust framework for automated behavioural annotation from videos, published in @methodsinecoevol.bsky.social!!

[paper] http://doi.org/10.1111/2041-210X.14502
[documentation] https://alexhang212.github.io/YOLO_Behaviour_Repo/index.html

Comments

canepaoneto.bsky.social•2 days ago

Do you think, it can be implemented for audio (spectrograms) data?
To characterize/monitor acoustic behaviour?

alexhhchan.bsky.social•1 day ago

In theory the object detector just looks for some pattern in an image then tries to localise it, so I guess yes? If a specific call is quite visually distinct on the spectrogram, I don't see why not. Would be cool if it would works!

canepaoneto.bsky.social•1 day ago

Many thanks!!

alexhhchan.bsky.social•15 days ago

Using the widely used object detector YOLO, we show that if you prepare your dataset in a slightly creative way (e.g drawing a box around a Siberian Jay's beak touching food), the model can be used to detect behviours from videos effectively

alexhhchan.bsky.social•15 days ago

Using the framework to extract coded behavioural metrics (e.g feeding rate, visit rates), we also show that the rates extracted are highly correlated

alexhhchan.bsky.social•15 days ago

If the framework is not accurate enough (no models are perfect!), the framework can also be used as a "hybrid approach" to first use identify events using the framework then manually confirm detections, saving valuable annotation time.

alexhhchan.bsky.social•15 days ago

More video results: https://youtu.be/YytSl9gEOHI?si=kcS5pdN_ugLvX49-

alexhhchan.bsky.social•15 days ago

If you are trying out the framework and have bugs/feedback/ comments on the documentation, please let me know. Hope the framework is useful and have fun!

alexhhchan.bsky.social•15 days ago

Thank you all co-authors, Prasetia, Harald, Johanna, Jana, Britta, @jj255.bsky.social, @willpearse.bsky.social, @itchyshin.bsky.social, Terry, @michael-griesser.bsky.social, Andrea, Saverio and Kano! @cbehav.bsky.social @uni-konstanz.de @mpi-animalbehav.bsky.social

jotatx.bsky.social•11 days ago

Hey Alex. I am using yolov11l model to detect action in sport. Will read your paper since we overlap on some of the main logic blocks.
Your paper is quite interesting

Comments

Posting Rules

Reply