Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
133 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

BaboonLand Dataset: Tracking Primates in the Wild and Automating Behaviour Recognition from Drone Videos (2405.17698v3)

Published 27 May 2024 in cs.CV

Abstract: Using drones to track multiple individuals simultaneously in their natural environment is a powerful approach for better understanding group primate behavior. Previous studies have demonstrated that it is possible to automate the classification of primate behavior from video data, but these studies have been carried out in captivity or from ground-based cameras. To understand group behavior and the self-organization of a collective, the whole troop needs to be seen at a scale where behavior can be seen in relation to the natural environment in which ecological decisions are made. This study presents a novel dataset from drone videos for baboon detection, tracking, and behavior recognition. The baboon detection dataset was created by manually annotating all baboons in drone videos with bounding boxes. A tiling method was subsequently applied to create a pyramid of images at various scales from the original 5.3K resolution images, resulting in approximately 30K images used for baboon detection. The tracking dataset is derived from the detection dataset, where all bounding boxes are assigned the same ID throughout the video. This process resulted in half an hour of very dense tracking data. The behavior recognition dataset was generated by converting tracks into mini-scenes, a video subregion centered on each animal; each mini-scene was manually annotated with 12 distinct behavior types, resulting in over 20 hours of data. Benchmark results show mean average precision (mAP) of 92.62\% for the YOLOv8-X detection model, multiple object tracking precision (MOTA) of 63.81\% for the BotSort tracking algorithm, and micro top-1 accuracy of 63.97\% for the X3D behavior recognition model. Using deep learning to classify wildlife behavior from drone footage facilitates non-invasive insight into the collective behavior of an entire group.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (63)
  1. Jeanne Altmann. Observational study of behavior: sampling methods. Behaviour, 49(3-4):227–266, 1974.
  2. Toward a science of computational ethology. Neuron, 84(1):18–31, 2014.
  3. Terrestrial animal tracking as an eye on life and planet. Science, 348(6240):aaa2478, 2015.
  4. Using very-high-resolution satellite imagery and deep learning to detect and count african elephants in heterogeneous landscapes. Remote Sensing in Ecology and Conservation, 7(3):369–381, 2021.
  5. Cnn-based action recognition and pose estimation for classifying animal behavior from videos: a survey. arXiv preprint arXiv:2301.06187, 2023.
  6. Multi-animal pose estimation, identification and tracking with deeplabcut. Nature Methods, 19(4):496–504, 2022.
  7. Sleap: A deep learning system for multi-animal pose tracking. Nature methods, 19(4):486–495, 2022.
  8. Trex, a fast multi-animal tracking system with markerless identification, and 2d estimation of posture and visual fields. Elife, 10:e64000, 2021.
  9. Deepposekit, a software toolkit for fast and robust animal pose estimation using deep learning. Elife, 8:e47994, 2019.
  10. Using deeplabcut for 3d markerless pose estimation across species and behaviors. Nature protocols, 14(7):2152–2176, 2019.
  11. Deep learning tools for the measurement of animal behavior in neuroscience. Current opinion in neurobiology, 60:1–11, 2020.
  12. Deepethogram, a machine learning pipeline for supervised behavior classification from raw pixels. Elife, 10:e63377, 2021.
  13. The mouse action recognition system (mars) software pipeline for automated analysis of social behaviors in mice. Elife, 10:e63720, 2021.
  14. Quantifying behavior to understand the brain. Nature neuroscience, 23(12):1537–1549, 2020.
  15. Pig-posture recognition based on computer vision: Dataset and exploration. Animals, 11(5):1295, 2021.
  16. Postural behavior recognition of captive nocturnal animals based on deep learning: a case study of bengal slow loris. Scientific Reports, 12(1):7738, 2022.
  17. Automated audiovisual behavior recognition in wild primates. Science advances, 7(46):eabi4883, 2021.
  18. Determination of optimal flight altitude to minimise acoustic drone disturbance to wildlife using species audiograms. Methods in Ecology and Evolution, 12(11):2196–2207, 2021.
  19. Quantifying the movement, behaviour and environmental context of group-living animals using drones and computer vision. Journal of Animal Ecology, 92(7):1357–1371, 2023.
  20. Capture, immobilization, and global positioning system collaring of olive baboons (papio anubis) and vervets (chlorocebus pygerythrus): lessons learned and suggested best practices. American journal of primatology, 81(6):e22997, 2019.
  21. Ecological countermeasures to prevent pathogen spillover and subsequent pandemics. Nature Communications, 15(1):2577, Mar 2024.
  22. Future coexistence with great apes will require major changes to policy and practice. Nature Human Behaviour, 8(4):632–643, Apr 2024.
  23. Deep learning methods for multi-species animal re-identification and tracking–a survey. Computer Science Review, 38:100289, 2020.
  24. A survey of machine learning approaches in animal behaviour. Neurocomputing, 491:442–463, 2022.
  25. Animaltrack: A benchmark for multi-animal tracking in the wild. International Journal of Computer Vision, 131(2):496–513, 2023.
  26. Mammalnet: A large-scale video benchmark for mammal recognition and behavior understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13052–13061, 2023.
  27. Animal kingdom: A large and diverse dataset for animal behavior understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19023–19034, 2022.
  28. Automated detection of wildlife using drones: Synthesis, opportunities and constraints. Methods in Ecology and Evolution, 12(6):1103–1114, 2021.
  29. Inferring the rules of social interaction in migrating caribou. Philosophical Transactions of the Royal Society B: Biological Sciences, 373(1746):20170385, 2018.
  30. Habitat and social factors shape individual decisions and emergent group structure during baboon collective movement. elife, 6:e19505, 2017.
  31. Aerial drone observations identified a multilevel society in feral horses. Scientific Reports, 11(1):71, 2021.
  32. Kabr: In-situ dataset for kenyan animal behavior recognition from drone videos. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 31–40, 2024.
  33. 2d/3d pose estimation and action recognition using multitask deep learning, 2018.
  34. Chimpact: A longitudinal dataset for understanding chimpanzee behaviors. Advances in Neural Information Processing Systems, 36, 2024.
  35. Macaquepose: a novel “in the wild” macaque monkey pose dataset for markerless motion capture. Frontiers in behavioral neuroscience, 14:581154, 2021.
  36. Monkeytrail: a scalable video-based method for tracking macaque movement trajectory in daily living cages. Zoological Research, 43(3):343, 2022.
  37. Deepwild: Application of the pose estimation tool deeplabcut for behaviour tracking in wild chimpanzees and bonobos. Journal of Animal Ecology, 92(8):1560–1574, 2023.
  38. The ava-kinetics localized human actions video dataset. arXiv preprint arXiv:2005.00214, 2020.
  39. Panaf20k: a large video dataset for wild ape detection and behaviour recognition. International Journal of Computer Vision, pages 1–17, 2024.
  40. Tracking together: estimating social poses. Nature Methods, 19(4):410–411, 2022.
  41. Norppa: novel ringed seal re-identification by pelage pattern aggregation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1–10, 2024.
  42. Fin-print a fully-automated multi-stage deep-learning-based framework for the individual recognition of killer whales. Scientific reports, 11(1):23480, 2021.
  43. Naps: Integrating pose estimation and tag-based tracking. Methods in Ecology and Evolution, 14(10):2541–2548, 2023.
  44. Towards automated visual monitoring of individual gorillas in the wild. In Proceedings of the IEEE International Conference on Computer Vision Workshops, pages 2820–2830, 2017.
  45. Claire L Witham. Automated face recognition of rhesus macaques. Journal of neuroscience methods, 300:157–165, 2018.
  46. Face recognition: Primates in the wild. In 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS), pages 1–10. IEEE, 2018.
  47. Lemurfaceid: A face recognition system to facilitate individual identification of lemurs. Bmc Zoology, 2:1–14, 2017.
  48. Chimpanzee face recognition from videos in the wild using deep learning. Science advances, 5(9):eaaw0736, 2019.
  49. Alexander Loos and Talat Anand Mohan Kalyanasundaram. Face recognition for great apes: Identification of primates in videos. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1548–1552. IEEE, 2015.
  50. Primate face identification in the wild. In PRICAI 2019: Trends in Artificial Intelligence: 16th Pacific Rim International Conference on Artificial Intelligence, Cuvu, Yanuca Island, Fiji, August 26-30, 2019, Proceedings, Part III 16, pages 387–401. Springer, 2019.
  51. idtracker.ai: tracking all individuals in small or large collectives of unmarked animals. Nature Methods, 16(2):179–182, Feb 2019.
  52. Shirley C Strum. Darwin’s monkey: why baboons can’t become human. American journal of physical anthropology, 149(S55):3–23, 2012.
  53. Baboon Ecology: African Field Research. University of Chicago Press, Chicago, 1970.
  54. J Wallis. Papio anubis. the iucn red list of threatened species 2020: e. t40647a17953200, 2020.
  55. Guy Cowlishaw. Vulnerability to predation in baboon populations. Behaviour, 131(3-4):293–304, 1994.
  56. Ultralytics YOLO, January 2023.
  57. Hollywood in homes: Crowdsourcing data collection for activity understanding. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14, pages 510–526. Springer, 2016.
  58. Bytetrack: Multi-object tracking by associating every detection box. In European conference on computer vision, pages 1–21. Springer, 2022.
  59. Bot-sort: Robust associations multi-pedestrian tracking. arXiv preprint arXiv:2206.14651, 2022.
  60. Quo vadis, action recognition? a new model and the kinetics dataset. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6299–6308, 2017.
  61. Slowfast networks for video recognition. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6202–6211, 2019.
  62. Christoph Feichtenhofer. X3d: Expanding architectures for efficient video recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 203–213, 2020.
  63. A satellite perspective on the movement decisions of african elephants in relation to nomadic pastoralists. Remote Sensing in Ecology and Conservation, 8(6):841–854, 2022.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets