Self-Supervised Learning for Few-Shot Bird Sound Classification (2312.15824v4)
Abstract: Self-supervised learning (SSL) in audio holds significant potential across various domains, particularly in situations where abundant, unlabeled data is readily available at no cost. This is pertinent in bioacoustics, where biologists routinely collect extensive sound datasets from the natural environment. In this study, we demonstrate that SSL is capable of acquiring meaningful representations of bird sounds from audio recordings without the need for annotations. Our experiments showcase that these learned representations exhibit the capacity to generalize to new bird species in few-shot learning (FSL) scenarios. Additionally, we show that selecting windows with high bird activation for self-supervised learning, using a pretrained audio neural network, significantly enhances the quality of the learned representations.
- Dan Stowell, “Computational bioacoustics with deep learning: a review and roadmap,” PeerJ, vol. 10, pp. e13152, 2022.
- “Overview of birdclef 2020: Bird sound recognition in complex acoustic environments,” in Conference and Labs of the Evaluation Forum, 2020.
- “BirdNET: A deep learning solution for avian diversity monitoring,” Ecological Informatics, 2021.
- “Learning to detect an animal sound from five examples,” Ecological Informatics, 2023.
- “Global birdsong embeddings enable superior transfer learning for bioacoustic classification,” arXiv:2307.06292, 2023.
- “Metaaudio: A few-shot audio classification benchmark,” in International Conference on Artificial Neural Networks, 2022.
- “A simple framework for contrastive learning of visual representations,” in International conference on machine learning, 2020.
- “Barlow twins: Self-supervised learning via redundancy reduction,” in International Conference on Machine Learning, 2021.
- “FroSSL: Frobenius Norm Minimization for Self-Supervised Learning,” arXiv:2310.02903, 2023.
- “Masked siamese networks for label-efficient learning,” in European Conference on Computer Vision, 2022.
- “Dinov2: Learning robust visual features without supervision,” arXiv:2304.07193, 2023.
- “Panns: Large-scale pretrained audio neural networks for audio pattern recognition,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2020.
- “Supervised contrastive learning,” Advances in neural information processing systems, 2020.
- “Regularized Contrastive Pre-training for Few-shot Bioacoustic Sound Detection,” in ICASSP IEEE International Conference on Acoustics, Speech and Signal Processing, 2024.
- “Contrastive learning of general-purpose audio representations,” in ICASSP IEEE International Conference on Acoustics, Speech and Signal Processing, 2021.
- “Unsupervised learning of semantic audio representations,” in ICASSP IEEE international conference on acoustics, speech and signal processing, 2018.
- “Unsupervised contrastive learning of sound event representations,” in ICASSP IEEE International Conference on Acoustics, Speech and Signal Processing, 2021.
- “BYOL for audio: Exploring pre-trained general-purpose audio representations,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2022.
- “Data augmentation approaches for improving animal audio classification,” Ecological Informatics, 2020.
- “Specaugment on large scale datasets,” in ICASSP IEEE International Conference on Acoustics, Speech and Signal Processing, 2020.
- “Understanding dimensional collapse in contrastive self-supervised learning,” arXiv:2110.09348, 2021.