BirdCLEF+ 2025: Bioacoustic ML Benchmark

• BirdCLEF+ 2025 Challenge is an applied machine learning benchmark focused on robust identification and classification of diverse species in complex soundscapes.
• It requires framewise, multi-label classification of 206 species using 5-second soundscape intervals under a 90-minute CPU-only inference limit.
• The challenge promotes innovations like transfer learning and spectrogram token skip-gram methods to balance high macro ROC-AUC with real-time deployment efficiency.

The BirdCLEF+ 2025 Challenge is an applied machine learning benchmark focused on robust identification and classification of a wide range of species—including birds, mammals, insects, and amphibians—across complex, real-world soundscape recordings under strict computational constraints. The challenge builds on the legacy of previous BirdCLEF tasks, explicitly extending the focus from avian taxa to broader bioacoustic monitoring and operationalizing methods for scalable, automated acoustic biodiversity assessment in natural soundscapes. BirdCLEF+ 2025 imposes a 90-minute CPU-only inference limit for a test set of thousands of one-minute soundscape files, forcing participants to innovate in both algorithmic efficiency and robust out-of-domain generalization.

1. Task Definition, Scope, and Constraints

The BirdCLEF+ 2025 Challenge requires framewise (5-second interval) multi-label classification of 206 species, comprising birds, mammals, insects, and amphibians, using real soundscape recordings simulating field operation scenarios. The task design introduces strict operational constraints:

• Inference Deadline: 90 minutes, CPU-only, for the entire test corpus (∼700 one-minute files), effectively prohibiting use of resource-heavy deep ensembles or non-optimized architectures.
• Multi-Taxa, Multi-Label: Models must handle varied taxa, acoustic morphologies, and significant class imbalance (both common and rare species).
• Strong Distribution Shift: Provided training data covers different biogeographic regions and typically cleaner “focal” conditions, while test soundscapes exhibit higher noise, overlapping vocalizations, and differing species occurrence patterns.

This paradigm enforces focus on both system efficiency (practical deployment feasibility) and generalization properties under distribution shift (Miyaguchi et al., 11 Jul 2025).

2. Methodological Advances and Baselines

Participants employed two primary methodological streams:

(a) Transfer Learning with Pre-Trained Bioacoustic Models

• Transfer Learning Strategy: Adapt off-the-shelf bird and animal sound recognition models by training slim classification heads on top of windowed embeddings, thereby leveraging learned representation spaces (Miyaguchi et al., 11 Jul 2025).
• Key Backbones Tested:
  • BirdNET
  • Perch
  • BirdSetEfficientNetB1
  • BirdSetConvNeXT
  • HawkEars
  • RanaSierraeCNN
  • YAMNet

Embeddings are first averaged within each 5-second window, with the classifier head retrained using task-specific labels. Model and embedding characteristics (e.g., dimensionality, source clip lengths) are carefully matched to the challenge granularity.

(b) Spectrogram Token Skip-Gram (STSG) Pipeline

• Discrete Token Representation: Converts Mel-spectrogram frames into discrete “spectrogram tokens” via large-scale Faiss K-means clustering (typically using 16k centroids), after dimensionality reduction (e.g., PCA to 128).
• Unsupervised Contextual Embedding: Learns token context representations using a Word2Vec skip-gram approach:

$$l_{\text{sgns}} = -\log(\sigma(\vec{w} \cdot \vec{c})) - \sum_{i=1}^{k} \log(\sigma(-\vec{w} \cdot \vec{c}_{N,i}))$$

where $(\vec{w}, \vec{c})$ are token and context embeddings.

• Classification: For each 5-second interval, token embeddings are averaged and input to a linear or shallow neural classifier; optionally, a student-teacher paradigm is explored via KL divergence minimization against predictions from Perch.

This approach (“STSG”, Editor's term) decouples computational cost from the depth of acoustic modeling by amortizing feature extraction and restricting learning/final inference to static embedding spaces (Miyaguchi et al., 11 Jul 2025).

3. Performance Metrics and Outcomes

Evaluation centers on macro ROC-AUC computed per species on public and private test sets. The table below summarizes performance and efficiency of notable models from (Miyaguchi et al., 11 Jul 2025):

Model	Public ROC-AUC	Private ROC-AUC	Inference Time (700 files)
BirdSetEfficientNetB1	0.810	0.778	~26 min
Perch (TFLite)	0.729	0.711	~16 min
BirdNET	0.719	0.718	~29–72 min
STSG (v2.1, token model)	0.559	0.520	~6 min

Key findings:

• Optimized transfer learning: BirdSetEfficientNetB1 achieved the best leaderboard scores (0.810/0.778) while meeting the runtime constraint.
• STSG tradeoff: Spectrogram token-based methods yielded lower macro ROC-AUC but vastly improved inference efficiency, with the fastest (STSG) requiring only ~6 minutes for the full test set.
• TFLite conversion: Applying TensorFlow Lite to existing models (e.g., Perch) resulted in a ~10x CPU speedup, crucial for meeting inference limits.

4. Engineering Innovations for Inference Efficiency

The computational bottleneck dictated several approaches:

• Model Graph Pruning and Conversion: Only inference-relevant nodes retained; TFLite used for optimal execution profiles.
• Averaging and Alignment Heuristics: For models with non-matching stride/window conventions (e.g., BirdNET’s 3s window, test set’s 5s intervals), careful averaging and alignment ensured fidelity while minimizing redundant computation.
• Token Pipeline Modularity: The STSG pipeline presents a fully modular, highly parallelizable process: tokenization and embedding learning occur offline, and inference reduces to simple embedding lookups and linear passes.

A plausible implication is that static, token-based pipelines may unlock orders-of-magnitude speedups for large-scale deployments, at the expense of some recognition granularity.

5. Methodological Implications and Future Directions

BirdCLEF+ 2025 surfaced trade-offs between high-accuracy, model-centric deep architectures and lightweight, token-based distributed representations. The challenge serves as a testbed for:

• Efficient Large-Scale Deployment: Demonstrates that state-of-the-art bioacoustic models, properly pruned and CPU-optimized, can deliver >0.8 macro ROC-AUC in real-time constraints.
• Viability of Unsupervised Token Embedding: STSG and related pipelines illustrate that “acoustic tokenization” is practical, modular, and can support broader instrument monitoring with further research.
• Transferability and Adaptation: Transfer learning remains the dominant paradigm; domain-specific backbone pretraining outperforms generic audio models, but models still exhibit performance drops under severe distributional and class-shift conditions (Hamer et al., 2023).
• Open Research Questions: Further domain adaptation, improved rare class detection, and hybridization of deep and token-based models are highlighted by observed gaps in species-level, few-shot, and OOD generalization performance.

6. Position Among Related Benchmarks and Community Impact

BirdCLEF+ 2025 continues themes of robustness and generalization central to recent benchmarks like BIRB (Hamer et al., 2023): use of public, citizen-science training data; evaluation on expert-annotated, “in the wild” soundscapes; and multi-taxa, multi-region targets. Key advances over previous BirdCLEF editions include the strict computational budgeting, broader taxonomic scope, and encouragement of reproducibility through open source codebases and leaderboards.

Recent methods from BirdCLEF+ have informed broader trends in bioacoustic ML, stimulating methodological innovation in representation learning, efficient deployment, and robust adaptation in challenging ecological monitoring tasks.

7. Prospective Research Directions

Based on challenge results, salient axes for further exploration include:

• Improved tokenization and embedding schemes (e.g., spectro-temporal patterns, unsupervised motif learning)
• Teacher-student transfer and lightweight sequential models for context exploitation without runtime penalties
• Tighter coupling of inference-efficient pipelines with metadata priors (e.g., co-occurrence, location, diel cycles)
• Evaluation under adversarial, distributional, and multi-modal test beds akin to BIRB for tracking genuine generalization advances.

Open-source implementations and benchmarking protocols are expected to persist as central enablers of machine learning progress on soundscape-scale biodiversity monitoring.