BirdMAE: Masked Autoencoder for Bird Sounds

• BirdMAE is a masked autoencoder-based bioacoustic foundation model that leverages tailored pretraining and specialized pipelines for fine-grained bird sound classification.
• It adapts the masked autoencoder paradigm with modifications such as a ViT-based decoder, mixup augmentation, and prototypical pooling to preserve discriminative patch-level features.
• Empirical results on the BirdSet benchmark demonstrate that BirdMAE outperforms general-purpose models, achieving significant improvements in multi-label bird classification.

BirdMAE, presented as Bird-MAE in the original report and as BirdMAE in a later comparative review, is a masked-autoencoder-based bioacoustic foundation model specialized for bird-sound analysis. It adapts the masked autoencoder paradigm to fine-grained bird classification by revising not only the pretraining corpus but also the training pipeline, fine-tuning procedure, and frozen-feature utilization. The primary motivation is that general-purpose audio MAE models do not generalize well to bird sounds, where subtle inter-species differences and high intra-species acoustic variability are central. In the BirdSet setting, BirdMAE is reported to achieve new state-of-the-art results in multi-label classification, while later comparative analysis identifies it as the strongest model on BirdSet among the reviewed foundation models (Rauch et al., 17 Apr 2025, Schwinger et al., 2 Aug 2025).

1. Problem setting and model rationale

BirdMAE is situated in the broader literature on self-supervised audio representation learning, but its design target is specifically fine-grained audio domains rather than general-domain audio classification. The original study argues that bird-sound classification exposes the limitations of general-domain Audio-MAE models because the task requires distinguishing subtle inter-species differences while managing high intra-species acoustic variability. The stated conclusion is that bridging this domain gap requires more than domain-specific pretraining data; adapting the entire training pipeline is crucial (Rauch et al., 17 Apr 2025).

This design rationale distinguishes BirdMAE from a simple transfer of Audio-MAE to bird audio. The model is described as a domain-specialized MAE trained on BirdSet and related Xeno-Canto-derived corpora, with downstream mechanisms intended to recover local spectral-temporal structure that would otherwise be underutilized. The later review places BirdMAE among large-scale pretrained bioacoustic foundation models and reports that it achieves the best performance on the BirdSet benchmark, while also noting that benchmark transferability is task-dependent: on BEANS, BEATs$_{NLM}$ is described as slightly better (Schwinger et al., 2 Aug 2025).

A common misconception in this area is that a single self-supervised model should transfer uniformly across bioacoustic tasks. The available results do not support that view. The original paper explicitly states that a one-size-fits-all SSL model is suboptimal for fine-grained audio domains, and the review likewise emphasizes model-selection differences between BirdSet and BEANS (Rauch et al., 17 Apr 2025, Schwinger et al., 2 Aug 2025).

2. Architecture and signal representation

BirdMAE follows an asymmetric masked autoencoder design built on a Vision Transformer. In the base recipe, the input is 5 s of audio at 32 kHz, converted to a $128 \times 512$ log-mel spectrogram. With a $16 \times 16$ patch size, this yields $H \times W = 8 \times 32$ patches. The encoder is a standard Vision Transformer mapping

$$x \in \mathbb{R}^{8 \times 32 \times (16 \times 16)} \rightarrow h_\beta(x) \in \mathbb{R}^{8 \times 32 \times D},$$

and the decoder is a lightweight ViT-based decoder that reconstructs only masked patches (Rauch et al., 17 Apr 2025).

The original work reports several bird-specific architectural modifications. The decoder is switched from Swin to ViT for stability on high masking ratios, the spectrogram width is reduced to 512 frames to focus on short bird calls, and mixup ($\alpha = 0.3$) is introduced during pretraining to encourage invariance (Rauch et al., 17 Apr 2025).

The comparative review describes the most widely used BirdMAE variant as using a ViT “Large” backbone. In that configuration, the input is a 5 s mel-spectrogram (128 mel bands $\times \approx 500$ time frames at 32 kHz) split into non-overlapping patches such as $16 \times 16$ frames, with each patch projected to a 1 024-dimensional embedding. The encoder consists of 24 transformer blocks, each with 1 024-dimensional hidden states, 16 attention heads, and a 4 096-dimensional feed-forward layer. Only unmasked patch embeddings are processed. The decoder is a lightweight transformer of 8 blocks with 512-dimensional hidden size (Schwinger et al., 2 Aug 2025).

For downstream use, BirdMAE can incorporate prototypical pooling. The review states that, in fine-tuning, BirdMAE may replace the standard CLS-token readout with a prototypical pooling layer that aggregates patch embeddings weighted by learned species prototypes before a final linear classifier. This is aligned with the original paper’s emphasis on exploiting patch-level structure rather than relying solely on a global pooled representation (Schwinger et al., 2 Aug 2025, Rauch et al., 17 Apr 2025).

3. Pretraining data and masked reconstruction objective

The original study organizes BirdMAE pretraining around a dataset termed “XCL-1.6M.” This corpus is drawn from the Xeno-Canto subset in BirdSet and is described as consisting of 528 K recordings and $\sim 3$ M events, curated to a maximum number of events per species and file to yield 1.6 M events spanning 9 735 species. Labels are weak, using file-level multi-species tags (Rauch et al., 17 Apr 2025).

The later review summarizes the pretraining data somewhat differently, stating that BirdMAE is pretrained on the BirdSet Xeno-Canto training split with $\approx 712$ K recordings, 9 734 species, and $128 \times 512$0 h total. Both descriptions identify the pretraining source as large-scale BirdSet/Xeno-Canto bird audio, but they correspond to different corpus summaries or curation stages (Schwinger et al., 2 Aug 2025).

The pretraining objective is masked spectrogram reconstruction. In the original paper, 75 % of patches are masked at random, reduced from 80 % in Audio-MAE, and reconstruction loss is computed on the masked set $128 \times 512$1:

$128 \times 512$2

The review presents the same objective as Masked Spectrogram Patch Modeling (MSPM) with a mean squared error over masked patches:

$128 \times 512$3

where $128 \times 512$4 is chosen uniformly with $128 \times 512$5 (Rauch et al., 17 Apr 2025, Schwinger et al., 2 Aug 2025).

The original paper also reports a concrete shift in pretraining recipe relative to Audio-MAE, including 150 epochs, batch size 1024, mixup 0.3, learning rate $128 \times 512$6, and BirdSet spectrogram normalization with mean and standard deviation $128 \times 512$7 (Rauch et al., 17 Apr 2025).

Recipe	Audio-MAE baseline	BirdMAE
Dataset	2 M AS	1.6 M XCL
Decoder	Swin	ViT
Epochs	32	150
Masking	0.80	0.75
Batch	512	1024
Mixup	0.0	0.3
LR	2e-4	2e-4

These changes are important because the original paper attributes BirdMAE’s gains not merely to domain-specific data, but to the adaptation of the full pretraining recipe to bird sounds. A plausible implication is that the model’s performance depends on the co-design of spectrogram geometry, masking ratio, decoder choice, and augmentation strategy rather than on dataset substitution alone (Rauch et al., 17 Apr 2025).

4. Fine-tuning, probing, and prototype-based readout

The original work defines a full fine-tuning recipe denoted “M2.” It combines domain augmentations at the waveform and spectrogram levels: time-shift, background noise mixing, gain, multi-label mixup, and freq/time masking. The downstream classifier is a prototypical classification head that learns $128 \times 512$8 prototypes per class, $128 \times 512$9 with $16 \times 16$0, computes cosine similarities

$16 \times 16$1

then max-pools over spatial indices,

$16 \times 16$2

and forms the class logit

$16 \times 16$3

This head is explicitly designed to retain discriminative patch-level evidence in the encoder output (Rauch et al., 17 Apr 2025).

For frozen-feature use, denoted “M3,” the encoder $16 \times 16$4 is frozen and only a lightweight probe is trained. The original paper states that linear, MLP, and attentive probing use global-pooled $16 \times 16$5-dimensional embeddings, whereas prototypical probing freezes $16 \times 16$6 and trains only the prototypes $16 \times 16$7 and linear $16 \times 16$8. In the few-shot form, for support examples $16 \times 16$9 of class $H \times W = 8 \times 32$0, the prototype is

$H \times W = 8 \times 32$1

and a test example is scored by cosine similarity:

$H \times W = 8 \times 32$2

The paper states that this matches the full fine-tuning prototypical head but without updating the encoder (Rauch et al., 17 Apr 2025).

The review describes two lightweight transfer protocols after pretraining. In linear probing, one extracts the single CLS-token embedding $H \times W = 8 \times 32$3 or a global average of all patch embeddings and trains a linear classifier

$H \times W = 8 \times 32$4

In attentive probing, one extracts the full set of patch embeddings $H \times W = 8 \times 32$5, introduces a trainable query vector $H \times W = 8 \times 32$6 or a small multi-head attention module, computes

$H \times W = 8 \times 32$7

and then applies a linear classifier. The review notes that this adds only $H \times W = 8 \times 32$8 parameters and often recovers more of the encoder’s fine-grained features (Schwinger et al., 2 Aug 2025).

Taken together, these protocols show that BirdMAE’s downstream behavior is strongly tied to how patch embeddings are aggregated. The original paper states directly that masked reconstruction alone underutilizes spatial patch features for discriminative tasks, and that prototypical pooling or probing recovers that signal (Rauch et al., 17 Apr 2025).

5. Benchmark performance on BirdSet

The principal empirical target in the original paper is BirdSet’s multi-label classification benchmark, reported in MAP % across eight tasks: POW, HSN, PER, NES, UHH, NBP, SSW, and SNE. Under full fine-tuning, BirdMAE improves substantially over both Audio-MAE and the supervised Perch baseline (Rauch et al., 17 Apr 2025).

Model	Average MAP
Audio-MAE B	34.91
Perch (sup.)	36.61
Bird-MAE B/16	39.90
Bird-MAE L/16	47.62
Bird-MAE H/16	46.53

The detailed task-level table in the original paper reports that Bird-MAE L/16 reaches 55.26 on POW, 55.28 on HSN, 34.64 on PER, 41.50 on NES, 30.17 on UHH, 71.69 on NBP, 40.82 on SSW, and 33.82 on SNE, for an average of 47.62. The same source states that Bird-MAE-L outperforms Perch by +14.1 p on POW, +13.9 p on PER, and sets new SOTA on all eight tasks (Rauch et al., 17 Apr 2025).

The comparative review reports complementary benchmark figures under a different evaluation framing. It states that, in the original BirdMAE paper and its re-evaluation, original cmAP$H \times W = 8 \times 32$9 on BirdSet multi-label soundscape classification is 44.0 % for BirdMAE versus $$x \in \mathbb{R}^{8 \times 32 \times (16 \times 16)} \rightarrow h_\beta(x) \in \mathbb{R}^{8 \times 32 \times D},$$0 % for ConvNext$$x \in \mathbb{R}^{8 \times 32 \times (16 \times 16)} \rightarrow h_\beta(x) \in \mathbb{R}^{8 \times 32 \times D},$$1 and $$x \in \mathbb{R}^{8 \times 32 \times (16 \times 16)} \rightarrow h_\beta(x) \in \mathbb{R}^{8 \times 32 \times D},$$2 % for Perch. In its own AUROC-based probing comparison, BirdMAE reaches an average AUROC of 0.8654 with attentive probing, compared with 0.8575 for ConvNext$$x \in \mathbb{R}^{8 \times 32 \times (16 \times 16)} \rightarrow h_\beta(x) \in \mathbb{R}^{8 \times 32 \times D},$$3 under linear probing, 0.8563 for Perch under linear probing, 0.8105 for AudioMAE, and 0.8228 for BEATs (Schwinger et al., 2 Aug 2025).

These results establish two distinct points. First, BirdMAE is highly competitive in BirdSet fine-tuning. Second, even under frozen-encoder adaptation, its representations remain strong when the probe is allowed to exploit patch-level information rather than only a single global token.

6. Frozen representations, few-shot behavior, and broader interpretation

A major contribution of the original paper is its analysis of frozen MAE representations. On the HSN task, the reported MAP % values are 8.77 for Audio-MAE B with linear probing, 10.45 with MLP, 19.37 with attentive probing, and 19.42 with prototypical probing. For Bird-MAE B, the corresponding values are 13.06, 17.23, 42.66, and 43.84. For Bird-MAE L, they are 15.22, 18.71, 41.51, and 49.03, while full fine-tune reaches 55.28 in the prototypical-head setting (Rauch et al., 17 Apr 2025).

The original paper states that prototypical probing improves over linear by +33.8 p on Bird-MAE-L and closes gap to fine-tuning to 6.25 p on HSN. Across all eight tasks, prototypical probing is reported to reduce the average gap to full fine-tuning to $$x \in \mathbb{R}^{8 \times 32 \times (16 \times 16)} \rightarrow h_\beta(x) \in \mathbb{R}^{8 \times 32 \times D},$$4 p. It is also described as parameter-efficient (<0.5 M params) and as outperforming linear or MLP probing by up to +37 p MAP (Rauch et al., 17 Apr 2025).

Few-shot performance follows the same pattern. With Bird-MAE-L frozen, the few-shot benchmark reports the following MAP ranges:

Shots per class	Linear	Prototypical
1	5–10	20–40
5	10–20	35–50
10	12–25	45–55

The same section states that with just 10 shots, prototypical probing reaches within a few percentage points of full-data prototyping on all test sets, where full-data prototyping lies in the 49–55 MAP range (Rauch et al., 17 Apr 2025).

The review generalizes this observation beyond the original experiments. It concludes that transformer-based models require attentive probing to extract the full performance of their representations, and specifically notes that BirdMAE performs best on BirdSet, whereas BEATs$$x \in \mathbb{R}^{8 \times 32 \times (16 \times 16)} \rightarrow h_\beta(x) \in \mathbb{R}^{8 \times 32 \times D},$$5 is slightly better on BEANS. It also notes that ConvNext$$x \in \mathbb{R}^{8 \times 32 \times (16 \times 16)} \rightarrow h_\beta(x) \in \mathbb{R}^{8 \times 32 \times D},$$6 and Perch remain competitive in linear probing settings for passive acoustic monitoring classification tasks of BirdSet (Schwinger et al., 2 Aug 2025).

This comparative picture suggests a more precise characterization of BirdMAE’s place in the literature. BirdMAE is not simply a generic audio foundation model applied to bird sounds; it is a domain-specialized MAE whose reported effectiveness depends on three coupled design choices: large-scale bird-centric pretraining, masked reconstruction with bird-specific training modifications, and downstream heads or probes that preserve discriminative patch structure. The original authors explicitly extend this implication to other fine-grained audio domains, including bioacoustics and medical auscultation, where specialized pretraining plus prototype-based heads may support both high accuracy and efficient deployment in edge or low-data scenarios (Rauch et al., 17 Apr 2025).