Scoring Backends Matter More Than Pooling: A Systematic Study of Training-Free Anomalous Sound Detection under Domain Shift

Abstract: Training-free anomalous sound detection (ASD) scores a test clip against a memory bank of normal embeddings from a frozen pretrained audio encoder. Recent work attributes domain-shift robustness mainly to how frame-level features are pooled over time; the scoring backend applied on top of the pooled embedding has received far less systematic attention. Using a single frozen BEATs encoder on the DCASE 2023 Task 2 development set (all seven machine types), we cross four classical backends -- nearest-neighbor cosine distance, Mahalanobis distance, locally density-normalized kNN, and PCA-subspace reconstruction residual -- with three temporal poolings (mean, GeM, max). Switching the backend moves target-domain AUC by 13.8 points on average (up to 53.8), whereas switching the pooling moves it by only 3.2 points: in this training-free regime, the backend, not the pooling, dominates domain-shift robustness. No backend wins everywhere, but the machine-dependent pattern reproduces on the DCASE 2025 development data (fan, bearing). Exploiting this, we propose a label-free score fusion that z-normalizes each backend with its training-bank self-scores and takes the minimum; it reaches a harmonic-mean target AUC of 63.3% versus 64.4% for the per-machine oracle, surpassing every fixed single backend while preserving source-domain accuracy. We also report a negative result: selecting a backend by source-domain pseudo-validation with proxy outliers fails, because all backends saturate on the proxy task.

• The paper demonstrates that scoring backends significantly impact target-domain AUC, with average improvements up to 13.8 points over pooling methods.
• It systematically evaluates four classical backends (kNN cosine, Mahalanobis, density-normalized kNN, PCA-residual) on DCASE datasets under strict training-free conditions.
• The study proposes a label-free fusion strategy that nearly matches per-machine oracle performance, underscoring the need for optimized backend selection.

Scoring Backend Dominance in Training-Free Anomalous Sound Detection under Domain Shift

Problem Setting and Motivation

Training-free anomalous sound detection (ASD) for machine condition monitoring operates on the principle of scoring a test clip against a database of normal embeddings, all derived from a frozen pretrained audio encoder. Recent advances have established the efficacy of transfer learning with self-supervised models (e.g., BEATs), allowing ASD systems to operate without additional training. However, domain shift—stemming from a mismatch between training (source) and deployment (target) conditions—remains a critical concern. Traditionally, studies have centered arguments for robustness on temporal pooling strategies, underestimating the significance of the scoring backend.

This work conducts a systematic and controlled comparison of classical scoring backends—nearest-neighbor cosine, Mahalanobis distance, locally density-normalized $k$NN, and PCA-subspace reconstruction residual—crossed with three common temporal poolings (mean, GeM, max), in a fully training-free regime using BEATs embeddings on standardized DCASE Task 2 datasets.

Experimental Methodology

The analysis covers all seven machine types in the DCASE2023 Task 2 development set and extends to two machines (fan, bearing) from the DCASE2025 set, ensuring validation across independent benchmark years. Each evaluation is performed with a strictly frozen architecture: BEATs embeddings, a fixed memory bank of 1,000 normal clips (with strong domain skew), and deterministic, parameter-free scoring for all backends and poolings.

The four backends are as follows:

• Nearest-neighbor cosine ($k$NN): Standard minimum cosine distance to bank.
• Mahalanobis distance: Using Ledoit–Wolf shrinkage for stable inverse covariance estimation due to high embedding dimensionality.
• Density-normalized $k$NN: Inspired by LOF, normalizes outlier score by local embedding density, mitigating bias from bank sparsity.
• PCA-subspace residual: Measures orthogonal deviation from the dominant subspace (retaining 90% variance), flagging samples outside the learned manifold.

Temporal poolings include mean, GeM ($p=3$), and max. All combinations are evaluated in terms of AUC (per-domain, per-machine) and pAUC following the DCASE protocol.

Empirical Findings: Backend Versus Pooling Sensitivity

A central result is the quantitative dominance of the scoring backend over the choice of pooling strategy in controlling domain-shift robustness. For target-domain AUC, switching the backend yields an average change of 13.8 points (with maxima up to 53.8), whereas varying the pooling alters scores by only 3.2 points on average (see Figure 1).

Figure 1: Backend choice impacts target-domain AUC substantially more than pooling, as shown by the per-machine range on DCASE2023/2025.

The backend effect is sharply machine-dependent, with no single backend universally optimal. Notably, density-normalized $k$NN achieves the highest target-domain AUC on four of seven machines, but may drastically degrade source-domain performance, exemplified by the "fan" case (source AUC $32.4$, target AUC $86.3$), linked to extreme source–target manifold geometry.

Figure 2: Target-domain AUC for four scoring backends on seven DCASE2023 machines with mean pooling, illustrating strong backend-dependent variation; no backend is dominant everywhere.

Cross-year validation on DCASE2025 confirms the reproducibility of both the backend-pooling asymmetry and the detailed per-backend behaviors, discounting the possibility of year- or dataset-specific artifacts.

Label-Free Score Fusion and Attempted Backend Selection

Given the backend instability, a key practical question is whether robust backend combination can approach the performance of an oracle (per-machine best backend). The study introduces a label-free fusion protocol: individual backend scores are z-normalized using bank self-scores, then fused by minimum, maximum, or mean.

• z-min fusion (minimum over normalized backends) nearly closes the gap to the per-machine oracle, achieving a harmonic-mean target-domain AUC of $63.3\%$ versus $64.4\%$ for the oracle in DCASE2023, while preserving high source-domain accuracy. This ensemble superiority is further confirmed for DCASE2025.

  Figure 3: z-min fusion of label-free, normalized backends achieves harmonic-mean target-domain AUC nearly that of the per-machine oracle on DCASE2023/2025.

The study also investigated pseudo-validation backend selection using proxy outliers from other machines' embeddings, finding that all backends saturate on this simplified task (pseudo-AUC $\approx 1.0$), rendering this approach ineffective.

Theoretical and Practical Implications

This controlled study establishes, with strong empirical evidence, the backend’s primacy over pooling in dictating domain-robustness for training-free ASD under challenging domain shift. The geometric structure of the memory bank—especially under heavy domain bias—profoundly influences the efficacy of different statistical outlier measures, dictating the necessity for machine- or domain-aware backend selection or combination.

The introduction of self-calibrated, label-free fusion significantly narrows the practical gap to an oracle, offering a robust, deployment-friendly enhancement without reliance on test labels or further data. The outlined failure of pseudo-validation highlights a fundamental limitation when proxy outliers are not sufficiently close to the in-manifold tail.

Limitations and Directions for Future Work

All analyses are restricted to clip-level memory banks and a single deep encoder (BEATs). Backends may interact differently with patch-wise memory banks or in systems incorporating adaptive or learned pooling (e.g., deviation-based, as in recent work). No sensitivity study of backend hyperparameters (e.g., number of neighbors, variance retention fraction) was performed, as parameter tuning would conflict with the training-free paradigm. Expanding the cross-year analysis to additional machines remains salient. The findings encourage development of advanced, potentially domain-adaptive backend selection schemes, leveraging more nuanced approaches to unsupervised manifold perturbation or proxy construction.

Conclusion

This work rigorously demonstrates that the design of the scoring backend eclipses that of temporal pooling in shaping domain-shift robustness for training-free ASD with frozen, pretrained audio representations. The implications are direct for practitioners: backend selection is not ancillary, but central, and simple, label-free ensemble strategies yield large, practical performance improvements. These findings motivate a reevaluation of past pooling-centric research in ASD, call for broader backend exploration, and suggest that robust backend fusion may become a standard component in training-free anomaly detection under domain shift.