OS-SCL: One-Stage Supervised Contrastive Learning

• The paper introduces OS-SCL, a one-stage framework that jointly optimizes supervised contrastive loss and classifier objectives to enhance machine-ID discrimination and reduce false alarms.
• OS-SCL integrates MixUp-induced label noise and a feature perturbation head to compress and normalize embeddings, leading to robust, compact intra-ID clusters.
• Experimental results on DCASE 2020 Task 2 show improved AUC/pAUC metrics, especially on challenging cases like ToyConveyor, using compact models with under 1.5M parameters.

to=arxiv_search.search 彩神争霸代理.json code {"query":"(Huang et al., 17 Sep 2025) Noise Supervised Contrastive Learning and Feature-Perturbed for Anomalous Sound Detection", "max_results": 5} to=arxiv_search.search 天天爱彩票中奖ium.json code {"query":"supervised contrastive learning Khosla arXiv", "max_results": 5} to=arxiv_search.search 彩神争霸提现.json code {"query":"ArcFace arXiv additive angular margin loss", "max_results": 5} One-Stage Supervised Contrastive Learning (OS-SCL) is a training technique for unsupervised anomalous sound detection (ASD) that is designed for the setting in which a model is trained only on normal audio and must detect unseen anomalous sounds at test time. It was introduced in the paper "Noise Supervised Contrastive Learning and Feature-Perturbed for Anomalous Sound Detection" (Huang et al., 17 Sep 2025). The method addresses a persistent failure mode of machine-ID self-supervised ASD systems: samples from different machines of the same type can sound very similar, making the decision boundaries between normal samples unclear and causing unstable predictions and frequent false alarms, most notably on ToyConveyor in DCASE 2020 Task 2. OS-SCL responds to this by jointly optimizing machine-ID-discriminative embeddings and the classifier in a single stage, combining supervised contrastive learning, MixUp-induced supervised-pair noise, and a feature perturbation head (FPH) that compresses and reconstructs embeddings before projecting them to the unit hypersphere (Huang et al., 17 Sep 2025).

1. Problem setting and motivation

In the formulation addressed by OS-SCL, ASD is posed as training on normal data only and detecting unknown anomalous sounds at test time. Modern machine-ID self-supervised classifiers train a classifier to predict the machine ID among many normal machines; an anomalous sample tends to deviate and be misclassified. According to the OS-SCL paper, such methods have surpassed reconstruction-based approaches, but still suffer when different machines of the same type sound very similar, because the boundaries between their normal samples are poorly defined (Huang et al., 17 Sep 2025).

OS-SCL is designed specifically for this “similar-but-different” regime. The method learns machine-ID-discriminative embeddings with a supervised contrastive loss in one single training stage together with the classifier, rather than pretraining contrastively and then fine-tuning. It also injects controlled noise through MixUp at the input or feature level while still constructing contrastive pairs from original labels, intentionally creating label noise for contrast. In addition, it perturbs features inside the embedding space through the FPH, which compresses and reconstructs features and projects them to the unit hypersphere. The stated goal of these design choices is to strengthen boundaries between highly similar machine IDs, encourage compact intra-ID clusters, and improve consistency across IDs (Huang et al., 17 Sep 2025).

A plausible implication is that OS-SCL reframes false-alarm reduction not as a post hoc thresholding problem, but as a representational geometry problem: if normal samples from similar machine IDs are organized with larger margins and tighter clusters, classifier confidence for normal data becomes more stable.

2. Core formulation of one-stage noisy supervised contrastive learning

The model begins with an audio sample or its time-frequency features, denoted $x$. A backbone encoder $f_\theta$ maps the input to an embedding

$e = f_\theta(x) \in \mathbb{R}^E,$

with MobileFaceNet used and $E = 128$ (Huang et al., 17 Sep 2025).

A projection head $g_\phi$ then maps embeddings to the space where contrastive learning is applied. In OS-SCL, this projection head is the Feature Perturbation Head:

$$\tilde{z} = \psi(W_d \sigma(W_e e)) \in \mathbb{R}^E,\qquad z = \tilde{z} / \|\tilde{z}\|_2,$$

where $W_e \in \mathbb{R}^{R \times E}$, $W_d \in \mathbb{R}^{E \times R}$, $\sigma$ is an activation, and $z$ is $f_\theta$0-normalized to lie on the unit sphere. The final contrastive embedding is therefore $f_\theta$1 (Huang et al., 17 Sep 2025).

MixUp is applied to both waveform and log-Mel within a batch. With coefficient $f_\theta$2, the mixed sample and its label pair are $f_\theta$3, where $f_\theta$4 is the original label and $f_\theta$5 is a randomly shuffled label. The embedding entering the FPH is $f_\theta$6, and $f_\theta$7 is its perturbed, normalized version (Huang et al., 17 Sep 2025).

For supervised contrastive learning, with batch index set $f_\theta$8, positives $f_\theta$9 for anchor $e = f_\theta(x) \in \mathbb{R}^E,$0 are samples with the same machine ID, and $e = f_\theta(x) \in \mathbb{R}^E,$1. Using temperature $e = f_\theta(x) \in \mathbb{R}^E,$2 and dot-product similarity,

$e = f_\theta(x) \in \mathbb{R}^E,$3

The distinctive aspect of OS-SCL is that the original labels $e = f_\theta(x) \in \mathbb{R}^E,$4 are used to define $e = f_\theta(x) \in \mathbb{R}^E,$5 even though $e = f_\theta(x) \in \mathbb{R}^E,$6 comes from a MixUp sample whose effective label can be closer to $e = f_\theta(x) \in \mathbb{R}^E,$7 when $e = f_\theta(x) \in \mathbb{R}^E,$8 is small. The paper identifies this deliberate mismatch as label noise in the contrastive pairing. It is not random feature corruption; it is supervised-pair noise arising from MixUp that is intended to force the embedding to rely on intrinsic acoustic differences rather than brittle label assignments (Huang et al., 17 Sep 2025).

The classifier component uses a Noisy-ArcMix cross-entropy on ArcFace logits. For class $e = f_\theta(x) \in \mathbb{R}^E,$9,

$E = 128$0

with scale $E = 128$1, margin $E = 128$2 for Log-Mel and $E = 128$3 for TFSTgram. Noisy-ArcMix is

$E = 128$4

The final one-stage loss is

$E = 128$5

The paper contrasts this with two-stage pipelines such as CLP-SCF, arguing that separate fine-tuning can erode contrastive structure, whereas OS-SCL optimizes both representation geometry and classification jointly in a single stage (Huang et al., 17 Sep 2025).

3. Feature perturbation and front-end representations

OS-SCL does not inject explicit Gaussian noise. Instead, perturbation is induced by an encoder-decoder bottleneck FPH applied to embeddings, followed by $E = 128$6 normalization:

$E = 128$7

The reduction from $E = 128$8 to $E = 128$9 and reconstruction back to $g_\phi$0 imposes an information bottleneck that, according to the paper, smooths idiosyncratic variations and yields compact, robust clusters, especially when combined with normalization to the hypersphere. No additional perturbation loss is introduced; there is no extra $g_\phi$1 term (Huang et al., 17 Sep 2025).

The paper reports that the bottleneck dimension controls perturbation strength. In ablations, too small an $g_\phi$2 over-perturbs and hurts performance, while $g_\phi$3 worked best. This suggests that the perturbation mechanism is intended to regularize representation geometry rather than maximize distortion.

Two front-end representations are used. The first is a standard log-Mel spectrogram with $g_\phi$4 mel bands and $g_\phi$5 time frames per clip, giving feature size $g_\phi$6:

$g_\phi$7

where $g_\phi$8 is the $g_\phi$9-th mel filter and $$\tilde{z} = \psi(W_d \sigma(W_e e)) \in \mathbb{R}^E,\qquad z = \tilde{z} / \|\tilde{z}\|_2,$$0 is for numerical stability (Huang et al., 17 Sep 2025).

The second is TFgram, a learned time-frequency feature extracted from raw waveform using TFgramNet. The layer summary in the paper is:

1. Conv1D: channels 64, kernel 11, stride 5, padding 5; BatchNorm + ReLU
2. ConvBlock $$\tilde{z} = \psi(W_d \sigma(W_e e)) \in \mathbb{R}^E,\qquad z = \tilde{z} / \|\tilde{z}\|_2,$$1: two Conv1Ds with kernel 3 and stride 1; the second uses dilation $$\tilde{z} = \psi(W_d \sigma(W_e e)) \in \mathbb{R}^E,\qquad z = \tilde{z} / \|\tilde{z}\|_2,$$2; followed by MaxPool1D(kernel 4)
3. ConvBlock $$\tilde{z} = \psi(W_d \sigma(W_e e)) \in \mathbb{R}^E,\qquad z = \tilde{z} / \|\tilde{z}\|_2,$$3: same pattern; followed by AdaptiveMaxPool1D(output length 626)
4. ConvBlock $$\tilde{z} = \psi(W_d \sigma(W_e e)) \in \mathbb{R}^E,\qquad z = \tilde{z} / \|\tilde{z}\|_2,$$4: same pattern; followed by AdaptiveMaxPool1D(output length 313)

The resulting TFgram has $$\tilde{z} = \psi(W_d \sigma(W_e e)) \in \mathbb{R}^E,\qquad z = \tilde{z} / \|\tilde{z}\|_2,$$5 time steps and learnable frequency channels $$\tilde{z} = \psi(W_d \sigma(W_e e)) \in \mathbb{R}^E,\qquad z = \tilde{z} / \|\tilde{z}\|_2,$$6. An optional fused feature, TFSTgram, is defined as

$$\tilde{z} = \psi(W_d \sigma(W_e e)) \in \mathbb{R}^E,\qquad z = \tilde{z} / \|\tilde{z}\|_2,$$7

where $$\tilde{z} = \psi(W_d \sigma(W_e e)) \in \mathbb{R}^E,\qquad z = \tilde{z} / \|\tilde{z}\|_2,$$8 is a time-only feature and $$\tilde{z} = \psi(W_d \sigma(W_e e)) \in \mathbb{R}^E,\qquad z = \tilde{z} / \|\tilde{z}\|_2,$$9 is TFgram (Huang et al., 17 Sep 2025).

The paper states that pure Log-Mel already achieves strong performance and higher stability, countering the common belief that ASD requires very high-frequency cues.

4. Architecture, training pipeline, and inference

The backbone encoder is MobileFaceNet, chosen as an efficient mobile CNN, and the projection or perturbation head is the FPH with a single encoder layer to $W_e \in \mathbb{R}^{R \times E}$0, a single decoder layer back to $W_e \in \mathbb{R}^{R \times E}$1, activation $W_e \in \mathbb{R}^{R \times E}$2, and final $W_e \in \mathbb{R}^{R \times E}$3 normalization (Huang et al., 17 Sep 2025).

The principal hyperparameters reported for training are: batch size $W_e \in \mathbb{R}^{R \times E}$4, epochs $W_e \in \mathbb{R}^{R \times E}$5, optimizer AdamW, initial learning rate $W_e \in \mathbb{R}^{R \times E}$6 with cosine annealing, EMA on weights, temperature $W_e \in \mathbb{R}^{R \times E}$7, ArcFace scale $W_e \in \mathbb{R}^{R \times E}$8, ArcFace margin $W_e \in \mathbb{R}^{R \times E}$9 for Log-Mel and $W_d \in \mathbb{R}^{E \times R}$0 for TFSTgram, MixUp coefficient $W_d \in \mathbb{R}^{E \times R}$1, FPH bottleneck $W_d \in \mathbb{R}^{E \times R}$2, embedding dimension $W_d \in \mathbb{R}^{E \times R}$3, feature size $W_d \in \mathbb{R}^{E \times R}$4, and MixUp on waveform and log-Mel features in batch (Huang et al., 17 Sep 2025).

The training pipeline described in the paper is compact but structurally specific. A batch is shuffled, a MixUp coefficient is sampled, mixed inputs are formed, the front-end produces Log-Mel or TFSTgram features, the backbone yields embeddings $W_d \in \mathbb{R}^{E \times R}$5, the FPH produces normalized embeddings $W_d \in \mathbb{R}^{E \times R}$6, positive sets $W_d \in \mathbb{R}^{E \times R}$7 are built from $W_d \in \mathbb{R}^{E \times R}$8, supervised contrastive loss is computed on $W_d \in \mathbb{R}^{E \times R}$9, ArcFace logits are computed from $\sigma$0, and the two loss terms are summed for backpropagation. EMA updates are applied during optimization (Huang et al., 17 Sep 2025).

At inference time, for a test clip with known machine ID $\sigma$1, the system feeds the sample through the same front-end and backbone, computes ArcFace logits and softmax probabilities, and derives a reconstruction-free anomaly score from the classifier confidence for the true ID:

$\sigma$2

or equivalently

$\sigma$3

Large $\sigma$4 indicates anomalous behavior. AUC and pAUC are obtained by sweeping a threshold over $\sigma$5, so no fixed threshold is needed during evaluation. For deployment, the paper notes that a per-ID threshold can be chosen on a validation set (Huang et al., 17 Sep 2025).

5. Evaluation on DCASE 2020 Task 2

The reported evaluation uses all normal data from the development set plus the additional set, totaling 41 machine IDs across Fan, Pump, Slider, Valve from MIMII and ToyCar, ToyConveyor from ToyADMOS. Testing uses development-test and evaluation-test splits. The metrics are AUC, pAUC for $\sigma$6, and mAUC, defined as the worst AUC across IDs within a machine type to capture lower-bound stability (Huang et al., 17 Sep 2025).

For a machine type with ID set $\sigma$7, the paper defines

$\sigma$8

This metric is central to the OS-SCL argument because it emphasizes worst-ID stability rather than only average behavior.

Setting	AUC / pAUC	mAUC
OS-SCL Log-Mel	94.64% / 88.42%	89.24%
OS-SCL TFSTgram	95.71% / 90.23%	91.23%
STgram-MFN	91.73% / 85.86%	84.86
CLP-SCF	93.75% / 88.48%	87.62
TASTgram (Noisy-ArcMix)	94.08% / 89.01%	—

Per-type AUC/pAUC for OS-SCL are also reported. For Log-Mel: Fan 95.66/88.59, Pump 94.34/85.80, Slider 99.55/97.64, Valve 99.83/99.09, ToyCar 96.83/90.44, and ToyConveyor 81.64/68.97. For TFSTgram: Fan 98.41/94.50, Pump 96.25/88.74, Slider 99.63/98.04, Valve 99.85/99.23, ToyCar 96.55/91.17, and ToyConveyor 83.56/69.87 (Huang et al., 17 Sep 2025).

The paper interprets the particularly large gains on ToyConveyor as indicating fewer false alarms across similar IDs. Because ToyConveyor is singled out earlier as the case where unclear boundaries most notably cause unstable predictions, this interpretation is internally consistent with the method’s stated objective.

6. Ablations, practical considerations, and research context

The ablation results isolate the contributions of the one-stage objective and the perturbation bottleneck. For one-stage OS-SCL versus ArcFace-only, the paper reports STgram(ArcFace) average 91.73/85.86 improving to STgram(OS-SCL) 94.63/89.34, and TFSTgram(ArcFace) 93.44/87.42 improving to TFSTgram(OS-SCL) 95.71/90.23. For the FPH capacity on TFSTgram, performance is 94.94 without FPH, 93.80 with $\sigma$9, 95.71 with $z$0, 95.46 with $z$1, and 94.55 with $z$2; the paper states that too small an $z$3 collapses performance, confirming that moderate bottleneck perturbation is beneficial but over-perturbation hurts (Huang et al., 17 Sep 2025).

The system is also compact. OS-SCL Log-Mel uses 0.88M parameters and TFSTgram uses 1.38M parameters. The paper compares these with large pretrained baselines in the 85–316M parameter range and states that OS-SCL achieves competitive or better averaged performance with dramatically smaller models (Huang et al., 17 Sep 2025).

The reported practical considerations are correspondingly operational. Temperature $z$4 is described as important because larger $z$5 weakens the contrastive signal. The FPH bottleneck should remain moderate, with $z$6 recommended. $z$7 normalization after FPH is used to focus on angular similarity. MixUp with $z$8 is presented as essential for the beneficial supervised-pair noise. The paper also notes limitations: gains on some machine types such as Valve are limited because performance is already near saturated; too aggressive bottlenecking harms performance; and if machine IDs are extremely imbalanced or there are too few positives per batch, supervised contrast may be less effective (Huang et al., 17 Sep 2025).

In relation to prior work, the paper states that OS-SCL adopts the supervised contrastive loss of Khosla et al. but injects structured label noise via MixUp while still constructing positive sets with original labels. It contrasts this with reconstruction-based methods, which can underfit anomalies that are easy to reconstruct, and with density estimators such as Glow-Aff, which require one model per machine ID and stable normal distributions. It also contrasts OS-SCL with two-stage contrastive pipelines such as CLP-SCF, where fine-tuning can dilute contrastive geometry. A plausible implication is that OS-SCL’s significance lies less in introducing a new standalone loss than in coupling supervised contrastive structure, classifier margins, controlled supervised noise, and embedding-space perturbation into a single objective targeted at highly confusable machine IDs (Huang et al., 17 Sep 2025).