SINGAD: Singing-Aware Speech Detection

• SINGAD is a singing-aware speech activity detection paradigm that reliably classifies frames containing spoken speech while rejecting pure singing as a negative class.
• It utilizes mel-spectrogram inputs processed through cascaded bidirectional GRU modules with skip connections, and includes a low-complexity variant for efficient computation.
• The method employs controlled speech-to-singing training ratios with extensive data augmentation, achieving a best overall AUC of 0.919 on challenging mixed audio benchmarks.

SINGAD is best understood as an informal label for singing-aware speech activity detection: a speech activity detector designed to identify spoken speech while rejecting singing voice as a hard negative class. In the cited literature, the exact name SINGAD is not introduced as a formal system name; the closest direct formulation is Singing-Robust Speech Activity Detection (SR-SAD), which defines the task as binary frame classification in monaural mixtures, with speech-only and speech + singing overlap mapped to the positive class, and singing-only, instrumental music, ambient noise, and silence mapped to the negative class (Grundhuber et al., 10 Dec 2025).

1. Definition and task semantics

The core SINGAD problem is not generic voice activity detection and not generic vocal activity detection. Its decision rule is specialized to the question of whether a frame contains speech, not whether it contains any human vocalization. In the SR-SAD formulation, the frame-level detector is written as

$$f(\mathbf{x}_i) = \begin{cases} 1 & \text{if } \mathbf{x}_i \text{ contains speech}\ 0 & \text{otherwise} \end{cases}$$

where $\mathbf{x}_i \in \mathbb{R}^N$ is the $i$-th frame (Grundhuber et al., 10 Dec 2025).

This definition is motivated by a specific failure mode of conventional SAD systems: they often misclassify singing as speech. The cited work attributes the difficulty to the fact that speech and singing are both human vocal signals and share many acoustic traits, even though singing is described as having higher fundamental frequency, lower temporal rate, and greater pitch stability. The boundary remains nontrivial because some non-speech vocal behaviors, including emotional speech, laughter, and crying, can overlap acoustically with singing (Grundhuber et al., 10 Dec 2025).

The task is particularly relevant to dialogue enhancement, automatic speech recognition, speech extraction from mixed audio, and other speech-processing front ends operating in media or music-rich environments. A common misconception is to equate SINGAD with singing voice detection. The SR-SAD formulation is narrower and more operational: singing-only frames are treated as negatives, but speech + singing overlap remains positive because spoken speech is still present (Grundhuber et al., 10 Dec 2025).

2. Model architectures and signal representation

The principal architectures associated with this task are SR-SAD and SR-SAD-LC. Both operate on mel-spectrogram input with the following front-end parameters: mel dimension $C = 80$, FFT size $N_{\mathrm{FFT}} = 512$, window length $512$ samples, hop length $256$ samples, and sample rate $16\,\mathrm{kHz}$ (Grundhuber et al., 10 Dec 2025).

The main SR-SAD architecture is described as a sequence of: input mel spectrogram, linear layer, tanh activation, and three cascaded bidirectional GRU modules. Each GRU module comprises two layers, and each module processes the output of the previous GRU concatenated with the original input, creating skip connections for frame-level prediction (Grundhuber et al., 10 Dec 2025).

The low-complexity variant SR-SAD-LC modifies this design by introducing downsampling along the time axis before the GRU via strided convolutional layers, a single-layer bidirectional GRU, and upsampling via transposed convolutional layers to restore the original time resolution (Grundhuber et al., 10 Dec 2025).

The reported computational profiles are as follows (Grundhuber et al., 10 Dec 2025):

Model	MACs	RTF	Parameters
SR-SAD-LC	15.6M	275	335K
SR-SAD	82.9M	32	870K
AS-pVAD	38.5M	125	279K
STA-VAD	3.35G	7	559K
ResNet50	80.05G	15	11.4M

These models were evaluated with chunk lengths of 0.05, 0.1, 0.2, 0.5, 1, 2 s for SR-SAD, SR-SAD-LC, and AS-pVAD, while ResNet50 was fixed at 10 s. The best SR-SAD results were obtained with 2 s chunks, indicating that discrimination between speech and singing benefits from nontrivial temporal context rather than purely local spectral evidence (Grundhuber et al., 10 Dec 2025).

3. Training protocol and controlled singing negatives

A defining methodological contribution of the SR-SAD work is the use of a controlled speech/singing ratio during training. Training examples are generated dynamically by selecting either speech with probability $p_s$ or singing with probability $1-p_s$ (Grundhuber et al., 10 Dec 2025).

The training corpora are drawn from three sources: DNS Challenge 2021 speech subsets for speech, MUSDB18-HQ for singing and music, and DNS Challenge 2021 noise for noise. Songs containing rap or speech were removed from MUSDB18-HQ (Grundhuber et al., 10 Dec 2025). If speech is selected, it is mixed with random noise. If singing is selected, the singing stem is mixed with its corresponding instrumental accompaniment. The labels are then binary: speech-containing frames are positive, and singing is negative (Grundhuber et al., 10 Dec 2025).

Ground-truth speech labels are generated from the clean speech signal using energy-based thresholding, followed by a postprocessing rule that bridges silent periods shorter than 300 ms (Grundhuber et al., 10 Dec 2025). The paper states that if more robustness to overlapping speech and singing is required, this should be explicitly included in training, which suggests that overlap robustness was not the dominant training configuration in the reported pipeline (Grundhuber et al., 10 Dec 2025).

Optimization uses binary cross-entropy, Adam, initial learning rate $\mathbf{x}_i \in \mathbb{R}^N$0, weight decay $\mathbf{x}_i \in \mathbb{R}^N$1, halving the learning rate if validation loss does not improve after 20 epochs, and early stopping patience 20. Each epoch contains 100,000 training audio/label pairs and 1,000 validation pairs, with total training lasting 60 to 200 epochs depending on hyperparameters (Grundhuber et al., 10 Dec 2025).

The augmentation pipeline includes SNR adjustment ($\mathbf{x}_i \in \mathbb{R}^N$2 dB) with probability 80%, frequency band rejection (100–4000 Hz) with probability 80%, high-pass filter (500–4000 Hz cutoff) with probability 30%, low-pass filter (3000–8000 Hz cutoff) with probability 10%, clipping with probability 10%, amplitude scaling (0.1–1.0) with probability 40%, white noise addition with probability 10%, and stereo-to-mono conversion with probability 100% (Grundhuber et al., 10 Dec 2025).

4. Corpora and evaluation metrics

The training and validation split is 80% train / 20% validation, with no overlapping speakers or artists between splits and all data resampled to 16 kHz (Grundhuber et al., 10 Dec 2025). The speech side uses DNS Challenge 2021 subsets including emotional_speech, french_data, german_speech, italian_speech, and read_speech. The singing side uses MUSDB18-HQ, and the noise side uses DNS Challenge 2021 noise (Grundhuber et al., 10 Dec 2025).

The test set is deliberately cross-domain and more difficult. It combines EARS speech samples, excluding those labeled melodic or vegetative; MoisesDB mixed songs for music and singing; and WHAM! babble noise (Grundhuber et al., 10 Dec 2025). The evaluation corpus totals 100 h, comprises 24,000 signals, and each signal is 15 s long. Speech-to-other-signal SNR is sampled uniformly from $\mathbf{x}_i \in \mathbb{R}^N$3 dB to $\mathbf{x}_i \in \mathbb{R}^N$4 dB, and loudness is sampled uniformly from $\mathbf{x}_i \in \mathbb{R}^N$5 LKFS to $\mathbf{x}_i \in \mathbb{R}^N$6 LKFS. If speech is present, a 12 s speech excerpt is randomly positioned within the 15 s sample (Grundhuber et al., 10 Dec 2025).

The label composition of the test set is central to the task definition: 34.7% speech, 23.4% singing, and 8.1% overlapping speech and singing (Grundhuber et al., 10 Dec 2025). This mixture structure ensures that the benchmark measures both ordinary speech detection and the ability to reject singing without suppressing true speech overlap.

Evaluation uses standard AUC and a singing-specific metric, $\mathbf{x}_i \in \mathbb{R}^N$7, introduced to quantify the tradeoff between speech recall and singing rejection. The paper defines Singing Rejection Rate (SiRR) as the true negative rate on frames containing singing:

$\mathbf{x}_i \in \mathbb{R}^N$8

(Grundhuber et al., 10 Dec 2025)

The motivation for $\mathbf{x}_i \in \mathbb{R}^N$9 is that ordinary SAD AUC can remain high even when a detector consistently mistakes singing for speech, because easy negatives such as silence and instrumental-only frames can dominate the non-speech pool (Grundhuber et al., 10 Dec 2025).

5. Empirical performance and operating trade-offs

The best reported overall result for SR-SAD is AUC = 0.919, while the paper also reports a compromise operating point at $i$0 chosen to balance overall SAD performance with singing rejection (Grundhuber et al., 10 Dec 2025). The distinction matters: ordinary AUC and singing-robust AUC do not peak at the same training ratio.

At the selected $i$1 operating point, the reported results are (Grundhuber et al., 10 Dec 2025):

Model	AUC	$i$2
SR-SAD	0.910	0.572
SR-SAD-LC	0.903	0.545
AS-pVAD	0.908	0.510
STA-VAD	0.848	0.385

When the speech probability is swept from 0% to 100% in 10% steps, the best ordinary AUC values occur around 70% speech: 0.919 for SR-SAD, 0.905 for SR-SAD-LC, 0.917 for AS-pVAD, 0.863 for STA-VAD, and 0.881 for ResNet50. By contrast, the best $i$3 values occur at 10% speech: 0.726 for SR-SAD, 0.696 for SR-SAD-LC, 0.663 for AS-pVAD, 0.688 for STA-VAD, and 0.595 for ResNet50 (Grundhuber et al., 10 Dec 2025). This establishes a direct operating trade-off: more singing negatives improve singing rejection, but too many can reduce overall speech detection accuracy.

The paper also reports a sharp degradation when singing is absent from training. At $i$4, $i$5 falls to 0.397 for SR-SAD, 0.421 for SR-SAD-LC, 0.422 for AS-pVAD, 0.323 for STA-VAD, and 0.398 for ResNet50 (Grundhuber et al., 10 Dec 2025). On singing-active MoisesDB segments, SR-SAD reaches ACC = 0.9786 at $i$6, but only 0.1215 at $i$7 (Grundhuber et al., 10 Dec 2025). This quantitatively anchors the central claim of SINGAD: explicit singing negatives are necessary if a speech detector is expected to reject singing.

Chunk length introduces another important trade-off. For SR-SAD, AUC rises from 0.715 at 0.05 s to 0.919 at 2 s. For SR-SAD-LC, the best reported value is 0.899 at 1–2 s, while AS-pVAD reaches 0.922 at 2 s (Grundhuber et al., 10 Dec 2025). This suggests that the speech-versus-singing boundary is not well characterized by instantaneous spectral evidence alone.

The main reported failure modes are rap, blues, and, to a lesser extent, electronic music with speech-like intros (Grundhuber et al., 10 Dec 2025). Rap is especially difficult because of its spoken-word character; blues and some electronic segments can also resemble speech acoustically. The paper further states that the use of 2 s chunk-based processing prevents real-time use in its current form, so SINGAD in this formulation is better viewed as a chunked front end than as a strictly low-latency detector (Grundhuber et al., 10 Dec 2025).

6. Terminological ambiguity and relation to adjacent research

The term SINGAD is not standardized across the cited arXiv literature. In the most direct sense, it aligns with singing-aware SAD, and SR-SAD is the clearest concrete instantiation of that problem setting (Grundhuber et al., 10 Dec 2025). This should be distinguished from several unrelated or only partially related systems whose names are similar.

A first source of confusion is the family of methods named SING, which in the cited corpus refers to unrelated topics such as Synthetic Intention Graph for Scalable Active Tool Discovery in LLM Agents (Xiao et al., 15 Jun 2026), SDE Inference via Natural Gradients (Hu et al., 21 Jun 2025), and Semantic Image Communications using Null-Space and INN-Guided Diffusion Models (Chen et al., 16 Mar 2025). A second source is SINGD, which denotes Structured Inverse-Free Natural Gradient Descent and concerns second-order optimization rather than audio analysis (Lin et al., 2023). None of these define singing-aware speech activity detection.

Within singing-audio research proper, several neighboring tasks remain distinct from SINGAD. FastSAG concerns Singing Accompaniment Generation, generating instrumental accompaniment from a vocal waveform (Chen et al., 2024). SingNet addresses real-time singing voice beat and downbeat tracking, using a causal CRNN and dynamic particle filtering for metrical inference from isolated vocals (Heydari et al., 2023). These systems are related at the level of singing-specific audio modeling, but they solve different problems: accompaniment synthesis and rhythmic tracking rather than spoken-speech detection.

The clearest encyclopedic definition, therefore, is that SINGAD denotes a singing-robust speech activity detection paradigm whose best-specified instance in the cited literature is SR-SAD: a mel-spectrogram-based recurrent detector trained with controlled proportions of speech and singing so that spoken speech remains detectable in mixed audio while singing is explicitly rejected as non-speech (Grundhuber et al., 10 Dec 2025).