Structured Perturbations in Healthy Spectrograms

• The paper demonstrates how structured perturbations in healthy spectrograms reveal distinctive geometric patterns that enable robust mode detection and anomaly estimation.
• It employs manifold parameterization and distortion metrics to quantify topological changes, integrating MFCCs for enhanced classification performance.
• The approach leverages learnable sparse representations and deep patching techniques to improve signal normalization and resilience against structured modifications.

Structured perturbations of healthy spectrograms encompass the deliberate or naturally induced modifications to spectrogram representations that possess interpretable, geometric, or statistical regularity. In contrast to random noise, these perturbations often have a strong connection to signal structure, domain-specific transformations, model architectures, or augmentation strategies. Research trends signal a convergence between stochastic geometry, manifold learning, Lie group frameworks, wavelet-based sparse representations, and data-driven patching protocols, each formalizing aspects of what constitutes “structured” change in a spectrogram.

1. Stochastic Geometry and Level Set Theory

Contemporary time-frequency analysis leverages the spectrogram $$S_\varphi f(\tau,\omega) = |V_\varphi f(\tau, \omega)|^2$$, where $V_\varphi f$ denotes the Short-Time Fourier Transform (STFT) for signal $f$ and window $\varphi$. The geometric paper of spectrogram level sets, as systematically established in (Ghosh et al., 2021), underpins the distinction between healthy and structurally perturbed states. For white noise, level sets

$$\Lambda(\gamma) = \{ (u, v)\in \mathbb{R}^2 : |V_{g} y(u,v)|\geq \gamma \}$$

exhibit regular grid-like organization due to connections with Gaussian analytic functions (GAFs). Additive perturbations—such as a nonzero signal with Hermite function structure—break this symmetry by generating zero-free (or "excursion") regions, typically manifesting as annuli in the time-frequency domain. Theoretical guarantees are established through non-asymptotic bounds on the spectrogram supremum within bounded domains. Such geometric changes are leveraged for robust detection and mode estimation (see Theorem 1 and related detection/estimation proofs).

Empirical evaluations confirm that structured perturbations (signal additions) yield distinctive annular structures, which can be detected and estimated with near-perfect accuracy given appropriate separation between underlying modes. The level set formalism thus provides a quantitative measure for detecting and analyzing structured deviations from the healthy spectrogram baseline.

2. Geometric Distortions and Manifold Parameterization

Modeling spectrograms as surfaces embedded in $\mathbb{R}^3$ enables the quantification of distortion via surface parameterization and mapping (Levy et al., 2021). Respiratory disease classification uses the spectrogram surface

$S(I) = \{ (x, y, z(x,y)) \mid x \in X, y \in Y \}$

with $(x, y)$ representing time and normalized frequency, and $z(x, y)$ the power. Structured perturbations—induced by disease or other mechanisms—change the spectrogram surface topology.

Flattening the spectrogram to a canonical domain via piecewise affine mappings and triangulation, distortion metrics (e.g., Symmetric Dirichlet Energy)

$$E_{SD}(\sigma_1, \sigma_2) = \frac{1}{4}(\sigma_1^2 + \sigma_1^{-2} + \sigma_2^2 + \sigma_2^{-2})$$

capture how local and global deformations differ from those of healthy signals. These features, when fused with Mel-frequency cepstral coefficients (MFCCs), provide high discriminative power for complex classification tasks.

This approach generalizes: spectrogram perturbations can be realized as manifold deformations, with distortion energies and invariant properties supporting the analysis of multi-modal biomedical or engineered signals.

3. Lie Group Transformations and Local Scalar Field Perturbations

Highly interpretable and invertible structured spectrogram perturbations are formalized via local Lie group transformations (Osipov, 16 Apr 2025). Spectrograms are warped through smooth, field-driven transformations in time, frequency, amplitude, and phase. The parametric form

$$\tilde{S}(f, t) = \rho(f, t) e^{-i\beta(f, t)} S(\omega(f, t), \tau(f, t))$$

uses scalar fields ($\phi_{\text{time}}$, $\phi_{\text{freq}}$, $\phi_{\text{amp}}$, etc.) to drive diffeomorphic mappings. Perturbations are constructed using the exponential map of the infinitesimal generator $X$: $$X = (v + u_t)\frac{\partial}{\partial t} + (w + u_f)\frac{\partial}{\partial f} + \alpha + i\beta;\quad \tilde{S} = \exp(\epsilon X)[S].$$ A neural network, trained on synthetically warped healthy speech, infers these fields and computes an approximate inverse to "normalize" distorted speech at test time. The addition of loss terms resembling spontaneous-symmetry-breaking potentials encourages nontrivial perturbation configurations.

Performance metrics (word error rate, character error rate) show consistent and significant improvement for pathological cases, with generalizability to further physiologically plausible perturbation domains (e.g., accent, channel effects) in data augmentation for robust ASR.

4. Learnable Sparse Representations and Spectrogram Adaptation

Wavelet packet transforms with learnable filters furnish structured, data-adapted spectrograms that are highly sensitive to anomalous deviations (Frusque et al., 2022). In the learnable WPT (L-WPT), node-wise filters $\theta$ and biases $\gamma$ are trained end-to-end, enforcing hard-thresholding activations $HT[\,\cdot\,;\gamma]$. In each recursive decomposition, coefficients associated with healthy signals remain sparse: $$HT[x; \gamma] = x(\sigma(-10(x+\gamma)) + \sigma(10(x-\gamma))),$$ where $\sigma(x)$ is the sigmoid.

Structured perturbations, such as faults or anomalies, manifest as energy deviations in specific frequency bands or time windows, making them detectable against the learned background. The L-WPT minimizes spectral leakage and achieves superior performance in anomaly detection (AUC $\sim$ 94.8%, compared to lower baselines), demonstrating that the learned sparse structure is both sensitive and robust to signal-specific perturbations.

5. Structured Patching, Masking, and Augmentation in Deep Architectures

Recent models such as AST and AuM have introduced patch-based approaches for spectrogram modeling. The Full-Frequency Temporal Patching (FFTP) technique (Makineni et al., 28 Aug 2025) addresses the time-frequency asymmetry by generating tall patches that span the entire frequency dimension while localizing in time: $$Z = \text{Conv2D}(X; W_c, s), \qquad Z' = \text{Transpose}(\text{Flatten}(Z)),$$ where $X$ is the log-mel spectrogram and $W_c$ a learnable kernel.

SpecMask, a patch-aligned augmentation strategy, institutes a fixed masking budget split between full-frequency (70%) and time-frequency (30%) masks. This preserves spectral continuity while enforcing temporal robustness. Patch-aligned augmentations are implemented to maintain semantic integrity in the processed spectrograms.

The combination of FFTP and SpecMask yields concrete gains: mAP improvements of up to +6.76 on AudioSet-18k, accuracy increases of up to +8.46 on SpeechCommandsV2, and computational reductions of up to 83.26%. These results indicate effective exploitation of structured spectrogram perturbations to enhance both accuracy and efficiency in large-scale classification tasks.

6. Robustness via Stochastic Differential Modeling

Spectrogram classifiers become robust to environmental and adversarial structured perturbations by leveraging neural stochastic differential equations (NSDEs) (Brogan et al., 3 Sep 2024). The NSDE formulation

$dX_t = f(X_t, t; \theta)dt + G(X_t, t; \theta)dW_t$

introduces domain-shaped stochasticity at the architectural level. Brownian surface noise is injected into residual blocks, serving as regularization and enhancing the model's resilience to input perturbations.

Explanation techniques—Integrated Gradients, NoiseTunnel—produce more coherent attribution maps due to the stabilizing effect of stochastic training. Confidence-calibrated outputs (Attribution-Based Confidence) and smoothness in attribution are crucial for deployment in critical infrastructure (smart grids, NILM, jamming/radar detection), where high SNR cannot be guaranteed.

7. Implications and Research Trajectories

Structured perturbations of healthy spectrograms frame a unifying paradigm for detection, normalization, augmentation, and robust modeling in time-frequency analysis. Theoretical innovations—ranging from level set geometry and manifold distortion metrics to Lie group transformations and learnable sparse representations—are operationalized in practical architectures that demonstrate marked improvements in detection accuracy, anomaly identification, classifier robustness, and computational efficiency. These developments suggest continued growth in the geometric and statistical formalism of spectrogram perturbations, with emerging applications in biomedical diagnostics, acoustic monitoring, smart infrastructure, and data augmentation for deep learning.

Future research will likely examine the extension of these methodologies to multi-modal and higher-order manifold domains, adaptive and physiologically plausible perturbation schemes, and increasingly sophisticated augmentation strategies governed by the intrinsic geometry and statistics of signal representations.