Psychoacoustic Masking Models

• Psychoacoustic masking models are algorithmic frameworks that quantify how audio components are hidden by other sounds using energetic and informational masking.
• They employ critical-band analysis, masker detection, and spreading functions to compute global thresholds that guide perceptual audio coding and enhancement.
• Modern applications integrate these models into neural networks and loss functions, optimizing audio quality in coding, adversarial attacks, and speech enhancement.

Psychoacoustic masking models are algorithmic and mathematical frameworks that quantify how audible an audio component is when presented in the presence of other sounds, exploiting properties of human auditory perception to enable audio coding, enhancement, and analysis. Such models are foundational in perceptual audio coding, speech enhancement, music production, auditory scene analysis, and related fields. Contemporary approaches extend basic energetic masking by incorporating cognitive effects, spatial (binaural) factors, information-theoretic concepts, and machine learning, reflecting increasing physiological, acoustical, and computational sophistication.

1. Foundations of Psychoacoustic Masking

Psychoacoustic masking refers to the phenomenon where the perception of one sound (the “signal” or “probe”) is obscured by the presence of another sound (the “masker”). Masking manifests as raised audibility thresholds in the spectral, temporal, and spatial domains.

• Energetic (Peripheral) Masking: Occurs at the cochlea and auditory nerve; the presence of a masker (tone or noise) elevates the detection threshold for nearby frequencies (“simultaneous or spectral masking”) or times (“forward or backward masking”). These processes are well-captured by peripheral auditory models that derive a frequency- and time-dependent threshold (e.g., Tmask(f,n)) using excitation patterns and known spread-of-masking curves (Delgado et al., 2023).
• Cognitive (Informational) Masking: Emerges from higher-level auditory processing, particularly with unpredictable or complex maskers. Here, the perception of the signal is impeded not because it is energetically submerged, but due to difficulty in segregating it from the masker, often quantified as a reduction in salience (Delgado et al., 2023).

Masking models thus range from classical energetic masking (quantitative thresholds based on local energy and spreading functions) to complex multi-layer frameworks calibrated to human subject data.

2. Classical Masking Metrics and MPEG-Style Approaches

Perceptual audio coding standards such as MPEG-1 Layer III (MP3) have institutionalized a family of psychoacoustic masking models that remain foundational in both research and practical systems. These models are built around a set of workflow steps:

Critical-band Decomposition: Signals are framed and divided into critical bands, often using the Bark or ERB scales, which align with the cochlear filterbank (Zhen et al., 2018, Zhen et al., 2020, Schönherr et al., 2018).

Masker Identification and Masking Curves: Within each frame, tonal and noise maskers are identified as local spectral maxima surpassing the absolute threshold of hearing. Each masker is assigned a masking curve using spreading functions that describe how masking propagates across frequency in Bark space: $$G(\Delta z) = \begin{cases} 27\,\Delta z & -3 \leq \Delta z < -1, \ -16-0.5\,\Delta z & -1 \leq \Delta z < 0, \ -16+1.6\,\Delta z & 0 \leq \Delta z < 1, \ 27\,\Delta z & 1 \leq \Delta z < 8, \ -\infty & \text{otherwise} \end{cases}$$ with $\Delta z$ the distance in Bark (Zhen et al., 2020).

Global Masking Threshold: The per-band masking curves and the absolute hearing threshold are combined (in the power domain) to yield a global masking threshold: $$T(f) = 10\log_{10}\left(10^{0.1Q_f} + \sum_r 10^{0.1 U_{f,r}} + \sum_b 10^{0.1 V_{f,b}} \right)$$ where $Q_f$ is the absolute threshold, $U_{f,r}$ and $V_{f,b}$ are contributions from tonal and noise maskers, respectively (Zhen et al., 2020, Zhen et al., 2018).

Use in Processing: Only those distortions or coding artifacts that exceed the instantaneous masking threshold at a given frequency and time are considered potentially audible and thus penalized in coding or enhancement objectives (Zhen et al., 2018, Zhen et al., 2020).

3. Extensions: Cognitive, Binaural, and Informational Masking

Recent modeling frameworks address the limitations of purely energetic models by incorporating both cognitive effects (informational masking) and spatial (binaural) effects:

• Informational Masking (IM): Models like that in (Delgado et al., 2023) extend the conventional metric by introducing measures of masker complexity and time-variance—specifically, statistical fluctuations (variance) in the proximity-to-threshold parameter $\beta$ over short time windows. High $\beta$-variance indicates masking dominated by cognitive factors, wherein the listener is distracted or unable to segregate the test signal, even above the energetic threshold.

The $\beta$-variance IM metric is:

$$\mathrm{IM}(n) = \frac{1}{K} \sum_{k=1}^K \operatorname{var}_{t \in \mathcal{W}_n} \{\beta(t,k)\}$$

where $\Delta z$0 quantifies near-threshold masking interaction, and the window $\Delta z$1 spans 100 ms (Delgado et al., 2023).

• Binaural (Spatial) Unmasking: Models now integrate interaural time and level differences. The hemispheric two-channel model frames binaural unmasking as a difference in population rates across broad neural channels, compactly formalized as a complex-valued correlation coefficient:

$\Delta z$2

Here, $\Delta z$3 are analytic representations of left/right ear signals, and $\Delta z$4 measures temporal stability of interaural phase differences. Temporal fluctuations in $\Delta z$5 underpin the detectability of signals in noise when binaural cues are present (Encke et al., 2021). This model quantitatively accounts for the majority of variance in classic binaural masking-release data.

• Harmonic and Modulation Masking: Masking models further encompass harmonicity and amplitude-modulation cues, shown to interact differently with diotic and dichotic listening. Computational models employing modulation-filter paths and equalization–cancellation architectures help predict masking release in complex-tone detection, highlighting the need for across-frequency (multi-channel) processing in some cases (Klein-Hennig et al., 2015).

4. Masking in Machine Learning, Speech Enhancement, and Audio Coding

The integration of psychoacoustic masking into loss functions and architecture design has enabled efficient and perceptually superior machine learning models:

• Perceptually Weighted Losses: In DNN-based denoising or audio coding, losses are modulated by psychoacoustic weights $\Delta z$6 derived from the ratio of PSD to masking threshold. This confines backpropagation to perceptually relevant bins, yielding smaller models with improved perceptual quality:

$\Delta z$7

$\Delta z$8

(Zhen et al., 2018).

• Neural Audio Codecs: Lightweight neural codecs using PAM-1 style masking achieve transparent coding at sharply reduced bitrates, employing both weighted spectral losses and explicit noise-to-mask-ratio penalties:

$\Delta z$9

(Zhen et al., 2020).

• Noise-Masking Enhancement: Neural models have been trained to maximize music’s ability to mask ambient noise by learning gain curves in Bark-space that are constrained by psychoacoustic masking models and listening-level preservation:

$$T(f) = 10\log_{10}\left(10^{0.1Q_f} + \sum_r 10^{0.1 U_{f,r}} + \sum_b 10^{0.1 V_{f,b}} \right)$$0

(Berger et al., 24 Feb 2025).

5. Application-Specific and Non-Standard Masking Models

Masking models have been specialized for various applications beyond standard audio coding and enhancement:

• Speech Coding under Extreme Bitrate Constraints: Highly simplified “one-tone-per-octave” masking schemes plus envelope shaping via Beta distributions have been used for conversational archiving at 2.5 kbps, sacrificing spectral detail but preserving intelligibility (Filho et al., 2015).
• Virtual Acoustics and Numerical Error Perception: Error signals from path-traced or Monte-Carlo acoustic simulations are evaluated for perceptual relevance by convolving their estimated spectrum with Zwicker-style stationary loudness models, producing a metric of masked error loudness (Cao et al., 2022).
• Adversarial Perturbation Hiding: Psychoacoustic thresholds (derived as in MP3 coding) are used as hard or soft constraints during gradient-based adversarial attacks, ensuring that injected perturbations remain below human detectability while still inducing target neural net misclassifications (Schönherr et al., 2018).
• Biological and Non-Human Systems: For mosquito hearing, optimal interference masks focus acoustic power into single frequencies or use rapid frequency modulation, in contrast to broadband noise. Information transfer metrics such as transfer entropy rigorously quantify the efficacy of masking strategies (Faber et al., 16 Oct 2025).

6. Methodological and Practical Considerations

• Algorithmic Workflow: Table 1 (below) summarizes prototypical algorithmic steps that recur in masking model implementations.

Step	Key Operations	Output
Framing/STFT	Windowed signal partition, transform to frequency	Time–frequency representation
Critical-band Analysis	Mapping to Bark/ERB or octave bands	Band energies, tonality measures
Masker Detection	Identification of tonal/noise maskers (spectral peaks, mod depth)	Set of maskers per frame
Spreading Function	Computing per-masker masking curves (triangular/Bark, parametric)	Spread energy profiles
Threshold Synthesis	Power-sum of masking curves + threshold-in-quiet	Per-band masking threshold
Application/Integration	Loss weighting, coding allocation, enhancement constraints	Perceptual optimization

• Model Calibration and Validation: Calibration employs listening tests (e.g., MUSHRA, MOS, AFC paradigms) and objective metrics (OPS, APS, IPS, STOI, transfer entropy). Models are validated against subjective ground truth, and data-driven selection of distortion-metric and masking-interaction terms is performed in current cognitive models (Delgado et al., 2023).
• Limitations and Future Directions: Contemporary models strive for greater physiological realism (e.g., finer time-frequency resolution, incorporation of true peripheral front-ends, listener variability) and tackle extensions to cross-channel, multi-source, or real-time scenarios. Unresolved issues include the optimal tradeoff between complexity and predictive accuracy, integration of temporal masking in non-stationary environments, and generalization across diverse content and listener populations.

7. Synthesis and Outlook

Psychoacoustic masking models provide the quantitative backbone for perceptual optimization in audio systems, enabling robust coding, enhancement, and analysis that maximally exploits human auditory tolerances. Increased physiological detail (binaural coding, harmonicity, informational complexity), methodological sophistication (neural models, transfer entropy, cognitive salience mapping), and statistical regularization (variance-based cognitive masking) now support state-of-the-art systems across domains. Continued synthesis of psychoacoustic, computational, and perceptual evidence holds potential for further advances in both theory and application.

Key references illustrating the breadth and technical detail of this field include (Zhen et al., 2020, Delgado et al., 2023, Faber et al., 16 Oct 2025, Encke et al., 2021, Berger et al., 24 Feb 2025, Schönherr et al., 2018, Klein-Hennig et al., 2015, Zhen et al., 2018, Cao et al., 2022), and (Filho et al., 2015).