Perceptual Separation (PS): Evaluation & Application

• Perceptual Separation (PS) is a metric that measures the effectiveness of source separation by distinguishing cross-talk leakage from self-distortion.
• It employs psychoacoustic distortions, high-dimensional encoding, and diffusion maps to achieve granular, frame-level evaluation that mirrors human listening.
• PS supports diagnostic evaluation, real-time monitoring, and training loss functions, enhancing model selection in audio and music separation applications.

Perceptual Separation (PS) refers to the quantification or computational assessment of how effectively a system separates mixed signals into perceptually distinct, non-interfering components. In source separation, PS is designed to measure, in a manner that is tightly coupled to human perception, the degree to which undesirable leakage or cross-talk from interfering sources remains in an estimated signal, independently from distortions of the signal itself. Recent advances propose PS as a granular, frame-level, differentiable metric that directly addresses the disjunction between conventional error-based metrics and subjective listening assessments in audio source separation, music analysis, and allied fields (Ivry et al., 11 Sep 2025).

1. Conceptual Framework

Perceptual Separation (PS) is established as a metric that quantifies the perceptual leakage—residual presence of undesired sources—within the output of a source separation system. Unlike conventional measures such as signal-to-distortion ratio (SDR), PS functionally isolates this leakage from self-distortion (the deformation of the target source itself). This separation is achieved by comparing the separated signal not only to its clean reference but also to a set of fundamental, psychoacoustically motivated distortions of the reference, as well as to other sources in the mixture.

The PS metric differs from prior aggregate energy-based or global perceptual scales by uniquely characterizing and disentangling the two dominant error modes—leakage (cross-talk) and self-distortion. Its granularity allows for diagnosis and optimization at high temporal resolution, suitable for both analysis and system training.

2. Methodological Pipeline

The PS methodology consists of four principal steps:

1. Synthesis of Fundamental Distortions: For each reference (ground-truth) signal in the mixture, a bank of controlled, psychoacoustically meaningful distortions is generated. These include manipulations such as notching, comb filtering, tremolo, additive noise, and reverberation, forming a perceptual neighborhood around each reference.
2. High-Dimensional Perceptual Encoding: All references, their distortions, and system outputs are independently embedded using a pre-trained self-supervised learning model (e.g., wav2vec 2.0 for speech or music). The encoder function, Φ: ℝᴸ → ℝᴹ, projects the waveform into a high-dimensional space where geometric distances best reflect perceptual similarity.
3. Manifold Construction via Diffusion Maps: The ensemble of encoded signals is nonlinearly projected onto a low-dimensional manifold using diffusion maps:

$$\Psi_t^{(d)}(x) = [\lambda_1^t u_1(x), \ldots, \lambda_d^t u_d(x)]^\top$$

where (λ_l, u_l) are eigenpairs of the data affinity matrix, and t is the diffusion time. This procedure aligns Euclidean distances with perceptual dissimilarity, allowing geometric operations in a tractable space.

1. Calculation of Mahalanobis Distances and PS Metric: Within the manifold, perceptual clusters are instantiated for each reference speaker/source, encompassing that source's clean reference and all its fundamental distortions. The Mahalanobis distance of a system output to its attributed cluster (Aᵢ^d) and to the nearest non-attributed cluster (Bᵢ^d) is computed:

$$A_i^{(d)} = d_M(\Psi_t^{(d)}(\hat{x}_i);\, \mu_i^{(d)},\, \Sigma_i^{(d)}), \quad B_i^{(d)} = \min_{j \neq i}\, d_M(\Psi_t^{(d)}(\hat{x}_i);\, \mu_j^{(d)},\, \Sigma_j^{(d)})$$

where d_M is the Mahalanobis distance, and (μ, Σ) are centroid and covariance of the respective clusters. PS is then formally defined as:

$$\mathrm{PS}_i^{(d)} = 1 - \frac{A_i^{(d)}}{A_i^{(d)} + B_i^{(d)}}$$

Lower values of A relative to B indicate stronger separation (low leakage); the metric is normalized to $[0,1]$, with higher values corresponding to better perceptual separation.

3. Error Quantification and Confidence Intervals

PS includes rigorous statistical quantification of its own reliability:

• Deterministic Error Radius: The projection to a d-dimensional manifold introduces truncation error; the error radius E(x) is explicitly bounded as:

$$E(x) = \Big(\sum_{l=d+1}^{N-1} \lambda_l^{2t}\Big)^{1/2}$$

reflecting the bias from neglecting higher diffusion dimensions.

• Non-Asymptotic Confidence Intervals: For derived distances and their mapping to PS, high-probability confidence intervals (typically 95%) are computed, accounting for both projection and empirical estimation uncertainty. These are directly used to report bounds on correlation with human perceptual judgments.

This self-quantification enables precise interpretation of PS under real-world operating conditions.

4. Experimental Validation and Comparative Effectiveness

Experiments on the SEBASS database, covering speech in English, Spanish, and music mixtures, found that PS (with its complementary Perceptual Match (PM) metric) achieves the highest or near-highest linear correlations with human mean opinion scores (MOS), compared to 14 other metrics (including SDR, SI-SDR, PESQ, STOI, DNSMOS, SpeechBERT, etc.). Specifically:

• For speech, PS and PM reach a Pearson correlation of up to 86.36% with MOS.
• For music, the correlation is as high as 87.21%.
• The error radius of these correlations does not exceed 1.39%, and the 95% confidence intervals are as tight as 12.21%.

PS and PM exhibit low normalized mutual information (NMI) between their outputs—never exceeding 0.2 and decreasing as system performance worsens. This suggests that, especially under failure modes, they capture complementary aspects of perceptual performance: PS is primarily sensitive to cross-source leakage, while PM quantifies self-distortion.

5. Applications and Diagnostic Utility

PS and PM are directly applicable as both evaluation metrics and differentiable loss functions in the development and deployment of source separation systems. Key applications include:

• Diagnostic Evaluation: PS isolates leakage errors, enabling targeted improvements in model architectures or optimization schemes. PM quantifies distortion of the source itself.
• Training Loss Functions: Because both metrics are differentiable and defined at high temporal granularity (down to 50 frames/s for speech, 10 for music), they can be used as training losses or for online hyper-parameter tuning.
• System Monitoring: Their resolution and confidence bounds suit real-time or interactive perceptual quality monitoring in teleconferencing, music production, and assistive hearing applications.

6. Future Directions and Open Problems

Planned improvements include:

• Extension of human perceptual benchmarking to additional languages, environments, and speaker configurations.
• Exploration of more sophisticated aggregation strategies to bridge the gap between frame-level PS and utterance-level quality judgments.
• Optimization of manifold learning and embedding steps for reduced computational overhead (currently ~1.2 × real time).
• Integration with and possible fusion of PS/PM with other advanced perceptual metrics for even broader coverage of failure modes.

The joint use of PS and PM, given their complementary error sensitivities and high empirical validity, is anticipated to enable substantially improved model selection and perceptual optimization in contemporary and future source separation technologies.