Perceptual Match (PM)

• Perceptual Match (PM) is a metric that defines how closely a processed signal aligns with its reference based on human perceptual features.
• It isolates self-distortion from interference by leveraging high-dimensional embeddings, diffusion maps, and a probabilistic Gamma model for precise measurement.
• PM is validated in audio-visual tasks and correlates strongly with human opinion scores, supporting both fine-grained and utterance-level system improvements.

Perceptual Match (PM) encompasses a class of metrics and system design principles aimed at quantifying or optimizing the perceived equivalence between signals, objects, or representations—explicitly from the standpoint of human or task-relevant perception. In contrast to purely physical or mathematical distance measurements, PM emphasizes the alignment of internal representations, subjective similarity, or perceptually salient distortions. Central to PM is the isolation (and, where applicable, functional separation) of distinct perceptual factors—such as self-distortion versus interference—via high-dimensional embeddings, statistical models, and perceptually motivated manifolds or metrics. In source separation and related audio-visual tasks, recent work operationalizes PM as an objective, differentiable, and granular measure that is tightly correlated with human mean-opinion scores, incorporating confidence intervals and error bounds to inform the reliability of system assessment (Ivry et al., 11 Sep 2025).

1. Definition and Theoretical Motivation

Perceptual Match (PM) refers to a measure that quantifies how closely a processed or estimated signal (for example, an output from a source separation system) resembles its designated reference in terms of human perception, abstracting away from direct physical measures or secondary interference effects such as leakage. The operational requirement is that PM should align with the "natural" manifold of perceptual variations—encoding both the invariances and salient sensitivity axes of the underlying domain.

Classical reference-based metrics (e.g., SDR, PESQ) either conflate multiple error sources or do not provide direct access to perceptual similarity. PM addresses this by functionally isolating self-distortion—how much the output differs from the reference in a manner that would be recognized as a perceptual mismatch—independent of contamination from other sources. This paradigm advances the evaluation and optimization of systems where human or domain-specific perceptual fidelity is critical.

2. Methodological Framework and Embedding Pipeline

The PM metric as instantiated in MAPSS (Ivry et al., 11 Sep 2025) begins by establishing a manifold of perceptual similarity for each reference using a "bank of fundamental distortions" motivated by psychoacoustics (e.g., notch filtering, additive noise, reverberation, pitch shifts). These distortions are engineered to span representative, perceptually significant departures from the original signal without introducing cross-source information. Each waveform (the reference, its distortions, and the system's output) is encoded into a high-dimensional vector space via a pre-trained self-supervised learning model (e.g., wav2vec 2.0, HuBERT, WavLM).

To recover a perceptually aligned geometry, all encoded representations are projected onto a low-dimensional manifold using diffusion maps. This construction is designed so that Euclidean distances within the manifold correspond to perceptual dissimilarities among the encoded signals. Within this manifold, the reference and its distortions form a perceptual cluster, modeling the range of acceptable, natural perceptual variation for that source.

The PM metric then quantifies the Mahalanobis distance of the test output’s embedding to this reference cluster. This distance is interpreted in terms of a probabilistic model (Gamma distribution fit to the cluster's intra-distortion distances), enabling the calculation of a regularized cumulative probability (using the regularized upper incomplete Gamma function) that reflects the degree of perceptual match. Scores near one denote outputs that remain within the natural perceptual variation of the reference; values near zero imply substantial perceptual deviation.

3. Mathematical Formulation

The formal steps, as defined in (Ivry et al., 11 Sep 2025), are:

• Reference $\mathbf{y}_i$ is encoded as $x_i = \Phi(\mathbf{y}_i)$.
• Each of $N_p$ reference distortions $\{\mathbf{y}_{i,p}\}$ are encoded as $x_{i,p} = \Phi(\mathbf{y}_{i,p})$.
• The system output $\hat{\mathbf{y}}_i$ is encoded as $\hat{x}_i = \Phi(\hat{\mathbf{y}}_i)$.
• All vectors are mapped to the $d$-dimensional diffusion map: $\Psi_{\tau}^{(d)}(x)$.

Define cluster for reference $i$ as $$c_i^{(d)} = \{\Psi_{\tau}^{(d)}(x_i), \Psi_{\tau}^{(d)}(x_{i,p})\}$$. The unbiased empirical covariance of the cluster (excluding the output) is: $$\tilde{\Sigma}_i^{(d)} = \frac{1}{N_p - 1}\sum_{p=1}^{N_p} (\Psi_\tau^{(d)}(x_{i,p}) - \Psi_\tau^{(d)}(x_i))(\Psi_\tau^{(d)}(x_{i,p}) - \Psi_\tau^{(d)}(x_i))^\top$$ For output, the squared Mahalanobis distance to the reference is: $$a_i^{(d)} = (\Psi_\tau^{(d)}(\hat{x}_i) - \Psi_\tau^{(d)}(x_i))^\top (\tilde{\Sigma}_i^{(d)} + \varepsilon I^{(d)})^{-1} (\Psi_\tau^{(d)}(\hat{x}_i) - \Psi_\tau^{(d)}(x_i))$$ Define $\{g_p^{(d)}\}$ as the squared Mahalanobis distances of each cluster member (excluding the output) to the cluster center. Fit a Gamma distribution to these squared distances to estimate parameters $\hat{k}_i^{(d)}$ (shape) and $\hat{\theta}_i^{(d)}$ (scale). The PM score is then: $$\mathrm{PM}_i^{(d)} = Q(\hat{k}_i^{(d)}, a_i^{(d)}/\hat{\theta}_i^{(d)})$$ where $Q$ is the regularized upper incomplete Gamma function.

This structure functionally separates self-distortion from cross-source interference (addressed by the complementary PS metric), ensuring that the PM value is not confounded by leakage.

4. Experimental Validation and Correlation with Human Judgment

Extensive experiments, as reported in (Ivry et al., 11 Sep 2025), were conducted on English, Spanish, and musical mixtures, validating PM against human mean-opinion scores (MOS). PM consistently achieved the highest (or nearly highest) linear correlation with human MOS among 14 metrics, with correlation coefficients as high as 86.36% for speech and 87.21% for music. The approach operates at fine temporal resolutions (as low as 50 frames per second), supporting both utterance and frame-level assessment.

The reliability and informativeness of PM are further substantiated by deterministic error radius calculations and probabilistic 95% confidence intervals (CIs)—the worst-case error radius of 1.39% and CIs of 12.21% for the correlation coefficients significantly improve assessment transparency. Additionally, mutual information analysis revealed very low normalized mutual information between PM and PS (typically < 0.2), substantiating that PM captures distinct self-distortion factors, not redundant with leakage measures.

5. Applications and Significance for System Development

PM supplies a robust, objective criterion for evaluating source separation systems, audio enhancement pipelines, and any perceptually grounded signal reconstruction task where the goal is to maintain the perceived identity or quality of a reference. Its ability to functionally isolate self-distortion enables developers and researchers to disentangle deficiencies due to inherent signal degradation from those due to cross-source leakage or interference.

Because the PM measure is differentiable and frame-granular, it is amenable to use as a loss function or evaluation metric in end-to-end system training—potentially driving perceptually informed parameter tuning and neural architecture optimization. When conjoined with the Perceptual Separation (PS) metric, PM provides a diagnostic toolset for distinguishing whether a degraded score is due to distortion or interference, facilitating targeted system debugging and improvement.

6. Limitations and Future Directions

Limitations of PM as reported in (Ivry et al., 11 Sep 2025) include the lack of frame-level human perceptual ratings in current validation datasets (such as SEBASS), necessitating heuristic aggregation (mean-pooling) for utterance-level correlations. The quality of the metric is sensitive to the choice and layer selection in the pre-trained perceptual encoder, and while current pipeline efficiencies achieve near-real-time performance, further gains are needed for real-time deployment or massive-scale architecture search. Additional extension to other languages, domains, and the collection of granular subjective data are suggested as future work to broaden generalizability and diagnostic granularity.

A plausible implication is that as the PM metric and its variants are adopted for training objectives, the alignment between automatic evaluation and human assessments of audio quality and intelligibility may further improve.

7. Summary Table: Key PM Methodological Steps

Stage	Operation	Purpose
Distortion Generation	Create psychoacoustically motivated variants of each reference	Model natural perceptual cluster
Encoding	Pre-trained SSL embedding (e.g., wav2vec 2.0)	High-dimensional perceptual representation
Manifold Projection	Diffusion maps	Align metric geometry with perception
Distance Computation	Mahalanobis to attributed cluster	Quantify self-distortion exclusively
Aggregation/Scoring	Gamma model + regularized upper incomplete Gamma	Probabilistic quantification of match

This methodological pipeline ensures that PM is mathematically principled, perceptually meaningful, and operationally separable from other error modes, supporting both fine-grained and utterance-level system assessment (Ivry et al., 11 Sep 2025).