Multi-Metric Blind Image Quality Assessment

• The paper introduces conformal prediction methods to construct hyperrectangular prediction regions that guarantee joint coverage for multiple image quality metrics.
• Methodologically, MM-BIQA leverages multivariate techniques like copula models, vine-copulas, and minimax formulations to calibrate uncertainty across metrics such as PSNR, SSIM, and LPIPS.
• Empirical evaluations reveal that advanced MM-BIQA approaches achieve tighter, balanced prediction sets and outperform naive or Bonferroni-based adjustments in practice.

Multi-metric blind image quality assessment (MM-BIQA) concerns constructing prediction sets or intervals for multiple image quality indices (e.g., PSNR, SSIM, LPIPS) using only the observed data and reconstructed images, without ground-truth reference for the test sample. These sets aim to quantify the uncertainty in predicted quality metrics, typically providing joint coverage guarantees: with high probability, all true metrics lie within their respective prediction sets. The state-of-the-art framework for this task is conformal prediction, particularly in its multi-target or multi-metric formulations, which allow for finite-sample guarantees and adaptivity to data and model uncertainty.

1. Problem Definition and Formal Setup

In MM-BIQA, the objective is to quantify uncertainty over multiple image quality metrics given only a degraded observation $y \in \mathcal{Y}$ and a learned image reconstruction operator $f(y)$. For $K$ metrics $m_1,\ldots,m_K$ (e.g., PSNR, SSIM, LPIPS, DISTS), define the true vector of metrics for a test sample as

$$Y = \big[m_1(f(y), x^\star), \dotsc, m_K(f(y), x^\star)\big] \in \mathbb{R}^K$$

where $x^\star$ is the (unobserved) true image.

On a calibration set with reference images available, the goal is to construct data-driven hyperrectangles $C(y) = \prod_{k=1}^K C_k(y) \subseteq \mathbb{R}^K$ such that, for a specified miscoverage $\alpha \in (0,1)$,

$\Pr\big[Y \in C(y)\big] \geq 1-\alpha.$

The setting can be extended to multi-task or multi-output assessment for other high-dimensional imaging downstream tasks (Wen et al., 17 Nov 2025).

2. Methodological Foundations

Conformal Prediction:

All recent advances in MM-BIQA derive from conformal prediction theory, which produces set-valued predictions with valid marginal or joint coverage under exchangeability. For multi-metric settings, naive strategies—such as independent one-dimensional conformal intervals or Bonferroni-corrected intervals—are either wasteful (overly conservative) or do not guarantee coverage under correlated errors (Wen et al., 17 Nov 2025, Sampson et al., 6 Jun 2024).

Multi-Target Conformal Prediction:

To address the dependency structure among quality metrics, state-of-the-art methods use multivariate conformal predictors, including:

• Copula-based and vine-copula estimators of the joint residual distribution (Wen et al., 17 Nov 2025, Messoudi et al., 2021, Park et al., 4 Nov 2024).
• Minimax conformal calibration to optimize the tightest joint coverage intervals (Wen et al., 17 Nov 2025).
• Hyperrectangular prediction regions with balanced marginal coverage (Sampson et al., 6 Jun 2024).
• Volume-sorted and non-convex region approaches using normalizing flows (Luo et al., 4 Mar 2025).

3. Algorithms for Multi-Metric BIQA

The leading families of algorithms for MM-BIQA can be grouped as follows:

a. Split-Conformal and Hyperrectangle Methods

• Construct per-metric nonconformity scores (residuals, normalized errors) based on a calibration set with reference images.
• Aggregate scores across metrics to a scalar (e.g., via the maximum or copula transform), calibrate a quantile threshold, and construct intervals for each metric at test time (Wen et al., 17 Nov 2025, Sampson et al., 6 Jun 2024).
• Hyperrectangles can be constructed to ensure tight and (asymptotically) balanced marginal coverage across metrics (Sampson et al., 6 Jun 2024).

b. Copula-Based and Vine-Copula Conformal Methods

• Explicitly model the joint distribution of residuals across metrics using (empirical) copulas, Gumbel families, or flexible vine copulas (Messoudi et al., 2021, Wen et al., 17 Nov 2025, Park et al., 4 Nov 2024).
• Calibrate the region such that the multivariate copula CDF at the chosen quantile level equals $1-\alpha$.
• Employ vine copulas for scalable, nonparametric multivariate modeling and semiparametric bias correction to further improve region tightness (Park et al., 4 Nov 2024).

c. Minimax and Efficiency-Optimal Methods

• Formulate the joint region as the solution to a minimax problem: minimize the largest marginal coverage under the joint $\geq 1-\alpha$ constraint (Wen et al., 17 Nov 2025).
• Use empirical CDFs to match the asymptotic solution, which yields uniformly tight regions with balanced marginal coverage even under dependent errors.

d. Non-Convex Regions via Normalizing Flows

• Use conditional normalizing flows to learn a transformation mapping the conditional joint distribution of metrics to a known base density (Luo et al., 4 Mar 2025).
• Construct density-based nonconformity scores (e.g., likelihood or Jacobian determinants), sort test candidates by local density, and define the prediction set as a union of high-density balls.
• Calibrate coverage through exchangeable split-conformal techniques, providing non-convex regions that adapt to the true joint density of metrics.

e. Surrogate and Differentiable ERM Approaches

• Frame conformal set construction as a constrained empirical risk minimization problem, optimizing the set volume under valid empirical coverage using differentiable surrogates (Bai et al., 2022).
• Employ Lagrangian methods and (optional) post-hoc reconformalization to match finite-sample guarantees.

4. Theoretical Guarantees and Coverage Properties

All methods above guarantee finite-sample marginal or joint coverage for the hyperrectangular prediction set under exchangeability. Specific guarantees include:

• Split-conformal, copula, vine-copula, and minimax methods provide

$\Pr\big[Y \in C(y)\big] \geq 1-\alpha$

with finite-sample calibration (Wen et al., 17 Nov 2025, Messoudi et al., 2021, Park et al., 4 Nov 2024, Sampson et al., 6 Jun 2024).

• Minimax methods additionally guarantee asymptotic balance and tightness across metrics, achieving equal marginal coverage and minimal width (Wen et al., 17 Nov 2025).
• For hyperrectangular regions, asymptotic marginal balance holds under identically distributed error or continuous quantile assumptions (Sampson et al., 6 Jun 2024).
• Advanced extensions support conditional coverage (e.g., under non-exchangeable or group-structured data) via weighted or stratified calibration (Wen et al., 17 Nov 2025).

5. Empirical Evaluation and Practical Performance

Empirical benchmarks on multi-metric BIQA—commonly using fastMRI, PSNR, SSIM, LPIPS, DISTS, and others—show:

• Minimax and copula-based conformal methods achieve valid joint coverage, sharper intervals, and balanced single-metric coverage compared to naive independence or Bonferroni adjustments (Wen et al., 17 Nov 2025).
• Vine-copula and semiparametric influence-corrected methods yield the smallest prediction set volumes for a given coverage, especially in the presence of correlated metric errors (Park et al., 4 Nov 2024).
• Non-convex, flow-based approaches (VSPS) significantly reduce prediction set volumes in the presence of complex joint or non-Gaussian metric distributions (Luo et al., 4 Mar 2025).
• Simulation and real-data studies confirm all valid multi-metric conformal approaches outperform separate calibration in joint and marginal coverage, efficiency, and robustness (Wen et al., 17 Nov 2025, Sampson et al., 6 Jun 2024, Messoudi et al., 2021).

Method	Joint Coverage Guarantee	Marginal Balance	Typical Shape
Minimax (Wen et al., 17 Nov 2025)	Finite-sample, joint	Asymptotic	Hyperrectangle
Copula/vine (Park et al., 4 Nov 2024, Messoudi et al., 2021)	Finite-sample, joint	Yes	Flexible
Volume-Sorted (Luo et al., 4 Mar 2025)	Finite-sample, marginal	N/A	Non-convex
Hyperrectangle (Sampson et al., 6 Jun 2024)	Finite-sample, joint	Asymptotic	Hyperrectangle
ERM/Surrogate (Bai et al., 2022)	Approximate	Yes	Data-driven

6. Extensions and Limitations

• MM-BIQA conformal methods straightforwardly apply to any set of continuous metrics, tasks (e.g., classification, segmentation), or multi-round measurement stopping (Wen et al., 17 Nov 2025).
• All approaches require exchangeability or i.i.d. calibration and test data; performance under covariate shift or nonstationarity may degrade unless weighted conformal or online extensions are used (Wen et al., 17 Nov 2025).
• Most constructions restrict to axis-aligned (hyperrectangular) sets for tractability, though flow- and copula-based methods enable adaptation to general shapes.
• Empirical copulas can be sample-inefficient for high-dimensional metric vectors; vine copulas and surrogate minimax methods mitigate this issue (Park et al., 4 Nov 2024, Wen et al., 17 Nov 2025).
• Conditional (e.g., subgroup or local) coverage, online calibration, and non-rectangular regions remain active areas for extension (Wen et al., 17 Nov 2025).

7. Representative Pipeline for Multi-Metric BIQA (Minimax Approach)

A canonical pipeline to construct multi-metric coverage regions:

1. Data Preparation: Assemble calibration samples with reconstructed images $f(y_i)$ and their known references $x_i^\star$.
2. Forecasting and Nonconformity Scoring: Compute metrics $Y_{i,k} = m_k(f(y_i), x_i^\star)$ and predicted distributions or quantiles per metric.
3. Marginal CDF Estimation: Use a portion of data to compute empirical marginal CDFs of residuals for each metric.
4. Joint Calibration: On a held-out calibration set, transform residuals to pseudo-uniforms and calibrate the maximal quantile to achieve joint $1-\alpha$ coverage (e.g., maximize the minimum single-metric acceptance rate under the constraint).
5. Prediction Region Construction: For a new $y$, output

$$C(y) = \prod_{k=1}^K \left\{ z: \hat{F}_{S_k}(s_k(z)) \leq \hat{\lambda} \right\}$$

where $s_k$ is the nonconformity score for metric $k$ and $\hat{\lambda}$ is the calibrated joint threshold (Wen et al., 17 Nov 2025).

This approach applies generally across MM-BIQA tasks, leveraging as a subroutine the advances in multi-target conformal prediction, copula modeling, and nonparametric quantile estimation under exchangeability.

---

References:

• "Minimax Multi-Target Conformal Prediction with Applications to Imaging Inverse Problems" (Wen et al., 17 Nov 2025)
• "Conformal Multi-Target Hyperrectangles" (Sampson et al., 6 Jun 2024)
• "Copula-based conformal prediction for Multi-Target Regression" (Messoudi et al., 2021)
• "Semiparametric conformal prediction" (Park et al., 4 Nov 2024)
• "Volume-Sorted Prediction Set: Efficient Conformal Prediction for Multi-Target Regression" (Luo et al., 4 Mar 2025)
• "Efficient and Differentiable Conformal Prediction with General Function Classes" (Bai et al., 2022)