Causal Disentanglement for Full-Reference Image Quality Assessment

Abstract: Existing deep network-based full-reference image quality assessment (FR-IQA) models typically work by performing pairwise comparisons of deep features from the reference and distorted images. In this paper, we approach this problem from a different perspective and propose a novel FR-IQA paradigm based on causal inference and decoupled representation learning. Unlike typical feature comparison-based FR-IQA models, our approach formulates degradation estimation as a causal disentanglement process guided by intervention on latent representations. We first decouple degradation and content representations by exploiting the content invariance between the reference and distorted images. Second, inspired by the human visual masking effect, we design a masking module to model the causal relationship between image content and degradation features, thereby extracting content-influenced degradation features from distorted images. Finally, quality scores are predicted from these degradation features using either supervised regression or label-free dimensionality reduction. Extensive experiments demonstrate that our method achieves highly competitive performance on standard IQA benchmarks across fully supervised, few-label, and label-free settings. Furthermore, we evaluate the approach on diverse non-standard natural image domains with scarce data, including underwater, radiographic, medical, neutron, and screen-content images. Benefiting from its ability to perform scenario-specific training and prediction without labeled IQA data, our method exhibits superior cross-domain generalization compared to existing training-free FR-IQA models.

• The paper presents a novel causal disentanglement framework that models image content and degradation using structural causal models to improve FR-IQA.
• It utilizes an intervention-based autoencoder and nonlinear causal layers to explicitly encode human visual masking effects and enhance cross-domain generalization.
• Empirical evaluations demonstrate superior performance with SRCC/PLCC scores above 0.9 across diverse datasets, highlighting its robustness and label-free adaptability.

Causal Disentanglement Framework for Full-Reference Image Quality Assessment

Motivation and Problem Abstraction

The paper "Causal Disentanglement for Full-Reference Image Quality Assessment" (2604.21654) presents a new paradigm for FR-IQA, challenging the prevailing dual-branch feature comparison architecture in favor of a causal disentangled representation framework. The authors observe that traditional models, dominated by deep feature map difference computation, suffer from restricted adaptability and heavy dependence on MOS-labeled datasets, especially under data-scarce conditions or for out-of-distribution domains. The proposed method leverages content invariance and integrates causal modeling, explicitly encoding the human visual masking effect to extract content-influenced degradation features.

Figure 1: Comparison of typical FR-IQA architecture (feature difference computation) with the causal disentanglement approach (content-influenced degradation extraction).

Causal Modeling in FR-IQA

The authors formalize the FR-IQA process using Structural Causal Models (SCMs), identifying image content ($C$) and degradation ($D$) as the primary generative factors for distorted images. The causal formulation $C \to I_d \leftarrow D$ encapsulates the content and degradation pathways. However, decoupling degradation alone is insufficient for human perceptual alignment due to the intrinsic content-degradation interaction governed by visual masking. The proposed SCM introduces explicit content-aware intervention, refining traditional FR-IQA concepts beyond mere difference computation.

Figure 2: SCM for FR-IQA illustrating the content-degradation paths in image generation and perception.

Visual Masking Effect Quantification

Distortion visibility is content-dependent: smooth content may mask blur but accentuate quantization noise, whereas textured content may exhibit the opposite effect. The paper substantiates this with controlled visual examples, reinforcing the necessity of explicit content-degradation causal modeling.

Figure 3: Differential distortion visibility under identical intensities on images with distinct content characteristics.

Methodological Pipeline

Pre-training and Fine-tuning Strategy

The pipeline comprises synthetic degradation pre-training, scenario-specific fine-tuning, and multi-modality prediction (supervised regression, few-shot, and zero-shot ranking). Synthetic data is generated using reference images, guaranteeing content invariance and facilitating unsupervised functional separation between degradation and content.

Figure 4: Pre-training and fine-tuning stages for FR-IQA with synthetic degraded datasets and adaptation to target scenarios.

Latent Representation Intervention

To achieve degradation-content disentanglement, the method utilizes an autoencoder structure with explicit intervention: content features from the reference image replace part of the distorted image's latent code, forcing the complementary latent space to encode degradation. This is theoretically motivated by content invariance in FR-IQA.

Figure 5: Standard and intervention-based autoencoder architectures for latent factor decoupling.

Explicit Causal Layer

A nonlinear causal layer, modulated by a learned content mask, governs degradation feature transformation as per a two-node DAG ($C \to D$), imposing a structured approximation of the masking effect. Decoder reconstruction leverages FiLM channel modulation for robust perceptual feature learning.

Figure 6: Decoder and causal layer integrating content modulation via channel gating and residual masking.

Figure 7: SCM for quality score prediction explicitly encoding the content-degradation causal path.

Score Prediction and Manifold Ranking

When labels are available, a fully connected regression head maps content-influenced degradation features to subjective scores. In label-free settings, UMAP recovers a 1D quality coordinate—relative quality ranking is achieved by projecting high-dimensional features onto a learned quality manifold, which exhibits locally monotonic order with empirical separation between high- and low-quality samples.

Figure 8: UMAP embedding on TID2013 illustrating quality-related, monotonic structure after pre-training.

Figure 9: Flowchart of zero-shot prediction via UMAP-based 1D manifold projection.

Numerical Evaluation and Generalization

Extensive experiments demonstrate highly competitive performance on LIVE, CSIQ, TID2013, KADID, and PIPAL, with PLCC and SRCC exceeding 0.9 in most cases, outperforming both traditional metrics and established deep feature comparison approaches even in cross-domain generalization scenarios where labeled IQA data are scarce. Specifically, the method achieves SRCC/PLCC of 0.948/0.956 on infrared, 0.942/0.947 on neutron radiography, and 0.915/0.926 on screen content images. Ablations reveal the necessity of causal masking, FiLM channel modulation, and the superiority of UMAP projection versus alternative dimensionality reduction methods. The framework exhibits resilience to UMAP parameter variations, maintaining robust performance under practical deployment conditions.

Figure 10: Reference and distorted images from diverse domains demonstrating cross-domain adaptability.

Theoretical and Practical Implications

The proposed paradigm bridges human visual system psychophysics and representation learning, forging explicit causal links between semantic content and perceptual degradation. It enables scenario-specific adaptation without the need for costly subjective labels, a critical advancement for FR-IQA deployment in medical, radiographic, remote sensing, and synthetic imagery contexts. The intervention-based disentanglement and causal modulation ensure that extracted features reflect not just mechanical signal differences but the perceptual impact as experienced by human observers.

This approach suggests future directions in leveraging causal graphs for other perceptual metrics, integrating domain knowledge into latent space interventions, and expanding the paradigm to other modalities (e.g., video, hyperspectral).

Figure 11: Visualization of transfer of degradations to reference content by the proposed method, enabling assessment of causal perceptual impact.

Limitations and Failure Cases

Despite its strength, the method relies on clean reference imagery and explicit synthetic degradation simulation for pre-training, which restricts "plug-and-play" applicability compared to purely training-free approaches. Performance diminishes on blockwise distortions due to breakdown in reconstruction-driven supervision and content-degradation separation.

Conclusion

This paper establishes a causal disentanglement paradigm for FR-IQA, enabling efficient, label-free domain adaptation and robust perceptual feature extraction through explicit modeling of content-influenced degradation. Empirical and theoretical evidence confirms superiority in both standard and cross-domain settings, providing a foundation for further causal representation methods in perceptual metric learning.