RSFIQA: Region-aware Semantic NR-IQA

• RSFIQA is a no-reference image quality assessment framework that evaluates images through semantic partitioning of regions.
• It integrates dynamic segmentation using SAM and MLLM-based extraction to quantify distortions like color, noise, and blur.
• The design employs region-aware semantic attention, ensuring quality scores closely reflect localized, human-like perception.

RSFIQA, short for Region-aware Semantic-guided Fine-grained Image Quality Assessment, is a no-reference image quality assessment (NR-IQA) framework that models image quality through semantically partitioned local regions rather than through a purely global representation. It was introduced to address two limitations of existing NR-IQA methods: a reliance on global representations that leads to limited insights into semantically salient regions, and the use of uniform weighting for region features that weakens sensitivity to local quality variations. The framework combines dynamic semantic segmentation with the Segment Anything Model (SAM), region-level description and distortion extraction with a Multi-modal LLM (MLLM), and a Region-Aware Semantic Attention (RSA) mechanism that injects region-constrained semantic bias into visual attention. In reported experiments, RSFIQA is backbone-agnostic and achieves competitive quality prediction performance across both synthetic and authentic benchmark datasets including LIVE, CSIQ, TID2013, KADID-10k, CLIVE, KonIQ-10k, and SPAQ (Song et al., 11 Aug 2025).

1. Conceptual basis and problem setting

RSFIQA is situated within no-reference image quality assessment, where the objective is to simulate the process of perceiving image quality aligned with subjective human perception without access to a pristine reference image. Its central premise is that image distortions are not uniformly distributed across a scene: different semantic regions can exhibit distinct degradation types and severities. The model therefore treats fine-grained regional analysis as a first-class component of quality prediction rather than as a secondary explanation layer (Song et al., 11 Aug 2025).

The framework is designed to capture multi-dimensional quality discrepancies. In the reported implementation, region-level analysis explicitly considers descriptive content together with distortion attributes spanning color, noise, artifact, blur, and overall quality. A common simplification would be to interpret the language-model component as a captioning module; RSFIQA instead uses it to couple local semantics with structured distortion perception. This suggests that the model’s quality signal is intended to be both semantically conditioned and distortion-aware, rather than merely text-augmented.

2. Dynamic semantic partitioning of the image

The first stage of RSFIQA uses SAM to dynamically partition an input image $I \in \mathbb{R}^{H \times W \times 3}$ into $L$ semantically meaningful, non-overlapping regions. SAM produces segmentation masks $M \in \mathbb{R}^{H \times W \times L}$ through post-processing. The top masks, ranked by predicted IoU, are retained as meaningful regions, and pixels covered by none are assigned as background (Song et al., 11 Aug 2025).

This design differs from conventional patch-based or globally pooled NR-IQA pipelines in that the segmentation is explicitly semantic. The purpose is not only spatial subdivision but also content-sensitive quality assessment. Because the regions are non-overlapping, later attention operations can be regularized around region membership. A plausible implication is that semantic decomposition serves as both a representation prior and an attention constraint, which is consistent with the later use of region-internal attention bias.

3. Region-level semantic and distortion extraction

For each segmented region $M_i \cdot I$, RSFIQA queries an MLLM, specifically Qwen2.5-VL in experiments, to generate two categories of outputs: descriptive content for the region and multi-dimensional distortion ratings and scores. The prompt format elicits both qualitative levels and quantitative scores for the attributes color, noise, artifact, blur, and overall quality. Outputs from all $L$ regions are then tokenized and encoded into a regional text embedding matrix $E \in \mathbb{R}^{n \times d}$ (Song et al., 11 Aug 2025).

This region-level language supervision is paired with a Multi-scale Hierarchical Feature Extraction (MHF) module. Features $F_i$ from different backbone stages are downsampled by gated convolution and pooling to a common resolution, forming $D_i$. Context is aggregated through self-attention and cross-attention: $D'_i = \mathrm{Attn}(D_i, D_i, D_i) + D_i$

$$D''_i = \mathrm{Attn}(D''_{i+1}, D'_i, D'_i) + D''_{i+1}$$

The function of MHF is to provide multi-granularity visual features before semantic fusion. In this formulation, region text embeddings and hierarchical image features are complementary: the former encode localized semantics and perceived distortion attributes, while the latter retain the visual evidence needed for regression.

4. Region-Aware Semantic Attention and score prediction

The defining fusion component in RSFIQA is the Region-Aware Semantic Attention (RSA) mechanism. For each region $L$0, the semantic embedding $L$1 is used as the query, while the masked region $L$2 provides key and value: $L$3 The resulting region-aware features are spatially interpolated and reshaped: $L$4

RSA then introduces an attention bias

$L$5

where $L$6 regularizes the influence. This bias is injected into the visual attention calculation: $L$7 with $L$8 derived from image features $L$9. The reported effect is that only pixels within the same semantic region contribute to each other during attention computation, suppressing unrelated or noisy interactions (Song et al., 11 Aug 2025).

After RSA fusion, the aggregated features $M \in \mathbb{R}^{H \times W \times L}$0 are passed through a self-attention block, pooling, and an MLP to produce the final predicted quality score: $M \in \mathbb{R}^{H \times W \times L}$1 Training uses mean squared error between normalized predicted and ground-truth scores: $M \in \mathbb{R}^{H \times W \times L}$2

A common misconception would be that semantic attention in RSFIQA simply reweights salient areas. The mathematical construction is narrower: RSA inserts an intra-region attention bias, so its role is not only emphasis but also restriction of interaction structure.

5. Backbone-agnostic design and experimental behavior

RSFIQA is described as backbone-agnostic. The image backbone is treated as a modular feature extractor, and the reported experiments instantiate the framework with both ResNet-50 and Swin Transformer architectures. Region-level processing—segmentation, semantic extraction, and attention fusion—operates independently of the precise backbone, allowing integration into different deep neural network architectures (Song et al., 11 Aug 2025).

Evaluation is reported on synthetic datasets—LIVE, CSIQ, TID2013, KADID-10k—and authentic datasets—CLIVE, KonIQ-10k, SPAQ—using PLCC and SRCC. The RSFIQA-ResNet50 variant attains 0.983/0.981 on LIVE, 0.972/0.968 on CSIQ, 0.957/0.946 on TID2013, 0.949/0.945 on KADID-10k, 0.872/0.833 on CLIVE, 0.937/0.932 on KonIQ-10k, and 0.923/0.916 on SPAQ. On TID2013, the paper reports a new high of 0.957 (PLCC) / 0.946 (SRCC). The model is further described as showing average improvements over strong baselines such as VISGA, QCN, and LoDa of about 1.0% in both PLCC and SRCC (Song et al., 11 Aug 2025).

Cross-dataset evaluation indicates robust generalization when trained on KonIQ-10k and tested on unseen datasets. Ablation studies report that SAM segmentation, MLLM, and RSA each contribute positively and jointly boost performance, and that including all five distortion attributes in the prompts yields the best accuracy. Qualitative visualizations are reported to show that region-level semantic analysis and scoring closely track human subjective quality rankings. Taken together, these findings position RSFIQA as a fine-grained NR-IQA system whose gains are tied to explicit regional semantics rather than to a backbone substitution alone.

6. Relation to adjacent IQA frameworks and acronym usage

Within the broader IQA landscape, RSFIQA belongs to a larger movement toward continuous score prediction and task-aware quality modeling, but its technical profile is distinct. In fundus image quality assessment, FundaQ-8 defines eight expert-validated parameters and trains a ResNet18-based regression model to predict continuous scores in $M \in \mathbb{R}^{H \times W \times L}$3 (Zun et al., 25 Jun 2025). FTHNet combines a transformer backbone, a distortion perception network, a parameter hypernetwork, and a target network to regress fundus Mean Opinion Scores on the FQS dataset (Gong et al., 2024). In surveillance facial image quality assessment, SFIQA-Bench and SFIQA-Assessor jointly model noise, sharpness, colorfulness, contrast, fidelity, and overall quality through multi-task regression (Jiang et al., 7 Feb 2026). Relative to these systems, RSFIQA’s distinctive contribution is the explicit integration of semantic segmentation, region-level MLLM prompting, and region-constrained attention bias for generic NR-IQA.

The acronym RSFIQA also carries a separate usage in surveillance face-quality literature, where it appears as shorthand for recognition-oriented surveillance facial image quality assessment (Jiang et al., 7 Feb 2026). That usage is conceptually different from Region-aware Semantic-guided Fine-grained Image Quality Assessment in generic NR-IQA. The overlap is terminological rather than methodological.

This distinction matters because the two lines of work optimize different targets. Recognition-oriented facial quality methods prioritize encoder-aligned recognizability or fidelity, as seen in frameworks such as tinyFQnet, which defines quality via cosine similarity to a class center (Peng et al., 2021), and IG-FIQA, which introduces intra-class variance guidance for robust pseudo-label-based FIQA training (Kim et al., 2024). RSFIQA in the sense of (Song et al., 11 Aug 2025), by contrast, addresses general image quality prediction through semantically localized distortion understanding.