Q-Hawkeye: RL for Visual IQA

• Q-Hawkeye is a reinforcement learning-based framework that formulates no-reference image quality assessment as a Markov Decision Process.
• It employs Uncertainty-Aware Dynamic Optimization (UADO) to scale policy updates based on sample reliability and reduce unstable predictions.
• The approach integrates Perception-Aware Optimization (PAO) by contrasting pristine and degraded images to enforce visual grounding.

Q-Hawkeye denotes a reinforcement learning-based framework for reliable visual policy optimization in no-reference image quality assessment (NR-IQA). The framework explicitly addresses sample-level reliability and perceptual grounding limitations that afflict prior multimodal LLM (MLLM) approaches to IQA. Q-Hawkeye introduces two architectural and algorithmic advances: Uncertainty-Aware Dynamic Optimization (UADO) and Perception-Aware Optimization (PAO), formulating policy learning as a Markov Decision Process and optimizing with respect to both predictive stability and perceptual sensitivity (Xie et al., 30 Jan 2026).

1. NR-IQA as a Vision-Language Markov Decision Process

Q-Hawkeye formulates NR-IQA as an episodic MDP, where the policy $\pi_\theta$ is a vision-language transformer generating a concise reasoning-and-answer sequence $o$ string in response to an image $I$ and a fixed prompt $q$. Each MDP state consists of $(I,q)$ and the partially generated output tokens $o_{<t}$. Actions correspond to generating the next token $o_t$ from vocabulary $V$, with deterministic token transitions until a special terminal token. Reward feedback is assigned only at episode completion, with:

$R = R_{\mathrm{acc}} + R_{\mathrm{fmt}},$

where $R_{\mathrm{acc}} = \exp(-|\hat{y} - y|/\alpha)$ (with $o$0 being the ground-truth mean opinion score (MOS) and $o$1 the scalar parsed IQA score from the predicted answer), and $o$2 if the output matches required formatting, else $o$3. The only stochasticity arises from policy sampling.

2. Uncertainty-Aware Dynamic Optimization (UADO)

Q-Hawkeye samples $o$4 rollouts for each $o$5 pair under the current policy $o$6, parsing predicted scores $o$7 and computing the sample mean and variance:

$o$8

This variance $o$9 operationalizes predictive uncertainty. To normalize, the maximal attainable variance for MOS in $I$0 is $I$1, so $I$2 is clipped and rescaled:

$I$3

A downweighting factor $I$4 with temperature $I$5 is applied to rescale the policy gradient update strength for the entire sample, yielding for advantage $I$6:

$I$7

where $I$8 uses within-sample group relative normalization. All subsequent policy gradient and GRPO surrogate loss terms are modified to use these weighted advantages, thereby reducing the impact of high-uncertainty (unstable) samples on learning.

3. Perception-Aware Optimization via Implicit Perception Loss

To enforce true visual grounding, for each pristine input image $I$9, Q-Hawkeye synthesizes a randomly degraded image counterpart $q$0 using noise, blur, JPEG compression, or darkening operators.

The implicit perception loss is defined as

$q$1

Maximizing this KL divergence encourages the policy’s output distribution to diverge when degradation alters visual quality, discouraging text-only or language-prior-driven judgments. To avoid trivial high-entropy solutions, a double-entropy regularization term penalizes excessive randomness in both pristine and degraded conditions.

The full PAO objective term becomes: $q$2 with $q$3, $q$4.

4. End-to-End Optimization Algorithm and Implementation Details

The training routine iteratively processes each mini-batch through uncertainty-aware sampling (UADO), degraded-input resampling (PAO), and updates the model via AdamW. All optimizations are conducted on Qwen2.5-VL-7B backbones. Key hyperparameters include $q$5 for reward tolerance, PPO-style clipping ($q$6), KL-regularization ($q$7), and batch size 32 for 15 epochs.

All data augmentations for degradation are human- and GPT-4o–filtered to ensure perceptual relevance; e.g., Gaussian noise ($q$8), blur ($q$9), JPEG quality=30, darken=0.6.

5. Empirical Evaluation and Comparative Results

Q-Hawkeye is trained only on the KonIQ-10k dataset and tested in-distribution (KonIQ test) and out-of-distribution (SPAQ, LIVE-Wild, FLIVE, KADID-10K, CSIQ, PIPAL, AGIQA-3K) without fine-tuning. Metrics are PLCC and SRCC between predicted and human MOS.

Across these benchmarks, Q-Hawkeye achieves 80.0 PLCC and 76.2 SRCC—outperforming all prior MLLM-based IQA methods trained solely on KonIQ. Even compared to multi-dataset state-of-the-art approaches like DeQA-Score and VisualQuality-R1, the single-dataset Q-Hawkeye remains competitive or superior.

Ablation studies demonstrate that both UADO and PAO contribute significant, complementary improvements. UADO alone yields 77.5/73.8 (PLCC/SRCC), PAO variants alone yield 76.9–78.9/73.4–75.5, but the integrated framework attains 80.0/76.6.

6. Analysis: Reliability, Robustness, and Generalization

Module ablations confirm that UADO stabilizes training, reducing reward variance and raising average reward. Hyperparameter sweeps for temperature $(I,q)$0, perception regularization $(I,q)$1, and entropic weights $(I,q)$2 reveal well-behaved optima near defaults. Increasing rollout count $(I,q)$3 brings rapidly diminishing returns beyond $(I,q)$4.

The perception gap (score differential between $(I,q)$5 and $(I,q)$6) monotonically increases during training under PAO, providing direct evidence that the policy becomes more visually sensitive rather than language-prior-driven. Crucially, the framework generalizes across highly diverse out-of-distribution datasets despite training on a single set.

7. Contributions and Broader Impact

Q-Hawkeye advances RL-based IQA by reformulating reliability as a function of sample-wise prediction stability and perception grounding. The UADO mechanism adaptively scales gradient updates to filter unreliable samples, while PAO enforces visual evidence dependence via original/vs/degraded contrast. This results in substantial improvements in robustness, cross-dataset generalization, and ultimately, visually grounded quantitative IQA performed by modern MLLM systems (Xie et al., 30 Jan 2026).