PixelQA: Visual Reasoning & Quality Assessment

Updated 12 July 2026

PixelQA is a term describing both pixel-level visual reasoning for object-centric tasks and dense quality assessment for localized image evaluation.
It distinguishes between tasks that require segmentation, tracking, and guided question answering and methods that produce pixel-wise utility or quality maps.
Recent advances like UniPixel leverage object memory banks, temporal encoding, and refined mask propagation to improve both reasoning and quality attribution.

PixelQA is an overloaded term in recent vision literature. In one usage, it denotes a benchmark for pixel-level visual reasoning in which a model receives an image or video, a text prompt, and optional visual prompts, and returns a textual response together with grounded spatial-temporal masks (Liu et al., 22 Sep 2025). In another usage, closely related work uses “PixelQA” or pixel-level quality assessment to denote dense spatial estimation of image utility or perceptual quality, replacing a single image-level score with a map over pixels or regions (Terhörst et al., 2021). The distinction is consequential: the former addresses grounding, referring, segmentation, tracking, and question answering; the latter addresses localized quality attribution, interpretability, and image assessment. The term is also susceptible to bibliographic confusion, since the item listed as “Pixel Perfect: Relational Image Quality Assessment with Spatially-Aware Distortions” is described as a “LaTeX Guidelines for Author Response” template rather than an actual IQA method paper (Khan et al., 4 May 2026).

1. Terminological scope

Recent arXiv usage supports two established meanings of PixelQA. The first is a task name introduced by UniPixel for object-centric reasoning at pixel granularity. The second is a problem family centered on dense image-quality estimation, especially when a scalar MOS or FIQA score is considered too coarse for explanation or targeted optimization (Liu et al., 22 Sep 2025, Terhörst et al., 2021).

Usage	Core formulation	Representative source
Pixel-level visual reasoning	Textual answer plus grounded masks from image/video, question, and optional visual prompts	UniPixel (Liu et al., 22 Sep 2025)
Pixel-level quality assessment	Dense local quality or utility map, sometimes aggregated into an image-level score	Face PixelQA (Terhörst et al., 2021), pMOS (Kim et al., 2022), FunPiQ (Wang et al., 24 Jun 2026)

This dual usage matters because the two lines of work optimize different targets. UniPixel’s PixelQA is about which object is intended, where it is, and how object-centric evidence should condition reasoning. Pixel-level quality assessment is about which pixels are useful, which regions are degraded, and how local quality should contribute to a global assessment. Treating them as a single task obscures both the evaluation protocol and the supervision regime.

2. PixelQA as pixel-level visual reasoning

UniPixel defines a broad family of pixel-level reasoning tasks with image or video $\mathcal{X}$ , text prompt $\mathcal{T}$ , and optional visual prompts $\{\mathcal{P}_i\}_{i=1}^{N}$ as inputs, and textual responses plus grounded spatial-temporal masks $\{\mathcal{M}_i\}_{i=1}^{K}$ as outputs. The paper states the formulation explicitly: visual prompts may be points, boxes, or masks on a specific frame, both $N$ and $K$ may be zero, and $K$ need not equal $N$ because the model may segment extra objects or regions not specified by the prompts (Liu et al., 22 Sep 2025).

Within that framework, PixelQA is introduced from VideoRefer-Bench $^Q$ by replacing the original mask prompts with more ambiguous point or box prompts applied on a single frame. The task requires a model to use the question together with the sparse prompt to determine which object is intended, predict the target mask in the prompted frame, propagate or track that object through the video, use the resulting object-centric features during reasoning, and return both the answer and the object masks. UniPixel summarizes the requirement as follows: given ambiguous visual cues, models are expected to correctly identify the target object according to the question and the visual prompt, then respond with both the textual answer and the corresponding object masks (Liu et al., 22 Sep 2025).

This formulation separates PixelQA from adjacent tasks. It is not standard RES, because segmentation is not the terminal objective; it is an intermediate reasoning primitive. It is not standard VQA, because output masks are part of the answer protocol. It is not region-level captioning or referred video QA with mask input, because the hard grounding step is not outsourced. A plausible implication is that PixelQA is best understood as a compositional stress test for referring, segmentation, tracking, and QA under sparse prompting, rather than as a mere extension of any one of those tasks.

3. UniPixel architecture and empirical profile

UniPixel is built on Qwen2.5-VL with an LLM backbone, a ViT-based visual encoder with dynamic resolution, a prompt encoder for visual prompts, an object memory bank, and a SAM 2.1 mask decoder. The system introduces special tokens <REF>, <MEM>, and <SEG>. Visual prompts are encoded into tokens consumable by the LLM; points are represented as $(x,y,t)$ , boxes as $\mathcal{T}$ 0, and mask prompts are converted by masked pooling over visual encoder features and an M-to-L projector. For sparse prompts, UniPixel uses 2D Fourier embeddings for spatial positions, 1D Fourier embeddings for frame indices, and a GELU-to-Linear projection to the LLM embedding space (Liu et al., 22 Sep 2025).

The defining mechanism is the object memory bank, a hashmap whose keys are object IDs and whose values are spatial-temporal masks. During memory pre-filling, <REF> triggers an auxiliary response that identifies relevant objects, emits <SEG>, predicts masks, and stores object-mask pairs in memory. During memory injection, the model downsamples each frame-level object mask to visual-token resolution, applies masked pooling to extract object-centric visual features, projects those features, and replaces corresponding <MEM> tokens. This design makes predicted masks internal computational objects rather than mere outputs, which is why UniPixel can continue reasoning over grounded objects across turns or frames (Liu et al., 22 Sep 2025).

The reported PixelQA results show that this design supports both mask generation and question answering. On mixed prompts, UniPixel-3B reaches 57.2 $\mathcal{T}$ 1, 64.1 $\mathcal{T}$ 2, 60.6 $\mathcal{T}$ 3, 70.8 Acc, while UniPixel-7B reaches 57.5, 64.7, 61.1, 71.0 Acc. The larger generic LMM baselines, InternVL2-26B and Qwen2-VL-72B, are reported only on QA accuracy, at 60.9 Acc and 69.1 Acc respectively. The ablations identify four particularly important factors: task unification, the object memory bank, temporal encoding, and mask propagation. On PixelQA mixed, the full object memory bank gives 49.0 $\mathcal{T}$ 4, 68.5 Acc, compared with 46.8, 64.5 for <REF> only; removing temporal encoding drops performance to 44.3, 63.7; and replacing propagation with independent per-frame decoding reduces performance from 49.0, 68.5 to 46.1, 66.2 (Liu et al., 22 Sep 2025).

These numbers suggest that PixelQA, in the UniPixel sense, is primarily a test of intermediate grounded state construction. The benchmark rewards not just recognition of the queried object but the ability to turn that object into a reusable latent unit for downstream reasoning.

4. PixelQA as pixel-level quality assessment

A distinct literature uses PixelQA-style terminology for dense quality or utility estimation. In face recognition, pixel-level face image quality is defined as the utility of single pixels in a face image for recognition, and the proposed method is explicitly framed as a training-free approach to assess the pixel-level qualities of a face image given an arbitrary face recognition network. It estimates a model-specific scalar quality value, constructs a sample-specific quality regression model, backpropagates quality-based gradients, and converts them into a final pixel-level quality map $\mathcal{T}$ 5 with values $\mathcal{T}$ 6 (Terhörst et al., 2021). In NR-IQA, pIQA introduces pixel-by-pixel MOS (pMOS) together with an ROI map and aggregates them into an image-level MOS through

$\mathcal{T}$ 7

with only image-level MOS supervision (Kim et al., 2022). In fundus imaging, FunPiQ introduces the first FIQA benchmark to provide pixel-level quality annotations and defines three classes—good, usable, and bad—through anatomical visibility (Wang et al., 24 Jun 2026).

Setting	Dense output	Key mechanism
Face recognition	Pixel utility map $\mathcal{T}$ 8	SER-FIQ, sample-specific quality node, gradient backpropagation
NR-IQA	pMOS + ROI	Local IQA, ROI prediction, high-level feature embedding
Fundus FIQA	Good/usable/bad map	EFIQA-CP with DINOv3 features and nnPU

These works share a rejection of purely global scoring. The face-recognition line emphasizes explainability for enrolment and ICAO-style compliance. The NR-IQA line emphasizes that pooling-heavy backbones lose spatial fidelity and therefore cannot produce genuine per-pixel MOS. FunPiQ emphasizes that image-level FIQA depends on downstream task criteria, whereas anatomical visibility is more task-agnostic and quantitatively evaluable at the local level (Terhörst et al., 2021, Kim et al., 2022, Wang et al., 24 Jun 2026).

The contrast with scalar IQA is instructive. VAE-QA remains a full-reference IQA method that requires both a pristine reference image and a distorted image and predicts a scalar MOS-aligned quality score; it improves cross-dataset generalization and efficiency, but it does not define pixel-level outputs (Raviv et al., 2024). Pixel-level quality assessment therefore occupies a different methodological niche: it is not merely “better FR-IQA” or “better BIQA,” but a shift from a scalar target to a spatial field.

5. Supervision, benchmarks, and calibration regimes

No single supervision paradigm defines PixelQA across the literature. UniPixel trains in three stages: prompt encoder pretraining on 851K regional captioning data, LLM–mask decoder alignment on 87K referring segmentation data, and joint multitask training on around 1M samples, including memory pre-filling data and general video understanding data (Liu et al., 22 Sep 2025). The benchmark itself evaluates mask prediction with $\mathcal{T}$ 9, $\{\mathcal{P}_i\}_{i=1}^{N}$ 0, and $\{\mathcal{P}_i\}_{i=1}^{N}$ 1, and evaluates QA with multiple-choice Accuracy. This is a fully multimodal training and evaluation stack.

Pixel-level quality assessment uses more heterogeneous supervision. pIQA learns dense outputs from image-level MOS only, with no ground-truth pMOS labels, no ROI labels, no fixation maps, and no distortion masks (Kim et al., 2022). PIQ23 instead uses pairwise comparisons collected from over 30 image quality experts for face detail preservation, face target exposure, and overall image quality, then derives scene-wise JOD scales and bootstrap confidence intervals through psychometric scaling (Chahine et al., 2023). PICNIQ generalizes that logic by predicting the preference likelihood of quality between image pairs,

$\{\mathcal{P}_i\}_{i=1}^{N}$ 2

and then converting comparison graphs into JOD-like scores with TrueSkill rather than training a direct absolute-score regressor (Chahine et al., 2024). FHIQA adds scene-dependent score rescaling through a weighted combination over the top- $\{\mathcal{P}_i\}_{i=1}^{N}$ 3 predicted scenes, reflecting the claim that portrait quality should be predicted conditional on image semantics and that unseen scenes may be semantically intermediate between multiple training scenes (Chahine et al., 2024). FunPiQ uses expert pixel-level labels for evaluation, but EFIQA-CP itself is trained with quality pseudo-labels based on anatomical visibility and Non-Negative Positive-Unlabeled learning,

$\{\mathcal{P}_i\}_{i=1}^{N}$ 4

with class prior $\{\mathcal{P}_i\}_{i=1}^{N}$ 5 (Wang et al., 24 Jun 2026).

An adjacent infrastructure line is also relevant. PixelWeb provides 100,000 annotated web pages with BGRA element bitmaps, masks, contours, BBoxes, layer positions, and computed style information. The dataset is not itself a PixelQA benchmark, because it does not include natural-language QA pairs, but this suggests a strong substrate for future GUI PixelQA centered on visibility-aware grounding, occlusion reasoning, and pixel-to-DOM alignment (Yang et al., 23 Apr 2025).

6. Misconceptions, limitations, and emerging directions

A common misconception is that PixelQA names a single standardized task. The literature does not support that view. UniPixel uses PixelQA for joint referring, segmentation, tracking, and question answering at the pixel level, whereas face-recognition, NR-IQA, and FIQA works use related terminology for localized quality assessment and dense quality attribution (Liu et al., 22 Sep 2025, Terhörst et al., 2021, Wang et al., 24 Jun 2026). A second misconception is that any “PixelQA” citation automatically refers to a concrete method. The supplied record for “Pixel Perfect: Relational Image Quality Assessment with Spatially-Aware Distortions” explicitly states that the document is not an IQA research article but a CVPR-style author rebuttal template, so it provides no factual basis for reconstructing a relational or spatially-aware PixelQA system (Khan et al., 4 May 2026).

The current limitations are equally domain-specific. In UniPixel, the discussion identifies limited scale of pixel-level training data, the fact that the mask decoder predicts on the first frame and propagates forward, and the small scale of ReasonSeg as constraints on further progress (Liu et al., 22 Sep 2025). In pIQA, the dense maps are learned indirectly from scalar MOS, so there are still no true pixel-level MOS labels (Kim et al., 2022). In face PixelQA, the method is model-specific and depends on access to model internals, stochastic dropout passes, and stable gradients (Terhörst et al., 2021). In FunPiQ, the benchmark has 300 images, is evaluation-only, and future work is explicitly directed toward multi-reader annotations and a larger benchmark (Wang et al., 24 Jun 2026).

Across these lines, a consistent pattern is emerging. PixelQA increasingly denotes tasks where the decisive variable is not merely what the model predicts, but where the evidence resides and whether that evidence can be reused. In visual reasoning, reusable evidence takes the form of mask-grounded object memory. In quality assessment, reusable evidence takes the form of pixel-level utility or degradability maps. This suggests that PixelQA is best regarded not as a single benchmark, but as a family of pixel-grounded assessment problems unified by dense spatial outputs, explicit intermediate structure, and a preference for explainable-by-design formulations over post-hoc localization.