Low-Level Distortion Perception Task

• Low-level Distortion Perception Tasks are diagnostic evaluations that assess a model’s ability to detect primitive signal degradations (e.g., blur, noise, and compression artifacts) across severity levels.
• Forced-choice classification tasks combined with fine-tuning of vision encoders elevate distortion recognition accuracy significantly, from 14.92% in the baseline up to 91.45% with optimized tuning.
• Methodologies integrate comprehensive datasets and rigorous mathematical alignment techniques to bridge the gap between perceptual fidelity metrics and high-level semantic interpretation.

Low-level distortion perception tasks constitute a class of diagnostic evaluations designed to probe an algorithm's or model's ability to recognize or differentiate primitive signal degradations—such as blur, noise, compression artifacts, phase distortions, and various parametric transformations—rather than relying solely on high-level assessment metrics or composite quality scores. These tasks are crucial for both understanding and improving low-level visual and audio models, especially in the context of vision-LLMs, image restoration, and representation learning, where a clear distinction between fine-grained distortion awareness and template-driven reasoning is sought.

1. Conceptual Basis and Motivation

The central motivation for low-level distortion perception tasks arises from the observation that high-performing models in image or signal quality assessment, particularly multi-modal vision-LLMs (MLLMs), often deliver plausible but unreliable assessments of fundamental distortions. While such models can generate coherent text explanations or scalar scores, they frequently fail to reflect actual perceptual sensitivity to canonical low-level degradations such as Gaussian blur, additive noise, or compression artifacts. This raises the following diagnostic question: does the model's internal representation genuinely encode low-level cues, or does it simply exploit high-level language or template mappings without robust perception of signal fidelity (Li et al., 10 Dec 2025)?

2. Representative Task Design and Datasets

A rigorous low-level distortion perception task typically involves forced-choice or classification of a broad spectrum of synthetic distortions applied to pristine samples. In the visual domain, Li et al. unify four established Image Quality Assessment (IQA) benchmarks—LIVE, CSIQ, KADID-10k, and TID2013—into a comprehensive 8-class dataset spanning pristine images and seven well-characterized synthetic degradations:

• Gaussian blur (parametric low-pass filtering)
• Additive Gaussian noise
• JPEG compression (block DCT quantization)
• Brightness shift
• Contrast change (rescaling)
• Colorfulness perturbation (random saturation/hue shift)
• Jitter (local displacement of patches)

Each distorted instance is synthesized at five severity levels per distortion and accompanied by an inquiry prompt (e.g., “Looking at the <image> above, what is the synthetic distortion in the image?”), forming a pool of 17,882 image-text pairs with an 80/20 train/test split. The explicit aim is not to regress to a continuous quality measure, but to assign one of the eight semantic classes, thereby providing a direct probe of the model's fine-grained discrimination power (Li et al., 10 Dec 2025).

3. Model Architectures and Component-wise Analysis

Modern approaches to low-level distortion perception in multi-modal models leverage compositional architectures. The mPLUG-Owl2 pipeline, as dissected by Li et al., comprises a frozen ViT-L/14 visual encoder, a low-dimensional projection layer mapping vision tokens into semantic space, and a Llama2-7B LLM for linguistic outputs. Historically, only the projector and LLM weights are adapted during IQA model alignment, which biases the system toward high-level response templates and washes out sensitivity to low-level differences.

Fine-tuning methodology is principal: the study compares selective unfreezing of the vision encoder, projector, or LLM6—with the critical finding that restoration of low-level perception is only achieved when the vision encoder itself is tuned. Specifically, fine-tuning the vision encoder, alone or in combination, raises distortion classification accuracy from near-chance (14.92%) to over 91% (vision encoder + partial LLM), as opposed to projector or LLM tuning which plateaus at 72–74% (Li et al., 10 Dec 2025).

Fine-tuning Regime	Test Accuracy (%)
Baseline (all frozen)	14.92
Projector only	72.12
LLM only	74.24
Vision Encoder only	83.43
Vision Encoder + Projector	88.80
Vision Encoder + partial LLM	91.45

This stratified approach unmistakably pinpoints the vision encoder as the dominant locus for recovery of low-level distortion representation.

4. Mathematical Formulations and Evaluation Protocols

Quantitative assessment relies on aligning visual embeddings ($V = f_\text{vis}(I)$ for image $I$) with semantic label embeddings ($T = f_\text{text}$(label)). Alignment is measured both by Euclidean distance $D(V,T) = \|V - T\|_2$ and by cosine similarity shifts pre- and post-fine-tuning:

$$\text{Similarity} = \frac{\cos(V, T) - \cos(V^{\text{init}}, T)}{|\cos(V^{\text{init}}, T)|}$$

The model is supervised using a smoothed cross-entropy loss over the eight-class label space:

$$L(\theta) = (1 - \epsilon) \left[ -\log p_\theta(y|x) \right] + \epsilon \frac{1}{C} \sum_{c=1}^C \left[ -\log p_\theta(c|x) \right]$$

where $\epsilon$ is the label smoothing factor and $C=8$. Optionally, a contrastive alignment loss can be added, but strong classification loss alone suffices for high discrimination.

Confusion matrix analysis reveals that models tuned without vision encoder adaptation collapse predictions to a single class (e.g., “blur”), while fine-tuning restores balanced sensitivity across all distortion types (Li et al., 10 Dec 2025).

5. Broader Theoretical Context: Distortion–Perception and Rate–Distortion–Perception Tradeoffs

Low-level distortion perception directly interfaces with fundamental tradeoff theories in signal processing and image restoration:

• Perception–Distortion: There exists a monotonic, convex frontier $P(D)$ mapping minimal achievable divergence in perceptual quality (as measured by, e.g., total variation, Wasserstein-2, LPIPS) as a function of allowed distortion $D$. Minimizing distortion (e.g., MSE) inevitably causes perceptual outputs to stray from natural image statistics, while optimizing for perfect perception requires an increase in distortion (Blau et al., 2017, Freirich et al., 2021).
• Rate–Distortion–Perception: The triple tradeoff $R(D, \pi)$ establishes that for a source $X$, lowest rate $R$ at fixed distortion and perception bounds is fundamentally limited. Enforcing high perceptual fidelity lifts the rate–distortion curve, often by a 3 dB (2×) penalty for exact source distribution matching (Blau et al., 2019, Salehkalaibar et al., 22 Jan 2024).
• Classification–Distortion–Perception: Adding a recognizability axis yields a three-way Pareto analysis. No method can simultaneously minimize distortion, match natural statistics, and optimize for downstream classification error (Liu et al., 2019).

In practice, these tradeoff curves provide operational guidance for model training (e.g., tuning $\lambda$ in mixed distortion–perceptual losses) and for evaluating low-level tasks with respect to both human perception and semantic usability.

6. Extensions to Audio and Data Augmentation

Low-level distortion perception also encompasses imperceptible transformations in the auditory domain. For example, phase-intercept distortion—frequency-independent phase shifts in audio signals—are mathematically significant but psychoacoustically undetectable, as confirmed by forced-choice listening experiments with mean accuracy indistinguishable from chance (Krishnan et al., 17 Jun 2025). This property enables phase-intercept transformations as data augmentation tools: models trained with these augmentations retain spectral content while improving generalization with no perceptual penalties, as evidenced by small but consistent gains in audio classification and blind source separation metrics.

7. Implications and Future Directions

The systematic construction of low-level distortion perception tasks reveals persistent limitations of high-capacity, template-driven models and elucidates the need for explicit architectural and training interventions—most notably, the fine-tuning of vision encoders in MLLMs. These findings argue for future IQA and vision-language pipelines to incorporate dedicated visual alignment losses operating directly on the feature representations most sensitive to primitive degradations (Li et al., 10 Dec 2025).

From a broader methodological standpoint, the rigorous design of such diagnostic tasks, coupled with careful analysis in the signal–perception–semantics plane, enables principled benchmarking and targeted improvement of both generative and discriminative systems in low-level vision and audio. The extension to more diverse, ecologically valid distortions, perceptual divergences beyond shallow statistics, and joint optimization for utility in downstream tasks (e.g., recognition, retrieval) defines a central research trajectory for the next generation of perceptually aligned signal processing models.