Perception-R1 Framework

• Perception-R1 is a comprehensive framework that models perception by mapping external stimuli to internal representations using classical Fisher information geometry and modern reinforcement learning techniques.
• It integrates chain-of-thought processes with vision-language models to optimize task-specific policies and enhance the alignment between perceptual grounding and reasoning.
• Empirical benchmarks demonstrate significant improvements in perceptual accuracy and reasoning fidelity across multimodal systems, highlighting its practical impact on complex scene understanding.

The Perception-R1 framework represents a set of foundational and modern theoretical and algorithmic advances for modeling perception as a process of mapping external or physical variables to internal representations, with special attention to their application in vision, language, and multimodal machine learning systems. It encompasses both a classic psychophysical formalism for perceptual scales, derived from Fisher information geometry, and a family of contemporary reinforcement learning and chain-of-thought post-training pipelines for LLMs and Multimodal LLMs (MLLMs), all tied to the central goal of aligning perception with explicit reasoning and task-optimal policies. The Perception-R1 paradigm is instantiated in neuro-cognitive theory (Vacher et al., 2023), explicit policy learning in vision-LLMs (Yu et al., 10 Apr 2025, Xiao et al., 8 Jun 2025, Ding et al., 29 Sep 2025, Ma et al., 17 Mar 2025), fine-grained visual reasoning (Yang et al., 13 Mar 2025), complex scene understanding (Park et al., 29 May 2025), and perception-cognition chains for video anomaly reasoning (Huang et al., 26 May 2025).

1. Classical Perceptual Scale: Fisher Information and Internal Representation

The foundational Perception-R1 theory formalizes perception as a deterministic–stochastic transform: a physical stimulus parameter $S$ is mapped by a transduction function (or perceptual scale) $\psi: S \rightarrow R$, yielding an internal, noise-free representation $R$, which is then corrupted by additive noise $N$ to produce a noisy internal measurement $M = \psi(S) + N$. Here, $N$ is modeled as zero-mean Gaussian noise with constant variance. Psychophysical tasks (typically difference scaling) probe how changes in $S$ are stretched or compressed in subjective space, yielding data that support the maximum-likelihood difference scaling (MLDS) recovery of $\psi$ up to affine transformation (Vacher et al., 2023).

For high-dimensional stimuli (e.g., images, textures), one posits a generative model $p(x|S)$ specifying the statistical structure of $x$ for a given $S$. The key theoretical object is the Fisher information $$I_S(s) = \mathbb{E}_{M|S=s}\left[(\partial/\partial s \ln p(M|s))^2\right]$$, and the perceptual scale derivative obeys the square-root law: $\psi'(s) \propto \sqrt{I_S(s)}$ Integration gives $\psi(s) = \int_{s_0}^s \sqrt{I_S(t)}dt$, framing the internal geometry of perception as induced by the Fisher information delivered by the generative model. Empirical MLDS experiments confirm near-exact accordance with these theoretical predictions for classical (spatial frequency, orientation) and nonclassical (texture interpolation, deep-feature space) domains (Vacher et al., 2023).

2. Perception-R1 for Visual Policy Learning via Reinforcement Learning

Modern Perception-R1 approaches in vision-language modeling treat perception as a policy learning problem over MLLMs, optimized using group-based policy-gradient methods such as Generalized Relative Policy Optimization (GRPO) (Yu et al., 10 Apr 2025). Here, the MLLM is treated as a sequential decision process (MDP):

• State $s_t$ is tuple $(I, q, o_{<t})$ capturing image, language prompt, and partial output
• Action $a_t$ is the next vocabulary token
• Reward $r_t$ is 0 for all $t<T$, with terminal reward $R(o)$ at sequence end.

The reward signal combines format correctness (for output structure), discriminative spatial/semantic accuracy (e.g., IoU for boxes, edit distance for OCR), and for complex tasks incorporates recall, class, count, and missingness penalties. The policy $\pi_\theta$ is optimized via GRPO, where $G$ rollouts per prompt allow group normalization of advantages and clipped surrogate losses stabilize learning.

Perceptual complexity—empirically linked to number and granularity of output labels—determines RL's effect: RL offers major gains for multi-object settings (PixMo counting, COCO detection), more modest benefit for single-object tasks (RefCOCO grounding, PageOCR), and can degrade performance if chain-of-thought is forced on "pure" perception outputs (Yu et al., 10 Apr 2025).

3. Multimodal Reasoning: Structured Perceptual Grounding and Reward Augmentation

Recent Perception-R1 pipelines explicitly decouple the perception (grounding) and reasoning (CoT) stages in MLLMs. In VTPerception-R1 (Ding et al., 29 Sep 2025), models are first trained with explicit perception-augmented supervised fine-tuning on structured triple fields—<description> (key perceptual facts), > (chain-of-thought), <answer>—then further refined by perception-aware RL with composite rewards. These rewards include not just answer accuracy and output format, but explicit key-info recall in both visual and textual terms, as well as consistency metrics across stages. Empirical ablation shows that these rewards provide robust improvements in accuracy and reasoning fidelity, with the consistency reward having the most substantial effect on difficult tasks.

In Perception-R1 with RLVR (Xiao et al., 8 Jun 2025), an explicit visual perception reward is defined by comparing generated outputs to atomic visual annotations extracted from reference CoT, judged for coverage by an external LLM. The final reward is a weighted sum: $$r(y, a, V) = \alpha r_f(y) + \beta r_a(y, a) + \gamma r_v(y, V) + r_p(y)$$ Ablation demonstrates that both the perception reward $r_v$ and an $n$-gram repetition penalty $r_p$ are essential for optimal reasoning and perception accuracy.

4. Chain-of-Thought for Perception–Reasoning Integration

Perception-R1 frameworks extend classic chain-of-thought to vision by formalizing multi-step perception-to-cognition templates (Huang et al., 26 May 2025, Ma et al., 17 Mar 2025, Park et al., 29 May 2025). In Vad-R1 for video anomaly reasoning, the P2C-CoT template structures the output as two perception (global, local) and two cognition (shallow, deep) stages, culminating in a tagged answer covering which/what/when/where/why/how dimensions. Training combines high-quality manually or model-curated P2C-CoT supervision with RL via AVA-GRPO, which introduces an anomaly verification reward to self-check prediction faithfulness.

Similarly, DeepPerception (Ma et al., 17 Mar 2025) combines knowledge-guided CoT rationales for fine-grained visual grounding with RL on continuous IoU-based spatial rewards. Studies confirm that the two-stage curriculum (supervised CoT pre-training followed by RL) confers major advantages in both in-domain and cross-domain performance, with RL focusing on refining spatial outputs and CoT scaffolding domain-relevant reasoning chains.

5. Broader Theoretical Models and Connections

Beyond RL and neural architectures, Perception-R1 as instantiated in spin-lattice models (Fanfarillo et al., 8 Jul 2025) addresses the emergence of macroscopic perceptual landscapes via local, contextualized internal evaluation. Here, evaluation states (spin variables $S_i \in \{+1,\,0,\,-1\}$) interact to yield collective perceptual stances, captured formally by a Landau–Ginzburg free energy $F[\phi]$. Neutral evaluation states (spin-1 or Blume–Capel models) lower perceptual thresholds and induce greater contextual sensitivity, providing a statistical–field-theoretic tool for modeling multistability, graded sensitivity, bistability and hysteresis in perceptual organization.

6. Benchmarks, Datasets, and Empirical Impact

Perception-R1 frameworks have catalyzed the release of new multimodal and perception-reasoning benchmarks:

• R1-Onevision-Bench: 942 image-based questions stratified by grade/domain, probing integrated perception and reasoning (Yang et al., 13 Mar 2025).
• KVG-Bench: 1336 manually curated fine-grained visual grounding samples, both seen and out-of-domain (Ma et al., 17 Mar 2025).
• MathVista, MathVerse, AI2D, C-MMBench: standardized generalization and perception-reasoning evaluation splits (Yu et al., 10 Apr 2025, Xiao et al., 8 Jun 2025, Ding et al., 29 Sep 2025).

State-of-the-art results have been reported for Perception-R1 methods, with substantial absolute gains (sometimes >15–25 AP or accuracy points) over strong SFT or non-RL baselines in high-perceptual-complexity tasks.

7. Limitations and Future Directions

Current Perception-R1 implementations face challenges including reward design brittleness, scaling costs of RL post-training, and dependence on reliable atomic annotations and judging LLMs for verifiable perception. Chain-of-thought stages can degrade dense perception outputs if not properly aligned, and synthetic or composite data may not capture all real-world scene complexities.

Future directions identified in the literature include:

• Developing learned (non–rule-based) reward models.
• Extending three-stage inspection and chain-of-thought paradigms to video, region-level captioning, and segmentation.
• Integrating dynamic external knowledge retrieval for unfamiliar domains.
• Tightening coupling between token-level visual semantics and perception-grounded reasoning.

Perception-R1 continues to inform not only the development of perceptual models in neuroscience and psychophysics but also drives progress in auditable, scalable multimodal intelligence (Vacher et al., 2023, Yu et al., 10 Apr 2025, Xiao et al., 8 Jun 2025, Park et al., 29 May 2025, Ma et al., 17 Mar 2025, Ding et al., 29 Sep 2025, Huang et al., 26 May 2025, Yang et al., 13 Mar 2025, Fanfarillo et al., 8 Jul 2025).