CAMVR: Context-Aware Multi-Turn Reasoning

• CAMVR is a multimodal reasoning framework that processes visual and language inputs over multiple turns, maintaining context with explicit memory and iterative attention.
• It employs dynamic memory and planning mechanisms to minimize visual hallucinations and context-loss, ensuring responses remain grounded in accumulated evidence.
• Benchmark evaluations, including CLEVR-Dialog and video segmentation tasks, demonstrate CAMVR’s high effectiveness and improved accuracy in complex visual reasoning scenarios.

Context-Aware Multi-Turn Visual Reasoning (CAMVR) denotes a class of multimodal reasoning problems and systems in which an agent must process visual inputs and linguistic inputs over multiple turns, maintain cross-turn coherence, track entities and prior observations, and generate actions or responses that remain grounded in accumulated context rather than only the current turn. In contemporary formulations, CAMVR is characterized by explicit memory state, iterative perception or retrieval, and recurrent reasoning over visual and textual evidence, with stated goals that include minimizing visual hallucinations and context-loss across long interaction horizons (Han et al., 21 Aug 2025, Shen et al., 6 Sep 2025).

1. Formal problem setting

A canonical CAMVR formulation models turn $t$ with dialogue history $D_t=\{u_1,a_1,u_2,a_2,\dots,u_t\}$, visual observation $V_t$, memory $M_{t-1}$, current instruction $I_t=u_t$, and response or action $A_t=a_t$. In the CoLVLM formulation, the task is to learn a policy

$(A_t,\,M_t)=\text{CoLVLM}(V_t,\;I_t,\;M_{t-1})$

for $t=1\ldots T$, such that the response sequence faithfully answers each query or executes each instruction, the memory states summarize past visual and linguistic context, and visual hallucinations and context-loss are minimized (Han et al., 21 Aug 2025).

Several later systems cast the same problem as sequential decision-making. DRIM defines CAMVR as a Markov Decision Process in which the state

$s_t=\{(Q,I_0),(X_1,O_1),\dots,(X_t,O_t)\}$

contains the original question, the original image, and the history of multimodal Chain-of-Thought outputs and returned observations; actions are either tool calls or final answers, and the policy $\pi_\theta(a\mid s)$ maximizes expected trajectory reward (Yang et al., 19 Dec 2025). VideoSEG-O3 uses an analogous MDP in video object segmentation, with state $D_t=\{u_1,a_1,u_2,a_2,\dots,u_t\}$0, exploration actions of the form <select> [t_start, t_end] key_frame_index, and termination actions of the form <answer> [SEG] (Dai et al., 5 Jun 2026).

CAMVR also includes multi-image, text-light, and image-only regimes. VisChainBench formalizes a session as $D_t=\{u_1,a_1,u_2,a_2,\dots,u_t\}$1, where $D_t=\{u_1,a_1,u_2,a_2,\dots,u_t\}$2 is a set of context images, $D_t=\{u_1,a_1,u_2,a_2,\dots,u_t\}$3 is either a short text prompt or an image, and the model predicts $D_t=\{u_1,a_1,u_2,a_2,\dots,u_t\}$4 from a finite answer set (Lyu et al., 7 Dec 2025). Taken together, these formulations suggest that CAMVR is not restricted to visual dialog; it also encompasses tool-mediated inspection, iterative video exploration, and sequential image-to-image inference.

2. Core architectural mechanisms

A persistent theme in CAMVR is explicit state retention. An early precursor is the Context-aware Attention and Memory extension to the MAC network for visual dialog. There, Multi-Turn Memory carries $D_t=\{u_1,a_1,u_2,a_2,\dots,u_t\}$5 into turn $D_t=\{u_1,a_1,u_2,a_2,\dots,u_t\}$6, while Context-Aware Attention attends over control states from previous dialog turns and prior reasoning steps before fusing the recalled context into the current control vector. The resulting system was designed specifically to reason over dialog history and coreference rather than only the current question (Shah et al., 2020).

Later LVLM systems generalize this design into modular memory architectures. CAMVR with a Visual-Textual Context Memory Unit introduces a dynamic read-write memory $D_t=\{u_1,a_1,u_2,a_2,\dots,u_t\}$7 that stores critical visual features, textual semantic representations, and cross-modal correspondences. Its read operation lets current text embeddings query memory through scaled dot-product attention, and its Adaptive Visual Focus Guidance uses retrieved context to modulate spatial attention over the current visual feature map (Shen et al., 6 Sep 2025). CoLVLM Agent uses a hierarchical state $D_t=\{u_1,a_1,u_2,a_2,\dots,u_t\}$8 inside a four-module cycle of Memory, Perception, Planning, and Execution; the Perception module applies cross-modal attention over visual and language tokens, and the Execution module contains a self-correction subloop that can re-invoke planning if a response diverges from the plan or user feedback (Han et al., 21 Aug 2025).

Another recurring mechanism is context compression rather than raw history accumulation. VisHarness maintains the original image representation, the referring text, a compact archive of past visual feedback as textual summaries, and only the most recent visual feedback in full tokenized form. Its dynamic visual memory archiving replaces older visual outputs with placeholders $D_t=\{u_1,a_1,u_2,a_2,\dots,u_t\}$9, enabling on-policy reinforcement learning over longer multi-turn interactions with heterogeneous experts (Fan et al., 28 May 2026). ILVR compresses context differently: it interleaves text with a latent visual state $V_t$0 and uses a Momentum Teacher Model to distill only the top-$V_t$1 most relevant patch features as sparse supervision targets. The stated effect is a dynamic “text $V_t$2 latent $V_t$3 text $V_t$4 latent …” loop that preserves evolving context while avoiding repeated re-encoding of pixel-dense images (Dong et al., 5 Dec 2025).

A related but distinct line replaces single-agent memory with shared evidence context. In M$V_t$5-ACE, the anchor agent iteratively updates a shared context $V_t$6, while assistant agents provide frozen reference evidence lists. The Summary Tool partitions evidence into consistent, complementary, and conflicting components, and the Refine Tool uses conflict ratio and consensus to decide whether another round is required (Xie et al., 9 Mar 2026). This suggests that CAMVR architectures increasingly treat “context” as structured evidence rather than only concatenated tokens.

3. Iterative reasoning procedures and optimization

Many CAMVR systems implement explicit turn-level reasoning loops. Visual-O1 organizes each turn into a reasoning step and a reflection step, updating a prompt buffer $V_t$7 either by concatenation for “instantial experience” or by replacement for “empirical experience,” and only then performing response synthesis (Ni et al., 2024). Video-MTR uses a policy $V_t$8 that emits a natural-language rationale and an executable action at each turn; if the action is Retrieve(t_start, t_end), the environment returns at most 8 new frames, and if the action is Answer(opt), the episode terminates. Internally, the MLLM attends over previous rationales and newly observed frames to refine its latent question representation (Xie et al., 28 Aug 2025).

Reward shaping is central in systems that cast CAMVR as reinforcement learning. Video-MTR combines a trajectory-level reward for answer correctness with a turn-level reward for frame-query relevance: $V_t$9 and

$M_{t-1}$0

Only one turn may receive $M_{t-1}$1, and frame-relevance reward is gated by final correctness. The policy is then trained with PPO using token-level advantage estimates (Xie et al., 28 Aug 2025).

DRIM uses a three-stage pipeline of data construction, cold-start SFT, and redundancy-penalized RL. Its penalty term targets failed trajectories that exhibit oscillatory micro-adjustments, defined through high-IoU overlap between repeated zoom boxes, and its optimization uses Group Relative Policy Optimization while masking observation tokens out of the language-model loss (Yang et al., 19 Dec 2025). VideoSEG-O3 adds a different form of dense grounding: SEG-aware logit calibration inserts mask likelihood into the [SEG] token logit, so that the policy does not optimize only text probability but also pixel-level segmentation feedback. The same system enforces a decoupled thinking trace ordered as temporal understanding, spatial detail capturing, and expression parsing (Dai et al., 5 Jun 2026).

An alternative line treats visual generation itself as the reasoning substrate. In “Thinking in Frames,” each turn produces a video $M_{t-1}$2, with intermediate frames serving as explicit reasoning steps and the final frame carried into the next turn as updated world state. In Maze Navigation, increasing generated video length beyond the training length $M_{t-1}$3 improved Exact-Match in temporal OOD 6 × 6 mazes with path-length $M_{t-1}$4 from $M_{t-1}$5 at $M_{t-1}$6 to $M_{t-1}$7 at $M_{t-1}$8, a result described as a visual test-time scaling law (Li et al., 28 Jan 2026).

4. Benchmarks and evaluation protocols

CAMVR evaluation spans synthetic dialog, ambiguous instruction following, long-video understanding, multi-image workflows, and referring video object segmentation. MMDR-Bench contains 300 expert-curated scenarios averaging 5–7 turns and scored by human experts on six dimensions: Visual Entity Tracking, Dialogue Consistency, Reasoning Depth, Instruction Adherence, Error Suppression, and Response Fluency. Scores are discrete $M_{t-1}$9–$I_t=u_t$0 values aggregated per dimension and then averaged (Han et al., 21 Aug 2025).

MMCR introduced both a large instruction-tuning corpus and a diagnostic benchmark. MMCR-310k contains 310K contextual dialogues, including 210K single-image dialogues with 4 turns and 100K multi-image dialogues with 8 turns. MMCR-Bench comprises 600 dialogues across eight domains and 40 sub-topics, and GPT-4o serves as judge along five diagnostic dimensions: Precision & Conciseness, Consistency of Contextual References, Logical Contextual Relationships, Clarity of Dialogue Theme, and Degree of Redundancy. The overall score is $I_t=u_t$1 (Yan et al., 24 Mar 2025).

VisChainBench targets multi-image, minimal-language reasoning at larger scale. It contains 1,457 tasks and 20,431 images across Image–Text Multi-Turn Reasoning, In-Context Image-Only Reasoning, and Image-Only Multi-Turn Reasoning, and evaluates turn-wise accuracy, task-completion rate, and context-retention score (Lyu et al., 7 Dec 2025). Visual-O1 uses ambiguous subsets of RefCOCO+ and VizWiz, with ambiguity categories including ellipsis, colloquialism, subjectivity, relativity, and other; it reports gIoU and cIoU for referring image segmentation and accuracy and BLEU-1 for VQA (Ni et al., 2024).

Video-oriented CAMVR uses distinct suites. Video-MTR evaluates on VideoMME, MLVU, and EgoSchema (Xie et al., 28 Aug 2025). “Thinking in Frames” evaluates Maze Navigation with Exact Match and Progress Rate and Tangram Puzzle with Strict Goal Completion, Progress Completion, and Mask-IoU (Li et al., 28 Jan 2026). VideoSEG-O3 evaluates Referring VOS and Reasoning VOS on MeViS, ReVOS, Long-RVOS, and Ref-SAV, and reports JF% overall together with auxiliary analyses such as keyframe mIoU and empty-frame selection ratio (Dai et al., 5 Jun 2026).

5. Empirical results and ablation findings

The reported results are heterogeneous because the evaluated tasks differ substantially, but several systems isolate the effect of memory, planning, or multi-turn exploration.

System	Setting	Headline result
CAM with CAA + MTM (Shah et al., 2020)	CLEVR-Dialog	98.25% accuracy; vanilla MAC 65.9%
CoLVLM Agent (Han et al., 21 Aug 2025)	MMDR-Bench	Avg. 4.03; GPT-4o 3.92; Gemini 1.5 Pro 3.85
Video-MTR (Xie et al., 28 Aug 2025)	Long-video QA	51.0 vMME-Long, 59.0 vMME-Overall, 48.4 MLVU-Test, 62.4 EgoSchema
CAMVR with VCMU + AVFG (Shen et al., 6 Sep 2025)	VisDial / A-OKVQA / MTIF	78.9 CIDEr, 64.3% Accuracy, 56.5% IFSR, 0.78 CCS
DRIM (Yang et al., 19 Dec 2025)	VisualProbe / V* / HR-Bench	53.4%, 92.2%, 83.1%
VideoSEG-O3 (Dai et al., 5 Jun 2026)	ReVOS / staged training	67.7 JF% overall on ReVOS; 60.29 avg J + F after RL

Ablations make the architectural claims more specific. On CLEVR-Dialog, removing MTM from the full CAM system drops accuracy from 98.25% to 89.43%, while removing CAA drops performance back to vanilla MAC at 65.9%; naive concat-history reaches 98.08% but incurs $I_t=u_t$2 overhead rather than $I_t=u_t$3 (Shah et al., 2020). On MMDR-Bench, CoLVLM’s drop from turns 1–3 to 7+ is 0.10 points, compared with 0.20 for GPT-4o, and removing the Memory module drops overall score by 0.59 while removing the Planning engine drops it by 0.61. The same paper reports a 66 % reduction in both Context-Loss and Visual Hallucination rates compared to GPT-4o (Han et al., 21 Aug 2025).

Video-MTR reports a +6.3% gain on vMME-Long, and its ablations show 59.0 on vMME-Overall and 48.4 on MLVU-Test for the full model, versus 57.8 and 44.4 without Bi-Level Reward and 57.2 and 44.3 for a single-turn variant. On MLVU, detail tasks gain +7.5% on single-detail and +8.1% on multi-detail. The paper explicitly notes that goal-gated shaping prevents reward hacking because total reward increases while accuracy decreases in the ungated variant (Xie et al., 28 Aug 2025).

Visual-O1 reports substantial gains on ambiguous tasks without changing the frozen multimodal model in the instantial setting. On ambiguous VQA, GPT-4o moves from 14.10 accuracy and 0.5960 BLEU-1 to 22.20 and 0.6475 with Visual-O1, while LLaVA moves from 8.58 and 0.4760 to 21.35 and 0.5123. On ambiguous RIS, SoM improves from 0.0752 gIoU and 0.1158 cIoU to 0.1278 and 0.1722 (Ni et al., 2024).

VideoSEG-O3 provides a particularly explicit training-stage analysis. VideoSEG-SFT yields avg J + F = 54.96; adding CoT Cold-Start raises this to 57.07 with avg rounds = 2.79; adding RL reaches 60.29 with avg rounds = 2.23. On ReVOS, VideoSEG-O3-4B reaches 67.7 JF% overall versus 61.3 for Veason-R1-7B, and GRPO alone degrades from 59.43 to 57.92 before SEG-aware calibration raises performance to 60.51 and auxiliary spatial+keyframe loss to 61.95 (Dai et al., 5 Jun 2026).

6. Misconceptions, limitations, and open directions

A common misconception is that CAMVR is equivalent to concatenating longer histories. The evidence is narrower. Early dialog results showed that concat-history can be competitive, but the same study reports different computational overhead and strong gains from structured Context-Aware Attention and Multi-Turn Memory (Shah et al., 2020). Later work similarly shows that individual modules are indispensable: removing memory or planning in CoLVLM causes large score drops, and removing reasoning or reflection in Visual-O1 lowers general VQA accuracy from 57.38 to 54.25, while removing response synthesis lowers it to 47.59 (Han et al., 21 Aug 2025, Ni et al., 2024).

Another misconception is that more turns or more reward necessarily yield better reasoning. Several papers state the opposite. Video-MTR introduces goal-gated shaping specifically to prevent reward hacking (Xie et al., 28 Aug 2025). M$I_t=u_t$4-ACE reports that the Refine Tool is essential to prevent oscillation and that without it iterative turns can degrade performance (Xie et al., 9 Mar 2026). VideoSEG-O3 shows that GRPO alone degrades performance unless mask quality is coupled back into the token-level objective through SEG-aware logit calibration (Dai et al., 5 Jun 2026).

The current limitations are also explicit. ILVR requires access to high-resolution auxiliary images at training time, and its latent capacity $I_t=u_t$5 trades off detail versus sequence length (Dong et al., 5 Dec 2025). VisHarness is capped by the precision and coverage of the chosen experts, and even with archiving, extremely long horizons $I_t=u_t$6 turns may still stress current MLLMs (Fan et al., 28 May 2026). VisChainBench is limited to Daily, Engineering, Natural Science, and IT, still uses small amounts of text for output formatting, and assumes discrete next-step workflows rather than branching or open-ended planning (Lyu et al., 7 Dec 2025). MMCR reports a “Less is More” phenomenon in which MMCR-Bench scores keep rising with more CAMVR-style data while public benchmark performance peaks and then declines (Yan et al., 24 Mar 2025).

The open directions named across the literature are correspondingly concrete: adaptive $I_t=u_t$7 per turn and user feedback loops for latent reasoning (Dong et al., 5 Dec 2025), streaming video or on-the-fly imagery (Dong et al., 5 Dec 2025), additional experts such as keypoint detectors and depth estimators (Fan et al., 28 May 2026), fully language-less prompting paradigms and domain extensions such as medical and mathematics (Lyu et al., 7 Dec 2025), and perception-centric collaboration that replaces final-answer aggregation with iterative evidence correction (Xie et al., 9 Mar 2026). A plausible implication is that CAMVR is converging on a broader design principle: robust multi-turn visual reasoning depends less on a single monolithic decoder than on explicit context management, selective evidence acquisition, and training objectives that align intermediate reasoning with final grounded performance.