Image-Only Multi-Turn Reasoning (IOMR)

• Image-Only Multi-Turn Reasoning is a paradigm where models perform complex reasoning by processing sequences of images without relying on textual intermediates.
• It leverages methods such as symbolic diagram chains, tool-driven reinforcement learning, and multi-image sequence processing to enhance spatial and diagrammatic reasoning.
• Empirical benchmarks indicate that IOMR achieves up to 100% improvement over text-based approaches, while facing challenges like error propagation and scalability.

Image-Only Multi-Turn Reasoning (IOMR) is a paradigm in which models process and manipulate sequences of images over multiple interaction steps to achieve complex reasoning, without relying on intermediate or supporting text. IOMR subsumes a family of techniques and benchmarks designed to assess or facilitate visual agents whose reasoning chains are carried purely, or almost purely, in the visual modality. This category includes symbolic diagram-based “Chain-of-Images” systems, tool-driven visual search procedures, multi-image or sequential image-to-image benchmarks, and graphical rationale renderers. The field stands at the intersection of vision-language modeling, multimodal reinforcement learning, and human-like diagrammatic reasoning.

1. Foundational Concepts and Definitions

IOMR defines a reasoning protocol where all intermediate computational steps are represented via images—either generated, selected, or manipulated—rather than via text strings or spoken language. The reasoning chain consists of a series of images $\{I_1, I_2, ..., I_T\}$, each encoding intermediate state, hypothesis, or subresult, with the final answer derived from the last image or from an aggregation over the sequence. The key requirements are:

• No reliance on intermediate textual rationales: images themselves must carry the reasoning states or cues.
• Multi-turn protocol: at each step, the model either generates, selects, or transforms an image, often conditioned on the history of all previous visual states.
• Closed-loop or stepwise feedback: outputs from one step inform the next, closely matching the “draw–perceive” cycles used by human problem solvers.

IOMR fundamentally generalizes Chain-of-Thought (CoT) to the visual domain, replacing step-by-step textual rationalization with visual “scratch-pad” processes or tool interactions (Meng et al., 2023, Lyu et al., 7 Dec 2025, Bian et al., 8 Jun 2026, Zhou et al., 4 Nov 2025).

2. Algorithms, Architectures, and Pipelines

Several system-level recipes have been deployed for IOMR, each with unique architectural or procedural features:

A. Symbolic Visual Reasoning Chains (e.g., Chain of Images—CoI)

• The model emits vector-graphic (SVG) commands conditioned on the reasoning context.
• At each step, collected SVG tokens are rasterized to pixel images, encoded via a vision backbone (e.g., CLIP-ViT), and fed back as embeddings to guide further generation.
• The process repeats iteratively until an <Answer> token is triggered (Meng et al., 2023).
• Training combines token-level cross-entropy for SVG/image generation with answer prediction.

B. Tool-Based Visual Search RL Agents (e.g., Mini-o3, DRIM)

• The agent operates over high-resolution scenes, issuing tool calls (zoom, crop, etc.) to progressively focus on targets.
• Reinforcement learning is employed, with reward for correct answers and often penalties or masking schemes for low-efficiency trajectories or failed explorations.
• Pipelines use supervised pre-training on trajectory demonstrations followed by policy optimization (e.g., Group Relative Policy Optimization, GRPO), often with special strategies such as “over-turn masking” to allow policies to scale to deep turn counts at inference (Lai et al., 9 Sep 2025, Yang et al., 19 Dec 2025).

C. Multi-Image Chain-of-Thought Systems (e.g., MiCo, ImageChain)

• Systems that concatenate embedded representations of multiple images (e.g., sequence of crops, video frames, or augmented views) as input, with reasoning performed over such chains in either an autoregressive or policy-driven manner.
• In MiCo, reasoning over image triplets is optimized through PPO-style RL without human-labeled QA, using self-supervision and prompt-engineered chain-of-thought emission (Chen et al., 27 Jun 2025).
• In ImageChain, interleaving scene images and text in a multi-turn conversational structure enables the model to perform contextually aware reasoning (e.g., next-scene description) (Villegas et al., 26 Feb 2025).

D. Visual-Only Rationale Rendering (e.g., Optical Reasoning)

• In Optical Reasoning, all intermediate rationales are rendered as composite images (either typographic or graphical panels); MLLMs then process these visual rationales as the sole carrier of stepwise logic (Bian et al., 8 Jun 2026).

3. Benchmarks and Evaluation Protocols

IOMR systems are evaluated on diverse, highly-structured benchmarks that isolate the visual reasoning capability across domains, task types, and interaction patterns. Key benchmarks include:

A. CoIEval (Meng et al., 2023)

• 15 domains spanning geometry, chess, logical deduction, spatial planning, etc.; quantitative results show CoI surpasses text-based CoT by factors of 1.5–2.3× in accuracy on domains such as intersect geometry and chess tactics.

B. VisChainBench (Lyu et al., 7 Dec 2025)

• 1,457 tasks, ~20,000 images, with three principal formats:
  • ITMR: Image-Text Multi-Turn
  • ICIR: In-Context Image-Only Reasoning
  • IOMR: Image-Only Multi-Turn Reasoning
• Each task requires selecting the correct next-step image in a sequential chain, minimizing language cues; overall correct-answer rates (CA) highlight a steep scaling curve with model size.

C. MIRA (Zhou et al., 4 Nov 2025)

• 546 problems, each requiring intermediate visual “scratch-pad” steps.
• Evaluation distinguishes direct (no rationale), text-CoT, and visual-CoT (image-supported) reasoning; supplying annotated intermediate images boosts accuracy by up to 33.7% relative to text-only CoT.

D. Optical Reasoning Benchmarks (Bian et al., 8 Jun 2026)

• ScienceQA, AquaRat, GSM8K, Zebra-CoT, etc.: studies show visual rationales achieve equivalent or better accuracy with ~29% fewer reasoning tokens, and up to 1.96× token efficiency compared to text rationales.

E. Visual Search and Multi-Image Reasoning Datasets (Lai et al., 9 Sep 2025, Chen et al., 27 Jun 2025)

• VisualProbe, V* Bench, HR-Bench: test agents’ ability to search, categorize, or compare across multi-image or high-resolution scenes over multiple tool-invoking turns.

Benchmark	Format	Task Count	Unique Aspects
CoIEval	Mixed	15 domains	SVG/image chain-of-thought
VisChainBench	Multi-image	1,457	Sequenced, minimal language
MIRA	Visual-CoT	546	Annotated visual rationales
Optical Reasoning	Visual-only	~1,000s	Panel/composite rationale imgs
VisualProbe	Visual search	~4,500	Exploratory, tool-driven RL

4. Empirical Findings, Model Analyses, and Insights

• Performance Gains: Image-only reasoning frameworks (CoI, Optical Reasoning, visual-CoT in MIRA) consistently outperform text-only baselines in accuracy, particularly as task complexity rises or as domains become more spatial/structural. Relative gains range from 20–100% over text CoT, particularly on geometric, spatial, or visual search tasks (Meng et al., 2023, Zhou et al., 4 Nov 2025, Lyu et al., 7 Dec 2025, Bian et al., 8 Jun 2026).
• Scaling and Error Propagation: IOMR performance exhibits strong scaling with model capacity. However, error propagation is acute—early mistakes in a sequence often preclude recovery, drastically lowering task-completion rates compared to per-turn answer rates (Lyu et al., 7 Dec 2025, Lai et al., 9 Sep 2025).
• Critical Role of Visual Intermediates: The provision (or model-generation) of precise intermediate sketches, diagrams, or sub-image crops often determines success. Qualitative examples reveal that even state-of-the-art LLMs hallucinate or over-abstract when deprived of visual intermediates (Zhou et al., 4 Nov 2025, Meng et al., 2023).
• Token Efficiency: Optical reasoning achieves substantial reductions in reasoning token budgets, due to high information density and spatial layout, demonstrating intrinsic efficiency advantages over textual chains (Bian et al., 8 Jun 2026).
• Tool-Driven Exploration: Systems incorporating tool actions—zoom, crop, manipulate—require specialized reward schemes. Redundancy penalties, over-turn masking, and reward shaping are effective in driving deep, self-corrective multi-turn visual reasoning (Yang et al., 19 Dec 2025, Lai et al., 9 Sep 2025).

5. Limitations, Open Challenges, and Future Directions

• Modality Constraints: Current image-only chains are most effective in diagrammatic, spatial, or structural domains where information can be encoded with high fidelity as SVG or panel drawings. Photo-realistic or semantically-loaded scenes may require richer symbol vocabularies or integration of generative diffusion models (Meng et al., 2023, Bian et al., 8 Jun 2026).
• Bottlenecks in Visual Encoding: Sequential conditioning using high-dimensional visual embeddings can bottleneck as the chain length increases or when pixel budgets are tight. Hierarchical, object-centric, or memory-augmented latent architectures are proposed as possible solutions (Meng et al., 2023, Lyu et al., 7 Dec 2025).
• Visual Hallucination and Misparsing: Models are prone to graphical hallucination (incorrect visual evidence, misleading diagrams) and may unevenly parse fine-grained layouts depending on architecture and rendering, underscoring the need for model-aware renderers and visual correctness feedback (Bian et al., 8 Jun 2026, Yang et al., 19 Dec 2025).
• Learning to Generate vs. Use Visual Chains: IOMR benchmarks such as MIRA and VisChainBench identify a gap between models' abilities to consume supplied visual chains versus autonomously generate effective intermediates. End-to-end agentic routines that “think by drawing” (via internal sketching tools or APIs) remain a target for future pretraining and RL scaffolding (Zhou et al., 4 Nov 2025, Lyu et al., 7 Dec 2025).
• Benchmarking and Dataset Expansion: Further progress hinges on developing benchmarks spanning new domains (e.g., medical procedures, code traces, architectural plans) and on open-sourcing data generation pipelines that minimize language priors and facilitate reproducibility (Lyu et al., 7 Dec 2025).
• System Integration: Advanced systems will likely combine symbolic vector formats, strong visual encoders, and flexible tool APIs into unified closed-loop “draw–perceive–act” architectures (Meng et al., 2023, Yang et al., 19 Dec 2025, Lai et al., 9 Sep 2025).

6. Design Principles and Methodological Guidance

Emergent methodological principles for IOMR system builders include:

• Employ symbolic vector graphics when precision is critical: SVG-based reasoning offers lossless, model-controllable visual “scratch-pads” (Meng et al., 2023).
• Fuse vision and language representations via cross-attention or gating: Effective conditioning on visual state histories is essential for multi-turn coherence.
• Structure tasks and prompts to coordinate multi-step visual dialogue: Scaffolding, whether by explicit turn-based protocols or example-driven tool chains, improves learning depth and reliability (Zhou et al., 4 Nov 2025, Yang et al., 19 Dec 2025).
• Leverage both supervised and RL-based curriculum: Cold-start SFT on multi-turn demonstration trajectories followed by redundancy-penalized RL matches or exceeds the robustness of closed-source baselines (Yang et al., 19 Dec 2025, Lai et al., 9 Sep 2025).
• Explicitly account for scalability in turn depth vs inference budget: Over-turn masking and similar schemes decouple training horizon from test-time depth, yielding systems that can naturally scale to long reasoning chains without premature answer emission (Lai et al., 9 Sep 2025).

In summary, IOMR is a rapidly advancing direction unifying chain-of-thought, reinforcement learning, visual tool invocation, and diagrammatic reasoning—shifting multimodal systems from “text that sees images” to agents that genuinely “think in pictures.” The field is defined by its drive toward human-like, image-centric intelligence, guided by rigorous benchmarks and a deepening analytical understanding of the bottlenecks and affordances of spatial reasoning with images (Meng et al., 2023, Yang et al., 19 Dec 2025, Lyu et al., 7 Dec 2025, Bian et al., 8 Jun 2026, Zhou et al., 4 Nov 2025, Lai et al., 9 Sep 2025, Chen et al., 27 Jun 2025, Villegas et al., 26 Feb 2025).