---
title: Image-Only Multi-Turn Reasoning (IOMR)
url: https://www.emergentmind.com/topics/image-only-multi-turn-reasoning-iomr
type: topic
---

# Image-Only Multi-Turn Reasoning (IOMR)

Image-Only Multi-Turn Reasoning (IOMR) is a paradigm in which models process and manipulate sequences of images over multiple interaction steps to achieve complex reasoning, without relying on intermediate or supporting text. IOMR subsumes a family of techniques and benchmarks designed to assess or facilitate visual agents whose reasoning chains are carried purely, or almost purely, in the visual modality. This category includes symbolic diagram-based “Chain-of-Images” systems, tool-driven visual search procedures, multi-image or sequential image-to-image benchmarks, and graphical rationale renderers. The field stands at the intersection of vision-language modeling, multimodal reinforcement learning, and human-like diagrammatic reasoning.

## 1. Foundational Concepts and Definitions

IOMR defines a reasoning protocol where all intermediate computational steps are represented via images—either generated, selected, or manipulated—rather than via text strings or spoken language. The reasoning chain consists of a series of images \(\{I_1, I_2, ..., I_T\}\), each encoding intermediate state, hypothesis, or subresult, with the final answer derived from the last image or from an aggregation over the sequence. The key requirements are:

- **No reliance on intermediate textual rationales**: images themselves must carry the reasoning states or cues.
- **Multi-turn protocol**: at each step, the model either generates, selects, or transforms an image, often conditioned on the history of all previous visual states.
- **Closed-loop or stepwise feedback**: outputs from one step inform the next, closely matching the “draw–perceive” cycles used by human problem solvers.

IOMR fundamentally generalizes Chain-of-Thought (CoT) to the visual domain, replacing step-by-step textual rationalization with visual “scratch-pad” processes or tool interactions [2311.09241][2512.06759][2606.09585][2511.02779].

## 2. Algorithms, Architectures, and Pipelines

Several system-level recipes have been deployed for IOMR, each with unique architectural or procedural features:

**A. Symbolic Visual Reasoning Chains (e.g., Chain of Images—CoI)**
- The model emits vector-graphic (SVG) commands conditioned on the reasoning context.
- At each step, collected SVG tokens are rasterized to pixel images, encoded via a vision backbone (e.g., CLIP-ViT), and fed back as embeddings to guide further generation.
- The process repeats iteratively until an <Answer> token is triggered [2311.09241].
- Training combines token-level cross-entropy for SVG/image generation with answer prediction.

**B. Tool-Based Visual Search RL Agents (e.g., Mini-o3, DRIM)**
- The agent operates over high-resolution scenes, issuing tool calls (zoom, crop, etc.) to progressively focus on targets.
- Reinforcement learning is employed, with reward for correct answers and often penalties or masking schemes for low-efficiency trajectories or failed explorations.
- Pipelines use supervised pre-training on trajectory demonstrations followed by policy optimization (e.g., Group Relative Policy Optimization, GRPO), often with special strategies such as “over-turn masking” to allow policies to scale to deep turn counts at inference [2509.07969][2512.17306].

**C. Multi-Image Chain-of-Thought Systems (e.g., MiCo, ImageChain)**
- Systems that concatenate embedded representations of multiple images (e.g., sequence of crops, video frames, or augmented views) as input, with reasoning performed over such chains in either an autoregressive or policy-driven manner.
- In MiCo, reasoning over image triplets is optimized through PPO-style RL without human-labeled QA, using self-supervision and prompt-engineered chain-of-thought emission [2506.22434].
- In ImageChain, interleaving scene images and text in a multi-turn conversational structure enables the model to perform contextually aware reasoning (e.g., next-scene description) [2502.19409].

**D. Visual-Only Rationale Rendering (e.g., Optical Reasoning)**
- In Optical Reasoning, all intermediate rationales are rendered as composite images (either typographic or graphical panels); MLLMs then process these visual rationales as the sole carrier of stepwise logic [2606.09585].

## 3. Benchmarks and Evaluation Protocols

IOMR systems are evaluated on diverse, highly-structured benchmarks that isolate the visual reasoning capability across domains, task types, and interaction patterns. Key benchmarks include:

**A. CoIEval** [2311.09241]
- 15 domains spanning geometry, chess, logical deduction, spatial planning, etc.; quantitative results show CoI surpasses text-based CoT by factors of 1.5–2.3× in accuracy on domains such as intersect geometry and chess tactics.

**B. VisChainBench** [2512.06759]
- 1,457 tasks, ~20,000 images, with three principal formats:
  - ITMR: Image-Text Multi-Turn
  - ICIR: In-Context Image-Only Reasoning
  - IOMR: Image-Only Multi-Turn Reasoning
- Each task requires selecting the correct next-step image in a sequential chain, minimizing language cues; overall correct-answer rates (CA) highlight a steep scaling curve with model size.

**C. MIRA** [2511.02779]
- 546 problems, each requiring intermediate visual “scratch-pad” steps.
- Evaluation distinguishes direct (no rationale), text-CoT, and visual-CoT (image-supported) reasoning; supplying annotated intermediate images boosts accuracy by up to 33.7% relative to text-only CoT.

**D. Optical Reasoning Benchmarks** [2606.09585]
- ScienceQA, AquaRat, GSM8K, Zebra-CoT, etc.: studies show visual rationales achieve equivalent or better accuracy with ~29% fewer reasoning tokens, and up to 1.96× token efficiency compared to text rationales.

**E. Visual Search and Multi-Image Reasoning Datasets** [2509.07969][2506.22434]
- VisualProbe, V* Bench, HR-Bench: test agents’ ability to search, categorize, or compare across multi-image or high-resolution scenes over multiple tool-invoking turns.

| Benchmark         | Format      | Task Count | Unique Aspects                 |
|-------------------|-------------|------------|-------------------------------|
| CoIEval           | Mixed       | 15 domains | SVG/image chain-of-thought     |
| VisChainBench     | Multi-image | 1,457      | Sequenced, minimal language    |
| MIRA              | Visual-CoT  | 546        | Annotated visual rationales    |
| Optical Reasoning | Visual-only | ~1,000s    | Panel/composite rationale imgs |
| VisualProbe       | Visual search| ~4,500     | Exploratory, tool-driven RL    |

## 4. Empirical Findings, Model Analyses, and Insights

- **Performance Gains**: Image-only reasoning frameworks (CoI, Optical Reasoning, visual-CoT in MIRA) consistently outperform text-only baselines in accuracy, particularly as task complexity rises or as domains become more spatial/structural. Relative gains range from 20–100% over text CoT, particularly on geometric, spatial, or visual search tasks [2311.09241][2511.02779][2512.06759][2606.09585].
- **Scaling and Error Propagation**: IOMR performance exhibits strong scaling with model capacity. However, error propagation is acute—early mistakes in a sequence often preclude recovery, drastically lowering task-completion rates compared to per-turn answer rates [2512.06759][2509.07969].
- **Critical Role of Visual Intermediates**: The provision (or model-generation) of precise intermediate sketches, diagrams, or sub-image crops often determines success. Qualitative examples reveal that even state-of-the-art LLMs hallucinate or over-abstract when deprived of visual intermediates [2511.02779][2311.09241].
- **Token Efficiency**: Optical reasoning achieves substantial reductions in reasoning token budgets, due to high information density and spatial layout, demonstrating intrinsic efficiency advantages over textual chains [2606.09585].
- **Tool-Driven Exploration**: Systems incorporating tool actions—zoom, crop, manipulate—require specialized reward schemes. Redundancy penalties, over-turn masking, and reward shaping are effective in driving deep, self-corrective multi-turn visual reasoning [2512.17306][2509.07969].

## 5. Limitations, Open Challenges, and Future Directions

- **Modality Constraints**: Current image-only chains are most effective in diagrammatic, spatial, or structural domains where information can be encoded with high fidelity as SVG or panel drawings. Photo-realistic or semantically-loaded scenes may require richer symbol vocabularies or integration of generative diffusion models [2311.09241][2606.09585].
- **Bottlenecks in Visual Encoding**: Sequential conditioning using high-dimensional visual embeddings can bottleneck as the chain length increases or when pixel budgets are tight. Hierarchical, object-centric, or memory-augmented latent architectures are proposed as possible solutions [2311.09241][2512.06759].
- **Visual Hallucination and Misparsing**: Models are prone to graphical hallucination (incorrect visual evidence, misleading diagrams) and may unevenly parse fine-grained layouts depending on architecture and rendering, underscoring the need for model-aware renderers and visual correctness feedback [2606.09585][2512.17306].
- **Learning to Generate vs. Use Visual Chains**: IOMR benchmarks such as MIRA and VisChainBench identify a gap between models' abilities to consume supplied visual chains versus autonomously generate effective intermediates. End-to-end agentic routines that “think by drawing” (via internal sketching tools or APIs) remain a target for future pretraining and RL scaffolding [2511.02779][2512.06759].
- **Benchmarking and Dataset Expansion**: Further progress hinges on developing benchmarks spanning new domains (e.g., medical procedures, code traces, architectural plans) and on open-sourcing data generation pipelines that minimize language priors and facilitate reproducibility [2512.06759].
- **System Integration**: Advanced systems will likely combine symbolic vector formats, strong visual encoders, and flexible tool APIs into unified closed-loop “draw–perceive–act” architectures [2311.09241][2512.17306][2509.07969].

## 6. Design Principles and Methodological Guidance

Emergent methodological principles for IOMR system builders include:

- **Employ symbolic vector graphics when precision is critical**: SVG-based reasoning offers lossless, model-controllable visual “scratch-pads” [2311.09241].
- **Fuse vision and language representations via cross-attention or gating**: Effective conditioning on visual state histories is essential for multi-turn coherence.
- **Structure tasks and prompts to coordinate multi-step visual dialogue**: Scaffolding, whether by explicit turn-based protocols or example-driven tool chains, improves learning depth and reliability [2511.02779][2512.17306].
- **Leverage both supervised and RL-based curriculum**: Cold-start SFT on multi-turn demonstration trajectories followed by redundancy-penalized RL matches or exceeds the robustness of closed-source baselines [2512.17306][2509.07969].
- **Explicitly account for scalability in turn depth vs inference budget**: Over-turn masking and similar schemes decouple training horizon from test-time depth, yielding systems that can naturally scale to long reasoning chains without premature answer emission [2509.07969].

In summary, IOMR is a rapidly advancing direction unifying chain-of-thought, reinforcement learning, visual tool invocation, and diagrammatic reasoning—shifting multimodal systems from “text that sees images” to agents that genuinely “think in pictures.” The field is defined by its drive toward human-like, image-centric intelligence, guided by rigorous benchmarks and a deepening analytical understanding of the bottlenecks and affordances of spatial reasoning with images [2311.09241][2512.17306][2512.06759][2606.09585][2511.02779][2509.07969][2506.22434][2502.19409].

Source: https://www.emergentmind.com/topics/image-only-multi-turn-reasoning-iomr