VICoT-Agent: Multimodal Reasoning Framework

• VICoT-Agent is a multimodal framework that interleaves language reasoning with visual/text tools for step-by-step remote sensing image analysis.
• The architecture employs a vision-interleaved chain-of-thought and a dynamic, stack-based reasoning structure, reducing token consumption by 65% and latency by 48%.
• The framework distills large teacher models into lightweight student models, enabling scalable deployment on resource-constrained satellite and edge devices.

VICoT-Agent is a multimodal agent framework introduced for interpretable, multi-round reasoning and dynamic tool usage in complex remote sensing image analysis. Unlike prior approaches focused on one-shot object recognition or decoupled tool-invocation, VICoT-Agent advances intelligence extraction by interleaving explicit language reasoning (chain-of-thought) with concrete visual and text tool operations at each inference step. This design supports high transparency, efficient execution, and modular extensibility, and further enables practical deployment on resource-constrained platforms by distilling large teacher agents into lightweight student models (Wang et al., 25 Nov 2025).

1. Motivation and Objectives

VICoT-Agent addresses the evolving requirements of remote sensing tasks, which demand step-by-step, interpretable intelligence extraction rather than simple object classification. Objectives include:

• Explicit, multi-round reasoning that interleaves natural language thought processes with vision-tool actions, ensuring each step is grounded in image evidence and transparent.
• Scalable deployment by distilling large multimodal agents (e.g., GPT-4o) into memory- and latency-efficient student models suitable for on-board satellite and edge environments.
• Modular tool suite integration to flexibly invoke vision and text utilities within the reasoning trajectory, without tightly binding tool logic to the core model architecture (Wang et al., 25 Nov 2025).

2. Architectural Components and Reasoning Workflow

The architecture of VICoT-Agent is centered on the Vision-Interleaved Chain-of-Thought (VICoT) methodology and a stack-based reasoning structure for efficient recursive processing.

2.1 Vision-Interleaved Chain-of-Thought

Traditional chain-of-thought approaches implement cyclical “think → act → observe” reasoning, but typically detach external tool invocation from explicit reasoning rounds. VICoT refines this by weaving tool calls directly into each reasoning phase:

1. Generate a language-based reasoning step $\varphi_t$.
2. Map $\varphi_t$ to a specific tool $m_t$ from the set $\mathcal{T}$ of available vision/text tools.
3. Invoke $m_t$, convert its output to textual evidence $e_t$ via a VLM bridge, and append the tuple $(\varphi_t, m_t, e_t)$ to the reasoning stack.

This explicit, region-grounded trajectory repeats until the completion of the task, producing a transparent stepwise reasoning history (Wang et al., 25 Nov 2025).

2.2 Stack-Based Reasoning Structure

VICoT maintains a dynamic stack $S_t$ to compress context and preserve reasoning causality without re-processing the entire inference history. Each stack frame $s_t = (\varphi_t, m_t, e_t)$ records:

• The reasoning step ($\varphi_t$)
• The chosen tool and arguments ($m_t$)
• The evidence returned ($e_t$)

Formally: $$\varphi_t = \begin{cases} h_\theta(x,\text{Prompt}), & t=1,\ h_\theta(S_{t-1}), & t\ge2 \end{cases}$$

$$m_t = g_\theta(\varphi_t, \mathcal{T}), \quad e_t = \tau_{i_t}(\alpha_{i_t})$$

$$S_t = \mathrm{push}(S_{t-1}, (\varphi_t, m_t, e_t))$$

This stack-based compression yields linear context-token growth $O(T)$—remarkably lower than quadratic $O(T^2)$ requirements of plan-replan loops—resulting in a 65% reduction in token consumption and 48% lower latency compared to previous paradigms. VICoT can also manage tool-selection ambiguity by forking parallel stacks $\mathcal{P}_t = \{ S_t^{(1)}, ..., S_t^{(W_t)} \}$, subsequently pruning suboptimal paths with lightweight heuristics (Wang et al., 25 Nov 2025).

2.3 Modular MCP-Compatible Tool Suite

VICoT integrates vision and text tools through a unified XML Function-Calling interface compliant with the Model Context Protocol (MCP). Vision tools include GroundingDINO for open-vocabulary detection, region cropping, super-resolution (Real-ESRGAN), binarization, denoising, deblurring, and cloud/rain removal. Text tools comprise web search and retrieval-augmented generation via vector databases. LLM outputs at each reasoning step are vectorized to query all tools in $\mathcal{T}$, dynamically selecting the highest-scoring interface, ensuring full decoupling and “plug-and-play” extensibility (Wang et al., 25 Nov 2025).

3. Reasoning Stack Distillation Methodology

3.1 Motivation and Technical Constraints

Teacher agents based on large LLMs like GPT-4o demonstrate superior multi-modal reasoning but exceed resource constraints of edge and orbital hardware. Distilling the explicit stack-based reasoning behaviors—including intermediate traces and tool selections—into student models (e.g., Qwen3-14B with AWQ 4-bit quantization) reduces model size and computational overhead:

• Model size: >50 GB (teacher) → ~12 GB (student)
• VRAM requirement: 16 GB
• Inference latency: 10–15 s/image
• Reasoning quality: BLEU score gain of +3; maintains or improves trajectory accuracy relative to teacher (Wang et al., 25 Nov 2025).

3.2 Distillation Protocol

A custom dataset, VICoT-HRSC ($N \approx 364$ UHR remote sensing images), provides full reasoning-stack trajectories $\{S_1, ..., S_T\}$ extracted from the teacher. The student’s parameters $\phi$ are trained to mimic both the sequence of reasoning steps and exact tool-selection distributions at each turn, using a distillation loss: $$\mathcal{L}_{\mathrm{distill}} = \sum_{t=1}^T \left[ \alpha\,\mathrm{CE}(\varphi_t^{(\mathrm{tea})}, \varphi_t^{(\mathrm{stu})}) + \beta\,\mathrm{KL}(p^{(\mathrm{tea})}(m_t|S_{t-1}) \,\Vert\, p^{(\mathrm{stu})}(m_t|S_{t-1})) \right]$$ where CE(·) is cross-entropy for reasoning trace tokens, KL(·) aligns tool selection probabilities, and $\alpha, \beta$ are loss balancing coefficients. This protocol ensures the student model acquires both thought process emulation and operational tool-procedure fidelity (Wang et al., 25 Nov 2025).

4. Experimental Results and Evaluations

4.1 Benchmarks and Tasks

VICoT-Agent was evaluated on:

• VICoT-HRSC: custom multi-turn remote sensing reasoning dataset
• RSVQA (Low-Res, High-Res): visual question answering
• Ultra-high-resolution VQA: MME-RealWorld-RS, LRS-VQA-FAIR
• Additional low/medium-resolution remote sensing datasets (Wang et al., 25 Nov 2025)

4.2 Comparative Quantitative Results

Model	Trajectory Accuracy (%)	BLEU	Token Consumption	Inference Latency
VisionGPT	61.7	—	—	—
ViperGPT	71.4	—	—	—
HuggingGPT	74.1	—	—	—
GPT-4o (tools only baseline)	68.8	0.73	—	—
VICoT (GPT-4o, teacher)	92.3	0.95	–65%	–48%
VICoT (Qwen3-14B distilled)	88.6	—	—	10–15 s / image

• On RSVQA-LR: VICoT-4o achieves 94% vs. 89.2% (GPT-4o baseline)
• On RSVQA-HR: VICoT-4o achieves 92.3% vs. 75.3% (GPT-4o baseline)
• On ultra-high-resolution MME-RW-RS and LRS-FAIR tasks, VICoT-4o gains +3.8% and +2.4% relative to GPT-4o
• Human evaluators and automated GPT-4.1 rating systems report a 20–30 point advantage in coherence, grounding, and information richness over alternative LLM+tool frameworks (Wang et al., 25 Nov 2025)

4.3 Transparency and Output Structure

Every reasoning step is explicitly documented in a stack frame $(\varphi_t, m_t, e_t)$, supporting retraceability and auditability. Final outputs adhere to the SOAP format (Subject, Objective, Assessment, Plan), streamlining integration into downstream intelligence workflows (Wang et al., 25 Nov 2025).

5. Tooling Architecture and Extensibility

VICoT’s tool suite leverages the MCP protocol with XML-based function calling. Tool categories include:

• Vision tools: open-vocabulary detection, cropping, super-resolution, binarization, denoising, deblurring, atmospheric artifact removal.
• Text tools: web search, retrieval-augmented generation. The design maintains full decoupling of tool logic, enabling independent enhancement and domain adaptation. Each reasoning step vectorizes the LLM output to rank and select tools dynamically, supporting seamless integration of future analytics capabilities (e.g., georegistration, change detection) with further interface and rule specification (Wang et al., 25 Nov 2025).

6. Limitations and Prospects for Advancement

VICoT-Agent’s current focus is on remote-sensing images. Extension to fully open-domain multimodal scenarios—such as incorporation of natural images and video—is recognized as an open challenge. Additional limitations include:

• Scalability and generalization for sub-10B parameter student models, especially under zero/few-shot conditions. Exploration of meta-distillation and efficient prompt tuning is anticipated.
• Existing MCP tool pipelines would require augmentation for real-time geospatial analytics.
• Catastrophic LLM tool-invocation failures, such as invalid crop coordinates, are presently mitigated via heuristic retries; more robust error recovery mechanisms are needed.

A plausible implication is that VICoT-Agent’s stack-based, interleaved reasoning and modular tooling framework provides a blueprint for future interpretable, resource-efficient multimodal agents across increasingly demanding visual reasoning domains (Wang et al., 25 Nov 2025).