Thyme: Beyond Static Image Reasoning

• Thyme is a multimodal paradigm that equips models with the capability to autonomously generate and execute code for dynamic image manipulation and real-time mathematical computations.
• It integrates supervised fine-tuning with reinforcement learning, such as the GRPO-ATS algorithm, to enhance performance in high-resolution image tasks and complex computations.
• Key innovations include autonomous decision-making for tool invocation and controlled exploration, ensuring robust accuracy and versatility in multimodal processing.

Thyme (Think Beyond Images) refers to a novel paradigm in multimodal model design and training that enables autonomous decision-making for advanced image processing and mathematical computation within the reasoning loop, transcending static or purely “think with images” approaches. By further integrating code generation for both visual and computational operations, Thyme is distinguished by its capacity to execute diverse manipulations and execute mathematical analysis on the fly, governed by “high autonomy” in deciding when and how to invoke such processes. The paradigm is activated through a carefully staged supervised and reinforcement learning protocol, resulting in consistent performance gains across a wide spectrum of perception and reasoning tasks, especially those requiring high-resolution image detail or complex computational pipelines (Zhang et al., 15 Aug 2025).

1. Conceptual Overview

Thyme is introduced to supersede existing multimodal and “think with images” models by equipping the Multimodal LLM (MLLM) with the capability to autonomously generate executable code for both image manipulation and mathematical computations. Rather than restricting the model to region cropping or auxiliary generation, Thyme provides mechanisms to actively transform its inputs—cropping, rotating, adjusting contrast, zooming, or running mathematical scripts—integrating these steps as part of its inference cycle. The core focus is on enabling models to take initiative in both perceptual analysis and logical reasoning, aggregating information before formulating an answer.

2. Feature Set and Operational Mechanism

Thyme’s feature set is significantly broader than previously established frameworks, supporting the following operations:

Image Operations	Mathematical Reasoning	Autonomous Control
Cropping regions of interest (ROI)	Code execution for symbolic computation	Decides when to apply code/image ops
Rotation and orientation correction	Solving equations: $t = a \cdot m^b$	Balances exploration and precision
Contrast and lighting enhancement	Logarithmic transformations, solving for constants	Integrates decision policy with RL
Scaling and zooming of regions	Multi-step mathematical chains via Python scripts	Controls granularity of manipulation

This capacity is sustained by the model’s internal logic: for example, upon encountering a question regarding small text in a high-resolution image, the model autonomously generates code to crop and enhance that region, possibly followed by OCR; for computational problems, it generates executable scripts that solve or simulate equations, leveraging operations such as simultaneous equations or logarithmic transformations. The paradigm’s built-in autonomy ensures that image manipulations and code executions occur only when required, with a claim of high reliability and extensibility (Zhang et al., 15 Aug 2025).

3. Training Protocol: Supervised and Reinforcement Learning

The training regime is bifurcated into two strategic phases:

• Supervised Fine-Tuning (SFT): Thyme is trained on a set of approximately 500,000 curated samples spanning over four million raw sources, covering direct Q&A, image operations, mathematical code generation, and multi-turn dialogues. SFT utilizes masking of intermediate sandbox outputs and loss computation based only on the final dialogue stage, minimizing gradient contamination and preventing model collapse to erroneous patterns.
• Reinforcement Learning (RL) – GRPO-ATS Algorithm: During RL, high-resolution question–answer pairs are annotated for increased learning difficulty. The Group Relative Policy Optimization with Adaptive Temperature Sampling (GRPO-ATS) algorithm is applied: sampling temperature is set high (e.g., 1.0) for text to foster reasoning diversity and low (e.g., 0.0) for deterministic code generation. The RL objective is expressed as:

$$J_\text{GRPO}(\theta) = \mathbb{E} \left[ \frac{1}{|\tau|} \sum_{\text{tokens in } \tau} (A - \beta \cdot KL(\pi_\theta \| \pi_\text{ref})) \right]$$

where $A$ is the advantage incorporating correctness and format rewards and the KL divergence penalizes straying from a reference policy.

This dual-phase protocol ensures the model learns both the mechanics of code/image manipulation and the strategic selection thereof, yielding robust performance and minimizing hallucination or execution errors.

4. Performance Evaluation and Benchmark Results

Thyme has been validated on nearly 20 benchmarks, encompassing three principal areas:

• Perception: High-resolution image recognition tasks (e.g., HRBench, MME-RealWorld) where targeted image cropping and enhancement produce materially improved answer accuracy.
• Reasoning: Mathematical benchmarks where the ability to translate stepwise logic into executable Python scripts confers substantial computational reliability and accuracy.
• General Multimodal Tasks: Notable improvements in reducing hallucinations, iterative refinement, and multi-turn reasoning strategy.

Ablation studies, including masking sandbox outputs and focusing loss calculation on the final round, demonstrate the necessity of these design choices. Adaptive temperature sampling in RL is empirically shown to balance exploration in text with stability in code, harmonizing reasoning diversity with execution correctness.

5. Algorithmic Innovation: GRPO-ATS

GRPO-ATS represents a targeted advancement in reinforcement learning for multimodal models. By differential temperature control during token generation, it resolves the trade-off between exploratory reasoning and precise tool invocation. Text tokens are generated under elevated temperature, allowing for varied logical exploration, while code tokens are generated under zero-temperature for deterministic completeness—a requirement in error-sensitive computational domains. The objective formulation explicitly ties the policy update to final correctness and structural consistency, rather than relying on intermediate proxies or process-based signals.

6. Implications and Future Research Trajectories

Thyme’s autonomous code and image operation framework paves the way for AI agents capable of real-time interactive modification, iterative analysis, and error correction in highly demanding environments. Plausible implications include:

• Enhanced support for real-time domains such as medical imaging, autonomous driving, or scientific visual analysis, where domain-specific input transformations are pivotal.
• Robust foundation for ongoing research into multimodal RL algorithms that tune exploration granularity in tool use.
• The architecture encourages further work in scaling up both dataset diversity and underlying model capacity for ever more complex multimodal perception and reasoning scenarios.

The paradigm also emphasizes the growing importance of end-to-end systems that can actively transform and reason over visual data beyond passive interpretation, with integration of code execution bolstering logical capability.

7. Comparative Context and Positioning

Relative to preceding models that either statically “think with images” or rely on rigid tool invocation, Thyme is positioned as the first open-source paradigm offering feature parity with proprietary approaches (notably O3). Its mechanism for autonomous multi-stage processing—combining image operations, mathematical computation, and adaptive learning—marks a transition toward holistic, context-aware multimedia reasoning. The paradigm’s strategy and consistency of benchmark gains set a new standard for open-source multimodal systems in both breadth and depth of reasoning.

---

Thyme (Think Beyond Images) thus encapsulates a paradigm shift: from static perception and constrained tool invocation to dynamic, code-empowered, high-autonomy reasoning that actively manipulates and analyzes input data during inference. Its technical rigor, broad operational range, and reinforcement learning innovations situate it as a foundational direction for future multimodal AI research and applications (Zhang et al., 15 Aug 2025).