Thyme: Beyond Image Processing

• Thyme is a multimodal framework that autonomously generates and executes code to enhance image manipulation and logical reasoning.
• It employs supervised fine-tuning and reinforcement learning via GRPO-ATS to optimize deterministic code execution while fostering diverse reasoning strategies.
• Thyme achieves significant performance gains in perception, reasoning, and general multimodal tasks, setting a new benchmark for open-source models.

Thyme refers to an emerging paradigm and framework advancing multimodal LLMs (MLLMs) that go beyond merely processing images as static context. Instead, Thyme enables models to autonomously generate and execute diverse image processing and computational operations via executable code, thus fusing rich visual manipulation capabilities with logical reasoning. This approach establishes a new degree of autonomy in end-to-end perception-reasoning, substantially enlarging the operational scope of open-source multimodal models.

1. Motivation and Historical Context

The introduction of “think with images” by OpenAI catalyzed several research efforts exploring methods for more actively leveraging visual information within AI reasoning. Prior open-source approaches primarily relied on auxiliary image generation or bounded box cropping. These methods were constrained by limited manipulation abilities, low image generation fidelity, and expensive computational requirements. Proprietary systems (e.g., O3), however, demonstrated richer suites of manipulations and tightly integrated logical reasoning, but this breadth of capability was inaccessible in open frameworks. Thyme was developed to address these gaps by empowering MLLMs to not only perform sophisticated image operations but also seamlessly integrate mathematical computations through code generation, effectively “thinking beyond images” (Zhang et al., 15 Aug 2025).

2. Core Innovations and Functionalities

Thyme’s distinguishing feature is its ability to autonomously synthesize executable code for image and reasoning tasks. Models trained under the Thyme paradigm can decide when an image requires tool usage, generate precise code for multiple operations—including cropping, scaling, rotation, and contrast enhancement—and perform high-level mathematical computations.

For example, given a visual task described by a formula such as $t = a \cdot m^b$, the model is capable of:

• Inferring values for $a$, $b$ from data points,
• Generating code to solve for these values,
• Executing the code in a sandbox, and
• Incorporating the output result into the reasoning process.

The entire code execution is managed in a secure sandbox, which also performs format correction and variable alignment to minimize the cognitive burden on the model (e.g., fixing minor cropping coordinate errors). Code segments and textual reasoning are segregated with specialized markers and distinct temperature settings, ensuring determinism in code and diversity in text.

3. Training Strategy: SFT and RL with GRPO-ATS

Thyme employs a two-stage training pipeline:

• Supervised Fine-Tuning (SFT): A curated dataset of 500,000 samples—extracted from more than 4 million raw sources—is used. This dataset spans tasks requiring direct answers, image manipulations, and mathematical computations. The SFT phase trains both chain-of-thought reasoning and correct code generation. The internal sandbox outputs are masked; loss computation focuses only on the final answer to stabilize learning and prevent gradient contamination.
• Reinforcement Learning (RL) Phase: The RL phase further refines decision-making and tool use autonomy. High-resolution, perceptually complex examples are manually curated to challenge the model. The training uses GRPO-ATS (Group Relative Policy Optimization with Adaptive Temperature Sampling). GRPO-ATS (Editor's term) sets temperature to zero for code generation (ensuring deterministic, executable code) and allows a temperature of one for text tokens (supporting reasoning exploration). The RL reward combines result accuracy, proper formatting, and consistency between reasoning and final answers.

4. Benchmark Results and Experimental Analysis

Thyme was evaluated on nearly 20 benchmarks across three core domains:

• Perception Tasks: Evaluated on HRBench, MME-RealWorld, and V*, Thyme demonstrated substantial improvements in high-resolution image recognition relative to established baselines (e.g., Qwen2.5-VL-7B), outperforming larger-scale models on some benchmarks.
• Reasoning Tasks: Conversion of complex logical problems into executable code led to enhanced performance on visual reasoning and mathematical benchmarks such as MathVision, MathVerse, and LogicVista.
• General Multimodal Tasks: The system improved performance in OCR, chart understanding, and effectively reduced hallucination rates due to its precise reasoning and tool integration.

Comprehensive analyses and ablation studies showed consistent and significant performance gains, particularly in challenging, high-resolution and multi-step computation scenarios.

5. Architectural Mechanisms: Code Execution and Autonomy

Thyme's architecture is characterized by its interaction between code generation and execution. The model autonomously chooses whether a problem requires code-based image manipulation or mathematical computation, generates the requisite Python code, executes it in a sandbox, and incorporates the result into ongoing reasoning. The training pipeline is tightly coupled with runtime error correction, result validation, and differential temperature settings.

A plausible implication is that decoupling reasoning exploration from code execution fidelity allows models to maintain creative autonomy while producing robust, verifiable tool outputs. This mechanism may underpin future advances in generalist MLLMs with strong reliability in both visual and logical tasks.

6. Implications and Future Directions

The Thyme paradigm holds implications for several domains:

• Unified Perception-Reasoning: Dynamic tool invocation and code execution enable closer approximation to human problem-solving capabilities, where perception and computation are interwoven.
• Benchmarked Generalization: By integrating datasets, sandboxing tools, and code, Thyme supplies a robust platform for benchmarking and for further exploration of complex multimodal tool chains and autonomous module interaction.
• Broader Applications: The model’s adaptive problem-solving could extend to robotics, GUI agents, real-time decision-making in high-resolution scenarios, and domains requiring on-the-fly perception and computation.
• Research Expansion: Research may focus on scaling Thyme to larger model sizes, incorporating additional toolchains, refining reward strategies in RL, and developing more advanced evaluation methodologies targeting realistic, visually and logically complex settings.

7. Comparative Perspective and Open-Source Impact

Thyme establishes a precedent for open-source models that combine dynamic, richly parameterized image manipulations and autonomous reasoning. The release of code, datasets, and sandboxing infrastructure supports replication, benchmarking, and further methodological innovation. This sets a foundation for the evolution of MLLMs that can not only comprehend but actively manipulate and reason through complex visual and mathematical phenomena in real time.

In summary, Thyme exemplifies a shift toward highly autonomous, tool-integrated multimodal LLMs that are able to “think beyond images” by generating and executing code for diverse image and reasoning operations. Its innovative training strategy, combining large-scale SFT and targeted RL via GRPO-ATS, yields consistent gains and provides a clear trajectory for future research in multimodal AI (Zhang et al., 15 Aug 2025).