Gym-V: A Unified Vision Environment System for Agentic Vision Research

Abstract: As agentic systems increasingly rely on reinforcement learning from verifiable rewards, standardized ``gym'' infrastructure has become essential for rapid iteration, reproducibility, and fair comparison. Vision agents lack such infrastructure, limiting systematic study of what drives their learning and where current models fall short. We introduce \textbf{Gym-V}, a unified platform of 179 procedurally generated visual environments across 10 domains with controllable difficulty, enabling controlled experiments that were previously infeasible across fragmented toolkits. Using it, we find that observation scaffolding is more decisive for training success than the choice of RL algorithm, with captions and game rules determining whether learning succeeds at all. Cross-domain transfer experiments further show that training on diverse task categories generalizes broadly while narrow training can cause negative transfer, with multi-turn interaction amplifying all of these effects. Gym-V is released as a convenient foundation for training environments and evaluation toolkits, aiming to accelerate future research on agentic VLMs.

• The paper introduces Gym-V, a unified gym-style framework with 179 vision environments designed for standardized RL and multimodal evaluations.
• It leverages procedural task generation, composable wrappers, and automated verifiers to enable controlled studies on prompt engineering and agent performance.
• Empirical results reveal significant inter-model performance gaps, highlighting the critical role of rich observation scaffolding and curriculum diversity.

Gym-V: A Unified Vision Environment System for Agentic Vision Research

Platform Overview and Motivation

The development of agentic vision-LLMs (VLMs) has been impeded by the lack of a unified gym-style benchmarking and training infrastructure analogous to what exists for text-based agentic LLMs. "Gym-V: A Unified Vision Environment System for Agentic Vision Research" (2603.15432) addresses this critical gap by introducing a framework comprising 179 visually grounded environments across 10 categories, designed for standardized RL training, offline learning, and consistent evaluation of multimodal agents.

Gym-V is built around several core design principles:

• Procedural generation of task instances to ensure scalability and continuous difficulty adjustment.
• Composability of observation wrappers for explicit experimental control over text scaffolding, context history, and rule descriptions.
• Automatic verifiers and integration with an evaluation-as-a-service (EaaS) backend for both discriminative and generative tasks.
• A unified, Gym-compatible API applicable to single-turn, multi-turn, multi-agent, tool-augmented, and generative tasks.

  Figure 1: Gym-V comprises 105 single-turn and 74 multi-turn environments across 10 categories, unified by a common reset/step interface for diverse workflows including both interactive and offline tasks.

System Architecture

Environment Interface

Gym-V extends the classical Gym API with multi-agent support (in line with Ray RLlib) and batch abstractions for seamless transition between online RL and offline dataset-driven learning, supporting consistent evaluation pipelines and data organization regardless of the underlying environment or modality.

Composable Wrappers

A crucial contribution of Gym-V is the explicit treatment of observation scaffolding, rules, captions, and context as first-class experimental variables. Wrappers allow modification of task presentation at the agent–environment interface, enabling controlled studies on how prompt construction, history window length, and auxiliary descriptions influence model behavior.

Distributed Evaluation Service

Evaluation fidelity and scalability are ensured by a Ray Serve-based EaaS architecture. This allows for unified scoring of both discriminative and generative tasks using integrated, scalable reward models (e.g., CLIP, HPSv3, VLMs, remote APIs), ensuring that custom reward models or external benchmarks can be hot-swapped without impacting environment code.

Figure 2: Evaluation outputs from Gym-V closely track official results for both VLM and generative vision benchmarks, ensuring reliable RL and comparative evaluation.

Environment Suite and Taxonomy

The environment suite is hierarchically organized to span the breadth of vision-agent challenges. It includes:

• Single-turn tasks: Perceptual reasoning, algorithmic problem solving, geometric and cognitive tasks, graph and logic reasoning, and classic puzzles.
• Multi-turn tasks: Board/card/video games, spatial 2D/3D navigation, temporal/reactive control in retro games.

Each environment provides parameterized difficulty for curriculum learning and scalability, deterministic seeding, and large solution spaces to mitigate reward hacking and brittle overfitting.

Empirical Evaluation and Benchmarks

Zero-shot Benchmarking

Extensive zero-shot evaluation of nine VLMs (open and closed source) demonstrates that even state-of-the-art models leave substantial room for improvement across Gym-V tasks. The strongest closed model, Gemini-3-Pro, significantly outperforms the best open-weight models (e.g., Qwen3-VL-32B), with the latter achieving only about half the average performance.

Figure 3: Performance across single-turn and multi-turn tasks shows steep inter-model gaps, with Gemini-3-Pro leading but ARC (abstract pattern tasks) and hard combinatorial domains remaining challenging even for frontier models. Right: Accuracy rapidly degrades with increasing difficulty, evidencing sharp “difficulty cliffs.”

Scale alone is insufficient for closing the capability gap: newer training recipes and better reasoning supervision yield greater lifts in single-turn visual reasoning than simply scaling open-weight models.

Algorithmic Insights: RL Method Comparison

Controlled experiments with GRPO, GSPO, and SAPO RL algorithms across diverse environments evidence that observation scaffolding (e.g., rich captions, explicit rules) has a far larger impact on learning success than the specific RL algorithm. All algorithms can effectively learn in well-scaffolded problems, but stability and final returns in long-horizon, multi-turn tasks reveal nuanced differences: GSPO excels in stability, while SAPO can collapse under large policy drift.

Figure 4: RL algorithm comparison across 16 environments. All settings are learnable; no method dominates outright. Multi-turn task difficulty manifests in slower convergence and lower returns.

Multi-Turn and Vision-specific Ablations

Systematic ablation studies crisply reveal the decisive role of prompt construction and scaffolding:

• Interaction history: Providing recent context (3 or 5 turns) versus memoryless (MDP) observation universally improves training in multi-turn environments.
• Explicit rules vs. exploration: Including explicit environment rules dramatically improves learning in irreversible-score contexts (Sokoban), but may be less critical in domains with dense reward feedback (Minesweeper).
• Textual captions: Image + caption input consistently outperforms image-only across single- and multi-turn settings, especially for perception-heavy tasks and situations where visual grounding is a bottleneck.

  Figure 5: Training reward curves for ablations over context and rules in multi-turn games. Models given rules or longer context improve faster and reach higher effective returns.

  

  Figure 6: Adding caption-based scaffolding to image observations yields robust and early gains across both single-turn and multi-turn environments, most pronounced in visually challenging domains.

Implicit Rule Induction via Context

Additional studies demonstrate that providing contextual trajectories enables models to induce environment rules without explicit prompting, closing much of the gap to fully-scaffolded performance for many domains, with exceptions in settings involving irreversible or latent constraints.

Figure 7: Increasing context window or providing explicit rules enables VLMs to approach the performance of rule-informed agents, evidencing powerful implicit rule discovery from sequence data.

Cross-Domain Generalization and Negative Transfer

Gym-V supports curriculum and transfer studies: broad, diverse curriculum training (as in Cognition and Puzzles) yields strong cross-domain improvements, while overspecialized training in narrow domains (e.g., Geometry) induces negative transfer. Transfer matrices are asymmetric, indicating skill hierarchy: Logic often transfers more to Cognition than vice versa.

Multi-turn environments exhibit pronounced fragility: while in-domain gains dominate, cross-game transfer is limited and negative transfer emerges, emphasizing the compounding effects of error propagation in sequential settings.

Implications and Future Directions

The findings of this work have multiple implications for VLM research and reinforcement learning more broadly:

• Observation scaffolding is paramount. Textual and rule-based context design, rather than optimization details, is the main bottleneck for complex vision-agent tasks. Future architectures must emphasize rich multimodal scaffolding and flexible prompt engineering.
• Evaluation paradigms must evolve to avoid saturation and test non-trivial generalization. The procedural, configurable Gym-V framework enables continual curriculum escalation and robustness-to-difficulty sweeps.
• Negative transfer and catastrophic overfitting are real risks: domain-specialized RL without sufficient curriculum diversity can suppress generalization, a phenomenon likely relevant for both vision and language RL pipelines.
• Multi-turn interaction amplifies all effects: both advantages of scaffolding and liabilities of algorithmic instability or representation bottlenecks are significantly magnified in sequential decision problems.
• Temporal/reactive control remains an open frontier: VLMs currently exhibit minimal progress on retro arcade tasks even after hundreds of actions, necessitating advances in temporal abstraction and action planning.

  Figure 8: VLM agents make little progress on fine-grained retro arcade benchmarks, highlighting temporal perception and control as open challenges.

Representative Task Visualizations

Figure 9: Algorithmic tasks challenge VLMs with rendered grids and transformations.

Figure 10: Cognition environments probe mental rotation, odd-one-out, and 3D spatial reasoning.

Figure 11: Geometry tasks require inference over 2D layouts and convex shapes.

Figure 12: Graph tasks assess pathfinding and reasoning over variable-sized visual graphs.

Figure 13: Logic tasks involve constraint-based grid filling and Boolean circuit tracing.

Figure 14: Puzzles such as KnightSwap and Tower of Hanoi induce search and planning from rendered states.

Figure 15: Games, including board and grid-based, support single and adversarial multi-agent play.

Figure 16: Spatial navigation in both 2D and 3D, requiring goal-directed reasoning under partial observability.

Figure 17: Temporal environments test the limits of continuous visual control and perception.

Figure 18: Heatmap of per-environment, per-model zero-shot results reveals persistent broad gaps and sharp specialization.

Conclusion

Gym-V provides a rigorous, scalable, unified system for studying agentic VLMs under RL, offline, and generative paradigms. Its design foregrounds observation scaffolding and prompt engineering as critical determinants of learning success, uncovering steep difficulty cliffs and broad limitations in current open- and closed-weight models. Rich curriculum support, procedural scaling, and unified evaluation enable robust transfer, ablation, and negative transfer analysis. As vision-agent research continues to progress toward more generalist, robust, and temporally competent systems, Gym-V serves as a foundational ecosystem for reproducible and comprehensive evaluation, guiding both theoretical advances in multimodal modeling and practical improvements in agentic RL algorithms.