LLM Agents Can See Code Repositories

Abstract: Coding agents powered by LLMs have demonstrated strong performance on software engineering tasks. Yet most agents consume repositories almost entirely as text, which differs from how human developers use visual structure such as folder hierarchies and dependency relationships to orient themselves in large codebases. With multimodal LLMs (MLLMs), it is an open question whether agents can effectively benefit from visual representations of repositories. This paper presents the first systematic empirical study of visual repository representations for LLM-based agents on repository-level issue resolution. We evaluate four recent multimodal models. Our results show that a strictly vision-only setup degrades accuracy and increases token cost, because agents lack sufficient symbolic detail and compensate with repeated visual queries. In contrast, integrating visual graphs of repository structure as a supplementary modality alongside standard text interfaces helps agents understand structure more efficiently: input token consumption decreases by up to 26% while issue-resolution accuracy is maintained or improved. Visualization is most useful during fault localization and when the agent autonomously controls exploration depth. These findings point to a practical hybrid text-and-vision design for next-generation coding agents.

• The paper demonstrates that integrating visual repository representations with text significantly enhances accuracy and efficiency by reducing token cost and API calls.
• It employs multimodal context via dependency graph layouts and dynamic depth queries to optimize performance across localization, repair, and validation stages.
• Empirical results reveal that early-stage visual context, especially during localization, minimizes navigation errors and optimizes computational resource allocation.

Visual Repository Representations in Multimodal Coding Agents

Motivation and Problem Statement

LLM agents have achieved notable proficiency in repository-level software engineering tasks, yet conventional approaches rely almost exclusively on text-based interfaces for code navigation and reasoning. In contrast, human developers exploit visual characteristics—directory hierarchies, file dependencies, syntax highlighting—to comprehend repositories efficiently. The rapid evolution of Multimodal LLMs (MLLMs) prompts the investigation: does supplementing textual code context with visual repository representations confer advantages in comprehension or efficiency for coding agents? The paper systematically quantifies the impact of integrating visual structural context and textual code in agentic workflows, introducing SeeRepo, a multimodal augmentation for repository-level issue resolution.

Methodology

Multimodal Context Integration

SeeRepo augments standard agent interfaces with visual dependency graphs derived from AST-based static analysis. Dependency relations—contains, imports, invokes, inherits—are rendered as structured images (via Graphviz) and supplied as supplementary modality alongside text-based navigation (Bash commands, file access, code editing). Four state-of-the-art MLLMs are evaluated: GPT-5-mini, GPT-5.1, Doubao-Seed-2.0-Lite, and Kimi K2.5. The study examines:

• The performance boundary between vision-only and hybrid modalities,
• Effects of different visualization layouts (graph, nested, tabular),
• Dynamic versus fixed neighborhood depth in visual structural queries,
• Visualization benefits across distinct agent pipeline stages (localization, repair, validation).

Benchmarking is conducted on SWE-bench Verified (500 Python repository instances), SWE-Rebench Leaderboard, and SWE-QA (repository code question answering), using Pass@1 and Overall Score for effectiveness, and token consumption, API calls, and monetary cost for efficiency.

Empirical Results

Vision-Only Versus Multimodal Context

A vision-only context severely degrades both performance and efficiency. Removing textual access in favor of exclusively visual structural cues reduces Pass@1 by 13.6–34.1 points across evaluated models, with Doubao-Seed-2.0-Lite suffering the most extreme drop—from 51.0% to 16.9%. Token cost increases up to 268% and API calls surge (up to +95%). Agents repeatedly invoke graph queries to compensate for the lack of symbolic reasoning, accumulating high computational overhead but failing to recover accuracy.

In contrast, supplementing text interfaces with visual context achieves substantial cost reductions while maintaining or improving accuracy. GPT-5-mini with SeeRepo exhibits Pass@1 of 55.4% (+0.4) and up to 26% lower token cost. GPT-5.1 achieves the most pronounced efficiency gain (−46% cost), despite a minor accuracy regression, attributed to overreliance on superficial structural matches. Kimi K2.5 realizes simultaneous accuracy improvement (+1.8) and reduced cost (−3%). The effect generalizes across SWE-Rebench and SWE-QA, where multimodal context yields higher effectiveness with substantial reductions in API calls and token consumption.

Visualization Layouts and Structural Query Depth

All visual layouts outperform the text-only baseline in efficiency. The graph layout achieves the greatest token reduction (−25%) with competitive accuracy (+0.4), whereas the tabular layout offers the highest Pass@1 (56.2%) but with smaller efficiency gains. The nested layout balances hierarchical visualization and context density. Dynamic graph depth, where the agent autonomously determines exploration breadth per query, attains the optimal trade-off; it reduces input tokens by 25% and cost by 26% with comparable accuracy to fixed-depth alternatives. Fixed depths beyond 2-hops (Depth=4) further boost accuracy (+2.2), but at diminishing returns in token efficiency.

Visualization Tool Invocation in Agent Pipeline

Visualization delivers maximal benefit when invoked in the localization stage. Early structural grounding (fault localization) enables efficient narrowing of candidate code regions, reducing redundant exploratory steps. Pass@1 improves marginally (+0.4) with 26% lower cost. Invocation during repair or patch validation phases degrades accuracy (−5.0 and −3.4 points respectively), as excessive or late-stage structural context introduces distraction or prompts unnecessary modifications.

Case Analysis

Qualitative case study on a SWE-bench instance demonstrates that SeeRepo-equipped agents minimize navigation errors, reduce total token consumption by 32.6% and agent rounds by 29%. By replacing iterative grep-based exploration with topology-aware queries, multimodal agents acquire decisive structural context with fewer steps, dedicating more computational resources to reasoning and patch synthesis.

Implications and Future Directions

Practical Relevance

Hybrid text+vision modalities enable coding agents to exploit high-information-density context for repository navigation. Token efficiency is paramount in long-horizon workflows involving large repositories; SeeRepo demonstrates that visual structural grounding accelerates localization and reduces input requirements without impairing downstream repair accuracy. The results reveal a modality-sensitive performance boundary: visual representations are synergistic only when integrated with text, not when they supplant it.

Theoretical Significance

The findings substantiate that the channel of structural information transfer—visual graphs versus serialized text—materially shapes agentic reasoning dynamics. Visualization scaffolds promote global context awareness and dependency reasoning, especially during initial exploration, aligning agent behavior closer to developer practices. However, untargeted or late-stage invocation of visual context introduces noise, suggesting that modality selection and timing should be governed by policy.

Prospects for Multimodal AI in SE

Adaptive visualization strategies, learning-based invocation policies (e.g., RL for tool calling), and dynamic integration of runtime signals (execution traces) present promising avenues. Extending multimodal repository grounding to languages and architectures beyond Python warrants investigation. Cross-modal agents may eventually bridge code and user experience artifacts, generalizing to GUI issue resolution and mixed visual/textual debugging.

Conclusion

The paper presents a comprehensive quantitative and qualitative assessment of visual repository representations in agentic software engineering workflows (2606.14061). SeeRepo, a multimodal augmentation for LLM-driven coding agents, systematically lowers token cost and preserves accuracy by combining visual structural context with symbolic text interfaces. Empirical results demonstrate:

• Vision-only modality is insufficient for effective issue resolution,
• Hybrid text+vision context delivers substantial efficiency gains,
• Graph layout and dynamic depth queries maximize cost-effectiveness,
• Early-stage visualization (localization) achieves the most pronounced benefit.

These findings delineate modality-specific boundaries for agentic code reasoning and inform the design of cost-efficient, structurally-aware multimodal coding agents. Structured visual grounding is poised to become integral in advanced repository navigation and localization, paving the way for future agent architectures optimized for large-scale, multimodal software engineering environments.