CodeVision: Multimodal Code & Visual Analysis

Updated 4 July 2026

CodeVision is a multi-faceted research domain that integrates programming visualization, image-to-code benchmarks, and multimodal code analysis.
It utilizes structured representations like CCR and CGR and innovative techniques such as 2D token probability mapping to enhance code evaluation.
Empirical studies across debugging, synthesis, and vulnerability detection highlight its potential to improve code comprehension and automated program repair.

In the arXiv literature, CodeVision denotes several distinct research artifacts at the interface of programming and visual computation rather than a single standardized system. The name is used for a CS2 program visualizer, a reference-code-free image-to-code benchmark, a flowchart-to-code evaluation suite, a detector for LLM-generated code based on 2D token probability maps, a multimodal code-generation framework built by merging vision and coding models, a code-as-tool framework for visual reasoning, and broader visual representations for source-code understanding and vulnerability prediction (Bansal et al., 2021, Periasami et al., 11 May 2026, Wang et al., 17 Feb 2025, Xu et al., 6 Jan 2025, Jiang et al., 13 Aug 2025, Guo et al., 3 Dec 2025, Shi et al., 2022, Bilgin, 2021).

1. Terminological scope

The term appears in several non-identical contexts. A concise taxonomy is therefore necessary before discussing individual systems.

Usage of “CodeVision”	Main object	Citation
Pedagogical program visualization	“Eye” as a web-based, interactive visual interpreter for C++ with STL	(Bansal et al., 2021)
Reference-code-free image-to-code evaluation	Multi-domain benchmark and evaluation suite for image reconstruction as executable code	(Periasami et al., 11 May 2026)
Flowchart-to-code reasoning	Benchmark for generating correct code from flowcharts	(Wang et al., 17 Feb 2025)
Flowchart-to-code repair	Automated repair framing used by SVRepair	(Tang et al., 5 Feb 2026)
LLM-generated code detection	2D token probability maps plus ViT/ResNet classification	(Xu et al., 6 Jan 2025)
Unified multimodal code generation	VisCodex, presented as merging vision and coding models	(Jiang et al., 13 Aug 2025)
Code-as-tool visual reasoning	Framework where the model emits executable Python for image operations	(Guo et al., 3 Dec 2025)
Visual source-code understanding	ASCII- or AST-based image representations of code	(Shi et al., 2022, Bilgin, 2021)

This distribution of meanings suggests that CodeVision functions as a recurring label for research that converts code into visual objects, converts visual objects into code, or uses vision methods to reason about programs and programming-related artifacts.

2. Pedagogical program visualization and the “Eye” model

In the CS2 setting, Eye is described as a web-based, interactive visual interpreter for C++ with STL that embodies the principles of a “CodeVision” tool. Its architecture is explicitly three-module: a Python front-end using rply to lex and parse C++, an interpreter in which each AST node implements an exec(env) method and emits a Canonical Graphics Representation (CGR) in JSON, and an HTML5/CSS/JavaScript renderer that supports “Next” and “Prev” stepping (Bansal et al., 2021).

The architecture is organized around two canonical intermediate representations. Module 1 produces a language-independent Abstract Syntax Tree called the Canonical Code Representation (CCR). Module 2 executes CCR nodes and emits CGR frames containing the current line number, a human-readable explanation, call-stack frames with variable names, values, and scopes, and snapshots of active data-structure instances such as arrays, maps, and queues. Module 3 incrementally consumes CGR JSON and draws the source pane, call-stack pane, and data-structure pane. The modularity is explicit: adding Java or Python support requires replacing the parser/AST generator, while instructors can customize exec methods or add new library classes without changing the display layer.

The runtime pipeline is line-by-line and operation-level. Parsing maps source code to CCR; interpretation walks the AST in execution order; each atomic action such as a variable read, variable write, branch decision, or data-structure access leads to a distinct CGR frame. For library calls such as vector.push_back or map.insert, the interpreter traverses the data structure’s internals and records sub-steps with color-coded events: “read,” “modify,” and “delete.” The renderer then visualizes arrays as contiguous indexed boxes, stacks and queues as lists with push/pop animations, binary search trees as node-and-pointer structures with traversal highlighting, and hash tables as arrays of buckets with chains. The reported worst-case redraw cost per frame is $O(n)$ , and a data-structure operation spanning $k$ internal steps yields approximately $k$ frames (Bansal et al., 2021).

The empirical evaluation comprised two controlled studies with $N=20$ CS undergraduates. In logical debugging, the average fraction of total time spent when using Eye was 48.6% versus 58.3% on one problem and 41.7% versus 51.4% on a second, with paired $p=0.0578$ . In code comprehension, quiz scores were 60.6% with Eye versus 59.2% without, with $p=0.446$ . An anonymous survey reported an estimated 61.4% average frustration reduction and 1.56× more problems solved when using the tool. The paper simultaneously reports limitations: familiarity bias, the p-value being slightly above conventional 0.05 in the debugging study, current coverage being limited to C++ STL, and $O(n)$ redraw cost for very large data structures (Bansal et al., 2021).

3. Benchmarks for image-to-code and flowchart-to-code

A major use of the name concerns evaluation rather than pedagogy. In one formulation, CodeVision is a reference-code-free benchmark and evaluation suite for image-to-code reconstruction. It contains 2,169 test cases and a 539-example “mini” subset drawn from 15 source datasets across 6 domains: Charts & Plots, Geometry, Graphs, Scientific Imagery, Documents, and 3D Spatial Scenes. The task is to output a self-contained Python program using NumPy/Matplotlib that saves a figure to a predefined path; programs execute in a sandbox with fixed DPI, Agg backend, and a 30 s timeout. Failures such as syntax errors or no-output receive 0.0 immediately. Successful renders are then compared to the source image by a VLM-based rater equipped with dataset-specific rubrics and deterministic guardrails (Periasami et al., 11 May 2026).

The evaluation protocol uses four weighted rubric categories per dataset, with weights summing to 1 and two categories designated critical. The raw score is

$r_{\mathrm{raw}}=\sum_{i=1}^4 w_i s_i,$

where each $s_i\in[0,5]$ . Deterministic caps suppress inflated scores under severe semantic failures. If any critical category is $\le 1.5$ , the final score is capped at 2.8; blank or unreadable renders are capped at 0.5. On the full 2,169-example test, Qwen3.5-9B achieved 73.1% render success and 1.60/5.0, Kimi-K2.5+Reasoning achieved 90.0% and 3.23/5.0, and GPT-5.4 Mini+Reasoning achieved 98.8% and 4.06/5.0. Performance was strongly domain-dependent: leading models scored >4.5/5.0 on chart- and graph-like visuals but only 2.42/5.0 at best on SpatialVLM-QA, while chemistry, circuits, and documents remained comparatively difficult. Human validation on 539 test-mini pairs rated by six STEM annotators found Pearson/Spearman correlations of 0.723/0.703, exceeding a generic rubric (0.626/0.633) and embedding-cosine baselines (0.407/0.498 and 0.311/0.457) (Periasami et al., 11 May 2026).

The same benchmark also supports rater-filtered self-training. Given a source image $k$ 0, the model generates code $k$ 1 producing render $k$ 2 with score $k$ 3, then re-encodes $k$ 4 to generate $k$ 5, render $k$ 6, and score $k$ 7. Examples are retained when

$k$ 8

with threshold $k$ 9. Fine-tuning Qwen3.5-9B on 1,271 such pairs, with 141 dev examples, improved its test-mini score from 1.60 to 1.86 and render success from 75.1% to 78.5% (Periasami et al., 11 May 2026).

A second benchmark, named Code-Vision, evaluates multimodal models on generating code from flowcharts rather than from natural images. It consists of three subsets: HumanEval-V with 164 Python functions, Algorithm with 149 LeetCode-style problems divided into Easy (45), Medium (75), and Hard (29), and MATH with 125 problems divided into Easy (45), Medium (40), and Hard (40). Flowchart complexity rises with difficulty: for Algorithm, average nodes increase from 12.62 to 18.86, and average edges from 13.29 to 19.14. Correctness is execution-based against private test sets, and generation uses zero-shot prompting with temperature = 0.2, top_p = 0.95, and max length = 1 024 tokens (Wang et al., 17 Feb 2025).

On this benchmark, twelve MLLMs were evaluated. GPT-4o achieved 93.9 on HumanEval-V, 87.9 on Algorithm Overall, 92.0 on MATH Overall, and 91.3 average. On Algorithm Hard, GPT-4o reached 79.3% pass@1, while the best open-source models were far lower; the paper highlights that the best open-source result is 15.0% on MATH Hard, but 0% on Algorithm Hard for all listed open-source models. A text-only ablation reduced GPT-4o from 87.9% to 24.8% on Algorithm Overall, whereas MMCode fell only from 17.0% to 14.8%, which the paper uses to argue that Code-Vision forces multimodal reasoning (Wang et al., 17 Feb 2025).

A related reinterpretation appears in SVRepair, which treats CodeVision flowchart-to-code instances as automated program repair: the “bug” is the absence of code, the visual artifact is the bug report, and the “patch” is the complete implementation. SVRepair converts the flowchart image into a Semantic Scene Graph serialized in Mermaid syntax, then uses a coding agent issuing SEARCH, READ, and EDIT operations. Evaluation is in Pass@1. In this setting, SVRepair achieves 95.12% on CodeVision, compared with 92.07 for GPT-4o, 82.30 for Claude 3.5 Sonnet, 84.75 for Claude 4.0, and 88.41 for Qwen3-VL-235B (Tang et al., 5 Feb 2026).

4. Detecting LLM-generated code with visual probability maps

In a separate line of work, CodeVision names a detector for distinguishing human-written from LLM-generated code by combining token log-probabilities with 2D visual structure. The pipeline first tokenizes code with the cl100k_base tokenizer, then queries an LLM for each token’s conditional log-probability under its preceding context, using temperature=0 and top-p=1. These values are arranged in an $k$ 0 matrix aligned to the original code’s lines and columns, with unused cells filled by a placeholder such as $k$ 1. The resulting matrix is treated as a single-channel image and classified by a lightweight Vision Transformer or ResNet (Xu et al., 6 Jan 2025).

The central representational claim is that code layout matters. By laying out log-probabilities in two dimensions, the method preserves indentation, brackets, and line breaks rather than flattening code to a sequence. The paper formalizes the representation as $k$ 2, and then feeds $k$ 3 either directly into a modified 56-layer ResNet or into a ViT with patch embedding, $k$ 4, $k$ 5 encoder layers, 4 heads, and MLP hidden size 256. Training uses binary cross-entropy with Adam, learning rate $k$ 6, and early stopping on validation AUC (Xu et al., 6 Jan 2025).

The evaluation dataset, derived from Project CodeNet, contains 28,342 samples across 6 languages: C, C++, Go, Java, Python, Ruby. Of these, 15,633 are LLM-generated by GPT-3.5 and GPT-4, and 12,709 are human-written. Across all six languages, ResNet and ViT achieve AUCs in [0.94, 0.99], with FPR/FNR ≈10–15%. For Python specifically, the reported example is ResNet AUC=0.98, FPR=0.09, FNR=0.07; by comparison, DetectGPT achieves AUC=0.97 and GPTZero AUC=0.39. On code from seven code-specialized LLMs, CodeVision still leads with AUCs ≥0.81 (Xu et al., 6 Jan 2025).

Robustness and efficiency are treated as first-class concerns. The paper studies code mixing (10/30/50% human vs. LLM tokens), code translation, dead-code insertion, identifier renaming, print injection, and try/catch wrapping. The worst degradation comes from code translation, with $k$ 7AUC = -0.25 and $k$ 8FPR/FNR up to +0.20; other attacks cause minor to moderate drops with $k$ 9AUC \le -0.11). Runtime on an NVIDIA A100 is dominated by API calls: 13.01 s per sample for log-prob extraction, versus 6.4e–5 s for ResNet inference and 6.8e–4 s for ViT inference. The paper notes limitations as dependence on an external LLM API and reduced interpretability relative to rule-based detectors (Xu et al., 6 Jan 2025).

5. Unified multimodal code generation by merging vision and coding models

Under the name VisCodex, and explicitly glossed as “CodeVision” in the supplied summary, a further research direction merges a strong vision-LLM with a specialized code LLM by task-vector model merging. Let $N=20$ 0 denote a shared base checkpoint, $N=20$ 1 a vision-LLM, and $N=20$ 2 a code model. Their task vectors are

$N=20$ 3

and the merged language-model backbone is

$N=20$ 4

At 8 B, the best setting is reported as $N=20$ 5; at 33 B, $N=20$ 6. Only the LLM backbone is merged; the ViT with 2D-RoPE vision encoder and projection MLP remain frozen (Jiang et al., 13 Aug 2025).

Training uses the Multimodal Coding Dataset (McD) with 598 K samples from four domains: 200 K enhanced HTML pairs, 210 K chart image-to-Python pairs, 59 K image-augmented StackOverflow QA pairs, and 129 K algorithmic code samples. The evaluation benchmark InfiBench-V contains 322 questions, manually selected from a much larger StackOverflow-derived pool, spanning 13 languages and 5 categories: Front-End (100), Back-End (75), DS & ML (95), IT Ops (11), and Mobile/Desktop (41). Evaluation combines Keyword Matching, Unit Testing, and a GPT-4o Judge, normalized to [0–100] and averaged (Jiang et al., 13 Aug 2025).

The reported results place VisCodex near proprietary systems. VisCodex-8B obtains 90.1 / 90.9 on Design2Code, 74.8 / 74.1 on ChartMimic, 11.0 on MMCode, 72.1 on InfiBench-V, and 68.8 average. VisCodex-33B reaches 90.5 / 91.1, 79.3 / 78.5, 15.6, 78.6, and 72.3 average, close to GPT-4o at 73.3 average. Ablations are central to the paper’s interpretation: without model merging, VisCodex-8B’s MMCode drops from 11.0 to 6.8, and ChartMimic Low-L from 74.8 to 73.4. Training on McD rather than prior Web2Code data raises Design2Code Block-Match from 84.4 to 89.6 and Color from 79.6 to 86.8 (Jiang et al., 13 Aug 2025).

A plausible implication is that, within this branch of the literature, CodeVision denotes a strategy for importing coding competence into visually grounded models without full retraining, using parameter arithmetic plus multimodal supervised fine-tuning.

6. Programming vision as code-as-tool reasoning

Another use of the label appears in “Thinking with Programming Vision: Towards a Unified View for Thinking with Images.” Here CodeVision is a code-as-tool framework in which a multimodal model emits executable Python rather than selecting from a fixed registry of named tools. The generated code may call PIL, OpenCV, or custom libraries; execution occurs in a sandbox, and outputs—either transformed images or error tracebacks—are fed back into the model. The claimed benefit is that any library function becomes a potential tool without retraining the model or redesigning prompt templates (Guo et al., 3 Dec 2025).

The inference loop is multi-turn. Inputs consist of a raw image encoded by a frozen visual encoder plus a tokenized prompt containing the question and prior tool outputs. Outputs may mix > ..., <code>...</code>, and <answer>...</answer>. When a code block is emitted, the sandbox executes it; if successful, the resulting image is serialized and appended to context, and if unsuccessful, the error log is appended instead. This allows revision and recovery rather than one-shot execution. The training procedure has two stages: Supervised Fine-Tuning (SFT) on approximately 5 000 examples and Reinforcement Learning (RL) on approximately 40 000 examples (Guo et al., 3 Dec 2025).

The RL setup uses a dense process reward,

$N=20$ 7

where outcome reward includes answer correctness and proper tag formatting, strategy reward credits required or beneficial tool use, and penalties apply to excessive turns, poor crops, or inappropriate orientation fixes. For suggested tools, the paper computes a necessity term by contrasting successful rollouts that do and do not use the optional tool over $N=20$ 8 rollouts. Benchmarks include an Orientation Robustness Benchmark based on OCRBench and ChartQAPro, single-tool benchmarks V*, HRBench4k, HRBench8k, and the multi-tool benchmark MVToolBench (Guo et al., 3 Dec 2025).

The reported gains are large. On rotated/flipped OCRBench, Qwen2.5-VL-7B base averages 56.0%, while CodeVision-7B reaches 73.4% (+17.4). Qwen3-VL-8B Thinking improves from 52.2% to 75.4%, and Qwen3-VL-32B Thinking from 55.7% to 79.5%. On MVToolBench, CodeVision-7B, CodeVision-8B, and CodeVision-32B achieve 60.1, 62.7, and 65.4, all marked as SOTA in the supplied summary. Ablations show MVToolBench dropping from 60.1 to 50.7 without strategy reward and to 55.9 without constraint penalties; RL from scratch fails to converge, so SFT is reported as essential. The same summary attributes emergent capabilities to the framework, including flexible tool composition, chained execution, and runtime error recovery (Guo et al., 3 Dec 2025).

7. Visual representations of source code for understanding and security

A broader CodeVision lineage is visible in work that treats source code itself as an image. CV4Code encodes raw snippets as 2D arrays of ASCII codepoints. Printable ASCII plus a special blank token defines a vocabulary of size 96. After filtering, each character is mapped to an index, lines are padded to the maximum width, and the result is a 2D integer array $N=20$ 9, optionally normalized to grayscale. Snippets are then cropped and padded to fixed size, typically $p=0.0578$ 0. Because the representation uses raw text rather than ASTs, the method is language-agnostic and can featurize syntactically incorrect code (Shi et al., 2022).

CV4Code evaluates several vision backbones, including a ResNet, ViT, and a Compact Convolutional Transformer (CCT). Training on CodeNetBench uses 237 distinct problem IDs with 171 300 train, 21 000 validation, and 21 000 test samples, alongside a similarity benchmark with 100 problems × 20 snippets each. Optimization uses AdamW, initial learning rate 1e-3, weight decay 1e-4, a 5-epoch linear warmup, cosine annealing over 100 epochs, batch size 256, and a single NVIDIA V100 GPU. On classification, CCT-L achieves 97.64 Top-1 and 98.99 Top-5 in the multilingual setting, and 97.13 / 98.79 in Java-only evaluation. On retrieval, CCT-L reaches mAP@R = 0.991, exceeding ResNet at 0.983 and a-Transformer at 0.980 (Shi et al., 2022).

Code2Image approaches the same problem from the AST side. It parses source code into a full AST, draws each node as a colored rectangle, draws 1-pixel-thick black lines for parent-child edges, and compacts the resulting RGB image by trimming redundant white rows and columns. The CNN used for vulnerability prediction has two convolutional blocks with 32 filters, ReLU, BatchNorm, Dropout, and MaxPool2D, followed by a Dense layer with 32 units and a Softmax output (2 units). Experiments use the Draper VDISC dataset, originally about 1 M functions, with roughly 50 000 C functions parsed via Pycparser. Five vulnerability classes are binarized as vulnerable versus safe, with oversampling to a 25% positive ratio in training (Bilgin, 2021).

The reported comparison against Ast2Vec shows stronger performance for Code2Image in four of five vulnerability categories. For CWE-119, Code2Image achieves F1/MCC/AUC = 0.668 / 0.648 / 0.554 versus 0.527 / 0.496 / 0.424 for Ast2Vec; for CWE-120, 0.528 / 0.463 / 0.477 versus 0.424 / 0.357 / 0.385; for CWE-469, 0.140 / 0.166 / 0.054 versus 0.082 / 0.115 / 0.031; for CWE-476, Code2Image and Ast2Vec are close at 0.617 / 0.613 / 0.524 and 0.599 / 0.622 / 0.535; and for CWE-others, 0.335 / 0.282 / 0.277 versus 0.283 / 0.224 / 0.186. Inference speed is reported as approximately 20.8 ms/sample on GPU (Bilgin, 2021).

Taken together, CV4Code and Code2Image illustrate two distinct visualizations of program artifacts: raw-source rasterization and AST-layout rendering. This suggests that, in one important sense, CodeVision research is not limited to multimodal LLMs; it also includes the direct recasting of program analysis as an image-processing problem.