RealChart2Code: Authentic Chart-to-Code Benchmark

• RealChart2Code is a chart-to-code benchmark that converts real-world chart images and data into executable visualization code with high fidelity.
• It systematically evaluates multi-turn refinement, chart replication, and reproduction tasks over diverse datasets and complex visualization types.
• The benchmark employs sandboxed Python environments and multi-agent LLM judging to ensure both syntactic correctness and semantic accuracy in generated code.

Searching arXiv for papers on RealChart2Code and chart-to-code generation. arxiv_search(query="RealChart2Code chart-to-code generation benchmark multi-task evaluation", max_results=10, sort_by="relevance") RealChart2Code is a chart-to-code benchmark and problem setting centered on generating executable visualization code from real-world charts rather than from simplified or synthetic proxies. In its benchmark formulation, chart-to-code generation is posed as $C=\mathcal{F}(V,P)$, where $V$ is a chart image, $P$ is a prompt, and $C$ is the generated code; the benchmark contains 2,896 instances grounded in authentic datasets and is described as the first benchmark to systematically evaluate chart generation from large-scale raw data and to assess iterative code refinement in a multi-turn conversational setting (Zhang et al., 26 Mar 2026). The broader chart-to-code literature frames code as a lossless representation of chart structure and appearance, in contrast to textual descriptions that omit dense visual and structural detail (Zhao et al., 11 Jan 2025).

1. Definition and conceptual foundations

RealChart2Code addresses a version of chart-to-code generation in which the target charts are tied to authentic analytical workflows, complex plot structures, and real data transformations rather than to small, stylized benchmark figures (Zhang et al., 26 Mar 2026). The benchmark was introduced because existing evaluations such as Plot2Code and ChartMimic had become too simple to measure whether modern vision-LLMs can reconstruct multi-panel scientific and business visualizations from realistic inputs. It therefore emphasizes authentic datasets, complex multi-panel visualizations, and multi-turn refinement, and organizes visualization tasks around seven high-level intents: Correlation, Deviation, Ranking, Distribution, Composition, Change, and Groups (Zhang et al., 26 Mar 2026).

A central premise shared with the wider chart-to-code literature is that executable plotting code is more faithful than chart captioning or chart description. ChartCoder states that interpreting charts with textual descriptions often leads to information loss, whereas parsing charts into code provides lossless representations that can contain all critical details (Zhao et al., 11 Jan 2025). ChartReasoner adopts the same premise in a different form, describing chart code as an executable specification intended to preserve layout and data semantics as losslessly as possible (Jia et al., 11 Jun 2025). This establishes the conceptual basis for RealChart2Code: the task is not merely to describe a chart, but to recover a programmatic artifact that can be executed, edited, and reused.

RealChart2Code also differs from earlier chart-to-code settings by treating analytic intent as part of the task rather than as incidental context. The benchmark’s charts are designed to reflect meaningful analytic questions over real datasets, so success depends on recovering not only visual form but also the data logic, transformations, and subplot organization that motivated the original visualization (Zhang et al., 26 Mar 2026).

2. Benchmark construction and task hierarchy

The benchmark is built from real datasets collected from Kaggle. The construction process begins with more than 8,000 candidate datasets, more than 100,000 data files, and roughly 30 billion rows, and after automatic filtering and expert curation yields 1,036 high-quality datasets, 3,271 raw data files, and about 860 million rows (Zhang et al., 26 Mar 2026). From these sources, the benchmark designers create 1,016 unique complex visualizations that span 50 chart types, 8 high-level domains, and 35 sub-topics, and then derive three task families from them (Zhang et al., 26 Mar 2026).

Task	Instances	Core input condition
Chart Replication	1,016	Chart image and prompt
Chart Reproduction	1,016	Chart image, prompt, and data_info for real files
Chart Refinement	864	Flawed chart/code context and natural-language correction instruction

Chart Replication asks for image-to-code generation without raw data access. The model sees a reference chart and a prompt instructing it to generate executable Matplotlib or Seaborn code that reproduces the figure as precisely as possible; synthetic data are therefore permissible so long as the rendered chart matches the reference (Zhang et al., 26 Mar 2026). Chart Reproduction uses the same target visualization but supplements the prompt with data_info describing real CSV or XLSX files, including filenames, shapes, columns, types, and sample rows, and requires the model to load and transform the actual data rather than fabricate a visually similar dataset (Zhang et al., 26 Mar 2026). Chart Refinement presents a chart containing injected errors together with a natural-language instruction specifying what to fix, and asks the model to generate corrected code while preserving everything else that remains valid (Zhang et al., 26 Mar 2026).

Ground-truth code for the 1,016 reference visualizations was written by an in-house team of five expert Python developers and checked for executability and reproducibility (Zhang et al., 26 Mar 2026). Refinement instances are then created by systematically injecting errors into a subset of these scripts and re-rendering the flawed outputs. The injected failures include wrong chart types, incorrect data mappings, layout and overlap problems, and style or semantic errors such as misaligned labels or incorrect axis ranges (Zhang et al., 26 Mar 2026). This design gives the benchmark a dual role: it evaluates both first-pass chart generation and the ability to revise code in response to visual defects.

3. Evaluation methodology

RealChart2Code evaluates models in a sandboxed Docker environment with Python 3.13, pre-installed scientific libraries, no network access, 128 CPU cores, 1 TB RAM, and a 120-second timeout per run (Zhang et al., 26 Mar 2026). Functional correctness is measured first. If the generated code fails with an exception or timeout, the instance receives zero for downstream visual metrics. Pass Rate is defined as

$\text{exec\_rate}=\frac{s}{t},$

where $s$ is the number of scripts that execute successfully and $t$ is the total number of tasks (Zhang et al., 26 Mar 2026).

For successful executions, RealChart2Code uses a multi-agent LLM judge ensemble with 0/1/2 scoring to measure visual and semantic fidelity (Zhang et al., 26 Mar 2026). Visual Structure Alignment comprises eight criteria: Chart Type Consistency, Spatial Layout Consistency, Text Element Consistency, Axis Configuration Consistency, Color Scheme Consistency, Style and Format Consistency, Component Completeness, and either Data Pattern Consistency or Data Alignment depending on task type (Zhang et al., 26 Mar 2026). The distinction between Data Pattern Consistency and Data Alignment is crucial. In Replication and Refinement, the judge assesses whether trends, peaks, distributions, and relative structures visually match. In Reproduction, the benchmark instead inspects whether the generated code loads the same files, uses equivalent columns, applies equivalent filtering and aggregation logic, and passes equivalent arrays into the plotting routines, while allowing syntactic variants such as $df['col']$, $df.col$, and $df.loc[:, 'col']$ to count as equivalent (Zhang et al., 26 Mar 2026).

Execution Quality is evaluated separately through Visual Clarity, Compositional Balance, and Typographic Quality (Zhang et al., 26 Mar 2026). These three measures score whether the generated figure is readable and usable even apart from reference matching. A chart can therefore be executable yet still lose points because legends occlude data, subplot proportions are distorted, or text is too small to read.

The benchmark also validates the reliability of this automatic evaluation. Reported inter-agent and human-alignment results are strong: the average Fleiss’ $V$0 across agents is 0.824 and the average Cohen’s $V$1 between the ensemble and human experts is 0.83 (Zhang et al., 26 Mar 2026). The metric-level values are especially high for Type and Layout, with Fleiss’ $V$2 of 0.921 and 0.998 and Cohen’s $V$3 of 0.91 and 0.99, respectively (Zhang et al., 26 Mar 2026). This matters because RealChart2Code relies on fine-grained, semantics-aware judging rather than on pixel-only similarity.

4. Empirical findings and the complexity gap

RealChart2Code evaluates 14 vision-LLMs, including proprietary systems such as Claude-4.5-Sonnet, Claude-4.5-Opus, GPT-5.1, Gemini-2.5-Flash, and Gemini-3-Pro-Preview, and open-weight systems such as Qwen3-VL-235B, Qwen3-VL-30B, Intern-VL-3.5-241B, Intern-VL-3.5-30B, DeepSeek-VL-7B, MiMo-VL-7B-RL, GLM-4.5V-106B, and GLM-4.1V-9B (Zhang et al., 26 Mar 2026). The results establish a large gap between simpler chart-to-code benchmarks and the realistic setting represented by RealChart2Code.

Among proprietary models, Claude-4.5-Opus achieves the strongest overall average, with Pass Rates of 87.7%, 86.1%, and 91.2% and Scores of 7.8, 7.4, and 9.4 on Replication, Reproduction, and Refinement, respectively, for an average score of 8.2 (Zhang et al., 26 Mar 2026). Gemini-3-Pro-Preview is close overall and achieves the highest Replication score, 9.0, with a Replication Pass Rate of 82.1% (Zhang et al., 26 Mar 2026). By contrast, the best open-weight model, Qwen3-VL-235B, records Pass Rates of 49.2%, 38.2%, and 65.1% and Scores of 3.3, 2.4, and 5.1, for an average of 3.6 (Zhang et al., 26 Mar 2026). The benchmark therefore reports a performance gap of more than 2× between top proprietary and best open-weight models (Zhang et al., 26 Mar 2026).

A common misconception is that strong performance on ChartMimic or Plot2Code implies readiness for real chart reverse engineering. RealChart2Code directly contradicts that assumption: the benchmark reports substantial performance degradation relative to those simpler evaluations, and its “Complexity Gap” analysis shows all model points far below the $V$4 line when comparing simpler-benchmark performance to RealChart2Code performance (Zhang et al., 26 Mar 2026). The interpretation is not that current models cannot generate chart code at all, but that they remain brittle when real data, multi-panel layouts, and iterative repair are introduced simultaneously.

The benchmark’s error analysis separates failure modes by model class. Open-weight models are dominated by syntax and execution failures as well as severe layout errors, including invalid APIs, undefined variables, and collapsed subplot arrangements (Zhang et al., 26 Mar 2026). Proprietary models execute more reliably but often fail in data reasoning, for example by loading the wrong columns or performing the wrong aggregation while still producing plausible-looking charts (Zhang et al., 26 Mar 2026). Refinement tasks surface an additional issue: regressive editing, in which a model fixes the requested problem but damages other previously correct parts of the chart (Zhang et al., 26 Mar 2026). These findings make RealChart2Code valuable not only as a leaderboard benchmark but also as a diagnostic tool for separating code-robustness failures from data-semantic and layout-compositional failures.

5. Methods and system designs associated with RealChart2Code

Although RealChart2Code itself is a benchmark rather than a single model, several contemporary chart-to-code systems are explicitly framed as blueprints for a RealChart2Code-style pipeline. C2C-MoLA, described as essentially the core of a “RealChart2Code” system, formulates chart-to-code generation as a mapping from a standardized chart image to a token sequence representing executable Python visualization code and imposes both syntactic correctness and semantic fidelity via render-based IoU constraints (Wang et al., 28 Nov 2025). Its architecture combines a multimodal backbone, a Mixture-of-Experts module with complexity-aware routing, and LoRA-based parameter-efficient adaptation. On Chart2Code-160k, it reports improvements of up to 17% in generation accuracy, an 18% reduction in peak GPU memory, and 20% faster convergence relative to standard fine-tuning and LoRA-only baselines, with the largest gains on complex charts (Wang et al., 28 Nov 2025). This directly addresses the heterogeneous chart complexity that RealChart2Code exposes.

A second line of work treats chart-to-code alignment as a preference-learning problem. Chart2Code introduces iterative dual preference learning, in which model outputs and structured code variants are compared using both a code-side heuristic F1 signal and a visual-side multi-dimensional binary score, and then optimized with DPO (Zhang et al., 3 Apr 2025). The benchmarked dual scoring strategy reaches 99.8% correct winner selection on the feedback evaluation set, while retaining 85.7% of candidate pairs after filtering inconsistent cases (Zhang et al., 3 Apr 2025). This is relevant to RealChart2Code because it offers a mechanism for improving out-of-distribution chart-to-code performance without relying only on token-level supervision.

A third response to RealChart2Code-like difficulty is to train on more realistic data and use visually grounded reinforcement learning. ChartMaster constructs ReChartPrompt-240K by using real-world, human-designed charts from arXiv papers as prompts for code generation and then applies ChartSimRL, a GRPO-based reinforcement learning method with attribute similarity and visual similarity rewards (Tan et al., 25 Aug 2025). On ChartMimic, ChartMaster-7B reports Execution Rate 93.8, Low-Level 78.2, and High-Level 77.3, substantially improving over the base Qwen2.5-VL-7B (Tan et al., 25 Aug 2025). CharTide goes further by redesigning both training and alignment data: it builds a 2M-sample tri-perspective supervised dataset and then applies Inquiry-Driven RL using an Inspector model and a WebSSL-based visual reward, reporting ChartMimic High-Level scores of 91.6 for CharTide-7B and 92.7 for CharTide-8B (Zheng et al., 24 Apr 2026). These methods suggest that the difficulty surfaced by RealChart2Code is not purely architectural; it is also a data and reward-design problem.

Editing-oriented work is especially pertinent to RealChart2Code’s refinement tasks. ChartEditor introduces ChartEditVista, a benchmark of 7,964 samples spanning 31 chart categories, and trains a 3B model with a rendering reward that jointly enforces code executability and visual fidelity (Chen et al., 19 Nov 2025). The framework also demonstrates that the same rendering reward improves pure chart-to-code performance on ChartMimic Direct Mimic, indicating that refinement and reproduction are technically coupled rather than separate capabilities (Chen et al., 19 Nov 2025).

6. Limitations and future directions

RealChart2Code is deliberately realistic, but its own design choices impose constraints. The benchmark is Matplotlib-centric, its evaluation depends on an LLM-based judge even though human correlation is strong, its data come from Kaggle and therefore reflect Kaggle’s domain distribution, and it focuses on static images rather than interactive visualizations (Zhang et al., 26 Mar 2026). These are not minor issues. A Matplotlib-centered benchmark rewards imperative chart construction patterns; interactive or declarative ecosystems such as Vega-Lite, Plotly, or ECharts are outside its direct scope. A plausible implication is that benchmark success does not by itself establish cross-library chart-to-code competence.

The broader literature points to additional open problems. Several of the most effective training pipelines remain synthetic-first and still acknowledge a gap to noisy, real-world charts with OCR difficulties, unusual layouts, or PDF artifacts (Wang et al., 28 Nov 2025). ChartMaster explicitly notes that its real-chart dataset is still biased toward scientific and technical arXiv figures and does not directly address low-resolution scans, watermarks, or business-dashboard conventions (Tan et al., 25 Aug 2025). RealChart2Code’s Reproduction task partly closes this gap by introducing authentic datasets and large raw tables, but it does not yet solve library diversity or multimodal document noise.

At the same time, future directions are visible. ChartReasoner treats chart code as a modality bridge for downstream reasoning, using chart-to-code as the first stage in a code-driven reasoning pipeline (Jia et al., 11 Jun 2025). Aligned multi-language work extends the problem beyond Python: Chart2NCode provides 176K charts paired with aligned Python, R, and LaTeX scripts, and CharLuMA uses a language-conditioned mixture of low-rank subspaces to share chart understanding while specializing generation to the target language (Zhang et al., 27 Apr 2026). This suggests that a mature RealChart2Code ecosystem may need to move beyond “image to Matplotlib” toward multi-language, tool-augmented, and reasoning-aware systems in which executable chart code functions as both reconstruction target and intermediate representation.

RealChart2Code therefore occupies a specific position in the chart-understanding literature. It is neither a generic chart captioning benchmark nor merely a harder version of ChartMimic. It is a benchmark for executable, data-grounded, multi-panel, refinement-aware chart reverse engineering, and it exposes the gap between current chart-to-code competence and the demands of real visualization workflows (Zhang et al., 26 Mar 2026).