CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolution

Abstract: Chart-to-code generation demands strict visual precision and syntactic correctness from Vision-LLMs (VLMs). However, existing approaches are fundamentally constrained by data-centric limitations: despite the availability of growing chart-to-code datasets, simply scaling homogeneous chart-code pairs conflates visual perception with program logic, preventing models from fully leveraging the richness of multimodal supervision. We present CharTide, a novel data-centric framework that systematically redesigns both training and alignment data for chart-to-code generation. First, we construct a 2M-sample dataset via a Tri-Perspective Tuning strategy, explicitly decoupling training into visual perception, pure-text code logic, and modality fusion streams, enabling a 7B model to surpass specialized baselines using only supervised data. Second, we reformulate alignment as a data verification problem rather than a heuristic scoring task. To this end, we introduce an Inquiry-Driven RL framework grounded in the principle of information invariance: a downstream model should yield consistent answers to identical visual queries across both original and generated charts. Moving beyond rigid rule matching or VLM scoring, we employ a frozen Inspector to objectively verify generated charts through atomic QA tasks, providing verifiable reward signals based on answer accuracy. Experiments on ChartMimic, Plot2Code, and ChartX show that CharTide-7B/8B significantly outperforms open-source baselines, surpasses GPT-4o, and is competitive with GPT-5.

• The paper introduces a tri-perspective supervised fine-tuning and inquiry-driven reinforcement learning framework that decouples visual, logical, and fusion streams for chart-to-code generation.
• It achieves state-of-the-art benchmarks by integrating objective fact verification and high-fidelity reward signals to ensure both semantic and visual alignment.
• The methodology demonstrates scalability and robustness across diverse chart types, outperforming larger models with its refined, data-centric approach.

CharTide: A Data-Centric Framework for High-Precision Chart-to-Code Generation

Overview and Problem Formulation

Chart-to-code generation is a core challenge for vision-LLMs (VLMs) due to strict requirements for pixel-wise visual fidelity and syntactic validity in the produced plotting code. Existing supervised fine-tuning (SFT) regimes suffer from diminishing returns ("scaling wall"), primarily due to information asymmetry and homogeneity in chart-code supervision. Previously, reinforcement learning (RL) alignment leveraged either heuristic rule-matching or black-box VLM-based scores, leading to instability and lack of objective progress.

This paper introduces "CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolution" (2604.22192). CharTide delivers a systematic redesign of both pre-training and RL data regimes, employing Tri-Perspective SFT for capability disentanglement and a novel Inquiry-Driven RL framework with verifiable, fact-based rewards. The approach achieves state-of-the-art (SOTA) results among open-source models and is competitive with proprietary closed models.

Figure 1: High-level overview of CharTide, with Tri-Perspective SFT decoupling capability streams and Inquiry-Driven RL leveraging objective, atomic evaluation.

Tri-Perspective Decomposed SFT

Motivation and Construction

The information asymmetry in end-to-end chart-to-code training leads to over-representation of non-visual tokens, templated syntactic patterns, and under-supervision of attribute values that must be visually inferred. To address the bottleneck, Tri-Perspective SFT decomposes supervision into orthogonal data streams:

• Visual Perception (Chart → Caption): Dense visual features are mapped to concise textual descriptions, ensuring grounding of visual perception.
• Code Logic (Caption → Code): Pure logic synthesis is learned without visual interference by generating plotting code solely from captions.
• Modality Fusion (Chart → Code): End-to-end mapping is enhanced via pairing regenerated code strictly with its rendered output, verified via structural similarity.

Each stream undergoes rigorous filtering (e.g., pre-trained LLM/VLM verification, WebSSL-1B structural similarity scoring) to maximize the orthogonality and quality of supervision. The result is a 2M-sample SFT dataset that, when used to train 7B/8B models, enables outperforming much larger (235B) prior baselines across metrics.

Figure 2: Schematic detail of the CharTide pipeline, illustrating the decoupled SFT streams and the reward-aligned RL verification loop.

Inquiry-Driven RL: From Subjective Scoring to Objective Verification

Verifiable Reward Design

Classical reward strategies (rule-based, VLM judge) provide noisy or rigid signals, impeding optimization for fine-grained and holistic chart reproduction. CharTide proposes Inquiry-Driven RL, reframing the RL alignment problem as one of explicit fact verification—grounded in the principle of information invariance: correct code will yield consistent answers to atomic visual queries, whether evaluated on the source or generated chart.

• Chart-VQA Dataset: ChartCap-based clustering and GPT-5 QA generation produce a diverse Query/Answer bank targeting chart type, layout, text elements, trends, and style, with tolerance annotations for data queries.
• Frozen Inspector: Reward signals are generated by checking whether the trained Inspector model can answer these QA pairs on generated charts, with rewards $r_{QA}$ for semantic consistency and $r_{vis}$ (WebSSL-1B cosine similarity) for visual fidelity. The final total reward combines both.

This approach fully decouples generator and evaluator and transforms evaluation into low-variance, deterministic, atomic QA signals, preventing reward hacking and instability.

Figure 3: Training dynamics in RL, with synchronized rise in train reward, pass rate, and consistency reward, indicating effective, stable optimization.

Experimental Results and Analysis

Benchmark Results

CharTide-7B/8B models outperform open-source and closed-source competitors on ChartMimic, Plot2Code, and ChartX. Specific highlights include:

• ChartMimic High-Level Score: 91.6 (CharTide-7B), surpassing MSRL-7B and GPT-4o, closing the gap with GPT-5.
• Strong gains on Plot2Code: Especially in visual similarity and text match, indicating enhanced semantic alignment.
• Ablation studies: Show the necessity of each SFT stream and the critical nature of Inquiry-Driven RL even when applied to external model checkpoints.
• OOD generalization: Demonstrated via strong performance on more challenging "Customized Mimic" and Seaborn sets.

Robustness and Reward Hacking Prevention

Metric tracking shows that, after normalizing by code execution rate, both semantic and visual rewards continue to improve during RL. This demonstrates that the model is not exploiting simple reward artifacts but is truly optimizing for higher-fidelity, executable outputs.

Figure 4: Reward hacking analysis, showing continuous improvement in normalized visual and semantic reward.

Visual Evaluation

Qualitative evaluation against challenging chart types reveals that CharTide produces structurally faithful, visually accurate, and semantically robust code outputs. The model excels on topologically complex scenarios (e.g., vector fields, radar plots) where prior baselines frequently collapse or omit critical style/annotation details.

Figure 5: Qualitative comparison on challenging chart types. CharTide maintains high-fidelity structure and attribute alignment with ground truth, outperforming prior models.

Data Contamination Checks

Worst-case and randomly sampled similarity analyses between test and training sets (using dual encoders) confirm no direct or indirect leakage. Retrieved neighbors may share style but differ in data and content, indicating CharTide’s generalization is not due to memorization.

Figure 6: Worst-case leakage check. Retrieved training images share layout or style but not data content—refuting direct data contamination.

Figure 7: Average-case leakage check. Random test queries retrieve semantically and visually distinct training images.

Reward Strategy and Visual Encoder Choices

Systematic ablation shows that:

• WebSSL-1B outperforms DINOv2 and SigLIP as a visual encoder for chart evaluation.
• Inquiry-Driven reward ($R_{inq}$) surpasses VLM-Judge-based reward both as a single and combined signal.
• The reward framework generalizes well for diverse base models and is not tailored for a particular pre-trained backbone.

  Figure 8: Qualitative comparison of WebSSL-1B’s visual similarity judgments: high correspondence with human-perceived alignment, outperforming standard visual encoders.

Implications and Future Directions

The results demonstrate that performance plateaus in chart-to-code synthesis are caused not merely by data insufficiency but by supervision homogeneity and reward subjectivity. CharTide’s approach—decoupling capability acquisition and grounding reward alignment in objective, atomic verification—enables SOTA performance without extreme model scaling. This suggests that for structured code generation from vision, data and evaluation regime design may be more critical than model size.

Future research directions include extending CharTide to larger backbone regimes, adapting the inquiry-driven reward strategy for chart editing, style transfer, or cross-library migration tasks, and exploring fully interactive multi-turn workflows.

(Figures 9–11)

Figures 9–11: Extensive qualitative visualizations on ChartMimic benchmark, further confirming style and semantic robustness of CharTide outputs.

Conclusion

CharTide sets a new standard for chart-to-code generation in open-weight VLMs by systematically redesigning both training and alignment supervision with Tri-Perspective SFT and Inquiry-Driven RL. Empirical and analytic results confirm substantial gains in visual fidelity and syntactic validity, stringent mitigation of memorization, and stable reward optimization. This framework substantiates the hypothesis that data-centric, verifiable alignment is essential for scalable structured code synthesis in multimodal models (2604.22192).