ChartSync: Benchmark for Visuo-Logical Chart Editing

• ChartSync is a benchmark for dependency-aware chart editing that rigorously evaluates how textual/numerical changes cascade to update corresponding geometric elements.
• It employs a structured pipeline with instruction proposing, code generation, and expert validation to build a dataset of 870 validated triplets with high retention and agreement rates.
• The evaluation framework combines objective metrics like OCR F1 and SSIM with VLM-based semantic scores to assess geometric synchronization and detect errors such as textual manipulation and logical inconsistencies.

ChartSync is a term used in contemporary chart research in two closely related senses. In its most specific sense, it denotes the benchmark introduced in "ChartSync: A Benchmark for Visuo-Logical Cascading Chart Editing", which formalizes instruction-guided chart editing as Visuo-Logical Cascading Editing (VLCE) and evaluates whether edited chart images preserve consistency between textual values and dependent geometry (Yu et al., 11 Jul 2026). In adjacent work, the term also appears as a broader label for systems that keep charts synchronized with video scenes, code, data tables, natural-language descriptions, or narration, treating synchronization as a cross-modal constraint rather than a purely graphical operation (He et al., 16 Jun 2025, Liu et al., 2024, Pinheiro et al., 2022).

1. Visuo-logical cascading editing

ChartSync’s core problem is not ordinary image editing but dependency-aware chart editing. A valid edit must update the requested textual or numerical content, propagate that change into all dependent geometric/chart elements, and leave unrelated content unchanged. The canonical motivating example is a pie-slice percentage edited from “20%” to “40%”: a text-only editor can repaint the label, whereas a VLCE-capable editor must also enlarge the corresponding slice and adjust the remaining slices so that the chart remains coherent (Yu et al., 11 Jul 2026).

The benchmark formalizes a chart as a visuo-logical graph

$C = (V, G, S),$

where $V = \{v_i\}$ are textual or numerical values, $G = \{g_j\}$ are geometric primitives, and $S$ is the dependency mapping from values to geometry. The visuo-logical constraint is

$\Delta G = S(\Delta V).$

An editing model $\mathcal{M}$ receives an original chart image $I_{ori}$ and an instruction $T_{inst}$ and produces

$I_{pred} = \mathcal{M}(I_{ori}, T_{inst}).$

A valid output must satisfy three conditions: textual realization of the requested $\Delta V$, geometric synchronization consistent with $V = \{v_i\}$0, and non-target preservation of unrelated chart elements (Yu et al., 11 Jul 2026).

This definition distinguishes ChartSync from benchmarks centered on localized text substitution. The benchmark targets cases in which a single value change induces a structured cascade through bars, slices, polygons, nodes, edges, or multi-panel dependencies. The authors characterize current failures as a deficit in geometric synchronization and dependency-aware cascading updates, not merely in rendering quality (Yu et al., 11 Jul 2026).

2. Benchmark construction and dataset composition

ChartSync is built through a programmatic rendering pipeline that guarantees deterministic visuo-logical coupling in the ground truth. The construction process has three phases. First, an Instruction Proposer analyzes source code and the original chart image to identify key, old_value, and new_value, while allowing edits that intentionally create logical contradictions if geometry is not updated. An Instruction Evaluator then filters proposals for localization, format, and usefulness, but does not reject visuo-logically contradictory edits. Second, a Code Generator edits the original plotting code with minimal but causally sufficient changes, and a Logic Validator checks instruction alignment, propagation discipline, logical consistency, and syntax validity before execution and re-rendering. Third, expert quality assurance validates the resulting triplets (Yu et al., 11 Jul 2026).

The expert protocol involved 935 candidate triplets, with 258 triplets flagged, a 27.6% correction rate, and 95.4% average pairwise agreement among 3 PhD-level AI researchers. After corrections, instruction refinement, code adjustment, re-rendering, and stratified sampling, the final dataset contains 870 expert-validated triplets, with a 93.0% retention rate (Yu et al., 11 Jul 2026).

The benchmark spans 9 chart categories and 4 task types. Its geometry-coupled core is the VLCE subset of 235 instances, consisting of 179 single VLCE and 56 multiple VLCE cases (Yu et al., 11 Jul 2026).

Benchmark stratum	Count
Bar	300
PIP	100
Box	100
Multidiff	80
Errorpoint	70
Graph	70
Radar	70
Errorbar	40
Pie	40
Single Text Edit	592
Multiple Text Edit	43
Single VLCE	179
Multiple VLCE	56

The chart families were selected to expose varied value-to-geometry mappings. Bar and pie charts provide the clearest scalar-to-extent or scalar-to-angle couplings. Radar, Graph, Errorbar, Errorpoint, Multidiff, and PIP introduce multi-entity, multi-panel, or structurally coupled edits in which propagation is more complex than a single localized deformation. This suggests that ChartSync is intended as a diagnostic benchmark for structural chart editing rather than a narrow test of OCR or label replacement (Yu et al., 11 Jul 2026).

3. Evaluation framework

ChartSync evaluates predictions with a two-tier framework combining objective visual metrics and a VLM-as-a-judge paradigm. The objective layer contains OCR F$V = \{v_i\}$1 and SSIM. OCR F$V = \{v_i\}$2 is computed after extracting text sequences from the predicted and ground-truth images with a GPT-5.4-based OCR extractor, normalizing them, and using longest common subsequence statistics to derive character-level precision and recall. SSIM measures structural similarity between prediction and ground truth, capturing luminance, contrast, and structural fidelity, but not higher-level semantics (Yu et al., 11 Jul 2026).

The semantic layer uses Gemini-3.1-Pro as the judge and produces three scores. TESR evaluates textual edit success, including both content correctness and spatial correctness of edited text, on a scale in $V = \{v_i\}$3. VLCS evaluates visuo-logical consistency on the VLCE subset only, on a scale in $V = \{v_i\}$4, and applies a strict no-change penalty: if geometry remains pixel-identical to the source after a value edit, VLCS must be 0.0. BFS evaluates background fidelity for non-target regions on a scale in $V = \{v_i\}$5 (Yu et al., 11 Jul 2026).

Aggregation is performed in two steps. VLM Avg is the macro-average of TESR, VLCS, and BFS. Overall Score is the equal average of the five metric means: OCR F$V = \{v_i\}$6, SSIM, TESR, VLCS, and BFS. The evaluation protocol was also validated against human expert ratings; the paper reports ICC = 0.87 and Overall Score $V = \{v_i\}$7 with humans, with similarly high correlation for VLCS (Yu et al., 11 Jul 2026).

The design of VLCS is particularly significant. It does not reward approximate visual plausibility unless the required geometric change is directionally and structurally correct relative to both the source and the ground truth. ChartSync therefore measures whether a model has implicitly reconstructed a value-to-geometry mapping, not merely whether it can draw a visually similar chart.

4. Empirical findings and failure paradigms

ChartSync evaluates 14 image editing models and one code-mediated pipeline. The code-mediated system first reconstructs plotting code from the input chart with GPT-5.5, edits that code according to the instruction, and re-renders the chart. Among proprietary image editors, Nano Banana Pro and GPT-Image-2 are the strongest performers. Among open-source models, Qwen-Image-Edit-2511 is the strongest baseline, but it still shows a severe deficit on geometric synchronization (Yu et al., 11 Jul 2026).

System	Overall	VLCS	TESR	BFS
Code-Mediated Pipeline	60.58	28.51	93.38	37.44
Nano Banana Pro	87.76	83.71	96.25	89.60
GPT-Image-2	81.78	74.47	88.90	80.29
Qwen-Image-Edit-2511	60.29	13.83	61.81	58.88

The central empirical pattern is the capability gap between text editing and geometric synchronization. Many models achieve reasonable TESR but collapse on VLCS, indicating that they can localize and rewrite text while failing to update the geometry that encodes the edited values. The code-mediated pipeline illustrates a different failure mode: it achieves OCR F$V = \{v_i\}$8 88.76 and TESR 93.38, yet only VLCS 28.51, BFS 37.44, and SSIM 54.83, reflecting information loss in image-to-code reconstruction and imperfect re-rendered fidelity (Yu et al., 11 Jul 2026).

The paper identifies three error categories. Textual Manipulation Failure (TMF) covers wrong, missing, illegible, truncated, or hallucinated target text. Visuo-Logical Inconsistency (VLI) covers cases where text changes but geometry is unchanged or changed in the wrong direction. Background Corruption (BC) covers collateral damage to axes, gridlines, unrelated marks, or global layout. For Qwen-Image-Edit-2511, TMF accounts for approximately 58.1% of errors and VLI for approximately 31.1%. For Nano Banana Pro, textual manipulation is largely resolved, and the remaining errors are dominated by BC ≈ 41.7% and VLI ≈ 38.1% (Yu et al., 11 Jul 2026).

From these findings, the benchmark isolates three hierarchical meta-abilities: foundation perception, synchronization and reasoning, and high-fidelity generation. The first concerns fine-grained semantic grounding and robust text rendering; the second concerns value-to-geometry reasoning and dependency propagation; the third concerns semantic isolation and artifact-free editing. The benchmark’s conclusion is therefore not simply that current models underperform, but that structured chart editing requires a composite capability that most generative image editors do not yet possess (Yu et al., 11 Jul 2026).

5. Broader meanings of ChartSync in adjacent research

Outside the benchmark paper, ChartSync is also used as a broader design idea for cross-modal synchronization. In video authoring, ChartBlender is described as essentially a “ChartSync” system for video: it lets users create charts, embed them as 3D objects into video scenes, and automatically synchronize them with camera motion and moving objects using Metric3D, Open3D RGBD odometry, BootsTAPIR, and temporal smoothing. The system was evaluated through two controlled experiments and expert interviews with five domain experts, and experts reported that tasks taking hours or days were reduced to “two or three minutes” (He et al., 16 Jun 2025).

In chart understanding, the term has been used for systems that keep multiple chart representations synchronized. SynChart frames ChartSync as keeping chart image, data table, plotting code, descriptions, and QA in step. Its dataset contains approximately 3.93M chart images, 3.93M CSV tables, 3.93M Python scripts, 7.86M text descriptions, 45.8M simple QA pairs, and 13.9M complex QA pairs, and its 4.2B chart-expert model reaches 84.60 ChartQA accuracy (Liu et al., 2024). ChartVerse pushes the same programmatic perspective further, introducing Rollout Posterior Entropy (RPE) for chart complexity, truth-anchored inverse QA synthesis, ChartVerse-SFT-600K, ChartVerse-RL-40K, and an 8B model that reaches 64.1 average across seven chart benchmarks (Liu et al., 20 Jan 2026).

Synchronization has also been formulated across plotting languages. Chart2NCode provides 176K aligned chart quadruples spanning Python, R, and LaTeX, and CharLuMA uses a language-conditioned mixture of low-rank subspaces to share chart understanding while specializing code generation to the target language. On the Chart2NCode test set, CharLuMA-6.7B reaches ER 98.0, DS 88.7, MJ 88.1, F1 83.5 for Python, ER 96.5, DS 81.8, MJ 80.9 for R, and ER 89.0, DS 72.5, MJ 74.2 for LaTeX (Zhang et al., 27 Apr 2026).

Related work also uses ChartSync in document and narrative settings. ChartText treats ChartSync as keeping charts and narration synchronized by linking phrases in text or speech to chart elements in two stages, matching and grouping; it reports F1 scores of 0.50 and 0.66 on two datasets, increasing to 0.68 and 0.84 in a semi-automatic correction setting, and demonstrates real-time voice-driven overlays for chart presentations (Pinheiro et al., 2022). ChartifyText uses a two-stage pipeline—Tabular Data Inference and Expressive Chart Generation—to synchronize charts with data-involved prose, explicitly encoding uncertainty, ranges, inferred values, and sentiment; in a user study with 15 participants, mean answer time dropped from 139.36s to 73.62s with no significant accuracy difference (Zhang et al., 2024). OpusAnimation extends the synchronization problem into time-varying media through Dynamic Chart Generation, using instruction-code-video triplets, QA-based code and video evaluation, and a Joint Code-Visual Reward for GRPO in Qwen2.5-VL-DCG-3B (Li et al., 2 Oct 2025).

Taken together, these lines of work suggest that ChartSync is not limited to one benchmark. It names a broader research agenda in which charts are treated as structured objects whose image form, code form, tabular form, textual description, temporal behavior, and narrative alignment should remain mutually consistent.

6. Technical substrates for ChartSync systems

A broader ChartSync stack depends on reliable structural extraction from existing charts. CHARTER provides an end-to-end chart-to-table pipeline combining a Faster R-CNN page detector, a CenterNet-based element detector with auxiliary domain-specific heatmaps, OCR, and a rule-based analysis module. It supports bar, pie, line, and scatter charts, trains its Stage-2 detector solely on synthetic data, and reports Stage-1 [email protected] of 98.0% for bar charts and 97.8% for pie charts, along with strong end-to-end performance on ICPR2020 bar and pie benchmarks (Shtok et al., 2021).

OneChart reformulates structural extraction as chart-to-dict generation and introduces a single auxiliary token <Chart> with an auxiliary number decoder. The hidden state of <Chart> is optimized with an L1 regression objective over normalized numeric values, and the resulting self-consistency score

$V = \{v_i\}$9

acts as a confidence signal. Despite using only 0.2B parameters, OneChart substantially outperforms prior chart parsing systems on multiple structural-extraction benchmarks and improves LLaVA-1.6 on ChartQA from 56.0 to 67.2 when used as a chart parsing agent with the figure input (Chen et al., 2024).

For line-chart extraction, ChartZero argues that synthetic priors are sufficient for zero-shot generalization if the system learns geometry rather than semantics. It trains exclusively on 100,000 synthetic line charts, introduces the Global Orthogonal Instance (GOI) loss, uses a VLM-guided legend matching strategy, and reports IoU = 0.82, NRMSE = 0.028, legend F1 0.945, axis accuracy 0.962, and ChartRM 0.9210 on its benchmark (Islam et al., 7 May 2026). This suggests that any future ChartSync editor operating directly on chart images will benefit from strong parsing modules that expose explicit structure before editing or reasoning begins.

Chart understanding on the language side supplies another substrate. ChartAssistant characterizes chart-to-table pre-training as the alignment stage that makes chart and structured text tightly coupled, and reports strong zero-shot transfer on real-world charts after multitask instruction tuning (Meng et al., 2024). ChartAdapter, in turn, specializes chart summarization with a cross-modal projector, learnable latent textual embeddings, and a three-stage hierarchical training procedure, reaching BLEU-4 35.55, ROUGE-1 41.49, ROUGE-2 15.75, and ROUGE-L 25.79 on Chart2Text PEW (Xu et al., 2024). A plausible implication is that ChartSync, in the broad sense, requires both structural extraction and semantically aligned language generation.

7. Limitations and outlook

The ChartSync benchmark paper identifies five explicit limitations. First, evaluation relies on a VLM as a judge, even though the protocol was validated against human ratings. Second, the benchmark focuses on standard statistical charts rather than more heterogeneous document graphics. Third, the dataset scale is still modest at 870 triplets and 235 VLCE cases. Fourth, the detailed error taxonomy was conducted on only two representative models rather than all evaluated systems. Fifth, the training data of proprietary models cannot be audited, so prior exposure to similar chart editing tasks cannot be excluded (Yu et al., 11 Jul 2026).

The benchmark nevertheless points toward clear architectural directions. The authors explicitly propose integrating explicit data structures and metrics into multimodal models, pursuing hybrid symbolic-neural systems, augmenting diffusion-based editors with explicit spatial reasoning modules, and combining chart understanding tasks such as chart-to-code and QA with editing tasks so that models internalize data-geometry mappings (Yu et al., 11 Jul 2026). This suggests that future ChartSync systems are unlikely to emerge from generic image editing alone. They will more plausibly combine programmatic chart representations, structural parsing, cross-modal alignment, and controlled rendering so that textual, numerical, geometric, and temporal states remain synchronized across edits.