FigEdit: A Benchmark for Chart Editing

• FigEdit is a benchmark that formalizes charts as renderings of structured specifications rather than free-form images.
• It supports diverse edit tasks—including single, multi, conversational, visual-guidance, and style transfer—using executable Vega/Vega-Lite specs.
• The framework emphasizes that preserving data-encoding, axis coherence, and legend integrity requires structured transformation beyond pixel-level edits.

to=arxiv_search 微信天天中彩票 大发时时彩怎么 code to=arxiv_search.search code FigEdit is a large-scale benchmark for scientific figure editing introduced to evaluate a specific claim: a chart is not merely an arrangement of pixels but a visual representation of structured data governed by a graphical grammar. In this formulation, a chart image is the rendering of an underlying specification rather than a free-form bitmap, so editing a chart is treated as a structured transformation problem rather than a pixel-manipulation task. FigEdit comprises over 30,836 samples, covers 10 chart types, and is organized into five task families—single edits, multi edits, conversational edits, visual-guidance-based edits, and style transfer—using executable Vega/Vega-Lite specifications, rendered before–after pairs, and natural-language instructions (Li et al., 30 Nov 2025).

1. Formal model and scope

The benchmark formalizes a natural image as a mapping

$I : \mathbb{R}^2 \rightarrow \mathbb{R}^3$

from coordinates to RGB values, but formalizes a chart as

$I = R(\sigma),$

where the specification $\sigma$ lives in a structured space $\Sigma$ and is decomposed as

$\sigma = (C, S).$

Here, content $C$ includes the dataset $D$, chart type $\tau$, and encodings, while style $S$ includes palettes, fonts, strokes and fills, gridlines, legend layout, spacing, and margins (Li et al., 30 Nov 2025).

On this view, an atomic edit is a function on specifications,

$f_e : \Sigma \rightarrow \Sigma,$

so the edited output is produced as

$I = R(\sigma),$0

Examples stated for this framework include adding a bar for a category with a specified value, or increasing axis-label font size while leaving data and geometry unchanged. The formulation emphasizes invariants that are specific to charts: data–encoding alignment, axis coherence, legend integrity, and text correctness (Li et al., 30 Nov 2025).

A common misconception is that chart editing can be handled as ordinary image editing with stronger prompting. FigEdit is built precisely to challenge that assumption. The benchmark argues that state-of-the-art editors trained on natural images are optimized for perceptual realism and prompt alignment, not for preserving executable chart semantics such as correct bar heights, consistent legends, or stable axis ranges (Li et al., 30 Nov 2025).

2. Benchmark composition and construction

FigEdit covers 10 chart types: area, line, bar, stacked-bar, pie, donut, box, violin, scatter, and dot. The benchmark uses curated real-world datasets from sources including Kaggle, UCI, World Bank WDI, OECD PISA, US Census, BLS, BEA, WHO, FRED, and NOAA, with chart-class preference lists used to encourage coherent dataset–chart pairings (Li et al., 30 Nov 2025).

Base figures are produced through LLM-guided Vega spec proposal. The model is instructed to output a JSON object

$I = R(\sigma),$1

with a valid Vega v6 schema, inline "values" data, and field structures appropriate to the chart type. Invalid outputs are rejected and resampled (Li et al., 30 Nov 2025).

Construction includes several explicit validation layers. Schema and field-shape validation enforce chart-type requirements. Duplicate and near-duplicate control uses exact and structural signatures, including

$I = R(\sigma),$2

to reject repeated or trivially similar specifications. Semantic diversity is further enforced by token-overlap control over categorical terms using a sliding window and Jaccard-style overlap thresholding (Li et al., 30 Nov 2025).

The underlying edit generation process uses Vega/Vega-Lite as the editable representation. For each base chart, the system samples feasible operations, generates natural-language instructions with OP tags, applies deterministic edit functions to the specification, validates the edited spec, and renders the ground-truth image pair. This yields paired images together with executable before–after specifications rather than only image-space annotations (Li et al., 30 Nov 2025).

3. Task families and edit vocabulary

FigEdit defines five task families of increasing difficulty. In the single-edit task, the input is an original chart image and a natural-language instruction describing exactly one atomic edit. In the multi-edit task, the instruction describes two or more edits composed in a canonical order. In the conversational task, the system must apply sequential edits across turns while preserving prior modifications. In the style-transfer task, a target chart must preserve its content while matching a style attribute taken from a reference chart. In the visual-guidance task, a separate guidance image marks the target region with a thin red circle, and the edit must apply only there (Li et al., 30 Nov 2025).

The benchmark’s atomic operation set is

$I = R(\sigma),$3

These are grouped in the paper as data-centric operations, text operations, style editing, layout or spacing adjustment, and font operations (Li et al., 30 Nov 2025).

The benchmark also reports the operation distribution. In the single-edit split, add element has 1,941 images, remove element 1,892, add title 1,942, change background color 1,944, change data color 1,729, adjust category spacing 1,729, and font adjustment 2,943, for a total of 14,105 single edits. Multi edit includes 3,370 dual-operation instances and 2,660 triple-operation instances. Visual guidance includes 1,666 change-data-color cases and 1,819 remove-element cases. Style transfer includes 1,511 transfer-style examples, 1,728 change-data-color examples, and 387 category-spacing examples (Li et al., 30 Nov 2025).

Instructions are natural-language sentences followed by canonical OP tags. The parameterization includes, for example, new_bg=<css-or-#hex> for background edits, axis_label_size=<int 6..30> for font changes, and structured arguments such as target_category, target_series, position, category, and value for datapoint addition, recoloring, or removal. This pairing of free-form language with deterministic operation metadata is one of the benchmark’s defining features (Li et al., 30 Nov 2025).

4. Evaluation protocol and the metric-gap problem

FigEdit evaluates outputs in image space but treats semantic correctness as central. The benchmark uses SSIM, PSNR, LPIPS, CLIP similarity, OCR similarity, and an LLM-based instruction score. OCR similarity is defined as

$I = R(\sigma),$4

where OCR strings are extracted from the generated image and the ground truth (Li et al., 30 Nov 2025).

The LLM-based judge uses GPT-4o and produces score_instruction, score_preservation, score_quality, and an overall score, each on a 1–5 scale, together with JSON reasoning. The paper’s point is not merely that charts need more metrics, but that chart edits can be visually close to the source while remaining semantically invalid. High SSIM or PSNR can therefore coexist with failure to execute the requested edit (Li et al., 30 Nov 2025).

This metric-gap is a central contribution of the benchmark. The analysis explicitly states that classic metrics overestimate performance in chart editing because only small regions may change while the semantic meaning changes severely. A model can ignore the requested modification, misplace a bar, distort a legend, or alter an axis range and still achieve strong pixel-level scores. FigEdit therefore uses OCR and LLM-as-judge evaluation to expose failures that image-reconstruction metrics miss (Li et al., 30 Nov 2025).

A second misconception addressed by the benchmark is that preservation can be judged solely by closeness to the input. In chart editing, preservation is constrained by chart grammar. Unedited components should remain intact, but edited components must also remain semantically valid with respect to the underlying data and encodings. This makes “preservation” stricter than ordinary visual similarity (Li et al., 30 Nov 2025).

5. Baseline models and empirical findings

The benchmark evaluates four representative image-editing systems: GPT-Image, Imagen 4, OmniGen2, and InstructPix2Pix. For single and multi edits, the models receive the original chart and the instruction. For conversational edits, multi-turn prompting is used where supported. For visual guidance, the original chart and the red-circled guidance image are provided. For style transfer, a target chart and a reference chart are supplied in a two-panel prompt (Li et al., 30 Nov 2025).

The results support the benchmark’s thesis. On single edits, Imagen 4 attains the highest SSIM at 0.7726 and PSNR at 13.04, but has low semantic scores: Instr 1.58, Preserv 1.51, and Qual 2.05. GPT-Image has lower SSIM and PSNR, at 0.7295 and 10.32, but much stronger LLM scores: Instr 3.47, Preserv 1.71, and Qual 2.45. InstructPix2Pix and OmniGen2 are intermediate, with OmniGen2 reaching Instr 3.35, Preserv 2.55, and Qual 2.85 on single edits (Li et al., 30 Nov 2025).

The multi-edit setting is harder for all models. Imagen 4 falls to Instr 1.26 and Preserv 1.32, while GPT-Image reaches Instr 2.51 and Preserv 1.63; InstructPix2Pix and OmniGen2 remain weak as well. In conversational editing, GPT-Image is notably stronger on instruction following, with Instr 4.59, but preservation remains only moderate at 2.51. In visual guidance, GPT-Image obtains the best OCR score at 0.4665 and strong quality at 3.95, yet even there the paper reports frequent failures to limit the edit to the circled region. In style transfer, Imagen 4 again has the highest SSIM at 0.8500 and PSNR at 14.00, while GPT-Image leads on semantic measures with OCR 0.5092 and LLM scores Instr 3.06, Preserv 3.57, and Qual 4.16 (Li et al., 30 Nov 2025).

The per-instruction breakdown identifies data-centric edits as especially difficult. “Add a new data point” and “Remove an existing data point” are the hardest categories across systems. Reported failures include misplacing bars or points, changing the wrong element, distorting axes, or neglecting the requested removal entirely. This supports the broader conclusion that current editors treat charts as generic images rather than as structured visualizations (Li et al., 30 Nov 2025).

6. Position in the broader editing literature

FigEdit occupies a distinct place within the rapidly expanding literature on image editing. General benchmarks and editors such as ImgEdit introduce 1.2 million instruction-based edit pairs, a VLM-based editor, and a benchmark spanning basic, hard single-turn, and multi-turn settings, but are designed for open-domain image editing rather than chart-specific graphical grammar (Ye et al., 26 May 2025). Fine-grained editors such as PartEdit target object parts by learning special textual tokens that produce localization masks and then blending source and edited features during denoising (Cvejic et al., 6 Feb 2025). Bounding-box-guided editors such as FineEdit use multi-level bounding box injection, a 1.2 million-pair dataset, and a dedicated region-editing benchmark to improve localization and background preservation (Xu et al., 13 Apr 2026).

Latency-optimized localized editors address a different axis. FlashEdit explicitly decouples speed, structure, and semantics through One-Step Inversion-and-Editing, Background Shield, and Sparsified Spatial Cross-Attention, reporting under 0.2 seconds per edit and a 150.84$I = R(\sigma),$5 speedup over DDIM+Prompt-to-Prompt while emphasizing background consistency and localized changes (Wu et al., 26 Sep 2025). FireRed-Image-Edit further scales instruction-based editing through a 1.6B-sample corpus, balanced generation and editing data, preference optimization, online RL, and layout-aware OCR rewards for text editing (Team et al., 12 Feb 2026). Earlier optimization-based work such as Forgedit focused on per-image fine-tuning, vector projection in text-embedding space, and UNet forgetting strategies to control identity similarity and editing strength (Zhang et al., 2023).

FigEdit’s contribution is not that these methods are irrelevant to figures, but that their success criteria are still largely image-space criteria. The benchmark argues that scientific charts require structure-aware models that can reason over executable specifications, data tables, encodings, legends, and typography jointly. This suggests that future figure editing systems will need to combine multimodal editing with chart parsing, specification editing, or other grammar-aware representations rather than relying on pixel-level manipulation alone (Li et al., 30 Nov 2025).

7. Research implications and future directions

FigEdit is intended as a benchmark, but it also functions as a problem statement for structure-aware figure editing. The paper points toward models that parse a chart into data, marks, encodings, and style, apply edits at the specification level, and re-render the result. It also identifies open problems: robust parsing of arbitrary scientific charts into executable specifications, faithful edit application when the original specification is unavailable, evaluation protocols that compare specifications or data when possible, and extension to more complex multi-panel figures and additional chart types (Li et al., 30 Nov 2025).

This position has direct implications for “FigEdit”-style systems in the broader sense used by adjacent editing work. FlashEdit, FineEdit, PartEdit, and ImgEdit all emphasize localized edits, preservation outside the edit region, and interactive workflows, which are properties that remain relevant for plots, diagrams, and document figures (Wu et al., 26 Sep 2025, Xu et al., 13 Apr 2026, Cvejic et al., 6 Feb 2025, Ye et al., 26 May 2025). FigEdit sharpens that agenda by showing that localization and preservation, while necessary, are not sufficient for scientific charts unless the model also respects the underlying structured transformation defined by the graphical grammar (Li et al., 30 Nov 2025).

In that sense, FigEdit marks a shift in how figure editing is conceptualized. It treats scientific charts as rendered programs rather than as ordinary images, supplies a common benchmark grounded in real data and executable specifications, and demonstrates that contemporary image editors remain weak on the semantic constraints that matter in scientific workflows. Its central legacy is therefore methodological: chart editing should be evaluated, and likely modeled, as specification-aware transformation rather than as perceptually plausible image synthesis (Li et al., 30 Nov 2025).