Intent-Based Chart Generation from Documents

• Intent-based chart generation refers to automated visualization synthesis that extracts and selects relevant evidence from lengthy documents based on communicative goals.
• The approach decomposes the process into stages such as evidence extraction, intermediate representation design, chart planning, and executable code generation to ensure data fidelity.
• Recent studies demonstrate improved chart accuracy and clarity through multi-stage pipelines, validated by benchmarks and mixed-initiative user studies for publication-ready outputs.

Intent-based chart generation from documents is the task of generating a chart that is both grounded in one or more documents and aligned with a user-specified or document-implied communicative goal. Unlike text-to-chart or table-to-chart settings, it requires an upstream phase that locates relevant evidence in long documents, determines what subset of that evidence should be visualized, and only then selects or synthesizes a chart whose data, type, layout, and sometimes narrative framing match the intended message. Recent work treats this as a compound problem spanning document understanding, intermediate representation design, chart planning, code generation, and verification, while adjacent work on chart reconstruction, chart editing, retrieval, and publication-aware rendering supplies reusable components without solving the full document-to-chart problem by itself (Jain et al., 20 Jul 2025, Ghosh et al., 26 Jul 2025, Zhang et al., 2024, Nguyen et al., 2024, Kondic et al., 31 May 2025, Yan et al., 13 Jan 2025).

1. Scope and relation to adjacent tasks

The most direct formulations define the problem at document scale rather than at table scale. Doc2Chart introduces the task of intent-based chart generation from documents: given a document $D$ and a user-specified intent $I$, generate a chart $C$ that adheres to the intent and is grounded in the document. Infogen defines a related but more specialized task, text-to-complex statistical infographic generation, in which the input is a text-heavy document and the output is a multi-subchart infographic rather than a single chart. ChartifyText starts from a user-selected statement plus its surrounding context and generates a chart that visually communicates the underlying data and claim. HoneyPlotNet, although motivated by cyber deception, is technically a document-conditioned chart generation system in which local document text is used to generate captions, chart text, chart type, and chart data before rendering (Jain et al., 20 Jul 2025, Ghosh et al., 26 Jul 2025, Zhang et al., 2024, Nguyen et al., 2024).

A common misconception is that neighboring tasks are equivalent to document-centric generation. They are not. ChartGen studies chart-to-code reconstruction from chart images, not document-to-chart generation. ChartEditor starts from an uploaded basic chart image and supports intent-driven pictorial transformation. ChartGPT generates charts from abstract natural language over tables rather than long documents. ChartGalaxy concentrates on infographic chart understanding, code generation, and example-based infographic generation once tabular data or a reference chart is already available. Kori, ChartText, InkSight, and WYTIWYR focus on chart-text linking, documentation, sketch-based finding authoring, or retrieval rather than end-to-end document-grounded chart synthesis (Kondic et al., 31 May 2025, Yan et al., 13 Jan 2025, Tian et al., 2023, Li et al., 24 May 2025, Latif et al., 2021, Pinheiro et al., 2022, Lin et al., 2023, Xiao et al., 2023).

This division of labor matters because intent-based chart generation from documents is not merely a rendering problem. The system must recover document evidence, infer a communicative target such as comparison, trend, composition, or ranking, and then choose a representation that is both data-faithful and rhetorically appropriate. Several papers make this point from different directions: Doc2Chart argues that long documents contain significantly more content than what appears in the final visualization, ChartifyText emphasizes ambiguity and intrinsic sparsity of data in text documents, and chart-plot argues that even when a model can already produce runnable plotting code, publication readiness remains a distinct “last mile” problem (Jain et al., 20 Jul 2025, Zhang et al., 2024, Tang et al., 8 Jun 2026).

2. Intent representations and canonical intermediate forms

A consistent pattern in the literature is the use of an intermediate representation between document semantics and final rendering. In Doc2Chart, the first stage produces a structured JSON object with values, x_axis_label, y_axis_label, and title; the chart data is described in the paper both as $(x,y,\text{category})$ tuples and as structured tuples $$\langle x\text{-axis}, y\text{-axis}, \text{value}\rangle$$. In Infogen, the full problem is formalized as $C = g(f(T))$, where $T$ is the input text, $M=f(T)$ is infographic metadata, and $g(M)$ generates executable code. The metadata includes infographic-level title and summary and sub-chart-level fields such as kind, axis, stats, text, position, alignment, background, dimensions, fonts, and sub-chart summary (Jain et al., 20 Jul 2025, Ghosh et al., 26 Jul 2025).

Other work uses different canonical forms. CycleChart makes chart schema the common interface across tasks and defines a renderer $$R : \mathcal{S} \times \mathcal{T} \rightarrow \mathcal{I}$$, where schema $I$0 mediates between data tables, images, and question answering. ChartGen argues that executable plotting code is a stringent chart-understanding target: given a chart image $I$1 originally generated by plotting code $I$2, the model must generate a script $I$3 that reconstructs the chart faithfully. HoneyPlotNet uses a caption as an intermediate semantic pivot, with the pipeline $I$4. ChartEditor introduces a hierarchical chart tree that organizes marks, axes, annotations, grouping relations, and legend links for downstream intent-driven editing (Deng et al., 22 Dec 2025, Kondic et al., 31 May 2025, Nguyen et al., 2024, Yan et al., 13 Jan 2025).

These representations encode different notions of intent. Executable code captures chart type, library choice, axes, labels, legends, colors, and render-time parameters. Schema captures marks, channels, transforms, and faceting. Metadata captures not only chart content but also layout and cross-chart organization. A plausible implication is that document-to-chart systems need more than one intermediate layer: a data-grounding layer such as CSV or structured tuples, a chart-structure layer such as schema or plotting code, and, for infographic or publication settings, a layout/style layer such as metadata or a style skill (Kondic et al., 31 May 2025, Deng et al., 22 Dec 2025, Ghosh et al., 26 Jul 2025, Tang et al., 8 Jun 2026).

3. End-to-end methodologies

Doc2Chart exemplifies the modular document-to-chart pipeline. It is an unsupervised, zero-shot, multi-stage framework with three stages: iterative data extraction and refinement, chart type prediction, and code generation. Extraction is intent-aware and produces JSON; a validation module decides whether to accept the result, refine it, or request re-extraction with feedback; a heuristic-guided chart-type module then recommends bar, line, grouped bar, stacked bar, or pie under explicit rules such as “Time-based: $I$5 points $I$6 Bar; 4+ points $I$7 Line” and “Intent: Magnitude $I$8 Bar; Trend $I$9 Line; Composition $C$0 Pie/Stacked Bar”; the final stage synthesizes executable Matplotlib code (Jain et al., 20 Jul 2025).

ChartifyText also uses staged decomposition, but starts from a selected statement and its surrounding context rather than a user intent string plus a long document. Its tabular data inference module performs topic generation, table schema creation, table data population, and missing-data/value inference. It then applies expressive chart generation, recommending one of bar, line, pie, or scatter and augmenting the result with explicit encodings for uncertainty, ranges, missing values, and sentiment. The paper’s key engineering choice is to populate tables by first extracting quoted evidence from context and only then converting those quotes into numeric values, which is intended to reduce hallucination (Zhang et al., 2024).

HoneyPlotNet follows a different route. For each chart instance it extracts the nearest $C$1 sentences around the figure reference; in the reported setup it uses 16 sentences on both sides of the first chart mention, with a 10% probability of random shortening or lengthening by one sentence. A multitask Transformer first generates a caption from document context, then chart text elements from the caption, and also maps the caption to a discrete sequence containing chart type and data tokens. Those tokens are decoded by a hierarchical VQ-VAE/VQGAN-style Plot Data Model into continuous chart data, scale parameters, and chart shape before rendering (Nguyen et al., 2024).

Infogen targets multi-subchart statistical infographics and adds an explicit planning layer. Stage 1 fine-tunes LLMs to generate infographic metadata, aligns them using DPO with synthetic preferences, and uses a fine-tuned ranker to select the best metadata candidate. Stage 2 converts metadata into Python code with Plotly and Plotnine, and a second LLM acts as a judge that checks subchart count, chart types, axes, statistics, positions, fonts, dimensions, backgrounds, and layout, iterating up to five times (Ghosh et al., 26 Jul 2025).

Chart-plot addresses a later stage in the pipeline: publication. It assumes a data table and intent string are already available, then adds a style-aware code generator conditioned on a textual style skill, a deployment-aware render loop that compiles the chart inside the target LaTeX context, and a structured edit layer that exposes each chart element through stable identifiers and directly manipulable handles. This work does not solve document semantic understanding, but it makes the target document template part of the chart-generation environment itself (Tang et al., 8 Jun 2026).

4. Datasets and benchmark resources

The field is now supported by several datasets and procedural generation corpora spanning direct document-to-chart tasks, chart reconstruction, infographic synthesis, and chart-structure perception.

Resource	Scope	Scale
Doc2Chart benchmark	$C$2intent, document, chart$C$3 tuples	1,242 tuples and 2.2K charts
Infodat	text-to-infographic metadata	3,463 data points
HoneyPlotNet dataset	document-chart pairs	5,418 pairs
ChartGen	chart image-code supervision	222.5K unique chart-image code pairs
ChartGalaxy	infographic chart corpus	million-scale
CycleChart-Bench	aligned generation/parsing/QA benchmark	6,507 charts
ChartSS	chart semantic segmentation	59,693 images

Doc2Chart’s benchmark is built from SEC 10-K filings and scientific papers, with source documents chosen for table density and annotated with creative intents, chart images, supporting tables, source page numbers, and chart descriptions. Infodat is derived from complex Pew infographics and pairs synthesized document-style input text with manually refined infographic metadata; it reports an average of 2.15 sub-charts per metadata sample and allows up to 21 sub-charts. HoneyPlotNet’s 5,418 document-chart pairs come from chart data linked to PubMedCentral Open Access documents and cover five chart types: line, scatter, vertical bar, horizontal bar, and box chart (Jain et al., 20 Jul 2025, Ghosh et al., 26 Jul 2025, Nguyen et al., 2024).

Adjacent resources greatly expand supervision for intermediate problems. ChartGen scales 13K seed charts into 222.5K unique chart image-code pairs, spanning 27 chart types, 11 plotting libraries, and aligned modalities including image, code, text, CSV, and DocTags. ChartGalaxy contributes a million-scale infographic corpus with 75 chart types, 330 chart variations, and 68 layout templates, plus a code benchmark and an example-based generation setting. CycleChart-Bench aligns raw tables, natural-language intent queries, Vega-Lite schemas, rendered images, visualization-level tables, and QA supervision over 6,507 charts, including 1,135 faceted charts. ChartSS supplies 59,693 annotated bar, line, and pie charts for chart semantic segmentation (Kondic et al., 31 May 2025, Li et al., 24 May 2025, Deng et al., 22 Dec 2025, Yan et al., 13 Jan 2025).

These resources imply a layered data ecosystem rather than a single benchmark. Direct document-to-chart corpora remain comparatively small and domain-specific, whereas synthetic or semi-synthetic corpora for chart code, infographic variations, and schema-aligned learning are already large enough to support representation learning at scale. This suggests that practical systems may continue to rely on transfer from adjacent tasks until large, diverse, fully document-grounded chart-generation datasets become commonplace.

5. Evaluation practices and empirical results

Evaluation in this area is heterogeneous because different systems solve different subproblems. Doc2Chart introduces ChartEval, an attribution-based metric over structured chart representations, and reports that it correlates with human judgments at Pearson’s $C$4, compared with $C$5 for a simpler LLM-based metric. On its benchmark, the method improves over the best baselines by up to 9 points in chart data accuracy and 17 points in chart type; for example, with Gemini-2.0 it reaches 71.53 chart data accuracy versus 62.51 for the best baseline, and with Claude-3.5-Sonnet it reaches 82.01 chart type best versus 64.80 (Jain et al., 20 Jul 2025).

ChartifyText evaluates the document-to-chart claim through comprehension and workload. In its user study, participants answered questions significantly faster with generated charts than with text alone, with $C$6 for ChartifyText and $C$7 for the text baseline, at $C$8, while answer scores were similar ($C$9 versus $(x,y,\text{category})$0, $(x,y,\text{category})$1). Expert ratings were also high for relevance, data accuracy, chart clarity, and visualization-guideline compliance (Zhang et al., 2024).

Representation-centric evaluation remains stringent. ChartGen shows that generating executable plotting code is substantially easier than faithfully recovering the underlying chart semantics. On its held-out chart-to-code benchmark, Granite-Vision-3.1 obtains 72.5% execution rate, 0.58 data score, 5.83 semantic score, and 7.48 image similarity, while InternVL-2.5 reaches the highest execution rate at 86.0% but lower data and semantic fidelity. This exposes a core issue for document-to-chart systems as well: runnable code does not guarantee correct data mappings, correct scaling, or correct style semantics (Kondic et al., 31 May 2025).

Infographic-oriented systems report different ceilings. On ChartGalaxy’s “Direct Mimic” benchmark for D3.js HTML generation, Gemini-2.5-Pro achieves overall 85.21 and Llama-4-Maverick-17B achieves 61.29. In ChartGalaxy’s example-based generation study, experts rate the template-based method at 4.63 fidelity versus 2.10 for GPT-Image-1, with similarly higher aesthetics and creativity scores. Infogen reports 74.69 subchart accuracy, 84.23 subchart type accuracy, and 89.56 statistical accuracy for its large model, alongside human ratings of 4.1 for readability and 4.1 for data accuracy and alignment (Li et al., 24 May 2025, Ghosh et al., 26 Jul 2025).

The evaluation literature also shows that intent capture and authoring interaction matter. ChartEditor reports Mask2Former ResNet-101 at 78.90 mIoU on ChartSS and shows that GLIGEN + Decomposition outperforms plain GLIGEN with LPIPS 0.35 versus 0.55, style accuracy 85% versus 65%, and user rating 4.5 versus 3.2. Kori and ChartText evaluate text-chart grounding rather than generation; ChartText reports F1 scores of 0.50 and 0.66 on two datasets, rising to 0.68 and 0.84 with semi-automatic correction. Talk Me Through It shows that spoken imagined-chart instructions are structurally different from typed existing-chart descriptions and that a system trained on spoken imagined-chart data outperforms a system trained on typed existing-chart data on both voice and text input (Yan et al., 13 Jan 2025, Pinheiro et al., 2022, Ponochevnyi et al., 21 Jan 2026).

6. Limitations, controversies, and open directions

The main unresolved issue is that document grounding remains harder than chart rendering. Doc2Chart explicitly notes that vague or underspecified intents remain difficult and that multiple long documents are challenging under context-window limits. Infogen relies on a semi-automated, Pew-derived dataset and does not independently benchmark code executability. ChartifyText depends on GPT-4 prompt orchestration, lacks a formal intent model, and supports only four chart types. HoneyPlotNet is largely confined to scientific and biomedical documents from PubMedCentral and covers only five chart types. ChartGalaxy focuses primarily on single-chart infographics and does not address multi-chart storytelling. ChartGen inherits representational biases from the underlying VLM and code models and does not address narrative discourse or chart-type selection from communicative goals (Jain et al., 20 Jul 2025, Ghosh et al., 26 Jul 2025, Zhang et al., 2024, Nguyen et al., 2024, Li et al., 24 May 2025, Kondic et al., 31 May 2025).

Another persistent controversy is evaluation. Image similarity, code similarity, and LLM-judge scoring all miss some aspect of chart faithfulness. Doc2Chart argues that visual-decoding metrics often fail to capture chart data effectively and therefore uses attribution over structured chart tuples. ChartGen relies partly on GPT-4o-based judgments of data, semantics, and image similarity. CycleChart uses schema-centric multitask consistency rather than a single end-to-end chart-generation objective. This suggests that future evaluation will likely need to remain multi-view, combining source attribution, schema correctness, execution validity, and human judgments of appropriateness (Jain et al., 20 Jul 2025, Kondic et al., 31 May 2025, Deng et al., 22 Dec 2025).

A further lesson is that intent capture is rarely solved by one-shot prompting. Kori models chart-text references as selections, unions, and intersections over chart elements. InkSight lets users express intent through open-path and closed-path sketches before fact recommendation. ChartEditor translates vague natural-language intent into chart-tree edits, and Talk Me Through It shows that realistic imagined-chart prompts are longer, more iterative, and more conversational than chart-description datasets suggest. This suggests that robust document-to-chart systems will need mixed-initiative interfaces rather than purely autonomous generation, especially when document evidence is incomplete or multiple chart realizations are equally defensible (Latif et al., 2021, Lin et al., 2023, Yan et al., 13 Jan 2025, Ponochevnyi et al., 21 Jan 2026).

Finally, publication context is becoming part of the problem definition. Chart-plot argues that the open problem is no longer only chart code generation but chart publication: style conformity, layout validity in a target template, and precise author edits after initial generation. A plausible implication is that mature document-to-chart systems will need three layers at once: document-grounded extraction and planning, executable chart synthesis, and deployment-aware adaptation to the medium in which the chart will actually appear (Tang et al., 8 Jun 2026).

In sum, intent-based chart generation from documents has moved from a loosely defined aspiration to a layered research area with direct task formulations, structured intermediates, new benchmarks, and several reusable adjacent technologies. Its central challenge remains unchanged: the system must infer not only the data to plot, but also the communicative reason for plotting it, and must preserve that reason through extraction, schema construction, code generation, and evaluation.