Infogen: Structured Generation in Traffic & Charts

• Infogen is an ambiguous term denoting two distinct 2025 systems that generate structured outputs for autonomous driving and infographic synthesis.
• The autonomous driving system formulates long-horizon traffic simulation as autoregressive next-token group prediction over maps, signals, agent states, and motions.
• The document visualization system converts text into structured metadata and executable Python code to render complex, multi-chart statistical infographics.

Searching arXiv for the two 2025 "Infogen/InfGen" papers to ground the article and citations. Infogen denotes two distinct 2025 research systems that share a near-identical name but address different generative problems. In autonomous driving, “InfGen: Scenario Generation as Next Token Group Prediction” formulates long-horizon traffic simulation as autoregressive generation over grouped scene tokens representing maps, traffic lights, agent states, and motion, with support for evolving agent populations and infinite scene generation (Peng et al., 29 Jun 2025). In document visualization, “Infogen: Generating Complex Statistical Infographics from Documents” defines a text-to-infographic pipeline in which fine-tuned LLMs first generate structured metadata and then render multi-chart statistical infographics through code generation and iterative feedback (Ghosh et al., 26 Jul 2025). The shared label therefore does not designate a single unified framework; rather, it refers to two contemporaneous lines of work centered on structured generation under explicit intermediate representations.

1. Terminological scope and naming

The term “Infogen” is ambiguous in the 2025 arXiv literature. One work, styled “InfGen,” concerns realistic and interactive traffic simulation for autonomous driving, with the scenario represented as a sequence of tokens and generated autoregressively over time (Peng et al., 29 Jun 2025). The other, styled “Infogen,” concerns complex statistical infographic generation from text-heavy documents, with a two-stage pipeline from text to metadata to executable Python code (Ghosh et al., 26 Jul 2025).

This naming collision is not merely cosmetic. The two systems operate in different domains, optimize different objectives, and rely on different structural priors. The traffic system is centered on closed-loop scene evolution, grouped causal attention, and kinematic rollout. The infographic system is centered on metadata planning, LLM fine-tuning with QLoRA and DPO, ranker-based ensembling, and agentic code refinement. A plausible implication is that “Infogen” should be read contextually: in autonomous driving literature it refers to scenario generation; in document visualization literature it refers to infographic synthesis.

2. InfGen in autonomous driving: problem formulation and motivation

InfGen is a data-driven traffic scenario generator that treats the entire evolution of a driving scene as an autoregressive sequence of discrete tokens (Peng et al., 29 Jun 2025). Its motivation is the inadequacy of three prevalent paradigms in autonomous driving simulation: log-replay, one-shot motion prediction, and two-stage scenario generation. Log-replay yields high visual realism but no interactivity, because background agents follow fixed recorded trajectories and do not react when the ego deviates from the logged behavior. One-shot motion predictors generate all future trajectories in a single forward pass, so closed-loop unrolling induces covariate shift and unrealistic states. Two-stage generators decouple initial state generation from motion prediction and force a fixed agent set, preventing new agents from entering later in the rollout.

InfGen is designed to address long-horizon closed-loop realism, evolving agent populations, and diversity beyond the limited distribution of logs. The framework jointly creates initial states and trajectories, supports insertion of new agents over time, and is intended for both evaluation and reinforcement learning training. This makes it a generative simulator rather than a static replay mechanism or a pure forecasting model.

The paper defines the dynamic scene state at time $t$ as

$$\mathcal{S}_t = \bigl(\{\mathbf{l}^{(k)}_t\}, \{\mathbf{a}^{(i)}_t\}\bigr),$$

where $\mathbf{l}^{(k)}_t$ are traffic light states and $\mathbf{a}^{(i)}_t$ are agent states. The scenario is then serialized as

$$\mathbf{x}_{1:T} =\bigl[ MAP; (TL,AS,MO)_1; (TL,AS,MO)_2; \ldots \bigr],$$

with $MAP$ for static map tokens, $TL_t$ for traffic lights, $AS_t$ for agent states, and $MO_t$ for motions. The generative model is

$$p_\theta(\mathbf{x}_{1:T} \mid \mathcal{M}) = \prod_{t=1}^{T} p_\theta(\mathbf{x}_t \mid \mathbf{x}_{<t}, \mathcal{M}).$$

3. Tokenization, grouped autoregression, and scene dynamics

A central feature of InfGen is “next token group prediction,” in which generation is structured at the level of semantically meaningful groups rather than a flat token stream (Peng et al., 29 Jun 2025). Within each time step, the model generates all traffic light tokens in a batch, then agent state tokens agent by agent and field by field, then motion tokens in a batch over agents. The grouped causal attention mask encodes semantic structure: motion tokens can attend to current and past states and traffic lights; traffic lights can attend to previous motions; and tokens within a group attend freely to one another.

Traffic lights are represented as

$$\mathcal{S}_t = \bigl(\{\mathbf{l}^{(k)}_t\}, \{\mathbf{a}^{(i)}_t\}\bigr),$$0

Agents are represented as

$$\mathcal{S}_t = \bigl(\{\mathbf{l}^{(k)}_t\}, \{\mathbf{a}^{(i)}_t\}\bigr),$$1

with category $$\mathcal{S}_t = \bigl(\{\mathbf{l}^{(k)}_t\}, \{\mathbf{a}^{(i)}_t\}\bigr),$$2. Map context is encoded from polyline segments through a PointNet-like polyline encoder and a full-attention Transformer encoder. The resulting map tokens act as static keys and values in the decoder’s cross-attention.

Agent state tokens are organized as four ordered tokens,

$$\mathcal{S}_t = \bigl(\{\mathbf{l}^{(k)}_t\}, \{\mathbf{a}^{(i)}_t\}\bigr),$$3

where SOA denotes Start-Of-Agent, TYPE is the agent class, MS is the map segment index, and RS is the relative state with respect to that segment. The relative state vector is

$$\mathcal{S}_t = \bigl(\{\mathbf{l}^{(k)}_t\}, \{\mathbf{a}^{(i)}_t\}\bigr),$$4

InfGen does not discretize the entire map globally; instead, each agent is expressed locally relative to a segment. The paper states that this reduces vocabulary size, preserves locality, and supports precise placement of new agents on lanes.

Motion is represented by discrete motion labels corresponding to $$\mathcal{S}_t = \bigl(\{\mathbf{l}^{(k)}_t\}, \{\mathbf{a}^{(i)}_t\}\bigr),$$5, using 33 bins for each component plus a start token, yielding 1090 motion tokens. State rollout follows a first-order bicycle model:

$$\mathcal{S}_t = \bigl(\{\mathbf{l}^{(k)}_t\}, \{\mathbf{a}^{(i)}_t\}\bigr),$$6

Because agent-state generation occurs at every step, InfGen explicitly supports new agent insertion, persistence of existing agents, and disappearance when agents are no longer declared.

4. Architecture, training procedure, and generation regime in InfGen

InfGen uses an encoder–decoder Transformer architecture (Peng et al., 29 Jun 2025). The encoder processes static map tokens, while the decoder operates over dynamic tokens with self-attention under a grouped causal mask and cross-attention to map tokens at every layer. Relative positional attention incorporates geometric relations $$\mathcal{S}_t = \bigl(\{\mathbf{l}^{(k)}_t\}, \{\mathbf{a}^{(i)}_t\}\bigr),$$7 into the attention score, and a KNN mask can restrict attention to spatial neighbors when both tokens have positions.

Generation proceeds stepwise. At each step, traffic light states are generated in batch; agent states are generated by sampling TYPE from SOA, then map segment ID, then the relative state via a Relative State Head; and motion tokens are predicted in batch over agents, with top-$$\mathcal{S}_t = \bigl(\{\mathbf{l}^{(k)}_t\}, \{\mathbf{a}^{(i)}_t\}\bigr),$$8 sampling at inference. New agents can be inserted indefinitely by generating new SOA–TYPE–MS–RS sequences, while agents can leave by no longer being declared in later steps. The paper describes this as supporting arbitrarily long rollouts with evolving sets of agents.

Training is end-to-end with cross-entropy losses over traffic light states, agent type, map segment ID, relative-state fields, and motion labels, under teacher forcing. The dataset is the Waymo Open Motion Dataset, downsampled from 10 Hz to 2 Hz for 19 steps per scenario, with up to 128 agents and HD maps. ScenarioNet is used for scenario management. Preprocessing includes polyline segmentation into at most 3000 map segments, alignment of traffic lights to map segments, filtering and sorting of agents, and motion-label construction by enumerating candidate $$\mathcal{S}_t = \bigl(\{\mathbf{l}^{(k)}_t\}, \{\mathbf{a}^{(i)}_t\}\bigr),$$9 values and selecting the best via Average Corner Error.

The reported training schedule is two-stage: pretraining on traffic lights and motion only, followed by fine-tuning with full agent-state generation. Optimization uses AdamW with learning rate $\mathbf{l}^{(k)}_t$0, cosine decay, 2000 warmup steps, and gradient clipping. The model has approximately 4.6M parameters and is trained on 8 RTX A6000 GPUs.

5. Empirical behavior and reinforcement-learning utility of InfGen

InfGen is evaluated on initial-state realism, motion prediction, qualitative traffic behavior, and reinforcement-learning utility (Peng et al., 29 Jun 2025). For initial-state realism, Maximum Mean Discrepancy is measured over position, heading, size, and velocity distributions. Under the strict protocol, InfGen attains MMD values of 0.1291 for position, 0.1270 for heading, 0.0743 for size, and 0.1970 for velocity. Its non-autoregressive ablation is worse on position and heading, and under the relaxed protocol the gap between full InfGen and the non-AR ablation is substantial across all four attributes: 0.2198 vs. 0.3237 for position, 0.0665 vs. 0.1203 for heading, 0.0279 vs. 0.0630 for size, and 0.0730 vs. 0.1183 for velocity. The paper interprets this as evidence that ordered token-group generation is crucial for realism.

As a motion predictor on WOMD, InfGen-Motion reports ADE_avg of 1.2100 m, FDE_avg of 3.5336 m, and ADD of 2.2115, while InfGen-Full reports ADE_avg of 1.3382 m, FDE_avg of 3.8740 m, and ADD of 2.6486. The full model therefore trades some pointwise forecast accuracy for greater trajectory diversity, which the paper identifies as more beneficial for simulation and RL training than pure forecasting accuracy.

For RL, the framework is integrated into MetaDrive with ScenarioNet using 500 training scenarios and TD3 as the learning algorithm. Policies are evaluated on 100 held-out real WOMD scenarios using reward, success rate, route completion, off-road rate, collision rate, and cost. Log-Replay yields reward 32.24, success 0.7244, completion 0.6726, off-road 0.2872, collision 0.0308, and cost 0.2852. InfGen-Motion with adaptive generation yields reward 39.23, success 0.7475, completion 0.7032, and collision 0.0187. The best variant, InfGen-Full with reject sampling and adaptive generation, yields reward 39.07, success 0.7620, completion 0.7345, off-road 0.2830, collision 0.0260, and cost 0.2610. The reported result is that training on InfGen-generated reactive traffic improves robustness and generalization when evaluated on unmodified real-world scenarios.

The paper also positions InfGen against log-replay, two-stage scenario generators such as TrafficGen and SceneGen, forecasting models such as PredictionNet, Trafficsim, and MotionLM, and scenario-generation frameworks such as UniGen. Its claimed novelties are unified tokenization of map, signals, agent state, and motion; token-group autoregression with structured attention; infinite-style generation through continuous agent injection and motion rollout; and direct integration into RL pipelines.

6. Infogen for document visualization: task definition, metadata, and dataset

The second Infogen introduces the task of generating complex statistical infographics directly from text-heavy documents (Ghosh et al., 26 Jul 2025). A complex statistical infographic is defined as a single visual containing multiple sub-charts, each with its own axes, titles, and textual annotations, arranged into a cohesive, information-dense layout. The target differs from simple chart generation because the system must infer how many sub-charts are needed, choose appropriate chart types, extract and align numerical values from prose, and maintain cross-chart consistency in scales, categories, fonts, colors, and placement.

The overall pipeline is formalized as

$\mathbf{l}^{(k)}_t$1

where $\mathbf{l}^{(k)}_t$2 is the input textual document, $\mathbf{l}^{(k)}_t$3 is structured infographic metadata $\mathbf{l}^{(k)}_t$4, and $\mathbf{l}^{(k)}_t$5 is executable Python code $\mathbf{l}^{(k)}_t$6 for rendering the infographic. Metadata generation is trained with standard sequence cross-entropy,

$\mathbf{l}^{(k)}_t$7

and preference alignment uses a DPO objective over synthetic preference triples $\mathbf{l}^{(k)}_t$8 judged by GPT-3.5 or GPT-4o.

The metadata schema is unusually explicit. At infographic level it includes a title and summary. At sub-chart level it includes kind, axis, stats, text, position_chart, position_chart_text, background, dimensions, fonts, alignment, and summary. This metadata is treated as a content plan specifying what to show, how to show it, and where to place it.

The associated benchmark, Infodat, is constructed from the Pew Research chart dataset and contains 3,463 complex infographic images. Construction proceeds in three stages: filtering complex multi-chart visuals with GPT-4o; synthesizing a single continuous input passage per infographic without leaking chart count or layout terms; and generating metadata from the images with GPT-4 followed by human reviewer correction. The final split is 80% train, 5% validation, and 15% test. Reported descriptive statistics include an average of 341.015 words in metadata, an average of 185.46 words in input text, an average of 2.15 sub-charts per metadata instance, a maximum of 21 sub-charts, and a minimum of 1.

7. Two-stage infographic generation, evaluation, and limitations

Infogen’s first stage is text-to-metadata generation through fine-tuned LLMs, DPO alignment, and a ranker-based ensemble (Ghosh et al., 26 Jul 2025). The paper fine-tunes Qwen-2 Large (72B), LLaMA 3 (70B), Phi-3 Medium and Large, and smaller 7–8B-scale variants using QLoRA. For DPO, each input is decoded twice at different temperatures, GPT-3.5 Turbo or GPT-4o ranks the outputs, and the preferred/dispreferred pairs are used to optimize large models including LLaMA 3 70B, Phi-3 large, and Qwen2 large. Because individual models still hallucinate or miscount sub-charts, a fine-tuned LLaMA 3 70B ranker chooses among candidate metadata outputs based on heuristically labeled training comparisons that prioritize correct sub-chart count, sub-chart type, and alignment with ground truth metadata.

The second stage is metadata-to-code-to-infographic generation. A GPT-4o coder agent generates Python plotting code conditioned on metadata, using plotting libraries such as Plotly and Plotnine. A GPT-4o feedback or judge agent then checks subchart count, chart types, axes, statistics, positions, alignment, fonts, dimensions, background, title, summary, general spacing, and runtime validity. If the code fails any constraint, the judge returns specific feedback, and refinement continues for up to five iterations.

Automatic evaluation targets the metadata stage through Sub-chart Accuracy, Root Square Error on sub-chart count, Sub-chart Type Accuracy, Statistical Accuracy, and ROUGE-L for titles and summaries. Human evaluation of final infographics is performed on a random 35% of test samples using Readability, Visual Appeal, and Data Accuracy and Alignment scores on a 1–5 scale.

The main quantitative result is that Infogen (large) achieves Subchart Accuracy 74.69%, RSE 1.80, Title ROUGE-L 0.56, Summary ROUGE-L 0.49, Subchart Type Accuracy 84.23%, Subchart Summary ROUGE-L 0.52, and Statistical Accuracy 89.56. The best GPT-4o prompting baseline with BM25 clustering and 20-shot prompting achieves 57.69% Subchart Accuracy, RSE 2.20, Title ROUGE-L 0.39, Summary ROUGE-L 0.35, Subchart Type Accuracy 79.07%, Subchart Summary ROUGE-L 0.43, and Statistical Accuracy 88.62. The best single large fine-tuned model, Phi-3 large with DPO, achieves 72.11% Subchart Accuracy and 89.44% Statistical Accuracy. Human evaluation assigns Infogen scores of 4.1 for Readability, 3.8 for Visual Appeal, and 4.1 for Data Accuracy and Alignment, exceeding GPT-4o (20-shot) and Phi-3 (DPO).

The paper’s ablations argue that fine-tuning is critical relative to in-context prompting, DPO improves all large models, the fine-tuned ranker is stronger than naive mixture strategies, and the feedback loop corrects overlapping labels, misaligned sub-charts, and poor spacing. Reported limitations include imperfect sub-chart alignment and selection, restricted dataset scale and domain coverage because all 3,463 examples are derived from Pew Research, limited style flexibility, and inference cost approximately 1.5 times that of a single-model baseline. Failure modes include misinterpretation of numerical values, missing or extra sub-charts, layout imperfections without feedback, and occasional metadata–code divergence.

Taken together, the two Infogen systems exemplify a shared research pattern: explicit intermediate structure is used to make high-dimensional generation tractable. In InfGen for traffic, the intermediate structure is grouped scene tokenization with causal semantics. In Infogen for documents, it is metadata describing chart content and layout. This suggests that the commonality is methodological rather than application-specific: both systems rely on decomposing generation into semantically constrained units so that long-horizon consistency, controllability, and evaluability can be improved within their respective domains (Peng et al., 29 Jun 2025, Ghosh et al., 26 Jul 2025).