VectorWorld: Streaming Vector Simulation

• VectorWorld is a streaming world model for closed-loop autonomous-driving simulation that incrementally generates ego-centric lane–agent vector-graph tiles.
• It employs a motion-aware gated VAE and an edge-gated relational DiT with interval-conditioned MeanFlow for solver-free one-step masked completion.
• Empirical results on datasets like Waymo and nuPlan demonstrate improved stability, reduced collision rates, and enhanced long-horizon feasibility.

Searching arXiv for the cited VectorWorld paper and adjacent vector-structured generation work. VectorWorld denotes a streaming world model for closed-loop autonomous-driving simulation that incrementally generates ego-centric lane–agent vector-graph tiles during rollout, rather than relying on rasterized scene synthesis or static log replay. In its canonical formulation, VectorWorld addresses three deployment constraints in interactive simulation: history-free initialization, multi-step sampling latency, and long-horizon kinematic infeasibility. The system combines a motion-aware gated VAE for policy-compatible interaction-state initialization, an edge-gated relational DiT trained with interval-conditioned MeanFlow and JVP-based large-step supervision for solver-free one-step masked completion, and a physics-aligned non-ego policy, denoted ASim in the paper, for stable long-horizon rollouts (Jiang et al., 18 Mar 2026).

1. Definition and research setting

VectorWorld is situated in the problem of closed-loop autonomous-driving evaluation, where planners require interactive environments that respond to ego actions, extend beyond recorded horizons, and support counterfactuals. The paper’s central claim is that existing generative world models often degrade online because they initialize from history-free states, require multi-step diffusion or flow solvers that violate per-frame real-time budgets, and accumulate small feasibility errors over kilometer-scale rollouts (Jiang et al., 18 Mar 2026).

The representation is explicitly vector-graph based. Scenes are encoded as heterogeneous graphs whose nodes correspond to lanes and agents, and whose typed edges encode lane-to-lane, agent-to-agent, lane-to-agent, and agent-to-lane relations. The generated content is streamed as ego-centric $64 \mathrm{m}\times 64\mathrm{m}$ tiles, decoded to vector geometry, and stitched into a global scene. This choice places structural priors directly in the latent scene representation rather than in a raster output space, which the paper argues improves topology preservation and agent–lane alignment in comparison with rasterized world models (Jiang et al., 18 Mar 2026).

A plausible implication is that VectorWorld belongs to a broader shift toward structured generative representations. Related work in text-to-SVG generation and amodal vectorization likewise treats vector primitives, layers, or graphs as primary modeling units rather than as post hoc outputs from raster synthesis (Jain et al., 2022, Hu et al., 13 Apr 2026, Feuerpfeil et al., 2024, Zhang et al., 2024).

2. Vector-graph representation and architecture

VectorWorld uses an ego-centric SE(2) scene parameterization. Lane nodes carry centerline polylines in the ego frame. Agent nodes carry a static state $$[x, y, speed, \cos\theta, \sin\theta, length, width]$$, a type one-hot, and a motion-history code. The graph includes typed heterogeneous edges $E = E_{L2L} \cup E_{A2A} \cup E_{L2A} \cup E_{A2L}$, with lane connectivity and interaction structure encoded directly in the graph topology (Jiang et al., 18 Mar 2026).

The initialization interface is a motion-aware gated VAE. Its encoder uses two agent branches, one for static information and one for motion, with a learned gate that selectively fuses motion when informative and suppresses it for static agents. The paper gives the gating and posterior parameterization as

$g_i = \sigma(f_{\mathrm{gate}([s_i, m_i, T_i])}),$

$$h_i = (1 - g_i)\,\odot\,\phi_{\mathrm{st}([s_i, T_i])} + g_i\,\odot\,\phi_{\mathrm{mot}([m_i, T_i])},$$

$$q_\phi(z_i \mid s_i, m_i, T_i) = \mathcal{N}(\mu_\phi(h_i), \mathrm{diag}(\sigma_\phi(h_i)^2)).$$

The stated purpose is to produce policy-compatible states aligned with history-conditioned policies and to permit warm-start initialization at $k=0$ through deterministic unrolling of motion codes into policy history windows (Jiang et al., 18 Mar 2026).

Scene generation beyond initialization is handled by an edge-gated relational DiT. Its attention alternates across typed edge sets and injects edge-conditioned bias and multiplicative gating:

$$\alpha_{ij}^{(h)} = \mathrm{softmax}_j\Big( \frac{\langle W_q^{(h)} x_i, W_k^{(h)} x_j \rangle}{\sqrt{d_h} + B^{(h)}(e_{ij})} \Big),$$

$$v_i^{(h)} = \sum_j \alpha_{ij}^{(h)} \big( \{1 + \tanh(G^{(h)}(e_{ij}))\} \odot W_v^{(h)} x_j \big).$$

According to the paper, the additive bias regulates connectivity and the multiplicative gate controls feature aggregation strength, with the intended effect of improving lane continuity and lane–agent alignment (Jiang et al., 18 Mar 2026).

3. MeanFlow formulation and one-step masked completion

VectorWorld’s generative core is an interval-conditioned MeanFlow model trained for large-step transport under masking. The rectified path is

$z_t = (1 - t) z + t \epsilon,\quad t \in [0,1].$

For clamped tokens, the model sets $$[x, y, speed, \cos\theta, \sin\theta, length, width]$$0, so that the target velocity becomes $$[x, y, speed, \cos\theta, \sin\theta, length, width]$$1. This constant-path conditioning is the mechanism used to preserve known content during outpainting (Jiang et al., 18 Mar 2026).

The JVP-corrected interval-mean velocity is defined as

$$[x, y, speed, \cos\theta, \sin\theta, length, width]$$2

with the training objective

$$[x, y, speed, \cos\theta, \sin\theta, length, width]$$3

The total derivative $$[x, y, speed, \cos\theta, \sin\theta, length, width]$$4 is implemented through a Jacobian–vector product with tangent $$[x, y, speed, \cos\theta, \sin\theta, length, width]$$5 along the rectified path. The paper presents this as the basis for large-step supervision and as the reason one-step inference remains viable in a masked completion setting (Jiang et al., 18 Mar 2026).

At inference, VectorWorld uses classifier-free guidance on the mean velocity,

$$[x, y, speed, \cos\theta, \sin\theta, length, width]$$6

followed by a single large-step update,

$$[x, y, speed, \cos\theta, \sin\theta, length, width]$$7

Setting $$[x, y, speed, \cos\theta, \sin\theta, length, width]$$8 yields the deployed solver-free one-step sampler. In the paper’s interpretation, this is the key enabler of real-time outpainting because it removes iterative solver costs while keeping conditioned structure fixed through the mask constraint (Jiang et al., 18 Mar 2026).

This design contrasts with DDPM and standard rectified-flow baselines, which the paper includes for comparison:

$$[x, y, speed, \cos\theta, \sin\theta, length, width]$$9

and

$E = E_{L2L} \cup E_{A2A} \cup E_{L2A} \cup E_{A2L}$0

The paper’s position is that these alternatives are less suitable for real-time streaming at the required latency budget (Jiang et al., 18 Mar 2026).

4. Streaming rollout and closed-loop behavior

The runtime system streams new content only when the ego approaches tile boundaries. Known latents are clamped, frontier tokens are completed, and decoded tiles are SE(2)-transformed and stitched into the global route. This produces a rollout loop in which new scene context is synthesized on demand rather than precomputed at episode start (Jiang et al., 18 Mar 2026).

The non-ego behavior model is ASim, described as a physics-aligned NPC policy with hybrid discrete–continuous actions. Its action parameterization is

$E = E_{L2L} \cup E_{A2A} \cup E_{L2A} \cup E_{A2L}$1

Kinematic feasibility is injected by differentiable logit shaping,

$E = E_{L2L} \cup E_{A2A} \cup E_{L2A} \cup E_{A2L}$2

and by the DKAL regularizer,

$E = E_{L2L} \cup E_{A2A} \cup E_{L2A} \cup E_{A2L}$3

The paper uses yaw-rate, curvature, and lateral acceleration proxies,

$E = E_{L2L} \cup E_{A2A} \cup E_{L2A} \cup E_{A2L}$4

to define differentiable costs for shaping and evaluation (Jiang et al., 18 Mar 2026).

The result is a three-part closed-loop engine. Initial Scene Generation is handled by the motion-aware VAE. Scene Outpainting is handled by the edge-gated relational DiT with one-step MeanFlow. NPC Behavior is handled by ASim. The paper presents these components as jointly necessary: initialization reduces cold-start jerk, one-step generation satisfies streaming latency, and ASim limits compounding infeasibility over long horizons (Jiang et al., 18 Mar 2026).

This suggests that VectorWorld is less a single generator than an integrated simulation stack. The architecture resembles structured pipelines in other vector domains, where a semantic or geometric backbone is paired with optimization or editing modules rather than treated as a monolithic decoder (Hu et al., 13 Apr 2026, Nowakowski et al., 2022).

5. Empirical results and operating characteristics

VectorWorld is evaluated on Waymo Open Motion and nuPlan. Scenes are represented as $E = E_{L2L} \cup E_{A2A} \cup E_{L2A} \cup E_{A2L}$5 ego-centric tiles with capped counts; the paper states Waymo defaults of $E = E_{L2L} \cup E_{A2A} \cup E_{L2A} \cup E_{A2L}$6 lanes and $E = E_{L2L} \cup E_{A2A} \cup E_{L2A} \cup E_{A2L}$7 agents, while nuPlan includes traffic-light semantics when available (Jiang et al., 18 Mar 2026).

The paper reports improvements in both offline scene quality and closed-loop behavior. For offline scene quality, it gives the following headline values. On nuPlan, VectorWorld achieves endpoint distance $E = E_{L2L} \cup E_{A2A} \cup E_{L2A} \cup E_{A2L}$8 m, collision rate $E = E_{L2L} \cup E_{A2A} \cup E_{L2A} \cup E_{A2L}$9, route length $g_i = \sigma(f_{\mathrm{gate}([s_i, m_i, T_i])}),$0 m, and Fréchet distance $g_i = \sigma(f_{\mathrm{gate}([s_i, m_i, T_i])}),$1. On Waymo, it reports Fréchet distance $g_i = \sigma(f_{\mathrm{gate}([s_i, m_i, T_i])}),$2, route length $g_i = \sigma(f_{\mathrm{gate}([s_i, m_i, T_i])}),$3 m, endpoint distance $g_i = \sigma(f_{\mathrm{gate}([s_i, m_i, T_i])}),$4 m, and collision rate $g_i = \sigma(f_{\mathrm{gate}([s_i, m_i, T_i])}),$5 (Jiang et al., 18 Mar 2026).

For quality–latency trade-offs, the paper states that MeanFlow-1 runs at approximately $g_i = \sigma(f_{\mathrm{gate}([s_i, m_i, T_i])}),$6 ms per tile with Waymo endpoint distance $g_i = \sigma(f_{\mathrm{gate}([s_i, m_i, T_i])}),$7 m, MeanFlow-5 runs at approximately $g_i = \sigma(f_{\mathrm{gate}([s_i, m_i, T_i])}),$8 ms per tile with endpoint distance $g_i = \sigma(f_{\mathrm{gate}([s_i, m_i, T_i])}),$9 m, and DDPM-100 runs at approximately $$h_i = (1 - g_i)\,\odot\,\phi_{\mathrm{st}([s_i, T_i])} + g_i\,\odot\,\phi_{\mathrm{mot}([m_i, T_i])},$$0 ms per tile with endpoint distance $$h_i = (1 - g_i)\,\odot\,\phi_{\mathrm{st}([s_i, T_i])} + g_i\,\odot\,\phi_{\mathrm{mot}([m_i, T_i])},$$1 m. The stated real-time budget is $$h_i = (1 - g_i)\,\odot\,\phi_{\mathrm{st}([s_i, T_i])} + g_i\,\odot\,\phi_{\mathrm{mot}([m_i, T_i])},$$2 ms per tile, so one-step and few-step MeanFlow configurations are presented as compatible with streaming, whereas DDPM-100 is characterized as offline only (Jiang et al., 18 Mar 2026).

For closed-loop rollouts, the paper reports that warm-start interaction states reduce jerk from $$h_i = (1 - g_i)\,\odot\,\phi_{\mathrm{st}([s_i, T_i])} + g_i\,\odot\,\phi_{\mathrm{mot}([m_i, T_i])},$$3 to $$h_i = (1 - g_i)\,\odot\,\phi_{\mathrm{st}([s_i, T_i])} + g_i\,\odot\,\phi_{\mathrm{mot}([m_i, T_i])},$$4 and increase success from $$h_i = (1 - g_i)\,\odot\,\phi_{\mathrm{st}([s_i, T_i])} + g_i\,\odot\,\phi_{\mathrm{mot}([m_i, T_i])},$$5 to $$h_i = (1 - g_i)\,\odot\,\phi_{\mathrm{st}([s_i, T_i])} + g_i\,\odot\,\phi_{\mathrm{mot}([m_i, T_i])},$$6 in streaming settings. It further states that VectorWorld supports stable, real-time $$h_i = (1 - g_i)\,\odot\,\phi_{\mathrm{st}([s_i, T_i])} + g_i\,\odot\,\phi_{\mathrm{mot}([m_i, T_i])},$$7 rollouts and that PPO retraining in VectorWorld increases success from $$h_i = (1 - g_i)\,\odot\,\phi_{\mathrm{st}([s_i, T_i])} + g_i\,\odot\,\phi_{\mathrm{mot}([m_i, T_i])},$$8 to $$h_i = (1 - g_i)\,\odot\,\phi_{\mathrm{st}([s_i, T_i])} + g_i\,\odot\,\phi_{\mathrm{mot}([m_i, T_i])},$$9 on the hardest $$q_\phi(z_i \mid s_i, m_i, T_i) = \mathcal{N}(\mu_\phi(h_i), \mathrm{diag}(\sigma_\phi(h_i)^2)).$$0 stress-test setting (Jiang et al., 18 Mar 2026).

ASim-specific ablations also emphasize feasibility gains. The paper reports ADE improving from $$q_\phi(z_i \mid s_i, m_i, T_i) = \mathcal{N}(\mu_\phi(h_i), \mathrm{diag}(\sigma_\phi(h_i)^2)).$$1 m for Ctrl-Sim to $$q_\phi(z_i \mid s_i, m_i, T_i) = \mathcal{N}(\mu_\phi(h_i), \mathrm{diag}(\sigma_\phi(h_i)^2)).$$2 m for ASim, controllability $$q_\phi(z_i \mid s_i, m_i, T_i) = \mathcal{N}(\mu_\phi(h_i), \mathrm{diag}(\sigma_\phi(h_i)^2)).$$3 increasing from $$q_\phi(z_i \mid s_i, m_i, T_i) = \mathcal{N}(\mu_\phi(h_i), \mathrm{diag}(\sigma_\phi(h_i)^2)).$$4 to $$q_\phi(z_i \mid s_i, m_i, T_i) = \mathcal{N}(\mu_\phi(h_i), \mathrm{diag}(\sigma_\phi(h_i)^2)).$$5, and feasibility violations such as lateral acceleration decreasing from $$q_\phi(z_i \mid s_i, m_i, T_i) = \mathcal{N}(\mu_\phi(h_i), \mathrm{diag}(\sigma_\phi(h_i)^2)).$$6 to $$q_\phi(z_i \mid s_i, m_i, T_i) = \mathcal{N}(\mu_\phi(h_i), \mathrm{diag}(\sigma_\phi(h_i)^2)).$$7 under DKAL and logit shaping (Jiang et al., 18 Mar 2026).

6. Context, neighboring uses, and limitations

Within autonomous-driving simulation, VectorWorld is presented as distinct from replay-only evaluation and from rasterized generative world models. Its novelty is described as a unified edge-gated relational DiT trained with interval-conditioned MeanFlow and JVP correction for solver-free one-step masked completion on heterogeneous vector graphs, together with a motion-aware initialization interface and a physics-aligned NPC policy (Jiang et al., 18 Mar 2026).

The broader literature shows that “vector world” ideas also arise outside driving. In text-to-SVG, VectorFusion distills raster diffusion priors into differentiable SVG parameterizations without paired captioned SVG data, using Score Distillation Sampling and DiffVG (Jain et al., 2022). Vector Grimoire learns a discrete codebook of vectorizable shapes under raster-only supervision and models text-conditioned SVG token sequences autoregressively (Feuerpfeil et al., 2024). “Text-to-Vector Generation with Neural Path Representation” instead regularizes path generation by optimizing in a learned latent manifold and refines outputs via layer-wise image vectorization (Zhang et al., 2024). AmodalSVG extends vectorization to semantically organized, amodally complete layers with explicit occlusion order, supporting object-level editing without background holes (Hu et al., 13 Apr 2026). FieldView, by contrast, uses mesh shaders for interactive 3D vector-field visualization, emphasizing on-GPU evaluation and geometry construction rather than generative scene synthesis (Nowakowski et al., 2022).

These neighboring systems are not variants of VectorWorld in the strict sense of the driving paper. A plausible implication is that they share a common methodological tendency: replacing pixels with editable, typed, or semantically structured primitives as the fundamental state space of generation, rendering, and interaction.

The limitations stated for VectorWorld are specific. Centerline-based maps omit some fine-grained elements such as curbs, detailed boundaries, and certain traffic-control semantics. One-step streaming leaves residual gaps in fine-grained distribution matching. ASim reduces violations but does not strictly guarantee physical feasibility under rare extreme maneuvers. The paper also notes that exact bicycle-model parameters are not specified, because the simulator uses delta-based SE(2) motion and differentiable feasibility proxies instead (Jiang et al., 18 Mar 2026).

From an encyclopedia perspective, VectorWorld is therefore best understood as a deployment-oriented world model: a system designed around initialization validity, real-time streaming, and long-horizon stability rather than around open-loop perceptual plausibility alone. Its distinctive contribution is to make vector-graph generative modeling operational in closed-loop simulation, while related vector-graphics and vectorization work demonstrates how similar structural commitments recur in adjacent areas of generative modeling and scientific visualization (Jiang et al., 18 Mar 2026, Jain et al., 2022, Hu et al., 13 Apr 2026, Feuerpfeil et al., 2024, Zhang et al., 2024, Nowakowski et al., 2022).