Infrastructure-Centric World Models: Bridging Temporal Depth and Spatial Breadth for Roadside Perception

Abstract: World models, generative AI systems that simulate how environments evolve, are transforming autonomous driving, yet all existing approaches adopt an ego-vehicle perspective, leaving the infrastructure viewpoint unexplored. We argue that infrastructure-centric world models offer a fundamentally complementary capability: the bird's-eye, multi-sensor, persistent viewpoint that roadside systems uniquely possess. Central to our thesis is a spatio-temporal complementarity: fixed roadside sensors excel at temporal depth, accumulating long-term behavioral distributions including rare safety-critical events, while vehicle-borne sensors excel at spatial breadth, sampling diverse scenes across large road networks. This paper presents a vision for Infrastructure-centric World Models (I-WM) in three phases: (I) generative scene understanding with quality-aware uncertainty propagation, (II) physics-informed predictive dynamics with multi-agent counterfactual reasoning, and (III) collaborative world models for V2X communication via latent space alignment. We propose a dual-layer architecture, annotation-free perception as a multi-modal data engine feeding end-to-end generative world models, with a phased sensor strategy from LiDAR through 4D radar and signal phase data to event cameras. We establish a taxonomy of driving world model paradigms, position I-WM relative to LeCun's JEPA, Li Fei-Fei's spatial intelligence, and VLA architectures, and introduce Infrastructure VLA (I-VLA) as a novel unification of roadside perception, language commands, and traffic control actions. Our vision builds upon existing multi-LiDAR pipelines and identifies open-source foundations for each phase, providing a path toward infrastructure that understands and anticipates traffic.

• The paper presents infrastructure-supported world models that unite temporal depth from fixed sensors with broader spatial insights from vehicle data.
• The methodology employs generative, uncertainty-aware, and physics-informed techniques to predict dynamic traffic scenarios and safety-critical events.
• The framework leverages V2X latent space alignment to enable collaborative perception and proactive infrastructure control in autonomous systems.

Infrastructure-Centric World Models: Bridging Temporal Depth and Spatial Breadth for Roadside Perception

Introduction and Motivation

The paper "Infrastructure-Centric World Models: Bridging Temporal Depth and Spatial Breadth for Roadside Perception" (2604.17651) articulates a comprehensive research agenda for shifting the locus of world modeling in next-generation autonomous driving from ego-vehicle-centered architectures to infrastructure-centric paradigms. The authors contend that infrastructure-supported world models (I-WM) are not merely an alternative to vehicle-centric world models but are fundamentally complementary. The key insight is rooted in a first-principles analysis of the spatio-temporal data regimes: stationary roadside sensors accumulate temporal depth (high-resolution behavioral distributions at persistent locations, including rare safety-critical events), whereas mobile, vehicle-mounted sensors survey broader spatial extents (diverse road geometries and traffic structures over wide networks). The epistemological and practical limitations of relying solely on either approach motivate the design of dual-domain, collaborative world models.

Technical Vision

The proposed framework outlines a three-phase research trajectory:

1. Phase I: Generative Scene Understanding with Quality-Aware Uncertainty Propagation. This phase leverages modular, annotation-free, and unsupervised multi-modal perception (drawing on pipelines such as FRGB3D, MulDet3D, and MulTrack3D) as a structured data engine. The downstream world model employs generative data-driven architectures (e.g., diffusion-based 4D scene generation and autoregressive occupancy forecasting) adapted for the infrastructure sensor regime and extended with explicit reliability annotations.
2. Phase II: Physics-Informed Predictive Dynamics with Multi-Agent Counterfactual Reasoning. I-WM integrates neural latent dynamics models with physical constraints (traffic flow theory, vehicle kinematics). This enables counterfactual reasoning not only at the ego-vehicle level, but at a system-wide, site-specific scale, exploiting the temporal recurrence of safety-critical events unique to the infrastructure perspective. The framework also supports language-conditioned scenario synthesis, incorporating causally relevant variables such as traffic signals and weather.
3. Phase III: Collaborative World Models for V2X via Latent Space Alignment. The long-term goal is cognitive and representational alignment between infrastructure and vehicle world models, realized through shared latent spaces and bandwidth-efficient V2X communication. This facilitates cooperative downstream tasks (multi-agent prediction, system-level control, and joint surrogate safety assessment), integrating temporal priors from infrastructure with spatial priors from vehicles.

Architectural and Methodological Distinctions

The I-WM framework is structured as a clean dual-layer architecture. The lower layer consists of site-adaptive, annotation-free perception, utilizing modular, uncertainty-propagating algorithms for 3D object detection, tracking, and background modeling. This layer produces richly structured, reliability-annotated observations (3D boxes, trajectories, static priors) feeding into the world modeling layer.

The world modeling layer is generative, self-supervised, and grounded in explicit 3D geometry and 4D occupancy sequences (cf. DynamicCity, OccWorld). This approach contrasts with the dominant paradigm of video prediction world models, whose output is 2D pixel-based and lacks geometric interpretability for safety metrics. By leveraging multi-LiDAR, camera, and new modalities (4D radar, event cameras, SPaT, environmental sensors), I-WM achieves high geometric fidelity, multi-modal forecasting, and environmental controllability.

A unique contribution of I-WM is the explicit propagation of sensor and perception uncertainty into the generative modeling pipeline, formalized as quality-aware channels within representations like HexPlane. This facilitates risk assessment and reliability-aware prediction crucial for safety-critical infrastructure applications.

Positioning Within Contemporary World Model Research

The authors establish a taxonomy distinguishing three paradigms: (A) video generation-centric, (B) 3D/4D scene generation-centric, and (C) decision/planning-centric world models. I-WM is technically anchored in Paradigm B but incorporates closed-loop RL-based planning components from Paradigm C. This strategic positioning is justified by the safety-critical need for explicit 3D reasoning and uncertainty quantification for downstream use cases (e.g., time-to-collision, post-encroachment time estimation).

The paper situates I-WM in the context of seminal frameworks such as LeCun's JEPA (predictive learning in abstract latent spaces), Fei-Fei Li's spatial intelligence agenda (grounded 3D/4D scene understanding), and the emerging Vision-Language-Action (VLA) paradigm, generalizing the latter to infrastructure domains (I-VLA), where "action" corresponds to signal control and system-level interventions rather than single-agent trajectory optimization.

Implications and Broader Impacts

I-WM embodies several practical and theoretical implications:

• Enhanced Safety Analytics and Surrogate Metrics: Long-term observation enables direct estimation of rare event distributions and site-specific safety baselines, supporting regulatory and planning functions beyond the purview of AV system validation.
• Proactive Infrastructure Control: Integration of learned world models with RL-based or model-predictive traffic management obviates the traditional need for hand-engineered simulators. I-WM enables stress-testing and optimization of control policies (e.g., SPaT, variable speed limits) under realistic, multi-agent dynamics.
• Scalable Data and Deployment Strategy: The modular dual-layer design, combined with phased sensor integration, provides a path toward city-scale, annotation-free deployment. Synthetic data, bootstrapped via world model rollouts, further addresses data scarcity for less-instrumented sites.
• Cognitive V2X Communication: Latent space alignment between infrastructure and vehicle world models supports fine-grained, bandwidth-efficient cooperative perception, control, and forecasting. This enables AVs and infrastructure to mutually compensate for each other's epistemic limitations (temporal vs. spatial coverage).

Future Directions

Anticipated developments include convergence toward unified, end-to-end, infrastructure foundation models that absorb both perception and world modeling into a joint, large-scale architecture—analogous to vehicle-centric world model trends. The proposed extension to Infrastructure VLA (I-VLA), integrating natural language, perception, and infrastructure actuation, introduces a new control-theoretic and human-interactive dimension for intelligent transportation systems.

Further research is required in scalable training of uncertainty-aware generative models, robust cross-domain latent alignment protocols, and integration of high-dimensional sensor modalities (event cameras, environmental telemetry). The simulation-to-real transfer for new sensor types (such as 4D radar) and cross-site generalization also present ongoing challenges.

Conclusion

The paper "Infrastructure-Centric World Models" (2604.17651) presents a rigorous, technically substantiated framework for transforming roadside perception from object detection and tracking toward proactive, generative, and uncertainty-aware world modeling. By formalizing and exploiting the complementarity between spatial and temporal data regimes inherent to vehicle and infrastructure sensing respectively, and by embedding model-driven physical constraints and V2X alignment capabilities, I-WM fills a critical gap in the state of the art. The proposed trajectory lays the groundwork for future research in safety-critical AI for intelligent transportation, emphasizing practical deployment scalability, theoretical soundness, and adaptability to evolving sensor and computation environments.