OpenLongTail: Generative Scaling of Long-Tail Driving Data

Abstract: Scaling robust driving policies is fundamentally bottlenecked by the scarcity of edge cases in curated datasets. While the real world continuously captures these critical events, such long-tail events remain underutilized when collected from heterogeneous sources. Specifically, diverse but valuable in-the-wild long-tail videos lack the full view coverage required for training policy models, often missing multi-view poses or originating solely from monocular dash cameras. This modality gap prevents these ubiquitous observations from being converted into scalable training data for long-tail generalization. We introduce OpenLongTail, an open-source generative data engine for scaling autonomous driving policies under long-tail events. To transform heterogeneous data sources into view-aligned and temporally coherent multi-view assets that are useful for policy learning, we develop a pose-informed extrapolative view synthesis pipeline that generates the missing views. We further enhance cross-view consistency and the temporal alignment for the newly generated views by injecting Plücker ray geometry into the scalable generation engine. By synthesizing heterogeneous long-tail data, we observe a significant improvement in closed-loop driving robustness in handling long-tail events. By measuring the extrapolative view synthesis and pose metrics, we validate the effectiveness of OpenLongTail in visual fidelity, cross-view consistency, and ego-trajectory recovery.

• The paper introduces a pipeline that converts unsynchronized dashcam footage into synchronized, geometry-aligned multi-view assets using pose-informed extrapolative view synthesis.
• It employs innovative modules—geometry encoder, temporal depth warp, and cross-view memory bank—to ensure spatial and semantic consistency across synthesized views.
• The generated long-tail data significantly enhances autonomous driving policies by improving robustness and safety in rare, challenging driving environments.

OpenLongTail: Generative Scaling of Long-Tail Driving Data

Motivation and Problem Formulation

Robust policy learning for autonomous driving is fundamentally limited by the scarcity and heterogeneity of long-tail event data. Standard multi-camera rigs facilitate the capture of most nominal cases, but rare scenarios—such as work zones, dynamic obstacles, and complex intersections—are severely underrepresented due to the practical challenges and costs associated with their collection. Meanwhile, abundant long-tail driving footage exists in the wild (e.g., from consumer dashcams), but typically lacks synchronized multi-view coverage or reliable pose annotations, making such data incompatible with current Vision-Language-Action (VLA) model training protocols. OpenLongTail addresses this acquisition bottleneck by enabling automatic, large-scale transformation of heterogeneous single-view or unposed dashcam videos into high-fidelity, temporally coherent, and geometry-aligned multi-view assets suitable for policy training.

Figure 1: OpenLongTail transforms heterogeneous driving videos into pose-grounded, synchronized multi-view assets, providing geometry-aware conditions for diffusion-based extrapolative view synthesis, which enables robust policy learning with long-tail events.

Methodology

Pose-Informed Extrapolative View Synthesis

The OpenLongTail pipeline comprises four stages. First, the input monocular video undergoes metric-scale ego-trajectory recovery using MapAnything, with further stabilization via Kalman filtering and Rauch-Tung-Striebel smoothing, yielding accurate, temporally consistent pose estimates essential for multi-view asset synthesis. Second, pose-informed, extrapolative view generation is realized through a frozen Wan 2.1-VACE-1.3B video backbone with LoRA adapters and three novel conditioning modules:

• Geometry Encoder: Injects per-token Plücker ray representations for each target-camera/image coordinate, ensuring latent consistency with the physical camera geometry across viewpoints.
• Temporal Depth Warp: Propagates pixelwise RGB priors and depth information from the observed (front) view to the target views using geometric reprojection; non-overlapping rear/side cameras are handled via past/future temporal offsets along the ego-trajectory.
• Cross-View Memory Bank: Maintains spatial and semantic consistency by conditioning the synthesis of each view on previously generated spatial neighbors, leveraging a Perceiver-style memory resampler and noise-dependent gating.

The synthesis objective is a velocity flow-matching loss over video latents, enabling efficient, pose-grounded denoising in the diffusion model for each target-view, robustly enforcing geometric consistency and preserving visual structure even under heavy extrapolation.

Figure 2: The OpenLongTail pipeline, which recovers ego-trajectory, encodes geometric constraints, and synthesizes synchronized multi-view assets suitable for downstream policy training.

Downstream Driving Policy Impact

A primary aim is to enable closed-loop training and evaluation of VLA policies for long-tail driving. Using the synthesized views, end-to-end policies such as Alpamayo-R1 can be fine-tuned and robustified on edge-case scenarios previously inaccessible due to data fragmentation or modality gaps. Closed-loop evaluation in the AlpaSim simulator demonstrates that policies trained with OpenLongTail-generated assets experience substantial improvements in driving score and collision rate, approaching or matching those trained on ground-truth multi-camera data. Notably, the addition of generated assets from both NVIDIA’s in-house datasets and external sources (e.g., Waymo E2E) further covers rare events such as cyclist encounters and complex intersections where out-of-domain generalization is essential.

Figure 3: Qualitative closed-loop rollout in a rare work-zone event; OpenLongTail-augmented policy maintains stable drivable corridors and avoids off-road failures even without ground-truth multi-view training data.

Quantitative and Qualitative Generation Results

OpenLongTail outperforms multiple state-of-the-art trajectory-aware video generation baselines such as TrajectoryCrafter, ReCamMaster, Gen3C, and Vista4D in both image/video fidelity and, crucially, cross-view geometric consistency. On unseen and out-of-distribution scenes, OpenLongTail attains higher PSNR and lower LPIPS across multiple target cameras, with the GeoKPM metric indicating a large gap in cross-view match validity compared to competitors—a key measure for downstream policy transferability.

Figure 4: Qualitative comparison of multi-view video generation; OpenLongTail produces synchronized target views preserving scene layout and cross-view consistency, outperforming prior work like Vista4D.

OpenLongTail generalizes to unconstrained crowd-sourced sources (e.g., Nexar dashcam footage) by leveraging only monocular front-view videos and recovering sufficient geometric structure for multi-view asset synthesis, thereby unlocking a substantial portion of previously unusable “in-the-wild” long-tail driving data.

Figure 5: Results on Nexar dashcam data: only front view is available, yet the system recovers ego trajectory and synthesizes consistent extrapolative surround views.

Analysis of Data Distribution and Source Alignment

One of the central findings is that benefits from generative scaling are distribution-aware and alignment-conditioned. Augmenting policy training with generated assets from unaligned external sources such as Waymo E2E demonstrates clear improvements in certain categories (e.g., cyclists, uncommon vehicles), but offers limited gains—sometimes even regressions—in categories not well covered by the source distribution (e.g., human-guided complex intersections which are abundant in NVIDIA PAV but scarce in Waymo). This highlights that generative data scaling is not a pure function of sample size; alignment between the long-tail domain of interest and the external source’s scenario coverage is critical.

Figure 6: Source/domain distribution analysis: the occurrence of complex intersection types differs dramatically between NVIDIA and Waymo subsets, explaining alignment-conditioned training improvements.

Figure 7: Closed-loop rollout comparison demonstrates that generative scaling via aligned and external sources improves actual policy behavior in rare long-tail testcases.

Metric-Scale Ego-Trajectory Recovery

Accurate, temporally smooth metric-scale pose recovery is essential for both synthesis and policy training. OpenLongTail’s pose recovery matches the best absolute trajectory error (ATE) among competitive methods (e.g., MapAnything), but yields substantially lower jerk and acceleration variance, leading to more stable conditioning and improved final asset quality.

Broader Implications and Future Directions

OpenLongTail fundamentally advances the state of scalable autonomous driving policy learning by eliminating the requirement for synchronized multi-view data capture in rare scenarios and enabling the integration of vast, previously inaccessible monocular dashcam datasets. Practical implications include immediate improvements to long-tail robustness and safety in real-world deployed driving policies. Theoretically, this work refines our understanding of generative data scaling: effective expansion of training assets must consider both modality/geometry alignment and distributional coverage, not just data quantity.

Several limitations remain, such as the computational cost of diffusion inference at scale, residual temporal or camera-related artifacts, and the challenge of maintaining perfect cross-view structure under extreme scene divergence. Future efforts should explore advances in efficient generative modeling, explicit camera-parameter conditioning, and distribution-aware data selection to maximize policy robustness with minimal computational overhead.

Figure 8: Diverse unseen-scene examples: OpenLongTail generalizes to new domains and weather, maintaining coherent scene layout and multi-view synchronization.

Conclusion

OpenLongTail represents a robust, scientifically validated approach for bridging the crucial data gap in long-tail autonomous driving scenarios. By coupling metric-scale ego-trajectory recovery with geometry-grounded multi-view synthesis in a scalable, distribution-aware framework, it unlocks large-scale generative scaling using heterogeneous real-world video. The methodology is validated across multiple quantitative and qualitative axes, with demonstrated policy benefits in closed-loop simulation. OpenLongTail serves as a foundation for future work on efficient, reliable, and robust generative data engines driving continued progress in autonomous systems.