TerraTransfer: Learning End-to-End Driving Policies Without Expert Demonstrations

Abstract: End-to-end autonomous driving has achieved state-of-the-art performance on benchmarks and real-world deployments. Its standard training recipe, however, is expensive across all stages: collecting and labeling millions of driving frames is costly, and closed-loop RL on images is bottlenecked by the per-step cost of photorealistic rendering plus a forward pass through a large vision backbone. Self-play in vectorized simulators changes the economics: millions of rollout steps per second, and a state distribution naturally rich in collisions, near-misses, and recoveries that no driving log contains. Our approach exploits this asymmetry by decoupling learning to drive from learning to see. We pretrain a single policy by self-play, then align its latent space with a pretrained vision backbone, through the action KL divergence and a batch-relational low-rank structural loss. The action target comes from the self-play policy, so alignment never supervises against a logged trajectory: a paired dataset of (image, scene-state) frames suffices, with no need for the curated expert demonstrations that imitation pretraining is built on. On photorealistic 3D Gaussian splatting closed-loop scenarios, the resulting end-to-end policy matches or exceeds prior end-to-end methods.

• The paper demonstrates a novel method that forgoes expert demonstrations by leveraging high-throughput self-play RL in a vectorized simulation environment.
• The approach decouples visual perception from decision-making through a two-phase training process that uses structured low-rank feature alignment and an action KL loss.
• Empirical results reveal that the vision-based policy achieves superior data efficiency and performance compared to traditional imitation learning baselines.

TerraTransfer: End-to-End Driving Policy Learning Without Expert Demonstrations

Overview of the TerraTransfer Paradigm

"TerraTransfer: Learning End-to-End Driving Policies Without Expert Demonstrations" (2606.17386) introduces a demonstration-free methodology for training end-to-end autonomous driving policies. The paper exploits recent advances in high-throughput, multi-agent self-play RL and leverages a two-phase workflow that decouples the representation learning related to visual perception from the policy optimization for closed-loop driving. Unlike canonical end-to-end approaches, TerraTransfer eliminates reliance on any supervised imitation learning or expert demonstrations at all stages, significantly reducing both annotation cost and domain adaptation overhead.

The key design consists of first learning driving competence in a fully simulated, vectorized environment using self-play RL, and then aligning the resulting policy with a vision-based frontend via a structured feature-alignment loss and an action KL term. This enables transfer to real-world-like, image-based driving without the requirement of curated human demonstration datasets.

Figure 1: Conventional end-to-end autonomous driving systems depend on extensive expert demonstrations and expensive image-based RL, whereas TerraTransfer decouples vision and planning through self-play vector RL followed by vision alignment.

Two-Phase Training Procedure

In Phase 1, a multi-agent self-play policy $\pi_\theta$ is optimized via PPO in a high-throughput vectorized simulation (TerraZero). Each agent directly interacts with other instances of itself, driving the interaction distribution toward naturalistic, rare, and diverse behavior modes—scenarios typically poorly covered by human logs. The architecture incorporates MLP-based ego encoding and DeepSets encoders for road and partner elements. Critical to the approach is reward randomization: reward weights are drawn independently per agent and per episode, instructing the policy to generalize over a continuum of behavioral preferences, further enhancing robustness.

In Phase 2, the optimal self-play policy is frozen and used as a privileged teacher. The vision-based student model ingests raw images, using a frozen DINOv3 backbone and linear adapters to map images to features corresponding to road and agent representations. Crucially, the alignment signal consists exclusively of the teacher’s latent space and action distribution in the form of per-frame (image, state) pairs—no expert actions are involved at any point. The alignment loss is a weighted sum of (1) a batch-relational low-rank structural loss, which enforces the student to replicate the pairwise relational geometry of teacher features in dominant subspace directions, and (2) an action-level KL loss between student and teacher policy distributions.

Figure 2: The two-phase pipeline: (left) self-play RL with vector input for policy learning, and (right) vision policy alignment, supervised only by the frozen teacher’s features and actions.

Empirical Results

Experiments are performed on the HUGSim benchmark (derived from nuScenes), evaluating closed-loop behavior in photorealistic settings with varied scenario difficulty tiers. The main findings include:

• On the aggregate benchmark, the vision policy aligned by TerraTransfer outperforms all published end-to-end imitation learning methods, reaching an HD-Score of 0.490 versus the best prior baseline of 0.452. The gap between the vector-state teacher and the vision-aligned student is minimal (0.520 vs. 0.490), indicating high-fidelity policy transfer.
• The vision-based student surpasses imitation-trained policies while using substantially less aligned data and with no need for trajectory annotation—achieving superior results with only 40% of the nuPlan training frames used by baselines such as ECO.

Figure 3: Alignment data efficiency: TerraTransfer matches or surpasses strong baselines with orders of magnitude less paired data, indicating effective exploitation of the vector RL-generated supervision signal.

Notably, the vision student transfers robust behavior even when aligned using non-expert or random rollouts, as only the coverage of state space (not the optimality of data collection) affects the alignment quality.

Analysis of Latent Feature Subspaces

An in-depth ablation reveals the teacher’s scene features are highly redundant and sharply low-rank: over 80% of the informative variation is captured in 9–13 principal directions (for road and partner features, respectively), supporting the design of the low-rank structural alignment loss. Ablating the rank truncation demonstrates closed-loop driving quality is maximized well before approaching the full coordinate dimensionality, while matching absolute coordinates is suboptimal.

Figure 4: The batch-wise low-rank bases extracted during alignment reliably capture the dominant axes of teacher feature variation; coverage degrades past the effective rank threshold.

Qualitative Driving Scenarios

Qualitative studies highlight that the learned driving policy exhibits nuanced social behavior and anticipatory decision-making across complex interactions such as intent-uncertain cut-in negotiation, overtaking under oncoming-traffic pressure, route-aware lead vehicle disengagement, and anticipatory deceleration for hazards ahead.

Figure 5: Cut-in negotiation: the ego vehicle dynamically adapts its commitment based on inferred intent, achieving smooth and safe interaction without overcommitting in ambiguous situations.

Figure 6: Overtaking with oncoming traffic: the learned policy executes context-sensitive, time-pressured overtaking maneuvers, reflecting robust judgment under safety constraints.

Figure 7: Narrow-lane passage: anticipatory deceleration and oscillatory speed modulation in response to mixed oncoming and vulnerable road user context.

Limitations and Future Outlook

The current pipeline's limitations are primarily due to simulator scope—specifically, the exclusive use of bounding-box inputs excludes fine-grained visual cues (e.g., brake lights, hand signals), and the scenario generator does not yet cover multi-agent pedestrian or complex traffic-rule situations. Expanding support for richer object types and signals is anticipated to further close remaining real-to-simulation transfer gaps.

On the theoretical front, the results demonstrate that policy extraction via high-throughput self-play RL in a vectorized domain, followed by low-rank feature alignment, is sufficient to instantiate image-based autonomous driving policies that achieve high real-world control fidelity without reliance on human demonstration. Practically, the framework shifts the dominant training cost from perpetual collection and annotation of rare, curated logs to scalable, one-time, and reusable RL. This paradigm points toward a hardware-limited, simulation-driven model scale-up regime for future end-to-end driving systems.

Conclusion

TerraTransfer establishes that competent, robust end-to-end driving policies can be learned without any supervision from expert trajectories or human demonstrations. This is accomplished by exploiting a vectorized self-play RL foundation as a source of privileged supervision and transferring the behavior to raw image space using relational subspace alignment. The resulting policies not only match but, in several aspects, exceed the capabilities of demonstration-dependent baselines, with dramatic improvements in data efficiency and overall cost-effectiveness. This study indicates that simulation-first training with decoupled perception alignment may be a dominant strategy for the next phase of large-scale autonomous driving policy development.