- The paper introduces a four-axis taxonomy covering inputs, outputs, supervision, and evaluation to classify autonomous-driving systems by their contribution to safe, feasible, and route-compliant planning rather than by backbone design.
- The paper finds that safety-aware open-loop metrics such as NAVSIM-style PDMS align better with closed-loop driving performance than ADE or FDE, while still producing ranking inversions that require evaluation across both real-log and reactive protocols.
- The paper recommends separating world-model and language roles, reporting deployment constraints and reproducibility levels, and combining learned planners with runtime assurance to address calibration, latency, long-tail safety, and action-grounding risks.
Motivation and central thesis
This survey addresses a definitional and methodological problem in end-to-end autonomous driving (E2E-AD): the term "end-to-end" has become overloaded, spanning everything from direct camera-to-steering regression to systems with rich intermediate representations such as BEV features, vectorized scene tokens, occupancy fields, world-model latents, and language tokens. The authors argue that the meaningful distinction is not whether intermediate structure exists, but whether that structure is learned, supervised, and evaluated so as to support safe, feasible, and route-compliant planning. They adopt the label planning-oriented E2E-AD for systems whose architectures and supervision signals are organized around the final driving plan.
The survey's positioning is distinctive because it treats three issues as inseparable: model architecture, evaluation protocol, and emerging foundation-model-based paradigms. Its central thesis is deliberately methodological — a system should not be judged by how structure-free it appears, but by whether its learned structure, supervision, and evaluation protocol jointly support safe and reproducible planning.
Scope and review methodology
The review follows a structured narrative protocol covering 2015 through June 2026, drawing on IEEE Xplore, ACM, SpringerLink, arXiv, OpenReview, CVF Open Access, benchmark repositories, and leaderboards, with backward and forward citation tracing from anchor works including PilotNet, Conditional Imitation Learning, Learning by Cheating, TransFuser, TCP, UniAD, VAD, NAVSIM, Bench2Drive, DriveLM, SimLingo, WOD-E2E, and LEAD. Inclusion requires at least one of: an influential E2E formulation, planning-relevant outputs, a benchmark or metric contribution, a world-model/VLM/VLA mechanism relevant to ego planning, public reproducibility resources, or an evaluation critique. Purely perception-only work and proprietary systems without comparable detail are excluded. The survey explicitly distinguishes itself from prior surveys on broad E2E driving, vision-language driving, world models, foundation models, and datasets by centering the taxonomy on output spaces, supervision, and evaluation validity rather than backbones or chronology.
Historical evolution
The survey traces four phases. Direct imitation: ALVINN, DeepDriving's affordance prediction, and PilotNet established sensor-to-control learning; the NoCrash analyses made covariate shift explicit, showing that once a policy deviates from expert states, errors compound into off-route behavior or collisions. Privileged distillation and robust imitation: Learning by Cheating distilled ground-truth-state teachers into image-based students; Learning by Watching and Learning From All Vehicles expanded supervision beyond ego demonstrations; RL contributed mainly as a complement for closed-loop interaction and recovery rather than a replacement for imitation. Structured planning interfaces: TransFuser, NEAT, TCP, PlanT, MILE, InterFuser, and culminating in UniAD's planning-oriented philosophy — upstream tasks should be designed by their contribution to planning — followed by VAD's vectorized representations, VADv2's probabilistic planning, DriveAdapter's coupling-barrier analysis, ThinkTwice's decoder-capacity findings, and PARA-Drive's real-time parallelization. Consequence-aware and language-grounded systems: world models (WoTE, World4Drive) and VLA systems (SimLingo, ORION, SpaceDrive, WAM-Flow).
A recurring lesson the authors draw is that each generation solved one bottleneck while exposing another: direct BC enabled learning but suffered covariate shift; conditional imitation improved controllability but not recovery; privileged distillation introduced learner-expert asymmetry; BEV methods improved geometric reasoning at the cost of label dependence; world models brought consequence modeling but raised calibration concerns; VLA systems added semantic reasoning but created a text-to-metric alignment problem.
The four-axis taxonomy
The organizing contribution is a taxonomy along four axes: input representation (front-view images through BEV/occupancy, object/vector tokens, latent world states, VLA token spaces), output space (steering/throttle, waypoints, trajectories, trajectory distributions, scene rollouts, language-conditioned action tokens), supervision signal (BC, perturbations, privileged teachers, auxiliary tasks, world-model prediction, language QA, rater preference), and evaluation protocol (open-loop replay, non-reactive simulation, reactive closed-loop, long-tail and preference-aware evaluation). Two transformer-based systems may belong to entirely different families depending on these axes; conversely, two architecturally dissimilar systems may be equivalent planning formulations. This framing usefully rejects backbone-centric classification.
On supervision, the survey makes several distinctions worth noting. It separates expert optimality, expert learnability, and student deployability, arguing that a perfect fully observable expert can be a poor teacher for a limited-sensor student — the issue LEAD addresses by designing experts to reduce visibility, uncertainty, and intent asymmetry. It also distinguishes auxiliary supervision (training-only losses) from auxiliary interfaces (structures visible at inference), and recommends papers report which signals are removed at deployment. On language, it separates three roles — training-time supervision, inference-time reasoning interface, and post-hoc explanation — and warns against conflating them, since each carries different compute, safety, and verification implications.
Benchmarks and the open-loop/closed-loop disagreement
The evaluation analysis is arguably the survey's strongest section. It documents the structural reasons open-loop and closed-loop metrics disagree: open-loop scores policies under the expert's state distribution rather than their own induced distribution; pointwise L2 penalizes valid multimodal alternatives; logged trajectories are one human realization under one interaction history, not universal optima; and open-loop protocols are vulnerable to ego-status shortcuts, as demonstrated by "Is Ego Status All You Need?" [2407.xxxx family, cited as li2024ego] and Hidden Biases [jaeger2023hidden].
The survey catalogs the benchmark landscape — CARLA, nuPlan, Bench2Drive, NAVSIM, TAD-E2E, WOD-E2E, Fail2Drive — and maps each protocol to what it can and cannot establish. A notable empirical finding comes from cross-benchmark correlation studies: safety-aware open-loop scores (NAVSIM-style PDMS) correlate with closed-loop driving score better than ADE/FDE, but ranking inversions persist and submetrics saturate. The practical consequence the authors draw is firm: no method should be called state of the art without specifying the protocol, and comparative tables should group results by metric family rather than mixing scores across protocols. They recommend triangulating any robust claim across at least one real-log proxy and one reactive closed-loop protocol, and propose a five-group reporting standard plus a detailed reproducibility checklist covering benchmark versions, sensor setup, controller details, runtime safety wrappers, compute budgets, and seed-level statistics.
World models and VLA systems
For world models, the survey proposes a four-role decomposition — representation pretraining (DriveWorld), generative scene prediction (GenAD, Drive-WM), trajectory evaluation via imagined consequences (WoTE, World4Drive), and reactive simulation (Bench2Drive-R, WorldDrive) — and insists these roles not be collapsed. The decisive evidence for a world model is whether imagined futures improve trajectory selection under distribution shift, not generation quality in isolation. The unresolved concern, stated plainly, is calibration: latent rollouts can be miscalibrated precisely in rare situations where planning most needs reliability.
For VLA systems, the survey is similarly measured. It acknowledges the appeal of semantic knowledge and inspectable reasoning (DriveLM, LMDrive, DriveVLM, OmniDrive, EMMA, DiMA, SimLingo, ORION, SGDrive) but holds that strong VLM reasoning does not imply safe control: latency, hallucination, non-actionable rationales, and explanation–action mismatch remain risks. The key demand is action-grounded evaluation — ablations showing trajectory improvement attributable to language, not merely better explanations. DiMA's distillation approach, which removes the LLM from the inference loop, is highlighted as a systems-relevant pattern.
Safety, deployability, and reproducibility
The survey is candid that formal guarantees are largely absent: public E2E work reports empirical mitigation (collision-aware losses, occupancy constraints, emergency braking, uncertainty thresholds) rather than certificates. It frames runtime assurance as compatible with end-to-end learning — the neural policy proposes, a verified or rule-constrained envelope disposes — and identifies the placement of the assurance boundary as the key open design question, noting trade-offs between post-hoc shields that mask planner weaknesses and differentiable shields that resist certification. On compute, it argues latency, memory, and verification constraints should be first-class scientific variables, citing PARA-Drive, DiMA, DrivoR, and WAM-Flow as responses.
The reproducibility section introduces a four-level scheme (paper-only, inference-reproducible, training-reproducible, benchmark-reproducible) and argues that citation value should track runnable baseline status, particularly for world-model and VLA work where nominal reproduction is often unrealistic.
Limitations and open questions
The authors concede three threats to the survey itself: bias toward public literature (industrial L4 stacks may already use these techniques unpublished); recency risk for 2025–2026 frontier work (WOD-E2E, SpaceDrive, SGDrive, Fail2Drive, WorldDrive are treated as signals, not consensus); and metric dependence, meaning comparisons should be read as a structured map of claims rather than a ranking. Open problems they enumerate include benchmark unification across realism, reactivity, long-tail coverage, and preference; learner-expert asymmetry requiring controlled experiments holding architecture and data scale fixed; world-model calibration under intervention; language-action grounding demonstrated by trajectory improvement; scaling formal envelopes to perception-conditioned multi-agent policies; and standardized reproducibility cards with independent reruns and public failure-case logs.
Conclusion
This survey's principal value lies less in cataloging architectures than in imposing evaluative discipline on a fast-moving field. Its four-axis taxonomy, its insistence on protocol-aligned comparison, its separation of world-model and language roles, and its reproducibility checklist together reframe how E2E-AD claims should be stated and audited. The field's next phase, as the authors frame it, is methodological as much as architectural: making claims comparable, failures localizable, and planning objectives faithful to the complexity of real driving.