DriveMoE: Autonomous Driving MoE Framework

• DriveMoE is a mixture-of-experts framework designed to overcome multi-camera input redundancy and mode averaging in end-to-end autonomous driving.
• It employs a scene-specialized Vision MoE for dynamic camera view selection and a skill-specialized Action MoE for behavior-specific planning.
• DriveMoE demonstrates state-of-the-art results, outperforming its Drive-π0 baseline in Driving Score, Success Rate, and closed-loop Bench2Drive metrics.

Searching arXiv for the cited DriveMoE paper and closely related autonomous-driving MoE works.

DriveMoE is a Mixture-of-Experts-based end-to-end autonomous driving framework that augments the Vision-Language-Action baseline Drive-$\pi_0$ with two conditional-computation mechanisms: a Scene-Specialized Vision MoE for dynamic camera-view selection and a Skill-Specialized Action MoE for behavior-specific planning. Built on a pretrained PaliGemma VLM backbone and a flow-matching trajectory module, it is designed to address two specific weaknesses of end-to-end driving systems: inefficient processing of surround-view, multi-camera, multi-timestep visual input, and mode averaging over diverse or rare driving behaviors such as aggressive turns, emergency braking, and yielding. In Bench2Drive closed-loop evaluation, the paper reports state-of-the-art performance, with substantial gains over its Drive-$\pi_0$ baseline in Driving Score, Success Rate, and open-loop trajectory error (Yang et al., 22 May 2025).

1. Problem setting and Drive-$\pi_0$ baseline

DriveMoE is situated in end-to-end autonomous driving, where the model maps raw sensory input directly to future trajectory prediction and downstream control. The paper motivates the method through two persistent difficulties. First, autonomous driving must process surround-view, multi-camera, multi-timestep visual input, and uniform encoding of all views causes token count and compute to grow rapidly. Second, driving behavior is multi-modal and long-tailed: frequent behaviors such as lane following dominate the data, while rare but safety-critical cases require qualitatively different responses. The paper argues that a single monolithic policy tends to average across such modes, which degrades rare maneuver handling (Yang et al., 22 May 2025).

DriveMoE inherits the overall Vision-Language-Action structure from Drive-$\pi_0$, itself described as an adaptation of the embodied-AI model $\pi_0$ to autonomous driving. The inherited components are a pretrained PaliGemma VLM backbone, a fixed text prompt such as “Please predict future trajectory,” structured vehicle state, and a flow-matching-based action module for future trajectory generation. The model output is a predicted future trajectory of 10 waypoints, which is converted by a common PID controller into throttle, brake, and steering commands (Yang et al., 22 May 2025).

The concrete input configuration used by DriveMoE is narrower than an exhaustive surround-view encoder. In practice, the model uses two sequential front-view images plus one dynamically selected camera view. The front-view sequence is intended to expose temporal cues such as surrounding-agent velocity, while the dynamic view is selected by the Vision MoE according to scene context. Together with the fixed prompt and vehicle state—including current and historical position, velocity, acceleration, and heading angle—these inputs are fused by the VLM backbone before trajectory planning (Yang et al., 22 May 2025).

2. Dual-MoE architecture

DriveMoE adds two MoE mechanisms to Drive-$\pi_0$: one before expensive visual processing and one inside the action decoder. The front half of the architecture performs conditional perception by selecting contextually relevant camera views. The back half performs conditional planning by activating behavior-specialized experts in the flow-matching decoder. This yields conditional computation at both perception and action levels (Yang et al., 22 May 2025).

Component	Function	Concrete implementation
Scene-Specialized Vision MoE	Selects relevant camera views	Two front views plus one Top-1 dynamic view
Skill-Specialized Action MoE	Activates behavior-specific planning experts	One shared expert and six non-shared experts in implementation
Base VLA stack	Multimodal fusion and trajectory generation	PaliGemma backbone plus flow-matching action module

The two MoE components solve different subproblems. The Vision MoE reduces redundancy in multi-view perception by selecting the most relevant camera according to the current scene and route intention. The Action MoE replaces dense FFNs in the trajectory transformer with skill-specialized experts so that behaviorally distinct cases, such as merging or emergency braking, are not forced into a single undifferentiated decoder. The paper explicitly frames this as an attempt to improve both efficiency in multi-view perception and robustness to diverse, rare, and safety-critical driving behaviors (Yang et al., 22 May 2025).

3. Scene-Specialized Vision MoE

The Vision MoE is driven by the claim that not all camera views are equally useful in every situation. Rather than encoding all views uniformly, DriveMoE uses a lightweight router $\boldsymbol{R}_{\text{vision}}$ that takes the front-view embedding $\boldsymbol{e}_t^{\text{front}}$ and the future goal waypoint $\boldsymbol{g}_t$ from the route planner, and predicts a probability distribution over camera views: $$\boldsymbol{p}_t = \text{Softmax}\left(\boldsymbol{R}_{\text{vision}}\left(\boldsymbol{e}_t^{\text{front}}, \boldsymbol{g}_t\right)\right),$$ where $\pi_0$0 and $\pi_0$1 is the selection probability of view $\pi_0$2 at timestep $\pi_0$3 (Yang et al., 22 May 2025).

Operationally, the method uses Top-1 dynamic view selection at test time. The “experts” in this part of the system are effectively the available camera streams or view pathways, not separate per-view neural FFN experts. Because the routing happens before the expensive visual backbone, non-selected views can be skipped entirely. The selected views are augmented with learnable view-specific positional embeddings so that camera identity and cross-view spatial structure are retained (Yang et al., 22 May 2025).

The router is trained with manually generated heuristic camera-view annotations derived from future trajectory, maps, and object context. The appendix-level rules described in the paper include the following mappings: for intersection turning, annotate the front-side camera facing the intended exit direction; for lane change, annotate a corresponding rear-side camera if the target lane has the same direction and a front-side camera if opposing-lane occupancy is required; for highway merging/cut-in, annotate the side camera facing the merging location; and for yielding to emergency vehicles, annotate the camera facing the approaching emergency vehicle. With binary camera-view labels $\pi_0$4, the vision router is trained by

$\pi_0$5

This makes the visual routing explicitly supervised rather than latent (Yang et al., 22 May 2025).

4. Skill-Specialized Action MoE

The Action MoE is inserted into the flow-matching trajectory transformer. The paper’s core modification is to replace each dense FFN in the decoder with a mixture of expert FFNs. For decoder layer $\pi_0$6, with hidden input $\pi_0$7, the router $\pi_0$8 produces routing probabilities

$\pi_0$9

and each expert computes

$\pi_0$0

The layer output is written as

$\pi_0$1

The paper notes that the notation is imperfect: the text uses $\pi_0$2 and $\pi_0$3 inconsistently, while the implementation description is more specific (Yang et al., 22 May 2025).

In implementation, the Action MoE uses 1 shared expert and 6 non-shared experts. The non-shared experts are aligned with Bench2Drive skill categories: Merging, Overtaking, Emergency Brake, Give Way, and Traffic Sign, with an additional ParkingExit expert in the best ablation. The routing supervision comes from driving skill labels, trained via the paper’s shorthand cross-entropy expression

$\pi_0$4

The overall action-side loss is

$\pi_0$5

The text also states that a load-balancing regularization loss $\pi_0$6 is used to avoid expert collapse and that router noise is injected following DeepSeekMoE, but the explicit load-balancing formula and coefficient are not provided in the paper (Yang et al., 22 May 2025).

The routing regime is described generically as sparse, with only the top-ranked experts activated. The narrative mentions Top-1 or Top-2 conceptually, but the concrete implementation states that the top-3 experts selected by the action router are utilized during training and inference. This is the most specific setting reported and therefore the operative one for the published results (Yang et al., 22 May 2025).

5. Training, inference, and control generation

DriveMoE is trained on the Bench2Drive official training set, using the base set of 1000 clips, with 950 train and 50 validation/test for fair comparison. The supervision is entirely supervised or imitation-style: future waypoint learning through flow matching, heuristic camera-view labels for the vision router, and benchmark-defined skill labels for the action router. No reinforcement-learning stage is described (Yang et al., 22 May 2025).

Training proceeds in two stages. In Stage 1, for the first 10 epochs, both Vision MoE and Action MoE use ground-truth experts only, while the routers are trained jointly. This is a teacher-forcing-style stabilization phase. In Stage 2, training continues for 5 additional epochs from the Stage-1 checkpoint, but selected camera views and action experts are taken from the routers’ own predictions rather than ground-truth annotations. This reduces train-test mismatch and exposes the model to its own routing errors during optimization (Yang et al., 22 May 2025).

The reported optimization settings are: separate optimizers for the VLA and Action MoE experts, learning rate $\pi_0$7, warmup steps enabled, gradient clipping with max norm $\pi_0$8, and gradient accumulation to simulate batch size $\pi_0$9. The loss weights are stage-dependent. In Stage 1, $\pi_0$0 for the Vision Router loss, $\pi_0$1 for the Action Router loss, and $\pi_0$2 for the flow-matching loss. In Stage 2, $\pi_0$3, while other hyperparameters remain the same (Yang et al., 22 May 2025).

At inference time, DriveMoE uses router predictions rather than annotations. The visual input is always the pair of sequential front-view images plus one Top-1 dynamic camera view. The action module uses the router-selected experts, concretely the top-3 experts in the implementation. The output is a trajectory of 10 future waypoints. A fixed PID controller then computes control. The reported gains are

$\pi_0$4

for steering and

$\pi_0$5

for speed. The desired speed is computed from the 7th waypoint and the steering angle from the 10th waypoint (Yang et al., 22 May 2025).

6. Empirical results and ablations

The main benchmark is Bench2Drive closed-loop evaluation in CARLA v0.9.15.1, with 220 short routes designed to stress challenging corner cases. The primary metrics are Driving Score (DS), Success Rate (SR), Efficiency, Comfort, and Avg. L2. Bench2Drive also reports ability categories: Merging, Overtaking, Emergency Brake, Give Way, and Traffic Sign (Yang et al., 22 May 2025).

The headline comparison against Drive-$\pi_0$6 is:

Model	DS	SR	Efficiency	Comfort	Avg. L2
Drive-$\pi_0$7	60.45	30.00	168.41	14.88	0.56
DriveMoE	74.22	48.64	175.96	15.31	0.38

The paper states that, relative to Drive-$\pi_0$8, DriveMoE improves Driving Score by 22.8% and Success Rate by 62.1%. It also surpasses strong external baselines reported in the same table, including DiffAD, DriveTrans, and DriveAdapter, and is presented as state of the art on Bench2Drive (Yang et al., 22 May 2025).

Ability-specific results also indicate differential gains. DriveMoE reaches 34.67 on Merging, 40.00 on Overtaking, 65.45 on Emergency Brake, 40.00 on Give Way, and 59.44 on Traffic Sign, for a mean of 47.91. The corresponding Drive-$\pi_0$9 scores are 29.35, 36.58, 48.83, 40.00, and 54.45, with mean 41.84. The largest gain occurs in Emergency Brake, rising from 48.83 to 65.45, which is consistent with the paper’s emphasis on rare or safety-critical behaviors (Yang et al., 22 May 2025).

Ablations isolate the two MoE components. Removing the Vision MoE yields DS 68.68 and SR 42.45; removing the Action MoE yields DS 67.31 and SR 40.56; full DriveMoE reaches DS 74.22 and SR 48.64. This suggests that the two mechanisms are complementary rather than redundant. The Vision MoE ablation is especially sharp: adding a fixed back, front-left, or front-right camera improves modestly over the Drive-$\pi_0$0 baseline, but a dynamic view without supervision rises to DS 69.71, and a supervised dynamic view reaches the full 74.22. On the action side, the number of non-shared experts also matters: 6 non-shared experts is best; 13 or 44 experts reduce performance, which the paper attributes to load imbalance and data fragmentation (Yang et al., 22 May 2025).

Open-loop router accuracy further clarifies the asymmetry between the two routing problems. On the validation set, the Vision Router accuracy is 88.85%, while the Action Router accuracy is 65.40%. A plausible implication is that view selection is a more directly supervised and geometrically constrained routing problem than behavior-skill selection, which is more abstract and multimodal. The paper does not provide latency, FLOPs, or throughput numbers for DriveMoE, so its efficiency claim remains architectural rather than numerically benchmarked (Yang et al., 22 May 2025).

7. Position within driving-oriented MoE research

DriveMoE occupies a specific point in the broader autonomous-driving MoE design space. Its distinctive feature is the combination of view-level routing before vision encoding and skill-level expert routing inside a trajectory planner within a VLA framework (Yang et al., 22 May 2025). That differs from several adjacent lines of work.

In planning-oriented MoE systems, TrajMoE argues that the appropriate trajectory prior is scene-dependent and applies MoE inside a trajectory-scoring transformer over a predefined trajectory vocabulary, with reinforcement learning used only to refine scoring heads rather than to control the vehicle directly (Xing et al., 8 Dec 2025). EMoE-Planner instead uses supervised scene-level hard routing and scene-specific query banks in a pure learning planner on NuPlan, emphasizing interpretable scenario experts and interaction-oriented loss shaping (Zhu et al., 18 May 2025). These systems are close in spirit to DriveMoE’s action-side specialization, but they are not VLA systems and do not combine that specialization with pre-backbone camera routing.

Other works place MoE in different parts of the stack. MiniDrive uses FE-MoE in the visual feature-engineering stage before the LLM, prioritizing token-efficient multi-view question answering rather than trajectory generation (Zhang et al., 2024). MoSE implements hierarchical skill-oriented routing across perception, prediction, and planning inside a small VLM, but is evaluated primarily on reasoning and suggestion tasks rather than closed-loop control (Xu et al., 10 Jul 2025). D$\pi_0$1-MoE and DriveFine represent later generative-planning variants: the former uses dual disentanglement with longitudinal and lateral expert pools in a diffusion planner (Feng et al., 3 Jun 2026), while the latter uses a block-MoE that separates generation and refinement experts in a masked diffusion VLA planner (Dang et al., 16 Feb 2026). In perception, MoE-RAM applies statistic-augmented, decoupled routing and aggregation to semantic segmentation rather than control (Kou et al., 7 Dec 2025). In driver monitoring, VDMoE uses spatial and temporal experts for drowsiness, cognitive load, HR, and RR estimation from RGB cabin video (Wang et al., 2024).

Taken together, this literature indicates that autonomous-driving MoE has developed along several axes: perception-side routing, planning-side routing, reasoning-oriented routing, diffusion-generation routing, and driver-monitoring routing. DriveMoE’s contribution is to combine two of those axes—scene-specialized perception and skill-specialized planning—inside a single VLA end-to-end driving system. Its strongest empirical evidence lies in closed-loop Bench2Drive gains; its least specified aspects are the exact flow-matching internals, the Action MoE load-balancing formulation, and the systems-level efficiency characteristics (Yang et al., 22 May 2025).