---
title: 'GEMINUS: Mixture-of-Experts in Autonomous Driving'
url: https://www.emergentmind.com/topics/geminus
type: topic
---

# GEMINUS: Mixture-of-Experts in Autonomous Driving

Searching arXiv for the GEMINUS paper and closely related autonomous-driving benchmark/method papers for citation support.
Searching for Bench2Drive and key related end-to-end driving references mentioned alongside GEMINUS.
GEMINUS is a mixture-of-experts framework for end-to-end autonomous driving that combines a globally trained planner, scenario-specialized planners, and an uncertainty-sensitive routing mechanism in order to address the diversity and multimodality of traffic behavior. The name expands to “Dual-aware Global and Scene-Adaptive Mixture-of-Experts for End-to-End Autonomous Driving.” Its central claim is that a single driving policy tends to learn an “average” behavior over heterogeneous traffic situations, whereas practical driving requires both scenario-specific specialization and a robust fallback when scene classification is ambiguous. In GEMINUS, a **Global Expert** is trained on the full dataset, a **Scene-Adaptive Experts Group** is trained on predefined scenario subsets, and a **Dual-aware Router** chooses between them by jointly considering scenario-level features and routing uncertainty [2507.14456].

## 1. Problem formulation and motivation

GEMINUS is defined in a planning-oriented end-to-end setting in which the model maps a **front-facing monocular RGB image** \(i\), the **ego speed** \(v\), a **high-level navigation command** \(c\), and a **goal point** \((x_g, y_g)\) to a short-horizon future trajectory. The predicted trajectory is a sequence of waypoints over \(K=4\) steps at \(2\text{ Hz}\), corresponding to a \(2\)-second horizon, and a PID controller converts those waypoints into throttle, brake, and steer commands [2507.14456].

The motivation is the paper’s diagnosis of **mode averaging** in single-policy imitation learning. When a single model is optimized over the full training distribution with conventional regression losses, qualitatively distinct behaviors—such as merging, overtaking, emergency braking, giving way, and obeying traffic signs—are forced into a single shared policy. The paper characterizes this as fitting a single Gaussian-like output mode to a genuinely multimodal distribution. In that interpretation, a planner may produce compromise trajectories that are globally plausible but locally suboptimal for the current scenario [2507.14456].

A further premise is that command conditioning is not sufficient to resolve this mismatch. Commands such as “turn left,” “go straight,” or “turn right” do not fully specify the interaction structure of a scene, and therefore do not by themselves supply the specialization signal required for differentiated driving skills. GEMINUS is proposed as a driving-specific alternative to both single-expert planning and generic MoE routing [2507.14456].

## 2. Architectural organization

The architecture has three principal components, each with a distinct function.

| Component | Training scope | Function |
|---|---|---|
| **Global Expert** | Overall dataset | Robust fallback policy |
| **Scene-Adaptive Experts Group** | Corresponding scene subsets | Scenario-specific adaptive behavior |
| **Dual-aware Router** | Scenario supervision plus uncertainty estimation | Selects specialist or global expert |

The **Global Expert** is trained on the entire dataset and is intended to provide broad coverage and stable behavior when scenario identity is unclear. The **Scene-Adaptive Experts Group** consists of five experts corresponding to the scenario subsets **Merging**, **Overtaking**, **Emergency Brake**, **Give Way**, and **Traffic Sign**. Each specialist is trained on its own subset so that it can learn a narrower policy distribution than the global model [2507.14456].

The **Dual-aware Router** is the core gating mechanism. It is “dual-aware” in two senses. First, it is **scenario aware**: it predicts which scene-adaptive expert is most appropriate for the current input. Second, it is **uncertainty aware**: it uses the entropy of its expert-probability distribution to decide whether that routing decision is reliable. At inference time, GEMINUS computes a probability distribution over the scene-adaptive experts; if routing uncertainty is below a threshold \(\tau\), it selects the top-ranked specialist, and if uncertainty is above or equal to \(\tau\), it falls back to the Global Expert [2507.14456].

This produces a hard-switching policy rather than a weighted average over all experts. In the paper’s interpretation, specialization is invoked when the scene is clearly identifiable, while the globally trained planner is retained as a robustness prior for ambiguous cases [2507.14456].

## 3. Representation, routing, and optimization

GEMINUS is built on a TCP-like single-expert backbone. A **ResNet34** pretrained on ImageNet encodes the front-view image into a **1000-dimensional image feature**, and a measurement encoder maps
\[
m = [v, c, x_g, y_g]
\]
to a **128-dimensional measurement feature**. These are concatenated into a joint feature representation, passed through linear layers to produce a **256-dimensional latent feature**, and then decoded by a GRU that autoregressively predicts the future waypoints [2507.14456].

The paper places this within a generic mixture-of-experts formulation:
\[
p_{\theta}(Y \mid X) = \sum_{k=1}^{K} q_{\theta}(Z=k \mid X)\cdot m_{\theta}(Y \mid Z=k, X).
\]
GEMINUS instantiates that idea with explicit scenario supervision and entropy-based routing uncertainty rather than a generic unsupervised gate [2507.14456].

For scenario-aware routing, the router predicts expert probabilities
\[
P(x) = [p_1, p_2, \dots, p_N],
\]
and is trained with a cross-entropy objective,
\[
\mathcal{L}_{\text{scenario}} = - \sum_{i=1}^{|E|} \mathbb{1}(h(s_m)=i)\log p_i(x),
\]
where \(h\) maps each scenario category to its designated expert. For uncertainty estimation, GEMINUS computes the Shannon entropy
\[
H(P(x)) = - \sum_{i=1}^{N} p_i \log(p_i)
\]
and normalizes it as
\[
U(x) = \frac{-\sum_{i=1}^{N} p_i \log(p_i)}{\log(N)}.
\]
Low \(U(x)\) indicates confident routing, and high \(U(x)\) triggers fallback to the Global Expert [2507.14456].

The global planner is trained with a trajectory imitation loss,
\[
\mathcal{L}_{\text{traj\_global}} = \sum_{t=1}^{K} \left\| w_t - \hat{w}_t \right\|_1,
\]
together with a feature alignment term
\[
\mathcal{L}_{F\_\text{global}} = \left\| j_{\text{global}} - j_{\text{expert}} \right\|_2
\]
and a value alignment term
\[
\mathcal{L}_{V\_\text{global}} = \left\| v_{\text{global}} - v_{\text{expert}} \right\|_2^2.
\]
Scene-adaptive experts use the analogous losses, but only the active expert receives gradient for a given sample. An auxiliary speed head is trained with an \(L_1\) speed loss. The full objective is a weighted sum of the global, adaptive, routing, and speed terms [2507.14456].

## 4. Data regime and implementation details

Training uses the **official Bench2Drive training dataset**, specifically the **base dataset (1000 clips)**, with **950 training clips** and **50 open-loop validation clips**. Each clip spans about **150 meters** and is associated with a specific traffic scenario. For closed-loop evaluation, the system is tested on **220 routes** organized into **44 interactive scenarios**, with **5 routes per scenario** [2507.14456].

The predefined scenario partition is not discovered automatically; it is imposed from Bench2Drive’s scenario definitions. This is a defining methodological choice. It gives the router explicit supervisory targets, but it also means that GEMINUS depends on an externally specified scenario taxonomy rather than latent expert discovery [2507.14456].

Implementation parameters are reported explicitly. The RGB image resolution is \(900 \times 256\). The uncertainty threshold is set to
\[
\tau = 0.5.
\]
The PID controller uses TransFuser-tuned parameters: longitudinal \(K_P = 5.0\), \(K_I = 0.5\), \(K_D = 1.0\), and lateral \(K_P = 0.75\), \(K_I = 0.75\), \(K_D = 0.3\). Optimization uses **Adam** with initial learning rate \(1\times10^{-4}\), weight decay \(1\times10^{-7}\), batch size **96**, and **32** training epochs, with the learning rate multiplied by \(0.5\) after \(30\) epochs. Training is reported on a single **NVIDIA GeForce RTX 4090** [2507.14456].

## 5. Empirical performance on Bench2Drive

The main empirical claim is that GEMINUS achieves **state-of-the-art performance in Driving Score and Success Rate** on the Bench2Drive closed-loop benchmark while using only **front-camera monocular input**. The reported results are:

| Method | Input | Avg. L2 \(\downarrow\) | Driving Score \(\uparrow\) | Success Rate \(\uparrow\) |
|---|---:|---:|---:|---:|
| **TCP-traj\*** | Front camera | 1.70 | 59.90 | 30.00 |
| **DriveTrans** | 6 cameras | 0.62 | 63.46 | 35.01 |
| **GEMINUS\*** | Front camera | 1.60 | 65.39 | 37.73 |

Against the strongest monocular baseline reported in the paper, **TCP-traj\***, GEMINUS improves Driving Score from **59.90** to **65.39**, Success Rate from **30.00** to **37.73**, and Avg. L2 from **1.70** to **1.60**. The paper summarizes these as a **+9.17\%** gain in Driving Score, a **+25.77\%** gain in Success Rate, and a **5.88\% reduction** in Avg. L2 [2507.14456].

The MultiAbility analysis is more differentiated. GEMINUS reports **11.11** on Merging, **37.50** on Overtaking, **55.00** on Emergency Brake, **40.00** on Give Way, and **45.26** on Traffic Sign, for a **MultiAbility-Mean of 37.77**. This is not the best overall MultiAbility-Mean in the comparison table—**DriveAdapter** reports **42.08**, and **DriveTrans** reports **38.60**—but GEMINUS is notably strong in **Overtaking** and **Emergency Brake**, and its closed-loop Driving Score and Success Rate remain the leading metrics claimed in the paper [2507.14456].

A plausible implication is that the architecture improves route completion and closed-loop robustness more strongly than it improves the average of the five decomposed ability scores. The paper itself emphasizes closed-loop competence over open-loop trajectory error as the more meaningful operational criterion [2507.14456].

## 6. Ablation results, interpretation, and limitations

The ablation study is central to the paper’s argument because it separates the effects of generic MoE, scenario-aware specialization, and uncertainty-aware fallback. The reported comparisons are:

| Variant | Driving Score | Success Rate | MultiAbility |
|---|---:|---:|---:|
| **SingleExpert-E2E** | 60.73 | 30.91 | 31.63 |
| **VanillaMoE-E2E** | 59.23 | 29.09 | 32.05 |
| **ScenarioMoE-E2E** | 62.38 | 32.27 | 34.46 |
| **GEMINUS** | 65.39 | 37.73 | 37.77 |

These results support three specific conclusions. First, **generic MoE alone does not help**: **VanillaMoE-E2E** is worse than the single-expert baseline in Driving Score and Success Rate. Second, **scenario-aware specialization helps**: **ScenarioMoE-E2E** improves over **SingleExpert-E2E**. Third, **the Global Expert plus uncertainty-aware routing contributes the largest robustness gain**, especially in Success Rate. Relative to **ScenarioMoE-E2E**, GEMINUS increases Driving Score by **+4.83\%**, Success Rate by **+22.06\%**, and MultiAbility by **+19.41\%**. Relative to **SingleExpert-E2E**, the paper highlights improvements of **7.67\% in Driving Score**, **22.06\% in Success Rate**, and **19.41\% in MultiAbility-Mean** [2507.14456].

The threshold study shows that performance peaks at
\[
\tau = 0.5,
\]
with weaker performance at both smaller and larger thresholds. The paper interprets this as a balance point: too small a threshold causes overuse of the global expert and loss of specialization, whereas too large a threshold overtrusts specialists under ambiguity [2507.14456].

The router analysis shows substantial heterogeneity across scenarios. Overall routing accuracy is **68.06\%**, but it varies from **91.35\%** for **Overtaking** and **90.45\%** for **Traffic Sign** to **32.85\%** for **Merging**, **54.03\%** for **Emergency Brake**, and only **2.87\%** for **Give Way**. Expert utilization follows the same asymmetry: the **Traffic Sign Expert** is used **49.73\%** of the time, the **Overtaking Expert** **19.22\%**, the **Emergency Brake Expert** **16.08\%**, the **Merging Expert** **8.44\%**, the **Global Expert** **6.29\%**, and the **Give Way Expert** only **0.23\%** [2507.14456].

The paper attributes the weakness on **Give Way** to two factors stated explicitly: **data imbalance**, because Give Way comprises only **3.16\%** of training and **4.00\%** of validation, and the limitations of **monocular sensing**, because some Give Way situations require awareness of rear-approaching vehicles outside the front camera’s field of view. More broadly, several limitations are built into the method’s design. It depends on predefined scenario categories; routing can still fail under confident misclassification; and its strongest evidence is currently simulator-based rather than real-world. This suggests that GEMINUS should be understood not as a general solution to expert routing in autonomous driving, but as a structured demonstration that, in this benchmark regime, the combination of explicit scene specialization and uncertainty-triggered fallback is more effective than either a single global planner or a generic MoE gate [2507.14456].

Source: https://www.emergentmind.com/topics/geminus