Hierarchical Interaction Decoder

• Hierarchical Interaction Decoder is a transformer-based module that performs structured social reasoning by decomposing attention into standard and neighbor-context pathways.
• It leverages a dual-pathway cross-attention architecture with a learnable gating mechanism to balance uniform geometric coverage and focused neighbor context.
• Empirical evaluations on highway-ramp scenarios show improved minADE and minFDE metrics, demonstrating robustness without reliance on HD map data.

The Hierarchical Interaction Decoder (HID) is a transformer-based module that performs structured social reasoning through decomposed cross-attention pathways, supporting intention-aligned multimodal prediction in contexts such as vehicle trajectory forecasting. It operates in tandem with a Motion-Aware Encoder (MAE), with both components comprising the GContextFormer architecture, which is designed for interpretable, map-free multimodal sequence modeling. HID distinguishes itself by fusing uniform geometric coverage and salient neighbor context, offering robustness and accuracy without reliance on HD map data (Chen et al., 24 Nov 2025).

1. Conceptual Foundations

The HID receives mode-wise context representations generated by a global context-aware encoder and processes them with respect to local agent observations (such as the trajectories of neighboring vehicles in traffic). Rather than using a single undifferentiated attention mechanism, HID decomposes social reasoning into dual attention pathways: a standard pathway ensuring geometric coverage across all agent-mode pairs, and a neighbor-context-enhanced pathway that integrates salient global neighbor features. This decomposition is mediated by a learnable gating mechanism to maintain an adaptive balance between broad coverage and context focus.

2. Dual-Pathway Cross-Attention Architecture

Each input mode embedding $c_k$ (where $k$ indexes trajectory modes) is processed through two parallel cross-attention modules:

• Standard Pathway employs conventional multi-head dot-product cross-attention: mode queries attend to the complete set of neighbor embeddings, providing uniform geometric coverage and allowing all potential interactions.

$$o_k^{\mathrm{std}} = \mathrm{MultiHead}(W^Q_dc_k,\,W^K_d[y_1,\dots,y_N],\,W^V_d[y_1,\dots,y_N],\,\mathbf{M})$$

Here, $y_i$ is the embedding of neighbor $i$, and $\mathbf{M}$ is a masking matrix to restrict invalid interactions.

• Neighbor-Context-Enhanced Pathway first aggregates a global neighbor context $G^n$ using a bounded scaled additive mechanism:

$$\beta_i = \mathrm{softmax}\left(\frac{1}{\sqrt{d_k}}\tanh(W^Q_ny_i+W^K_ny_i)\right),\qquad G^n = \sum_{i=1}^N \beta_iv^n_i$$

Each mode embedding is then contextually enhanced before attention:

$$c'_k = c_k + \gamma_kG^n,\qquad o_k^{\mathrm{enh}} = \mathrm{MultiHead}(W^Q_dc'_k, K^\mathrm{std}, V^\mathrm{std}, \mathbf{M})$$

The scaling factor $\gamma_k$ is learned per mode via a sigmoid-parameterized affine transformation.

3. Gating Fusion Mechanism

After both pathways produce their respective outputs, a learnable gate $g_k$ is applied to mediate their aggregation: $$g_k = \sigma(w^gc_k + b^g),\qquad o_k = g_k\,o_k^{\mathrm{std}} + (1-g_k)\,o_k^{\mathrm{enh}}$$ This gating enables the decoder to adaptively select between broad coverage and salient interaction, a crucial capability in high-curvature and transition regions of highway-ramp scenes, as demonstrated in empirical analyses (Chen et al., 24 Nov 2025).

4. Training Objectives and Output Heads

The output from HID for each mode $k$ is used by two prediction heads:

• A classification head yields mode probabilities $\hat p_k$ via softmax.
• A regression head produces mode-specific trajectory predictions $\hat Y_k$.

Soft mode labels are computed based on proximity to ground-truth trajectories, supporting cross-entropy mode classification. Regression loss is computed only for the closest mode's output via Smooth-$\ell_1$ objective. The overall loss function is a combination of weighted classification and regression components: $$\mathcal{L} = \lambda_{\mathrm{cls}}\mathcal{L}_{\mathrm{cls}} + \lambda_{\mathrm{reg}}\mathcal{L}_{\mathrm{reg}}$$

5. Empirical Evaluation and Impact

HID, when paired with the MAE as part of GContextFormer, has been empirically shown to outperform transformer-based and graph-based baselines in multimodal trajectory prediction on the TOD-VT highway-ramp dataset. On aggregate metrics (minADE, minFDE, and miss rate), the full stacking of MAE and HID yields the strongest results compared to partial replacements: | Model | minADE (m) | minFDE (m) | MR-2 | MR-3 | CVaR (m) | |----------------|------------|------------|------|------|----------| | TUTR | 0.69 | 1.50 | 0.28 | 0.15 | 4.00 | | G-MAE | 0.65 | 1.27 | 0.25 | 0.13 | 3.40 | | G-HID | 0.71 | 1.52 | 0.29 | 0.16 | 4.05 | | GContextFormer | 0.63 | 1.25 | 0.22 | 0.12 | 3.38 |

Spatial error distribution analyses indicate that the hierarchical fusion is especially effective in complex transition and merging zones, where naive pairwise attention fails due to either mode suppression or overconfigured neighbor bias (Chen et al., 24 Nov 2025).

6. Interpretability and Analysis

Attention visualizations demonstrate that HID ranks neighbors by context saliency, often prioritizing agents that are semantically critical to the target’s intent (lead vehicle, merging neighbor, etc.) for different modes. The gating mechanism modulates the contribution of each pathway per mode, exposing the attribution of predictive reasoning in a transparent fashion.

7. Robustness and Extensibility

HID’s hierarchical structure, with its context-aware fusion and gating, offers robust prediction in settings with missing or corrupted map data and partial neighbor detection. The modular framework supports extension to domains beyond vehicle prediction, including urban intersection reasoning, multi-robot coordination, and pedestrian forecasting, via adaptation of the mode bank and context encoders. Its architecture allows for late fusion with coarse semantic signals and is resilient to distributional anomalies in agent configuration (Chen et al., 24 Nov 2025).