---
title: HPNet for Dynamic Trajectory Forecasting
url: https://www.emergentmind.com/topics/historical-prediction-network-hpnet
type: topic
---

# HPNet for Dynamic Trajectory Forecasting

Searching arXiv for HPNet-related papers to ground the article in the current literature.
Historical Prediction Network (HPNet) denotes a family of neural architectures built around explicit use of historical predictions to improve temporal consistency in sequential forecasting. In the contemporary trajectory-forecasting literature, HPNet most specifically refers to the dynamic multi-agent motion prediction model introduced in "HPNet: Dynamic Trajectory Forecasting with Historical Prediction Attention" [2404.06351], where current forecasts attend not only to observed agent and map context but also to embeddings of previous forecasts. A distinct later usage appears in infrastructure-enabled freeway work-zone safety, where HPNet is adapted for roadside sensing and embedded in a Predictive Digital Twin pipeline for multimodal trajectory prediction and proactive warning generation [2508.06544]. The acronym should be distinguished from the earlier "Hierarchical Prediction Network" for spatiotemporal video prediction in visual-cortex-inspired analysis-by-synthesis modeling [1901.09002], which shares the same abbreviation but addresses a different problem domain and architectural tradition.

## 1. Terminological scope and historical usage

The term HPNet is used in at least two technically different senses in the cited literature. In autonomous-driving trajectory forecasting, HPNet is a "dynamic trajectory forecasting method" that introduces Historical Prediction Attention (HPA) to encode the relationship between successive predictions and thereby improve both stability and accuracy [2404.06351]. In proactive work-zone safety, the same HPNet is adapted from an ego-centric setting to an infrastructure-mounted sensing setting, where it operates over fused roadside trajectories and Lanelet2 HD map features inside a Digital Twin environment [2508.06544].

An earlier and unrelated use of the acronym appears in the neurally inspired video-prediction paper "A Neurally-Inspired Hierarchical Prediction Network for Spatiotemporal Sequence Learning and Prediction" [1901.09002]. That model concerns hierarchical recurrent prediction in the visual cortical hierarchy, not road-agent motion forecasting. The coexistence of these usages creates a potential ambiguity in citations and literature searches. In current autonomous-driving contexts, "HPNet" usually refers to the Historical Prediction Attention model rather than the hierarchical predictive-coding video model. This suggests that precise disambiguation by title or arXiv identifier is necessary in scholarly writing.

## 2. Problem setting in dynamic trajectory forecasting

HPNet in the trajectory-forecasting sense is motivated by a critique of the "static paradigm" in which a predictor uses a fixed-duration historical window and generates each forecast independently at adjacent time steps [2404.06351]. Because neighboring forecasting times share heavily overlapping observations, independent prediction can yield temporal inconsistency: overlapping trajectory segments may disagree, mode identities may flip, and downstream planning can be destabilized. HPNet reformulates forecasting as a dynamic process in which successive predictions are explicitly correlated.

The formal setting in [2404.06351] considers a scene with $N$ road agents and an HD map $M$. For agent $n$, the observed state sequence up to time $t$ is denoted $S_{1:t}^{(n)}$, and the objective is to predict multimodal futures $Y_{t+1:t+T}^{(n)}$ with $K$ modes and associated scores. The observed states are given as
$$
S_{1:t}^{(n)} = \{(p_x^{i,n}, p_y^{i,n}, \theta^{i,n}, v_x^{i,n}, v_y^{i,n}, c_a^{i,n})\}_{i=1}^t,
$$
while future trajectories are
$$
Y_{t+1:t+T}^{(n)} = \{Y^{(n,k)}_{t+1:t+T}\}_{k=1}^K,
$$
with $y_{t+\tau}^{(n,k)} \in \mathbb{R}^2$ [2404.06351].

The work-zone adaptation preserves this core predictive objective but changes the sensing regime and deployment context. There, HPNet consumes fused trajectories from roadside LiDAR, radar, and camera streams at $10$ Hz, together with Lanelet2 HD map features derived from an OpenDRIVE-based co-simulation environment, to produce multimodal forecasts for freeway work-zone interactions [2508.06544]. The motivation is that work zones induce lane closures, tapers, speed changes, and constrained buffers that create irregular maneuvers such as merges and lateral compression, making temporally coherent trajectory forecasting operationally important for proactive safety.

## 3. Architectural organization

The core HPNet architecture in [2404.06351] comprises three main stages: spatio-temporal context encoding, Triple Factorized Attention (TFA), and multimodal decoding with refinement. Agent features, map features, and relative spatio-temporal edge features are first embedded into latent spaces. Learnable mode queries $q_{t,n,k}$ then aggregate temporal agent history and spatial lane context to produce initial prediction embeddings
$$
P_{t,n,k} = q_{t,n,k}^T + q_{t,n,k}^S \in \mathbb{R}^D
$$
[2404.06351].

The TFA module is the architectural core. It applies attention in three factorized stages rather than using one joint attention over agents, time, and modes. Agent Attention (AA) models inter-agent interaction under the same mode and time. Historical Prediction Attention (HPA) links the current prediction embedding to historical prediction embeddings for the same agent and mode. Mode Attention (MA) models interactions across multimodal hypotheses. The prescribed order is AA $\rightarrow$ HPA $\rightarrow$ MA, and the full TFA block is repeated $N_{\text{attn}}=2$ times for Argoverse and $3$ times for INTERACTION [2404.06351].

The decoder is also two-stage. Proposal decoding maps the mode-conditioned latent representation to an initial trajectory:
$$
L_{t,n,k}^1 = \mathrm{MLP}(P_{t,n,k}^{M}) \in \mathbb{R}^{F \times 2}.
$$
These proposal trajectories are then re-encoded into queries, passed through the same attention pipeline again, and refined via residual prediction and mode probability estimation. Final trajectories are obtained as
$$
L_{t,n,k}^2 = L_{t,n,k}^1 + \Delta L_{t,n,k}
$$
[2404.06351].

The work-zone adaptation in [2508.06544] describes a closely related but deployment-oriented variant. Inputs include agent motion sequences consisting of position, heading, velocity, and static attributes, embedded as $E_a$; map features from Lanelet2 lane boundaries and centerlines, embedded as $E_m$; and relative features, embedded as $E_e$. The decoder is explicitly described as hierarchical and two-stage, with a proposal MLP `traj_propose`, anchor extraction at the middle predicted step $F//2$, additional temporal, spatial, and mode attention over anchor embeddings, and a refinement MLP `traj_refine` producing corrections before transformation back to global coordinates [2508.06544]. A plausible implication is that the work-zone version emphasizes engineering integration with map-constrained infrastructure sensing while preserving the temporal-consistency mechanism central to HPNet.

## 4. Historical Prediction Attention and Triple Factorized Attention

Historical Prediction Attention is the defining mechanism of HPNet. Its purpose is to allow current prediction embeddings to attend to historical prediction embeddings rather than relying solely on raw observed history. In [2404.06351], for agent $n$ at time $t$ and mode $k$, with temporal span $I_2$, HPA is written as
$$
P_{t,n,k}^{HP} = \mathrm{MHA}\big(P_{t,n,k}^{A}, [P_{t-I_2 \dots t,n,k}^{A}, E_e], [P_{t-I_2 \dots t,n,k}^{A}, E_e]\big).
$$
This coupling is designed to encode the dynamic relationship between successive predictions, so that overlapping predicted segments remain consistent when appropriate, while still permitting changes in the near-term trajectory if the road situation warrants it [2404.06351].

A central claim of the method is that HPA extends the effective temporal receptive field "for free" by reusing historical prediction embeddings. Without HPA, temporal attention over raw frames is limited to $[t-I_1, t]$. With HPA, historical prediction embeddings already summarize earlier observation windows, so when $I_2 = I_1$, the effective range becomes $[t-2I_1, t]$ without additional attention over raw frames [2404.06351]. This is an architectural rather than merely algorithmic modification: it changes what constitutes temporal context by incorporating prior forecast states into the current reasoning process.

The remaining factors in TFA perform complementary roles. Agent Attention is
$$
P_{t,n,k}^{A} = \mathrm{MHA}\big(P_{t,n,k}, [P_{t,n',k}, E_e]_{n' \in N(n)}, [P_{t,n',k}, E_e]_{n' \in N(n)}\big),
$$
capturing inter-agent interaction among neighboring agents [2404.06351]. Mode Attention is
$$
P_{t,n,k}^{M} = \mathrm{MHA}\big(P_{t,n,k}^{HP}, [P_{t,n,1 \dots K}^{HP}, E_e], [P_{t,n,1 \dots K}^{HP}, E_e]\big),
$$
promoting diversity and reducing redundancy among multimodal hypotheses [2404.06351].

The work-zone paper presents the same idea in an implementation-oriented vocabulary. It describes "learnable mode tokens" as queries throughout the attention stack, temporal attention over historical dynamics, spatial attention over map vectors, and a "historic prediction attention" sublayer operating across the evolving predicted trajectory sequence to ensure temporal coherence [2508.06544]. It states the generic scaled dot-product attention equation
$$
\mathrm{Attn}(Q, K, V) = \mathrm{softmax}\left(\frac{QK^\top}{\sqrt{d_k}}\right)V,
$$
and specializes the historic prediction attention context as
$$
\alpha_t^{(k)} = \mathrm{softmax}\left(\frac{(q^{(k)}W_Q)(h_t^{(k)}W_K)^\top}{\sqrt{d_k}}\right), \quad
c^{(k)} = \sum_{t=1}^{T_{\text{pred}}} \alpha_t^{(k)} (h_t^{(k)}W_V),
$$
where $h_t^{(k)}$ denotes intermediate predicted embeddings at future step $t$ for mode $k$ [2508.06544]. The paper does not specify causal masking or gating, but it states that the purpose is to "ensure temporal coherence by capturing dependencies within predicted trajectories."

## 5. Training objectives, datasets, and empirical performance

The training objective in [2404.06351] depends on whether prediction is marginal or joint. For marginal prediction, a winner-takes-all mode is selected by endpoint displacement:
$$
k_{t,n} = \arg\min_{k \in [1,K]} \|l_{t+F,n,k}^1 - g_{t+F,n}\|_2.
$$
Huber losses are applied to both proposal and refined trajectories, and a cross-entropy loss is applied to mode probabilities. The total loss is
$$
\mathcal{L} = \frac{1}{TN} \sum_{t=-T+1}^{0} \sum_{n=1}^{N}
\big(L_{reg1}^{t,n} + L_{reg2}^{t,n} + L_{cls}^{t,n}\big)
$$
[2404.06351]. For joint prediction on INTERACTION, one mode is selected for the whole scene by joint endpoint displacement, and the loss aggregates across all agents [2404.06351].

Implementation details are reported with relatively high specificity. On Argoverse, HPNet uses $1$ Spatio-Temporal Attention layer, $2$ TFA layers, radius $R=50$, $I_1=I_2=20$, dropout $0.1$, about $4.1$M parameters, AdamW, batch size $16$, weight decay $1e^{-4}$, cosine learning-rate schedule, initial learning rate $5e^{-4}$, and $64$ epochs on $8 \times$ RTX 4090 GPUs [2404.06351]. On INTERACTION, it uses $1$ Spatio-Temporal Attention layer, $3$ TFA layers, radius $R=80$, $I_1=I_2=10$, about $5.3$M parameters, initial learning rate $3e^{-4}$, and the same optimizer family and epoch count [2404.06351]. Data augmentation includes horizontal flip $(0.5)$ and lane occlusion $(0.2)$ for both datasets, plus agent occlusion $(0.05)$ for Argoverse [2404.06351].

Reported results place HPNet at state of the art in the cited benchmarks.

| Dataset / Setting | Metrics reported for HPNet |
|---|---|
| Argoverse, single model | b-minFDE 1.7375; minFDE 1.0986; MR 0.1067; minADE 0.7612 |
| Argoverse, ensembled | b-minFDE 1.6768; minFDE 1.0856; MR 0.1075; minADE 0.7478 |
| INTERACTION, single model | minJointFDE 0.8231; minJointADE 0.2548 |

On Argoverse, the single model improves over GANet by $0.052$ b-minFDE, $0.062$ minFDE, and $0.045$ minADE [2404.06351]. On INTERACTION, it improves over FJMP by $-0.099$ FDE and $-0.020$ ADE [2404.06351]. The work-zone adaptation reports the same benchmark values for Argoverse and INTERACTION under its own evaluation protocol and further reports lower errors on its infrastructure-enabled work-zone dataset: minJointFDE $= 0.3228$ m and minJointADE $= 0.1327$ m, with $K=6$ modes [2508.06544]. The authors attribute these lower errors partly to reduced behavioral variability in simulation and partly to the infrastructure vantage with fewer occlusions [2508.06544].

The work-zone paper also reports training dynamics: HPNet was trained for $60$ epochs with Adam, initial learning rate $5e^{-4}$, decayed by a factor of $1e^{-4}$ after $4$ warmup epochs; training loss drops "from over 37 down to approximately 0.39" in the first $10$ epochs and then to a final value of $0.06$, while validation loss decreases "from an initial 31 to 0.08 by epoch 60" [2508.06544]. Because that paper states that the exact loss is not explicitly formalized, these convergence figures should be read as empirical training observations rather than as direct evidence for a precisely specified optimization functional.

## 6. Digital Twin deployment and work-zone safety application

In [2508.06544], HPNet is integrated into an infrastructure-enabled Predictive Digital Twin for proactive freeway work-zone safety. The system combines roadside LiDAR, radar, and camera streams at $10$ Hz with HD maps and a co-simulation environment built from SUMO and CARLA. The pipeline is described as comprising Design, Co-Simulation, Prediction Model Training, and Proactive Warning Generation [2508.06544].

The data and map-processing stack is explicitly heterogeneous. SUMO provides traffic dynamics, including the sublane model SL2015 with `--lateral-resolution` set to $0.25$ m and parameters such as `lcSublane`, `lcAssertive`, `lcImpatience`, `lcAccelLat`, `latAlignment`, and `lcMaxSpeedLatStanding`, which are said to be crucial in taper zones [2508.06544]. CARLA provides the $3$D environment and sensor simulation, ingesting OpenDRIVE `map.xodr`, while SUMO uses `net.xml` converted via `netconvert`, and OpenDRIVE is converted to Lanelet2 by CommonRoad Scenario Designer so that HPNet can operate over a vectorized lane graph [2508.06544]. The dataset follows INTERACTION format; each CSV corresponds to a $4$ s scenario with $40$ frames, about $18$--$22$ vehicles, and roughly $800$ rows per file, and a total of $20{,}000$ s of simulation yields $5{,}000$ cases, approximately $5.56$ hours [2508.06544].

The predictive output is converted into a conflict-warning signal through a bounding-box probabilistic model. For vehicle $i$ at step $k$, with center $(x_i^{(k)}, y_i^{(k)})$, orientation $\psi_i^{(k)}$, length $l$, and width $w$, local points are transformed by
$$
(x_i^{(k)}, y_i^{(k)}) + \mathcal{R}(\psi_i^{(k)}) \cdot (u, v)^\top,
$$
where
$$
\mathcal{R}(\psi)=
\begin{bmatrix}
\cos\psi & -\sin\psi \\
\sin\psi & \cos\psi
\end{bmatrix}
$$
[2508.06544]. For a pair of vehicles $(i,j)$, the minimum pointwise bounding-box distance at step $k$ is
$$
d_{ij}^{(k)} = \min_{a,b} \|p_{ia}^{(k)} - p_{jb}^{(k)}\|_2,
$$
and conflict probability is defined as
$$
P_{ij}^{(k)} = \exp\left(-\frac{d_{ij}^{(k)}}{\lambda}\right)
$$
[2508.06544]. Potential conflicts are identified when $d_{ij}^{(k)} < \delta_d = 7$ m, and warnings are generated if $P_{ij}^{(k)} > \delta_P$ with $\delta_P \approx 0.70$ [2508.06544].

This application situates HPNet beyond benchmark forecasting as a scene-understanding module inside a real-time safety loop. The paper states that the system issues alerts containing vehicle IDs, prediction step, and risk score via a communication gateway for proactive intervention [2508.06544]. A plausible implication is that HPNet’s temporal-consistency mechanism is particularly valuable in this setting because false oscillations in predicted intent would directly propagate into unstable warning policies.

## 7. Relation to prior approaches, strengths, and limitations

HPNet is positioned against methods that either forecast independently at each time step or lack an explicit mechanism for correlating successive predictions. The 2024 paper states that HPNet differs from the static paradigm by explicitly making successive forecasts aware of each other through HPA [2404.06351]. In the work-zone deployment paper, the comparison is expressed more concretely: vanilla LSTMs are characterized as pure sequence encoders, social pooling as fixed-neighborhood interaction models, lane-GCNs as map-graph models without historical predicted-state attention, and Transformers as temporal-attention models without explicit historical-prediction refinement [2508.06544]. In that framing, HPNet’s distinctive contribution is not merely attention per se, but attention over previously formed predictive embeddings.

Ablation results in [2404.06351] provide direct evidence for the contribution of different components. Removing TFA increases b-minFDE from $1.506$ to $1.832$. Removing AA, HPA, or MA degrades b-minFDE by $0.205$, $0.021$, and $0.025$, respectively [2404.06351]. The same study also reports a stability gain: the summed ADE of overlapping segments across successive predictions is reduced from approximately $2.90$ to approximately $2.25$ [2404.06351]. Qualitative analysis indicates that HPNet better preserves motion-goal identity across time steps than a baseline that alternates between incompatible goals such as left-turn and straight [2404.06351].

The principal limitations stated in the sources are likewise specific. HPNet assumes availability of HD maps and reliable agent tracks, uses a fixed prediction horizon, and relies on the quality of relative pose encoding; hyperparameters such as $I_1$, $I_2$, $R_1$, and $R_2$ can affect the balance between stability and reactivity [2404.06351]. Potential failure cases include sudden intent changes with sparse history, occlusions or missed detections, nonstandard map topologies, and large distribution shifts [2404.06351]. Because HPNet does not impose an explicit consistency loss, stability is learned implicitly through attention; in extremely noisy settings, HPA may over-smooth unless mismatched past predictions are downweighted [2404.06351]. The work-zone paper adds practical deployment concerns such as accurate map conversion, potential sensor latency or occlusions, and reduced coverage of rare behaviors in simulation, and identifies real-world deployment and V2X-based cooperative warnings as future work [2508.06544].

From a computational standpoint, the factorized design is intended to reduce complexity relative to joint attention over flattened agents $\times$ time $\times$ modes, and the added overhead from HPA is reported as modest [2404.06351]. On Argoverse with a V100, inference latency is reported as $27.62$ ms for serial processing per time step versus $22.76$ ms for a baseline without HPA, and $92.02$ ms for parallel processing over $20$ steps versus $81.08$ ms without HPA [2404.06351]. This suggests that HPNet trades a limited increase in latency for improved temporal coherence and predictive accuracy.

In summary, Historical Prediction Network in the trajectory-forecasting literature is defined by the use of historical prediction embeddings as first-class conditioning signals. Its characteristic technical signature is the combination of Historical Prediction Attention with Agent Attention and Mode Attention in a Triple Factorized Attention stack, followed by multimodal proposal-and-refinement decoding [2404.06351]. Subsequent work demonstrates that this design can be adapted from ego-centric autonomous-driving benchmarks to infrastructure-enabled Digital Twin systems for work-zone safety, where temporally coherent multi-agent forecasting supports probabilistic early-warning generation [2508.06544]. The acronym’s earlier use for hierarchical video prediction [1901.09002] remains historically relevant but refers to a different research lineage.

Source: https://www.emergentmind.com/topics/historical-prediction-network-hpnet