HPNet for Dynamic Trajectory Forecasting

• HPNet is a dynamic forecasting model that leverages historical prediction embeddings to improve coherence and stability in trajectory prediction.
• It uses Triple Factorized Attention by integrating agent, historical prediction, and mode attention to capture inter-agent interactions and temporal dependencies.
• Applied in autonomous driving and work-zone safety, HPNet adapts to varied sensing modalities while achieving state-of-the-art performance metrics.

Searching arXiv for HPNet-related papers to ground the article in the current literature. Historical Prediction Network (HPNet) denotes a family of neural architectures built around explicit use of historical predictions to improve temporal consistency in sequential forecasting. In the contemporary trajectory-forecasting literature, HPNet most specifically refers to the dynamic multi-agent motion prediction model introduced in "HPNet: Dynamic Trajectory Forecasting with Historical Prediction Attention" (Tang et al., 2024), where current forecasts attend not only to observed agent and map context but also to embeddings of previous forecasts. A distinct later usage appears in infrastructure-enabled freeway work-zone safety, where HPNet is adapted for roadside sensing and embedded in a Predictive Digital Twin pipeline for multimodal trajectory prediction and proactive warning generation (Ahmad et al., 5 Aug 2025). The acronym should be distinguished from the earlier "Hierarchical Prediction Network" for spatiotemporal video prediction in visual-cortex-inspired analysis-by-synthesis modeling (Qiu et al., 2019), which shares the same abbreviation but addresses a different problem domain and architectural tradition.

1. Terminological scope and historical usage

The term HPNet is used in at least two technically different senses in the cited literature. In autonomous-driving trajectory forecasting, HPNet is a "dynamic trajectory forecasting method" that introduces Historical Prediction Attention (HPA) to encode the relationship between successive predictions and thereby improve both stability and accuracy (Tang et al., 2024). In proactive work-zone safety, the same HPNet is adapted from an ego-centric setting to an infrastructure-mounted sensing setting, where it operates over fused roadside trajectories and Lanelet2 HD map features inside a Digital Twin environment (Ahmad et al., 5 Aug 2025).

An earlier and unrelated use of the acronym appears in the neurally inspired video-prediction paper "A Neurally-Inspired Hierarchical Prediction Network for Spatiotemporal Sequence Learning and Prediction" (Qiu et al., 2019). That model concerns hierarchical recurrent prediction in the visual cortical hierarchy, not road-agent motion forecasting. The coexistence of these usages creates a potential ambiguity in citations and literature searches. In current autonomous-driving contexts, "HPNet" usually refers to the Historical Prediction Attention model rather than the hierarchical predictive-coding video model. This suggests that precise disambiguation by title or arXiv identifier is necessary in scholarly writing.

2. Problem setting in dynamic trajectory forecasting

HPNet in the trajectory-forecasting sense is motivated by a critique of the "static paradigm" in which a predictor uses a fixed-duration historical window and generates each forecast independently at adjacent time steps (Tang et al., 2024). Because neighboring forecasting times share heavily overlapping observations, independent prediction can yield temporal inconsistency: overlapping trajectory segments may disagree, mode identities may flip, and downstream planning can be destabilized. HPNet reformulates forecasting as a dynamic process in which successive predictions are explicitly correlated.

The formal setting in (Tang et al., 2024) considers a scene with $N$ road agents and an HD map $M$. For agent $n$, the observed state sequence up to time $t$ is denoted $S_{1:t}^{(n)}$, and the objective is to predict multimodal futures $Y_{t+1:t+T}^{(n)}$ with $K$ modes and associated scores. The observed states are given as

$$S_{1:t}^{(n)} = \{(p_x^{i,n}, p_y^{i,n}, \theta^{i,n}, v_x^{i,n}, v_y^{i,n}, c_a^{i,n})\}_{i=1}^t,$$

while future trajectories are

$$Y_{t+1:t+T}^{(n)} = \{Y^{(n,k)}_{t+1:t+T}\}_{k=1}^K,$$

with $y_{t+\tau}^{(n,k)} \in \mathbb{R}^2$ (Tang et al., 2024).

The work-zone adaptation preserves this core predictive objective but changes the sensing regime and deployment context. There, HPNet consumes fused trajectories from roadside LiDAR, radar, and camera streams at $M$0 Hz, together with Lanelet2 HD map features derived from an OpenDRIVE-based co-simulation environment, to produce multimodal forecasts for freeway work-zone interactions (Ahmad et al., 5 Aug 2025). The motivation is that work zones induce lane closures, tapers, speed changes, and constrained buffers that create irregular maneuvers such as merges and lateral compression, making temporally coherent trajectory forecasting operationally important for proactive safety.

3. Architectural organization

The core HPNet architecture in (Tang et al., 2024) comprises three main stages: spatio-temporal context encoding, Triple Factorized Attention (TFA), and multimodal decoding with refinement. Agent features, map features, and relative spatio-temporal edge features are first embedded into latent spaces. Learnable mode queries $M$1 then aggregate temporal agent history and spatial lane context to produce initial prediction embeddings

$M$2

(Tang et al., 2024).

The TFA module is the architectural core. It applies attention in three factorized stages rather than using one joint attention over agents, time, and modes. Agent Attention (AA) models inter-agent interaction under the same mode and time. Historical Prediction Attention (HPA) links the current prediction embedding to historical prediction embeddings for the same agent and mode. Mode Attention (MA) models interactions across multimodal hypotheses. The prescribed order is AA $M$3 HPA $M$4 MA, and the full TFA block is repeated $M$5 times for Argoverse and $M$6 times for INTERACTION (Tang et al., 2024).

The decoder is also two-stage. Proposal decoding maps the mode-conditioned latent representation to an initial trajectory:

$M$7

These proposal trajectories are then re-encoded into queries, passed through the same attention pipeline again, and refined via residual prediction and mode probability estimation. Final trajectories are obtained as

$M$8

(Tang et al., 2024).

The work-zone adaptation in (Ahmad et al., 5 Aug 2025) describes a closely related but deployment-oriented variant. Inputs include agent motion sequences consisting of position, heading, velocity, and static attributes, embedded as $M$9; map features from Lanelet2 lane boundaries and centerlines, embedded as $n$0; and relative features, embedded as $n$1. The decoder is explicitly described as hierarchical and two-stage, with a proposal MLP traj_propose, anchor extraction at the middle predicted step $n$2, additional temporal, spatial, and mode attention over anchor embeddings, and a refinement MLP traj_refine producing corrections before transformation back to global coordinates (Ahmad et al., 5 Aug 2025). A plausible implication is that the work-zone version emphasizes engineering integration with map-constrained infrastructure sensing while preserving the temporal-consistency mechanism central to HPNet.

4. Historical Prediction Attention and Triple Factorized Attention

Historical Prediction Attention is the defining mechanism of HPNet. Its purpose is to allow current prediction embeddings to attend to historical prediction embeddings rather than relying solely on raw observed history. In (Tang et al., 2024), for agent $n$3 at time $n$4 and mode $n$5, with temporal span $n$6, HPA is written as

$n$7

This coupling is designed to encode the dynamic relationship between successive predictions, so that overlapping predicted segments remain consistent when appropriate, while still permitting changes in the near-term trajectory if the road situation warrants it (Tang et al., 2024).

A central claim of the method is that HPA extends the effective temporal receptive field "for free" by reusing historical prediction embeddings. Without HPA, temporal attention over raw frames is limited to $n$8. With HPA, historical prediction embeddings already summarize earlier observation windows, so when $n$9, the effective range becomes $t$0 without additional attention over raw frames (Tang et al., 2024). This is an architectural rather than merely algorithmic modification: it changes what constitutes temporal context by incorporating prior forecast states into the current reasoning process.

The remaining factors in TFA perform complementary roles. Agent Attention is

$t$1

capturing inter-agent interaction among neighboring agents (Tang et al., 2024). Mode Attention is

$t$2

promoting diversity and reducing redundancy among multimodal hypotheses (Tang et al., 2024).

The work-zone paper presents the same idea in an implementation-oriented vocabulary. It describes "learnable mode tokens" as queries throughout the attention stack, temporal attention over historical dynamics, spatial attention over map vectors, and a "historic prediction attention" sublayer operating across the evolving predicted trajectory sequence to ensure temporal coherence (Ahmad et al., 5 Aug 2025). It states the generic scaled dot-product attention equation

$t$3

and specializes the historic prediction attention context as

$t$4

where $t$5 denotes intermediate predicted embeddings at future step $t$6 for mode $t$7 (Ahmad et al., 5 Aug 2025). The paper does not specify causal masking or gating, but it states that the purpose is to "ensure temporal coherence by capturing dependencies within predicted trajectories."

5. Training objectives, datasets, and empirical performance

The training objective in (Tang et al., 2024) depends on whether prediction is marginal or joint. For marginal prediction, a winner-takes-all mode is selected by endpoint displacement:

$t$8

Huber losses are applied to both proposal and refined trajectories, and a cross-entropy loss is applied to mode probabilities. The total loss is

$t$9

(Tang et al., 2024). For joint prediction on INTERACTION, one mode is selected for the whole scene by joint endpoint displacement, and the loss aggregates across all agents (Tang et al., 2024).

Implementation details are reported with relatively high specificity. On Argoverse, HPNet uses $S_{1:t}^{(n)}$0 Spatio-Temporal Attention layer, $S_{1:t}^{(n)}$1 TFA layers, radius $S_{1:t}^{(n)}$2, $S_{1:t}^{(n)}$3, dropout $S_{1:t}^{(n)}$4, about $S_{1:t}^{(n)}$5M parameters, AdamW, batch size $S_{1:t}^{(n)}$6, weight decay $S_{1:t}^{(n)}$7, cosine learning-rate schedule, initial learning rate $S_{1:t}^{(n)}$8, and $S_{1:t}^{(n)}$9 epochs on $Y_{t+1:t+T}^{(n)}$0 RTX 4090 GPUs (Tang et al., 2024). On INTERACTION, it uses $Y_{t+1:t+T}^{(n)}$1 Spatio-Temporal Attention layer, $Y_{t+1:t+T}^{(n)}$2 TFA layers, radius $Y_{t+1:t+T}^{(n)}$3, $Y_{t+1:t+T}^{(n)}$4, about $Y_{t+1:t+T}^{(n)}$5M parameters, initial learning rate $Y_{t+1:t+T}^{(n)}$6, and the same optimizer family and epoch count (Tang et al., 2024). Data augmentation includes horizontal flip $Y_{t+1:t+T}^{(n)}$7 and lane occlusion $Y_{t+1:t+T}^{(n)}$8 for both datasets, plus agent occlusion $Y_{t+1:t+T}^{(n)}$9 for Argoverse (Tang et al., 2024).

Reported results place HPNet at state of the art in the cited benchmarks.

Dataset / Setting	Metrics reported for HPNet
Argoverse, single model	b-minFDE 1.7375; minFDE 1.0986; MR 0.1067; minADE 0.7612
Argoverse, ensembled	b-minFDE 1.6768; minFDE 1.0856; MR 0.1075; minADE 0.7478
INTERACTION, single model	minJointFDE 0.8231; minJointADE 0.2548

On Argoverse, the single model improves over GANet by $K$0 b-minFDE, $K$1 minFDE, and $K$2 minADE (Tang et al., 2024). On INTERACTION, it improves over FJMP by $K$3 FDE and $K$4 ADE (Tang et al., 2024). The work-zone adaptation reports the same benchmark values for Argoverse and INTERACTION under its own evaluation protocol and further reports lower errors on its infrastructure-enabled work-zone dataset: minJointFDE $K$5 m and minJointADE $K$6 m, with $K$7 modes (Ahmad et al., 5 Aug 2025). The authors attribute these lower errors partly to reduced behavioral variability in simulation and partly to the infrastructure vantage with fewer occlusions (Ahmad et al., 5 Aug 2025).

The work-zone paper also reports training dynamics: HPNet was trained for $K$8 epochs with Adam, initial learning rate $K$9, decayed by a factor of $$S_{1:t}^{(n)} = \{(p_x^{i,n}, p_y^{i,n}, \theta^{i,n}, v_x^{i,n}, v_y^{i,n}, c_a^{i,n})\}_{i=1}^t,$$0 after $$S_{1:t}^{(n)} = \{(p_x^{i,n}, p_y^{i,n}, \theta^{i,n}, v_x^{i,n}, v_y^{i,n}, c_a^{i,n})\}_{i=1}^t,$$1 warmup epochs; training loss drops "from over 37 down to approximately 0.39" in the first $$S_{1:t}^{(n)} = \{(p_x^{i,n}, p_y^{i,n}, \theta^{i,n}, v_x^{i,n}, v_y^{i,n}, c_a^{i,n})\}_{i=1}^t,$$2 epochs and then to a final value of $$S_{1:t}^{(n)} = \{(p_x^{i,n}, p_y^{i,n}, \theta^{i,n}, v_x^{i,n}, v_y^{i,n}, c_a^{i,n})\}_{i=1}^t,$$3, while validation loss decreases "from an initial 31 to 0.08 by epoch 60" (Ahmad et al., 5 Aug 2025). Because that paper states that the exact loss is not explicitly formalized, these convergence figures should be read as empirical training observations rather than as direct evidence for a precisely specified optimization functional.

6. Digital Twin deployment and work-zone safety application

In (Ahmad et al., 5 Aug 2025), HPNet is integrated into an infrastructure-enabled Predictive Digital Twin for proactive freeway work-zone safety. The system combines roadside LiDAR, radar, and camera streams at $$S_{1:t}^{(n)} = \{(p_x^{i,n}, p_y^{i,n}, \theta^{i,n}, v_x^{i,n}, v_y^{i,n}, c_a^{i,n})\}_{i=1}^t,$$4 Hz with HD maps and a co-simulation environment built from SUMO and CARLA. The pipeline is described as comprising Design, Co-Simulation, Prediction Model Training, and Proactive Warning Generation (Ahmad et al., 5 Aug 2025).

The data and map-processing stack is explicitly heterogeneous. SUMO provides traffic dynamics, including the sublane model SL2015 with --lateral-resolution set to $$S_{1:t}^{(n)} = \{(p_x^{i,n}, p_y^{i,n}, \theta^{i,n}, v_x^{i,n}, v_y^{i,n}, c_a^{i,n})\}_{i=1}^t,$$5 m and parameters such as lcSublane, lcAssertive, lcImpatience, lcAccelLat, latAlignment, and lcMaxSpeedLatStanding, which are said to be crucial in taper zones (Ahmad et al., 5 Aug 2025). CARLA provides the $$S_{1:t}^{(n)} = \{(p_x^{i,n}, p_y^{i,n}, \theta^{i,n}, v_x^{i,n}, v_y^{i,n}, c_a^{i,n})\}_{i=1}^t,$$6D environment and sensor simulation, ingesting OpenDRIVE map.xodr, while SUMO uses net.xml converted via netconvert, and OpenDRIVE is converted to Lanelet2 by CommonRoad Scenario Designer so that HPNet can operate over a vectorized lane graph (Ahmad et al., 5 Aug 2025). The dataset follows INTERACTION format; each CSV corresponds to a $$S_{1:t}^{(n)} = \{(p_x^{i,n}, p_y^{i,n}, \theta^{i,n}, v_x^{i,n}, v_y^{i,n}, c_a^{i,n})\}_{i=1}^t,$$7 s scenario with $$S_{1:t}^{(n)} = \{(p_x^{i,n}, p_y^{i,n}, \theta^{i,n}, v_x^{i,n}, v_y^{i,n}, c_a^{i,n})\}_{i=1}^t,$$8 frames, about $$S_{1:t}^{(n)} = \{(p_x^{i,n}, p_y^{i,n}, \theta^{i,n}, v_x^{i,n}, v_y^{i,n}, c_a^{i,n})\}_{i=1}^t,$$9--$$Y_{t+1:t+T}^{(n)} = \{Y^{(n,k)}_{t+1:t+T}\}_{k=1}^K,$$0 vehicles, and roughly $$Y_{t+1:t+T}^{(n)} = \{Y^{(n,k)}_{t+1:t+T}\}_{k=1}^K,$$1 rows per file, and a total of $$Y_{t+1:t+T}^{(n)} = \{Y^{(n,k)}_{t+1:t+T}\}_{k=1}^K,$$2 s of simulation yields $$Y_{t+1:t+T}^{(n)} = \{Y^{(n,k)}_{t+1:t+T}\}_{k=1}^K,$$3 cases, approximately $$Y_{t+1:t+T}^{(n)} = \{Y^{(n,k)}_{t+1:t+T}\}_{k=1}^K,$$4 hours (Ahmad et al., 5 Aug 2025).

The predictive output is converted into a conflict-warning signal through a bounding-box probabilistic model. For vehicle $$Y_{t+1:t+T}^{(n)} = \{Y^{(n,k)}_{t+1:t+T}\}_{k=1}^K,$$5 at step $$Y_{t+1:t+T}^{(n)} = \{Y^{(n,k)}_{t+1:t+T}\}_{k=1}^K,$$6, with center $$Y_{t+1:t+T}^{(n)} = \{Y^{(n,k)}_{t+1:t+T}\}_{k=1}^K,$$7, orientation $$Y_{t+1:t+T}^{(n)} = \{Y^{(n,k)}_{t+1:t+T}\}_{k=1}^K,$$8, length $$Y_{t+1:t+T}^{(n)} = \{Y^{(n,k)}_{t+1:t+T}\}_{k=1}^K,$$9, and width $y_{t+\tau}^{(n,k)} \in \mathbb{R}^2$0, local points are transformed by

$y_{t+\tau}^{(n,k)} \in \mathbb{R}^2$1

where

$y_{t+\tau}^{(n,k)} \in \mathbb{R}^2$2

(Ahmad et al., 5 Aug 2025). For a pair of vehicles $y_{t+\tau}^{(n,k)} \in \mathbb{R}^2$3, the minimum pointwise bounding-box distance at step $y_{t+\tau}^{(n,k)} \in \mathbb{R}^2$4 is

$y_{t+\tau}^{(n,k)} \in \mathbb{R}^2$5

and conflict probability is defined as

$y_{t+\tau}^{(n,k)} \in \mathbb{R}^2$6

(Ahmad et al., 5 Aug 2025). Potential conflicts are identified when $y_{t+\tau}^{(n,k)} \in \mathbb{R}^2$7 m, and warnings are generated if $y_{t+\tau}^{(n,k)} \in \mathbb{R}^2$8 with $y_{t+\tau}^{(n,k)} \in \mathbb{R}^2$9 (Ahmad et al., 5 Aug 2025).

This application situates HPNet beyond benchmark forecasting as a scene-understanding module inside a real-time safety loop. The paper states that the system issues alerts containing vehicle IDs, prediction step, and risk score via a communication gateway for proactive intervention (Ahmad et al., 5 Aug 2025). A plausible implication is that HPNet’s temporal-consistency mechanism is particularly valuable in this setting because false oscillations in predicted intent would directly propagate into unstable warning policies.

7. Relation to prior approaches, strengths, and limitations

HPNet is positioned against methods that either forecast independently at each time step or lack an explicit mechanism for correlating successive predictions. The 2024 paper states that HPNet differs from the static paradigm by explicitly making successive forecasts aware of each other through HPA (Tang et al., 2024). In the work-zone deployment paper, the comparison is expressed more concretely: vanilla LSTMs are characterized as pure sequence encoders, social pooling as fixed-neighborhood interaction models, lane-GCNs as map-graph models without historical predicted-state attention, and Transformers as temporal-attention models without explicit historical-prediction refinement (Ahmad et al., 5 Aug 2025). In that framing, HPNet’s distinctive contribution is not merely attention per se, but attention over previously formed predictive embeddings.

Ablation results in (Tang et al., 2024) provide direct evidence for the contribution of different components. Removing TFA increases b-minFDE from $M$00 to $M$01. Removing AA, HPA, or MA degrades b-minFDE by $M$02, $M$03, and $M$04, respectively (Tang et al., 2024). The same study also reports a stability gain: the summed ADE of overlapping segments across successive predictions is reduced from approximately $M$05 to approximately $M$06 (Tang et al., 2024). Qualitative analysis indicates that HPNet better preserves motion-goal identity across time steps than a baseline that alternates between incompatible goals such as left-turn and straight (Tang et al., 2024).

The principal limitations stated in the sources are likewise specific. HPNet assumes availability of HD maps and reliable agent tracks, uses a fixed prediction horizon, and relies on the quality of relative pose encoding; hyperparameters such as $M$07, $M$08, $M$09, and $M$10 can affect the balance between stability and reactivity (Tang et al., 2024). Potential failure cases include sudden intent changes with sparse history, occlusions or missed detections, nonstandard map topologies, and large distribution shifts (Tang et al., 2024). Because HPNet does not impose an explicit consistency loss, stability is learned implicitly through attention; in extremely noisy settings, HPA may over-smooth unless mismatched past predictions are downweighted (Tang et al., 2024). The work-zone paper adds practical deployment concerns such as accurate map conversion, potential sensor latency or occlusions, and reduced coverage of rare behaviors in simulation, and identifies real-world deployment and V2X-based cooperative warnings as future work (Ahmad et al., 5 Aug 2025).

From a computational standpoint, the factorized design is intended to reduce complexity relative to joint attention over flattened agents $M$11 time $M$12 modes, and the added overhead from HPA is reported as modest (Tang et al., 2024). On Argoverse with a V100, inference latency is reported as $M$13 ms for serial processing per time step versus $M$14 ms for a baseline without HPA, and $M$15 ms for parallel processing over $M$16 steps versus $M$17 ms without HPA (Tang et al., 2024). This suggests that HPNet trades a limited increase in latency for improved temporal coherence and predictive accuracy.

In summary, Historical Prediction Network in the trajectory-forecasting literature is defined by the use of historical prediction embeddings as first-class conditioning signals. Its characteristic technical signature is the combination of Historical Prediction Attention with Agent Attention and Mode Attention in a Triple Factorized Attention stack, followed by multimodal proposal-and-refinement decoding (Tang et al., 2024). Subsequent work demonstrates that this design can be adapted from ego-centric autonomous-driving benchmarks to infrastructure-enabled Digital Twin systems for work-zone safety, where temporally coherent multi-agent forecasting supports probabilistic early-warning generation (Ahmad et al., 5 Aug 2025). The acronym’s earlier use for hierarchical video prediction (Qiu et al., 2019) remains historically relevant but refers to a different research lineage.