Future-Guided Trajectory Refinement

• Future-Guided Trajectory Refinement (FGTR) is a technique where a coarse future hypothesis serves as a guidance signal to refine trajectories.
• It leverages proposal-specific context and residual correction mechanisms to improve prediction accuracy in dynamic environments.
• FGTR methods enhance motion forecasting and planning by integrating future predictions into iterative, context-aware refinement processes.

Searching arXiv for FGTR-related motion forecasting and refinement papers. Future-Guided Trajectory Refinement (FGTR) denotes a class of trajectory prediction and planning methods in which an initial future hypothesis is generated first and then used as guidance for a later correction stage. In the narrowest sense, the term is explicitly used in ResWorld, where a Future-Guided Trajectory Refinement (FGTR) module refines a prior ego trajectory by interacting waypoint queries with predicted future BEV features (Zhang et al., 11 Feb 2026). In a broader literature-level sense, FGTR refers to coarse-to-fine, future-conditioned refinement pipelines in which predicted futures are not treated only as terminal outputs, but are reused to retrieve context, structure interactions, or drive residual correction. This broader family includes proposal-conditioned scene refinement in R-Pred (Choi et al., 2022), scenario-adaptive future-guided refinement in SmartRefine (Zhou et al., 2024), topology-aware iterative refinement in SRefiner (Xiao et al., 6 Jul 2025), and several adjacent architectures that refine latent features or dense trajectories from explicit future cues rather than from history alone.

1. Definition and conceptual scope

FGTR is best defined by two coupled properties. First, a model produces a coarse future representation before final prediction. Second, that coarse future is reused as a guidance signal for a later stage that improves trajectories, features, or planning outputs. In ResWorld, the guidance object is a prior trajectory predicted from current BEV features, and the refinement stage uses it as reference points for deformable attention over future BEV features (Zhang et al., 11 Feb 2026). In SmartRefine, coarse trajectory hypotheses are used to choose anchors, retrieve “critical” context, and decide whether further refinement is worthwhile (Zhou et al., 2024). In R-Pred, each trajectory proposal conditions local scene pooling and proposal-level interaction selection in the second stage (Choi et al., 2022).

This suggests two useful readings of FGTR. A strict reading reserves the term for explicit trajectory-space refinement after a first-pass future is available. A broader reading includes methods in which future hypotheses refine latent representations, interaction structure, or decoder state, provided that predicted futures actively guide later computation. DGFNet, for example, is described as a one-shot future-guided feature refinement method rather than a full iterative trajectory refinement framework, because predicted future trajectories are filtered by estimated difficulty and then reused to refine latent agent features before final prediction (Xin et al., 2024). FINet similarly introduces potential future trajectories into scene encoding and then refines them temporally, making it a closely related future-guided refinement method rather than a canonical iterative output-space refiner (Li et al., 9 Mar 2025).

A central distinction within the FGTR family is therefore what is refined. Some methods refine dense future coordinates directly; some refine proposal features; some refine scene representations; some refine ego plans through world-model interaction. The most explicit trajectory-space formulations include residual updates such as

$$\mathbf{s}_f^i=\mathbf{s}_f^{i-1}+\delta\mathbf{s}_f^i$$

in SmartRefine (Zhou et al., 2024) and

$Y_l = \varphi(F_l) + Y_{l-1}$

in SRefiner (Xiao et al., 6 Jul 2025). By contrast, DGFNet uses intermediate future trajectories to refine features before final prediction, so its relation to FGTR is partial rather than exact (Xin et al., 2024).

2. Canonical pipeline and mathematical pattern

Across the literature, FGTR usually follows a two-stage or multi-stage structure. A backbone first predicts a future hypothesis, typically multimodal. A refinement module then conditions on that future hypothesis to gather additional context or compute corrections. The cleanest canonical form appears in trajectory-refinement methods that operate on offsets added to coarse futures. SRefiner formalizes this as

$$Y_0 = \mathcal{F}(X, L), \qquad Y = Y_0 + \mathcal{R}(Y_0, L),$$

where $Y_0$ is the coarse multimodal future produced by a base forecasting model and $\mathcal{R}$ predicts an offset on top of it (Xiao et al., 6 Jul 2025). SmartRefine uses the same residual logic,

$$(\delta\mathbf{s}_f^{i}, \mathbf{h}_f^{i}, \mathbf{p})=f_r(\mathbf{h}_f^{i-1}, \mathbf{s}_f^{i-1}, \mathbf{c}^{i-1}), \qquad \mathbf{s}_f^i=\mathbf{s}_f^{i-1}+\delta\mathbf{s}_f^i,$$

with the additional feature that context $\mathbf{c}^{i-1}$ is itself selected according to the current future hypothesis (Zhou et al., 2024).

The future guidance signal can take several forms. In R-Pred, it is an explicit set of multimodal trajectory proposals $\hat{\mathbf{y}}_1^m$, each of which drives proposal-specific scene pooling and proposal-level interaction reasoning (Choi et al., 2022). In SmartRefine, it is a set of anchors sampled along the predicted future path, with anchor-centric context retrieved by an adaptive radius

$$R_{i,v}=\mathcal{F}(i)\cdot v, \qquad \mathcal{F}(i)=\beta\left(\frac{1}{2}\right)^{i-1},$$

clipped to $[R_{\min},R_{\max}]$ (Zhou et al., 2024). In ResWorld, the future guidance is a prior trajectory

$Y_l = \varphi(F_l) + Y_{l-1}$0

which becomes the reference geometry for refinement against predicted future BEV features (Zhang et al., 11 Feb 2026).

A minimal abstract FGTR template is therefore:

1. produce a coarse future hypothesis;
2. use that hypothesis to determine where to look or whom to interact with;
3. refine features or trajectories using the selected future-conditioned information;
4. decode a final future, often with residual correction.

This suggests that FGTR is less a single architecture than a design principle: future predictions become internal conditioning variables for later refinement.

3. Principal refinement mechanisms

Several distinct mechanisms recur in FGTR-like work.

A first mechanism is future-conditioned scene retrieval. R-Pred’s Tube-Query Scene Attention (TQSA) pools scene vectors only within a radius $Y_l = \varphi(F_l) + Y_{l-1}$1 around any waypoint of a candidate proposal and then attends to those pooled features using the proposal feature as query (Choi et al., 2022). This makes scene encoding proposal-specific rather than global. SmartRefine uses a related but more explicitly adaptive strategy: predicted future trajectories define anchors, local map and agent context is retrieved around those anchors, and the retrieved context is encoded in anchor-centric coordinates (Zhou et al., 2024). Both methods instantiate the same FGTR premise: once a future hypothesis exists, context should be gathered along that anticipated path rather than only around the current state.

A second mechanism is future-conditioned interaction modeling. R-Pred’s Proposal-Level Interaction Attention (PIA) constructs a proposal group for the target by selecting neighboring proposals that are close in future trajectory space under a confidence threshold and a distance threshold, using

$Y_l = \varphi(F_l) + Y_{l-1}$2

as proposal distance (Choi et al., 2022). SRefiner replaces standard implicit interaction with explicit future-induced topology. It computes a same-time closest-approach relation

$Y_l = \varphi(F_l) + Y_{l-1}$3

defines “soft intersection points,” and constructs soft-braid features that include both agents’ velocities, accelerations, relative distance, and relative orientation at that time (Xiao et al., 6 Jul 2025). This makes the future guidance signal not just a latent token but a structured relational descriptor.

A third mechanism is future-aware world-model querying. ResWorld’s FGTR applies deformable attention between waypoint queries and future BEV features using the prior trajectory as reference points:

$Y_l = \varphi(F_l) + Y_{l-1}$4

This allows the planner to inspect whether a prior trajectory will collide with other objects or drive out of the drivable area, and then correct it using anticipated future road conditions (Zhang et al., 11 Feb 2026). Here the coarse future does not retrieve local context tokens from a map encoder; it indexes into a predicted future latent scene.

A fourth mechanism is future-guided coarse-to-fine decomposition. G2LTraj first predicts sparse global key steps

$Y_l = \varphi(F_l) + Y_{l-1}$5

then recursively fills intermediate points between head and tail key steps by learned midpoint generation, making each local prediction conditioned on already predicted farther-future structure (Zhang et al., 2024). MGTraj uses a similar but explicitly multi-granularity hierarchy: a predicted goal seeds an initial proposal, then a recursive refinement network predicts residual corrections from coarse to fine temporal granularities,

$Y_l = \varphi(F_l) + Y_{l-1}$6

This suggests that FGTR can be organized not only as repeated generic refinement, but as a hierarchy over temporal scales (Sun et al., 11 Sep 2025).

4. Future guidance signals

The literature differs most sharply in the form of the future signal.

The most common signal is a coarse predicted trajectory. SmartRefine’s coarse hypotheses $Y_l = \varphi(F_l) + Y_{l-1}$7 determine anchors and critical context (Zhou et al., 2024). R-Pred’s $Y_l = \varphi(F_l) + Y_{l-1}$8 trajectory proposals guide both scene and interaction refinement (Choi et al., 2022). SRefiner’s initial futures $Y_l = \varphi(F_l) + Y_{l-1}$9 are the substrate from which soft-braid topology is computed (Xiao et al., 6 Jul 2025).

A second signal is a goal or endpoint. MGTraj begins from a pretrained multimodal goal predictor,

$$Y_0 = \mathcal{F}(X, L), \qquad Y = Y_0 + \mathcal{R}(Y_0, L),$$0

then linearly interpolates an initial future proposal and recursively refines it (Sun et al., 11 Sep 2025). Goal guidance also underlies earlier goal-guided completion paradigms to which MGTraj is explicitly responding.

A third signal is a sparse set of interpretable keypoints. K-Gen uses an MLLM to generate sparse future keypoints $$Y_0 = \mathcal{F}(X, L), \qquad Y = Y_0 + \mathcal{R}(Y_0, L),$$1 and reasoning tokens $$Y_0 = \mathcal{F}(X, L), \qquad Y = Y_0 + \mathcal{R}(Y_0, L),$$2, constructs a coarse dense trajectory $$Y_0 = \mathcal{F}(X, L), \qquad Y = Y_0 + \mathcal{R}(Y_0, L),$$3 by interpolation, and then refines it with a Transformer-based residual module,

$$Y_0 = \mathcal{F}(X, L), \qquad Y = Y_0 + \mathcal{R}(Y_0, L),$$4

The keypoints are the future guidance; TrajRefiner is the refinement engine (Mu et al., 5 Mar 2026).

A fourth signal is an explicit future scene representation. ResWorld predicts future BEV features with TR-World, then uses the prior trajectory to query those future features in FGTR (Zhang et al., 11 Feb 2026). FINet similarly creates future trajectory tokens early and lets them participate in scene interaction before time-expanded temporal refinement (Li et al., 9 Mar 2025).

A fifth signal is topology derived from future trajectories. SRefiner’s most distinctive move is to turn coarse futures into soft-braid topological structure for both trajectory-trajectory and trajectory-lane interaction:

$$Y_0 = \mathcal{F}(X, L), \qquad Y = Y_0 + \mathcal{R}(Y_0, L),$$5

This pushes FGTR beyond generic future token attention toward structured future-conditioned interaction graphs (Xiao et al., 6 Jul 2025).

These variants suggest that “future guidance” should be understood functionally rather than by datatype. Any future-derived variable can serve as guidance if later computation depends on it in a proposal-specific way.

5. Representative implementations and comparative structure

The table summarizes prominent FGTR or FGTR-like formulations in the supplied literature.

Method	Future guidance	Refinement target
ResWorld	prior trajectory + future BEV features	final ego trajectory (Zhang et al., 11 Feb 2026)
SmartRefine	coarse multimodal future trajectories and anchor-centric context	trajectory coordinates, trajectory embeddings, probabilities (Zhou et al., 2024)
R-Pred	trajectory proposals	proposal features, then refined trajectories (Choi et al., 2022)
SRefiner	coarse futures converted to soft-braid topology	future trajectories by iterative residual correction (Xiao et al., 6 Jul 2025)
DGFNet	difficulty-filtered intermediate futures	latent agent features before final prediction (Xin et al., 2024)
FINet	latent future trajectory tokens	scene encoding and temporally refined futures (Li et al., 9 Mar 2025)
MGTraj	predicted goals and multi-granularity proposals	augmented state trajectory via recursive residual updates (Sun et al., 11 Sep 2025)
K-Gen	sparse keypoints and reasoning	dense trajectory via residual correction (Mu et al., 5 Mar 2026)

A notable structural split runs between predictor-agnostic second-stage refiners and integrated future-aware predictors. SRefiner and SmartRefine are explicit plug-in refinement modules designed to operate on coarse predictions from arbitrary backbones (Xiao et al., 6 Jul 2025, Zhou et al., 2024). R-Pred is also two-stage, though its second stage is more tightly coupled to proposal features (Choi et al., 2022). By contrast, FINet and DGFNet inject future hypotheses back into the network before the final output is produced, making them future-guided but not strictly post-hoc refiners (Li et al., 9 Mar 2025, Xin et al., 2024).

Another split concerns single-agent versus joint multi-agent prediction. SmartRefine is explicitly single-agent and identifies extension to multi-agent joint prediction as future work (Zhou et al., 2024). SRefiner is designed for multi-agent joint trajectories and uses joint winner-takes-all supervision across agents (Xiao et al., 6 Jul 2025). ResWorld addresses ego planning, where the refined output is a single ego trajectory but the future guidance comes from a world model of dynamic objects (Zhang et al., 11 Feb 2026).

6. Training objectives, supervision, and recurrent themes

FGTR methods usually retain standard trajectory losses while changing how context is selected or how hypotheses interact. SmartRefine combines classification, trajectory regression, and quality-score supervision:

$$Y_0 = \mathcal{F}(X, L), \qquad Y = Y_0 + \mathcal{R}(Y_0, L),$$6

with a learned quality score used for adaptive stopping (Zhou et al., 2024). SRefiner trains all refinement iterations with a joint winner-takes-all Huber objective,

$$Y_0 = \mathcal{F}(X, L), \qquad Y = Y_0 + \mathcal{R}(Y_0, L),$$7

and notably does not add classification, topology supervision, collision, drivable-area, or auxiliary losses (Xiao et al., 6 Jul 2025). ResWorld supervises both prior and final trajectories with

$$Y_0 = \mathcal{F}(X, L), \qquad Y = Y_0 + \mathcal{R}(Y_0, L),$$8

using final trajectory loss to backpropagate sparse spatial-temporal supervision into the future BEV representation (Zhang et al., 11 Feb 2026).

A recurrent theme is that some form of auxiliary structure improves future-guided refinement. SmartRefine adds a learned quality score to avoid over-refinement (Zhou et al., 2024). DGFNet uses endpoint-dispersion-based difficulty masking, because future feature interaction alone can worsen performance if noisy futures are allowed to influence later modules (Xin et al., 2024). MGTraj uses velocity prediction as an auxiliary task,

$$Y_0 = \mathcal{F}(X, L), \qquad Y = Y_0 + \mathcal{R}(Y_0, L),$$9

arguing that velocity supervision helps suppress abnormal trajectories and improve dynamics consistency (Sun et al., 11 Sep 2025). K-Gen uses motion loss, final point loss, and kinematic consistency loss in its TrajRefiner (Mu et al., 5 Mar 2026).

This suggests an important generalization: future guidance is powerful but brittle. Many methods therefore include explicit mechanisms to control when or how future hypotheses are trusted.

7. Empirical patterns, benefits, and limitations

The empirical literature supplied here supports four consistent claims.

First, future-guided second-stage refinement improves over coarse predictors. SmartRefine reports consistent gains across several backbones on Argoverse and Argoverse 2 (Zhou et al., 2024). R-Pred reports significant performance improvements over the single-stage baseline and state-of-the-art performance in some categories (Choi et al., 2022). SRefiner shows substantial gains over multiple backbones and strong refinement-stage baselines on Argoverse v2 and INTERACTION, with full soft-braid topology outperforming “No Topology” and hard braid topology (Xiao et al., 6 Jul 2025). ResWorld’s FGTR reduces average L2 and collision rate relative to its baseline and improves further when combined with TR-World (Zhang et al., 11 Feb 2026).

Second, proposal-specific context beats global or fixed context. SmartRefine’s anchor-centric encoding outperforms fixed agent-centric encoding across all tested backbones (Zhou et al., 2024). R-Pred’s TQSA and PIA explicitly use proposal geometry to select relevant scene and interaction context (Choi et al., 2022). SRefiner’s ablations show that recomputing topology from the latest refined futures matters, so future-conditioned interaction structure should not be frozen from the initial proposal (Xiao et al., 6 Jul 2025).

Third, the form of future guidance matters. SRefiner’s results support the paper’s claim that explicit topological structure is more useful than standard implicit interaction, and that soft topology is more expressive than binary crossing topology (Xiao et al., 6 Jul 2025). DGFNet’s ablation shows that future feature interaction without difficulty masking hurts performance, whereas future feature interaction plus difficulty masking helps, implying that noisy future guidance can be harmful (Xin et al., 2024). K-Gen’s TrajRefiner ablation shows that interpolation alone is much worse than learned residual refinement, and that relative coordinate encoding, kinematic consistency, and final point loss all matter (Mu et al., 5 Mar 2026).

Fourth, refinement is not universally beneficial unless controlled. SmartRefine explicitly argues that some scenarios gain little or can even be harmed by over-refinement, motivating scenario-adaptive stopping with a learned quality score (Zhou et al., 2024). SRefiner’s gains are smaller on stronger backbones such as HPNet, suggesting diminishing returns when the base predictor already models interactions well (Xiao et al., 6 Jul 2025). A plausible implication is that FGTR is most valuable when the first-stage future is informative enough to guide context selection, but still imperfect enough to benefit from correction.

Several limitations recur across the literature. FGTR methods usually assume a meaningful coarse proposal already exists; if the initial hypothesis is catastrophically wrong, future-guided retrieval or topology induction may also be misleading (Zhou et al., 2024, Xiao et al., 6 Jul 2025). Scalability analysis is often incomplete, especially for dense scenes with many candidate interactions (Xiao et al., 6 Jul 2025). Methods differ in whether guidance is available at inference or only during training; for example, some training-time future-aware or reward-guided methods are only partially FGTR in the strict sense. This suggests that the taxonomy of FGTR should keep training-time future-aware supervision distinct from inference-time trajectory refinement.

8. Taxonomic interpretation

A practical encyclopedia-level taxonomy of FGTR can be stated in three tiers.

Canonical FGTR refers to two-stage or multi-stage systems that first predict coarse futures and then explicitly refine them in trajectory or proposal space. ResWorld’s FGTR module, SmartRefine, R-Pred, and SRefiner belong here, though they differ in whether refinement is residual coordinate correction, feature refinement, or waypoint-query updating (Zhang et al., 11 Feb 2026, Zhou et al., 2024, Choi et al., 2022, Xiao et al., 6 Jul 2025).

Extended FGTR includes methods that generate future hypotheses early and reuse them to refine latent representations or scene encoding before final decoding. DGFNet and FINet are the clearest examples (Xin et al., 2024, Li et al., 9 Mar 2025).

Related coarse-to-fine future-guided methods include hierarchical goal-guided and keypoint-guided decoders in which a sparse future representation is progressively densified or corrected. MGTraj, G2LTraj, and K-Gen fit this category particularly well (Sun et al., 11 Sep 2025, Zhang et al., 2024, Mu et al., 5 Mar 2026).

Within this taxonomy, ResWorld is notable because it uses the term Future-Guided Trajectory Refinement explicitly and ties it to world-model supervision. SRefiner is notable because it sharpens the concept by making the future guidance signal an explicit topology-aware interaction structure rather than a generic latent future (Xiao et al., 6 Jul 2025). SmartRefine is notable because it makes refinement adaptive rather than fixed-depth (Zhou et al., 2024). R-Pred is notable because it provides an early and direct proposal-conditioned attention formulation that is mechanistically very close to a textbook FGTR pipeline (Choi et al., 2022).

FGTR, in this literature-level sense, is therefore best understood as a family of methods that convert predicted futures into operational guidance for later motion forecasting or planning. Its core insight is simple: a preliminary future should not be wasted as a mere output; it should become a conditioning object that reshapes context retrieval, interaction modeling, and final trajectory generation.