Hierarchical Trajectory Search (HTS)

Updated 5 July 2026

Hierarchical Trajectory Search (HTS) is a methodology that organizes search over temporally ordered candidate trajectories using a coarse-to-fine approach.
It is applied in diverse domains such as recommendation systems, diffusion test-time scaling, and hierarchical reinforcement learning, utilizing progressive candidate pruning and dynamic scoring.
HTS replaces uniform exhaustive search with multi-stage processes that reallocate computational resources based on uncertainty and information gain, improving efficiency and planning quality.

Hierarchical Trajectory Search (HTS) denotes a class of coarse-to-fine search procedures that operate over temporally ordered candidate trajectories rather than flat one-shot outputs. In recent arXiv usage, the term appears explicitly in recommendation and diffusion-model test-time scaling, and it is closely aligned with latent-space trajectory planning in hierarchical reinforcement learning. Across these settings, the shared structure is a hierarchy in which nodes correspond to temporally extended candidate states, search width is largest early, and scoring or pruning progressively concentrates computation on promising paths. A related but distinct line of work, HiT-JEPA, adopts the same multi-scale trajectory philosophy for representation learning, but it is not a search system with an explicit multi-stage candidate pruning tree in the classic algorithmic HTS sense (Xia et al., 26 May 2025, Co-Reyes et al., 2018, Xin et al., 22 Dec 2025, Bai et al., 2 Feb 2026, Li et al., 17 Jun 2025).

1. Terminological scope and conceptual core

The most explicit definition of HTS in the provided literature appears in "Hierarchical Tree Search-based User Lifelong Behavior Modeling on LLM" (Xia et al., 26 May 2025). There, the HTS module is the core mechanism that turns lifelong user-behavior modeling into a hierarchical search problem over a sequence of interest states. Each behavior chunk is treated as one step in a user-interest trajectory, and the method uses LLM-generated candidates plus learned process rewards to choose the best interest state at each step.

In diffusion test-time scaling, HTS is defined differently but with the same coarse-to-fine logic. In "dMLLM-TTS: Self-Verified and Efficient Test-Time Scaling for Diffusion Multi-Modal LLMs" (Xin et al., 22 Dec 2025), HTS is an efficient hierarchical search algorithm that adaptively expands and prunes sampling trajectories. In "Prism: Efficient Test-Time Scaling via Hierarchical Search and Self-Verification for Discrete Diffusion LLMs" (Bai et al., 2 Feb 2026), HTS maintains a pool of full diffusion trajectories and dynamically reallocates compute across denoising time, exploring broadly under high uncertainty and pruning as the solution skeleton becomes clearer.

Hierarchical reinforcement learning provides a closely related antecedent. "Self-Consistent Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory Embeddings" (Co-Reyes et al., 2018) does not use the same HTS label as a named module, but it explicitly turns hierarchical planning into search over latent trajectory segments via model predictive control in a learned latent space. This suggests that HTS is not a single standardized algorithm. A plausible implication is that the term functions as a family resemblance: hierarchical search over temporally extended states, with width reduction or abstraction-based planning replacing exhaustive flat search.

2. Search spaces, nodes, and trajectories

The central design choice in HTS is the object being searched. In recommendation, the search state is a candidate interest summary; in hierarchical RL, it is a latent trajectory segment; in diffusion inference, it is an evolving denoising trajectory. Despite the domain shift, the hierarchy is always indexed by temporal progression.

Setting	Search state	Hierarchical organization
HiT-LBM	$Node_{i,k}$ , a candidate interest summary for chunk $B_i$	Root node, one tree layer per behavior chunk, root-to-leaf route as an interest trajectory
SeCTAR	Latent variable $z_t$ representing a trajectory segment	Planning over approximately $H/T$ latent decisions rather than raw actions
dMLLM-TTS / Prism	Diffusion trajectory $Z_t$ or $\mathbf{z}_t$	Wide early search, progressive thinning, final refinement

In HiT-LBM, the search space is an interest tree: the root node initializes the search, the $i$ -th tree layer corresponds to the $i$ -th behavior chunk $B_i$ , and each node $Node_{i,k}$ is a candidate interest summary for chunk $B_i$ 0. A root-to-leaf route corresponds to an interest trajectory, with selected nodes forming the optimal path

$B_i$ 1

and the extracted interest sequence

$B_i$ 2

This tree formalization is introduced to mitigate quality degradation in the cascading paradigm, where a poor summary from chunk $B_i$ 3 can be passed into chunk $B_i$ 4 and degrade further (Xia et al., 26 May 2025).

In SeCTAR, the analogous hierarchy is defined over latent trajectory embeddings. Each latent $B_i$ 5 corresponds to a short, temporally extended behavior, and planning occurs over approximately $B_i$ 6 latent decisions. The paper’s key claim is that the effective horizon becomes shorter by a factor of the segment length $B_i$ 7, because the higher-level controller reasons over chunks of behavior rather than individual actions (Co-Reyes et al., 2018).

In diffusion HTS, the search state is not a left-to-right prefix but an entire denoising trajectory. That distinction is explicit in Prism: a dLLM state is partially masked and globally conditioned, so the natural unit of search is an entire diffusion trajectory rather than a prefix tree. This differs from autoregressive search, where branching occurs over growing prefixes and errors accumulate left to right (Bai et al., 2 Feb 2026).

3. Expansion, branching, and hierarchical thinning

The mechanics of HTS are domain-specific, but all variants implement a coarse-to-fine allocation of search width.

In HiT-LBM, candidate generation uses a Best-of- $B_i$ 8 expansion. Starting from the current parent interest node, the LLM is prompted with the parent node’s interest summary and the current behavior chunk $B_i$ 9, then sampled multiple times to produce $z_t$ 0 candidate child interests: $z_t$ 1 The method is greedy hierarchical expansion: each layer keeps multiple candidates, scores them, and selects the best one to continue (Xia et al., 26 May 2025).

SeCTAR implements a different branching regime through latent-space model predictive control. The planning procedure samples many candidate sequences of latents from the prior, rolls out each latent sequence through the state decoder, scores the predicted trajectories by reward, picks the best latent sequence, executes only the first latent with the policy decoder for $z_t$ 2 steps, and replans. The planner therefore searches over trajectory-level primitives rather than raw control sequences (Co-Reyes et al., 2018).

The diffusion variants formalize hierarchical thinning explicitly. In dMLLM-TTS, HTS has three stages: $z_t$ 3 The active width follows a geometric schedule

$z_t$ 4

so the pool shrinks over time until only $z_t$ 5 trajectories remain (Xin et al., 22 Dec 2025).

Prism introduces an analogous three-stage structure, but places pruning in an early-to-mid denoising window. The schedule is

$z_t$ 6

with exploration, progressive thinning, and final refinement separated by $z_t$ 7 and $z_t$ 8. A distinctive mechanism is local branching via partial remasking: $z_t$ 9 where only low-confidence positions are re-masked. This preserves a high-confidence logic skeleton while allowing alternative realizations in uncertain regions (Bai et al., 2 Feb 2026).

4. Scoring functions, verification, and information gain

HTS depends on an internal criterion for deciding which trajectories to keep. The form of that criterion varies from process rating in recommendation to self-verification in diffusion models.

HiT-LBM uses two process rating models. SRM (Sequence Rating Model) measures continuity with previous interests, and PRM (Point Rating Model) measures effectiveness or validity of the current interest for the current chunk. For chunk $H/T$ 0, the paper builds an evaluation set from the next chunk $H/T$ 1, computes “Yes” probabilities under three prompting conditions, and derives

$H/T$ 2

Information gain is defined operationally rather than with a Shannon-style formula: if the AUC with the current interest exceeds the AUC with the previous $H/T$ 3 interests, the current interest is treated as having continuity plus new information; if it exceeds the AUC using the current behavior chunk alone, it is treated as better representing the user’s present interest. Final node selection is then based on

$H/T$ 4

with

$H/T$ 5

The method is designed to reduce LLM hallucination effects and make chunk-wise interest learning more reliable (Xia et al., 26 May 2025).

SeCTAR uses a different scoring regime. Its defining technical idea is self-consistency between a latent-conditioned policy and a latent-conditioned predictive model: the policy-generated trajectory should match the model-predicted trajectory for the same latent. At planning time, latent sequences are evaluated by predicted reward under the state decoder, so the high-level search objective is a model-based assessment of trajectory value rather than a verifier over text or images (Co-Reyes et al., 2018).

The diffusion HTS papers replace external verifiers with Self-Verified Feedback (SVF). In dMLLM-TTS, the model is asked whether the image matches the prompt, and the logit or probability of “yes” becomes the alignment score: $H/T$ 6 This score ranks trajectories during hierarchical thinning and also selects the final result (Xin et al., 22 Dec 2025).

Prism defines SVF more explicitly as

$H/T$ 7

where $H/T$ 8 and $H/T$ 9 are derived from a binary self-evaluation prompt over a completed hypothesis. The paper emphasizes that HTS and SVF are tightly coupled: HTS supplies the search skeleton, and SVF supplies the ranking signal used for pruning and branching (Bai et al., 2 Feb 2026).

5. Efficiency, horizon reduction, and empirical behavior

One reason HTS recurs across domains is that it reallocates computation away from uniformly expensive flat search.

In hierarchical RL, SeCTAR reduces the effective planning horizon by a factor of the segment length $Z_t$ 0, because planning operates over approximately $Z_t$ 1 latent decisions. The paper reports that this trajectory-level model is effective at reasoning over long horizons with sparse rewards and outperforms TRPO, A3C, VIME, a model-based baseline from cited prior work, FeUdal Networks, and Option-Critic on 2D navigation, wheeled locomotion, object manipulation, and swimmer navigation (Co-Reyes et al., 2018).

In recommendation, HiT-LBM positions HTS as the module that addresses quality degradation in the cascading paradigm after chunking. Its ablation is organized as CUBE $Z_t$ 2 TIF $Z_t$ 3 HTS. The paper states that CUBE improves performance by alleviating context-length limitations, TIF gives a further sizable boost by modeling interest evolution over time, and HTS gives the final improvement and yields the best performance, suggesting that searching for the optimal interest path matters beyond simple chunking and fusion. In comparisons against baselines, HiT-LBM outperforms DIN, SIM, KAR, TRSR, and LIBER on both MovieLens-1M and Amazon-Book, and the full framework improves AUC and LogLoss across multiple backbones (Xia et al., 26 May 2025).

In diffusion test-time scaling, efficiency claims are explicit. dMLLM-TTS contrasts linear trajectory search, with cost $Z_t$ 4, against HTS with

$Z_t$ 5

which simplifies under practical assumptions to approximately $Z_t$ 6. On GenEval, the paper reports overall score improvements from $Z_t$ 7 for Lumina-DiMOO, $Z_t$ 8 for MMaDA, and $Z_t$ 9 for Muddit, with HTS about $\mathbf{z}_t$ 0 faster on Lumina-DiMOO and about $\mathbf{z}_t$ 1 faster on MMaDA and Muddit (Xin et al., 22 Dec 2025).

Prism makes an analogous performance-efficiency argument for discrete diffusion LLMs. Its compute cost is

$\mathbf{z}_t$ 2

which the paper simplifies to approximately $\mathbf{z}_t$ 3. Representative results include LLaDA 8B on GSM8K, where Prism with $\mathbf{z}_t$ 4 achieves $\mathbf{z}_t$ 5 at $\mathbf{z}_t$ 6 NFE versus best-of-16 at $\mathbf{z}_t$ 7 and $\mathbf{z}_t$ 8 NFE, and on MATH500, where Prism $\mathbf{z}_t$ 9 achieves $i$ 0 at $i$ 1 NFE versus best-of-16 at $i$ 2 and $i$ 3 NFE (Bai et al., 2 Feb 2026).

A common misconception is to equate any hierarchical trajectory model with HTS. The provided literature makes a sharper distinction.

HiT-JEPA is hierarchical and trajectory-centered, but the hierarchy is semantic and representational rather than an external search index. The model builds a three-level hierarchy,

$i$ 4

and propagates higher-level attention downward as a top-down spotlight. For retrieval, it uses $i$ 5, enriched by the full hierarchy of multi-scale abstractions, as the final trajectory representation. The paper explicitly states that it is not a search system with an explicit multi-stage candidate pruning tree in the classic algorithmic HTS sense and does not build an explicit retrieval tree or hierarchical index for candidate elimination (Li et al., 17 Jun 2025).

Another misconception is to treat HTS as interchangeable with autoregressive tree search. Prism explicitly distinguishes diffusion-native HTS from autoregressive best-of- $i$ 6, self-consistency, Tree-of-Thoughts, or MCTS. In dLLMs, the search state is a diffusion trajectory, compute is reallocated across denoising time rather than token positions, and branching is local and remasking-based rather than prefix extension (Bai et al., 2 Feb 2026).

The limitations reported across the literature are also domain-specific. In HiT-LBM, HTS is introduced because long sequences exceed LLM context limits and naive cascading can propagate low-quality interest summaries forward; HTS mitigates this degradation rather than abolishing the underlying challenge (Xia et al., 26 May 2025). In dMLLM-TTS and Prism, self-verification is efficient and self-contained, but external verifiers such as VILA-Judge, GPT-4o, or stronger external VLMs can outperform SVF in some cases, so ranking quality depends on the model’s own multimodal or reasoning competence (Xin et al., 22 Dec 2025, Bai et al., 2 Feb 2026). HiT-JEPA, meanwhile, keeps its hierarchy fixed at three levels and does not present a dedicated complexity analysis or an explicit deep hierarchy beyond that design (Li et al., 17 Jun 2025).

Taken together, these usages suggest that HTS is best understood as a methodological pattern for structured search over temporally extended candidate states. Its specific instantiation depends on the searched object—interest summaries, latent behaviors, image-generation trajectories, or language denoising paths—but its recurring technical purpose is the same: replace uniform exhaustive search with a hierarchy that preserves diversity early, identifies a promising trajectory or survivor set, and spends later computation where it is most informative.