GLAD: Grounded LookAhead Distillation

• The paper introduces a supervised distillation method that compresses verbose search traces into concise causal reasoning chains, enhancing LLM planning efficiency.
• GLAD leverages explicit ground-truth environment trajectories and a two-step process of data generation followed by structured compression to simulate future states.
• Empirical results on tasks like 2048 and Sokoban show reduced error accumulation and improved foresight-driven decisions without repeated environment access.

Grounded LookAhead Distillation (GLAD) is a supervised distillation framework for training LLM agents to perform multi-step lookahead reasoning in interactive environments. Rather than relying on inference-time search, GLAD leverages ground-truth environment trajectories to explicitly teach LLMs how to internally simulate future states and make foresight-driven decisions. The approach compresses expensive search traces into concise causal reasoning chains, enabling efficient and robust planning at inference without repeated environment access (Yu et al., 5 Feb 2026).

1. Formal Setup and Objective

GLAD operates within the Markov Decision Process (MDP) formalism: $M = (S, A, P, R, \gamma)$, where $S$ is the state space, $A$ the action space, $P$ the transition kernel, $R$ the reward function, and $\gamma$ the discount factor. The conventional RL objective is to maximize the expected return,

$$J(\theta) = \mathbb{E}_{\tau \sim \pi_\theta} \left[ \sum_{t=0}^\infty \gamma^t r_t \right]$$

In GLAD, the LLM agent’s policy $\pi_\theta$ is decomposed hierarchically:

$$\pi_\theta(z_t, a_t | s_t) = \pi_\theta(z_t | s_t) \cdot \pi_\theta(a_t | s_t, z_t)$$

where $z_t = (z_t^1, ..., z_t^L)$ is a sequence of "reasoning" or "thought" tokens and $a_t$ is the selected action. The goal is to train $\pi_\theta(z | s)$ to approximate search-grounded reasoning, as generated by a strong (but costly) oracle planner such as Monte-Carlo Tree Search (MCTS). Supervised fine-tuning is performed on triplets $(s,z^*,a^*)$ by minimizing the negative log-likelihood:

$$L_{\mathrm{GLAD}}(\theta) = - \mathbb{E}_{(s, z^*, a^*) \in D} \left[ \sum_{j=1}^{|z^*|} \log \pi_\theta(z^*_j | s, z^*_{<j}) + \log \pi_\theta(a^* | s, z^*) \right]$$

2. Data Generation via Grounded Environment Search

At each decision point $s_t$, ground-truth lookahead data is produced through environment-based search:

• A planner (typically short-horizon MCTS or K-step rollouts) queries the real transition function $P$, yielding a set of trajectories $T_{\mathrm{real}} = \{\tau_i\}$, where each $$\tau_i = (s_t, a_t^i, r_t^i, s_{t+1}^i, ..., s_{t+T}^i)$$.
• The current LLM policy $\pi_\theta$ is prompted with $s_t$ and $T_{\mathrm{real}}$ to analyze likely futures and suggest an action $a_t$.
• The model’s corresponding "raw" reasoning chain and action selection form the basis for subsequent data distillation.

This process ensures that the reasoning is tightly coupled to the environment’s true dynamics, rather than hypothetical or model-based simulations.

3. Compression and Cognitive Distillation

Raw search traces are characteristically verbose, tree-structured, and contain artifacts from planning (such as backtracking markers). GLAD incorporates a structured distillation phase:

• Format simplification: MCTS-specific markers and tree metadata are stripped. Information is rewritten into natural language that is compatible with LLM tokenization.
• Explicit causal reasoning: Each reasoning step is presented as an Observation→Analysis→Conclusion triplet, clarifying the causal chain behind each recommendation.
• Trend estimation and action justification: The chain explicitly details the rationale behind each action’s (in)effectiveness.
• Reflect search diversity: Trade-offs or points of uncertainty revealed by the planner are preserved in the distilled reasoning, capturing the full complexity of the search outcome.

The overall compression is formally denoted as $z = \text{Compress}(s_t, T_{\mathrm{real}}, a_t)$, providing a dense "chain of thought" for each trajectory.

4. Model Architecture

GLAD employs a standard decoder-only Transformer including:

• Embedding Layer: Input tokens receive standard token, positional, and type embeddings (identifying state, reasoning, or action tokens).
• Transformer Decoder Stack: $L$ identical transformer layers comprising self-attention and feedforward sub-layers.
• Dual Head Output: The output hidden state fans out into two softmax heads:
  • $\pi_\theta(z_j | \cdot)$ for reasoning token prediction
  • $\pi_\theta(a | \cdot)$ for action prediction

During supervised fine-tuning, contexts are serialized as: <state> [s] </state> <chain> [z tokens so far], and the model is trained via teacher-forcing to incrementally generate the complete reasoning chain $z^*$ and finally the action $a^*$.

5. Supervised Fine-Tuning Procedure

The procedure involves:

• Dataset Construction: Multiple environment episodes ($M$) are autonomously rolled out, collecting tuples $$(s_t, T_{\mathrm{real}}, \text{raw\_analysis}, a_t)$$ at each step. Each is compressed to $(s_t, z_t^*, a_t)$, building a distilled dataset $D$ (e.g., ~25K samples for 2048, 8K for Sokoban).
• Training Hyperparameters:

| Task | Batch Size | Epochs | Adam LR | |----------|------------|--------|------------| | 2048 | 16 | 4 | $5 \times 10^{-5}$ | | Sokoban | 16 | 10 | $2 \times 10^{-5}$ |

• Optimization: Minimize $L_{\mathrm{GLAD}}(\theta)$ with cross-entropy teacher-forcing on both reasoning and action tokens.

The training routine is summarized algorithmically as follows:

Algorithm 1: Grounded Lookahead Distillation Input: environment E, policy π_θ, rollout depth d, samples k, episodes M Output: distilled dataset D Initialize D←∅ for e=1…M do s₀←E.reset(); H←∅; t←0 while not done(sₜ) do T_real ← ProbeEnvironment(sₜ;d,k) (raw_analysis,aₜ) ← π_θ(prompt(sₜ,T_real)) if aₜ==<BACKTRACK> then revert sₜ, continue else (s_{t+1},rₜ)←E.step(aₜ); H.append((sₜ,T_real,raw_analysis,aₜ)); t+=1 end for each (sₜ,T_real,raw_analysis,aₜ) in H do z*←Compress(sₜ,T_real,aₜ); D←D∪{(sₜ,z*,aₜ)} end end return D

6. Inference Mechanism and Empirical Insights

At inference time, the model receives a serialized state and optionally a partial reasoning chain, and autoregressively generates a 10–20 token chain $z_t$ capturing short- and medium-term foresight. The final action $a_t$ is predicted in one forward pass. This eliminates the need for runtime MCTS or environment rollouts, significantly reducing computational overhead.

Empirical results on both stochastic (2048) and deterministic (Sokoban) domains show that GLAD-trained agents exhibit reduced error accumulation during long-horizon reasoning and improved generalization to novel environment variants. By grounding multi-step lookahead in the true environment, GLAD addresses hallucination and compounding error problems inherent to unguided chain-of-thought in LLMs (Yu et al., 5 Feb 2026).

7. Benefits, Limitations, and Implications

The principal benefit of GLAD is its ability to distill high-fidelity lookahead reasoning directly into the LLM, obviating costly search or rollouts at deployment. The compression and grounding steps enable the model to simulate the planner’s logic token-efficiently, yielding robust, foresight-driven action selection even in long-horizon tasks and unseen settings.

This suggests that GLAD provides a bridge between explicit search-based planning and pure neural policy learning, with the LLM internally imitating the trajectory of a powerful planner within a short chain of tokens. A plausible implication is that similar strategies may benefit other domains where reasoning chains can be extracted and compressed from expert search, such as combinatorial games or robotic planning, though further work is required to confirm generality beyond the presented tasks (Yu et al., 5 Feb 2026).