---
title: 'ALPINE: Autoregressive Planning in Networks'
url: https://www.emergentmind.com/topics/autoregressive-learning-for-planning-in-networks-alpine
type: topic
---

# ALPINE: Autoregressive Planning in Networks

Autoregressive Learning for Planning In NEtworks (ALPINE) is a theoretical and empirical investigation into how standard Transformer-based large language models (LLMs), trained via autoregressive next-token prediction, acquire and execute planning capabilities when such planning is cast as path-finding over networks. The ALPINE framework formalizes planning as network path-finding, characterizes precisely what is learned by Transformers under cross-entropy training, and reveals fundamental limitations—most notably, the inability to infer transitive reachability—from both theoretical and experimental perspectives [2405.09220].

## 1. Formalization of Path-Finding as Autoregressive Prediction

ALPINE models planning as a discrete token sequence generation problem: given a directed graph $\mathcal{G}$ with node set $\mathcal{V}$, adjacency matrix $A^{\mathrm{true}} \in \{0,1\}^{|\mathcal{V}| \times |\mathcal{V}|}$ (where $A_{i,k}^{\mathrm{true}} = 1$ iff edge $(i \to k)$ exists), and reachability matrix $R^{\mathrm{true}} \in \{0,1\}^{|\mathcal{V}| \times |\mathcal{V}|}$ (where $R_{t,k}^{\mathrm{true}} = 1$ iff a path $k \to \ldots \to t$ exists), the task is to generate a token sequence $u = (s, t, s, u_1, u_2, \ldots, u_p, t, \text{\textbackslash n})$, representing a path from source $s$ to target $t$. The vocabulary size is $M = |\mathcal{V}| + 1$.

Transformers are trained to autoregressively predict each next token using standard architecture:
- Input encoding: $H_0 = U W_t + W_p$, where $U$ is one-hot, $W_t$, $W_p$ are learned weights.
- Layerwise propagation: $H_\ell = \mathrm{TransformerLayer}(H_{\ell-1})$.
- Output: $\hat{u}_{n+1} = \mathrm{softmax}(\mathrm{LN}_t(H_L)_{(n,:)} W_o)$.
- Training loss: $\ell = -\sum_{n=1}^{N-1} U_{(n+1)} \cdot \log \hat{u}_{n+1}$.

This formulation provides a direct bridge between next-token prediction and the mechanistic implementation of planning as path-finding.

## 2. Representation of Graph Structure in Self-Attention and Output Probabilities

A core result (Theorem 3.1) demonstrates that a 1-layer, 1-head Transformer with embedding dimension $d = O(|\mathcal{V}|)$ can exactly implement optimal path-finding via appropriately chosen weights:
- **Attention:** Weights $W^Q = \sqrt{d}\,I$, $W^K$ ensure all attention mass focuses on the target token position.
- **Values:** Weights $W^V$ at the target token position embed the row $R_{(t,:)}^{\mathrm{true}}$ of the true reachability matrix.
- **Feedforward/MLP:** Weights $W_1, W_2$ extract the row $A_{(u_k,:)}^{\mathrm{true}}$, representing outgoing edges from the current node $u_k$.
- **Final logits:** At each next-token step,
  $$
  \mathrm{logit}_k \propto c_1 R_{t,k}^{\mathrm{true}} + c_2 A_{u_k,k}^{\mathrm{true}},
  $$
  where $c_1, c_2$ are scaling factors. The softmax over these logits assigns nonzero mass precisely to those $k$ reachable from $u_k$ and, ultimately, from $t$.

This construction unveils how Transformers, in principle, can internalize both adjacency ($A$) and reachability ($R$) information in their parameters and use these to make path-finding decisions stepwise.

## 3. Learning Dynamics: Adjacency and Partial Reachability

The analysis of SGD-driven learning dynamics (Section 4) employs a simplified Transformer (single layer/head, no nonlinearity/normalization, input/output weights as identities) to clarify which graph-theoretic structures are reliably encoded:
- For each observed triple $(i, j, k)$ (current node, target, next node), the final logit is $L_{i,j,k} = W^M_{i,k} + W^V_{j,k}$.
- Gradients with respect to $W^M$ (adjacency) drive $W^M_{i,k}$ up if edge $(i \to k)$ is seen in the training paths, and down or unchanged otherwise—resulting precisely in $W^M$ encoding $A^{\mathrm{obs}}$, the adjacency matrix of the paths present in training.
- For $W^V$ (reachability), the same dynamics cause $W^V_{j,k}$ to reflect $R^{\mathrm{obs}}$, that is, direct evidence of $k$ appearing on any path to $j$ in training.

Crucially, concatenated or composite reachabilities—cases where $(t, k)$ is reachable through chaining but never observed as a direct $(t, k)$ pair—are not learned. Thus, the trained model does not internalize the transitive closure required for inferential planning beyond observed data.

## 4. Empirical Evaluation on Synthetic DAGs and Blocksworld

Extensive experiments validate the theoretical predictions using both synthetic graph environments and the Blocksworld planning benchmark:
- **Synthetic DAGs:** Training on random graphs ($n=100$–500 nodes, $p=0.05$ edge prob.), up to 120-dimensional embeddings, varying layers/heads, and evaluating exact-path accuracy.
  - For $n \leq 200$, 1-layer/1-head models achieve >95% accuracy; for larger graphs, accuracy falls modestly.
  - Attention always focuses on the target token.
  - Feedforward matrix $W^{M'}$ recovers $A^{\mathrm{true}}$ on seen edges.
  - Value matrix $W^{V'}$ correlates with $R^{\mathrm{obs}}$; pairs requiring transitive closure are not captured.
  - Partitioning by “degree” (number of required concatenations): accuracy is near-100% for degree 0, falls to 20–50% for degree $\ge 2$.
- **Blocksworld:** The state graph with 73 nodes (legal 4-block configurations) is encoded analogously, trained on 80% of $(s, t)$ pairs, sampled over 50,000 paths.
  - A 1-layer, 1-head, $d=120$ Transformer achieves near-100% accuracy on directly observed reachabilities; attention and weights mirror synthetic results.

These results confirm both the successful learning of adjacency and observed reachability, and the consistent failure to generalize to purely compositional inferences.

## 5. Fundamental Limitation: No Transitive Closure

Theoretical (Section 4) and empirical (Figures 8, 9, 10) evidence demonstrate that autoregressive next-token training on path sequences fails to instantiate a mechanism for learning transitive closure:
- Gradients for $W^V_{t,k}$ do not increase unless $(t, k)$ appears directly in the data—as such, the model has no way to “compose” two or more partial reachabilities and thus cannot infer $R_{t,k}^{\mathrm{true}} = 1$ for any $(t, k)$ not observed.
- Empirically, entries in $W^{V'}$ corresponding to unobserved (but actually reachable) node pairs remain indistinguishable from truly unreachable pairs.
- Accordingly, test accuracy collapses on pairs requiring nontrivial concatenation (degree $\ge 2$), even on large models and extensive training sets.

This limitation underscores that current Transformer architectures, when trained exclusively via next-token prediction, do not implement compositional or algorithmic reasoning required for generalizable planning.

## 6. Insights and Prospects for Enhanced Planning in LLMs

ALPINE’s synthesis of theory and experiment yields several insights for advancing the planning capabilities of LLMs:
- Transformers can reliably learn and utilize direct adjacency and observed reachability within their weights under next-token cross-entropy training.
- However, absence of a compositional mechanism precludes the discovery of novel, transitive reachabilities—planning generalizes only across instances seen during training.
- To instill robust, compositional planning, promising modifications include:
  - Algorithmic or compositional supervision (e.g., chain-of-thought traces incorporating path composition).
  - Architectural augmentation using explicit graph reasoning modules, such as GNN encoders capable of dynamic-programming-style inference.
  - Expansion of the training objective to include new reachability queries, beyond observed instances.
  - Memory or deduction layers designed for transitive inference.
  - Hybrid losses that encourage generalized reasoning across unseen node pairs.

Collectively, these directions point toward future LLMs with genuinely generalizable planning and path-finding abilities, contingent upon integrating architectural, training, or supervision innovations beyond standard autoregressive objectives [2405.09220].

Source: https://www.emergentmind.com/topics/autoregressive-learning-for-planning-in-networks-alpine