Latent Span-Slot Alignment in LLM Reasoning

• The paper introduces a novel latent span-slot alignment mechanism that uses entropy-regularized optimal transport to compress and semantically align reasoning spans in LLMs.
• Latent span-slot alignment is defined as a technique to map variable-length explicit text spans into single latent tokens, maintaining semantic fidelity while reducing token count.
• Applied in the SPOT framework, this method improves accuracy by an average of 2.3 pp and reduces token length by about 37.5%, demonstrating significant efficiency gains.

Latent span-slot alignment is a mechanism for compressing explicit chain-of-thought (CoT) traces in LLMs into compact, interpretable latent representations, enabling efficient and semantically faithful reasoning. This approach generalizes beyond rigid point-to-point alignment by mapping variable-length explicit spans to single latent “slots” using entropy-regularized optimal transport (OT). It enhances both interpretability and efficiency, as demonstrated in the SPOT (“Span-level Pause-of-Thought”) framework, where explicit reasoning steps are replaced with special latent tokens that remain semantically grounded in the original content via a frozen LLM head (Chu et al., 6 Mar 2026).

1. Problem Setup and Formalism

Given an input $x$ and a teacher-model CoT trace $y = (y_1, \dots, y_T)$, the sequence is partitioned into $M$ consecutive, non-overlapping spans $S_1, S_2, \dots, S_M$. Each $S_i$ is typically a text block (e.g., paragraph delimited). A special latent token, $\langle$pause$\rangle$, is added to the vocabulary.

During training, a subset of spans $D \subseteq \{1, \dots, M\}$ is randomly selected to be replaced with $\langle$pause$\rangle$. The resulting compressed sequence, $y = (y_1, \dots, y_T)$0pause$y = (y_1, \dots, y_T)$1, S_{i+1}, \dots)$y = (y_1, \dots, y_T)$2h^{\text{tea}}_ty=(y1,…,yT)y = (y_1, \dots, y_T)3y_ty=(y1,…,yT)y = (y_1, \dots, y_T)4S_iy=(y1,…,yT)y = (y_1, \dots, y_T)5H^{\text{tea}}(S_i) \in \mathbb{R}^{|S_i|\times d}$y = (y_1, \dots, y_T)$6z_i \in \mathbb{R}^dy=(y1,…,yT)y = (y_1, \dots, y_T)7\langley=(y1,…,yT)y = (y_1, \dots, y_T)8\rangle</sup>position(<ahref="/papers/2603.06222"title=""rel="nofollow"data−turbo="false"class="assistant−link"x−datax−tooltip.raw="">Chuetal.,6Mar2026</a>).</p><h2class=′paper−heading′id=′frozen−head−decoding−constraint′>2.Frozen−HeadDecodingConstraint</h2><p>Toensureinterpretabilityof</sup> position (<a href="/papers/2603.06222" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Chu et al., 6 Mar 2026</a>).</p> <h2 class='paper-heading' id='frozen-head-decoding-constraint'>2. Frozen-Head Decoding Constraint</h2> <p>To ensure interpretability of y = (y_1, \dots, y_T)$9, the frozen pretrained LLM head (linear head ($M$0, $M$1) and embedding matrix $M$2) is used. For any hidden state $M$3, the token distribution is:

$M$4

where $M$5 is the vocabulary size. The projected embedding is:

$M$6

No explicit cross-entropy is applied to $M$7pause$M$8 tokens; instead, $M$9 is shaped by the span-alignment loss described below. The fixed LM head and embedding matrix guarantee that $S_1, S_2, \dots, S_M$0 remains decodable as a token distribution, allowing top-$S_1, S_2, \dots, S_M$1 keyword extraction for latent interpretation. This enforces that latent slots represent lexically meaningful mixtures of tokens (Chu et al., 6 Mar 2026).

3. Span-Level Semantic Alignment via Sinkhorn OT

The core of latent span-slot alignment is the soft coupling of latent slots to explicit reasoning spans using OT. For span $S_1, S_2, \dots, S_M$2 aligned to slot $S_1, S_2, \dots, S_M$3:

• Project the student slot and teacher span tokens: $S_1, S_2, \dots, S_M$4; $S_1, S_2, \dots, S_M$5 for $S_1, S_2, \dots, S_M$6.
• Compute the ground cost $S_1, S_2, \dots, S_M$7, where $S_1, S_2, \dots, S_M$8.
• Marginals: source $S_1, S_2, \dots, S_M$9 (all mass on student slot); target $S_i$0 (uniform over the span).

The Sinkhorn-regularized OT objective is:

$S_i$1

subject to row/column sum constraints, yielding a soft alignment between slot and span (Chu et al., 6 Mar 2026). The span-alignment loss is averaged over all dropped spans:

$S_i$2

This objective enables one-to-many semantic coverage of spans by a single latent slot and improves upon endpoint-only (e.g., last-token) alignment, which shows marked accuracy degradation (–4–15 pp) when OT is replaced by naive endpoint KL (Chu et al., 6 Mar 2026).

4. Training Methodology

Latent span-slot alignment is embedded in a two-stage training pipeline:

• Stage I (OT alignment training): For each mini-batch, the teacher is run on full $S_i$3 to produce $S_i$4; the student processes $S_i$5 to yield $S_i$6 at $S_i$7pause$S_i$8 positions. The total loss combines cross-entropy over explicit (non-$S_i$9pause$\langle$0) tokens and $\langle$1 (default $\langle$2), with only LoRA parameters updated. Sinkhorn iterations (20–50), with exponential scaling of $\langle$3, are used for stability; very long spans ($\langle$4) are subsampled.
• Stage II (Rejection-Sampled Fine-Tuning, RFT): Full explicit traces are generated, and $\langle$5 candidates are sampled by inserting $\langle$6pause$\langle$7 in various locations. Only candidates matching the original answer are kept; the shortest correct sequence $\langle$8 is selected and the model is fine-tuned with cross-entropy masking out the $\langle$9pause$\rangle$0 token.

At inference, $\rangle$1pause$\rangle$2 is inserted at deterministic intervals (e.g., every $\rangle$3 spans), and autoregressive decoding is performed. The choice of $\rangle$4 determines the tradeoff between compression and interpretability.

5. Interpretability and Semantic Faithfulness

Latent slots $\rangle$5 are directly decodable via the frozen LM head to yield $\rangle$6. The most probable tokens (Top-$\rangle$7) of this distribution form a keyword-style summary of the corresponding reasoning span. Empirical results show that the Top-$\rangle$8 tokens extracted from $\rangle$9 cover $D \subseteq \{1, \dots, M\}$0 of actual teacher span tokens under SPOT, compared to $D \subseteq \{1, \dots, M\}$1 for vanilla models, indicating semantically faithful anchoring (Chu et al., 6 Mar 2026). The frozen-head projection ensures that generated latent states remain readable and interpretable at the token level.

6. Empirical Outcomes and Ablation

On the DeepSeek-R1-Qwen-7B backbone, latent span-slot alignment yields substantial gains:

Benchmark	Accuracy Gain (pp)	Token Reduction (%)
GSM8K	+3.1	–52.1
MATH500	+1.4	–43.0
AIME’25	+3.3	–15.8
GPQA	+4.5	–49.3

Averaged over five benchmarks, accuracy improves by 2.3 points and generated token length is reduced by 37.5%. Ablations indicate that non-OT alternatives (e.g., endpoint KL, MSE to mean span embedding) significantly degrade performance or compression. Optimal trade-offs are achieved with $D \subseteq \{1, \dots, M\}$2 (one span per $D \subseteq \{1, \dots, M\}$3pause$D \subseteq \{1, \dots, M\}$4), and the alignment-regularization weight $D \subseteq \{1, \dots, M\}$5 maximizes performance (Chu et al., 6 Mar 2026).

7. Significance and Implications

Latent span-slot alignment, as instantiated in SPOT, constitutes a robust semantic compression and reasoning mechanism within LLMs. By leveraging entropy-regularized optimal transport and frozen-head interpretability constraints, it supports compact latent tokenization without loss of accuracy and with substantial reduction in generation cost. The mechanism provides a model-agnostic interface for interpretable, one-to-many alignment between latent spaces and explicit reasoning segments. A plausible implication is the broader applicability of such techniques to other forms of structured latent reasoning and compression within large-scale neural architectures (Chu et al., 6 Mar 2026).