---
title: 'Latent-DARM: Latent Space Planner–Executor Interface'
url: https://www.emergentmind.com/topics/latent-darm
type: topic
---

# Latent-DARM: Latent Space Planner–Executor Interface

Latent-DARM is a latent-space communication framework for multi-agent reasoning that bridges a discrete diffusion language model (DDLM) used as a planner with an autoregressive language model (ARM) used as an executor. The method is motivated by a division of labor: ARMs are effective for fluent sequential text generation but “limit global reasoning and plan revision,” whereas DDLMs enable “non-sequential, globally revisable generation” and have shown strong planning capabilities, yet their limited text fluency makes direct collaboration with ARMs brittle. Latent-DARM replaces the usual decode-to-text and re-encode interface with a learned projection from the planner’s final hidden state into the executor’s embedding space, with the goal of preserving planning structure while avoiding the discretization bottleneck of text [2603.09184].

## 1. Problem formulation and conceptual motivation

The framework is introduced in the setting of planner–executor systems for reasoning. In that setting, Agent 1 produces an intermediate plan, hint, or set of salient facts, and Agent 2 uses that information together with the original question to generate the final answer. The paper characterizes the planner as corresponding to a “thinking” stage and the executor as the component that produces the final answer “without additional explicit reasoning.” DDLMs are assigned the planner role because their masked denoising process supports non-sequential revision over the whole sequence, whereas ARMs are assigned the executor role because they can realize outputs fluently and reliably [2603.09184].

A central claim is that text-space communication is a poor interface between these model families. In the baseline formulation, the planner’s representation is decoded into text and then re-encoded for the executor,
$$
h_{\text{DDLM}} \xrightarrow{\text{decode}} T \xrightarrow{\text{encode}} h_{\text{ARM}} .
$$
The paper argues that this is problematic because DDLMs and ARMs do not share the same latent geometry. Their hidden states are described as living on different manifolds,
$$
h^{(1)} \in H_{\text{DDLM}} \subset \mathbb{R}^{d_1}, \qquad h^{(2)} \in H_{\text{ARM}} \subset \mathbb{R}^{d_2},
$$
with typically $d_1 \neq d_2$. On this account, forcing communication through text reintroduces a lossy bottleneck: the DDLM must first produce fluent text, and any disfluency, repetition, or incompleteness degrades the plan before the ARM ever receives it. The paper’s broader thesis is that text is not the only viable communication medium in multi-agent reasoning; latent exchange may preserve more of the planner’s structured intent.

## 2. Architectural structure of the latent bridge

Latent-DARM formalizes an alternative communication pathway in which the DDLM’s final latent state after denoising is projected directly into the ARM’s input space,
$$
h_{\text{DDLM}} \xrightarrow{f_\theta} h_{\text{ARM}} .
$$
The projected vector is then concatenated with the encoded question embedding,
$$
h_{\text{input}} = [\, f_\theta(h_{\text{DDLM}})\, ;\, \mathrm{embed}_{\text{ARM}}(q)\,] ,
$$
and the executor conditions on this combined representation to generate the answer [2603.09184].

The projection module $f_\theta$ is the key learned component. In the architectural description it is introduced as a **Linear–GELU–Linear** network. In the reported training details, the projector is specified more concretely as **3 linear layers with 2 GELU activations**, with **input dimension 4096**, **bottleneck 1024**, **output the LLaMA hidden dimension**, followed by **LlamaRMSNorm**. The design intent is not geometric identity between planner and executor representations, but a representation-aware translation that places planner information into a region of the executor’s embedding space that is useful for downstream answering.

The method is explicitly presented as a communication bridge rather than a full planner–executor co-training scheme. The paper states that **only the projector is trained** and that both the DDLM planner and ARM executor remain frozen. In the detailed training configuration, the executor is also described as using **LoRA adapters on attention and FFN projection layers**, while the backbone and LM head are frozen. Taken together, these statements indicate that the paper’s emphasis is on learning the interface rather than retraining the core models.

## 3. Diffusion-planner background and training objective

The DDLM component is a masked discrete diffusion language model over token sequences
$$
x \in \{1,\ldots,V\}^L .
$$
During training, a masking ratio $t \sim \mathrm{Unif}[0,1]$ is sampled, each token is independently replaced by a mask token $m$ with probability $t$, and the model is trained to predict the masked tokens in the partially corrupted sequence $z$ by minimizing
$$
\mathbb{E}_{x\sim q_{\text{data}},\, t,\, z}\sum_{i:z_i=m} - \log p_\theta(x_i \mid z). \tag{1}
$$
Generation proceeds by starting from a fully masked sequence and iteratively unmasking tokens over multiple denoising steps. The paper notes a known drawback of this regime: tokens revealed in a denoising step are predicted independently given the current unmasked context, which can yield outputs that are less fluent than those of ARMs [2603.09184].

Latent-DARM does not attempt to solve that fluency problem by modifying the diffusion model itself. Instead, it trains the projection module indirectly through downstream task performance. The paper explicitly rejects a naive objective that would align the projected DDLM representation to some “ideal” ARM hidden state, on the grounds that there is no unique canonical executor state corresponding to a correct solution. The projector is therefore trained with a task-aligned negative log-likelihood objective:
$$
\min \mathbb{E}_{(q,a)\sim \mathcal{D}}
\Big[-\log p_{\text{ARM}}\big(a \mid f_\theta(h_{\text{DDLM}}(q)), q\big)\Big]. \tag{5}
$$
This objective trains the bridge to preserve whatever aspects of the planner latent actually improve executor success.

The paper also gives explicit training and inference workflows. Training consists of initializing the projector, running the DDLM planner on each question to obtain $h_{\text{DDLM}}$, computing $h_{\text{proj}} = f_\theta(h_{\text{DDLM}})$, concatenating it with the ARM question embedding, computing cross-entropy or NLL against the answer, and updating the projector parameters. Inference consists of the same latent handoff without gradient updates.

## 4. Experimental configuration and model pairings

The empirical study covers **ARC-E**, **ARC-C**, **MMLU**, **AIME 2024**, and **DART-1 through DART-5**, combining science question answering, broad multitask knowledge, competitive mathematics, and progressively difficult mathematical reasoning benchmarks [2603.09184].

The DDLM planners evaluated include **LLaDA-8B-Instruct** and **Dream-v0-Instruct-7B**. The ARM executors include **Qwen2.5-7B-Instruct**, **Llama-3.1-8B-Instruct**, **Llama-3.2-3B-Instruct**, and **Qwen2.5-3B-Instruct**. For reference, the paper also lists **Qwen3-1.7B** and **DeepSeek-R1-Distill-Qwen-7B** as reasoning models. The main latent-space experiments use **LLaDA-8B-Instruct → Llama-3.2-3B-Instruct**, with planner sequence lengths of **64**, **128**, and **256 tokens**.

Projector training uses **35,000 samples uniformly drawn from ARC-E, ARC-Challenge, and DART1–DART5**. The optimizer is **AdamW** with **learning rate $5 \times 10^{-4}$**, **weight decay 0.001**, **warmup 300 steps**, and a **cosine** schedule. The reported batch configuration is **4 per device**, **gradient accumulation 2**, and **effective batch size 8**, trained for **10 epochs**. These details matter because the paper later attributes part of the benchmark profile of the learned bridge to the composition of this reasoning-heavy training set.

## 5. Reported performance and token-efficiency profile

The paper’s headline empirical claim is that latent-space communication consistently outperforms text-space communication on planning-intensive tasks. In the main **LLaDA-8B → Llama-3.2-3B** comparison, the text-space baseline reports **ARC-E 90.5**, **ARC-C 82.5**, **DART-1 53.5**, **DART-2 43.0**, **DART-3 35.5**, **DART-4 30.0**, **DART-5 27.0**, **AIME24 0.0**, and **MMLU 52.5**. The latent interface changes this profile substantially, especially on DART and AIME [2603.09184].

| Interface / plan length | DART-5 | AIME24 |
|---|---:|---:|
| Text-space baseline | 27.0 | 0.0 |
| Latent-space, 64-token plan | 54.0 | 12.5 |
| Latent-space, 128-token plan | 36.0 | 14.0 |
| Latent-space, 256-token plan | 52.5 | 14.0 |

The abstract summarizes the improvement as **27.0% to 36.0% on DART-5** and **0.0% to 14.0% on AIME2024**. The main table for the principal model pairing reports a larger DART-5 value of **54.0** for the **64-token** latent plan, alongside **36.0** for the **128-token** plan and **52.5** for the **256-token** plan. This suggests that the abstract is highlighting one reported latent configuration or an average rather than the best table entry. Across the reported settings, the most important gains remain concentrated on the harder reasoning benchmarks.

The method is not uniformly best on every dataset. The paper explicitly notes that it underperforms text-space communication on **MMLU** in the reported main table, interpreting this as evidence that the learned projector—trained on reasoning-heavy datasets—preserves planning structure better than broad factual recall. This qualification is important because it constrains the scope of the paper’s claims: the latent bridge is presented as especially useful for tasks requiring multi-step reasoning and structural planning rather than as a universal replacement for text-based collaboration.

A second major empirical theme is token efficiency. The **64-token** latent planner configuration is highlighted as the best overall trade-off. With only **64 planner tokens** and an average of **5 executor tokens**, Latent-DARM is reported to **surpass Qwen3 on DART-5 while using only 2.2% of the tokens**, and to **outperform it on AIME with only 1.9% of the tokens**. The abstract further states that the system “approaches the results of state-of-the-art reasoning models while using less than **2.2%** of its token budget.” The paper also notes that longer plans are not always better: **64 tokens** often gives the best balance, while **128** and **256** tokens can increase repetition and sometimes hurt performance.

## 6. Diagnostics, failure attribution, and scope conditions

The paper includes a diagnostic analysis intended to separate planner failures from executor failures. It defines **Setup X** as cases where **DDLM→ARM fails but ARM→ARM succeeds**, suggesting that the planner side is the bottleneck, and **Setup Y** as cases where **DDLM→DDLM succeeds but DDLM→ARM fails**, suggesting that the executor side is the bottleneck. Failure attribution is quantified as
$$
\text{Percentage}_i
=
\frac{\#\{\text{Setup } i \text{ samples}\}}
{\#\{\text{Incorrect samples in DDLM} \to \text{ARM}\}},
\qquad i \in \{X,Y\}.
$$
The reported result is that text-space collaboration produces failures mostly attributable to **Setup X**, whereas under Latent-DARM the failure mass shifts toward **Setup Y**. The paper interprets this as evidence that the latent bridge preserves planner quality more effectively, moving the dominant bottleneck from plan communication to executor limitations [2603.09184].

Additional appendix diagnostics quantify repetition in DDLM plans using **Distinct-3**, **Repetition-4**, and lexical repetition. The general trend reported is that longer diffusion plans become more repetitive, and that the **64-token** configuration is the least repetitive among the longer DDLM plans while remaining competitive with the autoregressive baseline. This aligns with the token-efficiency result: shorter latent plans are not merely cheaper, but may also preserve a cleaner planning signal.

Several scope conditions and limitations are stated directly. The projector is trained on a limited set of reasoning datasets, so transfer to broad knowledge tasks such as MMLU is weaker. Performance still depends on a frozen executor, meaning the bridge cannot fully correct executor deficiencies. The current study examines **one-way planner-to-executor communication**, not multi-hop or bidirectional collaboration. The paper also describes the work as a **preliminary empirical study**, leaving broader theoretical guarantees for future work. A common misconception would be to treat Latent-DARM as a generic finetuning recipe for heterogeneous models; in the paper’s formulation, it is specifically a representation-aware communication interface between a DDLM planner and an ARM executor.

## 7. Position within reasoning-model research

Latent-DARM is situated at the intersection of discrete diffusion language modeling, autoregressive generation, and multi-agent reasoning. Its central scientific claim is not that DDLMs should replace ARMs, nor that ARMs are intrinsically inadequate for reasoning, but that the two model classes have complementary strengths when connected through an appropriate interface. The DDLM supplies globally revisable planning structure, and the ARM supplies fluent realization. The learned projector is the mechanism that mediates between these regimes without routing the handoff through text [2603.09184].

Within that framing, the method advances a specific view of inter-model collaboration: communication bandwidth and representation compatibility matter as much as raw model quality. The paper argues that the relevant information passed from planner to executor is often abstract structure rather than surface form. A plausible implication is that latent communication may be especially useful whenever a planner’s internal organization is stronger than its textual fluency. At the same time, the benchmark profile in the paper shows that this advantage is task-dependent and training-dependent. Latent-DARM therefore stands less as a general theory of reasoning and more as a concrete demonstration that planner–executor systems can benefit from replacing text-space interfaces with learned latent bridges when the participating agents have heterogeneous generative mechanisms.

Source: https://www.emergentmind.com/topics/latent-darm