---
title: 'ComposableNav: Composable Diffusion Planner'
url: https://www.emergentmind.com/topics/composablenav
type: topic
---

# ComposableNav: Composable Diffusion Planner

ComposableNav is a framework for instruction-following navigation in dynamic environments that treats an instruction as a composition of independently satisfiable specifications, learns a diffusion model for each corresponding motion primitive, and composes these models at deployment to generate trajectories satisfying novel combinations of specifications unseen in training. In the formulation introduced in "ComposableNav: Instruction-Following Navigation in Dynamic Environments via Composable Diffusion" [2509.17941], the central problem is combinatorial: as a robot’s skill set expands, each instruction may contain multiple specifications, and the number of possible combinations grows exponentially. ComposableNav addresses this by separating primitive learning from primitive composition, and by using a two-stage training procedure consisting of supervised pre-training for dynamic navigation followed by reinforcement learning fine-tuning that molds the base model into distinct primitives, thereby avoiding primitive-specific demonstration datasets [2509.17941].

## 1. Problem formulation and representational assumptions

ComposableNav considers the task of generating a time-parameterized motion trajectory $\tau$ from an instruction $I$ and observations $O$ of a dynamic scene, such that the trajectory achieves the navigation goal while satisfying all specifications embedded in the instruction with respect to dynamic entities and regions. The paper assumes that an upstream language module, such as an LLM, decomposes the instruction into $K$ independent specifications,
$$
I \rightarrow \langle \phi^{(1)}, \ldots, \phi^{(k)} \rangle,
$$
where each $\phi^{(i)} : \tau \times O \rightarrow \{0,1\}$ is a binary predicate indicating whether $\tau$ satisfies the $i$-th specification. A trajectory follows the instruction iff
$$
\forall i \in [1,\ldots,k],\ \phi^{(i)}(\tau, O) = 1.
$$
This formalization makes the compositional target explicit: instruction following is reduced to jointly satisfying a set of binary specification predicates [2509.17941].

The trajectory representation is explicitly time-dependent. Trajectories are sequences of 2D waypoints at fixed time intervals, and shorter sequences are padded to a uniform length. This representation yields time-dependent plans suitable for dynamic scenes and low-latency control. Observations are object-centric: each relevant entity is encoded separately, with dynamic agents represented by predicted future positions under a constant-velocity model and regions represented as rectangles specified by four corner points. A transformer encoder attends over entity embeddings to produce a global context feature. Execution is receding-horizon: the generated trajectory is tracked by a model predictive controller, specifically MPPI, and replanned online as observations change [2509.17941].

The decomposition assumption is central. It does not claim that instructions are generated by a monolithic conditional policy; rather, it assumes that constituent specifications can be modeled independently and then recombined. This suggests a deliberate shift away from enumerating full instruction combinations during training and toward learning reusable motion primitives whose joint use is deferred to inference time.

## 2. Conditional diffusion model and compositional mechanism

The base planner models $p(\tau \mid O)$ using a conditional denoising network $f_\theta(\tau_t, t, O)$ that predicts Gaussian noise $\epsilon$ at diffusion step $t$. Training uses the “simple” denoising objective
$$
L_{MSE}(\theta) = E_{x_0,\epsilon,t,c} [\|\epsilon - f_\theta(x_t, t, c)\|^2],
$$
where $c$ is the conditioning context, consisting of environment observations $O$ and time index $t$. The forward diffusion process progressively adds Gaussian noise, and the learned reverse step is
$$
p_\theta(x_{t-1} \mid x_t, c) = N(x_t - f_\theta(x_t, t, c), \sigma_t^2 I),
$$
with $\sigma_t^2$ a time-dependent hyperparameter. At inference, sampling starts from $x_T \sim N(0, I)$ and iteratively denoises to $x_0$ [2509.17941].

The denoising network is a 1D UNet operating over trajectory coordinates, augmented with a context encoder. Each observation, including humans, regions, and the goal, is encoded with MLPs, then fused via a vision transformer into a global context feature. Conditioning is injected into UNet blocks. Classifier-free guidance is implemented by randomly zeroing the context $20\%$ of the time during training. The state space is 2D waypoint space over time; dynamic obstacles are encoded by predicted future positions, with humans modeled as spheres under constant-velocity forecasts and static regions modeled as rectangles [2509.17941].

The compositional mechanism rests on conditional independence. With $\phi^{(i)}$ and its observation $o^{(i)}$, the target conditional over trajectories factorizes as
$$
p(\tau \mid \phi^{(1)}, o^{(1)}, \ldots, \phi^{(k)}, o^{(k)}) \propto p(\tau) \prod_{i=1}^k \frac{p(\tau \mid \phi^{(i)}, o^{(i)})}{p(\tau)}.
$$
Each primitive $i$ has a denoising network $f_\theta^{\phi^{(i)}}(\tau_t, t, o^{(i)})$ trained to represent $p(\tau \mid \phi^{(i)}, o^{(i)})$, together with an unconditioned form $f_\theta^{\phi^{(i)}}(\tau_t, t, \emptyset)$ under classifier-free guidance. The composed noise used during sampling is
$$
\hat{\epsilon} = \frac{1}{k} \sum_{i=1}^k f_\theta^{\phi^{(i)}}(\tau_t, t, \emptyset) + \sum_{i=1}^k w_i \left[ f_\theta^{\phi^{(i)}}(\tau_t, t, o^{(i)}) - f_\theta^{\phi^{(i)}}(\tau_t, t, \emptyset) \right],
$$
with user-set weights $w_i$, set to $1$ in experiments. The reverse step for the composed planner is
$$
p_{\text{compose}}(\tau_{t-1} \mid \tau_t, \{\phi^{(i)}, o^{(i)}\}_{i=1}^k) = N(\tau_t - \hat{\epsilon}, \sigma_t^2 I).
$$
In intuitive terms, the per-primitive scores or noise estimates are added so that sampling is guided toward trajectories jointly likely under all primitives [2509.17941].

At deployment, the procedure is: parse $I$ into $\{\phi^{(i)}\}$ and extract corresponding observations $\{o^{(i)}\}$; initialize $\tau_T \sim N(0, I)$; for $t=T,T-1,\ldots,1$, compute conditioned and unconditioned noise for each primitive, form $\hat{\epsilon}$ via the composition equation, sample $\tau_{t-1}$, and return $\tau_0$ for MPC tracking with adaptive replanning. No additional temperature or scheduling terms are used, and $\sigma_t^2$ follows the diffusion noise schedule [2509.17941].

## 3. Two-stage training procedure

ComposableNav uses a two-stage training procedure. The first stage is supervised pre-training of a base diffusion model for dynamic navigation. The training data are synthetic dynamic environments with varying numbers of agents and regions and randomized goal locations. Collision-free, smooth, goal-reaching trajectories are generated using a spatio-temporal Hybrid A* planner. To diversify feasible behaviors, RRT samples candidate paths, subgoals along these paths are tracked by Hybrid A*, and planner hyperparameters such as velocity costs are varied. The model is trained on time-parameterized trajectory sequences conditioned on object-centric observations, using $L_{MSE}(\theta)$ for noise prediction, classifier-free guidance with $20\%$ null conditioning, EMA over parameters, an exponential noise schedule, and $T=25$ denoising steps. The scale and hyperparameters reported are approximately $2$ million trajectories, $2000$ epochs, learning rate $2 \times 10^{-4}$, and dropout $0.1$ [2509.17941].

The second stage is reinforcement learning fine-tuning using DDPO, or Denoising Diffusion Policy Optimization. DDPO treats each reverse denoising step $p_\theta(x_{t-1}\mid x_t,c)$ as a Gaussian policy in a Markovian MDP with state $s_i \triangleq \langle x_t,t,c\rangle$, action $a_i \triangleq x_{t-1}$, reward only at $t=0$, and zero reward otherwise. PPO updates maximize expected return over denoising trajectories using exact Gaussian log-likelihoods. The objective is
$$
J(\theta) = E\left[\sum_t \gamma^t r_t\right],
$$
instantiated with reward only at the final sample, so that $r_t=0$ for $t>0$ and $r_0=r(x_0,c)$. Advantages are computed over denoising trajectories, and PPO clips ratios while using policy and value heads implicit in the diffusion parameterization via Gaussian log-probabilities [2509.17941].

The rewards are primitive-specific and rule-based, aligned with predicate definitions. For “yield,” the reward penalizes waypoint overlap with the region in front of a person. For “follow,” it requires the robot to remain within a region behind the person over a specified terminal time window. For “walk-through,” overlap with a target region is rewarded. For “avoid,” overlap is penalized. The paper characterizes these rewards as molding the base model into a distinct energy landscape for each primitive. During fine-tuning, each primitive is trained in environments containing only relevant entities, trajectories and rewards are generated and stored in replay buffers, and PPO updates are performed with $32$ environments per epoch, $1000$ epochs, and learning rate $1 \times 10^{-6}$, which the paper reports empirically stabilizes DDPO and is consistent with observations in the literature [2509.17941].

A key significance of this training design is methodological rather than purely architectural. Primitive-specific demonstrations are not required. Instead, a general-purpose planner is first learned from synthetic navigation data and then specialized into interaction behaviors via reinforcement learning. This suggests a scalable route for extending the primitive set, although the paper also identifies reward design as a scaling bottleneck.

## 4. Primitive repertoire, execution, and online replanning

The implemented primitive repertoire has $K=6$ elements: pass a person from the left ($L$), pass a person from the right ($R$), follow behind a person ($F$), yield to a person ($Y$), walk through a specified region ($W$), and avoid walking through a specified region ($A$). Success criteria for each primitive are evaluated by the corresponding $\phi^{(i)}(\tau,O)$ via rule-based region membership checks over time. Novel compositions tested include $24$ instructions spanning $2$ to $4$ specifications, such as $L+R$, $P+F$, $Y+P$, $Y+F$, $W+P$, $W+Y$, $A+P$, and $A+F$, where $P$ denotes pass left or right unified. ComposableNav was not explicitly trained on these compositions; it composes learned primitives at inference [2509.17941].

Execution on robot hardware is receding-horizon. MPPI tracks the generated time-dependent trajectory using a differential drive kinematic model and minimizes deviation over a short horizon subject to acceleration and velocity constraints. Adaptive online replanning uses “Replan from Previous Context,” described as diffusion-based replanning. The current trajectory is perturbed by five forward diffusion steps and then partially denoised by five reverse steps, with states already visited fixed and only future segments updated. Because all per-primitive denoisers share the same architecture, inference can be batched with PyTorch `vmap` and compiled with `torch.compile` [2509.17941].

The reported runtime characteristics are explicit. Control runs at $10$ Hz and replanning occurs every $1.5$ s, or $0.67$ Hz. On a Clearpath Jackal with an Intel i7-9700TE CPU and NVIDIA RTX A2000 GPU, initial planning latencies for $1$ to $4$ composed primitives are $0.144\pm0.014$ s, $0.243\pm0.009$ s, $0.329\pm0.014$ s, and $0.413\pm0.010$ s, respectively. Replanning latencies are $0.027\pm0.002$ s, $0.036\pm0.004$ s, $0.049\pm0.003$ s, and $0.060\pm0.005$ s. Perception uses a Zed 2i camera for human detection and tracking, an Ouster LiDAR for obstacle point clouds and occupancy mapping, and ENML for localization; pedestrian positions are predicted under a constant-velocity model [2509.17941].

These deployment details matter because the method is not presented as an offline planner. The diffusion model generates time-dependent trajectories, MPC tracks them, and replanning modifies only the future portion of the plan. A plausible implication is that compositionality is being used not only to improve semantic coverage of instructions but also to maintain low-latency adaptation in changing scenes.

## 5. Evaluation protocol and empirical results

Simulation experiments are conducted in a $20\times20$ m 2D arena with dynamic humans represented as spheres and static regions represented as rectangles. For each of $24$ instructions, with $2$ to $4$ specifications and $8$ instructions per complexity level, $20$ random environments are generated. The control step is $\Delta t = 0.1$ s and episodes last up to $30$ s. The baselines are VLM-Social-Nav, CoNVOI, and BehAV. In the reported configuration, the baselines are adapted to the setting with GPT-4.1, ground-truth annotations are provided for BehAV to isolate planning, and motion planner hyperparameters are tuned via grid search. Evaluation uses three criteria: Instruction Alignment (IA), Collision-Free (CF), and Goal Reaching (GR). Success requires all three, so $SR=1$ iff $IA=CF=GR=1$ [2509.17941].

| Primitive count | ComposableNav SR | Baseline SRs |
|---|---:|---|
| 1 primitive | 99.8% | VLM-Social-Nav 23.3%; CoNVOI 55.8%; BehAV 75.8% |
| 2 primitives | 75.5% | VLM-Social-Nav 5.0%; CoNVOI 22.5%; BehAV 38.1% |
| 3 primitives | 58.6% | VLM-Social-Nav 0.0%; CoNVOI 10.6%; BehAV 10.0% |
| 4 primitives | 34.9% | VLM-Social-Nav 0.0%; CoNVOI 8.1%; BehAV 6.9% |

As instruction complexity increases, all baselines degrade sharply, reaching at most $11\%$ SR for four primitives, whereas ComposableNav maintains $58.6\%$ for three specifications and $34.9\%$ for four. The paper also reports that primitive-specific fine-tuning via DDPO yields near-perfect primitive execution, with examples such as $L/R/Y/W/A$ achieving approximately $100\%$ IA/CF/GR, while the pre-trained base model alone performs poorly on instruction alignment except for “avoid region,” which is aligned with collision avoidance [2509.17941].

Real-world deployment is evaluated in narrow-doorway and outdoor spaces using six instructions, including passing two people from right or left, passing one and following another, and following one person. Ten trials are conducted per instruction, and the success rates are described as consistently high. Qualitatively, the paper emphasizes preference-aligned behaviors, such as following a person through a doorway instead of overtaking, beyond default geometric navigation, while maintaining low latency on onboard compute [2509.17941].

The empirical pattern is therefore specific. The method is strongest when multiple simultaneous interaction behaviors must be respected in dynamic scenes. The data also show that increased compositional complexity degrades performance, but more slowly than in non-compositional VLM policies and costmap composition baselines.

## 6. Limitations, failure modes, and relation to other compositional navigation paradigms

The paper identifies several limitations. Performance declines as more primitives are composed because independence is imperfect and the noise-summation rule is simplistic. Conflicts between specifications, including mutually incompatible region constraints, can arise. Dense crowds and perception errors, particularly human tracking loss, can degrade instruction alignment and control smoothness. The authors also note that handcrafted rewards for each primitive do not scale well and suggest integrating VLM-based verifiers for DDPO to automate reward design and extend the primitive set. Stronger samplers, such as Hamiltonian Monte Carlo, are proposed as a future direction for improving composition at higher $K$, alongside end-to-end integration of instruction parsing and perception and extension to richer primitive repertoires and social norms [2509.17941].

Within the broader literature, “composable navigation” names several distinct but related design patterns. In NavCon, a low-bandwidth framework for context-aware embodied robot navigation, composability is realized through an intermediate Python-code layer that bridges an LLM and a robot navigation stack; the LLM is constrained to produce Python that only calls allowed API functions, while classical mapping and planning modules remain onboard and enforce safe execution [2306.09523]. In $NavA^3$, composability is hierarchical: a global Reasoning-VLM infers the target object and likely region from high-level instructions and annotated top-down scene views, while a local Pointing-VLM, NaviAfford, performs spatial-aware open-vocabulary object localization for precise control [2508.04598]. In NavComposer, composability concerns language data generation rather than control, decomposing trajectories into semantic entities—actions, scenes, and objects—and recomposing them into navigation instructions that are evaluated by an annotation-free system, NavInstrCritic [2507.10894]. In composable model-free RL for input-affine systems, the principle is expressed at the control level: value functions and optimal policies are learned per environment element and composed online through a quadratically constrained quadratic program to achieve goal reaching and collision avoidance with value-function-level-set guarantees [2602.12492].

These works share a family resemblance but operate at different layers of the navigation stack. ComposableNav [2509.17941] is specifically a compositional diffusion planner for dynamic instruction-following navigation. Its distinctive contribution lies in learning motion primitives as diffusion policies, specializing them by DDPO rather than primitive-specific demonstrations, and composing them by score or noise addition during sampling. This positions it between monolithic instruction-conditioned planners and modular classical systems: more structured than end-to-end language-to-action policies, but more trajectory-generative and dynamics-aware than costmap composition or code-only interfaces.

Source: https://www.emergentmind.com/topics/composablenav