ComposableNav: Composable Diffusion Planner

• ComposableNav is a framework for instruction-following navigation that composes independent motion primitives using diffusion models to generate time-dependent trajectories in dynamic scenes.
• It employs a two-stage training procedure, starting with supervised pre-training on synthetic navigation data followed by reinforcement learning fine-tuning with DDPO to specialize distinct primitives.
• Empirical evaluations in simulation and real-world tests show that ComposableNav outperforms baseline methods in multi-specification tasks, achieving higher rates of instruction alignment, collision avoidance, and goal reaching.

ComposableNav is a framework for instruction-following navigation in dynamic environments that treats an instruction as a composition of independently satisfiable specifications, learns a diffusion model for each corresponding motion primitive, and composes these models at deployment to generate trajectories satisfying novel combinations of specifications unseen in training. In the formulation introduced in "ComposableNav: Instruction-Following Navigation in Dynamic Environments via Composable Diffusion" (Hu et al., 22 Sep 2025), the central problem is combinatorial: as a robot’s skill set expands, each instruction may contain multiple specifications, and the number of possible combinations grows exponentially. ComposableNav addresses this by separating primitive learning from primitive composition, and by using a two-stage training procedure consisting of supervised pre-training for dynamic navigation followed by reinforcement learning fine-tuning that molds the base model into distinct primitives, thereby avoiding primitive-specific demonstration datasets (Hu et al., 22 Sep 2025).

1. Problem formulation and representational assumptions

ComposableNav considers the task of generating a time-parameterized motion trajectory $\tau$ from an instruction $I$ and observations $O$ of a dynamic scene, such that the trajectory achieves the navigation goal while satisfying all specifications embedded in the instruction with respect to dynamic entities and regions. The paper assumes that an upstream language module, such as an LLM, decomposes the instruction into $K$ independent specifications,

$$I \rightarrow \langle \phi^{(1)}, \ldots, \phi^{(k)} \rangle,$$

where each $\phi^{(i)} : \tau \times O \rightarrow \{0,1\}$ is a binary predicate indicating whether $\tau$ satisfies the $i$-th specification. A trajectory follows the instruction iff

$$\forall i \in [1,\ldots,k],\ \phi^{(i)}(\tau, O) = 1.$$

This formalization makes the compositional target explicit: instruction following is reduced to jointly satisfying a set of binary specification predicates (Hu et al., 22 Sep 2025).

The trajectory representation is explicitly time-dependent. Trajectories are sequences of 2D waypoints at fixed time intervals, and shorter sequences are padded to a uniform length. This representation yields time-dependent plans suitable for dynamic scenes and low-latency control. Observations are object-centric: each relevant entity is encoded separately, with dynamic agents represented by predicted future positions under a constant-velocity model and regions represented as rectangles specified by four corner points. A transformer encoder attends over entity embeddings to produce a global context feature. Execution is receding-horizon: the generated trajectory is tracked by a model predictive controller, specifically MPPI, and replanned online as observations change (Hu et al., 22 Sep 2025).

The decomposition assumption is central. It does not claim that instructions are generated by a monolithic conditional policy; rather, it assumes that constituent specifications can be modeled independently and then recombined. This suggests a deliberate shift away from enumerating full instruction combinations during training and toward learning reusable motion primitives whose joint use is deferred to inference time.

2. Conditional diffusion model and compositional mechanism

The base planner models $p(\tau \mid O)$ using a conditional denoising network $I$0 that predicts Gaussian noise $I$1 at diffusion step $I$2. Training uses the “simple” denoising objective

$I$3

where $I$4 is the conditioning context, consisting of environment observations $I$5 and time index $I$6. The forward diffusion process progressively adds Gaussian noise, and the learned reverse step is

$I$7

with $I$8 a time-dependent hyperparameter. At inference, sampling starts from $I$9 and iteratively denoises to $O$0 (Hu et al., 22 Sep 2025).

The denoising network is a 1D UNet operating over trajectory coordinates, augmented with a context encoder. Each observation, including humans, regions, and the goal, is encoded with MLPs, then fused via a vision transformer into a global context feature. Conditioning is injected into UNet blocks. Classifier-free guidance is implemented by randomly zeroing the context $O$1 of the time during training. The state space is 2D waypoint space over time; dynamic obstacles are encoded by predicted future positions, with humans modeled as spheres under constant-velocity forecasts and static regions modeled as rectangles (Hu et al., 22 Sep 2025).

The compositional mechanism rests on conditional independence. With $O$2 and its observation $O$3, the target conditional over trajectories factorizes as

$O$4

Each primitive $O$5 has a denoising network $O$6 trained to represent $O$7, together with an unconditioned form $O$8 under classifier-free guidance. The composed noise used during sampling is

$O$9

with user-set weights $K$0, set to $K$1 in experiments. The reverse step for the composed planner is

$K$2

In intuitive terms, the per-primitive scores or noise estimates are added so that sampling is guided toward trajectories jointly likely under all primitives (Hu et al., 22 Sep 2025).

At deployment, the procedure is: parse $K$3 into $K$4 and extract corresponding observations $K$5; initialize $K$6; for $K$7, compute conditioned and unconditioned noise for each primitive, form $K$8 via the composition equation, sample $K$9, and return $$I \rightarrow \langle \phi^{(1)}, \ldots, \phi^{(k)} \rangle,$$0 for MPC tracking with adaptive replanning. No additional temperature or scheduling terms are used, and $$I \rightarrow \langle \phi^{(1)}, \ldots, \phi^{(k)} \rangle,$$1 follows the diffusion noise schedule (Hu et al., 22 Sep 2025).

3. Two-stage training procedure

ComposableNav uses a two-stage training procedure. The first stage is supervised pre-training of a base diffusion model for dynamic navigation. The training data are synthetic dynamic environments with varying numbers of agents and regions and randomized goal locations. Collision-free, smooth, goal-reaching trajectories are generated using a spatio-temporal Hybrid A* planner. To diversify feasible behaviors, RRT samples candidate paths, subgoals along these paths are tracked by Hybrid A*, and planner hyperparameters such as velocity costs are varied. The model is trained on time-parameterized trajectory sequences conditioned on object-centric observations, using $$I \rightarrow \langle \phi^{(1)}, \ldots, \phi^{(k)} \rangle,$$2 for noise prediction, classifier-free guidance with $$I \rightarrow \langle \phi^{(1)}, \ldots, \phi^{(k)} \rangle,$$3 null conditioning, EMA over parameters, an exponential noise schedule, and $$I \rightarrow \langle \phi^{(1)}, \ldots, \phi^{(k)} \rangle,$$4 denoising steps. The scale and hyperparameters reported are approximately $$I \rightarrow \langle \phi^{(1)}, \ldots, \phi^{(k)} \rangle,$$5 million trajectories, $$I \rightarrow \langle \phi^{(1)}, \ldots, \phi^{(k)} \rangle,$$6 epochs, learning rate $$I \rightarrow \langle \phi^{(1)}, \ldots, \phi^{(k)} \rangle,$$7, and dropout $$I \rightarrow \langle \phi^{(1)}, \ldots, \phi^{(k)} \rangle,$$8 (Hu et al., 22 Sep 2025).

The second stage is reinforcement learning fine-tuning using DDPO, or Denoising Diffusion Policy Optimization. DDPO treats each reverse denoising step $$I \rightarrow \langle \phi^{(1)}, \ldots, \phi^{(k)} \rangle,$$9 as a Gaussian policy in a Markovian MDP with state $\phi^{(i)} : \tau \times O \rightarrow \{0,1\}$0, action $\phi^{(i)} : \tau \times O \rightarrow \{0,1\}$1, reward only at $\phi^{(i)} : \tau \times O \rightarrow \{0,1\}$2, and zero reward otherwise. PPO updates maximize expected return over denoising trajectories using exact Gaussian log-likelihoods. The objective is

$\phi^{(i)} : \tau \times O \rightarrow \{0,1\}$3

instantiated with reward only at the final sample, so that $\phi^{(i)} : \tau \times O \rightarrow \{0,1\}$4 for $\phi^{(i)} : \tau \times O \rightarrow \{0,1\}$5 and $\phi^{(i)} : \tau \times O \rightarrow \{0,1\}$6. Advantages are computed over denoising trajectories, and PPO clips ratios while using policy and value heads implicit in the diffusion parameterization via Gaussian log-probabilities (Hu et al., 22 Sep 2025).

The rewards are primitive-specific and rule-based, aligned with predicate definitions. For “yield,” the reward penalizes waypoint overlap with the region in front of a person. For “follow,” it requires the robot to remain within a region behind the person over a specified terminal time window. For “walk-through,” overlap with a target region is rewarded. For “avoid,” overlap is penalized. The paper characterizes these rewards as molding the base model into a distinct energy landscape for each primitive. During fine-tuning, each primitive is trained in environments containing only relevant entities, trajectories and rewards are generated and stored in replay buffers, and PPO updates are performed with $\phi^{(i)} : \tau \times O \rightarrow \{0,1\}$7 environments per epoch, $\phi^{(i)} : \tau \times O \rightarrow \{0,1\}$8 epochs, and learning rate $\phi^{(i)} : \tau \times O \rightarrow \{0,1\}$9, which the paper reports empirically stabilizes DDPO and is consistent with observations in the literature (Hu et al., 22 Sep 2025).

A key significance of this training design is methodological rather than purely architectural. Primitive-specific demonstrations are not required. Instead, a general-purpose planner is first learned from synthetic navigation data and then specialized into interaction behaviors via reinforcement learning. This suggests a scalable route for extending the primitive set, although the paper also identifies reward design as a scaling bottleneck.

4. Primitive repertoire, execution, and online replanning

The implemented primitive repertoire has $\tau$0 elements: pass a person from the left ($\tau$1), pass a person from the right ($\tau$2), follow behind a person ($\tau$3), yield to a person ($\tau$4), walk through a specified region ($\tau$5), and avoid walking through a specified region ($\tau$6). Success criteria for each primitive are evaluated by the corresponding $\tau$7 via rule-based region membership checks over time. Novel compositions tested include $\tau$8 instructions spanning $\tau$9 to $i$0 specifications, such as $i$1, $i$2, $i$3, $i$4, $i$5, $i$6, $i$7, and $i$8, where $i$9 denotes pass left or right unified. ComposableNav was not explicitly trained on these compositions; it composes learned primitives at inference (Hu et al., 22 Sep 2025).

Execution on robot hardware is receding-horizon. MPPI tracks the generated time-dependent trajectory using a differential drive kinematic model and minimizes deviation over a short horizon subject to acceleration and velocity constraints. Adaptive online replanning uses “Replan from Previous Context,” described as diffusion-based replanning. The current trajectory is perturbed by five forward diffusion steps and then partially denoised by five reverse steps, with states already visited fixed and only future segments updated. Because all per-primitive denoisers share the same architecture, inference can be batched with PyTorch vmap and compiled with torch.compile (Hu et al., 22 Sep 2025).

The reported runtime characteristics are explicit. Control runs at $$\forall i \in [1,\ldots,k],\ \phi^{(i)}(\tau, O) = 1.$$0 Hz and replanning occurs every $$\forall i \in [1,\ldots,k],\ \phi^{(i)}(\tau, O) = 1.$$1 s, or $$\forall i \in [1,\ldots,k],\ \phi^{(i)}(\tau, O) = 1.$$2 Hz. On a Clearpath Jackal with an Intel i7-9700TE CPU and NVIDIA RTX A2000 GPU, initial planning latencies for $$\forall i \in [1,\ldots,k],\ \phi^{(i)}(\tau, O) = 1.$$3 to $$\forall i \in [1,\ldots,k],\ \phi^{(i)}(\tau, O) = 1.$$4 composed primitives are $$\forall i \in [1,\ldots,k],\ \phi^{(i)}(\tau, O) = 1.$$5 s, $$\forall i \in [1,\ldots,k],\ \phi^{(i)}(\tau, O) = 1.$$6 s, $$\forall i \in [1,\ldots,k],\ \phi^{(i)}(\tau, O) = 1.$$7 s, and $$\forall i \in [1,\ldots,k],\ \phi^{(i)}(\tau, O) = 1.$$8 s, respectively. Replanning latencies are $$\forall i \in [1,\ldots,k],\ \phi^{(i)}(\tau, O) = 1.$$9 s, $p(\tau \mid O)$0 s, $p(\tau \mid O)$1 s, and $p(\tau \mid O)$2 s. Perception uses a Zed 2i camera for human detection and tracking, an Ouster LiDAR for obstacle point clouds and occupancy mapping, and ENML for localization; pedestrian positions are predicted under a constant-velocity model (Hu et al., 22 Sep 2025).

These deployment details matter because the method is not presented as an offline planner. The diffusion model generates time-dependent trajectories, MPC tracks them, and replanning modifies only the future portion of the plan. A plausible implication is that compositionality is being used not only to improve semantic coverage of instructions but also to maintain low-latency adaptation in changing scenes.

5. Evaluation protocol and empirical results

Simulation experiments are conducted in a $p(\tau \mid O)$3 m 2D arena with dynamic humans represented as spheres and static regions represented as rectangles. For each of $p(\tau \mid O)$4 instructions, with $p(\tau \mid O)$5 to $p(\tau \mid O)$6 specifications and $p(\tau \mid O)$7 instructions per complexity level, $p(\tau \mid O)$8 random environments are generated. The control step is $p(\tau \mid O)$9 s and episodes last up to $I$00 s. The baselines are VLM-Social-Nav, CoNVOI, and BehAV. In the reported configuration, the baselines are adapted to the setting with GPT-4.1, ground-truth annotations are provided for BehAV to isolate planning, and motion planner hyperparameters are tuned via grid search. Evaluation uses three criteria: Instruction Alignment (IA), Collision-Free (CF), and Goal Reaching (GR). Success requires all three, so $I$01 iff $I$02 (Hu et al., 22 Sep 2025).

Primitive count	ComposableNav SR	Baseline SRs
1 primitive	99.8%	VLM-Social-Nav 23.3%; CoNVOI 55.8%; BehAV 75.8%
2 primitives	75.5%	VLM-Social-Nav 5.0%; CoNVOI 22.5%; BehAV 38.1%
3 primitives	58.6%	VLM-Social-Nav 0.0%; CoNVOI 10.6%; BehAV 10.0%
4 primitives	34.9%	VLM-Social-Nav 0.0%; CoNVOI 8.1%; BehAV 6.9%

As instruction complexity increases, all baselines degrade sharply, reaching at most $I$03 SR for four primitives, whereas ComposableNav maintains $I$04 for three specifications and $I$05 for four. The paper also reports that primitive-specific fine-tuning via DDPO yields near-perfect primitive execution, with examples such as $I$06 achieving approximately $I$07 IA/CF/GR, while the pre-trained base model alone performs poorly on instruction alignment except for “avoid region,” which is aligned with collision avoidance (Hu et al., 22 Sep 2025).

Real-world deployment is evaluated in narrow-doorway and outdoor spaces using six instructions, including passing two people from right or left, passing one and following another, and following one person. Ten trials are conducted per instruction, and the success rates are described as consistently high. Qualitatively, the paper emphasizes preference-aligned behaviors, such as following a person through a doorway instead of overtaking, beyond default geometric navigation, while maintaining low latency on onboard compute (Hu et al., 22 Sep 2025).

The empirical pattern is therefore specific. The method is strongest when multiple simultaneous interaction behaviors must be respected in dynamic scenes. The data also show that increased compositional complexity degrades performance, but more slowly than in non-compositional VLM policies and costmap composition baselines.

6. Limitations, failure modes, and relation to other compositional navigation paradigms

The paper identifies several limitations. Performance declines as more primitives are composed because independence is imperfect and the noise-summation rule is simplistic. Conflicts between specifications, including mutually incompatible region constraints, can arise. Dense crowds and perception errors, particularly human tracking loss, can degrade instruction alignment and control smoothness. The authors also note that handcrafted rewards for each primitive do not scale well and suggest integrating VLM-based verifiers for DDPO to automate reward design and extend the primitive set. Stronger samplers, such as Hamiltonian Monte Carlo, are proposed as a future direction for improving composition at higher $I$08, alongside end-to-end integration of instruction parsing and perception and extension to richer primitive repertoires and social norms (Hu et al., 22 Sep 2025).

Within the broader literature, “composable navigation” names several distinct but related design patterns. In NavCon, a low-bandwidth framework for context-aware embodied robot navigation, composability is realized through an intermediate Python-code layer that bridges an LLM and a robot navigation stack; the LLM is constrained to produce Python that only calls allowed API functions, while classical mapping and planning modules remain onboard and enforce safe execution (Biggie et al., 2023). In $I$09, composability is hierarchical: a global Reasoning-VLM infers the target object and likely region from high-level instructions and annotated top-down scene views, while a local Pointing-VLM, NaviAfford, performs spatial-aware open-vocabulary object localization for precise control (Zhang et al., 6 Aug 2025). In NavComposer, composability concerns language data generation rather than control, decomposing trajectories into semantic entities—actions, scenes, and objects—and recomposing them into navigation instructions that are evaluated by an annotation-free system, NavInstrCritic (He et al., 15 Jul 2025). In composable model-free RL for input-affine systems, the principle is expressed at the control level: value functions and optimal policies are learned per environment element and composed online through a quadratically constrained quadratic program to achieve goal reaching and collision avoidance with value-function-level-set guarantees (Sang et al., 13 Feb 2026).

These works share a family resemblance but operate at different layers of the navigation stack. ComposableNav (Hu et al., 22 Sep 2025) is specifically a compositional diffusion planner for dynamic instruction-following navigation. Its distinctive contribution lies in learning motion primitives as diffusion policies, specializing them by DDPO rather than primitive-specific demonstrations, and composing them by score or noise addition during sampling. This positions it between monolithic instruction-conditioned planners and modular classical systems: more structured than end-to-end language-to-action policies, but more trajectory-generative and dynamics-aware than costmap composition or code-only interfaces.