---
title: Conditional Neural Movement Primitives
url: https://www.emergentmind.com/topics/conditional-neural-movement-primitive-cnmp
type: topic
---

# Conditional Neural Movement Primitives

Conditional Neural Movement Primitives (CNMPs) are conditional, probabilistic auto-encoders that model continuous time-series such as robot trajectories. The CNMP framework is designed for learning from demonstration, enabling unsupervised discovery and encoding of complex skills, as well as providing high-fidelity reconstruction and inference of trajectories. Vector-Quantized CNMP (VQ-CNMP) augments the CNMP architecture with a discrete skill bottleneck, supporting unsupervised clustering of demonstrations into symbolic skills while retaining the expressiveness of continuous representations for trajectory execution. VQ-CNMP serves as the backbone of a neuro-symbolic bi-level planning pipeline, enabling both symbolic high-level planning with large language models and differentiable low-level trajectory refinement via gradient-based optimization [2410.10045].

## 1. Foundations of Conditional Neural Movement Primitives

CNMPs represent an unsupervised sequence modeling framework for robot trajectories, operational over datasets of demonstrations 
$$
\mathcal{D} = \{ D_1, \ldots, D_M \},
$$ 
where each demonstration $D_j$ is a time-ordered series of sensorimotor pairs:
$$
D_j = \{ (t_i, x_i) \}_{i=1}^{n_j}, \quad t_i \in \mathbb{R}, \ x_i \in \mathbb{R}^d.
$$
Each $(t_i, x_i)$ is encoded via a neural function $E_\theta$, yielding pointwise latents $z_i$:
$$
z_i = E_\theta(t_i, x_i).
$$
Contextual aggregation is performed by averaging these latents:
$$
z_e = \frac{1}{n} \sum_{i=1}^n z_i.
$$
The aggregated latent $z_e$ is concatenated with a query timestamp $t^*$ and passed to a decoder $Q_\phi$, producing the parameters of a Gaussian distribution over the sensorimotor state:
$$
(\mu(t^*), \sigma^2(t^*)) = Q_\phi([z_e; t^*]).
$$
This defines the conditional likelihood of the state $x$ at time $t^*$:
$$
p(x \mid t^*, z_e) = \mathcal{N}(x; \mu(t^*), \sigma^2(t^*)).
$$
The training objective is the negative log-likelihood over held-out queries, 
$$
\mathcal{L}_{\text{CNMP}} = - \sum_{(t^*, x) \in \text{queries}} \log p(x \mid t^*, z_e).
$$
This configuration enables CNMPs to learn flexible, temporally-conditioned generative models from unordered demonstration segments.

## 2. Vector-Quantized CNMP: Discrete Skill Bottleneck

VQ-CNMP introduces a discrete codebook $\mathcal{V} = \{ v_1, \ldots, v_K \}$, $v_k \in \mathbb{R}^D$, creating a symbolic bottleneck for skill discovery. After context aggregation, the continuous latent $z_e$ is quantized:
$$
k^* = \arg \min_{m = 1, \ldots, K} \| z_e - v_m \|_2, \quad z_q = v_{k^*}.
$$
Decoding then proceeds as in the original architecture:
$$
(\mu(t^*), \sigma^2(t^*)) = Q_\phi([z_q; t^*]).
$$
The VQ-CNMP loss combines the CNMP likelihood with the vector-quantization and commitment penalties:
$$
\mathcal{L} = \mathcal{L}_{\text{CNMP}}
+ \| \text{sg}(z_e) - z_q \|_2^2
+ \beta \| z_e - \text{sg}(z_q) \|_2^2,
$$
where $\text{sg}(\cdot)$ denotes the stop-gradient operator and $\beta$ is a small constant (standard choice $\beta = 0.25$).

Codebook optimization employs standard SGD, updating each codeword towards encoder outputs selected during the quantization step.

## 3. Self-Supervised Fine-Tuning for Latent Precision

Following initial unsupervised training, each trajectory demonstration is assigned a codebook index $k^*$. Demonstrations are clustered by their discrete assignments. A second CNMP is then re-trained in a fully supervised regime: trajectories from each cluster are encoded and decoded with the corresponding fixed latent $z_q = v_k$. This procedure greatly enhances trajectory reconstruction precision and ensures higher success rates in downstream low-level planning tasks.

## 4. Neuro-Symbolic Bi-Level Planning Pipeline

The bi-level pipeline exploits the structure learned in VQ-CNMP, integrating symbolic planning at the skill level with continuous refinement for physical task execution.

- **High-Level Skill Planning**: Each codebook vector $v_k$ is associated with a skill description (e.g., "take tomato from left cupboard and add to pan"). A multi-modal LLM (e.g., ChatGPT-4o) is prompted with an overhead image, the list of skill descriptions, and an NL-formulated goal. The LLM returns an ordered list of skill indices $[k_1, \ldots, k_N]$ to achieve the specified goal.

- **Low-Level Trajectory Planning (Gradient-Based Optimization)**: For each high-level step $k_i$, the latent $z^{(0)} = v_{k_i}$ is refined to align trajectory predictions with precise environmental constraints (e.g., object positions). The loss at contact time $t_c$ is
$$
L^{(j)} = \| \mu_c - p_{\text{obj}} \|_2^2,
$$
with $p_{\text{obj}}$ as the target 3D position. Gradient descent is applied to $z$:
$$
z^{(j+1)} = z^{(j)} - \alpha \nabla_z L^{(j)}.
$$
Refinement iterates until convergence ($\| \mu_c - p_{\text{obj}} \| < \epsilon$), after which the decoder is used to generate the full low-level trajectory for the skill step. Final plans concatenate these trajectories sequentially.

## 5. Modifications to Inference and Comparison to Standard CNMP

At inference, standard CNMPs encode context and decode from the continuous latent $z_e$, preserving the original trajectory distribution. In VQ-CNMP, inference defaults to hard quantization: the nearest codebook vector $v_k$ is used for decoding, optionally refined through the aforementioned gradient-based update for task-specific adjustments. Post self-supervised fine-tuning, inference rolls in using the assigned $v_k$ directly, eschewing nearest-neighbor search in favor of deterministic encoding.

This bifurcation between standard and vector-quantized inference regimes enables flexible trade-offs between symbolic skill abstraction and low-level control fidelity.

## 6. Empirical Validation: Skill Discovery, Symbolic Planning, and Control

Experiments were performed in a simulated kitchen with five parametrized skills, each with 100 demonstration trajectories and randomized object placements. Key quantitative results include:

- **Skill Discovery**: Training 100 VQ-CNMPs with $K=5$ codewords, perfect cluster-to-skill mapping was achieved in 27% of runs; mean clustering accuracy was 0.97. Increasing $K$ above the true number of skills gradually reduced clustering performance, as excess codes subdivide true skill classes.
- **Skill Labeling by LLM**: Using decoded trajectories as visual prompts for ChatGPT-4o, labeling accuracy with five consecutive snapshots was approximately 60% (randomized snapshots up to 71% for "pickup tomato"), indicating incomplete but non-trivial recognition by state-of-the-art LLMs.
- **High-Level Planning Accuracy**: Given only the five required skills, plans generated for assembling "stew of X, Y, Z" had 59.7% symbolic correctness. Introducing distractor skills (relevant or irrelevant) decreased performance, but explicit provision of object locations and hidden-object skills increased success up to 98.4%.
- **Low-Level Trajectory Execution**: Self-supervised fine-tuned VQ-CNMP significantly exceeded unsupervised baselines in control success (e.g., for "pickup": 100% vs. 80-90%, for "put-in-pan": 80-90% vs. 0%), with convergence times of ~70 versus ~400 gradient steps.

| Skill      | Pickup (unsup.) | Pickup (self-sup.) | Put-in-pan (unsup.) | Put-in-pan (self-sup.) |
|------------|-----------------|--------------------|---------------------|------------------------|
| Tomato     |      80%        |      100%          |         0%          |          90%           |
| Mushroom   |      40%        |      100%          |         0%          |          85%           |
| Potato     |       0%        |       90%          |         0%          |          80%           |
| Salt       |      40%        |      100%          |         0%          |          85%           |
| Oil        |      90%        |      100%          |         0%          |          80%           |

This combination of discrete skill discovery, symbolic planning, and differentiable trajectory refinement illustrates the capacity of VQ-CNMP for integrated neuro-symbolic planning in continuous control domains [2410.10045]. 

## 7. Significance and Prospects

VQ-CNMP extends the expressiveness of CNMPs by introducing discrete, interpretable skill symbols, supporting unsupervised skill abstraction and reliable low-level action planning. Self-supervised fine-tuning resolves cluster assignment ambiguities and increases trajectory precision. The bi-level pipeline unifies symbolic reasoning and differentiable control, facilitating scalable skill discovery, labeling, planning, and execution in complex, sensorimotor environments. 

A plausible implication is that future neuro-symbolic planners may further enhance generalization by incorporating active skill discovery, context-driven codebook adaptation, or closed-loop integration with multimodal LLMs for semantically grounded task specification and robust execution in unstructured environments [2410.10045].

Source: https://www.emergentmind.com/topics/conditional-neural-movement-primitive-cnmp