---
title: 'FlowMPPI: Deep Learning MPC'
url: https://www.emergentmind.com/topics/flowmppi
type: topic
---

# FlowMPPI: Deep Learning MPC

FlowMPPI is a family of sampling-based Model Predictive Control (MPC) algorithms that leverages conditional normalizing flows to learn expressive, context-aware distributions over optimal control sequences. These methods fuse classical MPPI with deep probabilistic inference, particularly variational inference, to produce control samples that are both goal-directed and sensitive to environmental constraints, including out-of-distribution (OOD) scenarios. The approach addresses sample efficiency, generalization, and the limitations of conventional Gaussian sampling in high-dimensional, cluttered, or previously unseen environments [2205.04667], [2212.02587].

## 1. Control-Sequence Distribution via Conditional Normalizing Flows

FlowMPPI replaces traditional factorized Gaussian samplers in MPPI with a conditional normalizing flow parameterization. The control sequence $U=(u_0,\ldots,u_{H-1}) \in \mathbb{R}^{H\cdot d_u}$ for horizon $H$ is drawn from a distribution $q_\phi(U|x_0,x_G,E)$, constructed as follows [2205.04667]:

- A variational autoencoder (VAE) encoder $q_\theta(h|E)$ maps the environment's signed distance function (SDF) $E$ into a low-dimensional latent vector $h$.
- A context MLP $g_\omega(x_0,x_G,h)$ produces a context vector $C$.
- A conditional normalizing flow $f_\phi$ transforms a base Gaussian $Z\sim N(0,I)$ into $U$, conditioned on $C$: 
  $$
  U = f_\phi(Z;C), \quad Z = f_\phi^{-1}(U;C).
  $$
The density is given by the change-of-variables formula:
$$
q_\phi(U|C) = N(Z;0,I)\cdot \exp(-\log|\det\partial f_\phi/\partial Z|), \quad Z=f_\phi^{-1}(U;C).
$$
Context encodes the current state $x_0$, goal $x_G$, and latent environment $h$. This amortized, conditional sampling mechanism enables the distribution to adapt both to robot dynamics and specific environmental geometries.

## 2. Objective: Variational Inference Perspective

The finite-horizon stochastic optimal control problem is reframed as inference on an optimality variable $o$ with
$$
p(o=1|\tau) \propto \exp(-J(\tau)), \quad \tau=(X,U),
$$
where $J(\tau)$ is the trajectory cost. The variational free-energy (up to constants) minimized by FlowMPPI is
$$
\mathcal{F}[\phi] = \mathrm{KL}(q_\phi(U)\,\|\,p(U|o=1)) = -\mathbb{E}_{q(\tau)}[\log p(o|\tau)] - H(q_\phi(U)),
$$
with $J(\tau)$ absorbing log-prior costs. The training objective combines maximizing likelihood of low-cost samples with regularization via entropy and flow terms:
$$
\mathcal{F} = -\mathbb{E}_{q(h)q_\phi(U|C)}[-J(\tau)] + \mathbb{E}_{p(Z)}[\log p(Z) - \log|\det\partial f_\phi/\partial Z|].
$$
In practice, this objective is optimized using weighted maximum likelihood on sampled trajectories.

## 3. MPPI Algorithm Augmented with Learned Flow

At each MPC step, FlowMPPI interleaves classic MPPI perturbations with environment-aware flow samples. The algorithmic procedure is:

1. Shift previous nominal control $U^0$ forward by one step.
2. Compute nominal latent $Z^0=f_\phi^{-1}(U^0;C)$.
3. Draw $K/2$ samples via classic Gaussian noise and $K/2$ via the flow.
4. For each, rollout trajectories, compute cost-augmented scores $S^k$ (including auxiliary terms penalizing deviation from nominal in control or latent space).
5. Compute softmax weights $w_k$ and form the weighted update for $U^0$.
6. Apply the first action to the plant.

The step integrates context-conditioned latent space sampling and explicit regularization paralleling KL-divergence in classic MPPI. The batch includes both noise-based perturbations and flow-induced proposals [2205.04667], [2212.02587].

## 4. Out-of-Distribution Projection for Robust Generalization

FlowMPPI incorporates an OOD-projection mechanism for test-time robustness when the environment $E$ is not covered by the training data. This projects the environment latent $h$ to a nearby in-distribution point by minimizing
$$
\hat{h} = \arg\min_h\ [\,b\cdot L_{OOD}(h) + L_{\mathrm{flow}}(h)\,],
$$
where
- $L_{OOD}(h) = -\log p_{\phi_{\mathrm{prior}}}(h)$ (the flow prior density from the VAE, high if $h$ is unlikely),
- $L_{\mathrm{flow}}(h)$ is the weighted negative log-likelihood for flow-sampled control sequences,
- $b$ modulates the in-distribution tradeoff.

This optimization adapts the flow's conditioning context for OOD environments, enhancing success rates and costs without retraining.

## 5. Architectural Details and Training Protocol

FlowMPPI and its variants employ the following architectural elements [2205.04667]:

- VAE over environment SDF ($E\in \mathbb{R}^{64\times64}$ or $\mathbb{R}^{64\times64\times64}$): 4-stride-2 convolutions $\to$ FC $\to$ $(\mu_h,\sigma_h)$; decoder reverses structure. Latent prior on $h$ uses a 4-layer Real-NVP flow.
- Context MLP $g_\omega$: concatenates $(x_0,x_G,h)$, 256-ReLU hidden, outputs $C$.
- Control-flow $f_\phi$: Real-NVP with 10 conditional coupling layers, batch-norm, and mixing.
- Planar system: $d_x=4$, $d_u=2$, $d_h=d_C=64$; quadrotor: $d_x=12$, $d_u=4$, $d_h=d_C=256$.
- Hyperparameters: $H=40$, sampling budgets $K\in\{256,512,1024\}$, Adam learning rate $1\mathrm{e}{-3}$, $\lambda=1$ temperature, and $\Sigma_U$ tuned per system. VAE loss is weighted and frozen after 100 epochs; control-perturbation variance is annealed.

For FlowMPPIProj (with projection), half the step's sample budget is allocated to $L_{\mathrm{flow}}$ computation for projection. All components are compatible with standard deep learning frameworks.

## 6. Empirical Evaluation and Performance

FlowMPPI is benchmarked on navigation and control tasks in both in-distribution and OOD scenarios:

- Planar 2D double-integrator navigation with disc obstacles (in-distribution) and narrow four-room environments (OOD).
- 12DoF quadrotor in 3D spheres (in-distribution), four-room corridor (OOD), and two real-world reconstructions.

The following table summarizes selected results [2205.04667]:

| Method         | Succ (2D OOD, K=512) | Cost (2D OOD, K=512) | Succ (3D OOD, K=512) | Cost (3D OOD, K=512) |
|----------------|----------------------|----------------------|----------------------|----------------------|
| MPPI           | 0.29                 | 2948                 | 0.11                 | 4724                 |
| iCEM           | 0.59                 | 2145                 | 0.47                 | 4157                 |
| FlowMPPI       | 0.75                 | 2155                 | 0.72                 | 3601                 |
| FlowMPPIProj   | 0.77                 | 2155                 | 0.83                 | 3443                 |

A similar trend is seen in real-world sim2real transfer, where FlowMPPIProj achieves substantial improvements in both success rate and cost. For example, in the stairway environment for the quadrotor:
- MPPI: succ 0.32, cost 3019
- iCEM: succ 0.58, cost 2623
- FlowMPPI: succ 0.50, cost 2463
- FlowMPPIProj: succ 0.85, cost 1745

Notably, baseline methods frequently get stuck in local minima in complex environments, where flow-based sampling yields goal-oriented, collision-free solutions from the first iteration.

## 7. Relationship to and Advances over Previous Methods

Earlier sampling-based MPC approaches typically employ Gaussian or control-space sampling, leading to suboptimal exploration and reliance on heuristics for parameter updates. FlowMPPI, especially when combined with bi-level optimization and end-to-end backpropagation-through-time (BPTT) training in a latent space [2212.02587], achieves:

- An order-of-magnitude improvement in sample efficiency,
- 10–20% lower median cost across domains,
- Consistent or improved success rates,
- Robust generalization to out-of-distribution and real-world environments.

The integration of normalizing flows enables expressive, tractable distributions that account for environment geometry and dynamics. The OOD-projection step further enables reliable sim2real transfer, as demonstrated on real-world datasets without requiring retraining.

FlowMPPI has demonstrated a marked advance in the application of probabilistic learning and inference to MPC, especially in scenarios where classical methods are challenged by non-Gaussianity, environmental complexity, or dataset shift [2205.04667], [2212.02587].

Source: https://www.emergentmind.com/topics/flowmppi