The Unreasonable Effectiveness of Discrete-Time Gaussian Process Mixtures for Robot Policy Learning (2505.03296v1)

Abstract: We present Mixture of Discrete-time Gaussian Processes (MiDiGap), a novel approach for flexible policy representation and imitation learning in robot manipulation. MiDiGap enables learning from as few as five demonstrations using only camera observations and generalizes across a wide range of challenging tasks. It excels at long-horizon behaviors such as making coffee, highly constrained motions such as opening doors, dynamic actions such as scooping with a spatula, and multimodal tasks such as hanging a mug. MiDiGap learns these tasks on a CPU in less than a minute and scales linearly to large datasets. We also develop a rich suite of tools for inference-time steering using evidence such as collision signals and robot kinematic constraints. This steering enables novel generalization capabilities, including obstacle avoidance and cross-embodiment policy transfer. MiDiGap achieves state-of-the-art performance on diverse few-shot manipulation benchmarks. On constrained RLBench tasks, it improves policy success by 76 percentage points and reduces trajectory cost by 67%. On multimodal tasks, it improves policy success by 48 percentage points and increases sample efficiency by a factor of 20. In cross-embodiment transfer, it more than doubles policy success. We make the code publicly available at https://midigap.cs.uni-freiburg.de.

• The paper introduces Mixture of Discrete-time Gaussian Processes (MiDiGaP) which models robot trajectories as sequences of conditionally independent Gaussian distributions, leveraging temporal data density and modal partitioning to learn multimodal policies from few demonstrations.
• MiDiGaP demonstrates superior performance over deep learning baselines in few-shot learning, speed, smoothness, and handling multimodal/constrained tasks in both simulation and real-world experiments.
• A key advantage of MiDiGaP is its probabilistic nature enabling inference-time adaptation via constrained Gaussian updating and robust cross-embodiment transfer using Variance-Aware Path Optimization (VAPOR).

This paper, "The Unreasonable Effectiveness of Discrete-Time Gaussian Process Mixtures for Robot Policy Learning" (2505.03296), introduces Mixture of Discrete-time Gaussian Processes (MiDiGaP) as a novel approach for robot policy learning, focusing on expressivity, multimodality, sample efficiency, and inference-time adaptability. The authors argue that existing methods like deep learning policies (Diffusion Policy [chi2023diffusionpolicy], Conditional Flow Matching [chisari2024learningroboticmanipulationpolicies]), Gaussian Mixture Models (GMMs) [calinon2006learning], and Continuous Gaussian Processes (CoGaps) [deisenroth2013gaussian] have limitations in areas critical for real-world robotic manipulation, such as needing large datasets, lacking interpretability, struggling with constrained or multimodal movements, or being computationally expensive.

MiDiGaP addresses these limitations by modeling trajectories as a sequence of conditionally independent Gaussian distributions over time. The core idea is to leverage the temporal density of demonstration data, rather than assuming global smoothness (like CoGaps) or local linearity (like GMMs).

The technical approach is structured as follows:

1. Discrete-time Gaussian Process (DiGaP): For unimodal trajectories, a DiGaP is defined as a finite sequence of $T$ Gaussian components $(\boldsymbol\mu_t, \boldsymbol\Sigma_t)_{t=1}^T$. Given a dataset of $N$ trajectories $\{\boldsymbol z_t^n\}_{t=1}^T\}_{n=1}^N$, the mean $\boldsymbol\mu_t$ and covariance $\boldsymbol\Sigma_t$ for each time step $t$ are estimated empirically from the $N$ data points at that time step (\eqref{eq:gp_fit}). The authors diagonalize the covariance matrices $\boldsymbol\Sigma_t$ to avoid learning spurious correlations between manifold dimensions, effectively modeling each dimension's distribution solely as a function of time. This approach requires no assumptions about global trajectory shape and scales linearly with the number of data points.
2. Modal Partitioning: To handle multimodal trajectory distributions, the paper proposes clustering the demonstrations into $M$ subsets, each representing a distinct mode. This is done by concatenating the data points of each trajectory (possibly subsampled in time) into a single vector and clustering these vectors. Three clustering strategies are explored: Riemannian GMM [zeestraten2018programming], Riemannian k-means [steinhaus1957], and DBSCAN [ester1996density]. The choice of clustering method impacts robustness and computation time (\label{tab:modal_part_times_multi}). k-means is found to be a good balance of robustness and speed. This process allows the method to automatically discover the number and nature of distinct behaviors within the demonstrations.
3. Mixture of Discrete-time Gaussian Processes (MiDiGaP): A MiDiGaP combines the DiGaP and modal partitioning. It consists of $M$ DiGaP components, each fitted to one of the subsets of demonstrations identified during modal partitioning. The prior probability $\pi^m$ of each mode $m$ is set as the fraction of demonstrations belonging to that mode's subset. Inference involves first sampling a mode according to these probabilities and then following the mean trajectory of the selected mode.
4. Segmenting and Sequencing of Skills: Long-horizon tasks are handled by automatically segmenting them into shorter skills using Task Auto-Parametrization And Skill Segmentation (TAPAS) [vonhartz2024art]. Each skill is modeled by a MiDiGaP. Skill models are sequenced for inference. The transitions between modes of consecutive skill models can be learned from demonstrated sequences or constructed based on KL divergence between end-points of modes in different skills.
5. Constrained Gaussian Updating: A key advantage of the probabilistic MiDiGaP is its ability to incorporate additional evidence (like obstacles or reachability limits) at inference time. This is done via constrained Gaussian updating, which involves two mechanisms:
   • Modal Updating: Adjusting the prior probabilities $\pi^m$ of the modes based on whether a mode is feasible given the evidence. For instance, if a mode leads to a collision, its likelihood can be reduced or set to zero (\eqref{eq:pi_update}).
   • Convex Updating: For constraints where only part of a mode's probability mass is infeasible (e.g., a mild workspace boundary intersection), the Gaussian parameters $(\boldsymbol\mu_t, \boldsymbol\Sigma_t)$ are updated via moment matching based on samples within the feasible region (\eqref{eq:moment_matching}). The feasibility region for this must be convex. Examples include simplified obstacle avoidance (\eqref{eq:convex-obst}) and basic reachability checks (\eqref{eq:reach_constr}).
6. VAPOR: Variance-Aware Path Optimization: To ensure kinematic feasibility, especially for redundant robots and during cross-embodiment transfer, the end-effector trajectory predicted by MiDiGaP is used to guide a trajectory optimization process in joint space (\eqref{eq:trajCost}). VAPOR incorporates the predicted pose covariance $\boldsymbol\Sigma_t$ from MiDiGaP into the objective function (\eqref{eq:modPoseCost}), effectively allowing the joint trajectory optimization to deviate from the mean end-effector trajectory in directions and magnitudes indicated by the policy's uncertainty, while penalizing deviations in low-variance directions. This process maximizes the likelihood of the resulting end-effector trajectory under the predicted distribution while satisfying joint limits and ensuring smoothness. The resulting optimized trajectory's likelihood can also be used as modal evidence for updating mode probabilities (\label{para:traj_opt_update}). Implementation uses Kineverse [rofer2022kineverse] for kinematics and Augmented Lagrangian [toussaint2017tutorial] for optimization.

For practical application, the method integrates with vision systems for task parameter extraction. The paper uses DINO features [caron2021emerging] for visual generalization and FoundationPose [wen2024foundationpose] for 6-DoF object pose estimation in real-world experiments.

Experimental results in simulation (RLBench) and on a real Franka Emika robot demonstrate MiDiGaP's effectiveness.

• Unimodal Tasks: MiDiGaP significantly outperforms deep learning baselines (Diffusion Policy, LSTM) and the GMM-based baseline (TAPAS-GMM) on few-shot tasks (5 demonstrations), particularly on highly constrained tasks (e.g., OpenMicrowave, WipeDesk) where trajectory shape is critical (\label{tab:success_rates_uni}, \label{tab:success_rates_real_uni}). It also produces substantially smoother trajectories (\label{tab:ee_acc_uni}). Computationally, MiDiGaP is orders of magnitude faster for training and inference than Diffusion Policy, even running on a CPU (\label{tab:inf_times}).
• Multimodal Tasks: MiDiGaP effectively learns multimodal policies from few shots (10-15 demonstrations), significantly outperforming baselines (\label{tab:success_rates_multi}, \label{tab:success_rates_real_multi}). The modal partitioning successfully identifies distinct modes, sometimes revealing unexpected variations in expert demonstrations (\label{fig:turntap_four_modes}).
• Constrained Gaussian Updating: Inference-time updating allows MiDiGaP to robustly handle novel obstacles and limited reachability by re-weighting modes and deforming trajectories, achieving near-perfect success rates in challenging scenarios where baselines fail (\label{tab:gu_collision}, \label{tab:success_rates_real_update}).
• Cross-Embodiment Transfer: VAPOR enables effective transfer of policies learned on a Franka Emika to a UR5 robot. By optimizing the joint trajectory considering the policy's variance and the target robot's kinematics, VAPOR significantly improves success rates compared to naive transfer or methods lacking variance awareness (\label{tab:success_rates_traj_opt_uni}, \label{tab:success_rates_traj_opt_mult}). Transferred policies often outperform those trained directly on the UR5, potentially due to capturing more variability from the 7-DoF Franka.
• Real-World Validation: Experiments on a real Franka robot confirm simulation findings, showing robust performance on diverse tasks, effective visual generalization using DINO and FoundationPose, and successful obstacle avoidance via constrained updating (\label{tab:success_rates_real_uni}, \label{tab:success_rates_real_multi}, \label{tab:success_rates_real_update}).

Limitations include reliance on the quality of underlying perception systems, lack of emergent behaviors like retrial (which might need to be added via external execution monitoring), focus on end-effector motions (less suited for tasks requiring learning in joint space like walking), and dependence on segmentable skills (as handled by TAPAS).

In summary, MiDiGaP provides a powerful, efficient, and interpretable framework for learning robot manipulation policies from few demonstrations. Its probabilistic nature enables flexible inference-time adaptation and robust cross-embodiment transfer via variance-aware trajectory optimization, making it a practical solution for real-world robotic tasks.