PAKE: Learning Whole-Body Loco-Manipulation with Partial Kinematic Embeddings

Abstract: Loco-manipulation has recently shown promising capabilities; however, achieving high-precision control, managing the high-dimensional action space induced by many degrees of freedom (DoFs), and fully exploiting the inherent redundancy of whole-body systems remain challenging. In this paper, we propose a novel whole-body control framework that effectively addresses these challenges by decomposing the complex loco-manipulation problem into partial reference motion generation and low-level imitation control. We introduce a new Kinematic Normalizing Flow (KNF) model, trained on a large-scale kinematic dataset, that generates diverse yet feasible partial reference motions. A high-level controller is then trained to navigate the KNF's latent space to exploit redundant solutions, while a low-level controller ensures physically feasible and accurate motion execution. We validate our approach on the quadrupedal robot equipped with a six-DoF robotic arm. In simulation, experimental results show that our approach significantly outperforms state-of-the-art methods in terms of tracking accuracy and feasible workspace coverage. For hardware deployment, we evaluate the system over 24 episodes across 8 different mobile loco-manipulation tasks. The system achieves end-effector pose-tracking errors of 4.5 cm and 0.14 rad, while maintaining accurate locomotion tracking with linear and angular velocity errors of 0.1 m/s and 0.01 rad/s, respectively, outperforming competitive baselines. Our method represents a practical and powerful solution for accurate and generalized whole-body loco-manipulation in high-DoF robotic systems, with promising potential for diverse downstream robotic tasks.

• The paper introduces a novel hierarchical framework integrating partial kinematic embeddings with low-level imitation and high-level policy control for robust and precise whole-body loco-manipulation.
• It employs a Kinematic Normalizing Flow trained on 170M samples to generate multimodal inverse kinematics solutions, enhancing workspace coverage and mitigating local minima.
• Empirical evaluations show PAKE outperforms baseline methods with low position (4.5 cm) and orientation (0.14 rad) errors, proving its efficacy in simulation and real-world deployments.

PAKE: Learning Whole-Body Loco-Manipulation with Partial Kinematic Embeddings

Overview and Motivation

The paper "PAKE: Learning Whole-Body Loco-Manipulation with Partial Kinematic Embeddings" (2607.11041) introduces a structured framework for high-precision whole-body control in high-DoF legged robotic systems equipped with manipulators. The motivation is to address the challenges inherent in loco-manipulation: coordinating multiple, often conflicting objectives across a high-dimensional action space, exploiting kinematic redundancy for both locomotion and manipulation, and achieving robust sim-to-real transfer for diverse real-world deployments. Existing solutions either overly restrict mobility, lose precision under dynamic conditions, or fail to utilize the full space of redundant solutions; PAKE advances the field by integrating a novel redundancy-aware motion embedding with a hierarchical control scheme designed for scalability and accuracy.

Hierarchical Framework Design

PAKE decomposes the complex task of whole-body loco-manipulation into two subproblems:

1. Partial Reference Motion Generation: Using a Kinematic Normalizing Flow (KNF), PAKE learns a distribution over feasible inverse kinematic solutions conditioned on end-effector targets, capturing the inherent redundancy of the arm and torso without over-constraining base mobility. The KNF is trained on a large-scale dataset (170M samples) synthesized via parallel forward kinematics sampling for the arm and torso (6 DoF + 3 DoF), offering multimodal, latency-minimized motion solutions.
2. Low-Level Imitation Control: A pre-trained low-level controller (LLC) tracks partial reference motions and velocity commands, filtering out physically infeasible trajectories, and producing actuator commands for all DoFs, effectively bridging the gap between purely kinematic planning and dynamic, contact-rich execution.
3. High-Level Redundancy-Aware Control: A high-level controller (HLC) explores the KNF latent space to optimize action selection, harnessing solution diversity to mitigate problems like local minima and adversarial objective conflicts between manipulation and locomotion tasks.

Figure 1: The pipeline illustrates dataset generation, KNF training, low-level imitation controller learning, and high-level policy optimization in PAKE.

Partial Kinematic Embeddings and Data Generation

The partial kinematic embedding comprises torso roll $\xi$, pitch $\theta$, height $h$, and arm configuration, intentionally leaving out torso position and yaw to preserve mobile manipulation capabilities Figure 2. The end-effector poses are defined within a heading frame to support downstream adaptation to diverse manipulation and locomotion tasks. Normalizing Flows, specifically the Glow architecture, are used to conditionally model the multimodal mapping between end-effector targets and partial kinematic solutions, allowing rapid sample generation for high-level policy optimization.

Figure 2: The partial reference motion model includes torso height, roll, pitch, and robotic arm DoFs, referenced in the heading frame.

Control Policy Design and Training

Low-Level Controller

The LLC is trained via PPO in an asymmetric actor-critic setup, tasked with tracking both 3D velocity commands and partial reference torso configurations. Curriculum sampling ensures progressive expansion of command space and enhances policy robustness. Input modalities for the actor include extended proprioceptive histories, action history, end-effector targets, and partial reference motion vectors. Domain randomization is employed to support zero-shot sim-to-real transfer.

High-Level Controller

The HLC is likewise trained via PPO, but operates in the latent space of the KNF. Rewards include end-effector tracking, velocity tracking, torso posture, smoothness, and regularization. The sampled end-effector trajectories use third-order rational Bézier curves, a departure from simple interpolation, enabling more complex, dynamic, and realistic manipulation.

Empirical Evaluation

PAKE is extensively evaluated in simulation and real-world experiments, addressing:

• Tracking Precision: Mean position error drops to 4.5 cm and mean orientation error to 0.14 rad, substantially outperforming baselines across all metrics.
• Workspace Coverage: PAKE achieves superior feasible workspace volumes under strict tracking error thresholds, indicating better exploitation of system redundancy and avoidance of local minima.

Figure 3: Tracking error distributions for position, orientation, and velocity metrics demonstrate PAKE's centralized, low-error performance against baselines.

Figure 4: 3D visualization shows spatial mapping of tracking errors, with lighter colors indicating increased error.

• Robustness Under Dynamic Conditions: In high-speed tracking scenarios, PAKE maintains performance with end-effector errors stabilized within 5 cm and 5 degrees shortly after command changes.

Figure 5: Performance of PAKE under highly dynamic chassis and manipulation commands in 30s.

• Real-World Deployment: Across 24 hardware episodes covering eight task families (e.g., cart pulling, sweeping, plugging, picking, placing), PAKE achieves robust, consistent performance, confirming robustness to task diversity and object interaction complexity.

Figure 6: Real-world deployments of PAKE across diverse manipulation tasks with mobile quadrupedal robots.

Numerical Results and Architectural Insights

The ablation study confirms that redundancy-aware latent space exploration via the HLC is necessary for optimal tracking; eliminating the HLC (i.e., using KNF at zero latent scale) degrades performance to baseline levels and suppresses solution diversity. PAKE outperforms both analytic Jacobian-based and RL-only methods in pose and velocity tracking, as well as workspace coverage. The architecture's ability to decouple high-level redundant kinematic planning from low-level dynamic feasibility enables stable, generalizable, and precise whole-body control—especially critical for tasks where manipulation and locomotion are tightly coupled.

Practical and Theoretical Implications

PAKE's integration of normalizing flow-based kinematic embedding, hierarchical decoupling, and curriculum-based training embodies a scalable approach for future high-DoF mobile manipulators. Practically, PAKE supports omni-directional chassis mobility without constraining workspace, enables robust sim-to-real deployment, and facilitates manipulation tasks requiring dynamic chassis and end-effector coordination. Theoretically, the framework offers a foundation for further exploration of structured latent space optimization, multimodal solution sampling, and redundancy exploitation in legged and mobile manipulation.

Future research could extend PAKE towards more generalized multi-task learning, hierarchical skill composition, vision-based input for end-effector target specification, and integration with semantic or language-based task planning systems, advancing towards general-purpose, highly capable mobile manipulators.

Conclusion

PAKE proposes a hierarchical, redundancy-aware framework for whole-body loco-manipulation in high-DoF legged robotic systems, combining Normalizing Flow-based partial kinematic embedding, curriculum-trained low-level imitation, and high-level latent space policy optimization. Extensive empirical evidence demonstrates superior tracking precision, workspace coverage, robustness to highly dynamic conditions, and applicability across diverse real-world manipulation tasks, providing both a technically robust and practically scalable approach for future developments in mobile robotic manipulation.