Whole-Body Manipulation: Taxonomy and Control

• Whole-Body Manipulation (WBM) is the coordinated use of a robot's entire body, integrating multi-contact interactions for locomotion, manipulation, and stability.
• Pose taxonomies classify configurations into standing, kneeling, and resting categories, enabling systematic planning and safe transitions between support states.
• Data-driven segmentation and motion primitive synthesis improve autonomous planning, facilitating real-time adaptation and robust control of complex tasks.

Whole-Body Manipulation (WBM) refers to the use of an entire robotic body—not just end-effectors—to interact physically with the environment in support of complex tasks such as locomotion, manipulation, stability enhancement, and coordinated multi-contact actions. This paradigm extends classical manipulation (primarily involving hands or grippers) to encompass rich contact interactions across legs, arms, torso, and in some cases non-standard body regions, enabling robustness, dexterity, and adaptivity in scenarios that demand integrated movement and force control.

1. Taxonomies of Whole-Body Poses

A foundational element in WBM is the systematic classification of possible bodily configurations that exploit environmental contacts. Borrowing from the principles of grasp taxonomies, a detailed whole-body pose taxonomy has been proposed that enumerates 46 distinct classes grouped into three principal categories: standing, kneeling, and resting poses (Borràs et al., 2015). Key features include:

• Standing Poses: Primarily stabilized by the feet (possibly augmented with arm contacts), offering a trade-off between mobility and static stability.
• Kneeling Poses: Utilize one or both knees for contact, reducing degrees of freedom but improving ground stability.
• Resting Poses: Involve considerable torso contact, ranging from dynamic (active stabilization) to static (passive rest), further sub-classified (r.1 to r.10).

Each class is differentiated by the number and type of support contacts (feet, knees, arms, torso; point vs. plane contacts), enabling formal reasoning over possible motion primitives and transitions (e.g., from single-leg to double-leg support, or from kneeling to standing). The taxonomy graph explicitly encodes feasible transitions as single-contact changes, thus providing a structure for decomposing complex motions into atomic pose transitions—akin to the edges of a contact graph in grasp planning. This structure simplifies the combinatorics of possible configurations and enables the systematic design and sequencing of whole-body movements.

2. Pose–Stability Relationships and Formalism

The relationship between pose selection and system stability is central to WBM. Each support configuration induces a unique set of equilibrium constraints, with greater numbers or larger surface contacts generally equating to increased resistance to disturbance. The formal representation of a support contact is

$C = \{l, m, c, n\}$

where $l$ is the contacting link, $m$ encapsulates the contact model (number/type of constraints), $c$ the global location, and $n$ the surface normal. An instantiated pose is described by

$$\{\text{id},\, p,\, \mathcal{C} = \{C_i\}_{i=1}^m,\, \mathcal{N}\}$$

with id referencing the taxonomy class, $p$ the center of mass, $\mathcal{C}$ the set of contacts, and $\mathcal{N}$ its neighboring classes for allowed transitions.

Stability, in this framework, is assessed analogously to force closure in grasping: poses with more and higher-dimensional constraint sets (multiple or planar contacts) admit a broader range of external disturbances before losing equilibrium (Borràs et al., 2015). This formalism underpins the design of motion planners and controllers capable of orchestrating transitions that exploit the spectrum of achievable stabilities.

3. Motion Primitives: Within-Class and Transition-Class Actions

WBM planning and synthesis are guided by two core classes of motion primitives:

• Inside-Class Motion: Actions that execute manipulation or locomotor subtasks without altering the underlying support configuration (e.g., manipulating a tool while in double-stance).
• Transition-Class Motion: Motions that change the support composition (e.g., shifting from kneeling to standing).

Motion primitives are linked to the taxonomy: inside-class motions allow for manipulation or local interaction under the same constraint set, while transition-class motions correspond to edges in the taxonomy graph. The system supports modular sequencing, where complex whole-body actions can be synthesized by concatenating primitives associated with individual poses and transitions (Borràs et al., 2015). Motion storage and recombination rules enable efficient imitation learning and autonomous composition of novel behaviors.

4. Data-Driven Validation and Segmentation

Application of the taxonomy to human motion data has been demonstrated through segmentation of whole-body motion capture (KIT whole-body motion database) (Borràs et al., 2015). Support contacts are detected via collision checking (between end-effectors and environment) and confirmed through velocity analysis with a 0.15 m/s threshold (support phases exhibit near-zero velocity at the point of contact). Actions are segmented into distinct support phases, with transition edges forming a temporal motion graph.

In empirical validation, such segmentation accurately differentiates locomotor phases (e.g., transitions while ascending stairs and manipulating a handrail) from manipulation phases (e.g., stationary push/pull actions while support is maintained). This ability enables not only semantic understanding of complex actions but also transfer of segmented primitives to robotic platforms for skill synthesis.

5. Implications for Autonomous Planning and Control

The pose taxonomy and its formal representation pave the way for autonomous decision making in complex environments:

• Pose Selection: Robots can dynamically select support configurations optimized for stability, reachability, or transition cost, as demanded by the task (e.g., using resting poses for fall recovery or multi-contact for manipulation in clutter).
• Motion Primitive Libraries: By associating libraries of primitives to taxonomy classes, robots can rapidly assemble novel action sequences, facilitating learning and on-the-fly adaptation.
• Structured Control Synthesis: The formal description of poses and neighbors ($\mathcal{N}$) enables the design of controllers that explicitly manage center-of-mass and contact forces for robust equilibrium maintenance throughout motion.
• Transition Generation: The taxonomy offers a formal grammar for safe transitions; planners can guarantee that no forbidden or dynamically unsafe transition is attempted.

Such frameworks are being adapted to real humanoid platforms (e.g., TORO, ARMAR-4) to enhance performance beyond prior single-contact or foot-centric approaches.

6. Future Research Trajectories

The integration of pose taxonomies with real-time perception and control is anticipated to enable:

• Autonomous Whole-Body Skill Synthesis: Automatic motion generation, based on environmental constraints and object affordances, that seamlessly blends locomotion, manipulation, and balance.
• Data-Driven Refinement: Refining taxonomy structure and transition rules through analysis of ever larger and more diverse human whole-body motion datasets may reveal new, highly functional pose classes or optimize transitions.
• Generalization to Multi-Contact Non-Humanoid Morphologies: The pose-centric perspective is extendable to quadrupeds, multi-limbed robots, or soft robots, provided contact models can be codified similarly.
• Benchmarking and Comparative Evaluation: The taxonomy provides a standardized space for comparison across robots, controllers, and task domains, supporting replicable benchmarking in WBM research.

A plausible implication is that structured pose taxonomies and motion primitive grammars may become foundational components in future autonomous, learning-enabled whole-body manipulation systems. This approach is expected to enhance robustness and task generality by systematic exploitation of the environment as a stabilizing resource.

---

The work described establishes whole-body pose taxonomies as a fundamental tool for structuring, segmenting, and synthesizing complex multi-contact behaviors in humanoid WBM, providing both the abstraction for high-level planning and the concrete representation needed for low-level control and data-driven learning (Borràs et al., 2015).