Unified Control Module

• Unified Control Module is a comprehensive control framework that integrates policy learning for simultaneous mobility and manipulation in robotic systems.
• It employs advanced strategies like command polynomial interpolation, residual action modeling, and curriculum learning to ensure smooth and precise motion.
• Empirical results demonstrate improved workspace coverage, reduced tracking errors, and enhanced robustness compared to traditional hierarchical control methods.

A Unified Control Module (UCM) is a control architecture designed to govern the entire operational space of a complex robotic system with a single, end-to-end policy or tightly-integrated control framework. In contrast with traditional hierarchical or segregated control protocols—where separate sub-policies manage subsystems such as locomotion and manipulation—UCMs are engineered to optimize coordination, workspace, and robustness by fusing perception and command streams across all degrees of freedom. Recent unified controllers, exemplified by the Unified Loco-Manipulation Controller (ULC) for humanoid robots, demonstrate state-of-the-art performance via joint policy learning, advanced feedback integration, and robustness mechanisms, motivating a shift away from decoupled control paradigms (Sun et al., 9 Jul 2025).

1. Unified Control Problem Formulation

Unified control modules for whole-body robots are fundamentally structured around goal-conditioned Markov Decision Processes (MDPs):

• $\mathcal{M} = \{ S, A, G, P, R, \gamma \}$, where $S$ denotes the observation space (multi-step proprioceptive states), $A$ the action space (e.g., joint positions across all DOFs), $G$ the goal-command space (velocity, pose, arm targets), $P$ the dynamics, $R$ the reward, and $\gamma$ the discount.
• The policy is parameterized as a Gaussian:

$$\pi_{\theta}(a_t | s_t, g_t) = \mathcal{N}(\mu_{\theta}(s_t, g_t), \Sigma_{\theta}(s_t, g_t)),$$

where the underlying neural network $f_{NN}$ stacks $k$ timesteps of proprioceptive features, past actions, and goals.

Actions are generally interpreted as processed joint targets via learned means and residual corrections: $$q_{\text{processed}} = \alpha_{\text{scale}} \cdot \mu_{\theta} + q_{\text{default}}, \quad q_{\text{final}}[\text{arms}] = q_{\text{processed}}[\text{arms}] + q_{\text{desired}}[\text{arms}],$$ with $\alpha_{\text{scale}}$ a normalization scalar (Sun et al., 9 Jul 2025).

2. Smooth Motion Generation and Fine-Grained Coordination

UCMs enforce trajectory smoothness and precise coordination by integrating advanced command-processing steps:

• Command Polynomial Interpolation: Target trajectories for arm and torso joints are interpolated over fixed horizons (e.g., $T=1$ s) using quintic polynomials:

$$s(\tau) = 10\tau^3 - 15\tau^4 + 6\tau^5,\quad q_\text{target}(\tau) = q_\text{start} + (q_\text{goal} - q_\text{start})\, s(\tau), \quad \tau \in [0,1],$$

ensuring $C^2$ continuity (zero velocity and acceleration at endpoints).

• Residual Action Modeling: Rather than learning full joint commands, the policy outputs residual corrections to finely tune base interpolated commands:

$$q_\text{final} = q_\text{desired} + (q_\text{processed} - q_\text{default}),$$

letting the network focus on environment-induced fine adjustments.

3. Progressive Skill-Acquisition via Curriculum Learning

Training involves a sequential skill-acquisition curriculum in which increasingly complex capabilities are gated by explicit thresholds:

• Stage T₁: Base velocity tracking.
• Stage T₂: Base height tracking, gated by strict velocity/height/hip tracking rewards:

$$R_\text{height}^{\text{avg}} \geq 0.85,\quad R_\text{vel}^{\text{avg}} \geq 0.8,\quad R_\text{hip}^{\text{avg}} \geq 0.2.$$

• Stage T₃: Torso and arm tracking, requiring upper-body mastery ($R_\text{upper}^{\text{avg}}\geq0.7$, $R_\text{torso}^{\text{avg}}\geq0.8$).

Command distributions vary by curriculum stage, transitioning from small, exponential-sampled motions to full workspace as proficiency grows.

4. Stability, Robustness, and Generalization Mechanisms

Central to unified policy feasibility is the explicit incorporation of real-time stability and robustness signals:

• Center-of-Gravity (CoG) Tracking: An explicit stability reward is imposed:

$$r_\text{CoG} = \exp\left(-\frac{\|p_\text{CoG}^{xy} - p_\text{feet}^{xy}\|^2}{\sigma_\text{CoG}^2}\right),\quad \sigma_\text{CoG}=0.2,$$

where $p_\text{CoG}$ is the mass-weighted sum of all CoMs, and $p_\text{feet}^{xy}$ the midpoint of ankle positions.

• Randomness Injection: Deployment and training robustness are assured through stochastic delay release (joint-level Bernoulli gating of command increments), load randomization (wrist masses, joint/actuator domain parameters), and other domain randomization strategies.

5. Policy Optimization and Reward Engineering

The learning objective is a highly structured, weighted reward sum balancing tracking accuracy, coordination, and regularization:

$$r_t = w_\text{vel} r_\text{vel} + w_\text{ang} r_\text{ang} + w_\text{height} r_\text{height} + w_\text{upper} r_\text{upper} + w_\text{yaw} r_\text{yaw} + w_\text{roll} r_\text{roll} + w_\text{pitch} r_\text{pitch} + w_\text{CoG} r_\text{CoG} + \dots$$

with additional penalties for foot slip, energy, joint limits, and motion regularity.

Policy optimization proceeds via PPO with a clipped surrogate objective: $$L^{\text{CLIP}}(\theta) = \mathbb{E}\left[\min(r_t(\theta)\hat{A}_t, \text{clip}(r_t(\theta),1-\epsilon,1+\epsilon)\hat{A}_t)\right],$$ employing Generalized Advantage Estimation for efficient gradient propagation.

6. Empirical Performance and Comparative Benchmarks

Unified controllers demonstrate marked improvements in both workspace coverage and whole-body tracking accuracy compared with state-of-the-art disentangled systems:

• Workspace coverage:
  • Root height: $[0.30,\,0.75]$ m.
  • Yaw: $\pm2.62$ rad, Roll: $\pm0.52$ rad, Pitch: $[-0.52,1.57]$ rad.
  • Full arm-joint range.
• Tracking errors (mean absolute error; ULC vs. baselines):
  • Velocity: $E_v=0.069 \pm 0.010$ m/s
  • Dual-arm: $E_a=0.083 \pm 0.012$ rad (FALCON: $0.096 \pm 0.014$ rad)
• Robustness under load/disturbance: Under external $2$ kg payload and command mutation, unified controllers degrade by $<15\%$, while modular baselines degrade by $>50\%$ in some degrees (Sun et al., 9 Jul 2025).

7. Architectural and Theoretical Implications

The unified control approach directly addresses the limitations of hierarchical control decomposition by:

• Enabling holistic, simultaneous optimization across mobility and manipulation DOFs, more closely mimicking human motor coordination.
• Allowing policy transfer, zero-shot deployment, and robust generalization to perturbations and hardware variation, owing to domain randomization and explicit stabilization terms.
• Reducing the engineering overhead associated with fine-tuning subsystem boundaries and inter-policy communication, since all feedback and commands are processed in one integrated policy block.

This design paradigm is emblematic of a growing trend across robotics and control engineering, where unified modules increasingly replace piecemeal or hierarchical control strategies, striving for natural, adaptive whole-body control with minimal hand-crafted switching or arbitration (Sun et al., 9 Jul 2025).

---

The Unified Control Module (ULC) provides a reproducible blueprint for deploying single-policy, fine-grained controllers in next-generation humanoid loco-manipulation, validated via large-scale benchmarks on the Unitree G1 platform and outperforming established hierarchical architectures in workspace, tracking, and robustness.