External Force Guided Curriculum Learning

• EFGCL is a curriculum learning approach that uses staged external forces to guide policy training in dynamic, contact-rich robotic tasks.
• Force signals are scheduled via linear, multiplicative, or performance-adaptive decay, mimicking human coaching to transition from assistance to autonomy.
• Empirical results show EFGCL improves sample efficiency and robustness in domains like legged locomotion, dexterous manipulation, and whole-body control.

External Force Guided Curriculum Learning (EFGCL) refers to a family of methods in robotics and reinforcement learning that leverage staged application of physically meaningful external forces or force-related constraints to shape policy learning for dynamic and contact-rich motor skills. These approaches are motivated by the observation that external guidance—in the form of force assistance or force-salient information—can both accelerate skill acquisition and promote robustness when gradually faded, mimicking principles found in human coaching and motor development. EFGCL has emerged as a central paradigm in recent work spanning dynamic legged locomotion, dexterous manipulation, and whole-body humanoid control, unites both policy-gradient and imitation-learning frameworks, and encompasses a diverse set of curriculum designs, from physically applied forces to curriculum-structured multimodal representation learning.

1. General Principles and Motivation

EFGCL is grounded in the hypothesis that dynamic or contact-rich robotic tasks pose a highly challenging exploration problem, particularly under sparse or failure-heavy reward regimes. A fundamental insight is that force cues—applied externally (physical assistance) or internally (force-salient representation)—can expose learners to successful motion trajectories otherwise inaccessible in early training stages. By progressively reducing reliance on these force cues, EFGCL aims to yield policies that are both sample efficient and robust once the guidance is withdrawn.

Key manifestations include:

• Application of physical assistive forces to the agent’s body or end effector during early training (Yoneda et al., 11 May 2026, Cao et al., 29 Jun 2025).
• Structured corruption or bottlenecking of non-force modalities to bias early learning towards underutilized or crucial force signals (Zhang et al., 13 Feb 2026, Liu et al., 24 Feb 2025).
• Scheduling of force signals or external perturbations as a curriculum variable, with linear, exponential, or performance-adaptive decay schemes (Tidd et al., 2020, Zhang et al., 10 May 2025).

This paradigm contrasts with reward-shaping or reference tracking, offering a direct mechanism for stabilizing exploration and representation in difficult environments.

2. Mathematical Formalism and Curricular Mechanisms

EFGCL methods formalize curriculum learning over force assistance or signal importance in both RL and supervised settings:

• External Force Augmentation in RL: Environment transitions are augmented with an external force term $F_\text{ext}(t)$, such that $$s_{t+1} = f(s_t, a_t) + \Delta t \cdot F_\text{ext}(t)$$. $F_\text{ext}(t)$ is composed as $F_\text{ext}(t) = \alpha_i \cdot F_\text{assist}$, where $\alpha_i$ is a curriculum parameter decayed across stages (Yoneda et al., 11 May 2026).
• Curriculum Schedules: Assistance magnitude $\alpha_i$ can decay linearly ($\alpha_i = 1-\varepsilon i$), multiplicatively ($\alpha_{k+1} = \alpha \cdot \alpha_k$), or be adaptively tied to proficiency (e.g., decay only after surpassing a minimum success rate) (Yoneda et al., 11 May 2026, Tidd et al., 2020).
• Force-Information Bottleneck and Corruption: In imitation settings, EFGCL may enforce prioritization of force cues by bottlenecking vision/language embeddings via a variational information bottleneck (VIB) loss term with a decaying multiplier (Zhang et al., 13 Feb 2026), or by systematically corrupting the visual stream (e.g., Gaussian blur) so that gradients concentrate on the force encoder (Liu et al., 24 Feb 2025).

The table provides salient curriculum scheduling mechanisms from representative works:

Paper	Curriculum Variable	Schedule/Update Rule
(Yoneda et al., 11 May 2026)	$\alpha_i$ force scaling	$\alpha_{i+1} = \max(0, 1-\varepsilon i)$, decay on success
(Tidd et al., 2020)	Guide force magnitude	$$s_{t+1} = f(s_t, a_t) + \Delta t \cdot F_\text{ext}(t)$$0, success-based multiplicative
(Zhang et al., 13 Feb 2026)	$$s_{t+1} = f(s_t, a_t) + \Delta t \cdot F_\text{ext}(t)$$1 (KL weight)	Exponential $$s_{t+1} = f(s_t, a_t) + \Delta t \cdot F_\text{ext}(t)$$2
(Liu et al., 24 Feb 2025)	Vision corruption $$s_{t+1} = f(s_t, a_t) + \Delta t \cdot F_\text{ext}(t)$$3	Linear/cosine/exponential decay to 0

3. Policy Architectures and Multimodal Fusion under Force Curriculum

EFGCL frameworks employ diverse policy representations tailored to their application domains:

• Vision–Language–Action Models: Curriculum integration of a Variational Information Bottleneck (VIB) on fused vision/language embeddings, coupled with direct concatenation of proprioceptive force data (joint torques) prior to the policy head, ensures early dependency on force signals (Zhang et al., 13 Feb 2026). The total loss is structured as $$s_{t+1} = f(s_t, a_t) + \Delta t \cdot F_\text{ext}(t)$$4, with $$s_{t+1} = f(s_t, a_t) + \Delta t \cdot F_\text{ext}(t)$$5 annealed to reintegrate perceptual cues.
• Force–Vision Transformers: FACTR instantiates EFGCL by progressive visual corruption (image blur or downsampling), maintaining a parallel uncorrupted force stream, and promoting force-attending in the transformer's cross-attention modules (Liu et al., 24 Feb 2025).
• Dual-Agent RL for Humanoids: Approaches such as FALCON and A2CF employ two-agent architectures in which a locomotion agent and a manipulation (or assistive force) agent operate with coupled goals. External force curricula target end-effector forces or full-body support and are implemented while respecting dynamic and actuation constraints (Zhang et al., 10 May 2025, Cao et al., 29 Jun 2025).

Policy architectures are typically built on PPO or transformer-based BC, with explicit monitoring of force-attention, cross-modality information flow, and adaptation to curriculum progression.

4. Experimental Protocols, Benchmarks, and Results

EFGCL demonstrates improved stability, robustness, and generalization across challenging robotic domains:

• Dynamic Motions (Quadrupeds & Humanoids): On tasks like jumping, backflips, and flips, applying EFGCL results in successful policy acquisition where conventional RL fails; EFGCL-enabled jump learning achieves target return in $$s_{t+1} = f(s_t, a_t) + \Delta t \cdot F_\text{ext}(t)$$6 samples vs. $$s_{t+1} = f(s_t, a_t) + \Delta t \cdot F_\text{ext}(t)$$7 greater for baseline PPO (Yoneda et al., 11 May 2026). In complex humanoid benchmarks, curriculum-guided assistive forces cut time-to-convergence by 30–50% and halve failure rates (Cao et al., 29 Jun 2025).
• Contact-rich Manipulation and Loco-Manipulation: Concretely, CRAFT augments VLA models to increase average task success (USB insertion, carton flipping, wiping, plasticine rolling) by $$s_{t+1} = f(s_t, a_t) + \Delta t \cdot F_\text{ext}(t)$$8–$$s_{t+1} = f(s_t, a_t) + \Delta t \cdot F_\text{ext}(t)$$9 percentage points, and improves out-of-distribution generalization to unseen objects and new variants from $F_\text{ext}(t)$0 to $F_\text{ext}(t)$1 (Zhang et al., 13 Feb 2026). FALCON demonstrates $F_\text{ext}(t)$2 improved upper-body joint tracking error and sustained gait stability under force disturbances, without robot-specific curriculum tuning (Zhang et al., 10 May 2025).
• Ablation Studies: Across works, ablations consistently show that omitting either the force curriculum or its structured decay eliminates the observed gains in both sample efficiency and robustness (Zhang et al., 13 Feb 2026, Liu et al., 24 Feb 2025, Yoneda et al., 11 May 2026, Tidd et al., 2020, Cao et al., 29 Jun 2025).

5. Variants and Extensions: Design Patterns

Several variants and extensions of EFGCL have been established:

• Physical Guidance vs. Curriculum over Modality: Physical guidance approaches inject real or simulated forces (spotting, PD guides) during early training and decay them, motivating the learning of endogenous balancing and control (Yoneda et al., 11 May 2026, Tidd et al., 2020, Cao et al., 29 Jun 2025). Curriculum over sensory modalities instead manipulates the information content of perception to force attention to critical signals, especially in high-dimensional multi-modal settings (Zhang et al., 13 Feb 2026, Liu et al., 24 Feb 2025).
• Assistive-force Policy Agents: Dual-agent systems, such as A2CF, learn an explicit assistive force policy that is jointly optimized with the skill policy, with the force application range decayed via performance thresholds and random masking to prevent overreliance (Cao et al., 29 Jun 2025).
• Jointly Torque-Feasible Curriculum: FALCON's force curriculum increases the end-effector force up to the joint torque limits, guaranteeing feasibility and promoting direct transfer across hardware platforms without the need for specific tuning (Zhang et al., 10 May 2025).

6. Empirical Impact and Domain Transfer

The empirical impact of EFGCL is substantiated by rigorous cross-task, cross-platform, and sim-to-real evaluations:

• Zero-shot Transfer: Policies trained with torque-aware or force-salient curricula in simulation robustly transfer to real robots. FALCON-trained policies are deployed on both Unitree G1 and Booster T1 hardware without per-robot curriculum adjustments, performing cart-pulling, payload carry, and door-opening under realistic force magnitudes (Zhang et al., 10 May 2025).
• Generalization: Methods such as CRAFT and FACTR achieve large increases in both generalization to unseen objects (≈43% improvement (Liu et al., 24 Feb 2025)) and adaptation to novel task arrangements (Zhang et al., 13 Feb 2026).
• Robustness and Recovery: Addition of force-aware curriculum controls promotes better "recovery" behavior (e.g., recovering from object drops, maintaining stability under perturbations), which standard vision-dominated or non-curricular methods do not achieve (Liu et al., 24 Feb 2025, Tidd et al., 2020).

7. Challenges, Limitations, and Future Directions

While EFGCL has demonstrated robust performance improvements, certain limitations and open areas remain:

• Force Curriculum Design: Most current methods rely on heuristic specification of force magnitude, application location, and decay schedule. Fully automatic, learned force-schedule design remains an open challenge (Yoneda et al., 11 May 2026).
• Overreliance on External Support: Improperly scheduled or excessive assistance can hamper autonomy; random masking and state-dependent decays are devised to counteract overreliance (Cao et al., 29 Jun 2025).
• Extension to Cyclical, Non-episodic Skills: While demonstrated on tasks with crisp success/failure structure (e.g., flips, insertions), expanding force-guided curricula to continuous, cyclic behaviors (e.g., dance, repetitive manipulation) is an area of ongoing research (Yoneda et al., 11 May 2026, Cao et al., 29 Jun 2025).
• Integration with Multi-agent and Multimodal RL: Extensions to settings with multiple interacting learners, and fusion with self-supervised or model-based exploration under force-aware structures, are plausible and under active study.

In sum, External Force Guided Curriculum Learning unifies a spectrum of methods for leveraging force information in staged, curriculum-based robotic skill acquisition. EFGCL methods enable efficient policy training in domains previously inaccessible due to exploration or representation bottlenecks, promote generalization and robustness, and demonstrate successful sim-to-real transfer across robot morphologies (Zhang et al., 13 Feb 2026, Yoneda et al., 11 May 2026, Zhang et al., 10 May 2025, Cao et al., 29 Jun 2025, Tidd et al., 2020, Liu et al., 24 Feb 2025).