Teacher–Student Network in Robotics

• Teacher–student networks are hierarchical models where a teacher provides privileged guidance to a student for efficient task learning in robotics and reinforcement learning.
• They utilize methods like force-guided curriculum learning, policy distillation, and adaptive assistance to overcome sparse rewards and challenging dynamics.
• Empirical evidence shows these architectures enhance sample efficiency, convergence, and robustness, with careful curriculum scheduling being key to successful skill transfer.

A teacher–student network structure in robotics and reinforcement learning refers to a hierarchical or dual-agent architecture where one agent (the teacher) provides privileged information, explicit guidance, or external signals (often via forces or demonstration) to facilitate the training of a student agent that must ultimately perform the task independently. Such structures are inherent in a broad spectrum of External Force Guided Curriculum Learning (EFGCL) methods and curriculum-based robotic control approaches. These systems encode physical and informational hierarchies, supervise policy learning with privileged modalities, or scaffold skill acquisition via gradually withdrawn guidance.

1. Formal Definition and Purpose

Teacher–student architectures in the EFGCL paradigm involve at least two interacting agents:

• Teacher (or assistive agent): Has access to privileged information, direct state knowledge, or task-specific heuristics. It may interact physically (applying explicit forces) or provide action/trajectory-level feedback.
• Student (or motion policy): Has limited sensing (e.g., proprioception, onboard sensors), must achieve proficiency in the designated task under increasingly autonomous conditions.

The overarching objective is to enable the student to succeed on challenging motor or manipulation tasks by leveraging the teacher’s guidance early in training, then autonomizing behavior as guidance is removed or reduced. Teacher–student frameworks facilitate learning under sparse rewards, difficult initializations, or unsafe exploration, by modifying the state visitation distribution to include more successful or informative trajectories (Yoneda et al., 11 May 2026, Zhang et al., 10 May 2025, Cao et al., 29 Jun 2025).

2. Instantiations in EFGCL and Related Methods

Teacher–student structures are instantiated in several prominent EFGCL frameworks:

System	Teacher Agent Role	Student Agent Role
CRAFT	Implicit: via external force and VIB bottleneck	Vision-language-action policy
EFGCL (quadruped)	Policy with privileged state + assistive forces	Policy with proprioception only
FALCON	Upper/Lower body reward shaping with force agent	Motion policy under external force
A2CF	Explicit RL-trained assistive force agent	Standard RL motion policy

• CRAFT: The teleoperation system (homologous leader–follower) serves as a data generation teacher, providing synchronized force and multimodal data; during fine-tuning, a variational information bottleneck forces the policy (student) to rely on force input first, then unmasks full input (Zhang et al., 13 Feb 2026).
• Explicit Teacher (EFGCL RL): In legged locomotion and humanoid control, the teacher is a policy operating with privileged (full-state, full-force) access and strong assistive forces. The student imitates, but is restricted to onboard sensors and receives progressively reduced assistance (Yoneda et al., 11 May 2026, Zhang et al., 10 May 2025, Cao et al., 29 Jun 2025).
• Dual-Agent RL (A2CF, FALCON): The system jointly trains two neural agents—an assistive force generator (teacher) and a motion controller (student)—under a shared reward but asymmetric information. The teacher agent deploys state-dependent assistance, withdrawn as skill indicators reach thresholds (Cao et al., 29 Jun 2025, Zhang et al., 10 May 2025).

3. Architectural Patterns and Data Flow

The internal workflow in teacher–student EFGCL systems often follows one of these patterns:

• Asymmetric Actor–Critic (AAC): Both agents act in parallel; the teacher receives privileged state and emits assistive actions/forces, the student receives standard observations. A global critic with full state evaluates both, with gradients updating both policies (Cao et al., 29 Jun 2025).
• Offline Demonstration + Bottleneck: Teleoperated or reference data (provided by a teacher) are filtered through an input bottleneck to ensure initial reliance on reliable modalities; as the bottleneck is relaxed, the student learns to integrate multimodal cues (Zhang et al., 13 Feb 2026, Liu et al., 24 Feb 2025).
• Policy Distillation: The teacher policy, acting in the modified environment with assistance, is used to supervise the student via action-matching objectives, while the student is constrained to test-time observations (Yoneda et al., 11 May 2026).
• Interactive Curriculum: The teacher maintains control over a curriculum parameter (e.g., magnitude and direction of assistive force), updating its schedule according to a success-based or adaptive rule, ultimately steering the student to unaided task performance (Tidd et al., 2020, Zhang et al., 10 May 2025, Cao et al., 29 Jun 2025).

4. Information, Modality, and Privileged Guidance

Teacher–student structures systematically exploit information asymmetry:

• Privileged Inputs: The teacher accesses full simulator state, future trajectory, or ideal force profiles. The student is limited to onboard sensoria, e.g., proprioception, limited vision, or real-time force sensing.
• Force and Action Guidance: The teacher, whether a human teleoperator or a policy with privileged access, applies explicit forces (external assistive, guiding joint torques) or provides ideal actions. Guidance may be physical (as in quadrupedal/humanoid curriculum learning (Yoneda et al., 11 May 2026, Zhang et al., 10 May 2025, Cao et al., 29 Jun 2025)) or informational (as in demonstration-imitation frameworks (Zhang et al., 13 Feb 2026)).
• Adaptive Assistance Scheduling: The teacher schedules the withdrawal of privileged information or physical assistance via discrete, continuous, or success-based curricula—e.g., force magnitude scaled by curriculum factor $\alpha$, information bottleneck weight $\lambda_{\mathrm{VIB}}(t)$, or external force bounds $\mathcal{B}_k$ (Zhang et al., 13 Feb 2026, Cao et al., 29 Jun 2025).

5. Curriculum Scheduling and Decoupling Process

The curriculum transition from teacher-dominated to student-driven operation is central:

• Force Annealing Schedules: Assistance (physical or informational) is reduced either by exponential decay, stage-wise thresholds, or success-based increments. For example, CRAFT decays $$\lambda_{\mathrm{VIB}}(t) = \lambda_{\mathrm{init}} \exp(-t/T_{\mathrm{decay}})$$; FALCON ramps $$\alpha(n_{\mathrm{epi}}) = \min\{1, n_{\mathrm{epi}}/N_{\mathrm{ramp}}\}$$ (Zhang et al., 13 Feb 2026, Zhang et al., 10 May 2025).
• Stage-based Training: Methods such as (Tidd et al., 2020) use multi-stage curricula: (1) teacher-provided guidance at full strength, (2) annealing of guiding signals, and (3) introduction of destabilizing perturbations for robustness training.
• Adaptive and Success-driven Decay: Several frameworks (e.g., A2CF (Cao et al., 29 Jun 2025)) adapt assistive force bounds based on student proficiency, accelerating independence when policy performance crosses thresholds.

6. Empirical Impacts and Best Practices

Empirical studies report substantial improvements in sample efficiency, convergence speed, robustness, and generalization:

• Policies trained via teacher–student curricula outperform baselines lacking such structure, especially on manipulation tasks requiring force sensitivity or legged/humanoid control involving dynamic contacts (Zhang et al., 13 Feb 2026, Liu et al., 24 Feb 2025, Yoneda et al., 11 May 2026, Zhang et al., 10 May 2025, Cao et al., 29 Jun 2025, Tidd et al., 2020).
• Ablation studies consistently show that removing either the teacher’s guidance or the curriculum schedule substantially degrades performance: for example, removing force-guided curricula in bipedal walking reduces success rates from above 69% to below 15% on gap terrains (Tidd et al., 2020).
• Properly chosen curriculum schedules (success-based, phase-aware, or adaptive) prevent premature over-dependence, destabilization upon withdrawal of guidance, or under-utilization of critical force features.

Best practices include the use of privileged information only in training (never at deployment), careful phase-wise curriculum initialization (especially in tasks with natural subphases, e.g., walking/jumping/landing), and the use of randomness and domain perturbation to ensure generalization across environments (Cao et al., 29 Jun 2025).

7. Limitations, Variants, and Extensions

Known limitations and active research areas include:

• Assistance Scope: Most current instantiations apply assistive forces only on global robot structures (e.g., pelvis) or at end effectors, rarely on limbs or via environment-robot interaction contacts. Extension to more granular or task-specific guidance remains open (Cao et al., 29 Jun 2025).
• Teacher Policy Quality: Teacher–student effectiveness depends on the quality and stability of the teacher (teleoperator or privileged policy); suboptimal guidance may produce brittle or suboptimal students (Zhang et al., 13 Feb 2026).
• Adaptive vs. Fixed Schedules: Excessively aggressive curriculum decay can result in “policy starvation,” while overly cautious withdrawal slows skill acquisition.
• Sim-to-Real Transfer: As privileged information is absent at deployment, robustness and domain randomization during training are critical for successful real-world transfer (Zhang et al., 10 May 2025).

A plausible implication is that the continued integration of teacher–student network structures, with increasingly sophisticated assistive agents and adaptive curricula, will further expand the range and reliability of skill acquisition in challenging robotic and embodied AI domains.