ForceMimic: Force-Centric Imitation Learning

• ForceMimic is a force-centric imitation learning framework that integrates force, torque, and motion data to enable precise, contact-sensitive robotic manipulation.
• It employs advanced hardware setups (e.g., bilateral control, force-capture devices) and learning architectures (RNNs, CNN-LSTM, diffusion models) to capture and predict time-varying wrench and pose data.
• Empirical results show that ForceMimic improves manipulation success by 30–45% over motion-only methods, demonstrating enhanced efficiency in tasks like drawing, peeling, and assembly.

ForceMimic is a class of imitation learning systems that explicitly incorporate force and torque information—alongside kinematic data—during both robotic demonstration capture and policy learning, with the goal of enabling robust, contact-sensitive manipulation. In contrast to traditional vision- or trajectory-based learning, ForceMimic approaches acquire, model, and execute time-varying wrench and pose data, leveraging innovations in hardware (force/motion capture, bilateral control) and algorithmic machinery (hybrid controllers, diffusion models, simulator-supervised training, adversarial learning) across a range of contact-rich tasks, from line-drawing and peeling to construction assembly (Adachi et al., 2018, Ehsani et al., 2020, Liu et al., 10 Oct 2024, You et al., 24 Jan 2025).

1. Force-Centric Demonstration Capture

ForceMimic systems integrate direct measurement of human-applied interaction forces (wrenches) with motion data at the tool or robot end-effector. Major paradigms include:

• Bilateral Control Separation: As in the original ForceMimic approach to object manipulation (Adachi et al., 2018), two 3-DoF haptic robots (master and slave) are coupled under four-channel bilateral control. The master’s measured reaction torques (τₘ res, acting force) and the slave’s (τₛ res, reaction force) are separated via synchronization and torque-symmetry constraints:

$$\theta_m - \theta_s = 0 \quad \text{and} \quad \tau_{m\,\text{res}} + \tau_{s\,\text{res}} = 0$$

This protocol yields time-series of fully separated position and force measurements.

• Handheld Force-Capture Devices: In contact-rich tasks such as vegetable peeling (Liu et al., 10 Oct 2024), ForceCapture is a handheld system integrating a six-axis force/torque sensor with a pose-tracking camera and optional gripper encoder. True environment wrenches are isolated via quasi-static gravity calibration across diverse poses using

$\min_{m,\,r_c} \;\sum_i \|\;w_i - m\,R_i\,g\,\|^2$

followed by subtraction of the estimated mass/gravity effect from each force signal.

• Teleoperation with Haptic Feedback: For assembly scenarios (You et al., 24 Jan 2025), human demonstrators manipulate a 6-DoF robot (e.g., Franka Emika Panda) remapped to a digital twin in Unity. Real-time F/T sensor readings and virtual contact forces are synchronously streamed for precise demonstration capture.

Synchronization and multi-modal data aggregation (force, pose, gripper width, point clouds, RGB-D) are universally performed at high temporal rates (≥30 Hz, up to 1 kHz) with calibration, downsampling, and normalization steps tailored to the task and hardware modalities.

2. Learning Architectures and Policy Representations

ForceMimic instantiates policies that jointly model force and motion, with architectures adapted to the richness of the input-output mapping in each domain:

• Sequential RNN Models with Force Inputs: For line drawing, two recurrent policy architectures were compared (Adachi et al., 2018):
  • Torque-Reference Learner: $x_t \in \mathbb{R}^{18}$ (master/slave positions, velocities, torques as inputs), outputting $\tau_{s}^{\text{ref}}$ for next-step torque control.
  • Command-Learner: $x_t \in \mathbb{R}^9$ (slave-side only), outputting position, velocity, and torque commands for a built-in controller.

Both utilize stacked LSTM layers (100 units), with a 20 ms prediction horizon.

• CNN-LSTM with Simulator Supervision: Prediction from video is realized via a ResNet-18 encoder feeding a 3-layer LSTM; outputs are 3D contact points and forces per finger. Policy optimization incorporates feedback from a differentiable physics simulator (PyBullet) (Ehsani et al., 2020).
• Diffusion Policy Networks for Trajectory+Wrench Output: In the hybrid force-motion architecture (Liu et al., 10 Oct 2024), the model ingests a concatenated observation vector (point cloud encoding, pose, gripper width), and outputs both $K$-step SE(3) pose deltas and force trajectories. It is trained to denoise randomly perturbed (via a diffusion kernel) action sequences over a short horizon, implemented as a U-Net over the denoising steps.
• Generative Adversarial Imitation and RL Fine-Tuning: Assembly manipulation leverages a GAIL framework with state inputs that blend real-time force observation and geometric progress measures, using MLP policies and discriminators. Final policies are PPO-refined with force-centric reward shaping (You et al., 24 Jan 2025).

A summary of representative architectures is provided in the following table:

Approach / Task	Input Modalities	Output
Bilateral/RNN (Line)	Force, joint states	Torque ref. or trajectory cmd
Sim-supervised (Video)	Video, pose, mesh	Contact pts, force vectors
Diffusion HybridIL (Peel)	Point cloud, wrench, pose	ΔPose seq, force seq
GAIL+PPO (Assembly)	Force, depth, alignment	Cartesian force, impedance

3. Learning Objectives, Losses, and Regularization

ForceMimic learning pipelines directly optimize outputs to match observed force and motion, incorporating both direct and indirect loss terms:

• MSE with Weight Decay: Recurrent models for line-drawing are trained by mean squared error between predicted and ground-truth torque/command outputs, with regularization ($\lambda \approx 10^{-4}$).

$$\mathcal{L}(\Theta) = \sum_{t=1}^T \| \hat{y}_t - y_t \|_2^2 + \lambda \| \Theta \|_2^2$$

A jerk penalty may be added to encourage smooth command trajectories.

• Diffusion Losses: HybridIL uses a diffusion-style denoising loss on stacked wrench and pose deltas, augmented by direct MSE on predicted endpoint pose and force:

$$\mathcal{L}_{\text{diff}}(\theta) = \mathbb{E}_{(s_t,z),\tau,\epsilon} \left[ \| \epsilon - \epsilon_{\theta}(s_t, z^{\tau}, \tau) \|^2 \right]$$

plus $L_2$ terms on predicted action sequences.

• Simulator-Driven Physics Losses: For video-to-force prediction, forward passes through a physics simulator tie predicted contact forces, points, and subsequent object effects to the observed video via keypoint reprojection losses and contact point regression (Ehsani et al., 2020).
• Adversarial and RL Objectives: In construction assembly, policies are first trained adversarially to match the expert joint distribution over force states and then refined with PPO on a reward blending insertion success and reduction of contact forces:

$$r_t = \begin{cases} +1, & \text{if insertion depth} \geq \text{target} \ +1/T_{\max}, & |F_{\text{normal}}(t)|<|F_{\text{normal}}(t\!-\!1)| \wedge |F_{\text{friction}}(t)|<|F_{\text{friction}}(t\!-\!1)| \ -1/T_{\max}, & \text{otherwise} \end{cases}$$

4. Control Paradigms for Force-Position Execution

ForceMimic controllers are uniformly hybrid, integrating both position and force control to ensure stable contact and compliance:

• Orthogonal Hybrid Control: During deployment, the learned policy predicts both motion direction ($\hat{d}_t$) and target force vector ($\hat{F}_t$). The control law splits:
  • Along $\hat{d}_t$: velocity or displacement is tracked with PD gains.
  • In the orthogonal plane: the measured force is servoed towards $\hat{F}_t^{\perp}$ using high-gain force control; this is realized by

  $$\tau = J^T \left[ K_f\,(\hat{F}_t^{\perp} - F_{\text{meas}}) + K_p\,((\hat{P}_{t+1} - P_t)\cdot\hat{d}_t)\,\hat{d}_t \right]$$

• Impedance-Based Inner Loops: For assembly, the desired Cartesian force $\mathbf{F}_{\text{cmd}}$ is mapped to joint torques via the Jacobian, with onboard gravity and Coriolis compensation.
• Primitive Switching: If predicted force magnitude drops below a threshold, the controller reverts to position-based tracking; an initial "press-in" force is applied to establish contact when required.

All layers operate at high servo rates (typically ≥500 Hz), maintaining feedback alignment with real robot dynamics.

5. Benchmark Tasks, Datasets, and Empirical Findings

ForceMimic approaches have been validated across diverse contact-centric benchmarks:

• Precision Drawing (Bilateral Control): On line drawing at novel ruler inclinations, force-augmented policies (Model 1/2) achieved 65–90% trial success, outperforming position-only baselines by 30–45%, and obtained torque errors as low as ~12 mNm (Adachi et al., 2018).
• Tool-Based Vegetable Peeling: The HybridIL architecture with force/point cloud inputs obtained 100% motion success and 85% continuous peel continuity, versus 80%/55% for pure vision-based diffusion policy. Pure vision led to excessive contact force, damaging the vegetable. Only the hybrid pipeline maintained both accuracy and force-matching (7.5 N vs. 6 N human) (Liu et al., 10 Oct 2024).
• Video-to-Force Prediction: Joint training of contact and force predictors in the simulator-supervised setting reduced keypoint errors by 5–10% and aligned predicted force arrows with human intent (e.g., twisting, lifting) (Ehsani et al., 2020).
• Assembly with Haptic Feedback: Two-phase approach achieved ≈95% success in randomized pipe-insertion, compared to ≈79% (visual imitation) and ≈55% (vanilla RL). Sample efficiency and generalization over variable geometries were significantly improved when force-based demonstrations and reward shaping were used (You et al., 24 Jan 2025).

6. Limitations, Practical Insights, and Future Directions

Several consistent findings emerge from the literature:

• Necessity of Force as Output: Providing force as merely an input (e.g., to vision-based policies) is generally insufficient; explicit prediction and closed-loop control of wrench parameters are critical for robust, compliant behavior, especially in non-static or highly variable contact dynamics (Liu et al., 10 Oct 2024).
• Demonstration Quality Sensitivity: Learning performance and convergence are highly sensitive to the fidelity and variability of force demonstrations—outliers or poorly calibrated samples can degrade both sample efficiency and final task success (You et al., 24 Jan 2025).
• Scalability and Generalization: Current systems are limited by the assumptions of rigid-body contact, relatively simple task structure (single object, single hand), and potential simulation bottlenecks (expensive finite-difference gradients). Extension to multi-object, deformable, or multi-contact settings will require additional innovations in sensing, network architectures (e.g., graph or multi-modal encoders), and efficient simulation-backends (Ehsani et al., 2020, Liu et al., 10 Oct 2024).
• Primitive Selection and Policy Modularity: Hybrid controllers currently rely on hand-crafted thresholds (e.g., force-magnitude) to switch control primitives; automated or learned primitive selection remains an open avenue (Liu et al., 10 Oct 2024).

Potential extensions include FEM-based simulation for high-fidelity contact, domain randomization for sim-to-real transfer, richer state observation (e.g., vision + haptics), and meta-learning for cross-tool/task generalization (You et al., 24 Jan 2025, Liu et al., 10 Oct 2024).

7. Summary of Contributions Across Research Domains

ForceMimic has established a unified methodological framework for force-centric imitation learning, with demonstrable impact across robotic manipulation subfields:

• Actionable, reproducible control strategies applicable to diverse tasks (drawing, peeling, assembly).
• Integration of hardware and learning advancements (bilateral separation, pseudo-state vector design, precise gravity compensation, multi-modal fusion).
• Empirically validated policies that robustly track human-like contact dynamics and outperform purely motion-based or indirect force-matching methods.
• Open-source implementations and datasets that enable broad adoption and further research (Liu et al., 10 Oct 2024).

By embedding physical interaction—via both data and control—at the core of imitation learning, ForceMimic represents a critical advance for dexterous, contact-rich robotic autonomy (Adachi et al., 2018, Ehsani et al., 2020, Liu et al., 10 Oct 2024, You et al., 24 Jan 2025).