FACTR 2: Learning External Force Sensing for Commodity Robot Arms Improves Policy Learning

Abstract: Contact-rich manipulation requires force sensitivity, but many robot arms lack dedicated force sensors due to their high cost. We present Neural External Torque Estimation (NEXT), a data-driven method that estimates external joint torques without needing any dedicated force sensors. NEXT trains in 1 minute from only 10 minutes of free-motion data, yet achieves estimates comparable to dedicated joint-torque sensors. NEXT enables force-feedback teleoperation on low-cost arms and improves policy learning through Force-Informed Re-Sampling Training (FIRST), which up-samples pre-contact and contact segments during behavior cloning. Across five long-horizon tasks, FIRST outperforms prior force-aware policies by over 17% in task progress. Together, NEXT and FIRST bring force-aware teleoperation and policy learning to off-the-shelf robots without additional sensing hardware. Video results and code are available at https://jasonjzliu.com/factr2

• The paper introduces NEXT, a self-supervised LSTM that accurately infers external joint torque from standard proprioceptive signals.
• The paper proposes FIRST, a re-sampling strategy that up-samples pre-contact and contact phases to enhance force-aware policy learning.
• The paper demonstrates that integrating NEXT and FIRST improves teleoperation and task performance on low-cost robot arms.

Data-Driven External Force Sensing for Commodity Robot Arms: FACTR 2

Introduction and Motivation

Contact-rich robotic manipulation tasks, such as assembly and insertion, fundamentally require precise force perception and control for success. However, most commodity robot arms lack force-torque (FT) sensors due to cost and integration challenges. Existing alternatives—such as analytical inverse dynamics, low-cost tactile sensors, or system identification—have proven insufficiently robust, especially on low-cost hardware with significant model inaccuracies and transmission nonlinearities. "FACTR 2: Learning External Force Sensing for Commodity Robot Arms Improves Policy Learning" (2606.12406) addresses these gaps through a unified paradigm: (1) Neural External Torque Estimation (NEXT), a self-supervised method to reliably infer external joint torque from standard proprioception without dedicated force sensing, and (2) Force-Informed Re-Sampling Training (FIRST), which leverages such estimated signals to improve robot policy training on contact-rich manipulation tasks.

Figure 1: (a) NEXT produces high-quality joint torque estimates from brief, contact-free data, requiring neither force sensors nor system identification. (b) FIRST segments demonstration data into free-space, pre-contact, and contact, upsampling contact-relevant phases for improved policy learning.

Neural Estimation of External Torque (NEXT)

NEXT replaces the need for dedicated FT sensors by learning a network to predict free-space torque directly from temporal sequences of proprioceptive signals, including joint positions, velocities, and tracking errors. The model is a history-based LSTM trained with less than ten minutes of free-space trajectories, requiring no contact data and no explicit parameter identification. At deployment, NEXT outputs an external torque estimate as the residual between the measured motor torque (from current sensing) and the predicted free-space torque.

This approach outperforms both analytical residual-based estimators (such as FILIC) and disturbance observers, particularly on arms with substantial unmodeled joint friction, stiction, and actuator nonlinearities. NEXT benefits from direct modeling of hardware-specific effects, yielding both denoised and accurately scaled external torque estimates.

The strong empirical validity of NEXT is demonstrated quantitatively in challenging regimes:

Figure 3: NEXT yields the lowest joint torque $L_1$ errors in both free-space and contact settings on the Franka platform; the method's estimates remain nearly noiseless in free space and faithfully track sensor readings during contact.

NEXT generalizes beyond high-end arms (Franka) to commodity systems (AgileX Piper, YAM), where joint-torque sensing is otherwise absent. On the Piper, NEXT achieves a mean joint $L_1$ error of only $0.018$ Nm, outperforming both FILIC and disturbance observers.

Figure 2: NEXT's error in free-space torque estimation on the AgileX Piper is substantially lower than all baselines. In user studies, participants rated teleoperation based on NEXT's feedback as more intuitive than with alternative methods.

Importantly, force estimation using NEXT supports robust force-feedback teleoperation on low-cost arms, an ability previously restricted to expensive platforms. User studies confirm higher usability and reduced exertion for operators when teleoperating with NEXT-based force feedback, with performance comparable to ground-truth torque-based feedback on the Franka platform.

Force-Informed Re-Sampling Training (FIRST)

Policy failure in contact-rich manipulation is disproportionately associated with precise alignment just before contact and during the contact phase itself. Standard behavioral cloning, which typically samples trajectories uniformly, fails to model these sparse, failure-prone events adequately.

FIRST introduces a data distribution shift at training time: using phase information computed via NEXT, it segments demonstration trajectories into free-space, pre-contact, and contact. Rather than uniformly sampling trajectory points, FIRST up-samples pre-contact and contact segments for policy training. This targeted sampling substantially lowers policy validation loss on critical pre-contact and contact datapoints, leading to improved generalization.

Empirically, FIRST achieves robust improvements in task completion rates and partial progress across five contact-rich, long-horizon manipulation benchmarks—including assembly, insertion, and dexterous screwing. FIRST outperforms baselines (such as FACTR, TA-VLA, and direct torque-conditioning) by at least 17% task progress across all tested tasks.

Figure 5: The five evaluation tasks are all long-horizon and contact-rich, demanding precise force sensitivity in multiple stages.

Figure 6: FIRST yields the highest task progress on all five manipulation tasks when using a flow-matching policy.

Ablations on up-sampling strategies indicate that emphasizing pre-contact segments alone provides the largest average benefit across tasks, with task-dependent gains from including both pre-contact and contact phases for specific manipulations requiring sustained interaction.

Comparison with Baselines and Policy Inputs

A critical empirical observation is that policies conditioned directly on raw joint current perform worse than those conditioned on external torques processed by NEXT. Joint current contains a superposition of actuator effort for both free-space motion and contact, which confounds policy representation learning. In contrast, the external torque signal provided by NEXT is a compact, contact-specific representation that enhances policy attention around critical interaction events.

Figure 4: Attention analysis during screw-cap manipulation reveals that policies using learned external torque focus their attention on key contact events, while those conditioned on joint effort do not.

Practical and Theoretical Implications

Practically, FACTR 2 extends the benefits of force-aware learning and closed-loop teleoperation to a wide range of affordable hardware, obviating the need for added FT hardware, expensive system identification, or complex sensor calibration. The system runs in real time, requiring minimal computational resources, and can be widely adopted with a simple ten-minute calibration procedure. The method democratizes high-quality force sensing and downstream contact-aware policy learning for platforms that have historically lacked these capabilities.

Theoretically, the data-centric, history-based approach to both sensorless force estimation and phase-sensitive imitation learning challenges the dominance of explicit dynamics modeling, value-based data selection, and raw proprioceptive augmentation in robotic learning pipelines. The empirical results suggest that phase-aware data biasing, driven by reliable event segmentation, dominates simple architectural or input-modality augmentations.

Limitations and Future Directions

NEXT's absolute force scaling depends on accurate knowledge of the actuator torque constant and may require calibration for tasks needing precise force control. Cross-platform generalization requires retraining due to hardware-specific dynamics. In future work, integrating meta-learning for cross-device generalization, combining discovery of contact-rich phases with semantic task understanding, and merging force-phase data selection with online reinforcement could further enhance contact-rich manipulation capabilities across diverse domains.

Conclusion

FACTR 2 demonstrates that with minimal sensor and compute requirements, commodity robot arms can gain robust, high-fidelity external torque estimation and leverage this for both teleoperation and advanced force-aware policy learning. The combination of NEXT and FIRST achieves policy performance rivaling sensorized systems, highlighting new opportunities for data-driven contact-rich manipulation at scale.