ManipForce: Force-Guided Contact Manipulation

• ManipForce is a framework that integrates high-frequency force-torque sensing and RGB data using a Frequency-Aware Multimodal Transformer for contact-rich tasks.
• The system combines a handheld demonstration platform with precise pose tracking and gravity compensation to accurately capture nuanced force interactions.
• Empirical results across six tasks show an average success rate of 83% with ManipForce versus 22% for RGB-only policies, highlighting its robust performance.

Searching arXiv for recent and directly relevant papers on ManipForce and force-aware manipulation. ManipForce is a force-guided policy-learning framework for contact-rich manipulation that couples a handheld demonstration system with a multimodal diffusion policy. Its stated objective is to capture high-frequency force-torque (F/T) and RGB data during natural human demonstrations for tasks in which success depends on precise control of interaction forces, including precision assembly, insertion into tight clearances, non-prehensile contact-driven manipulation, prying or disassembly, and stable contact maintenance. The associated policy model, the Frequency-Aware Multimodal Transformer (FMT), encodes asynchronous RGB and F/T streams using frequency- and modality-aware embeddings and fuses them through bi-directional cross-attention within a transformer diffusion policy. In six real-world tasks—gear assembly, LAN plug insertion, box flipping, open lid, battery disassembly, and battery insertion—FMT achieves an average success rate of 83%, while an RGB-only baseline achieves about 22% (Lee et al., 23 Sep 2025). A broader reading of adjacent force-aware manipulation research suggests that ManipForce also names a wider technical program in which force is treated as a demonstration signal, a servo objective, a policy condition, or a planning constraint rather than as a purely low-level safety variable (Collins et al., 2023, Yang et al., 2023, Noseworthy et al., 2024).

1. Demonstration system and sensing stack

ManipForce is built around a handheld RGB–F/T demonstration collection interface intended for natural human demonstrations of contact-rich manipulation. The hardware includes two hand-eye RGB cameras, an Intel RealSense D455 and an Intel RealSense D405, a wrist-mounted AIDIN AFT200 F/T sensor, ArUco markers for pose tracking, and a custom gripper with rack-and-pinion parallel-jaw actuation, a trigger interface, deformable pin-ray fingertips, linear guides, and a trigger-lock. A key mechanical detail is that all gripper components are mounted downstream of the F/T sensor, so the sensor measures the complete interaction wrench during manipulation. RGB and wrist-pose data are recorded at 30 Hz, while F/T is recorded at 200 Hz; the system preserves the higher-rate force stream and aligns it to visual frames by timestamp-based windowing rather than downsampling force to image rate (Lee et al., 23 Sep 2025).

Pose tracking is obtained with an externally positioned Azure Kinect that detects 3D ArUco markers and is described as viewpoint-independent with sub-millimeter accuracy. End-effector actions are represented as 6-DOF pose deltas between consecutive frames, transformed from the marker reference frame to the robot TCP frame using CAD-derived transformation matrices. Markers attached to the gripper jaws are tracked to infer gripper open-close state. The same sensing configuration is reproduced on the robot, with hand-eye cameras positioned to match the TCP-to-lens distances of the handheld setup, so that demonstrations can transfer directly into robot policy learning without task-specific calibration (Lee et al., 23 Sep 2025).

The system applies tool gravity compensation because the measured wrench contains both contact interaction and gravitational loading. With the IMU in the D455 camera, the gravity vector in the F/T frame is computed as

$$\mathbf{g}_{ft} = R^{ft}_{imu} \, \mathbf{g}_{imu},$$

with corresponding gravitational force and torque

$$\mathbf{F}_{g} = m_{tool} \, \mathbf{g}_{ft}, \quad \boldsymbol{\tau}_{g} = \mathbf{r}_{com} \times \mathbf{F}_{g},$$

and compensated wrench

$$\mathbf{F}_{comp} = \mathbf{F}_{measure} - \mathbf{F}_{g}, \quad \boldsymbol{\tau}_{comp} = \boldsymbol{\tau}_{measure} - \boldsymbol{\tau}_{g}.$$

The paper validates this compensation by tilting the device by $\pm90^\circ$ about the $x$- and $y$-axes: without compensation, force readings reached up to $\pm 8$ N and torques up to $0.4$ N·m, whereas with compensation the residual forces were within $\pm 1$ N and residual torques within $0.15$ N·m (Lee et al., 23 Sep 2025).

2. Frequency-Aware Multimodal Transformer

FMT extends Transformer Diffusion Policy to asynchronous multimodal observations. Its inputs are tokenized visual streams from the two cameras and a tokenized F/T stream:

$$\mathbf{F}_{g} = m_{tool} \, \mathbf{g}_{ft}, \quad \boldsymbol{\tau}_{g} = \mathbf{r}_{com} \times \mathbf{F}_{g},$$0

with

$$\mathbf{F}_{g} = m_{tool} \, \mathbf{g}_{ft}, \quad \boldsymbol{\tau}_{g} = \mathbf{r}_{com} \times \mathbf{F}_{g},$$1

RGB images are square padded and resized to $$\mathbf{F}_{g} = m_{tool} \, \mathbf{g}_{ft}, \quad \boldsymbol{\tau}_{g} = \mathbf{r}_{com} \times \mathbf{F}_{g},$$2, then encoded by DINOv2-B; force is encoded by a 1D CNN. The model adds spatial embeddings

$$\mathbf{F}_{g} = m_{tool} \, \mathbf{g}_{ft}, \quad \boldsymbol{\tau}_{g} = \mathbf{r}_{com} \times \mathbf{F}_{g},$$3

frequency-aware embeddings

$$\mathbf{F}_{g} = m_{tool} \, \mathbf{g}_{ft}, \quad \boldsymbol{\tau}_{g} = \mathbf{r}_{com} \times \mathbf{F}_{g},$$4

and modality embeddings

$$\mathbf{F}_{g} = m_{tool} \, \mathbf{g}_{ft}, \quad \boldsymbol{\tau}_{g} = \mathbf{r}_{com} \times \mathbf{F}_{g},$$5

For the camera streams, the frequency embeddings are resampled via linear interpolation to align approximately with camera timestamps (Lee et al., 23 Sep 2025).

The central fusion mechanism is bi-directional cross-attention. Vision tokens are enhanced by force tokens, and force tokens are enhanced by vision tokens:

$$\mathbf{F}_{g} = m_{tool} \, \mathbf{g}_{ft}, \quad \boldsymbol{\tau}_{g} = \mathbf{r}_{com} \times \mathbf{F}_{g},$$6

$$\mathbf{F}_{g} = m_{tool} \, \mathbf{g}_{ft}, \quad \boldsymbol{\tau}_{g} = \mathbf{r}_{com} \times \mathbf{F}_{g},$$7

The fused observation representation is then

$$\mathbf{F}_{g} = m_{tool} \, \mathbf{g}_{ft}, \quad \boldsymbol{\tau}_{g} = \mathbf{r}_{com} \times \mathbf{F}_{g},$$8

With the reported settings, the visual token count is $$\mathbf{F}_{g} = m_{tool} \, \mathbf{g}_{ft}, \quad \boldsymbol{\tau}_{g} = \mathbf{r}_{com} \times \mathbf{F}_{g},$$9, so the final multimodal observation sequence has length $$\mathbf{F}_{comp} = \mathbf{F}_{measure} - \mathbf{F}_{g}, \quad \boldsymbol{\tau}_{comp} = \boldsymbol{\tau}_{measure} - \boldsymbol{\tau}_{g}.$$0 (Lee et al., 23 Sep 2025).

FMT conditions a transformer diffusion policy on this multimodal representation. The paper describes the policy as generating action sequences by iterative denoising, with multimodal observation tokens serving as conditioning input, and states that end-effector actions are represented as 6-DOF pose deltas between consecutive frames. The exact denoising loss is not written in the reported text, but the architecture is explicitly positioned as a transformer diffusion policy rather than a deterministic regression model (Lee et al., 23 Sep 2025).

3. Real-world task suite and empirical performance

ManipForce is evaluated on six real-world tasks: Gear Assembly, LAN Plug Insertion, Box Flipping, Open Lid, Battery Disassembly, and Battery Insertion. Each task uses approximately 100 demonstration episodes for training and 20 randomized trials for evaluation. The principal comparison is against an RGB-only diffusion policy that uses the same Transformer Diffusion Policy backbone and DINOv2-B image encoder but excludes F/T (Lee et al., 23 Sep 2025).

Task	RGB-only	FMT
Gear Assembly	0.35	0.95
LAN Plug Insertion	0.40	0.85
Box Flipping	0.05	0.90
Open Lid	0.20	1.00
Battery Disassembly	0.20	0.65
Battery Insertion	0.10	0.60

These results yield an average success rate of about 22% for RGB-only and 83% for FMT. The largest absolute gains occur in Box Flipping and Open Lid, where the paper attributes the advantage to force-sensitive behavior in sustained contact and transient contact events. Gear Assembly and Battery Insertion are described as especially dependent on subtle torque changes, sticking, jamming, and alignment under rapidly changing contact forces (Lee et al., 23 Sep 2025).

The paper also reports three ablations: Low-Freq F/T (30 Hz), w/o Freq-aware Multimodal Embeddings, and w/o Cross-Attention. Open Lid shows the sharpest degradation when force is downsampled to 30 Hz, which the paper interprets as evidence that the task depends on a single transient high-frequency force spike. In a sampling-frequency analysis on Gear Assembly, performance rises monotonically from 0.40 at 30 Hz to 0.95 at 200 Hz, with 60 Hz and 120 Hz also evaluated in the figure. The authors conclude that higher F/T sampling captures short-duration force spikes, subtle torque changes, and sticking or jamming events that are not preserved at image rate (Lee et al., 23 Sep 2025).

4. Relation to visual-force imitation and goal-conditioned force prediction

ManipForce belongs to a recent cluster of systems that elevate force from an execution-side variable to an interpretable task representation. ForceSight defines a visual-force goal

$$\mathbf{F}_{comp} = \mathbf{F}_{measure} - \mathbf{F}_{g}, \quad \boldsymbol{\tau}_{comp} = \boldsymbol{\tau}_{measure} - \boldsymbol{\tau}_{g}.$$1

with

$$\mathbf{F}_{comp} = \mathbf{F}_{measure} - \mathbf{F}_{g}, \quad \boldsymbol{\tau}_{comp} = \boldsymbol{\tau}_{measure} - \boldsymbol{\tau}_{g}.$$2

where the applied force is a 3D vector and the grip force is a scalar. On a 100-trial real-world benchmark in unseen environments with unseen object instances, ForceSight achieves $$\mathbf{F}_{comp} = \mathbf{F}_{measure} - \mathbf{F}_{g}, \quad \boldsymbol{\tau}_{comp} = \boldsymbol{\tau}_{measure} - \boldsymbol{\tau}_{g}.$$3 success, and in a controlled comparison success drops from $$\mathbf{F}_{comp} = \mathbf{F}_{measure} - \mathbf{F}_{g}, \quad \boldsymbol{\tau}_{comp} = \boldsymbol{\tau}_{measure} - \boldsymbol{\tau}_{g}.$$4 ($$\mathbf{F}_{comp} = \mathbf{F}_{measure} - \mathbf{F}_{g}, \quad \boldsymbol{\tau}_{comp} = \boldsymbol{\tau}_{measure} - \boldsymbol{\tau}_{g}.$$5) to $$\mathbf{F}_{comp} = \mathbf{F}_{measure} - \mathbf{F}_{g}, \quad \boldsymbol{\tau}_{comp} = \boldsymbol{\tau}_{measure} - \boldsymbol{\tau}_{g}.$$6 ($$\mathbf{F}_{comp} = \mathbf{F}_{measure} - \mathbf{F}_{g}, \quad \boldsymbol{\tau}_{comp} = \boldsymbol{\tau}_{measure} - \boldsymbol{\tau}_{g}.$$7) when force goals are ignored (Collins et al., 2023). In that formulation, force is predicted as part of a one-keyframe-ahead subgoal rather than merely measured during execution.

MOMA-Force uses a different decomposition: retrieval-based visual imitation, wrench imitation, and admittance whole-body control. Its demonstrations contain RGB image, kinematic action, gripper action, wrench, and terminate flag, with the wrench defined as

$$\mathbf{F}_{comp} = \mathbf{F}_{measure} - \mathbf{F}_{g}, \quad \boldsymbol{\tau}_{comp} = \boldsymbol{\tau}_{measure} - \boldsymbol{\tau}_{g}.$$8

On six household articulated-object tasks, the reported average success rates are 20.0% for BC (single-task), 45.0% for MOMA-Force w/o FC, and 73.3% for full MOMA-Force, together with smaller contact forces and smaller force variances than baselines without force imitation (Yang et al., 2023). Here force is not encoded as a high-rate asynchronous stream, but it is retrieved as an expert target and converted into pose corrections through admittance control.

A broader interpretation is that ManipForce differs from both systems in where it places the multimodal bottleneck. ForceSight predicts force-sensitive subgoals from a single RGBD observation and text prompt, whereas MOMA-Force retrieves an expert wrench target from visual similarity; ManipForce instead retains the native 200 Hz F/T stream and fuses it with 30 Hz RGB through frequency-aware embeddings and bi-directional cross-attention. This suggests a shift from force as a predicted waypoint or retrieved target toward force as a temporally structured observation modality in its own right (Lee et al., 23 Sep 2025, Collins et al., 2023, Yang et al., 2023).

5. Force as action space and robustness parameter

Several neighboring works reinterpret force not only as sensed feedback but also as action space or policy parameter. FLEX formulates object-centric manipulation policies directly in force space for sustained-contact articulated-object manipulation. Its action space is

$$\mathbf{F}_{comp} = \mathbf{F}_{measure} - \mathbf{F}_{g}, \quad \boldsymbol{\tau}_{comp} = \boldsymbol{\tau}_{measure} - \boldsymbol{\tau}_{g}.$$9

and the actor outputs a direction head and a scale head. In simulation, the same learned policies transfer to Kinova Gen3, Panda, and UR5e without retraining, and on unseen objects the reported Panda success rates are $\pm90^\circ$0 on cabinets, $\pm90^\circ$1 on trashcans, $\pm90^\circ$2 on microwaves, and $\pm90^\circ$3 on dishwashers (Fang et al., 17 Mar 2025). This is a markedly different abstraction from ManipForce: force is the policy action rather than a sensed demonstration stream.

FORGE uses a force-threshold mechanism for sim-to-real contact-rich manipulation under uncertainty. The policy is explicitly conditioned on a maximum allowable force,

$\pm90^\circ$4

and excessive contact is penalized by

$\pm90^\circ$5

The method targets low-clearance peg insertion, gear meshing, nut threading, and planetary gearbox assembly under up to $\pm90^\circ$6 of fixed-part position estimation error. On real hardware, FORGE reports success rates of 0.84 for peg insertion and 0.98 for medium gear meshing, together with mean-force reductions from 11.81 N to 5.51 N on the peg task and from 11.67 N to 7.95 N on the gear task relative to a non-robust baseline (Noseworthy et al., 2024). Here force serves as both policy input and safety-budget parameter.

Taken together, these systems show three distinct but compatible readings of force-guided manipulation: force can be the action space itself, as in FLEX; a conditioning variable that defines allowable aggressiveness, as in FORGE; or an asynchronous sensory stream learned from natural demonstrations, as in ManipForce. This suggests that the term “ManipForce” can be read both narrowly, as the specific FMT-based framework, and more broadly, as a design principle for policies that organize contact-rich behavior around physically meaningful force variables (Lee et al., 23 Sep 2025, Fang et al., 17 Mar 2025, Noseworthy et al., 2024).

6. Planning, contact mechanics, and passive force shaping

Force-aware manipulation has also been developed at the planning and mechanics levels. “Planning for Multi-stage Forceful Manipulation” augments PDDLStream with controllers that exert wrenches and with constraints that explicitly consider torque and frictional limits. It introduces a new planning variable type, (Wrench ?w), and models strategy feasibility through forceful kinematic chains, small circular patch limit surfaces,

$\pm90^\circ$7

generalized friction cones,

$\pm90^\circ$8

and manipulator torque bounds

$\pm90^\circ$9

Its bottle domain includes four twisting methods—GT, FT, PT, TT—and four fixturing methods—RF, VF, SF(T), SF(M)—illustrating how force feasibility can reshape symbolic strategy choice (Holladay et al., 2021). “Robust Planning for Multi-stage Forceful Manipulation” extends the same line with cost-sensitive planning under uncertainty, where the action cost is

$x$0

and demonstrates bottle opening, nut twisting, and vegetable cutting with Monte Carlo robustness estimation over friction, wrench, and contact perturbations (Holladay et al., 2022).

At the control-mechanics level, “Compliant Manipulation of Free-Floating Objects” argues that free-floating alignment should be treated as a direct force-control problem rather than solely as a compliance problem. Its proposed method maintains a small constant force along the motion direction and uses measured contact wrench feedback to produce an apparent reduction in manipulator inertia along the remaining directions. In the reported setup, alignment is achieved at $x$1 N normal force with inertia reduction, versus $x$2 N for force control alone, while impedance control fails to maintain continuous contact (Sharma et al., 2020). “Manipulation via Force Distribution at Contact” extends the mechanics side further by replacing point contact with a Force-Distributed Line Contact (FDLC) model, approximated by two points connected by a virtual spring-damper system. In a box-rotation task, FDLC generates torque through non-uniform force distribution along the contact line and yields lower control effort, less robot motion, and continuous contact forces compared with point-contact modeling (Lee et al., 3 Feb 2026).

A different, fully passive route appears in “Compliant Constant Output/Input Force Mechanisms—Topology Optimization with Contact,” which synthesizes monolithic constant output force mechanisms and constant input force mechanisms while accounting for finite displacements, buckling, self-contact, mutual contact with external surfaces, and contact with flexible workpieces. The paper emphasizes that external surfaces may not be required for singlepiece mechanisms to attain constant force characteristics, though mutual contact can yield interesting solutions, and it notes that the desired force characteristics may not remain the same with alteration in the workpiece shape and material properties (Reddy et al., 2022). For ManipForce, these planning and mechanics results indicate that force-guided behavior can be learned, planned, or even embedded directly into morphology.

7. Limitations and open technical directions

The ManipForce paper carries several explicit limitations. It depends on wrist F/T sensing, dual hand-eye cameras, external Azure Kinect tracking during demonstration, and a custom handheld tool; transfer further depends on matching camera geometry and F/T measurement setup between the handheld system and the Franka Panda. The evaluation is confined to six relatively short-horizon single-skill tasks, and the paper omits several practical training details, including optimizer, learning rate, batch size, transformer depth, diffusion timestep count, and exact action-horizon specification (Lee et al., 23 Sep 2025). These constraints place it closer to a carefully engineered real-world platform than to a minimal software-only policy recipe.

The broader force-aware literature indicates several unresolved representational questions. ForceSight predicts a 3-axis applied force vector and a scalar grip force but does not predict torque components, full wrench trajectories, or temporal force profiles (Collins et al., 2023). FLEX is restricted to single-DOF prismatic and revolute articulations and does not dynamically adjust force magnitude for unforeseen factors like friction or spring tension (Fang et al., 17 Mar 2025). FORGE shows that force observations can hurt transfer on nut threading when the simulated force-observation distribution is not realistic enough (Noseworthy et al., 2024). These limitations suggest that high-frequency force sensing alone is not sufficient; contact-model fidelity and task structure remain central.

A plausible continuation is the unified multimodal trajectory modeling developed in “Unified Multimodal Diffusion Forcing,” which treats partial point cloud, full point cloud, force, action, proprioception, and reward as six modalities in a single generative model. MDF uses a 2D Time–Modality Noise Level Matrix

$x$3

to support policy mode, planner mode, dynamics prediction, state estimation, denoising, and anomaly localization. In simulation, removing wrench drops Nut Thread from 100% to 72%, Gear Mesh from 86% to 78%, and Peg Insert from 80% to 74% (Huang et al., 6 Nov 2025). This suggests that future ManipForce systems may move beyond policy conditioning toward a unified treatment of force, geometry, action, and latent contact state.

The ManipForce paper itself states that its future scope is longer-horizon tasks, more dexterous tasks, and tasks requiring more sophisticated gripper behaviors (Lee et al., 23 Sep 2025). Within the surrounding literature, that trajectory points toward force-aware policies that combine natural demonstrations, asynchronous multimodal sensing, explicit contact reasoning, and richer representations of extended or distributed contact.