Terminal Wrench Dataset Overview

• Terminal Wrench Dataset is a structured collection recording time-stamped 6-DoF object poses with corresponding 6D wrench vectors for detailed manipulation analysis.
• It can be acquired via high-frequency physical F/T sensor instrumentation or computed augmentation from motion capture, ensuring accurate data synchronization at 100–200 Hz.
• The dataset underpins applications in robotics, biomechanics, and human–robot interaction by enabling precise force analysis, regression modeling, and machine learning-driven manipulation control.

A terminal wrench dataset is a structured collection that, for each time-stamped sample of a manipulation or interaction task, records both the 6-degree-of-freedom (DoF) pose of a manipulated object and the corresponding 6-dimensional interaction wrench vector (forces and moments) at the interface between a manipulator (robotic or human) and that object. These datasets represent a critical resource for research in robotics, biomechanics, human–robot interaction, and manipulation learning, as they enable quantitative analysis of the dynamic coupling between actions, forces, and object responses.

1. Formal Definition of Terminal Wrench

The canonical terminal wrench for a single sample is a 6-dimensional vector

$$w = \begin{bmatrix} \mathbf{f} \ \boldsymbol{\tau} \end{bmatrix} = \begin{bmatrix} f_x \ f_y \ f_z \ \tau_x \ \tau_y \ \tau_z \end{bmatrix} \in \mathbb{R}^6,$$

where $\mathbf{f}$ is the 3D force (in N) and $\boldsymbol{\tau}$ is the 3D moment (in N·m), both typically measured or estimated at the contact or tool frame defined by the end-effector or hand. In a terminal wrench dataset, each time sample $t$ is paired with the pose of the object

$$T_o(t) = \begin{bmatrix} R_o(t) & p_o(t) \ 0 & 1 \end{bmatrix} \in SE(3),$$

where $R_o$ is rotation matrix and $p_o$ is position, in the global or task-specific coordinate frame. Each sequence (trial) yields a set $\{(T_o(t), w(t))\}_{t=0}^{T-1}$ (Herneth et al., 2024, Martín-Martín et al., 2018).

2. Methods for Acquisition and Computation

Terminal wrench datasets can be collected using direct physical instrumentation, or through computational augmentation of marker-based motion recordings. The two primary methodologies are:

Physical Instrumentation:

A calibrated 6-axis force/torque (F/T) sensor is mounted between the manipulator (robot wrist or human hand tool) and the manipulated object. The F/T sensor directly records $w$ at high frequency (e.g., 100 Hz), while motion capture systems provide synchronized rigid-body poses for the object, tool, and manipulator links. Sensor frames are mapped to canonical object frames via known transformations, with corrections for bias, gravity, and tool acceleration (Martín-Martín et al., 2018).

Object Augmentation Algorithms (OAA):

When only marker trajectories are present (e.g., optical motion capture), the OAA pipeline reconstructs the hand/object frame, fits a virtual object via inverse kinematics (IK) to the marker data, and computes the interaction wrench using inverse dynamics (ID). This requires knowledge of the manipulator kinematics, segment inertias, and CAD parameters, producing time-aligned $\{T_o(t), w(t)\}$ by solving

$\mathbf{f}$0

where $\mathbf{f}$1, $\mathbf{f}$2, $\mathbf{f}$3 are the manipulator joint states, $\mathbf{f}$4 is the end-effector Jacobian, and $\mathbf{f}$5 is the Moore–Penrose pseudo-inverse augmented with regularization (Herneth et al., 2024).

3. Dataset Structure, Synchronization, and Validation

Terminal wrench datasets are organized per trial or interaction, with columns for: time, manipulator joint states ($\mathbf{f}$6), object pose ($\mathbf{f}$7), and the six interaction wrench components. High-fidelity datasets sample at 100–200 Hz for motion/wrench data, synchronized to a master timebase (common clock or timestamps; alignment via nearest-neighbor interpolation or direct time-matching) (Martín-Martín et al., 2018). Metadata includes object mass, inertial parameters, marker identities, and coordinate frame conventions.

Validation is performed against ground-truth on robotic or instrumented systems. For OAA, reconstructed 6-DoF trajectories show marker-to-marker $\mathbf{f}$8 errors of $\mathbf{f}$9 mm (Pearson $\boldsymbol{\tau}$0); torques computed via ID correlate with measured F/T sensor data at $\boldsymbol{\tau}$1; mean absolute torque errors are $\boldsymbol{\tau}$2–$\boldsymbol{\tau}$3 Nm, with peak errors up to 10–20% of typical joint-torque range. Statistical tests (Kruskal–Wallis $\boldsymbol{\tau}$4) confirm repeatability. Minimum requirements are five hand markers and two wrist markers for robust hand frame definition; loss of a single marker yields $\boldsymbol{\tau}$510–15 mm centroid shift with negligible wrench impact (Herneth et al., 2024).

4. Usage in Robotics and Biomechanics

Terminal wrench datasets are foundational for:

• Learning-based Manipulation: Training observation-to-action or force-prediction models requires paired pose–wrench data. Data-driven approaches for manipulator control, grasp evaluation, and dynamics modeling depend on such datasets (Herneth et al., 2024, Rojas et al., 2016).
• Contact Task Introspection: Wrench signals encode moment-by-moment contact events, compliance, slip, and task completion. Action grammars constructed from segmented wrench sequences provide interpretable high-level summaries for classification and monitoring (Rojas et al., 2016).
• Human–Robot Collaboration and Prosthetics: Quantitative analysis of the wrenches induced by object manipulation enables regression analyses of required joint torques and the derivation of principal components for prosthesis or exoskeleton design (Herneth et al., 2024).
• Biomechanics and Ergonomics: Analysis of terminal wrenches during activities of daily living elucidates how external loads translate to joint loading in both able-bodied and impaired populations (Herneth et al., 2024).

5. Notable Datasets and Public Resources

Several landmark datasets exemplify the terminal wrench paradigm:

Dataset/Algorithm	Modality	Wrench Source	Data Access
RBO Dataset (Martín-Martín et al., 2018)	Human–object, RGB-D	Direct F/T sensor	https://tu-rbo.github.io/articulated-objects/
OAA Augmented (Herneth et al., 2024)	Human/robot, MoCap	Computed via ID	By running OAA scripts on user datasets
Robot Introspection (Rojas et al., 2016)	Sim/real robot	Direct F/T sensor	http://www.juanrojas.net/2017icra_wrench_introspection

The RBO Dataset provides 78 interactions with synchronized kinematics and 6D wrench data at 100 Hz, including articulated object CAD models and calibration details. OAA outputs (notably from (Herneth et al., 2024)) can be created for any marker-based trial, yielding object pose and wrench trajectories at mocap frequencies (100–200 Hz) with documented ground-truth validation.

6. Segmentation, Taxonomies, and Machine Learning Representations

Raw wrench streams can be segmented into primitives, motion compositions, and low-level behaviors (LLB) as in the robot introspection pipeline (Rojas et al., 2016). Segmentation is via recursive linear regression, with slope-based classification using well-defined gradient cut-offs: $\boldsymbol{\tau}$6 Subsequent layers aggregate primitives to behavioral tokens (Push, Pull, Fixed, etc.), enabling “sentence”-based description of manipulative actions and supporting robust classification accuracy ($\boldsymbol{\tau}$795%) with SVMs or Mondrian Forests. Tokenization is zero-padded for consistent dimensionality across trials (Rojas et al., 2016).

OAA- and sensor-based datasets are also used for regression (e.g., object pose $\boldsymbol{\tau}$8 wrench prediction), inverse-dynamics model training, and biomechanical analytics. Principal component analysis and time-series modeling require synchronized, normalized pose–wrench pairs as provided in terminal wrench datasets (Herneth et al., 2024, Martín-Martín et al., 2018).

7. Practical Aspects: Formats, Preprocessing, and Access

Raw terminal wrench datasets are typically distributed as CSV files per trial, with columns: time (s), joint states ($\boldsymbol{\tau}$9, $t$0, $t$1), object position and orientation ($t$2, $t$3 as quaternion or Euler angles), $t$4, $t$5. Metadata and object descriptors are included for reproducibility. Preprocessing involves bias/gravity correction (for sensor data), 6 Hz low-pass filtering (for marker-derived data), and marker labeling. For ROS users, wrenches and kinematics are often available as topics in .bag files for replay and analysis (Martín-Martín et al., 2018).

The OAA provides open-source scripts for dataset generation and validation. Direct sensor-acquired datasets require tool calibration, transformation to object frames, and synchronization with motion and video modalities (Herneth et al., 2024, Martín-Martín et al., 2018).

---

References:

• "The RBO Dataset of Articulated Objects and Interactions" (Martín-Martín et al., 2018)
• "Object Augmentation Algorithm: Computing virtual object motion and object induced interaction wrench from optical markers" (Herneth et al., 2024)
• "Robot Introspection via Wrench-based Action Grammars" (Rojas et al., 2016)