GRASPTrack: Multi-System Robotics & MOT

• GRASPTrack is a polysemous term for systems in robotic manipulation, reactive grasping, and depth-aware multi-object tracking, each using distinct sensor fusion methods.
• The autonomous grasping system integrates visual and tactile feedback via ROS nodes to achieve refined 6-DoF pose estimation, delivering up to 80% pick-and-place success.
• Reactive grasping and MOT variants employ geometry-based tracking (e.g., voxelized IoU, depth estimation) and dynamic planning to ensure robustness under occlusion and motion.

GRASPTrack is a recurrent system name in robotics and computer vision literature, but it does not denote a single canonical method. In the sources considered here, the name is attached to three distinct technical systems: a ROS-based autonomous grasping and manipulation architecture that tightly couples visual and tactile sensing for the IROS 2016 Autonomous Grasping and Manipulation track; a reactive grasping pipeline for previously unseen objects built around 6-DoF tracking, grasp-point sampling, and dynamic trajectory planning; and a monocular-video multi-object tracking framework that performs explicit 3D geometric reasoning through depth estimation, instance segmentation, voxelized intersection-over-union, and depth-aware motion modeling (Patel et al., 2017, Tuscher et al., 2021, Han et al., 11 Aug 2025).

1. Name, scope, and research usage

In the literature represented here, “GRASPTrack” refers to multiple systems that share a coupling of tracking with downstream decision-making, but differ sharply in domain, sensing stack, and objective. One system is a manipulation architecture deployed in a robotics competition, another is a reactive grasping framework for unknown objects, and the most recent is a depth-aware MOT method for monocular video (Patel et al., 2017, Tuscher et al., 2021, Han et al., 11 Aug 2025).

Variant	Domain	Principal components
Autonomous “GRASPTrack” system	Robotic grasping and manipulation	Vision, Tactile, Perception–Fusion, Control, ROS Integration
GRASPTrack for reactive grasping	Unknown-object robotic manipulation	THOR + G-ICP tracking, Grasp-Point Sampling and Ranking, Dynamic Trajectory Planning
GRASPTrack for MOT	Monocular multi-object tracking	Depth estimation, instance segmentation, Voxel-Based 3D IoU, DANC, DOCM

This multiplicity is important for citation practice. References to GRASPTrack in robotic manipulation usually concern sensor fusion, 6-DoF pose tracking, or reactive grasp planning, whereas references in video understanding concern geometry-reasoned association under occlusion. A common misconception is to treat the name as if it identified one algorithmic lineage; the available record instead shows three separate systems with different problem formulations.

2. Autonomous grasping and manipulation architecture

The autonomous “GRASPTrack” system described by Patel et al. is organized as a set of ROS nodes that implement five logical modules: Vision, Tactile, Perception–Fusion, Control, and ROS Integration. Its salient feature is the tight coupling between visual perception from an Asus Xtion RGB-D camera and tactile perception from Robotic Materials in-hand sensors, with the explicit aim of reducing uncertainty in sensing and actuation during the final stages of grasping on a Kinova robotic arm (Patel et al., 2017).

The vision module subscribes to /camera/depth/points and /camera/rgb/image_rect_color, performs tabletop segmentation via RANSAC by fitting the plane $\pi:\{n\cdot x + d = 0\}$ to the point cloud and removing inliers, then applies Euclidean clustering to the remaining points. For each cluster it computes per-point normals by PCA on a local neighborhood, builds SHOT or VFH histograms $H_i$, matches $H_i$ against a template library, and publishes /detected_objects with an object id and a coarse 6 D pose. The tactile module uses a Kinova three-finger hand with Robotic Materials arrays; each finger has a proximity channel, described as SA-I analog and slow-changing, and a contact channel, described as FA-I analog and fast-changing. FA-I is extracted with a high-pass filter,

$y[n] = \alpha \cdot (y[n-1] + x[n] - x[n-1]),$

where $\alpha = \tau/(\tau+\Delta t)$ and $\tau \approx 1/(2\pi \cdot 20\ \mathrm{Hz})$. Event detection uses calibrated thresholds:

• $contact\_event_i = (FAI_i < -\theta_{touch})$
• $release\_event_i = (FAI_i > +\theta_{touch})$
• $object\_detect_i = (SAI_i < -\theta_{obj})$
• $separation\_event_i = (SAI_i > +\theta_{obj})$

The perception–fusion stage is sequential rather than probabilistic. The system first uses 3 D vision to compute $H_i$0, approaches along a fixed Cartesian offset $H_i$1, and then switches to velocity-controlled search guided by proximity toward the centroid of a shell around the finger. If a proximity_event_i fires, velocity is zeroed and the controller switches to final grasp close. The paper explicitly notes that no probabilistic filter is described; although the update can be written suggestively as

$H_i$2

the implemented system uses hard thresholds rather than full Bayesian fusion.

The control module alternates between Cartesian mode for large motions and velocity mode for fine search. The position-control law is

$H_i$3

and the task-space velocity law is

$H_i$4

with $H_i$5 dynamically updated by proximity events. Motion is terminated when any contact_event fires. Finger closure is executed immediately after contact, with a constant fingertip closure velocity until contact_count \geq 2 or time-out. Camera-to-robot calibration uses an AR tag on the wrist and forward kinematics to obtain

$H_i$6

with residual registration error reported as $H_i$7.

The reported competition results make clear both the utility and the limits of the architecture. In a 30 min pick-and-place run over 10 objects, the system placed $H_i$8 objects, corresponding to $H_i$9 success; failures were due to large pose estimation error on small or shiny items. In 10 manipulation tasks, it succeeded in spoon scooping and depositing, stirring coffee, towel pick-and-hang, straw grasp, and syringe pumping with human assist; it partially succeeded in salt shaker shaking and USB-light unplug; and it failed at hammer and nail and scissors cutting, yielding an overall $H_i$0 tasks, or $H_i$1 success. The error statistics are equally informative: camera–arm calibration RMSE was $H_i$2, tactile event timing detected contact within $H_i$3 after first physical touch, and closed-loop search reduced approach error from $H_i$4 to $H_i$5 prior to grasp close. All software was implemented as ROS nodes and made available at https://github.com/correlllab/cu-perception-manipulation-stack.

3. Reactive grasping of unknown objects

A second system carrying the GRASPTrack name addresses reactive grasping of previously unseen objects through a fully integrated real-time pipeline composed of three main components: Object Tracking (THOR + G-ICP), Grasp-Point Sampling and Ranking, and Dynamic Trajectory Planning. The pipeline operates on commodity hardware, detects new objects on a table with a top-down depth camera by background subtraction, initializes each object with a 2D bounding box and an estimated 6-DoF pose, and then uses a side-view RGB-D camera to update the pose at approximately $H_i$6 (Tuscher et al., 2021).

The tracking stack combines Siamese template matching with point-cloud registration. A pre-trained Siamese network $H_i$7 extracts $H_i$8-channel feature maps from the initial template $H_i$9 and a search image $y[n] = \alpha \cdot (y[n-1] + x[n] - x[n-1]),$0, and object localization is obtained through depth-wise cross-correlation,

$y[n] = \alpha \cdot (y[n-1] + x[n] - x[n-1]),$1

THOR maintains two template banks, a long-term module of size $y[n] = \alpha \cdot (y[n-1] + x[n] - x[n-1]),$2 and a short-term module of size $y[n] = \alpha \cdot (y[n-1] + x[n] - x[n-1]),$3. Every $y[n] = \alpha \cdot (y[n-1] + x[n] - x[n-1]),$4 frames the newest crop is inserted into the STM in FIFO order, and it is occasionally inserted into the LTM if it increases the volume of the feature parallelotope. The criterion for adding a new template $y[n] = \alpha \cdot (y[n-1] + x[n] - x[n-1]),$5 to the LTM is

$y[n] = \alpha \cdot (y[n-1] + x[n] - x[n-1]),$6

where $y[n] = \alpha \cdot (y[n-1] + x[n] - x[n-1]),$7, $y[n] = \alpha \cdot (y[n-1] + x[n] - x[n-1]),$8, and $y[n] = \alpha \cdot (y[n-1] + x[n] - x[n-1]),$9. The 3D component extracts masked RGB-D pixels, projects them into world-space points

$\alpha = \tau/(\tau+\Delta t)$0

filters the observation cloud with a voxel grid, and solves for the rigid-body transform $\alpha = \tau/(\tau+\Delta t)$1 minimizing

$\alpha = \tau/(\tau+\Delta t)$2

implemented by Generalized ICP. The final tracked pose is the composition of the previous pose update, or the kinematic map if the robot holds the object, with $\alpha = \tau/(\tau+\Delta t)$3.

Grasp synthesis is formulated over candidates $\alpha = \tau/(\tau+\Delta t)$4, where $\alpha = \tau/(\tau+\Delta t)$5 is the joint configuration, $\alpha = \tau/(\tau+\Delta t)$6 is a cost vector, and $\alpha = \tau/(\tau+\Delta t)$7 is age. Candidates are generated by solving a regularized inverse-kinematics problem,

$\alpha = \tau/(\tau+\Delta t)$8

The task maps include position $\alpha = \tau/(\tau+\Delta t)$9, alignment, and collision,

$\tau \approx 1/(2\pi \cdot 20\ \mathrm{Hz})$0

with threshold $\tau \approx 1/(2\pi \cdot 20\ \mathrm{Hz})$1. To diversify approach directions, the system adds a random alignment map $\tau \approx 1/(2\pi \cdot 20\ \mathrm{Hz})$2 using a random orthonormal basis $\tau \approx 1/(2\pi \cdot 20\ \mathrm{Hz})$3 and schedules the weights with logistic ramps

$\tau \approx 1/(2\pi \cdot 20\ \mathrm{Hz})$4

where $\tau \approx 1/(2\pi \cdot 20\ \mathrm{Hz})$5. The reported optimizer settings are $\tau \approx 1/(2\pi \cdot 20\ \mathrm{Hz})$6, $\tau \approx 1/(2\pi \cdot 20\ \mathrm{Hz})$7, $\tau \approx 1/(2\pi \cdot 20\ \mathrm{Hz})$8, $\tau \approx 1/(2\pi \cdot 20\ \mathrm{Hz})$9, $contact\_event_i = (FAI_i < -\theta_{touch})$0, $contact\_event_i = (FAI_i < -\theta_{touch})$1, and $contact\_event_i = (FAI_i < -\theta_{touch})$2 candidates. Candidates whose end-effector pose lies outside the 3D object bounding box are discarded; the remaining candidates are ranked by distance of the end-effector contact to the cuboid center, with homing cost used as a tie-breaker. In each control loop, age is incremented, costs are recomputed, and candidates are re-ranked.

The dynamic planner produces collision-free joint-space trajectories by a backstepping rule. The nominal trajectory is the straight line in joint space connecting the current configuration $contact\_event_i = (FAI_i < -\theta_{touch})$3 to the top-ranked grasp configuration $contact\_event_i = (FAI_i < -\theta_{touch})$4 with step size $contact\_event_i = (FAI_i < -\theta_{touch})$5. On each iteration the system attempts the next point on this path; if that point is in collision with the environment or with the updated object pose, it steps back one or more points toward $contact\_event_i = (FAI_i < -\theta_{touch})$6 and retries. This planner is explicitly incremental rather than sampling-based.

Quantitative evaluation emphasizes robustness to occlusion and perturbation. In simulation with up to $contact\_event_i = (FAI_i < -\theta_{touch})$7 visual occlusion by a planar blocker, THOR + G-ICP maintained translation error $contact\_event_i = (FAI_i < -\theta_{touch})$8 and rotation error $contact\_event_i = (FAI_i < -\theta_{touch})$9, whereas a PCL particle-filter tracker degraded linearly to $release\_event_i = (FAI_i > +\theta_{touch})$0 at $release\_event_i = (FAI_i > +\theta_{touch})$1. In a manipulation handover experiment, THOR + G-ICP sustained errors $release\_event_i = (FAI_i > +\theta_{touch})$2 and $release\_event_i = (FAI_i > +\theta_{touch})$3 even during full occlusion by a gripper, while the particle filter lost track at handover. On a real robot, grasp success was reported as $release\_event_i = (FAI_i > +\theta_{touch})$4 $release\_event_i = (FAI_i > +\theta_{touch})$5 on a cuboid, $release\_event_i = (FAI_i > +\theta_{touch})$6 on a Pringles can under pushes, $release\_event_i = (FAI_i > +\theta_{touch})$7 on a measuring tape, and $release\_event_i = (FAI_i > +\theta_{touch})$8 on a lashing strap, with an average end-to-end reaction time of $release\_event_i = (FAI_i > +\theta_{touch})$9 from new observation to updated motion command. The paper also records limitations: the segmentation branch of SiamMask relies on color and texture, the grasp sampler assumes a cuboid-aligned pose, and the backstepping planner may fail in highly cluttered 3D environments.

4. Geometry-reasoned association for multi-object tracking

The 2025 GRASPTrack is a depth-aware MOT framework for monocular video. It augments a standard tracking-by-detection pipeline by fusing monocular depth estimation and instance segmentation to recover object-level 3D geometry, and then uses that geometry for more robust spatial and motion association under occlusion and depth ambiguity (Han et al., 11 Aug 2025).

The pipeline begins with a frame $object\_detect_i = (SAI_i < -\theta_{obj})$0. A pretrained depth network, Depth Anything v2, produces a dense depth map

$object\_detect_i = (SAI_i < -\theta_{obj})$1

and for each 2D detection $object\_detect_i = (SAI_i < -\theta_{obj})$2, EfficientTAM yields a binary mask

$object\_detect_i = (SAI_i < -\theta_{obj})$3

Pixels inside the mask are projected into camera space via

$object\_detect_i = (SAI_i < -\theta_{obj})$4

producing the per-instance point cloud

$object\_detect_i = (SAI_i < -\theta_{obj})$5

For association, the method computes a common 3D bounding volume over a detection cloud and a track cloud, discretizes the space with voxel size $object\_detect_i = (SAI_i < -\theta_{obj})$6, and defines the Voxel-Based 3D IoU as

$object\_detect_i = (SAI_i < -\theta_{obj})$7

The reported operating value is $object\_detect_i = (SAI_i < -\theta_{obj})$8.

Two further mechanisms specialize the association stage. Depth-Aware Adaptive Noise Compensation extends the Kalman state to

$object\_detect_i = (SAI_i < -\theta_{obj})$9

marks object $separation\_event_i = (SAI_i > +\theta_{obj})$0 as occluded if there exists $separation\_event_i = (SAI_i > +\theta_{obj})$1 with $separation\_event_i = (SAI_i > +\theta_{obj})$2 and $separation\_event_i = (SAI_i > +\theta_{obj})$3, defines occlusion strength as

$separation\_event_i = (SAI_i > +\theta_{obj})$4

and scales the process covariance by

$separation\_event_i = (SAI_i > +\theta_{obj})$5

with $separation\_event_i = (SAI_i > +\theta_{obj})$6. Depth-enhanced Observation-Centric Momentum extends image-plane motion consistency into 3D through displacement vectors and the cosine similarity

$separation\_event_i = (SAI_i > +\theta_{obj})$7

The association cost combines appearance score, $separation\_event_i = (SAI_i > +\theta_{obj})$8, and $separation\_event_i = (SAI_i > +\theta_{obj})$9.

The benchmark results place the method against ByteTrack, OC-SORT, and UCMCTrack on MOT17, MOT20, and DanceTrack.

Benchmark	GRASPTrack result	Comparison stated in the data
MOT17	$H_i$00 HOTA / $H_i$01 IDF1 / $H_i$02 MOTA / $H_i$03 AssA	Higher HOTA, IDF1, and AssA than ByteTrack, OC-SORT, and UCMCTrack
MOT20	$H_i$04 HOTA / $H_i$05 IDF1 / $H_i$06 MOTA / $H_i$07 AssA	Highest values among the listed trackers
DanceTrack	$H_i$08 HOTA / $H_i$09 IDF1 / $H_i$10 MOTA / $H_i$11 AssA	Higher than OC-SORT and UCMCTrack on all listed metrics

The ablation studies identify the contribution of each component on the DanceTrack validation set relative to an appearance-only baseline with HOTA $H_i$12. Adding Voxel-IoU gives HOTA $H_i$13, adding DANC gives $H_i$14, adding DOCM gives $H_i$15, adding Voxel-IoU + DANC gives $H_i$16, and the full model gives HOTA $H_i$17. Voxel size shows a trade-off between accuracy and speed: $H_i$18 FPS, $H_i$19 FPS, $H_i$20 FPS, and $H_i$21 FPS, with the reported optimal trade-off at $H_i$22. Occlusion sensitivity peaks at $H_i$23, and mask-guided point-cloud generation outperforms full-box generation, $H_i$24 versus $H_i$25 HOTA. Qualitatively, the method is reported to maintain identities through heavy occlusions and rapid movements, such as dancers crossing or pedestrians in dense crowds, more effectively than 2D-only IoU or 2D momentum methods.

5. Methodological contrasts and conceptual continuities

The three GRASPTrack systems differ most strongly in what is being tracked and how geometric information is operationalized. In the 2016 manipulation architecture, vision provides a coarse object pose and in-hand proximity and contact sensing refine the final approach through event-driven switching; the fusion is explicitly sequential and based on hard thresholds rather than a full Bayesian filter (Patel et al., 2017, Tuscher et al., 2021, Han et al., 11 Aug 2025). In the 2021 reactive grasping framework, the tracked quantity is a full 6-DoF object pose updated by Siamese template matching plus Generalized ICP, and the tracked pose directly parameterizes a grasp-configuration optimizer and a backstepping motion planner. In the 2025 MOT framework, the target is object identity persistence in monocular video, and geometry is expressed through per-instance point clouds, voxelized 3D IoU, depth-aware Kalman noise scaling, and 3D motion consistency.

The role of sensing also changes across the three systems. The 2016 architecture is multimodal in a literal hardware sense, combining RGB-D vision with in-hand tactile channels labeled SA-I and FA-I. The 2021 system is RGB-D but does not yet incorporate tactile feedback; the paper explicitly lists tactile feedback and online grasp-point refinement as future extensions. The 2025 framework is monocular at input, reconstructing depth from a pretrained network and using instance segmentation to recover object-specific 3D structure. This suggests that the shared name “GRASPTrack” is not tied to one sensor modality or one optimization formalism, but to a broader pattern in which tracking output is made immediately actionable by downstream association, control, or planning.

There is also a shift in the level of abstraction at which geometry is used. The 2016 system uses geometry primarily for coarse alignment and reactive closing, the 2021 system uses geometry for rigid registration and constrained inverse kinematics in $H_i$26 and joint space, and the 2025 system uses geometry as an association primitive through occupancy grids and depth-augmented motion descriptors. These differences explain why direct performance comparison across the three systems is not meaningful: their evaluation metrics range from pick-and-place success rates and tactile latency, to translation and rotation error under occlusion, to HOTA, IDF1, MOTA, and AssA.

6. Limitations, failure modes, and interpretive cautions

The documented limitations are domain-specific. In the 2016 robotic manipulation system, pick-and-place failures were due to large pose estimation error on small or shiny items, while hammer and nail failed because gripper compliance was too high and scissors cutting failed because of loss of contact during open/close (Patel et al., 2017, Tuscher et al., 2021, Han et al., 11 Aug 2025). Partial success on salt shaker shaking was attributed to too low a dispensing rate, and USB-light unplug was limited by unreliable re-insertion. These observations locate the main failure sources in calibration, object appearance, and end-effector mechanics rather than in the basic ROS-node architecture.

In the 2021 reactive grasping system, the limitations are more explicitly algorithmic. The tracker relies on color and texture in the segmentation branch of SiamMask, so very low-contrast objects in poor lighting may fail. The grasp sampler assumes a cuboid-aligned pose, which makes highly irregular shapes a poor fit for the current contact model. The backstepping planner is intentionally simple, but the paper notes that it may fail in highly cluttered 3D environments and suggests that integrating sampling-based motion planning could improve convergence. Future work proposed in that paper includes predicting grasp success probabilities directly from RGBD, incorporating tactile feedback and online grasp-point refinement, and extending tracking to textured deformable objects via dense-flow + ICP.

For the 2025 MOT framework, the reported evidence is benchmark-centered rather than failure-mode-centered. The method is evaluated on MOT17, MOT20, and DanceTrack, and the ablations isolate the effects of Voxel-IoU, DANC, DOCM, voxel size, occlusion sensitivity, and mask-guided point-cloud generation. A plausible implication is that performance depends materially on the fidelity of the depth map and the segmentation mask, because those quantities determine the quality of the 3D point clouds and the resulting voxelized overlap measure. The data provided, however, frames the system primarily through comparative benchmark performance and ablation structure rather than through a catalog of specific failure cases.

Taken together, these systems show that GRASPTrack is best understood as a polysemous research label. In robotic manipulation, it denotes architectures for reactive physical interaction with objects; in MOT, it denotes a geometry-reasoned association framework for preserving identities in video. Accurate interpretation therefore depends on the surrounding citation context, task definition, and mathematical machinery rather than on the name alone.