TrackOR: Long-Term OR Staff Tracking

• The paper introduces a framework that uses 3D geometric signatures for robust, long-term tracking and re-identification of operating room staff.
• It combines an online tracking stage with an offline global recovery stage to link fragmented tracklets into complete staff trajectories.
• Experimental results on MM-OR show 82.3% association accuracy, outperforming baselines by nearly 11 percentage points.

TrackOR is a framework for long-term multi-person tracking and re-identification in the operating room (OR), designed to maintain a persistent identity state over entire surgical procedures rather than only within short visible intervals. It is formulated as an end-to-end “tracking by detection” system with an online tracking stage and an offline global recovery stage, and its central technical claim is that 3D geometric signatures derived from segmented human point clouds are more reliable than conventional RGB appearance cues in the OR, where staff exhibit severe occlusions, crowded interactions, homogeneous attire, and prolonged exits and re-entries. On MM-OR, TrackOR reports 82.3% Association Accuracy and improves the strongest baseline by about +11 percentage points, while also enabling downstream analyses such as temporal pathway imprints for staff-centric OR intelligence (Wang et al., 11 Aug 2025).

1. Problem setting and formalization

TrackOR addresses operating-room scene understanding at the level of specific staff members over long time spans. The paper distinguishes between a tracklet, which is a single continuous visible segment of one person, and the person’s full trajectory, which may contain multiple separated visible episodes because OR personnel can leave and later re-enter the room. This distinction is formalized as

$\text{trk}_m^a = \{s_t^m\}_{t=b_m^a}^{e_m^a},$

where $s_t^m$ is the state of person $m$ at time $t$, and $b_m^a, e_m^a$ are the start and end frame indices for the $a$-th visible period. The complete trajectory is then

$$\tau_m = \{ \text{trk}_m^1, \text{trk}_m^2, ..., \text{trk}_m^{A_m}\},$$

where $A_m$ is the number of separate appearances of person $m$ (Wang et al., 11 Aug 2025).

The OR setting combines several classical failure modes of multi-object tracking. The paper explicitly emphasizes severe occlusions, crowded scenes, homogeneous attire, long procedures, and the “revolving door” reality of surgery, in which personnel such as a circulating nurse may leave the OR and later return. In this regime, short-term tracking is insufficient because a tracker must solve not only frame-to-frame association but also re-identification after long absences. TrackOR is therefore motivated by a shift from role-based understanding to staff-centric understanding: personalized OR intelligence requires distinguishing which specific nurse, technician, or surgeon is present, not merely inferring an abstract role (Wang et al., 11 Aug 2025).

2. End-to-end framework and sensing configuration

TrackOR uses multi-view RGB-D data from ceiling-mounted cameras. In the MM-OR dataset used for evaluation, each frame includes three annotated RGB views and a corresponding 3D point cloud. These inputs are used in a complementary manner: multi-view RGB is used for 3D human pose detection, while the 3D point cloud is segmented per person and converted into depth-based representations for re-identification (Wang et al., 11 Aug 2025).

The framework has two major stages. The first is an online tracking stage that performs framewise association while attempting to preserve identity as robustly as possible. The second is an offline global recovery stage that links fragmented tracklets into person-level trajectories and produces analysis-ready global trajectories. The detection component follows prior “3D-first” OR approaches and directly detects 3D human poses from multi-view RGB input using VoxelPose. The paper summarizes VoxelPose as predicting 2D heatmaps in each camera view, aggregating them into a unified 3D voxel volume, using a 3D CNN to predict each person’s root location, and then using a second 3D CNN to regress the detailed 3D pose (Wang et al., 11 Aug 2025).

After pose detection, the full scene point cloud is segmented to isolate each human, and each 3D pose is associated with its corresponding person point cloud. TrackOR then renders each segmented 3D human point cloud into 8 virtual viewpoints arranged equidistantly in a circle around the person. These projections produce 2D depth maps, which are processed by a re-identification network to yield a feature tensor

$\mathcal{F}_t^{i} \in \mathbb{R}^{8 \times C},$

where $s_t^m$0 is the feature dimension. This feature is the system’s central identity representation and is explicitly geometric rather than texture-based (Wang et al., 11 Aug 2025).

3. 3D geometric signatures and online association

The main technical contribution of TrackOR is the use of 3D geometric signatures for person re-identification in the OR. A geometric signature is constructed from a person’s segmented 3D point cloud, rendered from 8 virtual camera viewpoints, converted into depth maps, and encoded by a ReID network. The intended identity cues are person-specific 3D structural properties such as body shape, proportions, silhouette, and multi-view geometric structure. The paper’s claim is that these cues are more robust than RGB appearance in the OR because gowns, masks, and caps suppress color and texture variation (Wang et al., 11 Aug 2025).

Framewise detections are denoted

$s_t^m$1

and the target output trajectories are

$s_t^m$2

For online tracking, each detection and each active trajectory is represented by a ReID feature vector and a 3D bounding box. TrackOR computes a cost matrix $s_t^m$3 whose entries combine a shape cost based on the cosine dissimilarity of the ReID features and a spatial cost based on 3D Generalized IoU (GIoU) between 3D boxes. Assignment is solved with the Hungarian algorithm, and any match whose final cost exceeds a threshold $s_t^m$4 is rejected. Matched detections update existing tracks, unmatched detections initialize new tracks, and unmatched trajectories are marked as lost (Wang et al., 11 Aug 2025).

This design is structurally close to a classical linear-assignment tracker, but its identity cue is specific to the OR. The paper explicitly states that TrackOR’s own method relies on 3D positional overlap and ReID geometry, rather than on a Kalman filter or an explicit motion model. The role of 3D GIoU is especially important under ceiling-mounted multi-view sensing, because 2D overlap can be ambiguous when people occlude one another around the operating table, whereas 3D position and extent are more physically meaningful (Wang et al., 11 Aug 2025).

4. Offline global trajectory recovery

TrackOR supplements online association with a separate offline recovery stage designed to repair fragmentation and long-term identity errors. This stage operates at the tracklet level rather than the frame level. For each tracklet, TrackOR stores a sequence of framewise descriptors of shape

$s_t^m$5

where $s_t^m$6 is the tracklet length. These descriptors are aggregated by temporal max-pooling to obtain one descriptor per virtual view. An SVM-Gallery classifier then predicts the tracklet identity, and the paper states that the final identity is assigned by majority vote over the 8 view-specific feature vectors. All tracklets assigned the same identity are then grouped into the complete trajectory of that person (Wang et al., 11 Aug 2025).

The offline stage is intended to repair precisely the long-term errors that matter in surgical workflows: fragmentation caused by occlusion, identity switches introduced during difficult interactions, and failures to reconnect a person who disappears and later re-enters the room. The paper characterizes this as a “straightforward and effective offline global trajectory recovery.” It is not presented as a graph-optimization formulation; instead, it is a tracklet classification-and-grouping procedure that depends on the stability of the geometric identity features (Wang et al., 11 Aug 2025).

This separation between online and offline outputs is operationally important. The online stage provides immediate framewise tracks and tracklets, while the offline stage produces global, merged, analysis-ready trajectories that span all appearances of each individual. A plausible implication is that TrackOR is designed not only for live scene understanding but also for retrospective workflow analysis over whole procedures.

5. Experimental evaluation on MM-OR

TrackOR is evaluated on the MM-OR dataset, which the paper argues is more appropriate for long-term identity tracking than MVOR or 4D-OR. MM-OR contains the challenging conditions emphasized by the method: frequent occlusions, multiple clinicians in tight spaces, homogeneous attire, and realistic OR dynamics. The experiments use all MM-OR takes with segmentation labels, yielding 20 takes total and 23,442 frames, split into 13 takes train (62%), 2 takes validation (16%), and 5 takes test (22%). Each take corresponds to a single surgery (Wang et al., 11 Aug 2025).

All methods are evaluated in 2D image space so that 2D and 3D trackers can be compared fairly. For 3D trackers, 3D poses are projected into each 2D image plane to produce boxes. Reported metrics include HOTA, AssA, DetA, IDF1, IDSW, MOTA, count metrics, and FPS. Because boxes derived from projected 3D poses differ geometrically from silhouette-derived 2D ground truth, the HOTA threshold range is modified to

$s_t^m$7

Baselines include 2D RGB trackers such as OC-Sort, ByteTrack, StrongSort, BoostTrack++, Deep OC-Sort, and BoT-SORT, as well as 3D pose-based trackers such as KSP Tracker, Nearest-Neighbor, and Kalman Filter (Wang et al., 11 Aug 2025).

From Table 1, TrackOR achieves:

• HOTA = 82.216
• AssA = 82.300
• DetA = 83.685
• IDF1 = 76.362
• IDSW = 125
• FPS = 17

The strongest baseline in association is BoT-SORT with AssA = 71.309, so the reported improvement is

$s_t^m$8

percentage points. Other comparisons in the same table include Deep OC-Sort: AssA 66.348, Nearest-Neighbor (3D): AssA 65.813, and Kalman Filter (3D): AssA 62.441. The pattern described by the paper is that 3D-only spatial tracking helps but is not enough, 2D appearance-ReID helps but is limited by OR visual homogeneity, and 3D geometry + ReID gives the strongest identity association (Wang et al., 11 Aug 2025).

The paper also notes a trade-off. Some 2D trackers achieve better detection-oriented metrics because their detectors were fine-tuned directly on ground-truth boxes, whereas TrackOR’s 3D pose detector is trained with the cited self-supervised approach because MM-OR does not have 3D pose annotations. The authors therefore argue that older detection-heavy metrics such as MOTA favor 2D methods, while TrackOR’s main strength lies in maintaining identity correctly over long OR sequences (Wang et al., 11 Aug 2025).

6. Applications, limitations, and subsequent context

TrackOR is positioned as an enabling technology for personalized intelligent operating rooms. Once persistent staff identities are available, the system can support individualized workflow analysis, personalized feedback, safety monitoring at the level of specific staff, longitudinal study of movement strategies, and tailored assistance for the surgical team. The paper’s concrete downstream example is temporal pathway imprints, which are generated by taking a person’s trajectory root positions, projecting them onto the X-Y floor plane, visualizing them from a bird’s-eye view, and adding context such as regions of interest or sterility zones. In Figure 1, pathway imprints for the same robot technician in two surgeries show that the technician’s main workspace was the MPS station in both surgeries, that the technician interacted with the robot twice in Surgery 1 and once in Surgery 2, and that in Surgery 2 the non-sterile technician came into close proximity with the sterile patient table (Wang et al., 11 Aug 2025).

The framework also has clear operational constraints. TrackOR runs at 17 FPS on a single RTX 2080 Ti. It requires ceiling-mounted multi-view RGB-D cameras, calibrated multi-view geometry, 3D point cloud generation, and person segmentation in 3D. Its performance depends on reliable 3D pose estimation, sufficiently accurate point cloud segmentation, and enough geometric completeness in the observed person cloud. The paper also notes that 3D-based methods show lower DetA than the strongest 2D methods in its experiments, and that online tracking can still fragment, which motivates the offline recovery stage. A further practical issue mentioned briefly is that privacy concerns make public OR datasets scarce, especially from ceiling-mounted cameras (Wang et al., 11 Aug 2025).

A later development in the same research area, “Geometry OR Tracker: Universal Geometric Operating Room Tracking”, shifts emphasis from long-term staff re-identification to the geometric reliability of multi-view OR tracking itself. That work argues that shared-frame OR tracking is often limited by bad calibration and RGB-D registration, and reports that a rectification front-end reduces cross-view depth disagreement on MM-OR by more than 30× compared to raw calibration while improving world-frame tracking (Shao et al., 28 Feb 2026). This suggests a broader trajectory in OR tracking research: TrackOR establishes persistent staff-centric identity tracking through 3D geometric signatures and offline recovery, while subsequent work emphasizes that world-scale, metrically consistent OR tracking also depends critically on cross-view geometric consistency.