DualTrack: Dual-Encoder 6-DOF Ultrasound Tracking

• DualTrack is a dual-encoder architecture for sensorless 3D ultrasound that decouples local speckle motion from global anatomical context to estimate 6-DOF probe poses.
• It employs separate CNN and Transformer pipelines that process detailed local and coarse global features, respectively, and fuses them via cross-attention to mitigate drift.
• Experimental results on the TUS-REC benchmark report a global pose error below 5 mm, outperforming prior methods and demonstrating robust trajectory reconstruction.

Searching arXiv for the DualTrack paper and closely related context. DualTrack is a dual-encoder architecture for sensorless 3D ultrasound that estimates a probe’s 6-D trajectory from a sequence of 2D B-mode images by explicitly separating local motion cues from global anatomical context and then fusing them to predict 6-DOF pose increments. In the sensorless 3D ultrasound setting, only image streams are available, so the system must recover probe motion without external position sensors. DualTrack is motivated by the observation that frame-to-frame speckle decorrelation provides accurate local motion cues but accumulates drift over long sweeps, whereas coarse anatomical structures provide global anchors for plausible large-scale trajectories. On the TUS-REC public benchmark, the method is reported to achieve state-of-the-art accuracy and globally consistent 3D reconstructions, with an average reconstruction error below 5 mm (Wilson et al., 11 Sep 2025).

1. Problem formulation and design rationale

In sensorless 3D ultrasound, the observation sequence is $X_1,\ldots,X_n \in \mathbb{R}^{H\times W}$, and the objective is to recover the 6-D probe pose $p_i \in \mathbb{R}^6$ at each time $i$, comprising 3 translations and 3 Euler angles. DualTrack operates on pose increments rather than absolute poses, producing estimates of $\Delta p_i$ between adjacent frames. This formulation reflects the underlying reconstruction problem: local image similarity constrains short-term relative motion well, while long-horizon trajectory consistency requires additional contextual information (Wilson et al., 11 Sep 2025).

The central motivation is the complementarity of two feature classes. Local features, especially speckle patterns, support frame-to-frame motion estimation; global features, such as bones, vessels, and organ boundaries, situate the scan relative to anatomy and constrain its coarse shape. Prior single-stream networks are described as either emphasizing local 3D-CNN features while ignoring anatomy, or appending an LSTM or Transformer that must simultaneously learn speckles and anatomy. DualTrack addresses this trade-off by decoupling the two tasks into separate encoders and deferring their interaction to a lightweight fusion stage. This suggests that the architecture treats short-range motion estimation and long-range anatomical grounding as distinct inference problems rather than as a single shared representation-learning objective.

2. Dual-encoder architecture

The local encoder receives a contiguous subsequence of $N=16$ full-resolution frames, for example $640\times 512$. Its backbone is a modified 3D-ResNet-18 with a $(2+1)$D stem, consisting of one spatial $7\times 7$ convolution followed by a $3\times 1\times 1$ temporal convolution, followed by residual blocks of $3\times 3\times 3$ convolutions. Temporal downsampling is disabled by setting all temporal strides to 1, while spatial strides of $p_i \in \mathbb{R}^6$0 are used at each ResNet stage, yielding final maps of size $p_i \in \mathbb{R}^6$1 with $p_i \in \mathbb{R}^6$2. The resulting temporal receptive field is stated to be $p_i \in \mathbb{R}^6$3–2 frames in early layers, corresponding to the speckle-correlation window. A per-frame spatial-attention pooling module then maps

$p_i \in \mathbb{R}^6$4

with $p_i \in \mathbb{R}^6$5. During local-only training, a linear head is attached to $p_i \in \mathbb{R}^6$6 to predict the $p_i \in \mathbb{R}^6$7 relative pose increments $p_i \in \mathbb{R}^6$8, and only contiguous subsequences are used so that the model cannot “cheat” with anatomy (Wilson et al., 11 Sep 2025).

The global encoder receives a subsampled stream of $p_i \in \mathbb{R}^6$9 images, for example every $i$0 frame from the sweep, with each frame resized to $i$1. The downsampling and subsampling are intended to destroy fine speckles and force reliance on anatomy. Its image backbone $i$2 may be a 2D-ResNet-18 trained from scratch, a USFM encoder, an iBOT ViT, or MedSAM; all produce per-frame embeddings of dimension $i$3. These embeddings are processed by a Transformer encoder $i$4 with $i$5 layers, hidden size $i$6, mlp-dim $i$7, and $i$8 heads, producing

$i$9

A linear head is attached during global-only training to predict $\Delta p_i$0, and non-contiguous subsequences are used to force reliance on anatomy rather than speckle flow.

Fusion is performed through a Transformer-decoder-style cross-attention module. The local tokens serve as queries,

$\Delta p_i$1

while the global tokens provide keys and values,

$\Delta p_i$2

One cross-attention block followed by a small FFN yields refined tokens $\Delta p_i$3, and a final linear layer maps each $\Delta p_i$4 to a predicted 6-D increment: $\Delta p_i$5

$\Delta p_i$6

$\Delta p_i$7

Architecturally, the defining feature is therefore not merely the presence of multiple branches, but the decoupled specialization of the branches followed by asymmetric fusion in which local motion tokens query a global anatomical memory.

3. Optimization objective and training protocol

All training stages optimize the mean squared error of 6-D pose increments. If $\Delta p_i$8 denotes ground-truth relative poses and $\Delta p_i$9 the predictions, the trajectory loss is

$N=16$0

The same objective is used for local-only pretraining, global-only pretraining, and the final fusion stage. The paper states that no extra regularization terms were needed to achieve SOTA performance, apart from weight decay in CNN pretraining (Wilson et al., 11 Sep 2025).

The training protocol is staged. The local CNN stem and ResNet blocks are pretrained for 4000 epochs with AdamW, learning rate $N=16$1, and weight decay $N=16$2, using one subsequence per scan. The local spatial-attention head is then trained with the CNN frozen for 500 epochs at learning rate $N=16$3 and weight decay $N=16$4. Global pretraining is run for 800 epochs at learning rate $N=16$5 and weight decay $N=16$6. Fusion training uses end-to-end fine-tuning for 500 epochs at learning rate $N=16$7 and weight decay $N=16$8. All phases use a cosine-annealed learning-rate schedule. A plausible implication is that the staged procedure is intended to preserve the intended division of labor between branches before permitting joint refinement.

4. Inputs, modules, and computational profile

Input preprocessing is branch-specific. The local branch uses full $N=16$9 resolution and contiguous windows of length 16. The global branch downsamples frames to $640\times 512$0 and randomly samples non-contiguous $640\times 512$1 frames per sweep, at approximately every $640\times 512$2 frame. This input asymmetry is integral to the method: the local branch retains image detail and temporal continuity, while the global branch is deliberately deprived of fine speckle information so that its embeddings encode anatomy and long-range dependencies rather than short-term optical similarity (Wilson et al., 11 Sep 2025).

The Transformer components are specified explicitly. The global Transformer uses 8 layers, hidden size 512, mlp-dim 1024, and 8 heads. The fusion decoder consists of 1 layer of cross-attention followed by a small FFN. An additional small Transformer is placed between the local encoder and the fusion module for stability, with hidden size 64, mlp-dim 32, and 4 layers and heads.

The reported inference cost on an RTX Quadro 6000 is approximately $640\times 512$3 s to reconstruct a full 546-frame sweep, corresponding to an approximately 27 s scan, with memory consumption below 7 GB. The paper notes that inference remains dominated by the 3D-CNN. This places DualTrack in a regime that is computationally moderate for offline or workstation reconstruction, while suggesting that real-time embedded deployment would require further architectural compression.

5. Experimental evaluation on TUS-REC

The reported evaluation uses the TUS-REC public benchmark, consisting of 1,248 forearm scans from 53 volunteers and 226 GB of data. Scan shapes include linear and wavy trajectories, the average path length is approximately 167 mm, and the median sequence length is 546 frames. The split is 50 subjects, corresponding to 1,176 scans, for train and validation, and 3 subjects, corresponding to 72 scans, for test. Four quantitative metrics are used: GPE, the mean Euclidean error of 5 image corners under the full trajectory; LPE, the same measure between adjacent frames; FDR, defined as $640\times 512$4; and Max Drift in millimeters (Wilson et al., 11 Sep 2025).

Against prior methods, DualTrack is reported to outperform DCL-net (Guo et al.), 2-Frame CNN (Prevost), MoNet (Luo et al.), and Hybrid Transformer (Ning). The quantitative comparison is:

Method	GPE (mm)	FDR (%)
DCL-net (Guo et al.)	10.77	11.68
2-Frame CNN (Prevost)	8.90	8.93
MoNet (Luo et al.)	9.19	8.87
Hybrid Trans. (Ning)	6.01	6.22
DualTrack	4.93	5.10

The remaining reported metrics are likewise favorable: LPE is $640\times 512$5 for DualTrack, compared with $640\times 512$6 for the Hybrid Transformer baseline, and Max Drift is $640\times 512$7 mm, compared with $640\times 512$8 mm for Hybrid Transformer. All improvements over the Hybrid Transformer baseline are reported as significant under a Wilcoxon test with $640\times 512$9. Error-distribution boxplots in Fig. 5 are said to show consistent gains.

The ablation study isolates the role of decoupling. A local-only encoder yields GPE $(2+1)$0 mm, a local-plus-coupled-global configuration yields $(2+1)$1 mm, and the full DualTrack model yields $(2+1)$2 mm.

Configuration	GPE (mm)
Local-only encoder	7.36
Local+coupled global	5.92
DualTrack (full)	4.93

These results support the paper’s central claim that decoupled local and global feature extraction is more effective than a coupled global stream added to a local encoder.

6. Interpretation, limitations, and disambiguation

The reported strengths are tied directly to the architecture. The decoupled design allows each encoder to specialize in speckle versus anatomy; cross-attention fusion integrates the two streams without forcing a single trunk to learn competing tasks; the model reaches global error below 5 mm on a challenging public dataset; and the global branch is flexible with respect to backbone choice, including USFM, iBOT, and MedSAM (Wilson et al., 11 Sep 2025).

The paper also identifies several limitations and future directions. Evaluation is restricted to forearm scans, and performance on highly deformable organs such as the heart and liver remains untested. Inference is still dominated by a 3D-CNN, so lighter 3D architectures are proposed as a path toward lower latency. The pose-increment MSE objective does not explicitly enforce anatomical realism, and future work is suggested to incorporate learned global consistency losses or adversarial priors on scan trajectories. Real-time on-device integration is presented as requiring further compression through quantization or distillation. These points indicate that the current formulation prioritizes geometric trajectory fidelity under the benchmark protocol rather than explicit biomechanical or anatomical plausibility constraints.

A common misconception in this area is that very accurate local frame-to-frame tracking is sufficient for globally accurate reconstruction. DualTrack is explicitly motivated by the opposite observation: local speckle cues are accurate but accumulate drift, while global anatomical context is required to anchor long sweeps. Another possible source of confusion is nomenclature. DualTrack, the ultrasound trajectory-estimation method discussed here, is distinct from DuTrack, a separate system for long-term indoor human tracking using dual-channel Wi-Fi and acoustic sensing (Li et al., 16 Jan 2026).