Model-Mediated Tele-Ultrasound

• Model-mediated tele-ultrasound is a technique that uses patient-specific models to mediate interaction between remote experts and patients, enhancing stability and safety in remote diagnostics.
• It integrates geometric, voxelized, and anatomy-aware models to generate accurate force feedback and delay-independent visual previews for ultrasound imaging.
• Empirical studies show that these systems improve diagnostic accuracy and reduce operator load, supporting both robotic and mixed-reality human teleoperation setups.

Model-mediated tele-ultrasound is teleoperated ultrasound in which a model of the patient, the probe–tissue interaction, the ultrasound appearance, or a combination of these elements mediates the interaction between the remote expert and the patient. Instead of relying on continuous direct reflection of delayed forces and images, the operator interacts with a local representation of the remote environment, while remote measurements refine that representation asynchronously. In contemporary systems, the mediating model may be a geometric surface, a voxelized potential field, a pre-acquired ultrasound volume, an anatomy-aware biomechanical body model, or a mixed-reality registration of predicted probe poses; the follower may be a robot, a mixed-reality-guided human, or a hybrid configuration (Yeung et al., 18 Sep 2025).

1. From direct tele-echography to model mediation

Early robot-based tele-echography established that remote ultrasound examination was clinically feasible under favorable network conditions, but it remained essentially a direct master–slave paradigm with bilateral force feedback rather than an explicitly model-mediated one. The TER system allowed the specialist to move a mock-up of the ultrasound probe at the master site while the robot reproduced the movements of the real probe, which sent back ultrasound images and force feedback; in 58 patients, all aneurysms were detected by both remote and bedside techniques, and the interobserver correlation coefficient was $0.982$ for aortic diameters (0711.4523).

The motivation for model mediation arises from the specific failure modes of long-distance tele-ultrasound. Teleoperated ultrasound can improve diagnostic medical imaging access for remote communities, but accurate force feedback is important because too little force leads to poor coupling and noisy images, while too much is uncomfortable or unsafe. Under large communication delays, direct force reflection becomes difficult to stabilize and less transparent; the same delayed feedback also disrupts ultrasound-guided probe navigation, because the ultrasound image is the primary navigation cue. Model-mediated teleoperation addresses this by sending an environment model to the sonographer, letting the sonographer interact locally with this model at kHz rates, and updating the model asynchronously based on remote measurements (Black et al., 11 Feb 2025).

A further step is visual-haptic model-mediated teleoperation, in which the local model is not only haptic but also visual. In this formulation, a pre-acquired ultrasound sweep is resliced in real time to provide a delay-independent preview image, while the delayed live stream is retained for confirmation and fine tuning. This makes the local model, rather than the network, the dominant source of immediate sensorimotor feedback (Jaeggi et al., 25 May 2026).

2. Architectural patterns and mediating models

Model-mediated tele-ultrasound has diversified into several architectural forms. Some systems remain robotic and place the model in the expert-side control loop; some use a mixed-reality-guided human follower, described as a “flexible, cognitive robot”; others implement shared control in which patient-specific anatomy constrains the robot’s motion. Across these variants, the defining feature is that the remote operator does not depend on a continuously closed bilateral loop for the primary haptic or visual interaction.

Representative system	Mediating model	Function in the loop
"Measurement and Potential Field-Based Patient Modeling for Model-Mediated Tele-ultrasound" (Yeung et al., 18 Sep 2025)	Voxelized potential field from torso point cloud	Local force and torque rendering
"Visual-Haptic Model Mediated Teleoperation for Remote Ultrasound" (Black et al., 11 Feb 2025)	Point-cloud virtual fixture plus pre-acquired ultrasound sweep	Local haptics and local preview image
"Mixed Reality Tele-Ultrasound over 750 km: A Feasibility Study" (Yeung et al., 2024)	Ellipsoid model of the patient	Simple haptic surface and MR spatial context
"An Anatomy-Aware Shared Control Approach for Assisted Teleoperation of Lung Ultrasound Examinations" (Nardi et al., 2024)	SMPL + SKEL torso and rib-derived forbidden regions	Visual feedback and QP-based command filtering
"Towards Automated Initial Probe Placement in Transthoracic Teleultrasound Using Human Mesh and Skeleton Recovery" (Lee et al., 11 Mar 2026)	RGB-derived body mesh and skeleton	Initial probe placement guidance

A generic master–follower structure appears across the literature. On the expert side, the interface is commonly a haptic device such as a Touch X or Haption Desktop 6D, plus displays for ultrasound and context video. On the patient side, the follower may be a Franka Emika Panda or UR3e robot with a tracked ultrasound probe, or a novice human guided by a HoloLens 2 or Magic Leap 2. WebRTC is frequently used for streaming pose commands, ultrasound images, POV video, and occasional model updates. In the mixed-reality human-teleoperation formulation, the expert manipulates a virtual transducer locally, the follower aligns a real probe to a virtual one in 6 DOF, and the expert feels haptics through a model rather than through direct delayed remote force streaming (Black et al., 10 Nov 2025).

This architecture supports a broader interpretation of model-mediated tele-ultrasound than classical bilateral robotics. The same local model can drive a robot follower, a human follower with mixed reality, or a hybrid arrangement. Comparative evidence indicates that, with the same virtual fixture and mesh, human and robotic teleoperation can yield nearly identical image-space tracking accuracy and completion time, while differing substantially in force consistency and practical deployment overhead (Black et al., 10 Nov 2025).

3. Patient and contact modeling

A central research direction is the construction of a patient-specific model that yields accurate forces and torques while remaining stable and computationally tractable. A recent formulation begins with a torso point cloud captured by a TOF camera, structures that surface in cylindrical coordinates, voxelizes the interior, and assigns to each voxel a scalar potential $p(v)$. The ultrasound transducer is modeled as a pointshell; force and torque are then computed as linear projections of the voxel field,

$$\mathbf{f}_t = N_t p,\qquad \boldsymbol{\tau}_t = W_t p,$$

where $N_t$ and $W_t$ are sparse matrices assembled from the pointshell normals and moment arms at the current pose. The field is initialized from Laplace’s equation with Dirichlet boundary conditions and then refined by solving

$$\min_p \; \|L p - b\|^2 + \lambda \|N p - f\|^2 + \lambda \|W p - \tau\|^2,$$

with $\lambda = 10^{-4}$. On volunteer patients $(n=3)$, adding measured forces reduced the force magnitude error by an average of $7.23$ N and the force vector angle error by an average of $9.37^\circ$ compared to using only Laplace’s equation (Yeung et al., 18 Sep 2025).

A much simpler model was used in long-distance mixed-reality tele-ultrasound over 754 km. There, the patient’s abdominal surface was approximated by an axis-aligned ellipsoid,

$p(v)$0

with parameters calibrated from four contact points collected by the follower using the probe’s position and force sensors. The expert-side haptic device then rendered a spring–damper contact force

$p(v)$1

where $p(v)$2 is penetration depth into the ellipsoid and $p(v)$3 is the surface normal. This model did not aim to capture tissue heterogeneity; its role was to provide a stable and relatively accurate haptic representation of the patient on which to move and rest the operator’s hand (Yeung et al., 2024).

A third line of work emphasizes anatomy rather than compliance. In lung ultrasound, a patient-specific SMPL torso model and a fitted SKEL skeleton are used to reconstruct rib geometry and define forbidden regions around ribs. The operator’s desired Cartesian increment $p(v)$4 is filtered through a quadratic program,

$p(v)$5

so that the probe remains in intercostal spaces. Here the model mediates not rendered force, but permissible motion and spatial awareness (Nardi et al., 2024).

Initial placement can also be model-mediated. An RGB-only mixed-reality framework reconstructs SMPL-X and SKEL from $p(v)$6 views, computes a consistent body pose in the headset world frame,

$p(v)$7

then generates anatomy-aware initial probe poses $p(v)$8 for transthoracic windows using bony landmarks and torso normals. Across all postures and views, the overall positional error was $p(v)$9 mm for $$\mathbf{f}_t = N_t p,\qquad \boldsymbol{\tau}_t = W_t p,$$0, $$\mathbf{f}_t = N_t p,\qquad \boldsymbol{\tau}_t = W_t p,$$1 mm for $$\mathbf{f}_t = N_t p,\qquad \boldsymbol{\tau}_t = W_t p,$$2, and $$\mathbf{f}_t = N_t p,\qquad \boldsymbol{\tau}_t = W_t p,$$3 mm for $$\mathbf{f}_t = N_t p,\qquad \boldsymbol{\tau}_t = W_t p,$$4, with tilt and spin errors on the order of $$\mathbf{f}_t = N_t p,\qquad \boldsymbol{\tau}_t = W_t p,$$5–$$\mathbf{f}_t = N_t p,\qquad \boldsymbol{\tau}_t = W_t p,$$6 and $$\mathbf{f}_t = N_t p,\qquad \boldsymbol{\tau}_t = W_t p,$$7–$$\mathbf{f}_t = N_t p,\qquad \boldsymbol{\tau}_t = W_t p,$$8 respectively (Lee et al., 11 Mar 2026).

4. Visual mediation and ultrasound appearance models

Haptic mediation alone does not remove visual delay. In tele-ultrasound, even visual delays as low as approximately $$\mathbf{f}_t = N_t p,\qquad \boldsymbol{\tau}_t = W_t p,$$9 ms disrupt precise manipulation, and delayed ultrasound makes it hard to localize and track targets. Visual-haptic model-mediated teleoperation addresses this by introducing a local visual model: a 3D ultrasound sweep is reconstructed in advance, and the current operator pose defines a slice plane whose real-time reslicing produces a preview image. The robot command is related to the master pose by calibrated rigid transforms,

$N_t$0

so that the preview image remains synchronous with local haptics even when the actual robot and live stream are delayed (Black et al., 11 Feb 2025).

A major limitation of conventional reslicing is that ultrasound is direction-dependent. DARE, or Directionality-Aware Reslicing, reconstructs a 3D voxel grid in which each voxel stores not only intensities but also the acquisition orientations associated with those intensities. At reslice time, it keeps only samples whose acquisition directions are close to the desired reslice direction, rejecting samples with $N_t$1 or $N_t$2, and combining the remainder with orientation-dependent exponential weights. With a voxel size and interpolation radius of $N_t$3 mm, the median reslice time was approximately $N_t$4 ms per 2D image; on 552 human-dataset samples, DARE improved SSIM from median $N_t$5 to $N_t$6 and NCC from median $N_t$7 to $N_t$8, both with $N_t$9, and all clinical experts preferred the $W_t$0 mm setting (Jaeggi et al., 25 May 2026).

Mixed reality extends visual mediation into the patient-side workspace. In long-distance human teleoperation, a HoloLens 2 renders the expert-controlled virtual transducer and the patient ellipsoid so that the follower can align the real and virtual probes in situ. In anatomy-aware lung ultrasound, the remote physician can instead see a VR or 3D model view of the patient-specific torso, ribs, and probe trajectory, which compensates for occlusions in RGB camera views. In both cases, the model replaces raw remote imagery as the dominant source of immediate spatial guidance (Yeung et al., 2024).

5. Comparative evidence and measured performance

Model-mediated tele-ultrasound has now been evaluated in multiple experimental configurations. In the force-augmented voxel-field formulation, training-trajectory errors dropped from up to $W_t$1 N to $W_t$2 N in one sweep example and from $W_t$3 to $W_t$4 in one press example; on new trajectories the augmented model was still generally better than the Laplace-only baseline, although some cases showed slightly increased magnitude error while angle error improved (Yeung et al., 18 Sep 2025).

In visual-haptic robotic tele-ultrasound, 15 volunteers performed vessel-finding and sweeping tasks under 0 ms, 500 ms one-way, and 1000 ms one-way delays. At 500 ms, completion time and effort with visual-haptic model mediation were not significantly different from MMT at 0 ms; compared with MMT alone, visual-haptic MMT significantly improved vessel finding and total completion time at both 500 ms and 1000 ms, reduced mental demand and effort, improved perceived performance, lowered lateral tracking RMSE, and reduced vessel eccentricity variance during sweeps (Black et al., 11 Feb 2025).

A complementary human-follower study over 754 km tested 11 total scans with 10 novices and 2 sonographers. The sonographers were tasked with acquiring 5 target images in the epigastric region, and 92% of the acquired images had sufficient quality for interpretation by both radiologists. The mean raw position RMSE between real and virtual probes was $W_t$5 mm and the mean normalized position RMSE was $W_t$6 mm; the corresponding orientation RMSE values were $W_t$7 and $W_t$8. No statistically significant correlation was found between tracking RMSE and image quality, and the reported task load was below reference values reported in the literature (Yeung et al., 2024).

The choice of follower is itself informative. In a direct comparison between robotic teleoperation and human teleoperation with the same expert-side virtual fixture and mesh, total completion time differed by approximately $W_t$9 and image-space tracking accuracy by approximately $$\min_p \; \|L p - b\|^2 + \lambda \|N p - f\|^2 + \lambda \|W p - \tau\|^2,$$0, neither difference being practically large. The principal divergence was in force consistency: for a branch vessel sweep, eccentricity was $$\min_p \; \|L p - b\|^2 + \lambda \|N p - f\|^2 + \lambda \|W p - \tau\|^2,$$1 for human teleoperation, $$\min_p \; \|L p - b\|^2 + \lambda \|N p - f\|^2 + \lambda \|W p - \tau\|^2,$$2 for robotic teleoperation, and $$\min_p \; \|L p - b\|^2 + \lambda \|N p - f\|^2 + \lambda \|W p - \tau\|^2,$$3 for direct scanning, with all differences statistically significant. Setup time also differed sharply, at approximately 0:34 min for human teleoperation versus 1:53 min for the robot, excluding mounting and calibration (Black et al., 10 Nov 2025).

These newer results should be read against older direct-teleoperation baselines. The TER abdominal aortic study showed that direct bilateral tele-echography could be reliable, acceptable, and effective under an experimental 1 Gb/s link, but the examinations were significantly longer than bedside ultrasound and the system did not address the large-delay regimes that later motivated model mediation (0711.4523).

6. Limitations, misconceptions, and emerging directions

A recurring limitation is that many models remain quasi-static. The force-augmented voxel model neglects dynamic effects and friction, treats tissue as effectively homogeneous and isotropic, and has not yet validated torque experimentally. Accuracy degrades in unseen regions, and the evaluation involved only three volunteers. The authors explicitly note that a real-time implementation was not yet done, although the SPD system and rank-one updates are amenable to recursive online solvers (Yeung et al., 18 Sep 2025).

Visual models have analogous constraints. Pre-acquired ultrasound sweeps and DARE-style volumes assume accurate calibration and quasi-static anatomy during acquisition. Sparse sampling can create holes, tissue deformation is not modeled, and DARE has not yet been integrated into a fully dynamic closed-loop visual-haptic teleoperation experiment. This suggests that current visual mediation is strongest as a delay-independent preview rather than a complete substitute for live ultrasound (Jaeggi et al., 25 May 2026).

A common misconception is that model-mediated tele-ultrasound is equivalent to fully robotic tele-ultrasound. Comparative evidence shows that the same environment model can mediate expert interaction with either a robot follower or a mixed-reality-guided human follower, and that follower type affects force consistency and practical deployment more strongly than image-space accuracy or task duration. A second misconception is that geometry alone is sufficient. The recent literature shows distinct gains from augmenting geometric models with measured forces, torque formulations, directionality-aware ultrasound appearance, and anatomy-aware skeletal priors (Black et al., 10 Nov 2025).

Several adjacent research threads indicate where the field is moving. Learning-based systems have modeled ultrasound scanning skill as a policy over ultrasound images, pose or orientation, and contact force, and have proposed guided exploration to refine that policy locally; this suggests a local autonomous or semi-autonomous layer for future model-mediated tele-ultrasound (Deng et al., 2021). A related multimodal framework learns a task-quality model from ultrasound images, probe pose, and force/torque and uses sampling-based adjustment to guide a newbie sonographer or a robot arm (Deng et al., 2021). At a higher abstraction level, LLM-enhanced graph planning has been proposed as a local task planner for an embodied robotic ultrasound assistant, turning natural-language intent into sequences of validated robot APIs; this is not tele-ultrasound per se, but it fits naturally as a mediating layer between a remote clinician’s intent and local safe execution (Chen et al., 18 Feb 2025).

The field’s current trajectory is therefore toward richer patient-specific models, not merely more communication bandwidth. Concrete directions already identified in the literature include real-time recursive updating of voxel fields, systematic torque validation, explicit modeling of drag and viscoelasticity, deformation-aware ultrasound reconstruction, image-based control that aligns live and preview ultrasound, and broader clinical studies on real patients and real networks (Black et al., 11 Feb 2025). In that sense, model-mediated tele-ultrasound has evolved from a workaround for delayed force reflection into a general framework for integrating geometry, anatomy, contact mechanics, image formation, and shared autonomy into remote ultrasound systems.