Intelligent Virtual Sonographer (IVS)

• IVS is a systems paradigm that combines real-time ultrasound perception, decision-making, and conversational interfaces to emulate expert sonographer roles.
• It employs modular architectures with integrated speech-to-text, image analysis, and planning modules for dynamic probe guidance and clinical decision support.
• Recent implementations report high accuracy in anatomical localization and enhanced patient trust, underscoring IVS’s potential to advance robotic ultrasound procedures.

Searching arXiv for recent and directly relevant IVS/robotic ultrasound papers. arXiv.search query: "Intelligent Virtual Sonographer robotic ultrasound IVS" arXiv.search results for "Intelligent Virtual Sonographer robotic ultrasound IVS":

1. (Song et al., 17 Jul 2025) — "Intelligent Virtual Sonographer (IVS): Enhancing Physician-Robot-Patient Communication"
2. (Chen et al., 18 Feb 2025) — "USPilot: An Embodied Robotic Assistant Ultrasound System with LLM Enhanced Graph Planner"
3. (Song et al., 14 Feb 2025) — "Enhancing Patient Acceptance of Robotic Ultrasound through Conversational Virtual Agent and Immersive Visualizations"
4. (Raina et al., 2024) — "Coaching a Robotic Sonographer: Learning Robotic Ultrasound with Sparse Expert's Feedback"
5. (Dimnaku et al., 16 Apr 2025) — "Decision-based AI Visual Navigation for Cardiac Ultrasounds"
6. (Jiang et al., 2023) — "Robotic Ultrasound Imaging: State-of-the-Art and Future Perspectives"
7. (Droste et al., 2019) — "Ultrasound Image Representation Learning by Modeling Sonographer Visual Attention" arXiv.search query: "ultrasound voice commands segmentation detectron2 sonologist" arXiv.search results for "ultrasound voice commands segmentation detectron2 sonologist":
8. (Mohamed et al., 2024) — "Automating Sonologists USG Commands with AI and Voice Interface"
9. (Guo et al., 17 Jan 2026) — "A multitask framework for automated interpretation of multi-frame right upper quadrant ultrasound in clinical decision support"
10. (Jia et al., 2022) — "Development of A Real-time POCUS Image Quality Assessment and Acquisition Guidance System"
11. (Dall'Alba et al., 2024) — "Imitation Learning for Robotic Assisted Ultrasound Examination of Deep Venous Thrombosis using Kernelized Movement Primitives"
12. (A et al., 2023) — "Immersive Virtual Reality Platform for Robot-Assisted Antenatal Ultrasound Scanning" arXiv.search query: "teleultrasound initial probe placement mixed reality ultrasound robot" arXiv.search results for "teleultrasound initial probe placement mixed reality ultrasound robot":
13. (Lee et al., 11 Mar 2026) — "Towards Automated Initial Probe Placement in Transthoracic Teleultrasound Using Human Mesh and Skeleton Recovery"
14. (A et al., 2023) — "Immersive Virtual Reality Platform for Robot-Assisted Antenatal Ultrasound Scanning"
15. (Song et al., 14 Feb 2025) — "Enhancing Patient Acceptance of Robotic Ultrasound through Conversational Virtual Agent and Immersive Visualizations"
16. (Song et al., 17 Jul 2025) — "Intelligent Virtual Sonographer (IVS): Enhancing Physician-Robot-Patient Communication" Intelligent Virtual Sonographer (IVS) denotes a class of ultrasound assistance systems that combine real-time perception, conversational interaction, and robotic or software-mediated action to emulate selected functions of an expert sonographer. In its most explicit recent formulation, IVS is an embodied conversational agent in Extended Reality (XR) that mediates communication among physician, robotic ultrasound system, and patient, executes physician commands, and explains robot actions to the patient (Song et al., 17 Jul 2025). In the broader research landscape, closely related systems implement subsets of the same role: anatomy-aware probe guidance, image-quality-driven navigation, learning from demonstrations, voice-controlled workflow automation, mixed-reality embodiment, and downstream interpretation support (Jiang et al., 2023).

1. Conceptual scope and defining characteristics

The defining feature of IVS is not ultrasound automation in isolation, but the coupling of three functions that are usually distributed across a human sonographer’s workflow: perception of the live ultrasound stream, decision-making about scanning actions, and communication with clinicians or patients. The physician-facing IVS described in XR uses a professional dialogue mode for physicians, an empathetic mode for patients, natural-language robot control, and transparent relaying of robot actions during scanning (Song et al., 17 Jul 2025). This formulation makes IVS a communication bridge rather than merely a probe controller.

A second strand of the literature treats IVS as a “virtual sonographer” in the embodied robotics sense. USPilot is explicitly designed to function as a virtual sonographer that can answer ultrasound-related queries and perform scans based on user intent, using an LLM-based semantic router, ultrasound-specific adapters, and an LLM-enhanced Graph Neural Network planner over robotic APIs (Chen et al., 18 Feb 2025). A related patient-facing line of work combines a conversational virtual agent with augmented reality, augmented virtuality, and fully immersive virtual reality to improve trust and comfort during robotic ultrasound (Song et al., 14 Feb 2025).

A third strand focuses on sonographic competence itself: how to represent what experts attend to, how they rate image quality, how they choose probe trajectories, and how they adapt force and pose. The survey literature frames this as recovery of the “language of sonography,” namely the expert’s implicit reasoning over anatomy, probe pose, force, timing, and image semantics (Jiang et al., 2023). This suggests that IVS is best understood as a systems-level abstraction that can be instantiated as communication middleware, autonomous acquisition logic, or expert-behavior modeling, depending on the application.

2. System architecture across the literature

Across published systems, IVS-like architectures are consistently modular. They typically separate speech and dialogue handling, image understanding, task or motion planning, robot control, and user-facing visualization. In the XR IVS architecture, the spoken interaction pipeline consists of OpenAI Whisper base for speech-to-text, two local Meta Llama 3.1 8B instances for physician-facing and patient-facing dialogue, and Kokoro text-to-speech, with the avatar and interfaces implemented in Unity 2022.3.55 and rendered on a Meta Quest 3 (Song et al., 17 Jul 2025). The robotic ultrasound system includes a KUKA LBR iiwa 14 R820 arm, Siemens ACUSON Juniper ultrasound machine, Siemens 12L3 probe, and an Intel RealSense D435i depth camera.

USPilot adopts a different but structurally similar decomposition. Its semantic router classifies input into question answering or executable robot-control instruction, formalized as

$A\bigvee G^*=\pi(I,G)$

where the policy $\pi$ maps text instruction $I$ and task graph $G$ to either a text answer $A$ or an executable command sequence $G^*$ (Chen et al., 18 Feb 2025). The robot-control branch then invokes an LLM-enhanced GNN over a text-attributed API graph

$G=(V,E,T)$

to select and order ultrasound robot APIs.

The following table summarizes recurring IVS functions across representative systems.

System	Core IVS function	Distinctive mechanism
IVS (Song et al., 17 Jul 2025)	Physician-robot-patient communication	Dual local LLMs in XR
USPilot (Chen et al., 18 Feb 2025)	Intent understanding and robotic task planning	Semantic router + LLM-enhanced GNN
Voice USG system (Mohamed et al., 2024)	Hands-free workflow control	Speech recognition + Mask R-CNN/Detectron2
AI visual navigation (Dimnaku et al., 16 Apr 2025)	Real-time anatomical localization	Decision-gated localization from classifier features
Patient-acceptance system (Song et al., 14 Feb 2025)	Trust and reassurance	Conversational agent + mixed reality

The common architectural pattern is therefore not an end-to-end monolith but a layered stack: language interface, perceptual abstraction, decision or planning module, execution layer, and transparency layer. A plausible implication is that IVS maturity depends less on any single model family than on robust integration across these layers.

3. Perception, image understanding, and modeling of expertise

A central technical problem for IVS is how to encode expert sonographic knowledge when direct labels are scarce or incomplete. One approach models visual attention. In fetal anomaly screening, sonographer gaze was used as self-supervision for representation learning from 403,070 video frames, with saliency prediction yielding an absolute F1 improvement of 9.6% overall and 15.3% for cardiac planes relative to random initialization in standard-plane detection (Droste et al., 2019). The saliency model used a half-width SE-ResNeXt-50 adapted with dilated convolutions, and its transferability suggests that gaze captures clinically meaningful semantic structure beyond explicit plane labels.

Another approach uses direct image understanding for workflow assistance. The voice-controlled ultrasound system employs Mask R-CNN through Detectron2 for organ detection and instance segmentation, with training data from Ramachandra Hospital ultrasound images, Roboflow annotations, and a COCO-format dataset (Mohamed et al., 2024). It reports segmentation confidence between 50% and 95% in the abstract, and 40% to 95% in the results discussion, while the liver histopathology module reports 98% overall accuracy, macro-averaged precision/recall/F1 of 0.97, test accuracy of 97.78%, test loss of 0.0976, and 98.6% fibrosis detection accuracy.

A third direction replaces explicit spatial supervision with decision-based localization. For cardiac ultrasound, a video model is trained offline to answer whether the inferior vena cava is present in a transthoracic echocardiogram video, and localization is activated only when

$p_i>0$

after which feature maps from the 3rd ResNet layer are normalized, interpolated, filtered, and converted into a green 20-pixel radius circle on the image (Dimnaku et al., 16 Apr 2025). This system achieved 97% localization accuracy on 693 hospital videos where the decision model had detected an IVC, and 99.67% zero-shot localization accuracy on 916 Butterfly iQ videos from 91 patient scans.

These examples illustrate different ways of recovering the “language of sonography”: from gaze, from image-quality surrogates, from semantic segmentation, or from classifier internals. The survey literature makes this framing explicit and treats machine learning and artificial intelligence as the enabling techniques for patient- and process-specific, motion- and deformation-aware acquisition (Jiang et al., 2023).

4. Probe guidance, autonomy, and robotic skill acquisition

IVS-like acquisition systems differ most strongly in how they generate scanning actions. One family learns from expert demonstrations. For deep venous thrombosis ultrasound, Kernelized Movement Primitives are used to learn a mapping from scan state to desired force from demonstrations acquired with an ergonomic recording device integrating a Clarius HD3 Linear probe, a WITTENSTEIN/WIKA Resense HEX 12 force sensor, and OptiTrack pose tracking (Dall'Alba et al., 2024). In phantoms, the method reproduced superficial-vessel force with RMSE about 0.64 ± 0.28 N and deep-vessel force with RMSE about 1.49 ± 0.62 N; in compression maneuvers it approximated compression forces around 26.5 N; and in volunteers force RMSE was below 5 N for 4 of 5 subjects.

A second family uses reinforcement learning with human correction. The coaching framework for robotic sonography combines Soft Actor-Critic with sparse expert kinesthetic interventions modeled as a POMDP, augmenting the original image-quality reward

$r_u = \eta (q==q_{max}) + q/q_{max}$

with coaching and trajectory terms (Raina et al., 2024). On urinary bladder phantoms, coaching increased the learning rate by 25% and the number of high-quality image acquisitions by 74.5%; on P1, coached policies reduced position, orientation, and force errors by 31.6%, 26.6%, and 59.0%, respectively.

A third family uses model-based search with expert priors. In autonomous robotic bladder ultrasound, Bayesian Optimization over $(x,y,f_z)$ uses a Gaussian-process quality map whose prior is learned from expert demonstrations, while a ResNet50-based multi-scale bilinear-pooling network estimates image quality from expert-rated bladder images (Raina et al., 2023). On three urinary bladder phantoms, this system achieved 98.73% average probe-position accuracy and force accuracies of 99.28% for P0, 98.25% for P1, and 96.11% for P2.

Classical closed-loop robotic scanning also remains relevant. In peripheral artery phantom scanning, a hierarchical CNN first determines whether a vessel is visible and then regresses the vessel center; the robot maintains 6 N total force, advances 2 mm in the scanning direction when the vessel is visible, and uses a 20-pixel insensitivity margin equivalent to 2.74 mm for lateral recentering (Haxthausen et al., 2020). During 14 cm scans, 100% of saved images contained the complete vessel lumen.

Together, these systems show that IVS autonomy is not a single paradigm. It may be encoded as demonstrated skill, reward-shaped exploration, Bayesian search, or hierarchical perception-control loops. A common property is that image interpretation and motor control are tightly coupled.

5. Communication, XR, voice interfaces, and patient-facing embodiment

A distinguishing expansion of IVS research is the move from probe-centric autonomy to triadic interaction among physician, robot, and patient. The XR IVS system formalizes this with two independent local LLM instances: a professional, control-oriented physician-facing model and an empathetic, explanatory patient-facing model, both based on Meta Llama 3.1 8B (Song et al., 17 Jul 2025). In a user study with 14 participants, it achieved 90.48% accuracy in relaying patient-specific information to the physician, 85.71% accuracy for patient-requested actions, and 92.86% accuracy for physician-requested actions, with 1.09 s total conversational latency per turn.

Patient-facing embodiment has also been studied independently of physician control. A force-compliant robotic ultrasound system augmented with a conversational virtual agent and three mixed-reality modes used a KUKA LBR IIWA R800, ROS Noetic, local Whisper tiny, Phi-3 3B in Unity, and Meta Quest Voice SDK + Wit.AI (Song et al., 14 Feb 2025). Trust scores increased from 3.12 ± 0.62 in the baseline RUS condition to 4.33 ± 0.42 in AR-VG, 4.29 ± 0.38 in AV-VG, and 4.06 ± 0.68 in FV-VG, with a significant Friedman test $\pi$0. AR-VG was also the most preferred mode.

Voice control offers a lower-overhead form of IVS interaction. The ultrasound command-automation system recognizes commands such as “freeze,” “deep freeze,” “continue,” “predict,” and organ-related commands such as “liver,” using the Speech Recognition library and Google Speech-to-Text API (Mohamed et al., 2024). It reports over 90% command recognition accuracy, near-real-time response, and failure modes including background noise, distorted speech, accents, multiple speakers, and rapid commands.

These systems collectively show that IVS is not reducible to perception or control. Communication quality, transparency of robot action, and patient reassurance are treated as first-class technical objectives. This also clarifies a common misconception: an IVS is not necessarily a fully autonomous scanner; it may instead be an interaction layer that coordinates and explains a robotic procedure.

6. Clinical translation, boundary cases, and open problems

The current literature presents IVS as a heterogeneous research program rather than a closed technical standard. Some systems address only initial probe placement. In transthoracic teleultrasound, Patient registration and anatomy-informed Initial Probe placement Guidance uses a Magic Leap 2, RGB-only body reconstruction, SKEL-based bony landmarks, and in-situ mixed-reality overlays to provide initial probe guidance, with overall guided-versus-ground-truth positional error of 24.1 ± 10.6 mm, tilt error of 6.79 ± 3.95°, and spin error of 4.87 ± 2.77° (Lee et al., 11 Mar 2026). This is IVS-inspired initialization, not full autonomous scanning.

Other systems extend in the opposite direction, beyond acquisition into interpretation. A multitask vision-language agent for right upper quadrant ultrasound performs 18-condition classification, report generation, and cholecystectomy decision support on 9,189 Johns Hopkins cases with external validation at Stanford and a Chinese center (Guo et al., 17 Jan 2026). A plausible implication is that mature IVS systems may ultimately span acquisition, reporting, and clinical decision support, not just probe navigation.

The literature is also explicit about limitations. Hallucination remains a failure mode in LLM-mediated communication (Song et al., 17 Jul 2025). Voice interfaces are sensitive to acoustic conditions (Mohamed et al., 2024). Many robotic scanning studies remain phantom-based (Haxthausen et al., 2020, Raina et al., 2024). Generalization across anatomy, devices, and institutions is still uneven, as seen in zero-shot successes for IVC localization (Dimnaku et al., 16 Apr 2025) but cross-site degradation in ultrasound interpretation (Guo et al., 17 Jan 2026). The survey literature therefore emphasizes safety, generalization, transparency, and clinically meaningful image objectives as unresolved prerequisites for broad deployment (Jiang et al., 2023).

Finally, the corpus itself contains bibliographic ambiguity. One listed item, “Intelligent Robotic Sonographer: Mutual Information-based Disentangled Reward Learning from Few Demonstrations” (Jiang et al., 2023), is not usable in the supplied material because the provided document is a SAGE LaTeX template and contains none of the robotic ultrasound content attributed to it. This underscores a practical issue in IVS scholarship: the field is interdisciplinary, terminologically fluid, and currently distributed across robotics, medical imaging, human-computer interaction, and multimodal AI.

Taken together, the available evidence supports a precise characterization of IVS as an emerging ultrasound systems paradigm in which expert-like perception, action selection, and human communication are progressively unified. What remains unsettled is not the existence of the paradigm, but which combination of autonomy, embodiment, interpretability, and clinical authority will define its mature form.