Vehicle-in-the-Loop (ViL) Overview
- Vehicle-in-the-Loop (ViL) is a testing modality that integrates physical vehicle actuation with simulated or mixed-reality environments to safely validate autonomous systems.
- It employs synchronized digital twins, closed-loop data flows, and multi-rate sensor integration to bridge the gap between pure simulation and full road trials.
- ViL facilitates evaluation of control strategies, energy optimization, and safety-critical scenarios, providing practical insights into autonomous vehicle performance.
Vehicle-in-the-Loop (ViL) denotes a class of closed-loop testing methods in which a physical vehicle, or a physical vehicle platform, is coupled in real time to a simulated environment, a digital twin, or a mixed-reality world. In the autonomous-driving literature, ViL has been instantiated as a physical test vehicle on a dynamometer synchronized with its virtual counterpart in CARLA, as a self-driving vehicle that receives manipulated camera images containing virtual traffic participants, and as experimental connected vehicles embedded into virtual traffic corridors or microsimulated traffic in real time (Wu et al., 5 Mar 2026, Shen et al., 2023, Ard et al., 2022, Ard et al., 2020). This suggests that ViL is best understood as an intermediate experimental modality between purely virtual simulation and full road testing: it preserves real actuation, real hardware, and, in some configurations, real sensing or real human behavior, while enabling safe, reproducible, realistic, and cost-effective exposure to scenarios that would be difficult, expensive, or unsafe to stage on public roads (Wu et al., 5 Mar 2026, Beck et al., 2024).
1. Conceptual scope and research lineage
ViL emerged in autonomous-driving research as a response to the limited transferability of purely virtual simulation and the practical difficulty of exhaustive field trials. Early ROS-based testbeds emphasized straightforward transition to hardware in the loop testing and execution, with minimal distinction between simulated and real components, and demonstrated that student researchers were able to reproduce their simulation results with fewer than 2 days of interfacing with the physical platform (Bhadani et al., 2018). Subsequent work broadened the concept from laboratory transfer of controllers to physically grounded evaluation of autonomy stacks, V2X applications, eco-driving strategies, and mixed-reality interaction with vulnerable road users (Kavas-Torris et al., 2023, Ard et al., 2022, Kaiser et al., 29 Jul 2025).
Recent formulations make ViL central to software-defined and centralized E/E validation. In one representative framework, the autonomous driving stack runs end-to-end on a Central Car Server (CeCaS-Computer), while the physical vehicle on a powertrain dynamometer is synchronized with its digital twin in CARLA; this removes the need to test individual physical ECUs and their communication protocols separately, and eliminates flashing and intermediate layers during deployment (Wu et al., 5 Mar 2026). Other work places the emphasis on safety-critical scenario injection rather than synchronized vehicle twins: Sim-on-Wheels runs on a self-driving vehicle operating in the physical world, creates virtual traffic participants with risky behaviors, and seamlessly inserts the virtual events into images perceived from the physical world in real-time (Shen et al., 2023).
This body of work suggests that ViL is no longer a single bench architecture. It now includes full-size vehicles on dynamometers, closed-track vehicles embedded into virtual traffic, VR-based driver simulators built around steer-by-wire platforms, scaled physical cars and mobile robots with digital twins, and coupled human–vehicle in-the-loop systems (Weiss et al., 2022, Zhang et al., 3 Jul 2025, Ahmad et al., 17 Jun 2026).
2. Core architectures and closed-loop operation
A recurring architectural pattern is staged escalation from all-virtual testing toward physical closure of the loop. In the centralized E/E framework, the validation pipeline is explicitly organized as Internal Test (SiL), where software runs on the same hardware as the simulation; External Test (HiL), where software runs on an isolated central car server and the environment on separate simulation hardware; and ViL Test, where the software is deployed on the CeCaS-Computer controlling the real vehicle on the dynamometer and synchronized with its digital twin in CARLA (Wu et al., 5 Mar 2026). The CAT Vehicle Testbed expressed the same progression through a ROS abstraction in which sensor data, controllers, and actuators are all ROS nodes, permitting simulated and real components to be swapped with minimal changes (Bhadani et al., 2018).
Closed-loop data flow is the defining technical feature. In one generic formulation, the loop is: sensor data, simulated or real, to controller; controller to actuator commands; actuator commands to a simulated or real vehicle (Bhadani et al., 2018). In mixed-traffic eco-driving experiments, a server layer runs PTV VISSIM and synchronizes the physical ego vehicle with surrounding virtual traffic at 10 Hz, while an autonomous client layer performs localization, high-level planning, and bidirectional communication with the simulation server (Ard et al., 2020). In V2X validation, the traffic and scenario simulator, external management unit, and ego vehicle exchange mobility, positioning, channel, and message information under stringent real-time constraints, with optimization of scanning ranges reducing heavy-traffic delays from approximately 2 s to below 200 ms (Mafakheri et al., 2021).
Real-time synchronization is handled through explicitly managed multi-rate loops. The CARLA-based digital twin framework uses synchronous mode so that the simulator ticks after all computations finish; signals are sent at 20 ms intervals for control and 50 ms for non-critical signals, while physical camera detections arrive asynchronously at 5 FPS and are aligned to simulation ticks (Wu et al., 5 Mar 2026). Other implementations rely on 100 Hz pose updates from motion capture, 90 Hz world-state synchronization in Unreal Engine 5, or 10 ms kinematic updates when a physical vehicle’s CAN signals are used to advance the virtual ego pose (Lambertenghi et al., 26 Sep 2025, Kaiser et al., 29 Jul 2025, Beck et al., 2024). This suggests that ViL is fundamentally a synchronization problem as much as a vehicle-testing problem.
3. Digital twins, sensor strategies, and mixed reality
Digital twin coupling is one of the dominant ViL strategies. In the CeCaS framework, the dynamometer measures yaw, roll, acceleration, and related kinematics and streams them via Profinet-CAN to the simulation computer, so that the digital twin scenario in CARLA mirrors the physical setup; control signals computed by the CeCaS-Computer are sent to the test vehicle, and real vehicle state updates feed back to the digital twin (Wu et al., 5 Mar 2026). A related mixed-reality HIL platform for AgileX Limo robots disables physics for the digital twin in CARLA and continuously “teleports” the twin to the measured pose of the physical robot, with a Roadside Coordinator Unit handling state synchronization and V2X-style coordination (Ahmad et al., 17 Jun 2026).
A second strategy uses perception-space injection rather than only vehicle-state synchronization. Sim-on-Wheels performs scenario authoring, insertion rendering, and evaluation on board a self-driving vehicle, with OpenGL-based rasterization, custom physically based rendering, environmental lighting estimation, shadow mapping, stereo-depth-based occlusion reasoning, and alpha blending. The manipulated image is then fed to either a modular or an end-to-end autonomy stack, so that the full pipeline runs on the actual vehicle and interacts with the physical world, but the safety-critical events it sees are virtual (Shen et al., 2023). A closely related arrangement replaces the physical camera stream with a virtual one: in an Openpilot-based ViL setup, images from CARLA’s virtual camera are directly fed into Openpilot, control commands are transmitted to a real Toyota RAV4 via USB-to-CAN, and physical vehicle kinematics update the virtual ego pose while CARLA’s internal physics is bypassed (Beck et al., 2024).
A third strategy stimulates onboard perception hardware with projected or displayed imagery. In the coupled cyclist–vehicle environment, UE5 renders a virtual scene at 60 Hz onto a screen in front of the vehicle’s physical camera, allowing the automated shuttle bus to react to cyclist gestures through stimulated camera input while the cyclist rides a stationary bicycle in the same synchronized virtual environment (Kaiser et al., 29 Jul 2025). The centralized E/E framework further extends the notion of twin coupling through sensor duality: both virtual sensors and a real Basler Ace 2 PoE camera can be used, and the real camera can perceive both projected simulation environments and actual physical objects (Wu et al., 5 Mar 2026).
These variants support different notions of realism. Some studies prioritize true physical dynamics and hardware latency; others prioritize perceptual realism; still others mix real and synthetic objects, or mirror both the ego vehicle and scenario participants as digital twins. This suggests that “digital twin” in ViL may refer either to a state-synchronized vehicle replica or to a broader virtual world that reproduces the physical scene, the physical objects in it, or both.
4. Relation to adjacent testing modalities
ViL is usually defined in relation to neighboring modalities rather than in isolation. The literature repeatedly distinguishes it from Software-in-the-Loop (SiL), Model-in-the-Loop (MiL), and Hardware-in-the-Loop (HiL), and more recent work adds Mixed-Reality (MR), Autonomous-Vehicle-in-the-Loop (AViL), Cyclist-in-the-Loop (CiL), and Twin-in-the-Loop (TiL) (Wu et al., 5 Mar 2026, Beck et al., 2024, Li et al., 2023).
| Modality | Characteristics in the literature | Relation to ViL |
|---|---|---|
| SiL | Software runs on the same hardware as the simulation; all virtual | Baseline before HiL and ViL (Wu et al., 5 Mar 2026) |
| HiL | Software on isolated hardware or real OBU/RSU/controller with simulated environment | Often precedes road or ViL deployment (Wu et al., 5 Mar 2026, Kavas-Torris et al., 2023) |
| MiL | Vehicle follows a pre-recorded or idealized trajectory; no control feedback and no physical vehicle | Efficient exploration, but no physical dynamics (Beck et al., 2024) |
| ViL | Real actuation with simulated or mixed environment and closed-loop physical response | Intermediate modality between virtual and road testing (Beck et al., 2024) |
| MR | Real actuation with mixed perception, including simulated objects blended into real sensor streams | Improves perceptual realism without compromising safety or control (Lambertenghi et al., 26 Sep 2025) |
| TiL | Digital twin used on-board as a plant replica or controller component in a closed-loop observer/control scheme | Related, but not primarily a test bench modality (Dettù et al., 2022, Riva et al., 2022) |
Two clarifications are recurrent. First, ViL should not be conflated with HiL. HiL may connect isolated control hardware, OBU/RSU units, or ECUs to simulated traffic or infrastructure, whereas ViL closes the loop through an actual vehicle or physical vehicle platform (Kavas-Torris et al., 2023, Mafakheri et al., 2021). Second, ViL should not be conflated with TiL. In TiL, the simulator is run on-board the electronic control unit so as to calculate the nominal control action in real time, or embedded inside a closed-loop observer for state and parameter estimation; the simulator is therefore part of the controller or estimator, not only part of the test environment (Dettù et al., 2022, Dettù et al., 2023).
The literature also introduces more specialized extensions. AViL is advocated for pre-deployment testing of low-speed urban autonomous shuttles, where the full AV control and decision stack is exercised in a synthetic, controllable virtual environment before public deployment (Li et al., 2023). CiL–ViL coupling generalizes the paradigm to bidirectional human–automation interaction with vulnerable road users (Kaiser et al., 29 Jul 2025).
5. Representative applications and quantitative findings
ViL has been used for end-to-end autonomous-driving validation, V2X application testing, multisensory driver emulation, mixed-traffic energy optimization, and reality-gap analysis. In the CeCaS-centered digital-twin framework, experimental case studies include manual drive, adaptive cruise control and lane keeping assist, and emergency brake scenarios in which a real person is detected by the physical camera and a digital twin of that person is spawned in CARLA. In those tests, the reported average lateral error remains below , computed as the minimum distance from the virtual vehicle’s position to CARLA’s map ground-truth waypoints (Wu et al., 5 Mar 2026).
Perception-space ViL has produced quantitative realism measurements. Sim-on-Wheels reports rendering and compositing at 10–23 FPS, PSNR approximately 22 and SSIM approximately 0.83 on image regions with insertions, segmentation mIoU approximately 0.84 between real object and digital twin scenario, action consistency with brake command and trajectory differing by less than 20 cm on average, and an end-to-end latency increase of approximately 18–22% (Shen et al., 2023). Reality-gap analysis across SiL, ViL, MR, and full real-world testing further shows that ViL drastically reduces actuation mismatches relative to SiL, for example reducing throttle speed error from m/s to m/s and waypoint-following deviations from up to 0.38 m to approximately 0.01 m, while still retaining perception limitations because the ADS continues to consume simulated sensor data (Lambertenghi et al., 26 Sep 2025).
ViL has also been used to emulate dynamics that cannot be safely realized with the physical vehicle alone. A VR-based ViL driving simulator built around a four-wheel steer-by-wire vehicle introduces a speed scaling factor , with
so that the real vehicle can remain at 15–20 mph while the driver experiences reference maneuvers at 30 or 60 mph in double lane change and highway weaving experiments. Reported results state that yaw motion is tracked within human perception limits and that lateral acceleration and steering feel are comparable to the same maneuvers driven manually (Weiss et al., 2022).
In connected and automated driving, ViL has been especially prominent for energy and traffic studies. A Pontryagin’s Minimum Principle and MPC strategy in a virtual signalized corridor reports up to 36% fuel savings over a human-modelled driver, up to 26% improvement over automation without connectivity, and up to 22% fuel savings in a human-driven vehicle following connectivity-enabled automated vehicles (Ard et al., 2022). A mixed-traffic anticipative car-following framework reports up to 30% improved energy economy compared to realistically calibrated human driver car-following, with energy measured via calibrated OBD-II readings (Ard et al., 2020). Energy-aware lane planning for a Hyundai IONIQ 5 in a CARLA-connected ViL setup reports motion energy reduced by up to 39% compared to human drivers and total energy usage reduced by 24% (Kim et al., 29 Mar 2025). In interactive lane-change scenarios formulated as a generalized Nash equilibrium problem, ROS-based ViL experiments report average energy savings of 11.2–15.5% and generally improved or unchanged travel time (Bhattacharyya et al., 2024).
Safety and human-interaction studies extend the scope further. A coupled CiL–ViL environment synchronizes an automated shuttle bus and a real human cyclist through a UE5 virtual environment and measures mean latencies of approximately 53 ms for avatar motion, approximately 23 ms for steering angle, approximately 71 ms for lean angle, approximately 18–24 ms for braking, and approximately 300 ms for pedaling power; resulting vehicle trajectories in the test-bench and proving-ground settings are reported to match closely (Kaiser et al., 29 Jul 2025). This suggests that ViL has become a general framework for testing not only control performance but also interactive behavior, embodied perception, and cross-agent response timing.
6. Limitations, controversies, and current directions
The central technical problems recur across implementations: the sim-to-real gap, synchronization of multi-rate heterogeneous data streams, safety, and computation load (Wu et al., 5 Mar 2026). Even when physical dynamics are preserved, ViL does not automatically solve perceptual realism. A multi-modality evaluation of the reality gap concludes that ViL improves actuation realism but retains perception limitations, because simulated camera and LiDAR streams still differ from real sensing; in perception-dominated scenarios, failures in ViL may not transfer to real-world testing, whereas MR more consistently captures both perceptual and behavioral fidelity (Lambertenghi et al., 26 Sep 2025). A plausible implication is that claims of “realism” in ViL must always be interpreted relative to the part of the stack that has been closed physically: actuation, perception, infrastructure communication, human behavior, or some combination of these.
Throughput and cost remain a second structural limitation. ViL yields more trustworthy data about potential AV safety failures than MiL or SiL, but it is expensive and slow; MiL can efficiently scan large environmental and scenario parameter spaces, whereas ViL typically evaluates fewer scenarios under tighter synchronization constraints (Beck et al., 2024). This trade-off has motivated hybrid pipelines in which broad parameter sweeps are performed in MiL or SiL and only selected cases are escalated to ViL or road testing (Beck et al., 2024, Wu et al., 5 Mar 2026).
Current directions attempt to redistribute this realism–efficiency trade-off rather than eliminate it. Scaled physical cars and AI-powered digital twins reduce space and cost while improving fidelity over conventional physics-based DT models (Zhang et al., 3 Jul 2025). Mixed-reality HIL platforms combine physical robots with CARLA, ROS1/ROS2 bridging, V2X-style coordination, and Control Barrier Functions so that online learning and deployment both remain safety-filtered (Ahmad et al., 17 Jun 2026). Centralized E/E “code-to-road” workflows aim to reduce development and integration efforts by moving the complete autonomous driving stack directly onto the target vehicle hardware early in the testing cycle (Wu et al., 5 Mar 2026). Coupled in-the-loop environments extend ViL toward social interaction with VRUs, and reality-gap studies increasingly evaluate ViL together with MR and real-world testing instead of treating it as a terminal validation modality (Kaiser et al., 29 Jul 2025, Lambertenghi et al., 26 Sep 2025).
Taken together, these developments indicate that ViL is not a single definitive test method but a family of physically grounded experimental configurations. Its enduring role is to expose closed-loop behaviors that are invisible in purely virtual testing while deferring some of the risk, cost, and irreproducibility of full road trials. Whether a given ViL system is adequate depends on which part of the autonomy stack must be made physically real, and which remaining gaps are acceptable for the question under study.