---
title: Hybrid gem5-Verilator Co-simulation
url: https://www.emergentmind.com/topics/hybrid-gem5-verilator-co-simulation-flow
type: topic
---

# Hybrid gem5-Verilator Co-simulation

Hybrid gem5-Verilator co-simulation flow denotes a method in which the gem5 system-level simulator and Verilator’s cycle-accurate RTL simulation are integrated, enabling joint execution of realistic workloads and operating systems with actual RTL hardware models, particularly for complex subsystems such as cache-coherent memory hierarchies. This approach merges the high-level architectural modeling capabilities of gem5 with the fidelity of Verilator’s RTL-based device under test (DUT), allowing cross-domain validation and rapid design iteration for modern multi-core system-on-chip (SoC) architectures [2508.03837][1702.00686][2007.03152].

## 1. Principles of Hybrid Co-Simulation

Hybrid co-simulation unifies discrete-event (DE) and cycle-accurate event-driven paradigms. In the state-of-the-art, as outlined by the taxonomy in [1702.00686], a "master" orchestrator synchronizes gem5’s system simulation domain (typically DEVS-driven) with the cycle-stepped RTL simulation produced by Verilator. The communication protocol between modules relies on encapsulating interface events, checkpoints, and data through stable APIs.

Generic hybrid approaches include:
- Treating Verilator as a DE unit and gem5 as a CT simulator with appropriate adaptation.
- Wrapping outputs/inputs via semantic adaptors so that abrupt RTL events synchronize seamlessly with gem5’s architectural state.
- Implementing a commensurate time-stepping strategy ($T_\text{verilator} = T_\text{gem5} + \Delta T$), ensuring simulation time alignment per cycle or macro-step [2007.03152][1702.00686].

## 2. Co-Simulation Workflow and Protocol

The workflow implemented in frameworks like Rhea [2508.03837] involves the following sequence:
- **Initialization**: gem5 instantiates the Verilator-compiled model, initializes input signals, and steps the clock.
- **Synchronization**: At every gem5 clock cycle, the framework invokes Verilator’s eval() function, exchanges packets (e.g., memory accesses), and polls for RTL output signals such as "ack".
- **Scoreboarding and Validation**: Each memory transaction triggers comparisons between gem5’s Ruby memory model and the RTL results for correctness.
- **Queue Management**: Requests are enqueued at both the gem5 and Verilator interface; valid acknowledgments pop these requests and commit results.

System time ($T_\text{co-sim}$) is expressed as:
$$
T_\text{co-sim} = T_\text{gem5} + T_\text{sync}
$$
where $T_\text{sync}$ is the overhead of synchronization and data exchange.

## 3. Performance Metrics and Scalability

Experimental results show that the hybrid co-simulation incurs a simulation overhead, quantified by the slowdown factor $S(n)$:
$$
S(n) = \frac{T_{\text{co-sim}}(n)}{T_{\text{MI}}(n)}
$$
where $n$ denotes the number of cores, $T_\text{co-sim}$ the hybrid simulation time, and $T_\text{MI}$ the time for gem5 Ruby’s MI model. Observed values:
- $S \approx 2.7\times$ for dual-core scenarios
- $S \approx 1.6\times$ for sixteen-core systems

A plausible implication is that synchronization overhead ($T_\text{sync}$) is amortized across higher core counts, improving scalability.

Performance also includes application speedup, compared to MI:
$$
\text{Speedup} = \frac{T_{\text{MI}}}{T_{\text{design}}}
$$
Single-level RTL cache designs achieve intermediate speedups, and two-level designs reduce execution time by up to 43% for sixteen-core workloads [2508.03837].

## 4. Taxonomy of Co-Simulation Methods

According to [1702.00686], co-simulation taxonomies address:
- **Non-Functional Requirements (NFRs)**: Performance, scalability, accuracy, IP protection, parallelism.
- **Simulation Unit Requirements (SRs)**: Time relations, availability, Jacobian/derivative exposure, rollback capability.
- **Framework Requirements (FRs)**: Domain coupling, communication strategy (Jacobi vs. Gauss-Seidel), step sizing.

Hybrid flows demand semantic interface adaptors, accurate time synchronization, and robust handling of algebraic constraints (e.g., via predictor–corrector iterations using
$$
\Delta F_e = -(\partial g/\partial F_e )^{-1} g(F_e(nH))
$$
for iterative coupling variable correction).

## 5. Implementation Challenges and Solutions

Key challenges:
- **Event Ordering & Causality**: Simultaneous events necessitate strict global ordering (DE orchestrator "Select" function); optimistic approaches require rollback capability.
- **Accuracy and Error Control**: Communication step ($H$) must be harmonized with local integrator error and input extrapolation error. Adaptive step sizing may use error estimators (e.g., Richardson extrapolation).
- **Algebraic Loops and Constraints**: Instantaneous bidirectional dependencies demand iterative convergence mechanisms, possibly requiring simulators to expose derivative/Jacobian information.
- **Semantic Adaptation**: Wrappers must synchronize the discrete gem5 events with RTL transitions from Verilator, translating clocked outputs and state changes (possibly via zero-order hold or interpolation).
- **Real-Time and Multi-Rate Operation**: Synchronization for real/simulated hardware must track worst-case execution time (WCET); step sizing may need dynamic adjustment to balance accuracy and computational cost.

Adaptive communication and modular orchestrator design (supporting multiple simulation units) enhance reusability, scalability, and protect intellectual property boundaries [1702.00686].

## 6. Comparative Analysis and Use Cases

Compared to pure gem5 simulation using MI, the flow with Verilator:
- Achieves higher fidelity by simulating the microarchitectural details of real RTL models, making it possible to detect subtle bugs and validate low-level timing/power characteristics.
- Enables full-system execution of realistic benchmarks, including complete operating systems.
- Delivers intermediate performance—slower than MI models but more accurate—providing valuable trade-offs during hardware/software co-design and validation [2508.03837].

Multi-threaded Verilator simulation allows for parallelization of the DUT, further mitigating synchronization overhead.

Use cases in Rhea include rapid design and validation of cache-coherent memory subsystems with MSI protocol, scaling from one to sixteen cores, and benchmarking with standard workloads for quantitative hardware/software co-design analysis.

## 7. Future Directions and Research Opportunities

The state-of-the-art co-simulation taxonomy reveals promising research directions:
- **Generic Semantic Adaptors**: Development of more sophisticated wrappers for bridging different computation models.
- **Adaptive Strategy and Parallel/Distributed Frameworks**: Using parallel co-simulation to balance load across multicore/cluster environments and optimizing communication step size selection dynamically.
- **Robust Coupling and Modularity**: Enhancement of coupling frameworks to handle complex algebraic, functional, and timing dependencies with predictor–corrector schemes and full rollback support.

Further research may focus on scaling hybrid co-simulation to tens or hundreds of cores while retaining manageable overhead and propagating improvements in both accuracy and practical simulation throughput.

---

Hybrid gem5-Verilator co-simulation flow represents a modular, robust methodology for combining high-level system simulation with cycle-accurate RTL validation. Through frameworks such as Rhea, it achieves quantifiable performance and scalability benefits, while addressing intrinsic synchronization and adaptation challenges outlined in the co-simulation taxonomy. These techniques enable informed hardware/software co-design for modern SoCs and embody current research imperatives in modular, stable, and accurate simulator coupling [2508.03837][1702.00686][2007.03152].

Source: https://www.emergentmind.com/topics/hybrid-gem5-verilator-co-simulation-flow