TimingSimpleCPU: In-Order Timed gem5 Model

• TimingSimpleCPU is an in-order, single-issue CPU model that uses timed memory accesses and event-driven simulation to realistically capture memory latency.
• It decouples the CPU core from the memory hierarchy by issuing non-blocking asynchronous requests, effectively modeling queuing, arbitration, and contention.
• Its execution profile is dominated by instruction fetch and Ruby memory system interactions, highlighting key optimization opportunities in cache and scheduling design.

Searching arXiv for the cited paper and closely related gem5 references. TimingSimpleCPU (TS CPU) is a gem5 CPU model defined by an in-order, single-issue execution model that uses timed memory accesses and event-driven simulation to decouple the core from the memory hierarchy. Unlike AtomicSimpleCPU (AS CPU), which completes memory accesses on the caller’s stack via recvAtomic, TS CPU sends non-blocking timing requests, receives responses asynchronously, and models queuing, arbitration, and contention. In the anatomical analysis reported for gem5, this design makes TS CPU more realistic from the perspective of memory latency and coherence, while also concentrating a large share of simulator time inside the Ruby memory system, especially along the instruction fetch path rather than in the CPU core itself (Söderström et al., 25 Aug 2025).

1. Architectural role within gem5

TS CPU is one of the three major CPU models examined alongside AtomicSimpleCPU and the Out-of-order CPU (O3 CPU). Within that triad, its distinguishing property is not speculative or superscalar execution, but timed interaction with the memory system through an event-driven request/response protocol. The model is therefore positioned between the sequential atomic flow of AS CPU and the more pipeline-centric execution structure of O3 CPU.

Its high-level design is organized around three principal top-level processes: a Ruby event wrapper, D-cache tick (D-tick), and I-cache tick (I-tick). It is explicitly in-order and single-issue. Memory interactions are carried by request/response messages with possible NACKs; when a request cannot be accepted, the sender stalls and retries upon a subsequent signal. The event-driven model schedules all activity on the EventQueue, and responses are pushed back to the CPU through Ruby’s Sequencer and PacketQueue. The relevant call-chains are driven by events queued through EventManager::schedule and EventQueue::schedule, with retry points on NACKs and stalls when caches or TLB are busy (Söderström et al., 25 Aug 2025).

2. Event-driven execution semantics

The central semantic feature of TS CPU is that memory operations are not completed on the caller’s stack. Instead, the CPU issues timing requests and later resumes work when Ruby returns a response. This makes the control flow inherently split-phase. The paper identifies several key functions and interfaces that realize this behavior: TS::advanceInst and complete i-fetch drive instruction fetch, execute, and post-execute; fetch, send_fetch_request, and TS::sendFetch generate timing requests; BaseMMU::translateTiming, TLB::translateTiming, and the Walker manage address translation and page walks; RequestPort::sendTimingReq sends packets into Ruby; RubyPort::MemResponsePort::recvTimingReq receives them; and Sequencer, PacketQueue::schedSendTiming, QueuedResponsePort::schedTimingResp, DCachePort::recvTimingResp, completeDataAccess, and instruction CompleteAcc complete the return path.

This organization means that TS CPU’s execution state is governed by callbacks, scheduled events, and explicit waiting states. A plausible implication is that the model’s timing fidelity derives less from elaborate core microarchitecture than from the fact that every instruction fetch and data access becomes a transaction whose latency is exposed through the simulator’s event machinery. That implication is consistent with the reported observation that the dominant costs in TS CPU are attached to fetch-side progression, translation, coherence handling, and response scheduling rather than to decode or execute logic in isolation (Söderström et al., 25 Aug 2025).

3. Instruction fetch path

Instruction fetch is the dominant path in the TS CPU anatomy. I-tick spends almost all of its time in complete i-fetch, and complete i-fetch triggers TS::advanceInst both to fetch the next instruction and, once a Ruby response is ready, to continue with decode, execute, and post-execute. TS::advanceInst first attempts ROM or static-instruction paths (TS::advanceInst-[ROM/S-Inst]), and otherwise enters fetch, which constructs the request and hands it to the TLB.

Address translation begins in BaseMMU::translateTiming, which calls TLB::translateTiming. On a TLB hit, FetchTranslation::finish returns immediately. On a miss, the Walker performs a page walk via Ruby. The study notes that TLB::translate time is dominated by lookup, while Walker invocation is pronounced for certain applications such as bfs_raw in GAPBS and canneal in PARSEC, after which the walker spends the majority of its time waiting on Ruby.

The send path proceeds through send_fetch_request, TS::sendFetch, TimingRequestProtocol::sendReq, and RequestPort::sendTimingReq, which deliver the instruction request to RubyPort::MemResponsePort::recvTimingReq. Ruby’s Sequencer then enqueues the request into the L1 cache controller (L1 CC) mandatory queue. If the line is resident, the L1 CC triggers a fetch hit callback; otherwise the request traverses the network-on-chip through Garnet. On the response path, Sequencer’s hitback call dominates its own time, largely in response-scheduling operations such as PacketQueue::schedSendTiming, QueuedResponsePort::schedTimingResp, PacketQueue::emplace_front, and PacketQueue::schedSendEvent. The callback delivers the instruction to the I-cache’s recvTimingResp, complete i-fetch is scheduled again, and TS::advanceInst resumes decode, execute, and post-execute (Söderström et al., 25 Aug 2025).

4. Data accesses and stall machinery

After the Ruby response for instruction fetch arrives, complete i-fetch performs preExec for decode and then executes the instruction. If the instruction is non-memRef, it executes directly; otherwise initiate Acc is called. The load path and store path then follow the same general timing discipline used for fetch, but through the data side of the memory system.

For loads, init Mem Read delegates to the D-cache; for stores, the path uses writeMemTiming. Address translation again passes through BaseMMU::translateTiming to TLB::translateTiming, and on a miss the Walker fetches the page-table entry via Ruby. finishTranslation marks the request with a physical address, the D-cache sends RequestPort::sendTimingReq, Ruby receives the packet through RubyPort::MemResponsePort::recvTimingReq, the request moves through Sequencer, L1 CC, and potentially Garnet, and DCachePort::recvTimingResp eventually returns the data. completeDataAccess then runs instruction Complete Acc, postExecute commits the instruction, and TS::advanceInst proceeds to the next instruction.

The store path is structurally similar: translation and sendTimingReq are followed by coherence actions after commit, and Ruby’s callback again spends most time in response-packet scheduling. The anatomy also makes stall behavior explicit. If the I-cache, D-cache, or Ruby cannot accept a packet, the CPU sets its status to waiting and retries upon a callback or signal. Page walks stall at the TLB Walker until Ruby delivers the page-table entry, and the D-cache can stall on blocked lines. The event-driven status machine centered on advanceInst and complete i-fetch resolves these stall points. In the measured profiles, most data-side completion time sits in the D-cache access completion path, while the core-side data-completion and D-inst buckets are tiny compared with the fetch-side advInst (Söderström et al., 25 Aug 2025).

5. Ruby interaction and runtime hotspots

The reported study uses the MESI_Two_Level Ruby configuration, that is, a two-level hierarchy with MESI coherence. TS CPU issues instruction fetch and data-access timing requests through MemResponsePort::recvTimingReq, after which the Sequencer converts the requests and enqueues them to the L1 CC mandatory queue. L1 cache-controller state machines then execute coherence operations and either serve hits by scheduling immediate responses or push requests downstream through message buffers, L2, and memory controllers via Garnet.

At the top level, Ruby and Ruby-Infra dominate TS CPU runtime across all applications, and the I-cache fetch path is especially heavy. Inside Ruby, Sequencer hitback callbacks are dominated by response scheduling through PacketQueue::* and QueuedResponsePort::schedTimingResp. L1 CC transitions spend most time in TransWorker (L1State), which executes state tasks. The actions with the highest reported time share are h_ifetch_hit, h_load_hit, hh_store_hit, hhx_store_hit/hx_load_hit, k_popMandatoryQueue, g_issuePUTX, and forward_eviction_to_cpu. Garnet contributes through router wakeup and arbitration (SwitchAllocator::wakeup, arbitrate_inports, arbitrate_outports), input-side wakeup and route computation (InputUnit::wakeup/route_compute, RoutingUnit::outportCompute, NetDest::intersectionIsNotEmpty, flitBuffer::isReady), and network-interface activity (NI::flitisizeMessage, NI::scheduleOutputlink, NI::scheduleOutputPort, Credit::Credit, RouteInfo::RouteInfo, NetDest::NetDest/resize, NI::checkReschedule). No single router or NI function individually dominates; the total Garnet share rises when workloads miss in L1 and require traversal. The study further reports that most instruction fetches hit in L1 through h_ifetch_hit, but even those hits incur substantial Ruby overhead because the timed path still requires scheduling and callbacks (Söderström et al., 25 Aug 2025).

6. Experimental basis and measured execution-time structure

The anatomical results were obtained in gem5 24.0 for the x86_64 ISA in full-system mode with Ruby memory under MESI_Two_Level. The host machine was Ubuntu 22.04.4 with Linux 6.5 on an Intel Core i9-12900K and 128 GB RAM. The benchmark suite comprised GAPBS (bc_raw, bfs_raw, cc_raw, cc_sv_raw, pr_raw, pr_spmv_raw, tc_raw), PARSEC-3.0 (blackscholes, bodytrack, dedup, facesim, ferret, fluidanimate, freqmine, raytrace, streamcluster, swaptions, vips), and SPEC2017 (bwaves, cactuBSSN, exchange2, gcc, imagick, lbm, leela, mcf, nab, omnetpp, perlbench, x264, xalancbmk). GAPBS and PARSEC were run at 1, 4, and 16 cores; SPEC used 1 core. Memory sizes were GAPBS (3/8 GB), PARSEC (3/16 GB), and SPEC (3 GB). PARSEC and GAPBS entered ROI at the parallel phase, while SPEC used simpoint checkpoints.

Profiling used a lightweight framework built on perf_event_open, sampling every 1000 ms, collecting call chains bottom-up, and storing them in JSON. A parser with whitelist and blacklist support produced stacked bars focused on relevant components. Builds used fast with debug symbols enabled to preserve function names. The dataset contained 458 JSON files, each run for 1 hour with an equal sampling window across configurations, and each run was isolated in a dedicated cgroup. The reported results therefore describe relative time shares rather than absolute completion time.

Within that empirical basis, the top-level and flattened views show Ruby as the largest component of TS CPU runtime across GAPBS, PARSEC, and SPEC, and confirm that Ruby plus Ruby-Infra dominate. I-tick is largely complete i-fetch, and advInst is the dominant subcomponent when executing an instruction. Fine-grained fetch analysis highlights the send chain TS::sendFetch → TimingRequestProtocol::sendReq → RequestPort::sendTimingReq → MemResponsePort::recvTimingReq together with translation through BaseMMU::translateTiming and FetchTranslation::finish. TLB Lookup dominates translateTiming, and the Walker dominates its own time by waiting on Ruby. The Garnet-versus-L1 split is application-dependent: all GAPBS applications and canneal and dedup in PARSEC spend proportionally more time in Garnet’s Router and NI, whereas blackscholes, bodytrack, ferret, fluidanimate, freqmine, raytrace, swaptions, and vips in PARSEC, together with several SPEC applications, are L1-hit-heavy (Söderström et al., 25 Aug 2025).

7. Comparative interpretation, optimization implications, and caveats

The study places TS CPU between AS CPU and O3 CPU in both mechanism and hotspot structure.

CPU model	Runtime concentration	Distinctive emphasis
AS CPU	Ruby is also the largest share of runtime	Atomic flow still invokes Ruby heavily
TS CPU	Ruby and instruction fetch path dominate	Timed queuing and scheduling overheads
O3 CPU	More balanced between Ruby and core time	buildInst and pipeline stages dominate

In AS CPU, Ruby is likewise the largest share of runtime and the fetch stage dominates even without pipeline stalls, but TS CPU adds timed queuing and scheduling overheads through components such as Sequencer and PacketQueue. In O3 CPU, the largest costs shift toward constructing dynamic instructions and pipeline stages such as Fetch, Decode, Rename, IEW, and Commit; IEW-exec-insts typically dominates, and Ruby’s share is comparatively smaller and more balanced with core time.

The immediate explanation for TS CPU’s Ruby-dominant profile is that it issues a timing request for virtually every instruction fetch and data access, including accesses that ultimately hit in L1 I-cache or D-cache. Each request carries translation cost, possible page-walk cost, Sequencer enqueue and dequeue activity, L1 CC state-machine processing in TransWorker, response scheduling through PacketQueue and QueuedResponsePort, and, on misses, Garnet routing and network-interface work. The measured execution path inside complete i-fetch is therefore dominated by advInst, which repeatedly returns to fetch, while decode, execute, and post-execute remain comparatively small.

The paper also outlines practical optimization opportunities for TS CPU users. It recommends reducing instruction fetch misses and Ruby traversal by increasing L1 I-cache capacity, associativity, or line size; minimizing TLB page walks through guest configurations that reduce TLB misses; cutting response-scheduling overhead by reducing the number of Ruby callbacks; managing Garnet load by reducing miss rates or concurrent-request volume; warming up with AS CPU and switching to TS CPU at ROI checkpoints; keeping core count modest when workloads show increased I-fetch cost under multi-core configurations; avoiding unnecessary large simulated memory sizes; and using optimized binaries while avoiding debug or tracing unless needed. These recommendations are derived from the observed hotspots rather than from an experimental sweep of cache sizes, associativities, prefetchers, or page sizes.

Several caveats delimit the interpretation. The study reports relative distributions over fixed 1-hour samples rather than total simulation time to completion, and it does not provide exact percentage values from the figures. Garnet shows no single killer function; rather, arbitration, route computation, credit handling, and flit buffering contribute together. Effects of memory size and core count are application-dependent in TS CPU. This suggests that TS CPU should be understood less as a simplified core model and more as an event-driven timing interface whose performance profile is fundamentally shaped by how often execution traverses Ruby, how deep those traversals are, and how much of the resulting work remains in L1 CC versus propagates into Garnet and page-walk traffic (Söderström et al., 25 Aug 2025).