Accurate Google Cloud Simulator (AGOCS)

• AGOCS is a trace-replay simulator that reconstitutes real Google cluster workload events to support realistic scheduling research.
• It synchronizes multi-source traces—including jobs, tasks, and resource usage—ensuring temporal fidelity through controlled replay windows.
• Built in Scala with an event-driven, parallel architecture, AGOCS emphasizes actual resource usage and workload dynamics for comparative studies.

Searching arXiv for the AGOCS paper and closely related simulation frameworks to ground the article in current literature. {"queries":[{"q":"arXiv (Sliwko et al., 30 Sep 2025) AGOCS Accurate Google Cloud Simulator Framework"},{"q":"arXiv (Bawa et al., 1 Jul 2026) CloudyGUI Python-based Framework for Auto-Scaling and Cloud Workload Analysis"},{"q":"arXiv (Andreoli et al., 2024) CloudSim 7G Integrated Toolkit Modeling Simulation Future Generation Cloud Computing Environments"},{"q":"arXiv (Wu et al., 2024) CloudNativeSim toolkit modeling simulation cloud-native applications"},{"q":"arXiv (Khan et al., 2021) PerfSim Performance Simulator for Cloud Native Microservice Chains"}]} I found the target AGOCS paper and the main comparison frameworks on arXiv, including AGOCS (Sliwko et al., 30 Sep 2025), CloudyGUI (Bawa et al., 1 Jul 2026), CloudSim 7G (Andreoli et al., 2024), CloudNativeSim (Wu et al., 2024), and PerfSim (Khan et al., 2021). Accurate Google Cloud Simulator (AGOCS) is a high-fidelity cloud workload simulation framework designed around trace replay rather than synthetic workload generation. It parses Google Cluster Data and reconstitutes a stream of timestamped state changes for jobs, tasks, machines, constraints, and actual resource usage, with the explicit aim of enabling realistic scheduling and load-balancing research on a desktop or laptop-class machine. Implemented in Scala with an event-driven, parallel architecture, AGOCS is best understood as a Google-cluster-trace replay system for workload realism, not as a full reproduction of Google’s internal scheduler logic or a provider-semantic emulator of all Google Cloud services (Sliwko et al., 30 Sep 2025).

1. Definition, scope, and research role

AGOCS was proposed to address two linked problems: the limited realism of many cloud simulators and the practical inaccessibility of production cloud systems for low-level systems research. Its central design choice is to use real workload traces from a Google cluster instead of generating synthetic jobs statistically. The framework therefore targets studies in which realistic task arrivals, machine changes, resource requests, actual resource consumption, and constraint evolution matter more than broad scenario flexibility. The paper explicitly positions it for scheduler and load-balancer research and notes its use in the MASB (Multi-Agent System Balancer) project, where multiple scheduling algorithms consume the same trace-driven workload stream in parallel (Sliwko et al., 30 Sep 2025).

The simulator’s stated scope is narrower than that of a general-purpose cloud modeling environment. It focuses on a Google cloud cell environment derived from Google Cluster Data, with support for jobs and tasks, task lifecycle updates, node additions and removals, resource-capacity changes, task constraints, node attributes, and actual resource usage over time. At the same time, the paper is explicit that AGOCS does not reproduce Google’s internal scheduler decisions directly: SCHEDULE records are treated as outcomes already produced by Google’s proprietary scheduler and are not re-enacted as simulator workload events. This makes AGOCS especially suitable as a workload driver for alternative schedulers rather than as a replay of Google’s control plane in full (Sliwko et al., 30 Sep 2025).

A further motivation is the mismatch between requested and actually used resources. The paper argues that simulation based only on requests can be misleading, and cites prior work indicating that users may waste up to 98% of requested resources. AGOCS therefore elevates actual usage to a first-class simulation input. This suggests that its notion of “accuracy” is primarily trace fidelity at the workload-state level, especially for low-level control problems, rather than broad service-level emulation.

2. Trace corpus and event semantics

AGOCS uses the Google Cluster Data project as its primary source. The paper identifies the input as a 12.5K-node Google cluster observed over one calendar month, specifically May 2011, with the logs stored in Google Cloud Storage bucket clusterdata-2011-2. The trace volume is substantial—approximately 41 GB compressed and approximately 191 GB uncompressed—which directly shaped the implementation strategy: AGOCS does not preload the entire dataset into memory, but instead parses incrementally during simulation (Sliwko et al., 30 Sep 2025).

The extracted information spans three major domains. First, job and task traces provide job submissions, cancellations, changes in job priority, task submission, task scheduling state changes, termination causes, task resource requirements, and task constraint updates. Second, machine traces provide node additions, node removals, changes in total available resources, and node attribute additions and removals. Third, usage traces provide requested and actually used resource statistics, including CPU usage and requested CPU cores, memory usage, assigned memory requested and used, canonical memory used, page cache memory used, local and remote disk space requested and used, disk I/O time used, cycles per instruction, memory accesses per instruction, local scheduler priority class, job and task priority class, and service timeout (Sliwko et al., 30 Sep 2025).

The internal event vocabulary reflects this extraction model. AGOCS defines AddTaskWorkloadEvent, UpdateTaskRequiredResourcesWorkloadEvent, UpdateTaskUsedResourcesWorkloadEvent, UpdateTaskConstraintsWorkloadEvent, RemoveTaskWorkloadEvent, AddNodeWorkloadEvent, UpdateNodeTotalResourcesWorkloadEvent, AddNodeAttributesWorkloadEvent, RemoveNodeAttributesWorkloadEvent, and RemoveNodeWorkloadEvent. Each event is immutable and includes a timestamp: Long, an action, and a source: String. The task-trace mapping is deliberately selective: SUBMIT becomes AddTaskWorkloadEvent; EVICT, FAIL, FINISH, KILL, and LOST become RemoveTaskWorkloadEvent; and UPDATE_PENDING and UPDATE_RUNNING become UpdateTaskRequiredResourcesWorkloadEvent. By contrast, SCHEDULE does not produce an AGOCS workload event because the framework does not attempt to replicate Google’s proprietary scheduling logic (Sliwko et al., 30 Sep 2025).

The trace corpus also contains known anomalies, and the paper treats them as part of the framework’s epistemic boundary. Disk time data is missing after the first 14 days because of monitoring changes. About 0.003% of jobs are absent because they ran on nodes not represented in the traces, about 70 jobs have no task information, about 0.013% of task events and 0.0008% of job events have missing fields, and some resource statistics—especially cycles per instruction and memory accesses per instruction—can be inaccurate or outside expected microarchitectural ranges. In addition, the data is obfuscated, and the traces are shifted by 60,000,000 microseconds, i.e. 10 minutes, so that pre-existing cluster state can be separated from new incoming requests (Sliwko et al., 30 Sep 2025).

3. Architecture and synchronized replay pipeline

AGOCS is architected as an event-driven replay framework in which several independent trace families are transformed into a unified, temporally ordered stream of immutable workload events. The main components are a central WorkloadGenerator, five independent trace parsers implemented as Akka Actors, parser-local event buffers, a shared ContextData state object, and node/task representations that consume the evolving state. The parser set consists of MachineEventsParser, MachineAttributesParser, TaskEventsParser, TaskConstraintsParser, and TaskUsageParser (Sliwko et al., 30 Sep 2025).

The replay logic can be summarized as a multi-source temporal merge. If $P$ denotes the parser set and $E_p$ the event set produced by parser $p$, the unified event pool is

$E = \bigcup_{p \in P} E_p.$

These events are then ordered by timestamp,

$E^\ast = \operatorname{sortByTimestamp}(E),$

and applied in synchronized windows. The WorkloadGenerator collects events every 5 seconds, so the active replay batch for simulation time $t$ is

$$E_t = \{ e \in E^\ast \mid t \le \operatorname{ts}(e) < t + \Delta t \},$$

with $\Delta t = 5$ seconds. This windowed merge is the core mechanism by which AGOCS preserves cross-trace temporal consistency among machine changes, task updates, usage reports, and constraint mutations (Sliwko et al., 30 Sep 2025).

Buffering is used to decouple parsing from replay. Each parser holds up to 30 minutes ahead of simulation time or a hard limit of 1,000,000 events. In formal terms, the paper’s synthesis describes the policy as

$|B_p| \le 10^6$

and

$$\max_{e \in B_p} \operatorname{ts}(e) - t \le 30 \text{ minutes},$$

where $E_p$0 is parser $E_p$1’s buffer and $E_p$2 is current simulation time. If a parser lacks enough buffered events when polled by the WorkloadGenerator, the request blocks until more data is loaded; otherwise buffered events are returned immediately. This design makes replay primarily a synchronization and state-application problem rather than a monolithic batch parse (Sliwko et al., 30 Sep 2025).

The implementation stack is chosen accordingly. AGOCS is written in Scala, uses Akka Actors for parser workers and workload generation, wraps blocking operations in Scala Futures, and stores shared workload state in thread-safe, lock-free hash array mapped tries via Scala’s TrieMap. The paper presents these choices as enabling multicore execution, modular parser separation, and extension to additional trace formats or distributed deployments (Sliwko et al., 30 Sep 2025).

4. Jobs, tasks, nodes, and scheduler-facing state

Although jobs are the conceptual root in Google Cluster Data, AGOCS is operationally task-centric because tasks are the units that consume resources and transition through pending and running states. A task is submitted and becomes pending, may be updated while pending, then begins running when scheduled, may be updated while running, and is eventually removed when it finishes, fails, is killed, is evicted, or is lost. The paper states that once allocated, a running task does not return to pending; when a task is evicted or lost in the original Google system, a clone task is created and re-added to the queue, and AGOCS reflects the state transitions through event generation (Sliwko et al., 30 Sep 2025).

Nodes are modeled as dynamic cluster entities with mutable total capacities, mutable attribute sets, and changing membership in the active cluster. AddNodeWorkloadEvent and RemoveNodeWorkloadEvent control node presence, UpdateNodeTotalResourcesWorkloadEvent updates total resources, and attribute changes are represented by AddNodeAttributesWorkloadEvent and RemoveNodeAttributesWorkloadEvent. The public traces obfuscate the semantics of these attributes, though the paper notes examples such as external IP availability or Linux kernel version (Sliwko et al., 30 Sep 2025).

The decisive modeling choice is the preservation of both required and used resources. Required-resource evolution is carried by UpdateTaskRequiredResourcesWorkloadEvent, while actual consumption is carried by UpdateTaskUsedResourcesWorkloadEvent. This lets AGOCS track fluctuating memory allocation, storage use, and other dynamic resource signals rather than holding each task at a static request envelope. For scheduler research, the consequence is important: the simulator can expose workload states closer to the trace-observed cluster than a request-only abstraction would permit (Sliwko et al., 30 Sep 2025).

AGOCS is intended to feed external schedulers rather than to embody a single policy. The paper describes experiments in which one AGOCS server drives multiple concurrently running schedulers, and reports that in the MASB use case up to five schedulers were run concurrently: Greedy, Tabu Search, Simulated Annealing, and four Genetic Algorithm variations with and without seeding. The framework can be paused so that researchers can inspect the current distribution of tasks and jobs and compare scheduler states during execution. Snapshot restore, however, had not yet been implemented. This suggests a research workflow centered on comparative algorithm analysis under a common, deterministic workload stream rather than on stochastic scenario generation (Sliwko et al., 30 Sep 2025).

5. Fidelity claims, performance characteristics, and limitations

The paper repeatedly characterizes AGOCS as high-fidelity, detailed, and accurate, but the operative meaning of accuracy is trace fidelity rather than formal predictive validation. The supporting argument is architectural: AGOCS replays real-world Google traces; preserves event timestamps; synchronizes multiple trace families; includes node changes, constraints, and attributes; and models actual used resources instead of requests alone. The paper does not provide a formal validation methodology based on held-out traces, error metrics, goodness-of-fit statistics, or confidence intervals (Sliwko et al., 30 Sep 2025).

Performance results are instead emphasized. AGOCS was reported running on a MacBook Pro with a dual-core Intel Core i5 and 8 GB RAM; another comparison section specifies MacBook Pro 11,1, 2.4 GHz dual-core Intel Core i5, 8 GB RAM, Java SE Runtime Environment 1.8.0_60, and OS X 10.11.3. Month-long simulation is reported as taking about 9 hours, at around 75x speed factor in one place and 100x speedup in another, with throughput of about 21.22 GB of data per hour. After an initial data-loading and buffering period of about 30 seconds, the system reportedly runs at about 10–15% CPU usage in one setup. The principal bottleneck is disk I/O rather than computation, and about 89% of the data volume—roughly 170.54 GB—comes from resource-usage logs (Sliwko et al., 30 Sep 2025).

The comparison with CloudSim is revealing. The paper states that CloudSim is faster for small simulations, while AGOCS scales better as workload size increases. CloudSim is described as single-threaded and confined to one CPU core, whereas AGOCS was designed for multithreading and all-core use. AGOCS is presented as more realistic because it supports node addition and removal, actual used-resource values, detailed secondary parameters, task constraints, and node attributes. This comparison positions AGOCS as a realism-oriented complement to more flexible high-level simulators rather than as a universal replacement (Sliwko et al., 30 Sep 2025).

The limitations are substantial and explicitly acknowledged. AGOCS lacks network-utilization values because Google Cluster Data does not provide them. It inherits trace anomalies, obfuscation, and incomplete metrics such as disk-time coverage only for the first 14 days. Replay is deterministic, which improves reproducibility but can encourage overfitting to a single dataset. The simulator does not reproduce Google’s internal scheduler logic, and some graphical-monitor functionality must run in the same environment as the server. Taken together, these constraints imply that AGOCS is strongest when the research question concerns realistic workload-state evolution, task-resource dynamics, and scheduler comparison under trace-grounded pressure, rather than full provider-semantic emulation (Sliwko et al., 30 Sep 2025).

6. Position within cloud simulation research

Within the broader simulator landscape, AGOCS occupies a distinct niche. CloudyGUI is a Python-based framework organized around workload generation, prediction using XGBoost and LSTM, and a MAPE-based predictive auto-scaling loop; its strongest evidence concerns workload-shape fidelity and predictive accuracy, not Google-cloud semantic fidelity (Bawa et al., 1 Jul 2026). CloudSim 7G, by contrast, is a re-engineered and extensible toolkit with standardized interfaces, nested virtualization support, and multi-extension integration, but it is not presented as a provider-specific Google Cloud simulator (Andreoli et al., 2024). CloudNativeSim extends CloudSim toward microservice DAGs, per-service scheduling, scaling, and QoS feedback for cloud-native applications, reporting response-time accuracy above 94.5% in a SockShop-based case study, but it remains a toolkit for cloud-native simulation rather than a trace-replay model of Google cluster operations (Wu et al., 2024). PerfSim offers a trace-driven discrete-event approach to cloud-native microservice chains and reports approximately 81–99% accuracy in average response time against a real Kubernetes cluster, yet its focus is microservice endpoint and network-performance approximation, not replay of Google cluster traces (Khan et al., 2021).

Against that background, AGOCS’s distinctive contribution is not generic extensibility, GUI-driven auto-scaling, or service-graph latency modeling, but synchronized replay of a month-long Google cluster workload with fine-grained task, node, constraint, and used-resource evolution. This suggests a complementary rather than competitive role. For studies centered on alternative scheduling logic under realistic Google-cluster pressure, AGOCS provides a trace-grounded substrate. For broader infrastructure design, cloud-native application graphs, or predictive auto-scaling experiments, frameworks such as CloudSim 7G, CloudNativeSim, PerfSim, and CloudyGUI provide capabilities that AGOCS does not prioritize. The enduring significance of AGOCS lies in demonstrating that production-scale Google workload traces can be replayed with task-level and machine-level detail on commodity hardware, while preserving enough structure to support controlled comparative research on low-level cloud control problems (Sliwko et al., 30 Sep 2025).