Vulcan: Instance-Optimal Systems Heuristics Through LLM-Driven Search

Abstract: Resource-management tasks in modern operating and distributed systems continue to rely primarily on hand-designed heuristics for tasks such as scheduling, caching, or active queue management. Designing performant heuristics is an expensive, time-consuming process that we are forced to continuously go through due to the constant flux of hardware, workloads and environments. We propose a new alternative: synthesizing instance-optimal heuristics -- specialized for the exact workloads and hardware where they will be deployed -- using code-generating LLMs. To make this synthesis tractable, Vulcan separates policy and mechanism through LLM-friendly, task-agnostic interfaces. With these interfaces, users specify the inputs and objectives of their desired policy, while Vulcan searches for performant policies via evolutionary search over LLM-generated code. This interface is expressive enough to capture a wide range of system policies, yet sufficiently constrained to allow even small, inexpensive LLMs to generate correct and executable code. We use Vulcan to synthesize performant heuristics for cache eviction and memory tiering, and find that these heuristics outperform all human-designed state-of-the-art algorithms by upto 69% and 7.9% in performance for each of these tasks respectively.

• The paper introduces an automated LLM-driven search that synthesizes instance-optimal systems heuristics personalized to workload and hardware conditions.
• It details a three-phase pipeline that partitions system policy design into user-defined mechanisms, LLM-synthesized scoring, and automated policy assembly.
• Empirical case studies show significant performance gains, with up to 69% increase in cache hit rates and measurable improvements in memory tiering over state-of-the-art baselines.

Vulcan: Synthesizing Instance-Optimal Systems Heuristics via LLM-Driven Search

Motivation and Background

The design of resource-management heuristics for systems tasks—such as cache eviction, memory tiering, congestion control, and scheduling—remains predominantly manual and labor-intensive. Despite decades of research, the optimality landscape for these heuristics is fundamentally instance-dependent: workload characteristics, hardware changes, and evolving system objectives continuously affect which heuristic performs best. Empirical evaluations, such as the extensive analysis of 17 caching algorithms on 106 CloudPhysics traces, demonstrate the absence of a universally superior heuristic even within homogeneous workload classes (Figure 1).

Figure 1: Count of CloudPhysics traces where each heuristic performs best (highest object hit rate), showing no single policy dominates across traces or cache sizes.

To address the challenge of rapidly generating heuristics optimized for each workload-hardware setting, Vulcan recasts heuristic design as an automated search problem, leveraging code-generating LLMs and evolutionary search. The goal is to close the gap between theoretical “instance-optimal” policies—best performing for a specific deployment—and practical, deployable policies that match or surpass human designs.

Vulcan Framework

Vulcan introduces a highly structured, three-phase pipeline that separates policy logic from mechanism and enables efficient automated search in a manner amenable to current LLM capabilities.

System Design and Division of Labor

Vulcan’s architecture divides responsibilities as follows (Figure 2):

• Users supply lightweight mechanisms for data collection and action (green).
• LLMs generate scoring functions that capture core policy logic (yellow).
• Vulcan assembles LLM outputs, via templates, into executable, interchangeable policy modules (blue).

  Figure 2: End-to-end overview of Vulcan, in which inputs specified by the user in Phase I are consumed by subsequent phases for policy synthesis, evaluation, and deployment.

  

  Figure 3: Users define data collection and action logic; LLMs synthesize stateless decision functions; Vulcan orchestrates code generation, validation, and injection into system scaffolding.

Narrow Interfaces: Value and Rank Abstractions

Central to Vulcan is the abstraction of systems heuristics into two minimal, LLM-friendly interfaces:

• Value interface: Heuristics are single stateless functions mapping system state to a value, e.g., congestion window in TCP, CPU frequency, or scaling level.
• Rank interface: Heuristics score and select among a set of candidate objects, e.g., cache objects for eviction, pages for migration.

By enforcing this interface separation, Vulcan shrinks the LLM search space to the synthesis of a single numerical function, greatly reducing catastrophic failure modes and validation complexity.

Instance Definition and Automated Instance Generation

Vulcan enables automatic specialization of heuristics to narrow “instances”—defined as specific workload/hardware/parameter combinations—by coupling to instance classifiers and clustering frameworks (Figure 4).

Figure 4: Automated instance generation and runtime policy selection pipeline, where workload traces are clustered, classifiers route to known policies, and new instances can trigger offline search as needed.

Policy Search via Evolutionary LLM-Driven Loop

Vulcan implements a population-based evolutionary algorithm (Figure 5): LLMs are prompted with natural-language templates combining task specifications, input features, performance objectives, and prior high-performing policies. New candidate policies are generated, compiled, evaluated in simulators or testbeds, and scored. Over multiple generations, the search converges to high-performing heuristics.

Figure 5: The evolutionary search process—user defines task, Vulcan generates policy candidates via LLM, evaluates via custom harness, and maintains a heuristic database to guide future explorations.

Case Studies

Cache Eviction: Specialized Rank-Based Heuristics

For the cache eviction task, Vulcan’s Rank interface is instantiated by supplying per-object and global cache state features (e.g., frequency, recency, age, object size). The search process specializes heuristics to clusters of workload traces; for each cluster, a dedicated ranking function is synthesized and integrated via a priority queue mechanism.

The result is a consistent pattern of Vulcan-discovered heuristics outperforming or matching the strongest manual baselines across all CloudPhysics clusters (Figure 6). Notably, in certain instances, the synthesized policies produce up to 69% higher object hit rates compared to the best baseline.

Figure 6: Performance comparison of Vulcan-generated cache eviction heuristics against state-of-the-art algorithms, per workload cluster.

Efficiency-Oriented Queue Topology Search

Vulcan further enables the exploration of queue-topology design spaces—specializing the structure and transition rules of multi-queue caches using only constant-time operations. The evolutionary search composes initial placement and queue-to-queue transition functions, generating new topologies that outperform all tested baselines in specialized object-based regimes (Figures 9, 10).

Figure 7: Visualization of the queue topology discovered for cluster $C_7$, with capacity partitioned across three FIFOs and specialized transition logic.

Figure 8: Object hit rate of Vulcan-synthesized queue topology policies vs. queue-based baselines (TwoQ, ARC, S3-FIFO, etc.) on clusters $C_7$ and $C_8$.

Memory Tiering: Instance-Specialized Page Promotion

Modern tiered memory systems benefit from policies that adapt to both application behavior and hardware tier characteristics. Vulcan expresses these policies via the Rank interface, providing rich per-page histories and global bandwidth/latency signals to the search space.

Across a set of real-world workloads (GUPS, GapBS, Silo), the Vulcan search synthesizes policies achieving 2.5–7.9% improvement over state-of-the-art (ARMS) for performance-critical objectives, often converging within tens of search iterations (Figure 9). The system identifies nuanced policy structures—such as bandwidth-aware penalties and burstiness-adaptive migration—that extend beyond manually tuned baselines.

Figure 9: Normalized application performance improvement of Vulcan-generated memory tiering policies over the vanilla ARMS baseline for various workloads.

Implications

Practical Impact

By reducing the labor and expertise barrier to specialization, Vulcan renders instance-optimal heuristic design tractable even for frequently shifting, fine-grained system conditions. The ability to synthesize high-performing, interpretable code—without neural inference overhead or opaque model structure—makes Vulcan-synthesized policies deployable and auditable in stringent production environments.

Theoretical Considerations

Vulcan’s abstraction-driven approach demonstrates that the most productive avenue for ML-for-systems is not end-to-end black-box modeling, but rather the imposition of narrow, mechanism-agnostic interfaces that align the synthesis search space with LLM competence. This structurally reduces both validation costs and runtime safety risks. The evolutionary search paradigm, when confined to code synthesis for such interfaces, outperforms prior neural and incremental-algorithm-modification approaches for a wide variety of systems control tasks.

Future Directions

Potential expansions for Vulcan include:

• Generalization across new systems domains (e.g., I/O scheduling, power management).
• Tighter integration of instance classification and real-time monitoring for runtime policy switching.
• Incorporation of automated policy verification and constraint enforcement.
• Adaptation to emerging workload/hardware heterogeneity via continual or online search.
• Extension to multi-objective and even multi-agent systems settings.

Conclusion

Vulcan establishes that LLM-driven, interface-constrained code synthesis can produce instance-specialized heuristics that not only match but frequently outperform human-expert designs in systems control tasks. By focusing on structured search in mechanism-agnostic spaces and narrowing the role of ML to stateless decision logic synthesis, Vulcan delivers practical, interpretable, and efficient policies. This framework highlights an inflection point for automated systems design: principled abstractions and evolutionary LLM-centric pipelines will underpin the next era of high-performance, adaptive systems.

---

Reference: "Vulcan: Instance-Optimal Systems Heuristics Through LLM-Driven Search" (2512.25065)