Software-Assisted Dynamic LLC Resizing

• Software-assisted dynamic LLC resizing is a technique that automatically adjusts last-level cache capacity using adaptive, software-driven policies based on workload characteristics.
• It leverages hardware features such as Intel CAT and performance counters to partition cache ways among high-priority and I/O-intensive tasks for improved efficiency.
• Evaluations show significant performance gains, with up to 51% improved throughput for high-priority workloads and 60% reduced latency in I/O-heavy environments.

Software-assisted dynamic LLC resizing is the class of mechanisms and policies in which last-level cache (LLC) capacity is automatically and adaptively partitioned or resized at runtime via software or firmware, based on workload characteristics or I/O pressure. These frameworks exploit architectural features such as hardware cache partitioning (e.g., Intel Cache Allocation Technology, CAT) and performance counters, as well as algorithmic advances in dynamic policy design, to continuously tune the available LLC capacity across tenants, devices, or cache objects. The goal is to optimize key metrics—such as hit ratio, throughput, and latency—for mixed CPU-I/O and multi-tenant workloads in datacenter and high-performance computing environments (Berend et al., 26 Nov 2025, Park et al., 12 Jun 2025, Yuan et al., 2020).

1. LLC Architectural Background and Software Interfaces

Modern multi-core CPUs provision a shared LLC (e.g., 11-way set-associative, 25 MiB/socket on Intel Xeon Scalable) which is dynamically accessible by all cores, and often by direct I/O through Direct Cache Access (DCA) or Data Direct I/O (DDIO). Hardware mechanisms such as Intel's CAT expose the ability to partition LLC ways among "Classes of Service" (COS) identified by mask bits in model-specific registers (MSRs). The direct mapping between cores or threads and their associated COS defines which LLC ways their memory traffic can access (Park et al., 12 Jun 2025).

Device-driven cache access, enabled by DDIO/DCA, allows certain PCIe devices to DMA-write directly into LLC-resident data, impacting both I/O performance and cache contention. On a fine timescale, CAT and per-device DDIO control registers (such as perfctrlsts_0 on Skylake SP) enable dynamic software adjustment of LLC way allocations and DCA enable/disable status per device, offering the basis for software-assisted, dynamic resizing (Park et al., 12 Jun 2025, Yuan et al., 2020).

2. Dynamic LLC Resizing Policies and Algorithms

The core of software-assisted dynamic LLC resizing is the periodic control-loop that reads system metrics and reallocates cache ways among priorities, thread classes, or access patterns:

1. Priority-based resizing: Policies such as A4 (Park et al., 12 Jun 2025) assign LLC ways first to high-priority workloads and then opportunistically allocate spare capacity to low-priority workloads, under the constraint that the aggregate LLC hit rate for high-priority workloads does not degrade by more than a user-defined threshold (e.g., 20%). Each second, the framework can shrink the high-priority region by one way if measured $\Delta$LLC hit rate drop is below the threshold.
2. I/O-aware partitioning: In the presence of DDIO/DCA, certain LLC ways are reserved exclusively for high-bandwidth network devices, with selective DCA disabling on storage I/O devices that do not benefit from cache residency but can starve latency-sensitive traffic. Policies detect contention using DCA and LLC miss counters, and adjust both LLC partitioning and DCA device enablement (Park et al., 12 Jun 2025, Yuan et al., 2020).
3. Antagonist thread isolation: Cache-unfriendly threads (those with both high mid-level cache and LLC miss rates) are dynamically mapped to a minimal LLC region ("trash" ways), determined by sustained miss rates above a configured threshold (e.g., 90%), to avoid polluting shared LLC capacity assigned to more cache-sensitive workloads.
4. Dynamic cache size in replacement policies: At the replacement-policy algorithm level, frameworks like DynamicAdaptiveClimb (Berend et al., 26 Nov 2025) internally resize the effective cache (i.e., software-allocated region or partition) itself using counters (e.g., "jump" and "jump$'$") that track recent hit/miss balance. Cache size is doubled if miss pressure is sustained (jump(t) ≥ 2K), and halved if the working set shrinks (jump(t) ≤ –K/2 and excess top-half hits), with all counters summarizing working-set phase changes.

3. Implementation Details and Monitoring Infrastructure

Dynamic LLC resizing systems leverage several hardware and system software hooks:

• LLC partition reconfiguration: Setting the way-masks using Intel's CAT via MSR writes (e.g., IA32_L3_CBM_*), with operation times ≈2 μs per call and negligible overhead per resize operation (Park et al., 12 Jun 2025).
• DCA/DDIO toggling: Enabling/disabling per-device DCA by setting control bits (e.g., NoSnoopOpWrEn) in device's PCIe configuration space.
• Performance monitoring: Polling per-core and per-device counters each second (LLC hit/miss, DCA miss, PCIe write throughput, memory bandwidth) using Intel PCM/uncore CHA PMUs and system libraries.
• Low overhead: Sampling, logic evaluation, and MSR/config writes are order-of-magnitude sub-millisecond per evaluation, allowing even per-second resizing to consume less than 0.1% of CPU cycles (Park et al., 12 Jun 2025, Berend et al., 26 Nov 2025, Yuan et al., 2020).

Table: Key Software/Hardware Hooks Used in Dynamic LLC Resizing

Function	Interface / Mechanism	Overhead per Action
LLC way partition	Intel CAT (IA32_L3_CBM_* MSRs)	~2 μs
DCA toggling	PCIe config write (perfctrlsts_0 etc.)	5–10 μs
Perf Monitoring	Intel PCM, CHA PMUs, /dev/msr	~800 μs per sample loop

4. Performance Impact and Design Trade-offs

Quantitative evaluation demonstrates significant gains in both application throughput and latency, especially for high-priority and network-I/O intensive workloads:

• A4 framework (Park et al., 12 Jun 2025): In HPW-heavy mixes, dynamic resizing improves high-priority workload throughput by 51% (LLC hit-rate to 95%) compared to baseline, with low-priority workload performance within ±3% of default. Network flow latency is reduced by up to 60% at large I/O block sizes, and antagonistic (cache-unfriendly) cores see their impact minimized.
• IOCA (Yuan et al., 2020): By dynamically adjusting DDIO LLC way allocation, memory bandwidth is reduced up to 15.6% under high I/O rates, and core IPC is increased by up to 11.4%. Fine-grained way-shuffling minimizes performance interference in multi-tenant settings.
• DynamicAdaptiveClimb (Berend et al., 26 Nov 2025): In trace-driven simulation, dynamic cache resizing yields up to 29% miss-ratio reduction and consistent throughput improvements across six real-world datasets, compared to LRU and static-size adaptive policies.

Design trade-offs include sensitivity to the aggressiveness parameter (e.g., ε in DynamicAdaptiveClimb), control interval (longer intervals may miss short-term phase changes), and hardware platform constraints (e.g., number of COS masks, DCA support granularity).

5. Algorithmic and System Integration Considerations

Robust integration in production requires:

• Partitioning Safety: Hard limits on minimum ways per class to meet hardware and vendor safety constraints.
• Phase-change handling: Resizing is reverted or stabilized during workload churn or abrupt phase transitions, with optional fallback to static partitioning for correctness.
• Top-down and bottom-up partitioning: Both application or OS-driven partitioning and cache-access policy-driven resizing are possible; in practice, these are often combined (DynamicAdaptiveClimb triggers CAT partition change on upsize/downsize).
• Cooperation with scheduling/orchestration: Tenants and priorities are typically communicated from orchestrators, and device hints (networking vs. storage) are needed at agent startup for optimal policy decisions (Berend et al., 26 Nov 2025, Park et al., 12 Jun 2025).

6. Applications and Broader Implications

Software-assisted dynamic LLC resizing is applicable across multiple domains:

• Cloud multi-tenancy: Fine-grained way partitioning enables performance isolation and quality-of-service for mixed-priority and I/O/compute-intensive tenant mixes (Berend et al., 26 Nov 2025, Park et al., 12 Jun 2025, Yuan et al., 2020).
• Datacenter I/O optimization: Selective DCA enablement maximizes network acceleration while preventing cache pollution by high-throughput storage devices (Park et al., 12 Jun 2025).
• Phase-adaptive caching: Replacement policy-level resizing (DynamicAdaptiveClimb) deals with fluctuating working sets in high-variability request traces, lowering miss penalties under dynamic load (Berend et al., 26 Nov 2025).
• Autonomous digital control: RL-based policies applied in LLC converter optimization demonstrate the potential to extend similar frameworks to hardware-level parameter tuning in embedded power and digital control settings (Kruse et al., 2023).

A plausible implication is that dynamic LLC resizing frameworks—integrating hardware hooks, counters, and adaptive software policies—are becoming foundational to resource management in both CPU-centric and I/O-dominated computing environments, with performance and efficiency gains unattainable by static partitioning or core-centric allocation alone.