- The paper introduces a dynamic, locality-aware scheduling framework that steers threads to 'Preferred Cores' to preserve microarchitectural state.
- It employs demand prediction and topology-aware allocation, achieving up to 12% throughput gains and significantly reducing cache miss rates.
- The study challenges traditional OS scheduling by integrating soft affinity hints with work conservation for scalable and efficient datacenter performance.
Affinity Tailor: Dynamic Locality-Aware Scheduling at Scale
Motivation and Problem Statement
Rapid growth in datacenter core counts and aggressive hardware overcommitment have produced environments where operating system schedulers routinely multiplex hundreds of latency-sensitive and throughput-oriented workloads across massive, shared multicore topologies. Traditional work-conserving schedulers (e.g., Linux's CFS) prioritize immediate CPU utilization, frequently migrating threads across cores to prevent idleness. This practice, while maximizing theoretical CPU efficiency, is increasingly mismatched to contemporary server hardware and the economics of warehouse-scale data centers.
Frequent thread migrations dilute microarchitectural state, leading to degraded cache locality (L1, L2, and LLC), increased branch mispredictions, impaired hardware prefetcher efficacy, and saturated memory bandwidth. These effects are most pronounced in chiplet-based designs where workloads are dispersed across LLC boundaries, elevating remote memory fetch latencies. Rigid CPU partitioning (via hard cpusets) holds appeal for state preservation but is operationally infeasible at scale due to workload burstiness and pervasive resource overcommitment, resulting in stranded capacity or pathological queueing.
Affinity Tailor Architecture
Affinity Tailor introduces a userspace-guided, kernel-supported scheduling framework that dynamically preserves spatial and microarchitectural locality. Its core concept is to steer threads towards workload-determined, demand-sized, topologically compact sets of "Preferred Cores," while maintaining a permeable boundary to ensure global work-conservation.
Preferred Cores Mechanism
Implemented within the Linux kernel as "Core-Aware Scheduling" (CAS), Preferred Cores supplies per-cgroup soft affinity hints. When scheduling, the kernel first attempts to place threads on their assigned Preferred Cores. If capacity is fully occupied, it gracefully permits spillover to the wider system, preventing severe thread queueing and tail latency spikes. CAS contrasts with hard affinity approaches by allowing bursts beyond nominal CPU assignments and retaining maximal compatibility with mainline Linux and arbitrary workloads.
Demand Prediction and Assignment
A userspace agent continually predicts each workload's near-term CPU demand using high-percentile (e.g., p99) CPU usage observed over a recent sliding window. This demand estimation enables dynamic resizing of Preferred Core regions, balancing the need to avoid both overprovisioning (which fragments affinity regions and diminishes locality) and underprovisioning (which increases spillover to non-preferred cores).
Topology-Aware Core Allocation
Affinity Tailor provides two primary assignment algorithms, tailored to hardware topology:
- Chiplet/Split-LLC Systems (Algorithm 1): Allocates soft affinity regions at the chiplet granularity, packing workload threads into the smallest feasible set of LLC domains to minimize cross-chiplet migrations and remote memory activity. This combinatorial allocation is tractable due to the typically small number of chiplets per socket.
- Monolithic-LLC Systems (Algorithm 2): Applies fine-grained, core-based allocation to maximize the efficacy of private core-local predictive hardware structures (L1/L2 caches, branch predictors) in the absence of large LLC domain boundaries.
Evaluation and Empirical Results
Affinity Tailor was deployed fleet-wide within Google's global datacenters, across thousands of machines and four hardware platforms representing both split-LLC and monolithic-LLC topologies.
Key Numerical Results:
- Throughput Gains: Geometric mean per-CPU throughput increases of up to 12% on chiplet-based systems and 3.6% on monolithic-LLC systems compared to a production-optimized CFS scheduler.
- Memory Efficiency: Per-GB throughput gains of up to 7.5%, attributable to lower memory residency due to faster workload completion.
- Microarchitectural Metrics:
- On split-LLC platforms, LLC miss rates decreased by up to 26%, directly reducing remote memory traffic and bandwidth saturation.
- Branch predictor MPKI reductions of up to 2.2% were observed, indicating improved retention of per-core historical state.
- On monolithic-LLC systems, LLC RPKI dropped by 0.9%, indicating more effective exploitation of private caches.
- Scheduling Latency Tradeoff: Notably, tail (P99) thread scheduling latency increased by up to 17%. Despite this, net application throughput improved, demonstrating that the queueing penalties incurred by waiting for warm cores are eclipsed by the benefits of locality preservation.
Prior approaches to workload isolation—hardware-level resource partitioning (e.g., [4] [5] [20] [27] [32]), hard cpuset binding, and LLC affinity kernels [6]—either (a) fail to operate at the necessary temporal granularity, (b) cannot accommodate overcommitment and burstiness, or (c) lack principled userspace demand integration. Systems such as Nest [17] offer dense packing to warm cores but do not address overcommitted, highly diverse, or bursty multi-tenant datacenter loads.
Affinity Tailor distinguishes itself by offering true demand-aware, topology- and application-informed, permeable affinity enforcement at kernel level, at full fleet scale and across workload modalities, without requiring new runtimes or hardware extensions.
Practical and Theoretical Implications
Affinity Tailor's findings robustly challenge the orthodoxy that work-conservation should remain the dominant principle in OS scheduling for large shared-memory systems. The empirical evidence underscores that aggressive load-balancing and thread migration can be actively detrimental in hardware environments with deep predictive state and bandwidth bottlenecks. These results argue for operating system schedulers that elevate spatial locality and microarchitectural state "warmth" to first-class design objectives, even at the expense of some queueing.
On the hardware front, there are implications for future microarchitectural design: if predictive state is highly valuable, mechanisms for expedited state recovery or OS-assisted state transfer may become paramount.
Potential Future Directions
- Hierarchical Scheduling: Merging chiplet- and core-granularity algorithms into a tiered approach, optimizing both inter-chiplet and intra-chiplet locality.
- Semantic Workload Classification: Integrating application-level insights from userspace agents (e.g., knowledge of I/O-bound or stateless service characteristics) to selectively apply locality-oriented scheduling.
- Scheduler Extensibility: Leveraging recent kernel modularization (e.g., sched_ext, ghOSt [10] [11]) to accelerate deployment of adaptive affinity policies.
Conclusion
Affinity Tailor demonstrates that dynamic, locality-aware soft affinity scheduling is both feasible and impactful at hyperscale datacenter scale. Its deployment results in significant application throughput increases and reduced microarchitectural interference by confining thread execution to warm microarchitectural domains aligned with near-term workload demand. The scheduling paradigm embraced by Affinity Tailor provides a compelling alternative to work-conserving designs on modern, deeply shared hardware, signaling clear directions for the evolution of OS scheduling and hardware-software co-design.
Reference: "Affinity Tailor: Dynamic Locality-Aware Scheduling at Scale" (2604.27915)