NUMA-Aware Scheduling Overview

• NUMA-aware scheduling is a method that aligns task placement and memory allocation with local memory nodes to minimize remote access penalties.
• It leverages runtime monitoring and dynamic metrics, including speedup and contention degradation factors, to drive migration and load balancing.
• Empirical evidence shows performance gains of up to 25%, enhancing throughput in scientific, database, and cloud workloads.

A Non-Uniform Memory Access (NUMA)-aware scheduling system refers to any hardware, runtime, user-level, or operating system-level mechanism that explicitly optimizes the assignment of computations and memory allocations to the hierarchical memory topology present in NUMA architectures. In such architectures, each CPU socket or node has local memory, with remote memory accesses incurring greater latency and lower bandwidth. NUMA-aware scheduling is key to maximizing both performance and scalability in scientific, database, cloud, and parallel workloads, as it minimizes costly remote accesses and resource contention.

1. NUMA-Aware Scheduling: Definition and Motivation

NUMA-aware scheduling describes a family of methods and algorithms that adapt thread placement, task assignment, and memory allocation to the physical memory and core topology of NUMA hardware. The main goal is to co-locate processing threads with their preferred memory nodes and to minimize the frequency and cost of remote memory accesses. This is motivated by the pronounced, often order-of-magnitude differences in access latency and bandwidth between local and remote nodes, especially in systems with multiple multicore sockets.

Traditional OS or runtime schedulers, which operate under symmetric multiprocessing (SMP) assumptions or uniform memory architectures, are typically ill-suited to NUMA hardware. They may migrate threads and allocate memory obliviously, resulting in highly variable application throughput, increased remote memory traffic, and degraded cache utilization.

2. Algorithmic Frameworks and User-Level Approaches

NUMA-aware scheduling can be implemented at the kernel, user, or runtime system level. A notable advance is presented in the "User-Level Memory Scheduler for Optimizing Application Performance in NUMA-Based Multicore Systems" (Lim et al., 2021), which advocates for a user-space memory scheduler to overcome the limitations of kernel-only approaches.

This user-level scheduler comprises three core modules:

• Runtime Monitor: Periodically collects per-process NUMA statistics (such as memory usage and node topology) from /proc/<pid>/numa_maps, /proc/<pid>/stat, and sysfs entries.
• Reporter: Processes monitored data, filters for NUMA-specific imbalances, and computes two decisive metrics: the speedup factor (which predicts the benefit of NUMA-aware migration) and the contention degradation factor (which estimates the expected slow-down from resource contention).
• User-Space Scheduler: Decides memory node allocation for processes, migrating tasks and memory pages as needed for both load balancing and minimizing contention.

These modules are executed according to a well-defined algorithmic workflow. The scheduler dynamically sorts and migrates processes by their computed speedup and degradation factors, ensuring that the most beneficial migrations are enforced and that resources are allocated according to current application importance and observed locality.

The scheduler's actions are based on runtime data, allowing for continuous adaptation as workload characteristics change. It performs memory page migrations in user space, selectively clustering "sticky" pages with processes to optimize locality, and it can scatter contending tasks to reduce cross-node contention.

The core metrics are given by:

$$\text{Speedup Factor} = \frac{\text{Execution Time}_{\text{baseline}}}{\text{Execution Time}_{\text{NUMA-scheduled}}}$$

$$\text{Contention Degradation Factor} = \frac{\text{Execution Time}_{\text{contention}} - \text{Execution Time}_{\text{optimal}}}{\text{Execution Time}_{\text{optimal}}}$$

These metrics govern migration and pinning decisions.

3. NUMA-Aware Optimization Strategies

NUMA-aware schedulers optimize application performance through several concrete mechanisms:

• Intelligent Node Selection: Memory and task allocations are made based on up-to-date runtime monitoring instead of static affinity or OS defaults. Memory used by threads is migrated to nodes where they spend the majority of execution time, reducing remote access penalties.
• Dynamic Adaptation: Unlike static approaches (manual affinity or initialization policies), user-level schedulers react to observed changes in workload distribution, process importance, or contention patterns.
• Fine-Grained Process Sorting: Allocation decisions are sorted by empirically computed speedup and degradation factors, enabling prioritization of migrations with greatest benefit.
• Page and Task Co-Migration: Both task and associated data ("sticky" memory pages) are jointly migrated, maintaining data locality and reducing TLB and cache misses.

This approach minimizes remote memory access, balances memory and compute loads across NUMA nodes, and prevents unnecessary resource contention without requiring expert manual tuning.

4. Empirical Impact and Performance Metrics

Comprehensive experiments on a Dell PowerEdge R910 (Intel Xeon E7-4850, 40 cores, 32 GiB RAM) running Linux 3.2 validate the effectiveness of user-level NUMA scheduling (Lim et al., 2021). Benchmarks from the PARSEC suite, as well as real-world server workloads (Apache and MySQL), were used.

Key empirical findings include:

Workload	Metric	Baseline (OS/Static)	User-Level NUMA Scheduler	Improvement
PARSEC Suite	Execution Time	1×	up to 0.75×	Up to 25%
Apache Server	Throughput	1×	1.126×	12.6%
MySQL	Throughput	1×	1.07×	7%

The proposed scheduler also achieves up to 85% improvement in priority-awareness (i.e., ability to recognize application importance) relative to OS-level automatic NUMA scheduling.

Significant technical highlights are:

• Outperforms manual optimization (affinity/policy tuning) in most cases, with only minor exceptions on applications that benefit from expert static placement.
• Application throughput and execution time improvements are consistent across workload types (both CPU-bound and memory-bound).
• No manual adjustment, kernel source modification, or privileged operations are required: all logic resides in user space.

5. Integration, Limitations, and System-Level Considerations

NUMA-aware scheduling at user level is compatible with unprivileged application environments and can be layered atop existing OS policies without requiring changes to kernel or system libraries. It is, however, dependent on accurate and frequent collection of per-process NUMA statistics and assumes user-level control over process and memory assignment via mechanisms such as mbind, set_mempolicy, and page migration APIs.

A plausible implication is that as system complexity and core counts increase, kernel-level NUMA optimizations may be insufficient to maintain locality and balance, especially in multi-tenant or dynamic application environments where user-space importance is opaque to the OS. In such scenarios, user-level NUMA-aware schedulers can serve as critical middleware to close the performance gap.

Advanced NUMA-aware scheduling frameworks synergize with both application-managed affinity and kernel-level policies, facilitating high performance on complex, heterogeneous many-core systems.

6. Summary Table: Scheduler Features and Performance

Scheduler Type	Affinity Awareness	Handles User/Application Priority	Runtime Adaptation	Page Migration	Performance Gain versus OS Default
OS/kernel-level	Static/dynamic	Limited	Limited	Yes	Baseline
Manual (static tuning)	Static	Via expert	No	No	~1.0×–1.2×
User-level (Lim et al., 2021)	Dynamic	Full	Full	Yes	1.0×–1.25×

7. Conclusions and Best Practices

NUMA-aware scheduling, particularly at user level, is maximally effective when it couples fine-grained, continuous runtime monitoring with dynamic prioritization and relocation of tasks and memory. Key principles are:

• Quantify per-process benefit from locality (speedup factor); prioritize migrations with largest expected gain.
• Assess and respond to runtime contention (degradation factor); redistribute or scatter tasks under contention.
• Migrate tasks and their "sticky" memory pages jointly for robust data locality.
• Continuously monitor and adapt to application behavior, system topology, and resource contention without reliance on offline profiling or manual intervention.

These practices deliver robust and scalable performance benefits on NUMA-based multicore systems, as shown by up to 25% speedup on scientific workloads and double-digit throughput gains on production servers—all without kernel modifications or manual scheduling, establishing a new standard for performant, NUMA-optimized system and user-level scheduling.