SLA-MORL: SLA-Aware Multi-Objective RL

• The paper introduces an adaptive MORL framework that minimizes training time by 67.2%, operational costs by 68.8%, and improves SLA compliance by 73.4% compared to static baselines.
• It employs an actor-critic network with a 21-dimensional state representation and nine discrete resource-adjustment actions to dynamically manage resource allocation based on real-time SLA violation severity.
• The framework leverages intelligent initialization and dynamic weight adaptation to reduce cold-start exploration by 60% and enhance overall resource utilization in cloud environments.

Searching arXiv for the cited papers and related acronym usage to ground the article. SLA-MORL is an adaptive multi-objective reinforcement learning framework for dynamic resource allocation for machine learning workloads in cloud environments. It is formulated to minimize training time and operational costs, maximize resource utilization, and enforce SLA compliance under deadline or budget constraints specified by the user. In its reported formulation, SLA-MORL combines an actor-critic network, a 21-dimensional state representation, nine discrete resource-adjustment actions, intelligent initialization through historical learning or efficient baseline runs, and dynamic weight adaptation based on real-time SLA violation severity. The framework was evaluated on 13 diverse ML workloads using production HPC infrastructure, where it reported a 67.2% reduction in training time for deadline-critical jobs, a 68.8% reduction in costs for budget-constrained workloads, and a 73.4% improvement in overall SLA compliance compared to static baselines (Mostafa et al., 5 Aug 2025).

1. Problem setting and optimization targets

SLA-MORL addresses dynamic resource allocation for machine learning workloads in cloud environments where static resource allocation or single-objective optimization can lead to either SLA violations or resource waste. Its optimization targets are explicitly multi-objective: minimize Training Time $T$, modeled via per-epoch latency $T_{\text{epoch}}(g,c)$; minimize Operational Cost $C$, modeled as an hourly cost $C_{\text{hourly}}(g,c)$; maximize Resource Utilization $U$, by keeping GPU and CPU utilizations near user-specified targets; and enforce SLA Compliance by satisfying deadline or budget constraints specified by the user (Mostafa et al., 5 Aug 2025).

The framework casts dynamic resource allocation as an MDP $$(\mathcal{S},\mathcal{A},\mathcal{T},\mathcal{R},\gamma)$$. In the reported formulation, the state space consists of 21-dimensional real vectors, the action space contains nine discrete adjustments in GPU and CPU counts, the transition kernel is induced by system dynamics and SLURM API calls, the reward is a scalarized multi-objective reward, and the discount factor is $\gamma=0.95$. The hard system-side constraints include $g_t \in [1, G_{\max}]$, $c_t \in [1, C_{\max}]$, and the requirement that deadline $D$ or budget $T_{\text{epoch}}(g,c)$0 must not be violated beyond tolerable severity (Mostafa et al., 5 Aug 2025).

A central design choice is that resource allocation is preference-conditioned. The framework supports three user-defined preferences—time, cost, or balanced—and treats SLA compliance not as an afterthought but as a control signal that affects reward construction and policy adaptation. This makes the method operationally distinct from one-shot recommendation procedures: the policy can reallocate GPUs and CPUs during training in response to both workload evolution and violation severity. A plausible implication is that the method is intended for workloads whose optimal resource profile is not stationary across epochs.

2. State representation, action space, and reward design

The 21-dimensional state representation $T_{\text{epoch}}(g,c)$1 is constructed from five groups of signals: current GPU and CPU allocation $T_{\text{epoch}}(g,c)$2; normalized GPU and CPU utilization rates; fraction of epochs completed and instantaneous throughput; binary flags for six SLA checks; normalized violation severities for each SLA dimension; and a one-hot encoding of user preference (time, cost, balanced) (Mostafa et al., 5 Aug 2025).

More specifically, the state includes $T_{\text{epoch}}(g,c)$3, $T_{\text{epoch}}(g,c)$4 for GPU and CPU utilization, $T_{\text{epoch}}(g,c)$5 for training progress and throughput, $T_{\text{epoch}}(g,c)$6 for six SLA checks such as deadline slices and budget slices, $T_{\text{epoch}}(g,c)$7 for normalized violation severities, and $T_{\text{epoch}}(g,c)$8 for user preference encoding. The reported ablation that removes $T_{\text{epoch}}(g,c)$9 and $C$0 yields a $C$1 overall performance change, which situates violation-aware state features as structurally important rather than auxiliary (Mostafa et al., 5 Aug 2025).

The action space is discrete: $C$2 Each action adjusts the current allocation $C$3. Examples given in the report include $C$4 to add one GPU and $C$5 to remove one GPU and add one CPU core (Mostafa et al., 5 Aug 2025).

Reward design is based on scalarized multi-objective optimization with adaptive weights. The base weights depend on the user preference: $C$6 corresponding to $C$7. When an SLA objective $C$8 is violated, that weight is increased proportionally to severity $C$9 with $C_{\text{hourly}}(g,c)$0, after which weights are normalized. The final reward is

$C_{\text{hourly}}(g,c)$1

Here, $C_{\text{hourly}}(g,c)$2 is positive if throughput increases relative to the previous epoch, $C_{\text{hourly}}(g,c)$3 is positive if cost per hour decreases, $C_{\text{hourly}}(g,c)$4 rewards GPU and CPU utilizations near targets of 80% GPU and 70% CPU, and $C_{\text{hourly}}(g,c)$5 is proportional to SLA violation count or severity (Mostafa et al., 5 Aug 2025).

This adaptive weighting mechanism is the framework’s main control-theoretic device. The paper describes it as creating a self-correcting system. The reported ablation that replaces adaptive weights with fixed weights reduces SLA compliance by 31%, which indicates that weight adaptation is not merely a preference interface but a primary mechanism for responding to constraint pressure (Mostafa et al., 5 Aug 2025).

3. Initialization strategy and actor-critic training

SLA-MORL divides operation into an initialization stage and an online training stage. The initialization stage is designed to avoid random cold-start and offers three paths. The first, Skip Offline, sets $C_{\text{hourly}}(g,c)$6 and $C_{\text{hourly}}(g,c)$7. The second, Historical Learning, loads prior runs $C_{\text{hourly}}(g,c)$8, pre-trains $C_{\text{hourly}}(g,c)$9 on logged $U$0 tuples, initializes $U$1 from the best similar configuration, and uses $U$2. The third, Baseline Runs, evaluates the configurations $U$3, $U$4, and $U$5 on 20% of the data and 10% of the epochs, scales up estimated metrics as $U$6, initializes the networks, and sets $U$7 (Mostafa et al., 5 Aug 2025).

The framework states two key innovations. The first is intelligent initialization through historical learning or efficient baseline runs that eliminates cold-start problems, reducing initial exploration overhead by 60%. The second is dynamic weight adaptation that automatically adjusts optimization priorities based on real-time SLA violation severity (Mostafa et al., 5 Aug 2025).

The actor network $U$8 takes a 21-dimensional state as input and uses the sequence Linear(128) $U$9 ReLU $$(\mathcal{S},\mathcal{A},\mathcal{T},\mathcal{R},\gamma)$$0 Linear(64) $$(\mathcal{S},\mathcal{A},\mathcal{T},\mathcal{R},\gamma)$$1 ReLU $$(\mathcal{S},\mathcal{A},\mathcal{T},\mathcal{R},\gamma)$$2 Linear(9) $$(\mathcal{S},\mathcal{A},\mathcal{T},\mathcal{R},\gamma)$$3 Softmax to output a probability distribution over the nine actions. The critic network $$(\mathcal{S},\mathcal{A},\mathcal{T},\mathcal{R},\gamma)$$4 takes a 30-dimensional vector $$(\mathcal{S},\mathcal{A},\mathcal{T},\mathcal{R},\gamma)$$5 and uses Linear(128) $$(\mathcal{S},\mathcal{A},\mathcal{T},\mathcal{R},\gamma)$$6 ReLU $$(\mathcal{S},\mathcal{A},\mathcal{T},\mathcal{R},\gamma)$$7 Linear(64) $$(\mathcal{S},\mathcal{A},\mathcal{T},\mathcal{R},\gamma)$$8 ReLU $$(\mathcal{S},\mathcal{A},\mathcal{T},\mathcal{R},\gamma)$$9 Linear(1)) to output a scalar Q-value (Mostafa et al., 5 Aug 2025).

During the online phase, the procedure observes $\gamma=0.95$0, computes $\gamma=0.95$1, and triggers policy adaptation if $\gamma=0.95$2 or any SLA violation occurs. The procedure then adapts weights, samples $\gamma=0.95$3 with $\gamma=0.95$4-greedy exploration, applies the action, waits one epoch for stability, computes reward, stores $\gamma=0.95$5 in a replay buffer of capacity 100 K, and performs minibatch updates. The hyperparameters reported are $\gamma=0.95$6, $\gamma=0.95$7, and $\gamma=0.95$8 (Mostafa et al., 5 Aug 2025).

The critic update is given by

$\gamma=0.95$9

and the actor update by

$g_t \in [1, G_{\max}]$0

These choices place the method within the family of actor-critic RL systems while preserving a scalarized MORL reward interface (Mostafa et al., 5 Aug 2025).

4. Experimental setting and reported results

The reported evaluation uses a SLURM cluster with NVIDIA Quadro RTX 8000 GPUs (48 GB), Intel Xeon Gold 6148 CPUs (40 cores), and 384 GB RAM. Monitoring is performed with pynvml for GPU metrics and psutil for CPU metrics at 1 Hz, and control is implemented via scontrol and CUDA environment variables (Mostafa et al., 5 Aug 2025).

The workload suite consists of 13 model-data pairs. The report lists remote sensing workloads CAM, gWaveNet, and DAMA, and computer vision workloads including VGG16, ResNet50/101 on CIFAR-10/100, and Transformers and ResNet101 on CIFAR-100 and ImageNet100. All workloads are trained for 200 epochs with no early stop (Mostafa et al., 5 Aug 2025).

The baselines are Basic, which uses a fixed 1 GPU plus associated CPU allocation; Static_recom, a one-time optimal allocation; SLA-MORL_lite, which is online RL with no offline initialization; SLA-MORL_base_runs, which uses only the three baseline initializations; and SLA-MORL_w_target_logs, which uses only historical logs (Mostafa et al., 5 Aug 2025).

The quantitative results are reported in several forms. Average percentage improvement by priority relative to Basic is 63.7 for Time, 49.8 for Cost, and 58.4 for Balanced for the full SLA-MORL system; 61.4, 48.3, and 56.1 for SLA-MORL_w_target; 58.2, 46.9, and 52.7 for SLA-MORL_base_runs; 47.6, 43.1, and 44.8 for SLA-MORL_lite; and 33.4, 38.7, and 39.2 for Static_recom (Mostafa et al., 5 Aug 2025).

SLA compliance rates are also broken out by preference. For Time priority, the report gives 61.8% for SLA-MORL versus 17.4% for static; for Cost priority, 77.1% versus 33.8%; and for Balanced, 82.1% versus 39.2% (Mostafa et al., 5 Aug 2025).

The overall performance table averaged over 13 workloads reports the following for SLA-MORL: 67.2% on Time $g_t \in [1, G_{\max}]$1, 68.8% on Cost $g_t \in [1, G_{\max}]$2, 73.4% on SLA $g_t \in [1, G_{\max}]$3, and 68.0% on Efficiency $g_t \in [1, G_{\max}]$4. The corresponding Static_recom values are 48.7%, 50.2%, 59.1%, and 49.5%, while SLA-MORL_lite reports 52.4%, 54.1%, 62.8%, and 53.3% (Mostafa et al., 5 Aug 2025).

The reported dynamic resource patterns are preference-sensitive. Time priority escalates to 4 GPUs aggressively to meet deadlines; Cost priority stays at 1–2 GPUs and 2–4 CPUs; Balanced exhibits fluid trade-offs. In the gWaveNet Pareto-frontier visualization, SLA-MORL solutions are described as lying strictly closer to the time-cost optimum than all baselines (Mostafa et al., 5 Aug 2025).

5. Interpretation, ablations, and limitations

The ablation studies isolate two dependencies. First, removing the SLA-check and violation-severity components $g_t \in [1, G_{\max}]$5 and $g_t \in [1, G_{\max}]$6 causes a $g_t \in [1, G_{\max}]$7 overall performance change. Second, replacing adaptive weights with fixed weights reduces SLA compliance by 31% (Mostafa et al., 5 Aug 2025). These ablations support the paper’s claim that the rich 21-dimensional state, including violation metrics, is critical for sensitivity to constraints, and that dynamic weight adaptation enables self-correction under SLA violations.

The paper’s main insights and applicability statements are narrowly scoped. It states that intelligent initialization reduces cold-start exploration by 60%, dynamic weight adaptation enables self-correction under SLA violations, and the 21-dimensional state is critical for sensitivity to constraints. It also states that the design principles generalize to any cloud or on-prem environment where multi-objective SLAs must be balanced automatically (Mostafa et al., 5 Aug 2025).

The limitations are explicit: the current formulation uses a linear cost model, is evaluated on a single HPC cluster, and does not include spot-instance pricing. Future directions listed in the report are extension to heterogeneous resources, continuous actions, and multi-cluster public clouds such as AWS, Azure, and GCP (Mostafa et al., 5 Aug 2025). This suggests that the published system should be interpreted as an HPC-focused instantiation rather than as a complete treatment of all cloud pricing and scheduling regimes.

A common misconception is that SLA-MORL is simply a static preference switch among time, cost, and balanced modes. The reported mechanism is more specific: user preference sets base weights, but final optimization priorities are changed online through severity-conditioned adaptation and an SLA penalty term. Another misconception is that intelligent initialization is equivalent to offline tuning. In the reported system, initialization only seeds policy and value estimates; the actor-critic still performs online adaptation during the workload run (Mostafa et al., 5 Aug 2025).

6. Terminological ambiguity and relation to adjacent research

The acronym “SLA-MORL” is not unique in the literature. In “Limitations of Scalarisation in MORL: A Comparative Study in Discrete Environments,” the term appears as shorthand for scalarisation-based learning approaches in multi-objective reinforcement learning, specifically MO Q-Learning with linear scalarisation and Chebyshev scalarisation, evaluated in an outer-loop multi-policy methodology (Shah et al., 20 Nov 2025). That usage differs materially from “SLA-MORL: SLA-Aware Multi-Objective Reinforcement Learning for HPC Resource Optimization,” where the acronym denotes Service Level Agreement awareness rather than scalarisation-based learning (Mostafa et al., 5 Aug 2025).

The distinction matters because the comparative scalarisation study emphasizes several limitations of outer-loop scalarisation: incomplete front coverage, solution stability problems or “forgetting,” sensitivity to Pareto-front shape, and exponential blow-up in weight tuning as the number of objectives increases. It reports that linear scalarisation can only recover the convex hull of the Pareto front, while Chebyshev can find non-convex Pareto points but requires careful tuning of $g_t \in [1, G_{\max}]$8 and $g_t \in [1, G_{\max}]$9, and it presents Pareto Q-Learning as an inner-loop alternative that learns the entire Pareto front in one run without manual weight-grid tuning (Shah et al., 20 Nov 2025).

That comparison provides useful context for the HPC framework. SLA-MORL in the HPC paper also uses scalarized multi-objective reward, but it does not frame its contribution as full Pareto-front recovery. Instead, it emphasizes adaptive control under user preferences and SLA pressure. A plausible implication is that its reward design should be understood as an operational decision mechanism for online scheduling rather than as a general solution to the front-coverage issues identified in outer-loop scalarisation studies.

A related but distinct line of work appears in PROTEUS, which treats SLA-aware LLM routing as a constrained MDP with cost as the primary objective and accuracy as a constraint. PROTEUS uses Lagrangian dual control, a learned dual variable $c_t \in [1, C_{\max}]$0, and a single $c_t \in [1, C_{\max}]$1-conditioned policy that operates across $c_t \in [1, C_{\max}]$2, with reported 100% floor compliance on RouterBench and SPROUT (Bhatti et al., 27 Jan 2026). A plausible implication is that SLA-aware MORL systems can be organized around at least two different control logics: adaptive scalarized reward weighting, as in the HPC SLA-MORL framework, and primal-dual constraint enforcement, as in PROTEUS. The two papers therefore occupy adjacent but non-identical positions in the design space of SLA-conditioned reinforcement learning.