Genetic Programming for Self-Adaptive Auto-Scaling of Microservices

Abstract: Microservice architecture is widely adopted in modern systems, where auto-scaling is critical for satisfying service-level objectives (SLOs). However, determining optimal scaling for microservices is difficult, and reactive resource allocation often leads to costly over- or under-provisioning. We propose AutoSLO, a learning-based, self-adaptive scaling framework that dynamically adjusts microservice replicas to meet SLOs while minimizing resource usage. AutoSLO uses a continuous monitoring-adaptation feedback loop and leverages genetic programming to learn and evolve scaling logic, enabling the deployed microservice system to proactively prevent SLO violations rather than repeatedly searching for one-off scaling actions. We evaluate AutoSLO on two case-study systems -- an online shopping platform and a chatbot based on LLMs -- and show that this framework substantially reduces resource usage while maintaining a low frequency of SLO violations, all of which are resolved within a short time window.

• The paper introduces AutoSLO, a GP-driven framework that evolves scaling formulas to balance resource allocation and SLO compliance in microservices.
• It employs a MAPE-loop and surrogate SLO prediction model to efficiently generate and evaluate scaling policies without risky online trials.
• Empirical results on CPU-bound and GPU-bound systems show up to 50% resource savings with rapid recovery from SLO violations, highlighting its cost-effectiveness.

Genetic Programming for Self-Adaptive Auto-Scaling of Microservices

Introduction and Motivation

Service orchestration in microservice architectures presents challenging trade-offs between operational resource utilization and strict Service Level Objectives (SLOs). Traditional threshold-based scaling and widely deployed solutions like Kubernetes’ Horizontal Pod Autoscaler (HPA) are insufficient for orchestrating complex, interacting microservices under dynamic workloads. These approaches typically focus on isolated resource metrics (e.g., CPU or memory utilization) and follow reactive, one-shot adaptation policies, leading to persistent over-provisioning or SLO violations, especially where SLOs are multifaceted or cross-microservice.

“Genetic Programming for Self-Adaptive Auto-Scaling of Microservices” (2605.01533) introduces AutoSLO, an online adaptation loop driven by Genetic Programming (GP) that learns, evolves, and applies application-level scaling logic correlated with live SLO feedback. The framework targets microservices deployed in Kubernetes, evolving parametric scaling formulas to minimize resource allocation while bounding SLO violations.

AutoSLO Architecture

AutoSLO is architected around a MAPE-style (Monitor, Analyze, Plan, Execute) self-adaptive control loop, targeting only microservices identified as application-level bottlenecks to restrict the adaptation search space. Two critical artifacts are constructed during system initialization:

1. Bottleneck Microservice Identification: Using Performance Bottleneck Analysis (PBA), AutoSLO statically identifies microservices whose scaling actions will meaningfully affect SLOs system-wide.
2. Surrogate SLO Prediction Model: An offline-trained regression model predicts SLO metric values (e.g., 90th-percentile latency, request success rate) for any hypothetical scaling action, given current operational metrics. This eliminates the need for repeated online trial-and-error scaling experiments.

At runtime, the continuous AutoSLO loop operates as follows: it monitors operational and SLO metrics; checks SLO thresholds; triggers planning if SLOs are violated or resource utilization is excessive; employs GP to evolve scaling formulas conditioned on runtime metrics; and enacts replica count changes through the Kubernetes API.

Figure 1: AutoSLO’s self-adaptation control loop enabling closed-loop, SLO-driven scaling of bottleneck microservices.

This architecture contrasts standard HPA or rule-based approaches by generating and incrementally optimizing symbolic scaling formulas—mathematical expressions mapping features like QPS or per-service CPU usage to recommended pod counts—rather than manipulating static thresholds.

Genetic Programming-Based Scaling Logic Evolution

The core innovation in AutoSLO is the application of GP to synthesize scaling policies. Each GP individual encodes, per bottleneck microservice, a formula tree convertible into a mathematical expression over monitored runtime metrics and ephemeral random constants. The search space is defined by a context-free grammar supporting real-valued arithmetic expressions, constructed via randomized grow and breed operators typical of genetic algorithm frameworks.

Figure 2: An example GP individual comprising formula trees for two bottleneck microservices, synthesizing parametric scaling logic.

Individuals are evaluated via the surrogate SLO predictor, simulating the impact of the scaling policy on future SLO attainment without enacting the adaptation online. The GP fitness function is a minimization objective with two branches:

• SLO-violating individuals: Penalty reflects the normalized magnitude of predicted violation above threshold.
• SLO-satisfying individuals: Reward for lower cumulative pod allocation, driving resource minimization.

Standard evolutionary operators—one-point crossover, point mutation, tournament selection—stochastically explore and exploit the scaling logic space. Planning is triggered either reactively (SLO violation or over-provisioning detected) or proactively where SLO breach is anticipated. Best-performing scaling formulas persist between adaptation cycles, reused unless performance degradation is observed.

Empirical Evaluation

AutoSLO was evaluated on two distinct microservice applications:

• Boutique Shop: A CPU-bound e-commerce system comprising eleven microservices.
• LLM-based Chatbot: A GPU-bound service hosting state-of-the-art LLMs.

The operational scope included SLOs on 90th-percentile latency (Boutique Shop) and request success rate (Chatbot), under workloads cycling between normal- and high-load phases. Baseline comparisons included:

• HPA (Horizontal Pod Autoscaler): Per-microservice, threshold-based Kubernetes scaling.
• AutoSLO-Ran: A random search variant lacking GP-driven formula evolution, employing direct sampling of possible replica setpoints.

Key findings include:

• 50.65% reduction in average pod usage (Boutique Shop) and 46.4% (Chatbot) versus HPA.
• SLO violations (<2.8 per experiment, per system) under AutoSLO are always recovered within two adaptation cycles (<2 minutes), by design.

AutoSLO outperforms random search in resource savings for the GPU-constrained Chatbot, while matching random for the CPU-bound workload. HPA achieves zero SLO violations but at the cost of consistently doubling resource allocation over AutoSLO.

Figure 3: Resource and SLO evolution under dynamic workloads in Boutique Shop, comparing AutoSLO, AutoSLO-Ran, and HPA.

These results demonstrate a deliberate trade-off: AutoSLO is tuned for resource parsimony, accepting and rapidly recovering short-lived, bounded SLO violations, whereas HPA relies on persistent over-provisioning to guarantee compliance.

Discussion and Implications

The AutoSLO framework substantiates a search-based software engineering (SBSE) paradigm for self-adaptive system orchestration, relying on online policy learning instead of statically encoded heuristics or thresholding rules. The GP-based planning layer systematically explores and exploits compositional scaling logic, capable of adapting to previously unseen operational contexts via evolution. The use of surrogate modeling ensures that adaptation is computationally viable in operational deployments, as each policy evaluation is cheap and does not require risky online experimentation.

There remain notable limitations. AutoSLO currently operates over a static bottleneck set and is developed for single-SLO targets; extension to online bottleneck detection and multi-SLO trade-off optimization would require significant methodological advances, likely integrating multi-objective GP and online bottleneck identification algorithms.

The empirical trade-off between bounded SLO violations and aggressive resource minimization is architecturally beneficial for cost-sensitive cloud providers; however, applicability to high-regret failure-intolerant domains is less clear. Furthermore, as microservice deployments increasingly blend diverse resource constraints (CPU, GPU, network), future instantiations should generalize the surrogate modeling layer to multi-resource, multi-metric settings.

Conclusion

AutoSLO embodies a practical, GP-driven approach to self-adaptive autoscaling for Kubernetes-style microservice environments. Its ability to halve resource usage relative to baseline autoscalers, while maintaining swift recovery from occasional SLO violations, positions it as an effective blueprint for SLO-centric, cost-aware orchestration in both CPU- and GPU-intensive deployments. Continued advances along online bottleneck detection, multi-SLO adaptation, and multi-resource policy synthesis are promising directions for future AI-driven cloud orchestration research.