MiCo: Hierarchical Framework for VM Scheduling

• MiCo is a hierarchical framework that uses language agents to discover and compose Python-based heuristics for virtual machine scheduling.
• It reformulates scheduling as a Semi-Markov Decision Process with Options, separating non-contextual heuristic discovery from context-aware selection.
• In large-scale tests, MiCo handles over 10,000 VMs with near offline-optimal performance while adapting to nonstationary request flows.

MiCo is a hierarchical language-agent framework for virtual machine scheduling in cloud computing, introduced for solving Online Dynamic Multidimensional Bin Packing (ODMBP) under large-scale complexity and fluctuating demands. It formulates ODMBP as a Semi-Markov Decision Process with Options (SMDP-Option) and implements a two-stage architecture—Option Miner and Option Composer—in which LLMs discover, refine, prune, and compose scheduling heuristics expressed as Python functions. In the reported large-scale enterprise setting, MiCo achieves a 96.9\% competitive ratio in scenarios involving more than 10,000 virtual machines, while maintaining high performance under nonstationary request flows and diverse configurations (Wu et al., 15 May 2025).

1. Formal problem setting

MiCo is defined for an ODMBP setting in which a set of $N$ physical machines (PMs) is given, each with $d$-dimensional capacity $C^{pm}_i \in \mathbb{Z}^d$, together with an online sequence of VM requests arriving one-by-one over time. Each VM request is specified by $(vm\_id, c^{vm}\in\mathbb{Z}^d, b\in\{0,1\})$, where $b=1$ denotes a creation request and $b=0$ a deletion request. At each VM-creation event, the scheduler must immediately assign the VM to one of the PMs, with action space $A=\{1,\ldots,N\}$, without knowledge of future arrivals. Upon deletion events, resources are freed, but the update is deferred to the next arrival for synchronization.

The optimization target is to maximize the total number of successfully placed VMs, or a suitably weighted sum of rewards, subject to capacity constraints. In the discrete-time MDP formulation, the state is given as

$$s_t=\left(\{c^{pm}_{i,t}\}_{i=1}^N;\; b_t,\; c^{vm}_t;\; \Phi_t\right),$$

where $c^{pm}_{i,t}\in\mathbb{Z}^d$ is the remaining capacity of PM $i$ at event $d$0, $d$1 and $d$2 describe the arriving VM or deletion event, and $d$3 is the VM-to-PM mapping history. The action $d$4 selects a PM for a creation event; there is no decision for deletion.

The immediate reward is binary:

$d$5

The process terminates on the first failure. When $d$6 and $d$7 is feasible, the remaining capacity is updated by

$d$8

and the mapping is extended by $d$9. When $C^{pm}_i \in \mathbb{Z}^d$0, the deleted VM is queued in $C^{pm}_i \in \mathbb{Z}^d$1 and actually freed at the next creation event. The objective is

$C^{pm}_i \in \mathbb{Z}^d$2

This formulation makes explicit that the scheduling problem is online, dynamic, multidimensional, and resource-constrained. It also fixes the point at which MiCo departs from conventional heuristics: instead of hard-coding a single rule, it learns a hierarchy of temporally extended heuristics.

2. Hierarchical SMDP-Option reformulation

MiCo introduces a finite set of options $C^{pm}_i \in \mathbb{Z}^d$3, each treated as a temporally extended policy. An option is defined as $C^{pm}_i \in \mathbb{Z}^d$4, where $C^{pm}_i \in \mathbb{Z}^d$5 is the initiation set, $C^{pm}_i \in \mathbb{Z}^d$6 is the intra-option policy, and $C^{pm}_i \in \mathbb{Z}^d$7 is the termination condition. Selecting among options induces an SMDP $C^{pm}_i \in \mathbb{Z}^d$8.

The micro-level objective optimizes the option policy:

$C^{pm}_i \in \mathbb{Z}^d$9

The macro-level objective optimizes the master policy $(vm\_id, c^{vm}\in\mathbb{Z}^d, b\in\{0,1\})$0:

$(vm\_id, c^{vm}\in\mathbb{Z}^d, b\in\{0,1\})$1

The resulting architecture is explicitly two-stage. Option Miner, described as the “micro-learner,” discovers a library of $(vm\_id, c^{vm}\in\mathbb{Z}^d, b\in\{0,1\})$2 non-contextual scheduling heuristics via LLM-based function optimization on segmented scenarios. Option Composer, described as the “macro-learner,” prunes and composes the mined options into a context-aware master policy. The distinction between non-context-aware and context-aware behavior is central: MiCo first searches for diverse reusable heuristics and only then learns how to switch among them according to workload context.

A plausible implication is that MiCo should be understood less as a monolithic policy network than as a language-driven hyper-heuristic system. That interpretation is consistent with the paper’s emphasis on temporal abstraction, robustness to nonstationary request flows, and interpretability (Wu et al., 15 May 2025).

3. Option Miner

Option Miner begins by partitioning the full VM stream of length $(vm\_id, c^{vm}\in\mathbb{Z}^d, b\in\{0,1\})$3 into $(vm\_id, c^{vm}\in\mathbb{Z}^d, b\in\{0,1\})$4 non-overlapping time windows of length $(vm\_id, c^{vm}\in\mathbb{Z}^d, b\in\{0,1\})$5, for example $(vm\_id, c^{vm}\in\mathbb{Z}^d, b\in\{0,1\})$6, thereby obtaining scenarios $(vm\_id, c^{vm}\in\mathbb{Z}^d, b\in\{0,1\})$7. Each scenario is intended to exhibit a distinct VM size distribution.

Within each scenario, an option scheduler is represented as a Python function of the form priority_v*(bin,item)->score. MiCo maintains a pool of top-$(vm\_id, c^{vm}\in\mathbb{Z}^d, b\in\{0,1\})$8 candidate policies and iteratively refines them via contrastive LLM prompting. The reported algorithm initializes a seed policy $(vm\_id, c^{vm}\in\mathbb{Z}^d, b\in\{0,1\})$9 with BestFit code for each scenario $b=1$0. For each iteration $b=1$1, each policy $b=1$2 is evaluated by simulation on $b=1$3 sampled subsequences from $b=1$4; the top-$b=1$5 policies are selected by average reward; the LLM is prompted with role_des, task_des, and the code of the top-$b=1$6 policies to generate $b=1$7; and the new policies are inserted and re-ranked. The final best policies are then returned as options.

The prompt template is also specified. In the scheduler prompt, the role is “You are an expert VM scheduler.” The task is “Given existing priority_v0, produce a more sophisticated priority_v* with multiple conditionals and loops.” The required output is Python code only. This design choice places the heuristic search space in directly executable source code rather than latent parameters, which is closely tied to the framework’s interpretability claims.

The Option Miner stage therefore operationalizes heuristic discovery as code generation guided by simulation feedback. The mined options are not learned embeddings or black-box action heads; they are explicit scheduling functions whose behavior can be inspected and compared.

4. Option Composer

Option Composer converts the mined heuristic library into a smaller and more context-aware control layer. It first performs option pruning by computing each option’s performance $b=1$8 across all $b=1$9 scenarios. An option is retained if it exceeds a single-scenario threshold $b=0$0 in at least one scenario, or if it is at least the mean performance in at least a $b=0$1 fraction of scenarios. This produces a reduced option set $b=0$2 with $b=0$3.

The master policy is represented as a Python function heuristic_selector_v*(condition)->{1,2,3,4}. The variable condition encodes the past $b=0$4 VM arrivals as histograms over VM types. As in Option Miner, MiCo applies LLM-based function optimization, here over $b=0$5 iterations. Each candidate master policy $b=0$6 is evaluated by simulating selections over $b=0$7 on $b=0$8 test sequences and measuring average reward $b=0$9. The LLM then receives the top-$A=\{1,\ldots,N\}$0 master policies’ code together with a context-aware scheduler template and generates $A=\{1,\ldots,N\}$1. Iteration proceeds until convergence, after which $A=\{1,\ldots,N\}$2 is output.

This stage is where MiCo becomes context-aware in the strict sense used in the paper. The lower level supplies non-context-aware heuristics; the upper level learns a composing strategy that integrates those heuristics with contextual information derived from recent arrivals. The framework therefore separates heuristic invention from heuristic selection.

A common misconception is that MiCo is simply a direct LLM-for-control policy. In the reported design, the LLM does not schedule each VM online token by token. Instead, it discovers and refines Python heuristics and a Python heuristic selector, which are then evaluated in simulation and deployed as scheduling logic.

5. Implementation and empirical performance

The reported implementation uses GPT-4 as the language agent, with temperatures $A=\{1,\ldots,N\}$3 and seed policies Best-Fit and First-Fit. The software and hardware environment is Python 3.7 on Ubuntu 20.04 with AMD EPYC 192 cores, 514 GB RAM, and NVIDIA A100. The core experimental setting uses $A=\{1,\ldots,N\}$4 PMs and $A=\{1,\ldots,N\}$5 resource dimensions, namely CPU and RAM. For the Huawei dataset, the configuration is $A=\{1,\ldots,N\}$6, $A=\{1,\ldots,N\}$7, $A=\{1,\ldots,N\}$8, $A=\{1,\ldots,N\}$9, $$s_t=\left(\{c^{pm}_{i,t}\}_{i=1}^N;\; b_t,\; c^{vm}_t;\; \Phi_t\right),$$0, and $$s_t=\left(\{c^{pm}_{i,t}\}_{i=1}^N;\; b_t,\; c^{vm}_t;\; \Phi_t\right),$$1. Simulations use 30 independent seeds per experiment, and a code validity ratio is monitored to ensure syntactic correctness (Wu et al., 15 May 2025).

Benchmark	Reported result	Comparison
Huawei “East-1” dataset ($$s_t=\left(\{c^{pm}_{i,t}\}_{i=1}^N;\; b_t,\; c^{vm}_t;\; \Phi_t\right),$$2 K requests, nonstationary over 1 year)	MiCo mean performance ratio: 96.9% vs. offline Gurobi optimal	Best-Fit 92.6%, First-Fit 89.7%, HindSight 90.5%, SchedRL 85.8%
AzurePublicDatasetV2 ($$s_t=\left(\{c^{pm}_{i,t}\}_{i=1}^N;\; b_t,\; c^{vm}_t;\; \Phi_t\right),$$3 K requests, more stationary)	MiCo mean ratio: 71.5% vs. Gurobi	$$s_t=\left(\{c^{pm}_{i,t}\}_{i=1}^N;\; b_t,\; c^{vm}_t;\; \Phi_t\right),$$4 over best heuristic at 65.5%

On the Huawei “East-1” dataset, MiCo outperforms SchedRL by 11.1\% on average and by 32.6\% in the hardest scenario, denoted $$s_t=\left(\{c^{pm}_{i,t}\}_{i=1}^N;\; b_t,\; c^{vm}_t;\; \Phi_t\right),$$5. On held-out test sequences, the reported boxplots show MiCo’s median and interquartile range exceeding all heuristics. In the large-scale setting emphasized in the abstract, the framework handles more than 10,000 VMs online with 96.9\% of offline optimal.

The ablation results specify which design choices matter. Without hierarchical composition, described as “fun-search only,” improvement over Best-Fit is at most 1.2\%, and the method fails to adapt to nonstationarity. Removing option pruning reduces the mean from 96.9\% to 95.4\% and leads to slower convergence and occasional scenario-mismatch. When the sample length $$s_t=\left(\{c^{pm}_{i,t}\}_{i=1}^N;\; b_t,\; c^{vm}_t;\; \Phi_t\right),$$6 varies from 100 to 800, MiCo remains above 92\% in all cases and peaks at 400–600. For temperature $$s_t=\left(\{c^{pm}_{i,t}\}_{i=1}^N;\; b_t,\; c^{vm}_t;\; \Phi_t\right),$$7, performance peaks around $$s_t=\left(\{c^{pm}_{i,t}\}_{i=1}^N;\; b_t,\; c^{vm}_t;\; \Phi_t\right),$$8–0.8, while code validity drops beyond $$s_t=\left(\{c^{pm}_{i,t}\}_{i=1}^N;\; b_t,\; c^{vm}_t;\; \Phi_t\right),$$9. Reported LLM variants are GPT-4 at 96.9\%, GPT-3.5-Turbo at 94.5\%, DeepSeekCoder v2 at 96.1\%, and DeepSeekCoder v1 at 96.2\%.

6. Interpretability, limitations, and broader significance

The paper identifies four principal strengths. First, MiCo supports automatic heuristic discovery and thereby eliminates manual rule-coding while adapting to workload shifts. Second, its hierarchical temporal abstraction is reported to be robust to nonstationary request flows and to extend the planning horizon. Third, the generated Python functions are interpreted as close to classical heuristics, including weighted-sum rules, dynamic weights, and lookahead. Fourth, the framework is reported to scale to more than 10,000 VMs online while approaching offline optimality (Wu et al., 15 May 2025).

The interpretability case study is especially important for situating MiCo within the scheduling literature. The Option Miner stage yields weighted-sum heuristics with ratio-based weighting, dynamic CPU/MEM trade-offs, and lookahead loops, and these are described as analogous to multidimensional bin-packing heuristics in Christensen et al., 2017 and Vidal & Farges, 2004. The Option Composer stage yields statistical feature extraction based on averages, trends, and accelerations, together with rule-based mapping to heuristics 1–4, and this is described as akin to context-aware hyper-heuristics. A plausible implication is that MiCo’s novelty lies not in abandoning hand-crafted scheduling ideas, but in automating their discovery and composition through LLM-guided search.

The limitations are also explicit. LLM inference cost is substantial because each iteration invokes GPT-4. Code validity is imperfect, with random failures reported at less than 5\% when $c^{pm}_{i,t}\in\mathbb{Z}^d$0, requiring retry logic. The framework also has a cold-start dependence on seed policies, specifically Best-Fit in the reported experiments, and “zero-knowledge startup” is identified as an open challenge.

Taken together, these features place MiCo at the intersection of combinatorial optimization, cloud scheduling, hyper-heuristics, and code-generating language agents. Its reported contribution is a concrete heuristic-design paradigm for ODMBP in which non-contextual options are mined, context-aware composition is learned, and both levels remain inspectable as executable scheduling code rather than opaque action distributions.