Dynamic Data Selection Schedulers

• Dynamic Data Selection Schedulers are adaptive algorithms that select tasks or data based on real-time system states and evolving resource demands.
• They leverage surrogate models, meta-scheduling, and bi-level optimization to reduce operational costs, energy use, and improve performance.
• Key applications include dynamic curriculum learning for ML training, data-aware inference serving on the edge, and randomized scheduling in large-scale systems.

Dynamic Data Selection Schedulers constitute a broad class of scheduling and control algorithms that dynamically allocate, reweight, or select data—be it tasks, training batches, or inference requests—according to evolving system state, model feedback, or data utility. While rooted in classical scheduling and resource allocation, contemporary developments encompass deep neural network surrogates, bi-level optimization, control-theoretic adaptations, and surrogate-guided meta-selection spanning machine learning, cloud systems, reinforcement learning, and distributed control. This survey synthesizes recent methodologies, architectural paradigms, and empirical advances in dynamic data selection scheduling, with technical focus on surrogate-based meta-scheduling in cloud environments, dynamic curriculum/sampling for ML model training, and data-aware model selection for inference serving.

1. Foundational Problem Statements and Taxonomy

Dynamic data selection scheduling problems arise in non-stationary environments where jobs, data, tasks, or inputs arrive over time, and resource constraints, model capabilities, or workload statistics may change unpredictably. The central challenge is to dynamically select among candidate policies, schedules, or data subsets to optimize a composite objective—typically balancing execution cost, resource utilization, accuracy, model improvement, and computational overhead. Formally, given

• an incoming data stream $J_t$ or set of jobs/tasks/inputs at interval $t$,
• a finite set $\mathcal P = \{p^1, \ldots, p^q\}$ of candidate schedulers or selection algorithms,
• feature representations of system state $W_t$ (resource demand), $H_t$ (host/utilization), $S_t$ (history, scheduling graph) or input features, the objective is to select, at each decision epoch $t$, a policy $s^*_t$ solving

$$s_t^* = \arg\min_{s \in \mathcal P} \Big[ C_s(W_t) + \lambda\, h_s(W_t) \Big]$$

where $C_s$ denotes operational cost (e.g., VM-hours, energy, or loss) and $h_s$ the scheduling/selection overhead (Tuli et al., 2022, Tuli et al., 2022).

Dynamic data selection scheduler families in the literature encompass:

• Surrogate-based meta-schedulers using neural or simulation surrogates for resource management and cost balancing (Tuli et al., 2022, Tuli et al., 2022, Mohammed et al., 2019).
• Dynamic curriculum/sampling schedulers for adaptive data selection in model training, leveraging task-based, loss-based, or bi-level optimization (Wang et al., 2019, Yu et al., 22 Jul 2025, Rao et al., 22 May 2025).
• Priority and data-aware schedulers for inference serving, integrating context-conditioned utility or per-request predictions (Wolfrath et al., 10 May 2025).
• Task allocation in distributed systems with randomness and data locality, using sampling-based scheduling to reduce complexity under large server populations (Moaddeli et al., 2019).
• Real-time neuroscheduling for remote estimation, where each data selection is optimized adaptively for estimation fidelity under sample or transmission constraints (Vasconcelos et al., 17 May 2024).

2. Surrogate-Driven Meta-Scheduling and Policy Selection

A key advance in dynamic scheduling is the use of surrogate models that efficiently predict the cost and overhead of candidate scheduling policies without exhaustively executing all of them. The MetaNet paradigm (Tuli et al., 2022, Tuli et al., 2022) formalizes this approach for cloud task scheduling. At each scheduling interval, a deep neural network surrogate $f_\theta$ ingests a compact system summary (concatenation of task workloads $W_{t-1}$, host utilization $H_{t-1}$, and scheduling graph $S_{t-1}$), and outputs normalized predictions $\{ \hat{C}_s, \hat{h}_s \}_{s=1}^q$ for all policies. The scheduler chooses the policy minimizing the predicted composite cost. Training is supervised: collected logs of true (cost, overhead) pairs for each scheduler are normalized and used to drive a joint $L_2$ loss.

Surrogate-based scheduling enables:

• Rapid adaptation to workload regime shifts through retraining or online fine-tuning.
• Significant cost and energy reductions: MetaNet achieved up to 11% lower execution cost, 43% lower energy, and 13% fewer SLA violations versus leading single-policy methods in Azure VM clusters (Tuli et al., 2022, Tuli et al., 2022).
• Generality: the surrogate framework is applicable wherever selection among heterogeneous algorithmic policies with differing tradeoffs is needed, beyond VM placement—for example, in database query planning or model selection pipelines.

MetaNet's architectural advances—including dual heads for decoupled prediction of quality and overhead, explicit encoding of graph-structured scheduling history, and cross-modal attention for heterogeneous state aggregation—were confirmed essential via ablation studies (Tuli et al., 2022).

SimAS (Mohammed et al., 2019) extends this concept to high-performance computing, using simulation-assisted model-predictive control for real-time loop self-scheduling. Here, a portfolio of dynamic loop self-scheduling (DLS) methods is simulated on-the-fly under current measured system perturbations, and the method yielding the minimum predicted makespan is chosen. SimAS consistently outperformed fixed single-schedulers across a wide range of imbalanced workloads and fluctuating CPU/network scenarios.

3. Dynamic Curriculum, Adaptive Sampling, and Bi-level Optimization in ML Training

Dynamic data selection is also fundamental in training deep neural models, where evolving model capabilities, data heterogeneity, and resource constraints necessitate adaptive curricula. Dynamic Curriculum Learning (DCL) (Wang et al., 2019) developed a two-level scheduler: the sampling scheduler adjusts class and difficulty distributions over training epochs, transitioning from the natural data distribution to a balanced or hard-sample-focused regime. Mathematically, the batch target distribution at epoch $l$ is a power $g(l)$ of the original distribution, with the schedule $g(l)$ typically convex-decreasing (e.g., $SF_{\cos}(l)$). This gradually shifts sampling from majority/easy to minority/hard examples. DCL's scheduler achieves a standalone gain of +5.41% mean accuracy on CelebA (face attribute classification) compared to static sampling (Wang et al., 2019).

For modern LLMs, dynamic bi-level optimization offers a principled scheduler. The Data Weighting Model (DWM) (Yu et al., 22 Jul 2025) is trained via a bi-level objective: at each iteration, sample weights are computed to optimize not just the immediate training loss but their effect on downstream validation loss. DWM utilizes a Llama-2-style transformer as a batch encoder, with multi-head attention between samples and a softmax scoring head, solving for weights via meta-gradient on validation loss after one step of gradient update on the weighted batch. This pipeline yields improvements of +1.0–1.3% absolute zero/few-shot accuracy on 9 tasks, with benefits most pronounced in later training stages and robust transfer across LLM sizes.

Self-aware, feedback-driven sampling schedulers—including SAI-DPO (Rao et al., 22 May 2025)—dynamically adapt data selection to match the evolving strengths and weaknesses of an RLHF or DPO-trained model. SAI-DPO clusters data by knowledge points, uses probe batches to assess model-specific error rates, then adaptively up-weights sampling for clusters or instances on the current learning frontier. The resulting scheduler achieves up to 21.3 percentage point improvement over strong static curricula on competition-level math benchmarks.

4. Data-Aware and Input-Conditional Scheduling for Inference Serving

Dynamic model and data selection schedulers are increasingly critical for edge inference serving, where bandwidth, memory, and deadlines are strict. The SneakPeek system (Wolfrath et al., 10 May 2025) introduces piggyback neural predictors estimating the per-input probability that each candidate model will predict correctly. For each incoming request $x_i$ and possible model $m_j$, the scheduler quickly computes $\hat a_{i,j}$ (predicted accuracy), combines it with latency, deadline slack, and resource constraints into a priority score, and greedily batches to maximize expected utility. On a 4 GB Jetson Nano, SneakPeek improved utility by 45% and reduced average latency by 28% relative to profile-based static model selection, while retaining nearly all achievable accuracy.

Key structural innovations:

• Per-request per-model accuracy prediction as opposed to global average accuracy, mitigating bias from heterogeneous data.
• Explicit tradeoff scheduling balancing predicted utility against real-time constraints, with tunable $\alpha/\beta$ for accuracy vs. deadline priorities.
• Greedy, batch-aware batching and model-eviction strategies that reduce model swap overhead, with demonstrated efficacy across image and speech tasks.

5. Randomized, Partitioned, and Sampling-Based Dynamic Schedulers for Large Systems

In data center-scale systems with heterogeneous servers and massive parallelism, complexity constraints motivate randomized and partitioned data selection schedulers. Balanced-Pandas-Pod (BP-Pod) (Moaddeli et al., 2019) combines robust queue-based server selection with "power-of-$d$-choices" routing. Each task is always considered for its 3 data-local servers and $d$ additional randomly sampled servers among thousands, reducing routing time from $O(M)$ to $O(1)$. BP-Pod achieves strict empirical improvement over the full Balanced-Pandas algorithm at low and medium loads, matches it at high load, yet is far less sensitive to the hyperparameter $d$ and dramatically reduces central scheduler complexity.

• Throughput-optimality and heavy-traffic optimality are retained with only minimal performance loss in worst-case regimes.
• Graceful locality-performance tradeoff: always include local servers but leverage random choice for load-peak mitigation.
• This architecture generalizes to other large-scale selection problems with rapid dynamic feedback needs and massive candidate sets.

6. Neuroscheduling and Nonlinear Data-Driven Selection Under Communication Constraints

Dynamic data selection scheduling under nonlinear system objectives and explicit observation constraints is exemplified in the neuroscheduling framework for remote estimation (Vasconcelos et al., 17 May 2024). Here, a device with access to multiple jointly distributed sources must dynamically select which observation to transmit under a strict per-interval budget. The optimal scheduler-estimator pair is shown to be nonlinear (even for the bivariate Gaussian case), and is learned via data-driven optimization of a min-cost surrogate over linear-in-parameters basis families. The framework specifies necessary and sufficient function-class conditions to avoid overfitting (e.g., requiring basis functions with non-vanishing derivatives almost everywhere).

Empirical validation underscores significant performance gains over both fixed and linear person-by-person schedulers, as well as the centrality of basis choice and overfitting-avoidance guarantees.

7. Common Architecture, Open Problems, and Future Directions

Common threads among dynamic data selection schedulers include:

• Framing scheduler selection/data weighting as an online or meta-decision process, guided by real-time system or model feedback.
• Surrogate or simulation-assisted prediction of downstream utility, latency, or cost, enabling tractable policy selection in high-dimensional heterogeneous environments.
• The joint optimization and adaptation of both scheduler (meta-policy) and task/model-specific parameters, frequently via bi-level, control-theoretic, or feedback-loop strategies.

Significant open challenges persist:

• Expanding theoretical understanding of convergence, stability, and efficiency in nonstationary, adversarial, or coupled selection processes.
• Extending surrogate and feedback-driven selection to broader classes of tasks, such as multimodal or multi-agent control, and developing lightweight, accurate surrogates for extremely large action spaces.
• Overcoming limitations in model and system profiling, surrogate transferability, and adapting to rapid regime shifts without excessive retraining cost.
• Tighter integration of dynamic data selection schedulers within cross-layer or federated orchestration, where tradeoffs span from edge devices through to cloud or HPC resources.

A plausible implication is that as complexity, scale, and heterogeneity of computational systems and ML workflows continue to increase, dynamic data selection schedulers—anchored by surrogate prediction, feedback adaptation, and control-theoretic integration—will become a central architectural component for robust, cost-efficient, and high-performing data and resource management across domains.

---

References:

• "Learning to Dynamically Select Cost Optimal Schedulers in Cloud Computing Environments" (Tuli et al., 2022)
• "MetaNet: Automated Dynamic Selection of Scheduling Policies in Cloud Environments" (Tuli et al., 2022)
• "LLM Data Selection and Utilization via Dynamic Bi-level Optimization" (Yu et al., 22 Jul 2025)
• "Dynamic Curriculum Learning for Imbalanced Data Classification" (Wang et al., 2019)
• "Dynamic Sampling that Adapts: Iterative DPO for Self-Aware Mathematical Reasoning" (Rao et al., 22 May 2025)
• "SneakPeek: Data-Aware Model Selection and Scheduling for Inference Serving on the Edge" (Wolfrath et al., 10 May 2025)
• "The Power of d Choices in Scheduling for Data Centers with Heterogeneous Servers" (Moaddeli et al., 2019)
• "Neuroscheduling for Remote Estimation" (Vasconcelos et al., 17 May 2024)
• "SimAS: A Simulation-assisted Approach for the Scheduling Algorithm Selection under Perturbations" (Mohammed et al., 2019)