Predictive Multi-Task Scheduling (PREMA)

• PREMA is a method that integrates predictive models with dynamic multi-task scheduling to optimize objectives such as latency, throughput, and model performance.
• It employs analytical, deep learning, and meta-learning techniques to forecast task features and allocate resources efficiently.
• PREMA has demonstrated measurable improvements in sequence learning, cloud resources, and HPC workflows, validating its practical impact across multiple domains.

Predictive Multi-Task Scheduling (PREMA) encompasses a class of methods that combine task and resource scheduling with predictive models over task characteristics and system state, in order to optimize objectives such as latency, throughput, or model performance in diverse problem domains. PREMA formalizes the joint exploitation of forecasted or learned task features and adaptive scheduling, applying this methodology across areas such as sequence learning, deep neural inference on specialized hardware, and high-performance distributed computing. Distinct instantiations include meta-scheduling for sequence modeling (Wu et al., 2020), preemption-aware cloud resource scheduling (Choi et al., 2019), and deep-learning-driven reservation-based workflow scheduling (Gritsenko, 2013).

1. Formal Problem Characterization

PREMA formulations share a stochastic or adaptive scheduling framework that integrates explicit prediction of task attributes. The core structure consists of:

• Task set: A collection $\mathcal{T} = \{T_1, \ldots, T_M\}$, where each $T_m$ may correspond to a temporal horizon, a user request, or an auxiliary learning objective.
• Resource model: Tasks compete for one or more resources (HPC nodes (Gritsenko, 2013), NPUs (Choi et al., 2019), parameterized neural models (Wu et al., 2020)) with constraints such as exclusivity, tokenized access, or backfilling policies.
• Predictive component: Each task $i$ is assigned features predicted by analytical models, empirical statistics, or neural predictors (e.g., execution time $\hat p_i$, validation loss, or empirical recurrence).
• Scheduling objective: Multi-criteria functions, e.g., maximizing target task performance (Wu et al., 2020), minimizing weighted normalized turnaround (Choi et al., 2019), or combining makespan, utilization, slowdown (Gritsenko, 2013).

In sequence learning, the tasks are temporally correlated auxiliary and main objectives, with scheduling governed by meta-learning over the joint task/model state (Wu et al., 2020). In inference-system PREMA, the scheduler predicts per-task processing time and dynamically arbitrates with preemptions to satisfy SLA and fairness constraints (Choi et al., 2019). In distributed computing, the scheduler forecasts future workflow structure and realizes reservations via a confidence-adaptive backfilling approach (Gritsenko, 2013).

2. Predictive Modeling and Feature Extraction

PREMA methodologies rely on tailored predictive models:

• Analytical and profile-based prediction (DNN inference): Execution time estimators combine analytic expressions for tiled operations (GEMM, convolution) with model profile statistics for unroll lengths; achievable with sub-2% error and >98% correlation to empirical times, ensuring robust response to variable workloads (Choi et al., 2019).
• Deep multi-layer periodic pattern extraction (HPC workflows): Historic job sequences are decomposed into recurring patterns by period and resource signature; future arrivals are extrapolated patternwise and equipped with a confidence score $\varphi_k$ derived from empirical likelihood of recurrence (Gritsenko, 2013).
• Low-dimensional meta-features for sequence task scheduling: Features include input/output length ratios, model state (training and validation loss history), and training progress, facilitating conditional task allocation in multi-task learning (Wu et al., 2020).

Task selection or reservation is then conditioned upon these predictions, with downstream scheduling policies explicitly linked to forecast fidelity.

3. Scheduler Algorithm Design and Training

Scheduler mechanisms vary with application:

• Bi-level optimization with learnable scheduler (sequence learning): The scheduler, a small MLP $\varphi(I_{x,y,\theta};\omega)$, outputs a categorical distribution over tasks based on meta-features. Model parameters $\theta$ and scheduler weights $\omega$ are updated jointly in an alternating loop: the model by inner-loop minimization of expected loss under $\varphi$, the scheduler by REINFORCE using validation-based reward shaping. No auxiliary regularization or entropy penalty is imposed; all improvements stem from meta-learning over validation returns (Wu et al., 2020).
• Token-based, preemption-aware dispatchers (NPU inference): Each task receives tokens scaled by priority and wait time; selection proceeds via thresholding and shortest-predicted-job-first within candidates. Upon selection, a mechanism (CHECKPOINT vs. DRAIN) is dynamically chosen by comparing degradation metrics on current and candidate tasks, balancing throughput and latency (Choi et al., 2019).
• Heuristic event-driven backfilling with prediction-based reservations (distributed systems): Predicted jobs are classified (ignored, soft, or hard reservation) based on confidence thresholds $(\tau_{\text{low}}, \tau_{\text{high}})$; the runtime RMS executes job arrivals and completions event-wise, admitting backfilling subject to reservation integrity. Thresholds adapt online based on empirical reservation success (Gritsenko, 2013).

No single objective function is universal; metrics are kept modular and algorithms are often evaluated via multi-criteria ranking frameworks.

4. Evaluation Domains and Quantitative Results

PREMA performance has been validated in several domains:

• Simultaneous machine translation: PREMA, applied to wait-$k$ sequence tasks, demonstrates BLEU improvements over both static and adaptive baselines (e.g., increase from 28.54 to 29.01 BLEU for wait-3 En$\rightarrow$Vi on IWSLT’15). PREMA-trained models also enable further gains when combined with external techniques (Wu et al., 2020).
• Stock trend forecasting: RankIC and MSE are improved for all tasks under PREMA, with clear dominance over MTL, CL, LightGBM, and GAT baselines. RankIC for PREMA(T$_5$) reaches 0.075, with MSE reduced to 1.849 (Wu et al., 2020).
• NPU inference servers: PREMA reduces average normalized turnaround (ANNT) by $7.8\times$ over non-preemptive FCFS, increases system throughput ($1.4\times$), and reduces SLA violations to below 10% (from 36%), with only minor tail-latency increase (Choi et al., 2019).
• Academic HPC task scheduling: The deep-learning reservation scheduler outperforms queue-based approaches (e.g., preference score 0.1468 vs 0.0535 for Cons-BF on Zewura traces), approaching offline schedule-based benchmarks (Gritsenko, 2013).

Representative experimental settings, model architectures, and hyperparameters are domain specific and reported in detail in the corresponding studies.

5. Key Insights and Implementation Considerations

PREMA methods yield several actionable insights:

• Meta-scheduling using temporally or structurally correlated tasks consistently outperforms uniform sampling and hand-crafted curricula when features describing model/data/training status are included (Wu et al., 2020).
• Lightweight, analytic task-length prediction combined with preemptible hardware constructs achieves near-optimal scheduling without requiring deep sequence models, provided error remains moderate (Choi et al., 2019).
• Practical implementations of PREMA incur modest computational overhead. The majority comes from the meta-feature computation (especially at runtime), which can be selectively simplified in resource-constrained scenarios (Wu et al., 2020).
• Effectiveness is robust across application domains, including language, finance, HPC, and DNN inference systems. The approach generalizes to settings such as speech streaming, weather prediction, and sequential curriculum learning (Wu et al., 2020, Choi et al., 2019, Gritsenko, 2013).
• Parameter tuning is critical: For bi-level meta-schedulers, the scheduler learning rate typically must be substantially lower than model parameters; length and structure of episodes/training steps trade off bias and variance in reward estimation (Wu et al., 2020).

6. Limitations and Extensions

PREMA schemes—while effective—impose several structural limitations:

• Assumption of recurring patterns: For reservation-based workflow scheduling, periodicity or recurrence in workload is necessary; non-periodic or highly bursty workloads remain challenging (Gritsenko, 2013).
• Homogeneous resource and preemption models: Current implementations assume single-type (or lightly heterogeneous) resource pools and limited migration or inter-node orchestration capabilities (Choi et al., 2019).
• Overhead of prediction and meta-feature extraction: In high-throughput, latency-sensitive systems some input features (such as on-the-fly loss computations) may become bottlenecks (Wu et al., 2020).

Suggested extensions include direct support of heterogeneous clusters, incorporation of complex preemption and migration policies, more sophisticated machine learning predictors (e.g., real-time LSTM for sequence forecast), and integration of energy cost models (Choi et al., 2019, Gritsenko, 2013).

7. Concluding Synthesis

PREMA demonstrates that integrating task attribute prediction into dynamic multi-task scheduling frameworks yields substantial, measurable benefits across disparate computational domains. Core ingredients—shared parameterization, feature-driven scheduling distributions, bi-level or tokenized learning loops, and reward-based adaptation—are domain agnostic but realization-specific. PREMA generalizes easily by defining temporally or structurally correlated task families, sharing core network or resource representations, and learning a scheduling distribution that conditions on real-time task and model state. When implemented, these methods consistently outperform static scheduling, uniform MTL, and handcrafted curricula in both model-centric and system-centric settings (Wu et al., 2020, Choi et al., 2019, Gritsenko, 2013).