A Multi-Stage Warm-Start Deep Learning Framework for Unit Commitment

Abstract: Maintaining instantaneous balance between electricity supply and demand is critical for reliability and grid instability. System operators achieve this through solving the task of Unit Commitment (UC),ca high dimensional large-scale Mixed-integer Linear Programming (MILP) problem that is strictly and heavily governed by the grid physical constraints. As grid integrate variable renewable sources, and new technologies such as long duration storage in the grid, UC must be optimally solved for multi-day horizons and potentially with greater frequency. Therefore, traditional MILP solvers increasingly struggle to compute solutions within these tightening operational time limits. To bypass these computational bottlenecks, this paper proposes a novel framework utilizing a transformer-based architecture to predict generator commitment schedules over a 72-hour horizon. Also, because raw predictions in highly dimensional spaces often yield physically infeasible results, the pipeline integrates the self-attention network with deterministic post-processing heuristics that systematically enforce minimum up/down times and minimize excess capacity. Finally, these refined predictions are utilized as a warm start for a downstream MILP solver, while employing a confidence-based variable fixation strategy to drastically reduce the combinatorial search space. Validated on a single-bus test system, the complete multi-stage pipeline achieves 100\% feasibility and significantly accelerates computation times. Notably, in approximately 20\% of test instances, the proposed model reached a feasible operational schedule with a lower overall system cost than relying solely on the solver.

• The paper introduces a multi-stage framework that couples transformer predictions with deterministic post-processing and MILP warm-starts to generate physically feasible schedules.
• The method achieves near-optimal cost performance and 100% feasibility with significant runtime acceleration compared to traditional MILP solvers.
• Empirical validation on a 72-hour, 51-generator test case demonstrates the approach's scalability and potential for real-time grid operations.

Multi-Stage Warm-Start Deep Learning for Unit Commitment: A Technical Overview

Problem Context and Motivation

The Unit Commitment (UC) problem is a high-dimensional, temporally-coupled MILP used by grid operators to determine generator schedules that minimize cost while maintaining operational feasibility. Traditional MILP solvers are computationally intensive and do not scale well under increasing horizon lengths and system heterogeneity, a situation exacerbated by the rapid adoption of intermittent renewables, storage, and decarbonization targets. The practical need to solve multi-day UC with increased dispatch frequency compels new algorithmic approaches that balance cost, feasibility, and tractability.

Proposed Multi-Stage Framework

This work introduces a multi-stage framework coupling deep sequence modeling, post-processing heuristics, and MILP solver warm-start strategies. The central objective is to produce physically feasible and cost-competitive generator schedules for 72-hour horizons with significant acceleration.

The pipeline includes three sequential stages:

1. Deep Learning Prediction: A transformer network predicts generator on/off schedules conditioned on exogenous load and renewables.
2. Feasibility Restoration: Deterministic post-processing rules ensure constraints like minimum up/down, ramp, and generation sufficiency are strictly enforced.
3. Warm-Started MILP: Refined predictions serve as a warm-start point for a downstream MILP solver, with high-confidence variables fixed to reduce combinatorial complexity.

Transformer-Based Generator Scheduling

The model employs a transformer architecture with multi-head self-attention over the 72-hour horizon, explicitly targeting UC’s temporal dependencies where classical architectures (LSTMs, GRUs, feedforward NNs) lose efficacy. A zero-initialized, learnable positional embedding replaces sinusoidal schemes, allowing the network to autonomously internalize temporal periodicities characteristic of demand and renewable profiles. The model architecture is as follows:

• Input Encoding: $[B, T, F]$ sequence with normalized load, solar, and wind.
• Latent Transformation: Linear embedding to $d_{model}$, addition of learned position embedding.
• Encoder Block: 8 transformer layers, Pre-LN, 64 attention heads ($d_k = d_{model}/h$).
• Classification Head: Projects encoded representation to $[B, N_g, T]$ logit tensor, binarized by tunable threshold $\tau$.
• Training Loss: Class-weighted BCE, $\alpha=1.5$ for under-commit penalties, AdamW optimizer.

This design facilitates direct mapping of exogenous operational context to actionable generator schedules, maintaining tractability as system and horizon scale.

Deterministic Post-Processing Heuristics

Direct neural predictions frequently violate hard physical constraints due to statistical error accumulation in high-dimensional binary decisions. The framework addresses this via three post-processing heuristics:

1. Surgical Repair: Enforces generator minimum up/down time by adjusting contiguous 'on' and 'off' blocks and sequentially turns online the cheapest units to meet net load and ramping needs at each timestep.
2. Excess Generation Minimization: Iteratively decommits most expensive generators, with feasibility verified by a fast economic dispatch LP, ensuring excess capacity is penalized.
3. Head/Tail Trim: Prunes generator startup/shutdown by analyzing utilization at block edges and selectively decommitting low-utilization segments.

Together, these rules systematically restore hard constraint feasibility, as shown in the visualization of generator schedule correction:

(Figure 1)

Figure 1: Predicted generator statuses on a sample. Grey indicates generators turned off by post-processing, light blue indicates generators turned on by post-processing.

MILP Warm-Starting and Search Space Reduction

Refined schedules are used to warm-start the MILP solver, either as soft hints (non-fixed) or with high-confidence variables fixed based on the model's prediction certainty. This reduces effective search space and brings computational savings without sacrificing solution quality. The pipeline further permits threshold-based tuning for variable fixation, offering a control knob for the tradeoff between exactness and solution time.

Empirical Validation and Comparative Results

Validation leveraged the EPRI ‘AI-ccelerating Unit Commitment’ challenge dataset (single-bus system, 72-hour horizon, 51 generators), plus large-scale synthetic data augmentation for robust generalization. Six model variants (M1-M6) were ablated for the presence of post-processing, warm-start, and fixed-variable thresholds. Comparative baselines included a standalone MILP (HiGHS), LSTM, and a simple neural network.

Results are summarized with core performance metrics:

• Optimality Ratio (O.R.): Operational cost relative to MILP baseline.
• Feasibility Rate: Percentage of physically admissible solutions.
• Time Ratio (T.R.): Runtime normalized to MILP baseline.

Key findings include:

• Raw transformer predictions (M1) are high-accuracy but low-feasibility (31.6%).
• Only with post-processing (M2) or warm-start (M3–M6) does feasibility approach or reach 100%.
• The full multi-stage model (M6), with all stages enabled, achieves 100% feasibility, lowest median cost increase (O.R. median ≈ 1.0007), and significant runtime acceleration (T.R. ≈ 1.46% of MILP alone).
• In ≈20% of cases, the ML-warmstarted pipeline discovers schedules with lower cost than the MILP solver with a 0.25% MIP gap.
• Comparative NN and LSTM baselines achieve greater speed-ups but at the expense of higher suboptimality.

The distribution of optimality ratios for all models, when restricted to feasible cases, is visualized as follows:

(Figure 2)

Figure 2: Distribution of optimality ratios for models M1–M6, capped at 1.01, only for feasible samples.

Further, main effects analyses quantify the isolated impact of each pipeline stage on feasibility, optimality, and runtime. Warm-starting the MILP brings most rapid feasibility and cost improvements, whereas post-processing delivers strict constraint compliance. Fixing variables further accelerates solver convergence.

Implications and Future Outlook

The demonstrated multi-stage framework materially advances neural-enhanced UC operations, with the validation indicating that transformer-based sequence models, when judiciously coupled with deterministic repairs and targeted solver interaction, realize the dual goals of physical feasibility and computational tractability. The framework bridges the gap between fast, but unconstrained, ML predictors and slow, but robust, combinatorial optimization solvers.

From a practical perspective, such pipelines could enable ISOs and market operators to handle finer scheduling granularity, integrate volatile stochastic renewables, and support grid-level resilience studies on expanded timescales. Theoretical implications include expanding the paradigms of "learning to optimize" over large, temporally-coupled, constraint-dominated combinatorial problems. However, it must be emphasized that the validation assumes perfect forecasts and neglects network (transmission) constraints. Generalization to multi-bus, N-1 secure systems, and stochastic or risk-aware formulations remains an open research avenue.

Conclusion

This work establishes that a transformer-driven, post-processed, and MILP-warmstarted pipeline can achieve strictly feasible, near-globally optimal UC schedules for large, long-horizon test cases while dramatically reducing computational load. These results are a substantive benchmark for subsequent work on learning-augmented discrete optimization in the power systems context and more generally across technical domains requiring real-time, reliable combinatorial decision-making (2604.21891).