Forecasting Intra-Hour Imbalances in Electric Power Systems (1902.00563v1)

Abstract: Keeping the electricity production in balance with the actual demand is becoming a difficult and expensive task in spite of an involvement of experienced human operators. This is due to the increasing complexity of the electric power grid system with the intermittent renewable production as one of the contributors. A beforehand information about an occurring imbalance can help the transmission system operator to adjust the production plans, and thus ensure a high security of supply by reducing the use of costly balancing reserves, and consequently reduce undesirable fluctuations of the 50 Hz power system frequency. In this paper, we introduce the relatively new problem of an intra-hour imbalance forecasting for the transmission system operator (TSO). We focus on the use case of the Norwegian TSO, Statnett. We present a complementary imbalance forecasting tool that is able to support the TSO in determining the trend of future imbalances, and show the potential to proactively alleviate imbalances with a higher accuracy compared to the contemporary solution.

• The paper introduces a Quantile Regression Forests model to forecast intra-hour power imbalances using 5-minute interval data.
• It treats each future time step and market area as an independent univariate forecasting problem to achieve granular predictions.
• The approach outperforms existing tools by delivering reliable 95% prediction intervals, supporting proactive reserve decisions.

This paper addresses the challenge of forecasting intra-hour imbalances in electric power systems, specifically focusing on the needs of the Norwegian Transmission System Operator (TSO), Statnett (1902.00563). The core problem is the increasing difficulty and cost of balancing electricity supply and demand in real-time, exacerbated by intermittent renewable energy sources and limitations in market mechanisms which operate on hourly resolutions. Deviations between planned and actual supply/demand lead to imbalances, causing frequency fluctuations that require costly interventions using balancing reserves.

The existing tool used by Statnett, the "Planning Table," relies on scheduled production and load forecasts but suffers from inaccuracies due to assumptions about market balance and generator/demand adherence to plans. This makes proactive management of imbalances difficult, leading to increased use of expensive reserves.

Proposed Solution: Quantile Regression Forests (QRF)

To improve upon this, the paper proposes a complementary forecasting tool based on Quantile Regression Forests (QRF). QRF is an extension of Random Forests (RF) that provides not only point predictions (conditional mean) but also prediction intervals (quantiles), offering an assessment of forecast reliability crucial for decision support.

• Objective: Predict the average imbalance (defined as Area Control Error Open Loop, ACE OL) for the next two hours at a 5-minute granularity. This involves predicting 24 future values ($I_{\alpha, t+1}, ..., I_{\alpha, t+24}$) for each of Norway's five market areas ($\alpha \in \{NO1, ..., NO5\}$).
• Modeling Strategy: Instead of a single complex model, the authors treat the prediction for each of the 24 future time steps ($\Delta$) and each market area ($\alpha$) as an independent univariate forecasting problem. This results in $5 \times 24 = 120$ separate QRF models.
• Why QRF?: Chosen for its ability to intrinsically generate prediction intervals, robustness to overfitting and noise, and relatively low sensitivity to hyperparameter tuning, which simplifies maintenance and retraining.

Implementation Details

• Data: The models were trained and tested using two years (2015-2016) of historical imbalance data provided by Statnett, sampled every 5 minutes.
• Feature Engineering: Due to data limitations (lack of decomposed imbalance contributors like consumption forecast errors or production deviations), the feature set was focused primarily on historical imbalances and temporal information. Through manual engineering and selection, the final feature set included:
  • 24 previous imbalance values relative to the current imbalance ($I_{\alpha, t - \delta}$ relative to $I_{\alpha, t + 0}$, for $\delta \in \langle1, 24\rangle$).
  • The current absolute imbalance value ($I_{\alpha, t + 0}$).
  • Temporal features: month, day of week, hour, minute, sine/cosine representations of time features, a holiday flag, and solar elevation.
• Target Variable: The models predict future imbalances also as values relative to the current imbalance ($I_{\alpha, t + \Delta}$ relative to $I_{\alpha, t + 0}$).
• Training: Models were trained using a rolling window approach: train on 12 months of data and test on the subsequent month, then retrain. Hyperparameters were set consistently across models (e.g., 100 trees, minimum leaf size 10) after initial tuning showed minimal gains from fine-grained optimization per model.
• Software: The Scikit Garden library implementation of QRF was used.

import pandas as pd from skgarden.quantile import QuantileForestRegressor from sklearn.model_selection import train_test_split df_no1 = df[df['market_area'] == 'NO1'].copy() features = [ 'imbalance_t_minus_1_rel', 'imbalance_t_minus_2_rel', # ... up to 24 lags 'imbalance_t_plus_0_abs', 'month', 'dayofweek', 'hour', 'minute', # ... other temporal features 'solar_elevation' ] target = 'imbalance_t_plus_1_rel' # Predicting the imbalance 5 minutes ahead, relative to t+0 X = df_no1[features] y = df_no1[target] qrf = QuantileForestRegressor( n_estimators=100, min_samples_leaf=10, n_jobs=-1, # Use all available cores random_state=42 # max_features=0.7 (not directly available, but RF part has it) )

Evaluation and Results

The proposed QRF models were evaluated against two benchmarks:

1. Naive: Predicts imbalance will be the same as the previous week.
2. Planning Table (proxy): An enhanced version of Statnett's tool, using actual consumption data (as forecast data was unavailable). This benchmark was only available for a subset of the test period and prediction horizon ($\Delta \in \langle1, 7\rangle$).

Metrics used were Mean Squared Error (MSE), Mean Absolute Error (MAE), and Coverage Probability (CP) for the 95% prediction intervals. The results showed:

• The QRF model consistently outperformed both benchmarks in terms of MSE and MAE across nearly all market areas and prediction horizons ($\Delta$).
• Performance gains over the Planning Table proxy were particularly significant in areas NO2, NO3, NO4, and NO5.
• Prediction accuracy decreased as the forecast horizon ($\Delta$) increased, as expected.
• The 95% prediction intervals achieved coverage probabilities close to the nominal 95% level across all predictions, indicating reliable uncertainty estimates.

Practical Implications and Limitations

• The research demonstrates that even without incorporating explicit forecasts or plans (like generation schedules or load forecasts), models leveraging historical imbalance patterns and temporal features can provide valuable short-term predictions.
• The tool is intended as a complementary decision support system for human operators, providing an alternative view focused on historical patterns, rather than a replacement for the Planning Table which incorporates forward-looking planned data.
• Its main benefit lies in potentially helping operators make more informed decisions about activating manual reserves (mFRR) proactively, improving system stability and reducing costs.
• A key limitation is the model's inability to account for known, planned future events (e.g., large changes in production or HVDC flows) since these features were not included. This reinforces its role as a complementary tool.
• The authors note that weather data did not significantly improve performance in their setup, hypothesizing it might become relevant if combined with consumption forecasts or in systems with higher shares of weather-dependent renewables.
• The paper highlights the importance of collaboration with the TSO (Statnett) for data access, domain expertise, and potential deployment in the control room for qualitative evaluation by operators.

Future work includes incorporating planned data (consumption forecasts, production plans) into the models and deploying the prototype system in Statnett's control room.