M4-Instruct Forecasting Framework

• M4-Instruct is a forecasting framework that fuses a five-method statistical ensemble with a cross-series correlator to produce robust 14-day daily forecasts.
• Its methodology leverages local segment matching and affine alignment to extract high-fidelity continuations when strict correlation and variability criteria are met.
• Retrospective analysis reveals that its improved performance was largely driven by data leakage and quality issues, underscoring the need for rigorous diagnostic controls.

Searching arXiv for papers on the M4 competition and the correlator-based daily forecasting method. M4-Instruct denotes a distilled forecasting framework for the M4 Daily task derived from the method study in "Correlated daily time series and forecasting in the M4 competition" (Ingel et al., 2020). In this usage, it refers to a daily point-forecasting procedure for the 14-day horizon that combines a five-method statistical ensemble with a cross-series "correlator" that searches for highly similar 14-day segments in other series, aligns their continuations by an affine transform, and uses the matched continuation whenever an eligibility criterion is satisfied. Its central methodological significance lies less in the individual baselines than in the interaction between local segment matching, evaluation under M4 metrics, and the paper’s subsequent demonstration that much of the correlator’s apparent success was amplified by data leakage and data-quality defects.

1. Problem setting and evaluation protocol

The M4 competition required point forecasts for 100,000 series across multiple frequencies. The Daily category contained 4,227 series, each with one value per day, lengths ranging from 93 to 9,919 days, and forecast horizon $h = 14$ days (Ingel et al., 2020). M4 evaluation used sMAPE and MASE, and ranked submissions by the Overall Weighted Average (OWA), defined relative to the seasonal naive benchmark Naive2.

The metrics are:

$$\operatorname{sMAPE} = \frac{1}{h} \sum_{t=1}^{h} \frac{2\lvert F_t - Y_t \rvert}{\lvert F_t \rvert + \lvert Y_t \rvert}$$

$$\operatorname{MASE} = \frac{\frac{1}{h} \sum_{t=1}^{h} \lvert F_t - Y_t \rvert}{\frac{1}{n - m} \sum_{t=m+1}^{n} \lvert Y_t - Y_{t-m} \rvert}$$

$$\operatorname{OWA} = \frac{1}{2}\left( \frac{\operatorname{sMAPE}_{\text{model}}}{\operatorname{sMAPE}_{\text{Naive2}}} + \frac{\operatorname{MASE}_{\text{model}}}{\operatorname{MASE}_{\text{Naive2}}} \right).$$

For daily data, $m$ is commonly taken as $7$ to reflect weekly seasonality when calendar dates are available. The paper’s retrospective tables also report relative metrics versus a simple Naive constant forecast, defined as the last observed value, but the official M4 OWA is relative to Naive2. This distinction is methodologically important because the paper’s strongest numerical comparisons for the daily subset often use the simpler reference, whereas competition ranking did not.

2. Statistical baseline ensemble

M4-Instruct uses five per-series statistical methods, trained with an internal validation scheme that withholds the last 14 days of the training series. The ensemble is not weighted; instead, it takes the per-horizon median across the five forecasts, which is intended to be robust to outlying base-model behavior (Ingel et al., 2020).

Method	Formulation	Key settings
Naive	Constant forecast	$F_t$ equals the last observed value
ETS	Innovations state space	forecast::ets with defaults
ARIMA 1	Automatic stepwise ARIMA	auto.arima, stepwise, $p=q=0$ start, CSS approximation
ARIMA 2	Exhaustive automatic ARIMA	non-stepwise search up to max order 8, MLE
Custom	Additive decomposition	frequency 2, moving-average weights $(0.25, 0.5, 0.25)$

The custom decomposition method is structurally unusual for daily forecasting. It applies classical additive decomposition with an artificial frequency of $2$, uses the moving-average filter with weights $$\operatorname{sMAPE} = \frac{1}{h} \sum_{t=1}^{h} \frac{2\lvert F_t - Y_t \rvert}{\lvert F_t \rvert + \lvert Y_t \rvert}$$0 to smooth the trend, forecasts the seasonal component by seasonal naive repetition, and chooses one of four trend/residual forecasting combinations by validation using MASE. Those four combinations are: linear extrapolation for both trend and residuals; linear extrapolation for trend and repetition of the last 14 residuals; repetition of the last 14 trend values and linear extrapolation for residuals; or repetition of the last 14 values for both trend and residuals.

The ensemble forecast is

$$\operatorname{sMAPE} = \frac{1}{h} \sum_{t=1}^{h} \frac{2\lvert F_t - Y_t \rvert}{\lvert F_t \rvert + \lvert Y_t \rvert}$$1

All negative predictions are clipped to $$\operatorname{sMAPE} = \frac{1}{h} \sum_{t=1}^{h} \frac{2\lvert F_t - Y_t \rvert}{\lvert F_t \rvert + \lvert Y_t \rvert}$$2, because the dataset had no negatives. In retrospective evaluation, this median ensemble only slightly improved on the simple Naive baseline, which is one reason the correlator became the decisive component.

3. Correlator construction and forecast selection

The correlator is the defining element of M4-Instruct. Its core idea is to exploit cross-series correlations by taking the last 14 days of a target series and matching them against every 14-day segment of every other series that is followed by at least 14 further observations. The continuation of the best-matching segment, after mean-variance alignment, becomes the target forecast if the match passes an acceptance rule (Ingel et al., 2020).

For target series $$\operatorname{sMAPE} = \frac{1}{h} \sum_{t=1}^{h} \frac{2\lvert F_t - Y_t \rvert}{\lvert F_t \rvert + \lvert Y_t \rvert}$$3 with last 14 values $$\operatorname{sMAPE} = \frac{1}{h} \sum_{t=1}^{h} \frac{2\lvert F_t - Y_t \rvert}{\lvert F_t \rvert + \lvert Y_t \rvert}$$4 and candidate series $$\operatorname{sMAPE} = \frac{1}{h} \sum_{t=1}^{h} \frac{2\lvert F_t - Y_t \rvert}{\lvert F_t \rvert + \lvert Y_t \rvert}$$5 with a segment ending at $$\operatorname{sMAPE} = \frac{1}{h} \sum_{t=1}^{h} \frac{2\lvert F_t - Y_t \rvert}{\lvert F_t \rvert + \lvert Y_t \rvert}$$6, the Pearson correlation is

$$\operatorname{sMAPE} = \frac{1}{h} \sum_{t=1}^{h} \frac{2\lvert F_t - Y_t \rvert}{\lvert F_t \rvert + \lvert Y_t \rvert}$$7

where $$\operatorname{sMAPE} = \frac{1}{h} \sum_{t=1}^{h} \frac{2\lvert F_t - Y_t \rvert}{\lvert F_t \rvert + \lvert Y_t \rvert}$$8 and $$\operatorname{sMAPE} = \frac{1}{h} \sum_{t=1}^{h} \frac{2\lvert F_t - Y_t \rvert}{\lvert F_t \rvert + \lvert Y_t \rvert}$$9.

After selecting $$\operatorname{MASE} = \frac{\frac{1}{h} \sum_{t=1}^{h} \lvert F_t - Y_t \rvert}{\frac{1}{n - m} \sum_{t=m+1}^{n} \lvert Y_t - Y_{t-m} \rvert}$$0 that maximizes $$\operatorname{MASE} = \frac{\frac{1}{h} \sum_{t=1}^{h} \lvert F_t - Y_t \rvert}{\frac{1}{n - m} \sum_{t=m+1}^{n} \lvert Y_t - Y_{t-m} \rvert}$$1, the method applies an affine alignment

$$\operatorname{MASE} = \frac{\frac{1}{h} \sum_{t=1}^{h} \lvert F_t - Y_t \rvert}{\frac{1}{n - m} \sum_{t=m+1}^{n} \lvert Y_t - Y_{t-m} \rvert}$$2

and sets

$$\operatorname{MASE} = \frac{\frac{1}{h} \sum_{t=1}^{h} \lvert F_t - Y_t \rvert}{\frac{1}{n - m} \sum_{t=m+1}^{n} \lvert Y_t - Y_{t-m} \rvert}$$3

The submission used two acceptance conditions: a high correlation threshold,

$$\operatorname{MASE} = \frac{\frac{1}{h} \sum_{t=1}^{h} \lvert F_t - Y_t \rvert}{\frac{1}{n - m} \sum_{t=m+1}^{n} \lvert Y_t - Y_{t-m} \rvert}$$4

and a forecast-variability constraint,

$$\operatorname{MASE} = \frac{\frac{1}{h} \sum_{t=1}^{h} \lvert F_t - Y_t \rvert}{\frac{1}{n - m} \sum_{t=m+1}^{n} \lvert Y_t - Y_{t-m} \rvert}$$5

Retrospective analysis showed that a lower threshold of $$\operatorname{MASE} = \frac{\frac{1}{h} \sum_{t=1}^{h} \lvert F_t - Y_t \rvert}{\frac{1}{n - m} \sum_{t=m+1}^{n} \lvert Y_t - Y_{t-m} \rvert}$$6 yielded better overall OWA, and that the standard-deviation constraint was unnecessary for the best combined result. The correlator is combined with the ensemble by hard selection rather than weighted blending:

$$\operatorname{MASE} = \frac{\frac{1}{h} \sum_{t=1}^{h} \lvert F_t - Y_t \rvert}{\frac{1}{n - m} \sum_{t=m+1}^{n} \lvert Y_t - Y_{t-m} \rvert}$$7

This design makes M4-Instruct a piecewise forecasting system. The ensemble supplies a robust default, while the correlator overrides it only when a near-perfect neighbor continuation is found.

4. Retrospective performance and ablation results

The paper reports that the five-model ensemble alone offered only a marginal gain over the Naive constant baseline, whereas the correlator accounted for most of the improvement (Ingel et al., 2020).

Configuration	OWA vs Naive	Note
Naive	1.000	baseline
ETS	0.996	best single model among the five
ARIMA 1	1.041	worse than Naive
ARIMA 2	1.033	worse than Naive
Custom	1.354	substantially worse
Five-method ensemble	0.995	slight improvement
Submission with correlator	0.930	two implementation bugs, threshold 0.9999
Corrected correlator system	0.903	threshold 0.9999, std multiplier 2.5
Best post hoc setting	0.848	threshold 0.999, no std constraint

The strongest daily performance appears when the correlator is applied selectively. With correlation threshold $$\operatorname{MASE} = \frac{\frac{1}{h} \sum_{t=1}^{h} \lvert F_t - Y_t \rvert}{\frac{1}{n - m} \sum_{t=m+1}^{n} \lvert Y_t - Y_{t-m} \rvert}$$8 and standard-deviation multiplier $$\operatorname{MASE} = \frac{\frac{1}{h} \sum_{t=1}^{h} \lvert F_t - Y_t \rvert}{\frac{1}{n - m} \sum_{t=m+1}^{n} \lvert Y_t - Y_{t-m} \rvert}$$9, the correlator was used on 518 of 4,227 daily series, or 12.3%, and achieved OWA $$\operatorname{OWA} = \frac{1}{2}\left( \frac{\operatorname{sMAPE}_{\text{model}}}{\operatorname{sMAPE}_{\text{Naive2}}} + \frac{\operatorname{MASE}_{\text{model}}}{\operatorname{MASE}_{\text{Naive2}}} \right).$$0 on that subset. More liberal thresholds increased coverage to about 44% but degraded correlator-only OWA and could worsen full OWA once the ensemble handled the remaining series.

The gains were concentrated in the Daily and Hourly categories. The correlator was used on 12.3% of Daily series and 6% of Hourly series, with subset OWA values of $$\operatorname{OWA} = \frac{1}{2}\left( \frac{\operatorname{sMAPE}_{\text{model}}}{\operatorname{sMAPE}_{\text{Naive2}}} + \frac{\operatorname{MASE}_{\text{model}}}{\operatorname{MASE}_{\text{Naive2}}} \right).$$1 and $$\operatorname{OWA} = \frac{1}{2}\left( \frac{\operatorname{sMAPE}_{\text{model}}}{\operatorname{sMAPE}_{\text{Naive2}}} + \frac{\operatorname{MASE}_{\text{model}}}{\operatorname{MASE}_{\text{Naive2}}} \right).$$2, respectively. For Weekly, Monthly, Quarterly, and Yearly data, usage was at most 2.8% and the subset OWA values were much weaker. This suggests that the method depended on local repeated motifs that were much more prevalent in Daily and Hourly data than at lower frequencies.

A common misconception is that these results establish a broadly superior cross-series nearest-neighbor paradigm for daily forecasting. The paper’s retrospective analysis does not support that interpretation. Instead, it shows that the correlator can be extraordinarily effective when very high-fidelity local analogues exist, but that such analogues were themselves often consequences of leakage or data anomalies.

5. Data leakage, corrupted similarities, and diagnostic taxonomy

The paper’s most consequential methodological contribution is its analysis of why the correlator worked so well. It identifies data leakage and quality issues as amplifiers of performance, including test segments selected from different time intervals, duplicated segments, rearrangements, and misaligned start dates (Ingel et al., 2020).

The authors classify correlated pairs into four categories:

Category	Count	Description
T1 self-correlations	12	end of a series correlates with its beginning
T2 mutual correlations	202	beginning of one series correlates with end of another and vice versa
T3 synchronised correlations	23	correlated regions correspond to the same calendar dates
T4 unsynchronised correlations	767	correlated regions correspond to different dates

The total analyzed correlated pairs was 1,004 after excluding trivial single-jump alignments. Among the cases where the correlator was applied, 26% used future data, a fact that became detectable only after the organizers released actual dates. When restricted to past-only usage, the method still achieved strong performance: OWA $$\operatorname{OWA} = \frac{1}{2}\left( \frac{\operatorname{sMAPE}_{\text{model}}}{\operatorname{sMAPE}_{\text{Naive2}}} + \frac{\operatorname{MASE}_{\text{model}}}{\operatorname{MASE}_{\text{Naive2}}} \right).$$3 for threshold $$\operatorname{OWA} = \frac{1}{2}\left( \frac{\operatorname{sMAPE}_{\text{model}}}{\operatorname{sMAPE}_{\text{Naive2}}} + \frac{\operatorname{MASE}_{\text{model}}}{\operatorname{MASE}_{\text{Naive2}}} \right).$$4 and OWA $$\operatorname{OWA} = \frac{1}{2}\left( \frac{\operatorname{sMAPE}_{\text{model}}}{\operatorname{sMAPE}_{\text{Naive2}}} + \frac{\operatorname{MASE}_{\text{model}}}{\operatorname{MASE}_{\text{Naive2}}} \right).$$5 for threshold $$\operatorname{OWA} = \frac{1}{2}\left( \frac{\operatorname{sMAPE}_{\text{model}}}{\operatorname{sMAPE}_{\text{Naive2}}} + \frac{\operatorname{MASE}_{\text{model}}}{\operatorname{MASE}_{\text{Naive2}}} \right).$$6.

The mechanism of leakage is straightforward. If calendars are synchronized, or if start dates are incorrect, a near-perfect match between a target’s terminal window and another series’ segment may effectively expose the target’s future. Duplications and rearrangements create similarly artificial neighbors. The paper therefore recommends diagnostics based on sliding-window cross-series correlation spikes over long overlaps, matches that tighten after standardization and align on calendar time, self-matches where the beginning approximates the end for long spans, and date-based sanity checks verifying that matched segments strictly precede the target forecast start date.

This section is also where the principal controversy resides. The correlator’s exceptionally low subset OWA should not be read as a clean demonstration of transferable structure in daily time series. The paper argues that unusually high short-window correlations require skepticism, especially when actual timestamps are absent or unreliable.

6. Reproducible workflow, constraints, and broader relevance

A reproducible M4-Instruct workflow begins with 4,227 daily series with known timestamps and horizon $$\operatorname{OWA} = \frac{1}{2}\left( \frac{\operatorname{sMAPE}_{\text{model}}}{\operatorname{sMAPE}_{\text{Naive2}}} + \frac{\operatorname{MASE}_{\text{model}}}{\operatorname{MASE}_{\text{Naive2}}} \right).$$7. Preprocessing consists of handling missing values, detecting or removing duplicated and rearranged segments, confirming start dates, and clipping negative forecasts to zero. Evaluation should implement Naive2 with the appropriate seasonal period, such as $$\operatorname{OWA} = \frac{1}{2}\left( \frac{\operatorname{sMAPE}_{\text{model}}}{\operatorname{sMAPE}_{\text{Naive2}}} + \frac{\operatorname{MASE}_{\text{model}}}{\operatorname{MASE}_{\text{Naive2}}} \right).$$8 for daily data when weekly seasonality is meaningful (Ingel et al., 2020).

Per-series model fitting then proceeds through Naive, ETS, ARIMA 1, ARIMA 2, and the custom decomposition method, with the last 14 points removed for internal validation. The median ensemble is formed across these five methods. The correlator then scans candidate 14-day segments, sorts them by correlation, applies the threshold and optional variability constraint, constructs the affine continuation, and accepts the forecast only if the matched segment’s dates precede the target’s forecast origin. If no acceptable match exists, the system falls back to the ensemble.

The paper emphasizes several computational and operational constraints. ARIMA 2 is expensive; retrospective analysis showed that dropping it and retaining ARIMA 1 improved OWA to approximately $$\operatorname{OWA} = \frac{1}{2}\left( \frac{\operatorname{sMAPE}_{\text{model}}}{\operatorname{sMAPE}_{\text{Naive2}}} + \frac{\operatorname{MASE}_{\text{model}}}{\operatorname{MASE}_{\text{Naive2}}} \right).$$9 versus the Naive baseline while saving time. The correlator is computationally heavy, with cost described as $m$0, so pre-indexing segments, parallelizing correlation computation, and early stopping when $m$1 falls below threshold are recommended. Memory-safe iteration is also necessary to avoid the implementation bug that limited correlator usage to D1-D2138 in the original submission.

The principal limitations are explicit. The correlator is highly sensitive to leakage and data hygiene; perfect or near-perfect short-window correlations are rare in clean real-world data; short-window matching with $m$2 can be spurious; strict thresholds reduce coverage; relaxed thresholds degrade accuracy; and the custom decomposition method underperformed on daily series. Without leakage, expected performance moves much closer to robust statistical baselines such as ETS, ARIMA, and median ensembles.

The broader relevance of M4-Instruct therefore lies in protocol design rather than in a universally dominant forecasting model. It shows that cross-series analogical forecasting can be powerful under extreme local similarity, but only when date-aware eligibility, leakage controls, and auditability are treated as first-class constraints. For other frequencies, the paper suggests adapting the correlator window $m$3 to the dominant seasonality, enforcing date-based eligibility, and maintaining strong baseline models such as ETS and ARIMA; it also notes TBATS, BATS, Theta, and STL+ETS as candidates when complex seasonality is present, although these were not selected in the final daily ensemble.