Global Explanations for Multivariate Time Series Forecasting Models via $K$-Order Markov Approximations

Abstract: While many explainable AI (XAI) methods have been proposed, most are not designed for time-series forecasting models and often rely on the implicit assumption that timestamp features are independent. This assumption ignores the fundamental property of temporal dependence and can lead to explanations that violate the sequential and causal structure of the data. We introduce \textsc{KARMA}, a method for explaining time-series predictors by constructing a Markov surrogate model that captures the temporal dependencies learned by the predictor. Our approach revolves around three main aspects: identifying the minimal history length $K$ that is predictively sufficient for the model, estimating the best-fitting $K$-order Markov transition kernel from the discretized history space, and a five-level global explanation hierarchy that can be derived from the Markov transition kernel, which we illustrate using real-world weather data (Beijing PM 2.5). We also certify using complex synthetic data with known true causal edges that KARMA (i) recovers the data causal structure as learned by the model via a controlled experiment and (ii) identifies temporal dependencies better than established attribution methods such as TimeSHAP.

• The paper presents KARMA, a novel global surrogate methodology that approximates black-box time series predictors with a K-order Markov chain to capture temporal and cross-variable dependencies.
• KARMA identifies the minimal sufficient lag (K*) and constructs a five-level explanation hierarchy—including variable importance, lag profiles, and intervention effects—by estimating conditional transition kernels.
• Empirical evaluations demonstrate KARMA’s superior attribution fidelity and certified baseline performance, validated through synthetic causal recovery and real-world benchmarks.

Global Markov Surrogate Explanations for Multivariate Time Series Forecasting Models

Motivation and Limitations of Existing XAI for Time Series

The opacity of deep sequence models such as TCNs, RNNs, and transformers in time series forecasting impedes their adoption in critical domains like finance, healthcare monitoring, and industrial control. Traditional Explainable AI (XAI) methods—e.g., LIME, SHAP, gradient-based attributions, or attention-weight visualizations—are poorly adapted to time-dependent data. Their independence assumptions and local/global approximation heuristics break down under temporal autocorrelation, inter-variable Granger causality, and regime-dependent distribution shift. Perturbation-based explanations can generate off-manifold counterfactuals; gradients describe local geometry rather than systematic influences; and attention is empirically unreliable as a proxy for causal attribution.

KARMA ("K-Order Markov Approximation") is introduced as a global, model-agnostic XAI methodology that sidesteps these issues by directly approximating the predictive behavior of a black-box forecaster with a $K$-order Markov chain surrogate. This surrogate captures the temporal and cross-variable dependencies learned by the model via its estimated conditional transition kernel. The approach yields a hierarchy of statistically controlled, interpretable, and certified-global explanations reflecting how the model encodes history and causality.

The KARMA Method: Surrogate Construction and Explanation Hierarchy

KARMA constructs a global Markov surrogate for a generic black-box time series predictor $f$ by (1) identifying the minimal predictively sufficient lag $K^*$, (2) compressing the input space accordingly, and (3) estimating transition kernels over a discretized history space. The process and subsequent explanation hierarchy can be summarized as follows:

1. Markov Surrogate Selection ($K^*$): The minimal lag $K^*$ is determined by searching for the shortest input suffix rendering predictions effectively invariant to further prefix truncation, given a tolerance $\varepsilon$ on predictive discrepancy. Surrogacy validity requires only repeated model queries, not explicit kernel estimation.
2. Certified Model Compression/Baseline: When $K^* < W$ (window length), the model is formally insensitive to all lags $>K^*$. Consequently, attributions for those lags are mathematically zero (up to estimation error), and a model-certified baseline $b^*$ resolves the fundamental baseline selection ambiguity that has hindered prior time series XAI.
3. Transition Kernel Estimation: For a fixed $K^*$, the core surrogate is the estimated conditional transition kernel $f$0 (discrete next-state given lag-$f$1 history $f$2), marginally or jointly over the $f$3 target variables. Estimation employs empirical counting, Monte Carlo sampling with certified baselines, or tree-structured pooling for high-dimensional sparsity, always yielding strict noise floor guarantees.
4. Five-Level Global Explanation Hierarchy: All explanation levels are derived from the surrogate kernel:
   • Level 1 – Variable Importance: Aggregates the total variation (TV) shift in forecast distributions under marginalization of each source variable/lag; produces global, certified importance scores and direct edge interpretation in induced causal graphs.
   • Level 2 – Lag Profiles: Resolves variable importance by lag, capturing whether the model encodes short/long-range memory, momentum, or mean-reversion.
   • Level 3 – Regime Explanation: Quantifies unique, regime-dependent history effects by the marginal interdependence index, distinguishing histories where conditional forecasts diverge sharply from typical trajectories.
   • Level 4 – Interventional Effects: Computes average interventional effects (AIEs) quantifying sensitivity to explicit input manipulations, grounding attributions in counterfactual perturbations within regime-valid contexts.
   • Level 5 – Explanation Uncertainty: Reports both aleatoric prediction entropy and epistemic uncertainty derived from kernel estimation noise floor, with a coverage map to warn on unreliable histories.

Figure 1: Five-level KARMA explanation hierarchy applied to a TCN (W=24, hidden 64, 2 layers, kernel 3) trained on the Beijing PM$f$4 dataset. Each level provides a progressively more refined explanation, from variable importance and lag profiles to regime separation, interventional effects, and uncertainty quantification.

Experimental Evaluation

Synthetic Causal Recovery

On synthetic VAR processes with analytically known dependency structures, KARMA consistently achieves perfect recall and high precision in model-induced causal graph reconstruction at moderate scale, outperforming TimeSHAP, WinIT, DynaMask, and Feature Occlusion for denser graphs ($f$5). Attribution ranking (measured by Kendall's $f$6) is optimal for medium to large systems, and all lag-attributions beyond $f$7 are provably zero with the compression certificates.

Real-world Time Series Benchmarks

Three real datasets (Electricity Transformer Temperature [ETTh1], Beijing PM$f$8, Exchange Rate [ExRA]) spanning financial, environmental, and macroeconomic settings are used to validate the robustness and informativeness of KARMA against perturbation- and gradient-based XAI. Performance is measured by a lag-aware AUC quantifying the effect of imputing (removing) top-attributed lags with VAR-conditional expectations.

Figure 2: Lag AUC removal curves for all datasets and architectures. Each curve shows cumulative $f$9 as the fraction of removed time steps increases, indicating the temporal fidelity of attributed importance.

KARMA outperforms all baselines on ETTh1 (notably opening a substantial margin on TCN), is competitive on Beijing PM$K^*$0, and, importantly, all methods collapse to near-zero effect on the near-random-walk ExRA. This indicates fidelity—when a model learns little temporal dependence, no lag-importance explainer should spuriously report high-effect lags.

Statistical Guarantees, Practical Scaling, and Limitations

A unique property of KARMA is its explicit, finite-sample control via total variation bounds on kernel estimation. The sample budget for transition kernel estimation scales linearly in the observed history support in practice, and the proposed tree-based pooling offers tractable reliability at high $K^*$1 or $K^*$2. Level-5 explanations expose noise floors and point out regions of inadequate statistical support, a critical requirement for trustworthy scientific and regulatory use.

A primary limitation is the inherent exponential scaling of discretized state space with $K^*$3. The method partially alleviates this via focus on marginal kernels, observed history support, and adaptive pooling, but scaling to massive multivariate systems (e.g., genomics-scale time series) remains a challenge for future work.

Implications and Future Directions

KARMA demonstrates that global surrogate modeling via order-$K^*$4 Markov approximations can deliver temporally and causally faithful explanations in complex, non-linear time series prediction—without reliance on questionable independence or local-sensitivity paradigms. The approach is particularly relevant for high-stakes domains governed by regulatory requirements (e.g., ESMA, EU AI Act), where certified attributions and model-internal causal auditability are necessary.

The Markov surrogate framework opens a pathway for rigorous causal explanation, systematic baseline selection, and uncertainty quantification in time series XAI. Extensions to handle contemporaneous couplings, higher-dimensional manifolds, and unstructured input histories are natural next steps. As deep time series models proliferate in operational analytics, the need for methods that can reliably uncover and quantify model-induced causal dependencies will continue to accelerate.

Conclusion

KARMA advances time series model explainability by leveraging $K^*$5-order Markov approximations to provide a statistically grounded, globally consistent, and computationally practical surrogate for arbitrary black-box predictors. Its five-level explanation hierarchy captures variable, lag, regime, intervention, and reliability dimensions, unifying XAI under a single kernel estimation paradigm with explicit compression and reliability certification. Empirical results validate its effectiveness in both synthetic and real-world regimes and highlight principal directions for scaling and application in AI safety-critical environments (2606.27599).