TopoCast: A Topological Fidelity Framework for Evaluating Transformer-Based Time Series Forecasting

Abstract: Deep learning-based models have achieved state-of-the-art performance in Time Series Forecasting (TSF), yet their evaluation remains dominated by pointwise error metrics such as Mean Squared Error (MSE), which quantify numerical accuracy but overlook structural properties of the forecast signal, including recurrent dynamics, oscillatory behavior, and phase alignment. As a result, forecasts exhibiting over-smoothing, phase shifts, or frequency distortions may achieve favorable error scores despite substantial structural degradation. To address this limitation, we propose TopoCast, a topology-driven framework for evaluating structural fidelity in TSF. TopoCast reconstructs phase-space representations of forecast and ground-truth sequences using Takens delay embedding and applies persistent homology to characterize their intrinsic dynamics. We derive four complementary topological fidelity measures from persistence diagrams and aggregate them into a Topological Fidelity Score (TFS). We further introduce dominant cycle overlap, a novel metric that maps persistent topological features to the temporal domain to assess whether dominant oscillatory patterns occur at the correct time points. Combined with TFS, this yields the Localized Topological Fidelity Score (LTFS), a phase-aware measure that captures temporal localization errors invisible to existing evaluation metrics. Experiments on five Transformer architectures across three real-world benchmark datasets demonstrate that models with similar forecasting errors can exhibit markedly different structural fidelity profiles, revealing failure modes overlooked by conventional evaluation and highlighting the value of topology-aware forecast assessment.

• The paper introduces TopoCast, a framework using phase-space reconstruction and persistent homology to quantify forecast fidelity with metrics like loop count and persistence entropy.
• The paper demonstrates that transformer models with similar pointwise error metrics can have divergent structural performance, as revealed by LTFS and dominant cycle overlap analyses.
• The paper outlines practical implications for model selection and architecture design in forecasting applications across electricity, finance, and healthcare.

TopoCast: A Topological Fidelity Framework for Evaluating Transformer-Based Time Series Forecasting

Motivation and Context

Transformer-based architectures have established themselves as high-performing models in multivariate time series forecasting, exhibiting strong predictive capability across electricity, finance, and healthcare domains. However, performance evaluations remain predominantly rooted in pointwise error metrics, such as MSE and MAE, which are agnostic to critical structural and dynamical characteristics of temporal signals—periodicities, oscillatory motifs, and phase relationships. Such metrics allow forecasts with heavily smoothed dynamics, temporal misalignments, or spurious oscillations to score favorably despite serious degradation of underlying system behavior.

Addressing this evaluative myopia, TopoCast introduces a rigorous topological data analysis (TDA) framework, based on persistent homology, for quantifying structural fidelity of forecasts produced by deep time series models. TopoCast measures the degree to which forecasted sequences accurately reproduce the global geometric and dynamical structure of the ground truth, even when pointwise metrics are unable to discriminate between structurally distinct forecasting errors.

Methodological Framework

The core methodology of TopoCast leverages phase-space reconstruction and persistent homology to extract robust, scale-invariant descriptors of time series evolution:

• Phase-Space Embedding: Using Takens embedding (with fixed parameters $m=3$, $\tau=2$), both forecast and ground-truth sequences are lifted to higher-dimensional point clouds. This enables capture of underlying attractor geometry, essential for representing cycles and recurrent motifs beyond trivial 1D topologies.
• Persistent Homology Extraction: The Vietoris-Rips filtration is applied to these point clouds to compute $H_1$ persistence diagrams via Ripser, focusing on loop structures (periodicities and oscillations). This yields a birth-death barcode summarizing the lifetime of topological features across scales.
• Fidelity Score Computation: Four interpretable scalar descriptors are extracted:
  • Loop count ($S_{B_1}$): significant independent cycles.
  • Dominant cycle strength ($S_{L_{max}}$): maximum feature lifetime.
  • Total persistence ($S_{TP}$): sum of all lifetimes (“topological energy”).
  • Persistence entropy ($S_H$): distribution complexity.

Each descriptor is compared (forecast vs. ground truth) using log-normalized ratios, combined via geometric mean into the Topological Fidelity Score (TFS).

• Phase-Aware Alignment: Dominant Cycle Overlap: TFS is intrinsically agnostic to temporal localization; therefore, TopoCast introduces the dominant cycle overlap metric. By tracing representative cocycles back to original time steps, it quantifies the Jaccard overlap of regions in time where dominant cycles occur in both signals. This results in the Localized Topological Fidelity Score (LTFS): $LTFS = TFS \times \text{Overlap}$, the only measure in this context sensitive to phase errors.

This separation between diagram-level structure (TFS) and temporal co-localization (overlap) offers fine-grained diagnostic resolution of failure types: structural collapse, over-smoothing, amplitude attenuation, missing recurrent events, and temporal misalignments.

Empirical Evaluation

TopoCast was evaluated on five leading Transformer-based models (Transformer, Informer, Autoformer, FEDformer, PatchTST) and three real-world benchmarks (ETTm2, Exchange Rate, ILI), spanning periodic, nonstationary, and volatile dynamics. Both synthetic (controlled) and real data scenarios were analyzed.

Key findings:

• Divergence Between Pointwise and Structural Fidelity: Models with nearly identical MSEs exhibited radically different LTFS profiles. For example, PatchTST achieved the lowest MSE on the Exchange dataset but was ranked worst by LTFS due to injection of spurious cycles—a phenomenon completely invisible to standard metrics.
• Detection of Phase Localization Errors: The dominant cycle overlap component uniquely captured phase shifts and temporal misalignments even when diagrammatic summaries (and hence, Wasserstein, DTW, and TDI metrics) failed to differentiate such errors. In synthetic experiments, forecasts with perfect cycle shape but shifted in phase yielded high TFS and zero LTFS, precisely characterizing localization failure.
• Architectural Inductive Biases and Structured Errors: Decomposition-based models (Autoformer, FEDformer) imposed periodic structure regardless of the data support, leading to loop injection on signal regimes with sparse true periodicity. Conversely, standard self-attention models systematically lost loop structures on periodic datasets, highlighting the importance of explicit structural priors.
• Comprehensive Metric Complementarity: LTFS was broadly consistent with shape-similarity and distributional metrics (DTW, W2), but crucially offered non-redundant information and diagnostic decomposability. It enabled attributing degradation to distinct properties (loop count, amplitude, complexity, or phase), while remaining robust to noise via the persistent homology stability theorem.

Practical and Theoretical Implications

TopoCast provides a powerful, model-agnostic diagnostic for structural fidelity in sequence modeling, with significant practical relevance for operational domains where oscillatory and phase-aware accuracy is critical (e.g., ECG morphology analysis, power grid control, financial rhythm detection). Topological evaluation can guide both model selection and architecture design, particularly in the calibration of inductive biases for applications with strict dynamical constraints.

Theoretically, the explicit breakdown of diagram-level versus temporal-localization errors sharpens the understanding of how neural forecasting models encode dynamical priors, and how they fail as a function of architectural choices and data regimes. The methodology extends naturally to consideration of higher homology dimensions, joint uncertainty quantification for probabilistic forecasts, and the development of topologically-informed training objectives.

Future Directions

Extensions of TopoCast could involve:

• Incorporating higher-order persistent features ($H_2$, voids) for multivariate systems with more complex recurrence.
• Adapting metrics to probabilistic or ensemble forecasting, integrating uncertainty quantification in topological descriptors.
• Formulating differentiable topological regularizers for end-to-end topology-aware training.
• Applying phase-aware topological evaluation to causality and mechanistic interpretability assessments.

Conclusion

TopoCast establishes that standard pointwise accuracy metrics inadequately capture structural and phase-localized forecast fidelity in deep TSF. Through persistent homology and dominant cycle overlap, TopoCast robustly diagnoses distinct forms of forecast degradation, revealing characteristic model failure modes and providing actionable metrics for both evaluation and model design (2606.25439). This framework sets a new reference for topology-aware assessment in time series analysis and opens new avenues for architecture development and evaluation in sequential AI.