BatteryMFormer: Multi-level Learning for Battery Degradation Trajectory Forecasting

Abstract: Early battery degradation trajectory forecasting (BDTF), which predicts the full-life state-of-health trajectory from early operational data, is critical for battery optimization, manufacturing, and deployment. Battery degradation data exhibit two key characteristics. First, degradation data present a multi-level structure, including regularities shared within aging conditions and trajectory patterns shared across batteries. Second, degradation-related variations in voltage-current profiles are often localized to specific state-of-charge (SOC) intervals. Existing approaches often fail to explicitly model these characteristics. To bridge this gap, we propose BatteryMFormer, a multi-level Transformer for early BDTF. BatteryMFormer integrates (1) an aging-condition-aware decoder that injects aging-condition priors via aging-condition-informed queries and aging-condition-aware attention, (2) a meta degradation pattern memory that learns and retrieves trajectory prototypes to guide long-horizon forecasting, and (3) a dual-view encoder that jointly captures temporal dynamics and SOC-localized variations from voltage and current time series. Extensive experiments on four battery domains show that BatteryMFormer consistently outperforms state-of-the-art baselines, marking a significant step toward reliable BDTF. Our code is available at https://github.com/Ruifeng-Tan/BatteryMFormer.

• The paper presents a multi-level Transformer architecture that combines dual-view encoding, aging-condition-aware decoding, and a meta degradation pattern memory for precise battery degradation trajectory forecasting.
• Empirical evaluations across four battery types show significant reductions in error metrics and robust performance even with limited training data.
• Ablation studies underscore the critical role of SOC-localized encoding and pattern memory in capturing heterogeneous degradation signatures and enhancing generalization.

Multi-level Transformer Architectures for Early Battery Degradation Trajectory Forecasting

Motivation and Problem Formulation

Early-stage prediction of battery degradation trajectories, termed Battery Degradation Trajectory Forecasting (BDTF), is crucial for battery optimization, manufacturing, and operational decision-making. Owing to the heterogeneity of battery chemistries, protocols, and aging conditions, operational data exhibit complex multi-level structures: (1) regularities shared within aging conditions, (2) global trajectory patterns common across batteries, and (3) degradation-relevant signatures localized to specific State-of-Charge (SOC) intervals. Existing ML approaches predominantly focus either on battery-level modeling or feature engineering, often failing to explicitly account for these hierarchical phenomena. BatteryMFormer, a multi-level Transformer architecture, is introduced to bridge this gap by integrating aging-condition-aware decoding, a meta degradation pattern memory, and complementary dual-view encoding of temporal and SOC-localized features.

Figure 1: Multi-level structure of battery degradation: operational time series, trajectory diversity under aging conditions, canonical degradation shapes, and atypical trajectory phenomena.

BatteryMFormer Architecture

BatteryMFormer leverages multi-level inductive biases in a Transformer framework to enhance early-stage BDTF. The architecture consists of:

1. Dual-view Encoder: Processes operational voltage and current time series into temporal-view tokens (capturing intra- and inter-cycle dynamics) and SOC-view tokens (capturing degradation signatures localized in specific SOC intervals).
2. Aging-condition-aware Decoder (ACDecoder): Injects aging-condition priors—extracted via LLM-based embeddings—into query initialization and attention mechanisms, promoting aging-condition-consistent representation learning.
3. Meta Degradation Pattern Memory (MDPM): Maintains learnable memory slots storing trajectory prototypes. Pattern retrieval integrates long-horizon forecasting priors through cosine similarity, and memory learning aligns retrieved embeddings to global trajectory encodings.

   Figure 2: BatteryMFormer model pipeline: dual-view encoding, aging-condition-aware decoding, and pattern memory integration for long-horizon prediction.

The dual-view encoder explicitly models SOC-localized variations, enabling selective attention to degradation-relevant intervals (Figure 3). MDPM facilitates pattern-level guidance for test batteries subjected to previously unseen aging conditions, enhancing generalization.

Figure 3: SOC-localized degradation: deviations in voltage–SOC and current–SOC profiles concentrate in particular SOC intervals even as global profiles evolve smoothly.

Empirical Evaluation

The model is evaluated on four battery domains (Li-ion, CALB, Na-ion, Zn-ion) using the largest public battery lifetime dataset, under aging-condition-exclusive splits to test generalizability. BatteryMFormer consistently surpasses state-of-the-art baselines, achieving reductions in MAPE and MAE on all domains. The model's hierarchical representation significantly improves robustness against domain heterogeneity.

A systematic investigation varying the number of early cycles shows BatteryMFormer outperforms leading baselines (IC2ML, TimeBridge) across diverse input lengths. Notably, error does not monotonically decrease with increased input length, emphasizing the trade-off between informative signatures and input redundancy in long-sequence modeling.

Figure 4: BatteryMFormer performance improves as more early cycles are used, maintaining accuracy superiority across multiple domains.

Ablation studies reveal that all architectural components—SOC-view encoding, MDPM, ACDecoder (including both query and attention modulation), and LLM-based embeddings—contribute substantially to accuracy and stability. The SOC-view mechanism and pattern memory are particularly impactful in domains with complex aging condition diversity.

Model Interpretability and Case Analysis

Case studies demonstrate that MDPM retrieves trajectory prototypes consistent with empirical battery degradation (superlinear, linear, sublinear shapes), enabling effective extrapolation. ACDecoder attention analyses reveal that temporal-view tokens generally dominate, but SOC-view tokens are selectively prioritized—specifically on intervals featuring known degradation mechanisms (peak shifts, polarization increases) per differential voltage analysis.

Figure 5: High attention weights in SOC-view tokens align with major differential voltage analysis peaks, revealing selective focus on electrochemical signatures.

Data-efficient BDTF and Practical Implications

Reducing training set size by 50% minimally impacts BatteryMFormer performance, with improvements over top baselines ranging from 2.8–17.7% in MAPE, particularly marked in domains with pronounced data scarcity and aging condition diversity. This underscores the data efficiency derived from multi-level representation, relevant to practical scenarios where full-life data is expensive to acquire.

Theoretical and Practical Implications

BatteryMFormer exemplifies the integration of domain knowledge into deep sequence modeling through hierarchical biases and selective attention. The architecture demonstrates the value of combining global trajectory-level prototype retrieval with SOC-localized operational feature mining, mediated by aging-condition-aware representations. This approach offers a path forward for rapid degradation assessment, accelerated manufacturing testing, and maintenance scheduling in battery-powered systems. The use of LLM-based metadata encoding further facilitates adaptation across diverse protocols and chemistry regimes.

For future work, adapting BatteryMFormer to handle highly irregular and noisy field data—such as electric vehicle operational logs—remains a significant challenge. Further architectural refinement to efficiently process ultra-long sequences and to encode uncertainty quantification for high-stakes deployment is warranted.

Conclusion

BatteryMFormer advances the state of early battery degradation trajectory forecasting by explicitly encoding multi-level structural biases encompassing aging conditions, global trajectory patterns, and SOC-localized signatures, delivered within a unified Transformer-based sequence model. Empirically, it achieves consistent improvement over specialized and general time series forecasting baselines, exhibits robustness under limited training data, and offers interpretable prediction through selective token attention and pattern memory retrieval. The model's conceptual framework and practical efficacy suggest broad applicability in battery informatics and materials science forecasting tasks (2605.27044).