CyFormer: Accurate State-of-Health Prediction of Lithium-Ion Batteries via Cyclic Attention (2304.08502v1)

Abstract: Predicting the State-of-Health (SoH) of lithium-ion batteries is a fundamental task of battery management systems on electric vehicles. It aims at estimating future SoH based on historical aging data. Most existing deep learning methods rely on filter-based feature extractors (e.g., CNN or Kalman filters) and recurrent time sequence models. Though efficient, they generally ignore cyclic features and the domain gap between training and testing batteries. To address this problem, we present CyFormer, a transformer-based cyclic time sequence model for SoH prediction. Instead of the conventional CNN-RNN structure, we adopt an encoder-decoder architecture. In the encoder, row-wise and column-wise attention blocks effectively capture intra-cycle and inter-cycle connections and extract cyclic features. In the decoder, the SoH queries cross-attend to these features to form the final predictions. We further utilize a transfer learning strategy to narrow the domain gap between the training and testing set. To be specific, we use fine-tuning to shift the model to a target working condition. Finally, we made our model more efficient by pruning. The experiment shows that our method attains an MAE of 0.75\% with only 10\% data for fine-tuning on a testing battery, surpassing prior methods by a large margin. Effective and robust, our method provides a potential solution for all cyclic time sequence prediction tasks.

• The paper introduces CyFormer, a novel transformer-based model with a cyclic attention mechanism that effectively captures intra-cycle and inter-cycle features for accurate State-of-Health prediction in lithium-ion batteries.
• CyFormer achieved state-of-the-art performance, demonstrated by an MAE of 0.75% and MAPE of 0.90% using only 10% of target battery data for fine-tuning, significantly surpassing previous methods.
• The model's cyclic attention and transfer learning strategy offer a robust and efficient approach to bridge the domain gap and accurately predict battery degradation under various conditions, with potential applications beyond battery management.

The paper "CyFormer: Accurate State-of-Health Prediction of Lithium-Ion Batteries via Cyclic Attention" presents a novel transformer-based model, CyFormer, for predicting the State-of-Health (SoH) of lithium-ion batteries, a critical aspect of Battery Management Systems (BMS) in electric vehicles. The key innovation of this research is the development of a cyclic attention mechanism aimed at effectively capturing both intra-cycle and inter-cycle features to improve prediction accuracy and mitigate the domain gap issues that arise between training and testing datasets.

Model Architecture

CyFormer eschews the traditional CNN and RNN architectures commonly used in time-series modeling in favor of a transformer-inspired encoder-decoder framework. This architecture is structured as follows:

• Encoder: Utilizes row-wise and column-wise attention blocks to extract cyclic features from historical cycle data. The row-wise attention focuses on capturing intra-cycle connections, while the column-wise attention handles inter-cycle interactions. This two-dimensional attention mechanism preserves essential intra-cycle information that might be lost in previous CNN-based models.
• Decoder: Uses cross-attention to process features extracted by the encoder. Randomly initialized SoH queries attend to these cyclic features to generate predictions for future cycles.

The encoder embeds input sequences, applies two-dimensional positional encoding, and outputs a feature vector sequence that the decoder uses for SoH prediction. Notably, the model incorporates transfer learning to adapt to different battery working conditions by fine-tuning on a small subset of target cycle data.

Numerical Results

The authors conducted extensive experiments using a lithium-ion battery dataset from NASA Ames Prognostics Center of Excellence. Key findings include:

• The CyFormer model achieved an MAE (Mean Absolute Error) of 0.75% and an MAPE (Mean Absolute Percentage Error) of 0.90% with just 10% of the target battery data for fine-tuning. This performance surpasses state-of-the-art methods by a significant margin, highlighting the model's capacity to provide accurate predictions with limited fine-tuning data.
• When compared to CNN-LSTM and CNN-Transformer baselines, CyFormer demonstrated superior performance even with minimal data for fine-tuning, confirming its effectiveness in capturing long-term sequential patterns and local feature interactions through its novel attention mechanisms.

Innovations and Contributions

• Cyclic Attention Mechanism: The introduction of row-wise and column-wise attention allows for the extraction and preservation of nuanced cycle-specific features, addressing the limitations of generic CNN filters in capturing cycle-dependent information.
• Transfer Learning Strategy: By using only 10% of the initial cycle data for model fine-tuning, CyFormer effectively bridges the domain gap, adapting to variable battery behaviors under different conditions without sacrificing accuracy.
• Efficiency and Pruning: A lightweight version of the model was proposed by optimizing depth and sampling rates, achieving significant reductions in computational complexity while maintaining near-original accuracy levels.

Conclusion

The proposed CyFormer model sets a new benchmark for SoH prediction tasks by effectively integrating transformer-based attention mechanisms specifically tailored for cyclic time sequence data. Its method of addressing intra-cycle and inter-cycle variations through cyclic attention can extend beyond battery management to other domains dealing with cyclic data, thereby broadening the applicability of this approach. Additionally, the paper provides a comprehensive framework for leveraging limited data via transfer learning to achieve highly accurate predictions, which could offer significant benefits for industrial applications in electric vehicle battery management.