Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Time-Series Contrastive Learning against False Negatives and Class Imbalance (2312.11939v2)

Published 19 Dec 2023 in cs.LG and cs.AI

Abstract: As an exemplary self-supervised approach for representation learning, time-series contrastive learning has exhibited remarkable advancements in contemporary research. While recent contrastive learning strategies have focused on how to construct appropriate positives and negatives, in this study, we conduct theoretical analysis and find they have overlooked the fundamental issues: false negatives and class imbalance inherent in the InfoNCE loss-based framework. Therefore, we introduce a straightforward modification grounded in the SimCLR framework, universally adaptable to models engaged in the instance discrimination task. By constructing instance graphs to facilitate interactive learning among instances, we emulate supervised contrastive learning via the multiple-instances discrimination task, mitigating the harmful impact of false negatives. Moreover, leveraging the graph structure and few-labeled data, we perform semi-supervised consistency classification and enhance the representative ability of minority classes. We compared our method with the most popular time-series contrastive learning methods on four real-world time-series datasets and demonstrated our significant advantages in overall performance.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (31)
  1. A public domain dataset for human activity recognition using smartphones. In Esann, volume 3, 3.
  2. The descriptive epidemiology of epilepsy—a review. Epilepsy research, 85(1): 31–45.
  3. Integrated oversampling for imbalanced time series classification. IEEE Transactions on Knowledge and Data Engineering, 25(12): 2809–2822.
  4. A simple framework for contrastive learning of visual representations. In International conference on machine learning, 1597–1607. PMLR.
  5. Big self-supervised models are strong semi-supervised learners. Advances in neural information processing systems, 33: 22243–22255.
  6. AF classification from a short single lead ECG recording: The PhysioNet/computing in cardiology challenge 2017. In 2017 Computing in Cardiology (CinC), 1–4. IEEE.
  7. Deep learning for event-driven stock prediction. In Twenty-fourth international joint conference on artificial intelligence.
  8. Time-series representation learning via temporal and contextual contrasting. arXiv preprint arXiv:2106.14112.
  9. Unsupervised scalable representation learning for multivariate time series. Advances in neural information processing systems, 32.
  10. Comparing similarity perception in time series visualizations. IEEE transactions on visualization and computer graphics, 25(1): 523–533.
  11. PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. circulation, 101(23): e215–e220.
  12. Dissecting supervised contrastive learning. In International Conference on Machine Learning, 3821–3830. PMLR.
  13. Learning dynamics and heterogeneity of spatial-temporal graph data for traffic forecasting. IEEE Transactions on Knowledge and Data Engineering, 34(11): 5415–5428.
  14. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 9729–9738.
  15. Boosting contrastive self-supervised learning with false negative cancellation. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, 2785–2795.
  16. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907.
  17. Contrastive clustering. In Proceedings of the AAAI conference on artificial intelligence, volume 35, 8547–8555.
  18. Nattel, S. 2002. New ideas about atrial fibrillation 50 years on. Nature, 415(6868): 219–226.
  19. The temple university hospital EEG data corpus. Frontiers in neuroscience, 10: 196.
  20. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748.
  21. U-time: A fully convolutional network for time series segmentation applied to sleep staging. Advances in Neural Information Processing Systems, 32.
  22. Learning Audio-Visual Source Localization via False Negative Aware Contrastive Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6420–6429.
  23. Self-supervised graph neural networks for improved electroencephalographic seizure analysis. arXiv preprint arXiv:2104.08336.
  24. Unsupervised representation learning for time series with temporal neighborhood coding. arXiv preprint arXiv:2106.00750.
  25. Contrastive learning based hybrid networks for long-tailed image classification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 943–952.
  26. Unsupervised feature learning via non-parametric instance discrimination. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3733–3742.
  27. Unsupervised embedding learning via invariant and spreading instance feature. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 6210–6219.
  28. Ts2vec: Towards universal representation of time series. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, 8980–8987.
  29. Imgcl: Revisiting graph contrastive learning on imbalanced node classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 11138–11146.
  30. Weakly supervised contrastive learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 10042–10051.
  31. Balanced contrastive learning for long-tailed visual recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6908–6917.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Xiyuan Jin (1 paper)
  2. Jing Wang (740 papers)
  3. Lei Liu (332 papers)
  4. Youfang Lin (52 papers)
Citations (1)
View on Semantic Scholar