Spectral Temporal Contrastive Learning (2312.00966v2)
Abstract: Learning useful data representations without requiring labels is a cornerstone of modern deep learning. Self-supervised learning methods, particularly contrastive learning (CL), have proven successful by leveraging data augmentations to define positive pairs. This success has prompted a number of theoretical studies to better understand CL and investigate theoretical bounds for downstream linear probing tasks. This work is concerned with the temporal contrastive learning (TCL) setting where the sequential structure of the data is used instead to define positive pairs, which is more commonly used in RL and robotics contexts. In this paper, we adapt recent work on Spectral CL to formulate Spectral Temporal Contrastive Learning (STCL). We discuss a population loss based on a state graph derived from a time-homogeneous reversible Markov chain with uniform stationary distribution. The STCL loss enables to connect the linear probing performance to the spectral properties of the graph, and can be estimated by considering previously observed data sequences as an ensemble of MCMC chains.
- Vicreg: Variance-invariance-covariance regularization for self-supervised learning. arXiv preprint arXiv:2105.04906, 2021.
- Provable guarantees for self-supervised deep learning with spectral contrastive loss. CoRR, abs/2106.04156, 2021. URL https://arxiv.org/abs/2106.04156.
- Tclr: Temporal contrastive learning for video representation. Computer Vision and Image Understanding, 219:103406, 2022.
- Temporal abstractions-augmented temporally contrastive learning: An alternative to the laplacian in rl. In James Cussens and Kun Zhang, editors, Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, volume 180 of Proceedings of Machine Learning Research, pages 641–651. PMLR, 01–05 Aug 2022. URL https://proceedings.mlr.press/v180/erraqabi22a.html.
- Plan2vec: Unsupervised representation learning by latent plans. In Learning for Dynamics and Control, pages 935–946. PMLR, 2020.
- One-4-all: Neural potential fields for embodied navigation. arXiv preprint arXiv:2303.04011, 2023.
- A theoretical analysis of contrastive unsupervised representation learning. CoRR, abs/1902.09229, 2019. URL http://arxiv.org/abs/1902.09229.
- Predicting what you already know helps: Provable self-supervised learning. CoRR, abs/2008.01064, 2020. URL https://arxiv.org/abs/2008.01064.
- Contrastive learning, multi-view redundancy, and linear models. CoRR, abs/2008.10150, 2020. URL https://arxiv.org/abs/2008.10150.
- Contrastive and non-contrastive self-supervised learning recover global and local spectral embedding methods, 2022.
- On spectral clustering: Analysis and an algorithm. Advances in neural information processing systems, 14, 2001.
- A laplacian framework for option discovery in reinforcement learning. In International Conference on Machine Learning, pages 2295–2304. PMLR, 2017.
- The laplacian in RL: learning representations with efficient approximations. CoRR, abs/1810.04586, 2018. URL http://arxiv.org/abs/1810.04586.
- Yehuda Koren. On spectral graph drawing. In Tandy Warnow and Binhai Zhu, editors, Computing and Combinatorics, pages 496–508, Berlin, Heidelberg, 2003. Springer Berlin Heidelberg. ISBN 978-3-540-45071-9.
- C. Eckart and G. Young. The approximation of one matrix by another of lower rank. Psychometrika, 1(3):211–218, 1936. doi: 10.1007/BF02288367.
- Daniel Spielman. Spectral and algebraic graph theory. Yale lecture notes, draft of December, 4:47, 2019.
- Vip: Towards universal visual reward and representation via value-implicit pre-training. arXiv preprint arXiv:2210.00030, 2022.
- Learning a kernel matrix for nonlinear dimensionality reduction. In Proceedings ICML, page 106, 2004.