Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
92 tokens/sec
Gemini 2.5 Pro Premium
50 tokens/sec
GPT-5 Medium
22 tokens/sec
GPT-5 High Premium
21 tokens/sec
GPT-4o
97 tokens/sec
DeepSeek R1 via Azure Premium
87 tokens/sec
GPT OSS 120B via Groq Premium
459 tokens/sec
Kimi K2 via Groq Premium
230 tokens/sec
2000 character limit reached

BarlowRL: Barlow Twins for Data-Efficient Reinforcement Learning (2308.04263v3)

Published 8 Aug 2023 in cs.LG

Abstract: This paper introduces BarlowRL, a data-efficient reinforcement learning agent that combines the Barlow Twins self-supervised learning framework with DER (Data-Efficient Rainbow) algorithm. BarlowRL outperforms both DER and its contrastive counterpart CURL on the Atari 100k benchmark. BarlowRL avoids dimensional collapse by enforcing information spread to the whole space. This helps RL algorithms to utilize uniformly spread state representation that eventually results in a remarkable performance. The integration of Barlow Twins with DER enhances data efficiency and achieves superior performance in the RL tasks. BarlowRL demonstrates the potential of incorporating self-supervised learning techniques to improve RL algorithms.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Deep reinforcement learning at the edge of the statistical precipice. In Neural Information Processing Systems, 2021.
  2. Self-supervised classification network. ArXiv, abs/2103.10994, 2021.
  3. Masked siamese networks for label-efficient learning. In European Conference on Computer Vision, 2022.
  4. Vicreg: Variance-invariance-covariance regularization for self-supervised learning. ArXiv, abs/2105.04906, 2021.
  5. A distributional perspective on reinforcement learning. In International Conference on Machine Learning, 2017.
  6. Dota 2 with large scale deep reinforcement learning. ArXiv, abs/1912.06680, 2019.
  7. Deep clustering for unsupervised learning of visual features. ArXiv, abs/1807.05520, 2018.
  8. Emerging properties in self-supervised vision transformers. 2021 IEEE/CVF International Conference on Computer Vision, pages 9630–9640, 2021.
  9. Exploring simple siamese representation learning. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
  10. Learning a similarity metric discriminatively, with application to face verification. 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), 1:539–546 vol. 1, 2005.
  11. Whitening for self-supervised representation learning. In International Conference on Machine Learning, 2020.
  12. Noisy networks for exploration. ArXiv, abs/1706.10295, 2017.
  13. Bootstrap your own latent: A new approach to self-supervised learning. ArXiv, abs/2006.07733, 2020.
  14. Deep reinforcement learning with double q-learning. In AAAI Conference on Artificial Intelligence, 2015.
  15. When to use parametric models in reinforcement learning? ArXiv, abs/1906.05243, 2019.
  16. Rainbow: Combining improvements in deep reinforcement learning. In AAAI Conference on Artificial Intelligence, 2017.
  17. Model-based reinforcement learning for atari. ArXiv, abs/1903.00374, 2019.
  18. Image augmentation is all you need: Regularizing deep reinforcement learning from pixels. ArXiv, abs/2004.13649, 2020.
  19. Transformers are sample efficient world models. ArXiv, abs/2209.00588, 2022.
  20. Human-level control through deep reinforcement learning. Nature, 518:529–533, 2015.
  21. The primacy bias in deep reinforcement learning. In International Conference on Machine Learning, 2022.
  22. Self-supervised learning with an information maximization criterion. ArXiv, abs/2209.07999, 2022.
  23. Prioritized experience replay. CoRR, abs/1511.05952, 2015.
  24. Mastering atari, go, chess and shogi by planning with a learned model. Nature, 588:604 – 609, 2019.
  25. Data-efficient reinforcement learning with self-predictive representations. In International Conference on Learning Representations, 2020.
  26. Pretraining representations for data-efficient reinforcement learning. In Neural Information Processing Systems, 2021.
  27. Bigger, better, faster: Human-level atari with human-level efficiency. ArXiv, abs/2305.19452, 2023.
  28. Mastering the game of go with deep neural networks and tree search. Nature, 529:484–489, 2016.
  29. Curl: Contrastive unsupervised representations for reinforcement learning. In International Conference on Machine Learning, 2020.
  30. Introduction to reinforcement learning. 1998.
  31. Representation learning with contrastive predictive coding. ArXiv, abs/1807.03748, 2018.
  32. Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature, pages 1–5, 2019.
  33. X. Wang and Abhinav Kumar Gupta. Unsupervised learning of visual representations using videos. 2015 IEEE International Conference on Computer Vision (ICCV), pages 2794–2802, 2015.
  34. Dueling network architectures for deep reinforcement learning. In International Conference on Machine Learning, 2015.
  35. Unsupervised feature learning via non-parametric instance-level discrimination. ArXiv, abs/1805.01978, 2018.
  36. Mastering atari games with limited data. ArXiv, abs/2111.00210, 2021.
  37. Barlow twins: Self-supervised learning via redundancy reduction. In International Conference on Machine Learning, 2021.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now