Neural architecture impact on identifying temporally extended Reinforcement Learning tasks (2310.03161v1)
Abstract: Inspired by recent developments in attention models for image classification and natural language processing, we present various Attention based architectures in reinforcement learning (RL) domain, capable of performing well on OpenAI Gym Atari-2600 game suite. In spite of the recent success of Deep Reinforcement learning techniques in various fields like robotics, gaming and healthcare, they suffer from a major drawback that neural networks are difficult to interpret. We try to get around this problem with the help of Attention based models. In Attention based models, extracting and overlaying of attention map onto images allows for direct observation of information used by agent to select actions and easier interpretation of logic behind the chosen actions. Our models in addition to playing well on gym-Atari environments, also provide insights on how agent perceives its environment. In addition, motivated by recent developments in attention based video-classification models using Vision Transformer, we come up with an architecture based on Vision Transformer, for image-based RL domain too. Compared to previous works in Vision Transformer, our model is faster to train and requires fewer computational resources. 3
- Abien Fred Agarap “Deep Learning using Rectified Linear Units (ReLU)” cite arxiv:1803.08375Comment: 7 pages, 11 figures, 9 tables, 2018 URL: http://arxiv.org/abs/1803.08375
- Jimmy Lei Ba, Jamie Ryan Kiros and Geoffrey E. Hinton “Layer Normalization”, 2016 arXiv:1607.06450 [stat.ML]
- Gedas Bertasius, Heng Wang and Lorenzo Torresani “Is Space-Time Attention All You Need for Video Understanding?” In CoRR abs/2102.05095, 2021 arXiv: https://arxiv.org/abs/2102.05095
- G. Bradski “The OpenCV Library” In Dr. Dobb’s Journal of Software Tools, 2000
- “OpenAI Gym”, 2016 arXiv:1606.01540 [cs.LG]
- “Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context” In CoRR abs/1901.02860, 2019 arXiv: http://arxiv.org/abs/1901.02860
- “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale” In International Conference on Learning Representations, 2021 URL: https://openreview.net/forum?id=YicbFdNTTy
- “IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures” In CoRR abs/1802.01561, 2018 arXiv: http://arxiv.org/abs/1802.01561
- “Visualizing and Understanding Atari Agents” In CoRR abs/1711.00138, 2017 arXiv: http://arxiv.org/abs/1711.00138
- “Deep Residual Learning for Image Recognition” In CoRR abs/1512.03385, 2015 arXiv: http://arxiv.org/abs/1512.03385
- “Deep Residual Learning for Image Recognition” In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778 DOI: 10.1109/CVPR.2016.90
- “Bridging Nonlinearities and Stochastic Regularizers with Gaussian Error Linear Units” In CoRR abs/1606.08415, 2016 arXiv: http://arxiv.org/abs/1606.08415
- Shakti Kumar, Jerrod Parker and Panteha Naderian “Adaptive Transformers in RL” In CoRR abs/2004.03761, 2020 arXiv: https://arxiv.org/abs/2004.03761
- “TorchBeast: A PyTorch Platform for Distributed RL” In arXiv preprint arXiv:1910.03552, 2019 URL: https://github.com/facebookresearch/torchbeast
- “Reproducing “Towards Interpretable ReinforcementLearning Using Attention Augmented Agents”” Submitted to NeurIPS 2019 Reproducibility Challenge, 2020 URL: https://openreview.net/forum?id=BJgtDa9GaH
- “Human-level control through deep reinforcement learning” In Nature 518.7540, 2015, pp. 529–533 DOI: 10.1038/nature14236
- “Towards Interpretable Reinforcement Learning Using Attention Augmented Agents” In CoRR abs/1906.02500, 2019 arXiv: http://arxiv.org/abs/1906.02500
- “Stabilizing Transformers for Reinforcement Learning” In CoRR abs/1910.06764, 2019 arXiv: http://arxiv.org/abs/1910.06764
- Scipy “Savitzky-Golay filter” URL: https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.savgol_filter.html
- “Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting” In CoRR abs/1506.04214, 2015 arXiv: http://arxiv.org/abs/1506.04214
- Richard S. Sutton and Andrew G. Barto “Reinforcement Learning: An Introduction” The MIT Press, 2018 URL: http://incompleteideas.net/book/the-book-2nd.html
- TensorFlow “Transformer model for language understanding” URL: https://www.tensorflow.org/text/tutorials/transformer
- “Attention is All you Need” In Advances in Neural Information Processing Systems 30 Curran Associates, Inc., 2017 URL: https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
- Xiaofeng Yuan, Lin Li and Yalin Wang “Nonlinear Dynamic Soft Sensor Modeling With Supervised Long Short-Term Memory Network” In IEEE Transactions on Industrial Informatics 16.5, 2020, pp. 3168–3176 DOI: 10.1109/TII.2019.2902129
- Victor Vadakechirayath George (1 paper)