Sparse Representation Learning with Modified q-VAE towards Minimal Realization of World Model
Abstract: Extraction of low-dimensional latent space from high-dimensional observation data is essential to construct a real-time robot controller with a world model on the extracted latent space. However, there is no established method for tuning the dimension size of the latent space automatically, suffering from finding the necessary and sufficient dimension size, i.e. the minimal realization of the world model. In this study, we analyze and improve Tsallis-based variational autoencoder (q-VAE), and reveal that, under an appropriate configuration, it always facilitates making the latent space sparse. Even if the dimension size of the pre-specified latent space is redundant compared to the minimal realization, this sparsification collapses unnecessary dimensions, allowing for easy removal of them. We experimentally verified the benefits of the sparsification by the proposed method that it can easily find the necessary and sufficient six dimensions for a reaching task with a mobile manipulator that requires a six-dimensional state space. Moreover, by planning with such a minimal-realization world model learned in the extracted dimensions, the proposed method was able to exert a more optimal action sequence in real-time, reducing the reaching accomplishment time by around 20 %. The attached video is uploaded on youtube: https://youtu.be/-QjITrnxaRs
- Kobayashi T, Dean-Leon E, Guadarrama-Olvera JR, et al. Whole-body multicontact haptic human–humanoid interaction based on leader–follower switching: A robot dance of the “box step”. Advanced Intelligent Systems. 2022;4(2):2100038.
- Ha D, Schmidhuber J. World models. arXiv preprint arXiv:180310122. 2018;.
- Okada M, Taniguchi T. Dreaming: Model-based reinforcement learning by latent imagination without reconstruction. In: IEEE International Conference on Robotics and Automation; IEEE; 2021. p. 4209–4215.
- Kingma DP, Welling M. Auto-encoding variational bayes. In: International Conference on Learning Representations; 2014.
- Okada M, Taniguchi T. Variational inference mpc for bayesian model-based reinforcement learning. In: Conference on robot learning; PMLR; 2020. p. 258–272.
- Kobayashi T. q-vae for disentangled representation learning and latent dynamical systems. IEEE Robotics and Automation Letters. 2020;5(4):5669–5676.
- Tsallis C. Possible generalization of boltzmann-gibbs statistics. Journal of statistical physics. 1988;52(1-2):479–487.
- Suyari H, Tsukada M. Law of error in tsallis statistics. IEEE Transactions on Information Theory. 2005;51(2):753–757.
- Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv preprint arXiv:14126980. 2014;.
- Loaiza-Ganem G, Cunningham JP. The continuous bernoulli: fixing a pervasive error in variational autoencoders. Advances in Neural Information Processing Systems. 2019;32.
- Leurent E. An environment for autonomous driving decision-making [https://github.com/eleurent/highway-env]; 2018.
- Hoyer PO. Non-negative matrix factorization with sparseness constraints. Journal of machine learning research. 2004;5(9).
- Kobayashi T, Enomoto T. Towards autonomous driving of personal mobility with small and noisy dataset using tsallis-statistics-based behavioral cloning. arXiv preprint arXiv:211114294. 2021;.
- Sagi O, Rokach L. Ensemble learning: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery. 2018;8(4):e1249.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
