Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
133 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Inferring Hierarchical Structure in Multi-Room Maze Environments (2306.13546v1)

Published 23 Jun 2023 in cs.AI

Abstract: Cognitive maps play a crucial role in facilitating flexible behaviour by representing spatial and conceptual relationships within an environment. The ability to learn and infer the underlying structure of the environment is crucial for effective exploration and navigation. This paper introduces a hierarchical active inference model addressing the challenge of inferring structure in the world from pixel-based observations. We propose a three-layer hierarchical model consisting of a cognitive map, an allocentric, and an egocentric world model, combining curiosity-driven exploration with goal-oriented behaviour at the different levels of reasoning from context to place to motion. This allows for efficient exploration and goal-directed search in room-structured mini-grid environments.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (22)
  1. Self-labelling via simultaneous clustering and representation learning. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=Hyx-jyBFPr.
  2. Unifying count-based exploration and intrinsic motivation. In Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016. URL https://proceedings.neurips.cc/paper_files/paper/2016/file/afda332245e2af431fb7b672a68b659d-Paper.pdf.
  3. Exploration by random network distillation. CoRR, abs/1810.12894, 2018. URL http://arxiv.org/abs/1810.12894.
  4. Minimalistic gridworld environment for openai gym. https://github.com/maximecb/gym-minigrid, 2018.
  5. Neural scene representation and rendering. Science, 360(6394):1204–1210, 2018. doi: 10.1126/science.aar6170. URL https://www.science.org/doi/abs/10.1126/science.aar6170.
  6. Active inference and learning. Neuroscience & Biobehavioral Reviews, 68:862–879, 2016. ISSN 0149-7634. doi: https://doi.org/10.1016/j.neubiorev.2016.06.022. URL https://www.sciencedirect.com/science/article/pii/S0149763416301336.
  7. Clone-structured graph representations enable flexible learning and vicarious evaluation of cognitive maps. Nature Communications, 12, 04 2021. doi: 10.1038/s41467-021-22559-5.
  8. Mastering atari with discrete world models. CoRR, abs/2010.02193, 2020. URL https://arxiv.org/abs/2010.02193.
  9. Planning and navigation as active inference. 12 2017. doi: 10.1101/230599.
  10. Choreographer: Learning and adapting skills in imagination. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=PhkWyijGi5b.
  11. Ratslam: a hippocampal model for simultaneous localization and mapping. 1:403–408 Vol.1, 2004. doi: 10.1109/ROBOT.2004.1307183.
  12. Structure learning enhances concept formation in synthetic active inference agents. PLOS ONE, 17(11):1–34, 11 2022. doi: 10.1371/journal.pone.0277199. URL https://doi.org/10.1371/journal.pone.0277199.
  13. Interesting object, curious agent: Learning task-agnostic exploration. CoRR, abs/2111.13119, 2021. URL https://arxiv.org/abs/2111.13119.
  14. Active Inference: The Free Energy Principle in Mind, Brain, and Behavior. The MIT Press, 2022. ISBN 978-0-262-36997-8. doi: 10.7551/mitpress/12441.001.0001. URL https://direct.mit.edu/books/oa-monograph/5299/Active-InferenceThe-Free-Energy-Principle-in-Mind.
  15. Evaluating long-term memory in 3d mazes, 2022.
  16. Curiosity-driven exploration by self-supervised prediction. CoRR, abs/1705.05363, 2017. URL http://arxiv.org/abs/1705.05363.
  17. Generalized simultaneous localization and mapping (g-SLAM) as unification framework for natural and artificial intelligences: towards reverse engineering the hippocampal/entorhinal system and principles of high-level cognition. October 2021. doi: 10.31234/osf.io/tdw82. URL https://doi.org/10.31234/osf.io/tdw82.
  18. The hippocampal formation as a hierarchical generative model supporting generative replay and continual learning. Progress in Neurobiology, 217:102329, 2022. ISSN 0301-0082. doi: https://doi.org/10.1016/j.pneurobio.2022.102329. URL https://www.sciencedirect.com/science/article/pii/S0301008222001150.
  19. Home run: Finding your way home by imagining trajectories. In Active Inference, pp.  210–221, Cham, 2023. Springer Nature Switzerland. ISBN 978-3-031-28719-0.
  20. Active vision for robot manipulators using the free energy principle. Frontiers in Neurorobotics, 15, 2021. ISSN 1662-5218. doi: 10.3389/fnbot.2021.642780. URL https://www.frontiersin.org/articles/10.3389/fnbot.2021.642780.
  21. Chunking space and time with information geometry. In NeurIPS 2022 Workshop on Information-Theoretic Principles in Cognitive Systems, 2022. URL https://openreview.net/forum?id=Fq_HdQj6fOE.
  22. Learning generative state space models for active inference. Frontiers in Computational Neuroscience, 14, 2020. URL https://www.frontiersin.org/article/10.3389/fncom.2020.574372.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now