Relational recurrent neural networks (1806.01822v2)

Abstract: Memory-based neural networks model temporal data by leveraging an ability to remember information for long periods. It is unclear, however, whether they also have an ability to perform complex relational reasoning with the information they remember. Here, we first confirm our intuitions that standard memory architectures may struggle at tasks that heavily involve an understanding of the ways in which entities are connected -- i.e., tasks involving relational reasoning. We then improve upon these deficits by using a new memory module -- a \textit{Relational Memory Core} (RMC) -- which employs multi-head dot product attention to allow memories to interact. Finally, we test the RMC on a suite of tasks that may profit from more capable relational reasoning across sequential information, and show large gains in RL domains (e.g. Mini PacMan), program evaluation, and LLMing, achieving state-of-the-art results on the WikiText-103, Project Gutenberg, and GigaWord datasets.

• The paper introduces a Relational Memory Core (RMC) that uses multi-head dot product attention to explicitly connect memory vectors for enhanced relational reasoning.
• It outperforms traditional models like LSTMs by achieving 91% accuracy on the Nth Farthest Task and lowering perplexity on large-scale language modeling benchmarks.
• The results demonstrate RMC’s effectiveness in reinforcement learning and program evaluation, highlighting its potential for complex temporal reasoning tasks.

Relational Recurrent Neural Networks

The paper introduces a novel architecture, the Relational Memory Core (RMC), designed to improve relational reasoning capabilities in memory-augmented neural networks. The importance of this advancement is anchored in the limitations of current memory architectures, which often struggle with tasks requiring complex temporal relational reasoning. RMC represents a significant enhancement by employing multi-head dot product attention to facilitate interactions among memories, aligning well with the inductive biases necessary for relational reasoning.

Key Contributions

The core contribution of the paper is the introduction of a Relational Memory Core (RMC), a module specifically designed to enhance relational reasoning in memory-based networks. The RMC leverages multi-head dot product attention, inspired by the Transformer architecture, to allow memories to interact with each other efficiently. This architectural choice enables the model to explicitly relate memory vectors, which is conjectured to improve performance on tasks requiring relational reasoning over time.

Experimentation and Results

1. $N^{th}$ Farthest Task: This task, crafted to test the relational reasoning capability, revealed the stark superiority of the RMC over traditional models like LSTMs and Dynamic Neural Computation (DNC). The RMC demonstrated significant robustness and accuracy improvements, achieving a 91% accuracy even in environments demanding high memory fidelity.
2. Program Evaluation: Evaluated on tasks from the Learning to Execute dataset, the RMC showcased its proficiency by outperforming standard methods and baselines, including LSTMs and EntNet, particularly in tasks requiring symbolic manipulation and programmatic reasoning.
3. Reinforcement Learning: The RMC brought substantial improvements in partially observable environments, like Mini Pacman, by excelling in memory-dependent reasoning and planning tasks, where it significantly surpassed LSTM baselines.
4. LLMing: Achieving lower perplexity scores across datasets like WikiText-103 and GigaWord, the RMC demonstrated improved capability in handling sequential reasoning tasks over extensive textual datasets, showcasing both data efficiency and effectiveness.

Implications and Future Directions

The RMC's design introduces the potential for enhanced relational reasoning capabilities in recurrent neural architectures. By explicitly modeling interactions among memories through attention mechanisms, the RMC aligns interaction processes with the inherent task requirements, resulting in improved performance across various domains. Future work could explore the integration of RMC-like components with scalable models or combine them with growing buffers for embedding past states, which may empower models to tackle more extensive and complex temporal problems even more effectively.

The RMC evidences a promising direction for advancing neural network architectures in tasks where relational reasoning across time is crucial. As research progresses, the RMC's novel approach may inspire further enhancements in memory-augmented networks and broader AI applications, contributing to the development of more adept models in reasoning and decision-making tasks.