Selected fromarXiv
Translated by Machine Heart
Contributors:Lu, Siyuan
Recently, researchers from DeepMind and the CoMPLEX group at University College London proposed a Relational Recurrent Neural Network that utilizes a novel memory module, RMC, to address the challenges of relational reasoning tasks that standard memory architectures struggle to perform. This method has made significant advancements in reinforcement learning domains (such as Mini PacMan), program evaluation, and language modeling, achieving state-of-the-art results on the WikiText-103, Project Gutenberg, and GigaWord datasets.
Paper: Relational Recurrent Neural Networks
Paper link: https://arxiv.org/abs/1806.01822
Abstract: Memory-based neural networks model temporal data through long-term memory information. However, it remains unclear whether they possess the capability to perform complex relational reasoning on memory information. In this paper, we first confirm that standard memory architectures may struggle with tasks that require a deep understanding of how entities are connected (i.e., tasks involving relational reasoning). We then improve these shortcomings using a new memory module, the Relational Memory Core (RMC), which employs multi-head dot-product attention to allow memories to influence each other. Finally, we test RMC on a series of tasks that can benefit from more robust relational reasoning across sequences, with results indicating significant advancements in reinforcement learning domains (such as Mini PacMan), program evaluation, and language modeling, achieving state-of-the-art results on the WikiText-103, Project Gutenberg, and GigaWord datasets.
1 Introduction
Humans use complex memory systems to acquire and reason about important information without needing to recall when the information was initially perceived [1, 2]. In neural network research, successful methods for modeling sequential data also utilize memory systems, such as LSTM [3] and memory-augmented neural networks [4–7]. With enhanced memory capacity, bounded computational overhead over time, and the ability to handle gradient vanishing, these networks learn to associate events across different times, thus excelling at storing and retrieving information.
Here we propose that considering memory interactions alongside information storage and retrieval can yield significant benefits. Although current models can learn to segment and associate distributed, vectorized memories, they do not excel at explicitly completing these processes. We hypothesize that models proficient in doing so may better understand the associations of memories, thereby gaining a stronger ability to perform relational reasoning on temporal data. We first demonstrate through a relational reasoning task that emphasizes sequential information that current models indeed struggle in this regard. The novel RMC, utilizing multi-head dot-product attention, facilitates memory interactions, allowing us to address and analyze this demonstration task. We then apply RMC to a series of tasks (which may benefit from more pronounced memory interactions), resulting in a potential increase in memory capacity to handle relational reasoning over time: on partially observable reinforcement learning tasks, program evaluation, and language modeling tasks on the WikiText-103, Project Gutenberg, and GigaWord datasets.
3 Model
Our primary design principle is to provide an architecture backbone that allows the model to learn to segment information and compute interactions between the segmented information. To this end, we combine LSTM building blocks, memory-augmented neural networks, and non-local networks (specifically, the Transformer seq2seq model [19]) to form the core network. Similar to memory-augmented architectures, we consider using a fixed set of memory cells, but we utilize attention mechanisms for interactions between memory cells. As previously mentioned, our approach differs from prior research in that we apply attention mechanisms to memories at a single time step, rather than across all previous representations computed from all prior observations.
Figure 1: Relational Memory Core. (a) RMC takes the previous memory matrix and input vector as inputs, which are passed to the MHDPA module (A). (b) Using Query row-shared weights W^q, Key row-shared weights W^k, and Value row-shared weights W^v, we compute the linear projection for each memory cell. (c) Queries, keys, and values are compiled into matrices to compute softmax(QK^T)V. The output of this computation is a new memory, where information is mixed according to the attention weights of the memories. An MLP is applied row-wise to the output of the MHDPA module (a), resulting in a memory matrix that serves as the gated matrix, which is the core output or the next memory state.
Figure 2: Tasks. We test RMC on a series of supervised and reinforcement learning tasks. The Nth Farthest demonstration task and language modeling task are noteworthy. In the former, the solution requires explicit relational reasoning, as the model must classify the relationships between distances of vectors rather than classifying the vectors themselves. The latter tests the model on a large amount of natural data, allowing us to compare performance with finely-tuned models.
5 Results
Figure 3: Model Analysis. Each row describes the attention matrix for a specific sequence at each time step. The text below indicates the specific task of that sequence, which is encoded and input into the model. We highlight the vectors referenced in the task in red: if the model selects the vector that is the 2nd farthest from vector 7, the highlighted point is the time point of vector 7 that was input into the model. A single attention matrix shows the attention weights from one specific memory cell (y-axis) to another memory cell (column), or input (offset column), with numbers representing memory cells and “input” indicating input word embeddings.
Table 1: Per-character test accuracy on program evaluation and memory tasks.
Table 2: Validation perplexity and test perplexity on the WikiText-103, Project Gutenberg, and GigaWord v5 datasets.
This article is translated by Machine Heart, please contact this public account for authorization to reproduce.
✄————————————————
Join Machine Heart (Full-time Reporter / Intern): [email protected]
Submissions or requests for coverage: content@jiqizhixin.com
Advertising & Business Cooperation: [email protected]