Neural Stored-program Memory

Neural networks powered with external memory simulate computer behaviors. These models, which use the memory to store data for a neural controller, can learn algorithms and other complex tasks. In this paper, we introduce a new memory to store weights for the controller, analogous to the stored-prog...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2019-12
Main Authors: Le, Hung, Tran, Truyen, Venkatesh, Svetha
Format: Article
Language:English
Subjects:
Algorithms
Computer memory
Computer simulation
Control systems
Controllers
Machine learning
Neural networks
Switching theory
Task complexity
Turing machines
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Neural networks powered with external memory simulate computer behaviors. These models, which use the memory to store data for a neural controller, can learn algorithms and other complex tasks. In this paper, we introduce a new memory to store weights for the controller, analogous to the stored-program memory in modern computer architectures. The proposed model, dubbed Neural Stored-program Memory, augments current memory-augmented neural networks, creating differentiable machines that can switch programs through time, adapt to variable contexts and thus resemble the Universal Turing Machine. A wide range of experiments demonstrate that the resulting machines not only excel in classical algorithmic problems, but also have potential for compositional, continual, few-shot learning and question-answering tasks.
ISSN:2331-8422