Simple Artificial Neural Networks That Match Probability and Exploit and Explore When Confronting a Multiarmed Bandit

• Published in: IEEE transaction on neural networks and learning systems 2009-08, Vol.20 (8), p.1368-1371
• Main Authors: Dawson, M., Dupuis, B., Spetch, M.L., Kelly, D.M.
• Format: Article
• Language: English
• Subjects: Animals; Applied sciences; Arm; Artificial Intelligence; Artificial neural networks; Choice Behavior; Computer science; control theory; systems; Computer Simulation; Conditioning, Operant; Connectionism. Neural networks; Contingency; Decision support systems; Economic forecasting; Exact sciences and technology; Frequency; Genetic algorithms; Humans; Instrumental learning; Law; Learning; Matching; multiarmed bandit; Networks; Neural networks; Neural Networks (Computer); operant conditioning; perceptron; Probability; probability matching; Psychology; Reinforcement; Reinforcement (Psychology); Reinforcement Schedule
• ISSN: 1045-9227; 2162-237X; 1941-0093; 2162-2388
• DOI: 10.1109/TNN.2009.2025588

The matching law (Herrnstein 1961) states that response rates become proportional to reinforcement rates; this is related to the empirical phenomenon called probability matching (Vulkan 2000). Here, we show that a simple artificial neural network generates responses consistent with probability matching. This behavior was then used to create an operant procedure for network learning. We use the multiarmed bandit (Gittins 1989), a classic problem of choice behavior, to illustrate that operant training balances exploiting the bandit arm expected to pay off most frequently with exploring other arms. Perceptrons provide a medium for relating results from neural networks, genetic algorithms, animal learning, contingency theory, reinforcement learning, and theories of choice.