Statistical Mechanics of On-Line Learning Using Correlated Examples and Its Optimal Scheduling

We theoretically study the generalization capability of on-line learning using several correlated input vectors in each update in a statistical-mechanical manner. We consider a model organized with linear perceptrons with Gaussian noise. First, in a noiseless case, we analytically derive the optimal...

Full description

Saved in:
Bibliographic Details
Published in:Journal of the Physical Society of Japan 2017-08, Vol.86 (8), p.84804
Main Authors: Fujii, Takashi, Ito, Hidetaka, Miyoshi, Seiji
Format: Article
Language:English
Subjects:
Algorithms
Correlation analysis
Learning
Normal distribution
Online instruction
Scheduling
Statistical mechanics
Statistics
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We theoretically study the generalization capability of on-line learning using several correlated input vectors in each update in a statistical-mechanical manner. We consider a model organized with linear perceptrons with Gaussian noise. First, in a noiseless case, we analytically derive the optimal learning rate as a function of the number of examples used in one update and their correlation. Next, we analytically show that the use of correlated examples is effective if the optimal learning rate is used, even when there is some noise. Furthermore, we propose a novel algorithm that raises the generalization capability by increasing the number of examples used in one update with time.
ISSN:0031-9015
1347-4073
DOI:10.7566/JPSJ.86.084804