A note on model selection for small sample regression

The risk estimator called “Direct Eigenvalue Estimator” (DEE) is studied. DEE was developed for small sample regression. In contrast to many existing model selection criteria, derivation of DEE requires neither any asymptotic assumption nor any prior knowledge about the noise variance and the noise...

Full description

Saved in:
Bibliographic Details
Published in:Machine learning 2017-11, Vol.106 (11), p.1839-1862
Main Authors: Kawakita, Masanori, Takeuchi, Jun’ichi
Format: Article
Language:English
Subjects:
Artificial Intelligence
Computer Science
Control
Derivation
Estimators
Machine learning
Mathematical models
Mechatronics
Natural Language Processing (NLP)
Regression
Risk
Robotics
Simulation and Modeling
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The risk estimator called “Direct Eigenvalue Estimator” (DEE) is studied. DEE was developed for small sample regression. In contrast to many existing model selection criteria, derivation of DEE requires neither any asymptotic assumption nor any prior knowledge about the noise variance and the noise distribution. It was reported that DEE performed well in small sample cases but DEE performed a little worse than the state-of-the-art ADJ. This seems somewhat counter-intuitive because DEE was developed for specifically regression problem by exploiting available information exhaustively, while ADJ was developed for general setting. In this paper, we point out that the derivation of DEE includes an inappropriate part, notwithstanding the resultant form of DEE being valid in a sense. As its result, DEE cannot derive its potential. We introduce a class of ‘valid’ risk estimators based on the idea of DEE and show that better risk estimators (mDEE) can be found in the class. By numerical experiments, we verify that mDEE often performs better than or at least equally the original DEE and ADJ.
ISSN:0885-6125
1573-0565
DOI:10.1007/s10994-017-5645-5