Missing value imputation on missing completely at random data using multilayer perceptrons

Data mining is based on data files which usually contain errors in the form of missing values. This paper focuses on a methodological framework for the development of an automated data imputation model based on artificial neural networks. Fifteen real and simulated data sets are exposed to a perturb...

Full description

Saved in:
Bibliographic Details
Published in:Neural networks 2011, Vol.24 (1), p.121-129
Main Authors: Silva-Ramírez, Esther-Lydia, Pino-Mejías, Rafael, López-Coello, Manuel, Cubiles-de-la-Vega, María-Dolores
Format: Article
Language:English
Subjects:
Algorithmics. Computability. Computer arithmetics
Algorithms
Applied sciences
Artificial intelligence
Computer science
control theory
systems
Computer Simulation
Connectionism. Neural networks
Data Interpretation, Statistical
Data processing. List processing. Character string processing
Exact sciences and technology
Hot-deck model
Humans
Imputation
Information Systems
Mean/mode model
Memory organisation. Data processing
Missing data
Multilayer perceptron
Neural Networks (Computer)
Regression model
Software
Theoretical computing
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Data mining is based on data files which usually contain errors in the form of missing values. This paper focuses on a methodological framework for the development of an automated data imputation model based on artificial neural networks. Fifteen real and simulated data sets are exposed to a perturbation experiment, based on the random generation of missing values. These data set sizes range from 47 to 1389 records. A perturbation experiment was performed for each data set where the probability of missing value was set to 0.05. Several architectures and learning algorithms for the multilayer perceptron are tested and compared with three classic imputation procedures: mean/mode imputation, regression and hot-deck. The obtained results, considering different performance measures, not only suggest this approach improves the quality of a database with missing values, but also the best results are clearly obtained using the Multilayer Perceptron model in data sets with categorical variables. Three learning rules (Levenberg–Marquardt, BFGS Quasi-Newton and Conjugate Gradient Fletcher–Reeves Update) and a small number of hidden nodes are recommended.
ISSN:0893-6080
1879-2782
DOI:10.1016/j.neunet.2010.09.008