A Novel Framework for Imputation of Missing Values in Databases

• Published in: IEEE transactions on systems, man and cybernetics. Part A, Systems and humans man and cybernetics. Part A, Systems and humans, 2007-09, Vol.37 (5), p.692-709
• Main Authors: Farhangfar, A., Kurgan, L.A., Pedrycz, W.
• Format: Article
• Language: English
• Subjects: Accuracy; Algorithms; Asymptotic properties; Biomedical engineering; Biomedical equipment; Complexity; Computation; Computational complexity; Councils; Cybernetics; Data analysis; Dealing; Filling; Human; Instruments; Medical services; missing values; multiple imputation (MI); single imputation; Studies; Testing
• ISSN: 1083-4427; 2168-2216; 1558-2426; 2168-2232
• DOI: 10.1109/TSMCA.2007.902631

Many of the industrial and research databases are plagued by the problem of missing values. Some evident examples include databases associated with instrument maintenance, medical applications, and surveys. One of the common ways to cope with missing values is to complete their imputation (filling in). Given the rapid growth of sizes of databases, it becomes imperative to come up with a new imputation methodology along with efficient algorithms. The main objective of this paper is to develop a unified framework supporting a host of imputation methods. In the development of this framework, we require that its usage should (on average) lead to the significant improvement of accuracy of imputation while maintaining the same asymptotic computational complexity of the individual methods. Our intent is to provide a comprehensive review of the representative imputation techniques. It is noticeable that the use of the framework in the case of a low-quality single-imputation method has resulted in the imputation accuracy that is comparable to the one achieved when dealing with some other advanced imputation techniques. We also demonstrate, both theoretically and experimentally, that the application of the proposed framework leads to a linear computational complexity and, therefore, does not affect the asymptotic complexity of the associated imputation method.