VAE-BRIDGE: Variational Autoencoder Filter for Bayesian Ridge Imputation of Missing Data

The missing data issue is often found in real-world datasets and it is usually handled with imputation strategies that replace the missing values with new data. Recently, generative models such as Variational Autoencoders have been applied for this imputation task. However, they were always used to...

Full description

Saved in:
Bibliographic Details
Main Authors: Pereira, Ricardo Cardoso, Abreu, Pedro Henriques, Rodrigues, Pedro Pereira
Format: Conference Proceeding
Language:English
Subjects:
Bayes methods
bayesian ridge
data imputation
Data models
Gaussian distribution
healthcare data
Informatics
Mathematical model
Medical services
missing data
variational autoencoder
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The missing data issue is often found in real-world datasets and it is usually handled with imputation strategies that replace the missing values with new data. Recently, generative models such as Variational Autoencoders have been applied for this imputation task. However, they were always used to perform the entire imputation, which has presented limited results when comparing to other state-of-the-art methods. In this work, a new approach called Variational Autoencoder Filter for Bayesian Ridge Imputation is introduced. It uses a Variational Autoencoder at the beginning of the imputation pipeline to filter the instances that are later fitted to a Bayesian ridge regression used to predict the new values. The approach was compared to four state-of-the-art imputation methods using 10 datasets from the healthcare context covering clinical trials, all injected with missing values under different rates. The proposed approach significantly outperformed the remaining methods in all settings, achieving an overall improvement between 26% and 67%.
ISSN:2161-4407
DOI:10.1109/IJCNN48605.2020.9206615