Efficient estimation with incomplete data via generalised ANOVA decomposition

We study the efficient estimation of a class of mean functionals in settings where a complete multivariate dataset is complemented by additional datasets recording subsets of the variables of interest. These datasets are allowed to have a general, in particular non-monotonic, structure. Our main con...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-09
Main Author: Berrett, Thomas B
Format: Article
Language:English
Subjects:
Asymptotic methods
Confidence intervals
Datasets
Decomposition
Error analysis
Function space
Influence functions
Lower bounds
Mean
Multivariate analysis
Statistical analysis
Variance analysis
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We study the efficient estimation of a class of mean functionals in settings where a complete multivariate dataset is complemented by additional datasets recording subsets of the variables of interest. These datasets are allowed to have a general, in particular non-monotonic, structure. Our main contribution is to characterise the asymptotic minimal mean squared error for these problems and to introduce an estimator whose risk approximately matches this lower bound. We show that the efficient rescaled variance can be expressed as the minimal value of a quadratic optimisation problem over a function space, thus establishing a fundamental link between these estimation problems and the theory of generalised ANOVA decompositions. Our estimation procedure uses iterated nonparametric regression to mimic an approximate influence function derived through gradient descent. We prove that this estimator is approximately normally distributed, provide an estimator of its variance and thus develop confidence intervals of asymptotically minimal width. Finally we study a more direct estimator, which can be seen as a U-statistic with a data-dependent kernel, showing that it is also efficient under stronger regularity conditions.
ISSN:2331-8422