On the Choice of a Model to Fit Data from an Exponential Family

• Published in: The Annals of statistics 1988-03, Vol.16 (1), p.342-355
• Main Author: HAUGHTON, D. M. A
• Format: Article
• Language: English
• Subjects: 62F12; 62H99; Coordinate systems; Curved models; Data models; Density; Exact sciences and technology; exponential families; Lebesgue measures; Mathematical functions; Mathematical integrals; Mathematical manifolds; Mathematics; Observed choices; Parametric inference; Parametric models; Probabilities; Probability and statistics; Schwarz's criterion; Sciences and techniques of general use; Statistics
• ISSN: 0090-5364; 2168-8966
• DOI: 10.1214/aos/1176350709

Let X1, ⋯, Xnbe iid observations coming from an exponential family. The problem of interest is this: Given a finite number of models mj(smoothly curved manifolds in Rk), choose the best model to fit the observations, with some penalty for choosing models with dimensions which are too large. A result of Schwarz is made more specific and is extended to the case where the models are curved manifolds. If S(Y, n, j) is--up to a constant C(n) independent of the model--the log of the posterior probability of the jth model, where the sample mean Yn= (1/n)∑n i=1Xihas been replaced by Y, Schwarz suggested an asymptotic expansion of S(Y, n, j) whose leading terms are$\gamma(Y, n, j) = n \sup_{\psi \in m_j \cap \Theta}(Y\psi - b(\psi)) - \frac{1}{2}k_j \log n$, in the case where the models are affine subspaces of Rk. We establish a similar asymptotic expansion, including the next term, with uniform bounds for Y in a compact neighborhood of ∇ b(θ), where θ is the true value of the parameter. We suggest a criterion for the choice of the best model that consists of maximizing the three leading terms in the expansion S(Y, n, j). We show that the criterion gives the correct model with probabilities Pn θ→ 1 as n → + ∞.