Bayesian nonlinear regression for large p small n problems

• Published in: Journal of multivariate analysis 2012-07, Vol.108, p.28-40
• Main Authors: Chakraborty, Sounak, Ghosh, Malay, Mallick, Bani K.
• Format: Article
• Language: English
• Subjects: Bayesian analysis; Bayesian hierarchical model; Bayesian hierarchical model Empirical Bayes Gibbs sampling Markov chain Monte Carlo Metropolis–Hastings algorithm Near infrared spectroscopy Relevance vector machine Reproducing kernel Hilbert space Support vector machine Vapnik’s ϵ-insensitive loss; Empirical Bayes; Gibbs sampling; Markov chain Monte Carlo; Metropolis–Hastings algorithm; Monte Carlo simulation; Near infrared spectroscopy; Regression analysis; Relevance vector machine; Reproducing kernel Hilbert space; Statistical inference; Studies; Support vector machine; Support vector machines; Vapnik’s [formula omitted]-insensitive loss
• ISSN: 0047-259X; 1095-7243
• DOI: 10.1016/j.jmva.2012.01.015

Statistical modeling and inference problems with sample sizes substantially smaller than the number of available covariates are challenging. This is known as large p small n problem. Furthermore, the problem is more complicated when we have multiple correlated responses. We develop multivariate nonlinear regression models in this setup for accurate prediction. In this paper, we introduce a full Bayesian support vector regression model with Vapnik’s ϵ-insensitive loss function, based on reproducing kernel Hilbert spaces (RKHS) under the multivariate correlated response setup. This provides a full probabilistic description of support vector machine (SVM) rather than an algorithm for fitting purposes. We have also introduced a multivariate version of the relevance vector machine (RVM). Instead of the original treatment of the RVM relying on the use of type II maximum likelihood estimates of the hyper-parameters, we put a prior on the hyper-parameters and use Markov chain Monte Carlo technique for computation. We have also proposed an empirical Bayes method for our RVM and SVM. Our methods are illustrated with a prediction problem in the near-infrared (NIR) spectroscopy. A simulation study is also undertaken to check the prediction accuracy of our models.