Divisive Gaussian Processes for Nonstationary Regression

• Published in: IEEE transaction on neural networks and learning systems 2014-11, Vol.25 (11), p.1991-2003
• Main Authors: Munoz-Gonzalez, Luis, Lazaro-Gredilla, Miguel, Figueiras-Vidal, Anibal R.
• Format: Article
• Language: English
• Subjects: Approximation; Approximation algorithms; Approximation methods; Covariance; Covariance matrices; Elliptical slice sampling; expectation propagation (EP); Gaussian; Gaussian distribution; Gaussian process (GP); Ground penetrating radar; heteroscedastic regression; Inference; Markov analysis; Mathematical analysis; Mathematical models; Monte Carlo simulation; Noise; nonstationarity; Regression
• ISSN: 2162-237X; 2162-2388
• DOI: 10.1109/TNNLS.2014.2301951

Standard Gaussian process regression (GPR) assumes constant noise power throughout the input space and stationarity when combined with the squared exponential covariance function. This can be unrealistic and too restrictive for many real-world problems. Nonstationarity can be achieved by specific covariance functions, though prior knowledge about this nonstationarity can be difficult to obtain. On the other hand, the homoscedastic assumption is needed to allow GPR inference to be tractable. In this paper, we present a divisive GPR model which performs nonstationary regression under heteroscedastic noise using the pointwise division of two nonparametric latent functions. As the inference on the model is not analytically tractable, we propose a variational posterior approximation using expectation propagation (EP) which allows for accurate inference at reduced cost. We have also made a Markov chain Monte Carlo implementation with elliptical slice sampling to assess the quality of the EP approximation. Experiments support the usefulness of the proposed approach.