Distributed Multi-Agent Gaussian Regression via Finite-Dimensional Approximations

• Published in: IEEE transactions on pattern analysis and machine intelligence 2019-09, Vol.41 (9), p.2098-2111
• Main Authors: Pillonetto, Gianluigi, Schenato, Luca, Varagnolo, Damiano
• Format: Article
• Language: English
• Subjects: Algorithms; average consensus; Basis functions; Bayes methods; Complexity theory; Control Engineering; distributed estimation; Eigenvalues and eigenfunctions; Eigenvectors; Estimation; Gaussian process; Gaussian processes; Kernel; kernel-based regularization; Kernels; Multiagent systems; Noise measurement; nonparametric estimation; Nonparametric statistics; Reglerteknik; Regularization; sensor networks; Statistical analysis
• ISSN: 0162-8828; 1939-3539; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2018.2836422

We consider the problem of distributedly estimating Gaussian processes in multi-agent frameworks. Each agent collects few measurements and aims to collaboratively reconstruct a common estimate based on all data. Agents are assumed with limited computational and communication capabilities and to gather MM noisy measurements in total on input locations independently drawn from a known common probability density. The optimal solution would require agents to exchange all the MM input locations and measurements and then invert an M \times MM×M matrix, a non-scalable task. Differently, we propose two suboptimal approaches using the first EE orthonormal eigenfunctions obtained from the Karhunen-Loève (KL) expansion of the chosen kernel, where typically E\ll ME≪M. The benefits are that the computation and communication complexities scale with EE and not with MM, and computing the required statistics can be performed via standard average consensus algorithms. We obtain probabilistic non-asymptotic bounds that determine a priori the desired level of estimation accuracy, and new distributed strategies relying on Stein's unbiased risk estimate (SURE) paradigms for tuning the regularization parameters and applicable to generic basis functions (thus not necessarily kernel eigenfunctions) and that can again be implemented via average consensus. The proposed estimators and bounds are finally tested on both synthetic and real field data.