Sublinear Time Low-Rank Approximation of Positive Semidefinite Matrices

• Main Authors: Musco, Cameron, Woodruff, David P.
• Format: Conference Proceeding
• Language: English
• Subjects: Approximation algorithms; Covariance matrices; Kernel; leverage score sampling; low-rank approximation; Matrix decomposition; matrix sketching; Runtime; sublinear time algorithms; Symmetric matrices
• ISSN: 0272-5428
• DOI: 10.1109/FOCS.2017.68

We show how to compute a relative-error low-rank approximation to any positive semidefinite (PSD) matrix in sublinear time, i.e., for any n x n PSD matrix A, in Õ(n · poly(k/ε)) time we output a rank-k matrix B, in factored form, for which ||A - B|| 2 F ≤ (1 + ε)||A - A k || F 2 , where Ak is the best rank-k approximation to A. When k and 1/ε are not too large compared to the sparsity of A, our algorithm does not need to read all entries of the matrix. Hence, we significantly improve upon previous nnz(A) time algorithms based on oblivious subspace embeddings, and bypass an nnz(A) time lower bound for general matrices (where nnz(A) denotes the number of non-zero entries in the matrix). We prove time lower bounds for low-rank approximation of PSD matrices, showing that our algorithm is close to optimal. Finally, we extend our techniques to give sublinear time algorithms for lowrank approximation of A in the (often stronger) spectral norm metric ||A - B|| 2 2 and for ridge regression on PSD matrices.