GPUSGD: A GPU-accelerated stochastic gradient descent algorithm for matrix factorization

• Published in: Concurrency and computation 2016-09, Vol.28 (14), p.3844-3865
• Main Authors: Jin, Jing, Lai, Siyan, Hu, Su, Lin, Jing, Lin, Xiaola
• Format: Article
• Language: English
• Subjects: Algorithms; Concurrency; Descent; Divergence; Factorization; GPU computing; over-writing problem; parallel matrix factorization; Recommender systems; Social networks; stochastic gradient descent; Stochasticity; thread divergence
• ISSN: 1532-0626; 1532-0634
• DOI: 10.1002/cpe.3722

Summary Matrix factorization is one of the leading techniques for many applications such as social network‐based recommendation systems. As of today, many parallel stochastic gradient descent (SGD) methods have been proposed to address the matrix factorization issue on shared‐memory (multi‐core) systems and distributed systems. However, these methods cannot be improved significantly on graphics processing unit (GPU) because the serious over‐writing problem and thread divergence may occur. The fundamental reason for such undesired results is that GPU is a parallel single instruction multiple data device, which only can greatly improve the applications with fine‐grained parallelism. In this paper, we propose an efficient GPU algorithm, named GPUSGD, to solve the matrix factorization problem based on SGD method. The major advantage of the proposed GPUSGD is that such method not only can handle the over‐writing problem but also can avoid the performance loss caused by the thread divergence. The experimental results show that GPUSGD performs much better in accelerating the matrix factorization compared with the existing state‐of‐the‐art parallel methods. To the best of our knowledge, this is the first work that develops a parallel SGD method to improve the matrix factorization on GPU. Copyright © 2015 John Wiley & Sons, Ltd.