Blockwise coordinate descent schemes for sparse representation

• Main Authors: Bao-Di Liu, Yu-Xiong Wang, Bin Shen, Yu-Jin Zhang, Yan-Jiang Wang
• Format: Conference Proceeding
• Language: English
• Subjects: Convergence; coordinate descent; Dictionaries; Dictionary learning; Encoding; Linear programming; Minimization; Optimization; sparse coding; Sparse matrices
• ISSN: 1520-6149; 2379-190X
• DOI: 10.1109/ICASSP.2014.6854608

The current sparse representation framework is to decouple it as two subproblems, i.e., alternate sparse coding and dictionary learning using different optimizers, treating elements in bases and codes separately. In this paper, we treat elements both in bases and codes ho-mogenously. The original optimization is directly decoupled as several blockwise alternate subproblems rather than above two. Hence, sparse coding and bases learning optimizations are coupled together. And the variables involved in the optimization problems are partitioned into several suitable blocks with convexity preserved, making it possible to perform an exact block coordinate descent. For each separable subproblem, based on the convexity and monotonic property of the parabolic function, a closed-form solution is obtained. Thus the algorithm is simple, efficient and effective. Experimental results show that our algorithm significantly accelerates the learning process.