Family of iterative LS-based dictionary learning algorithms, ILS-DLA, for sparse signal representation

• Published in: Digital signal processing 2007, Vol.17 (1), p.32-49
• Main Authors: Engan, Kjersti, Skretting, Karl, Husøy, John Håkon
• Format: Article
• Language: English
• Subjects: Compression; Dictionary design; Dictionary learning; Frame design; Least squares; Matching pursuit; Overcomplete dictionary; Signal representation; Sparse approximation
• ISSN: 1051-2004; 1095-4333
• DOI: 10.1016/j.dsp.2006.02.002

The use of overcomplete dictionaries, or frames, for sparse signal representation has been given considerable attention in recent years. The major challenges are good algorithms for sparse approximations, i.e., vector selection algorithms, and good methods for choosing or designing dictionaries/frames. This work is concerned with the latter. We present a family of iterative least squares based dictionary learning algorithms (ILS-DLA), including algorithms for design of signal dependent block based dictionaries and overlapping dictionaries, as generalizations of transforms and filter banks, respectively. In addition different constraints can be included in the ILS-DLA, thus we present different constrained design algorithms. Experiments show that ILS-DLA is capable of reconstructing (most of) the generating dictionary vectors from a sparsely generated data set, with and without noise. The dictionaries are shown to be useful in applications like signal representation and compression where experiments demonstrate that our ILS-DLA dictionaries substantially improve compression results compared to traditional signal expansions such as transforms and filter banks/wavelets.