Dictionary Learning Using Rank-One Projection (ROP)

Abstract: Dictionary learning aims to find a dictionary that can sparsely represent the training data. Methods in the literature typically formulate the dictionary learning problem as an optimisation with respect to two variables, i.e., dictionary and sparse coefficients, and solve it by alternating between two stages: sparse coding and dictionary update. The key contribution of this work is a Rank-One Projection (ROP) formulation where dictionary learning is cast as an optimisation with respect to a single variable which is a set of rank one matrices. The resulting algorithm is hence single staged. An alternating direction method of multipliers (ADMM) is derived to solve the optimisation problem and guarantees a global convergence despite non-convexity of the optimisation formulation. Also ROP reduces the number of tuning parameters required in other benchmark algorithms. Numerical tests demonstrate that ROP outperforms other benchmarks for both synthetic and real data especially when the sample number is small.