Online optimization in dynamic environments: a regret analysis for sparse problems

Abstract: Time-varying systems are a challenge in many scientific and engineering areas. Usually, estimation of time-varying parameters or signals must be performed online, which calls for the development of responsive online algorithms. In this paper, we consider this problem in the context of the sparse optimization; specifically, we consider the Elastic-net model, which promotes parsimonious solutions. Following the rationale in \cite{mok16}, we propose an online algorithm and we theoretically prove that it is successful in terms of dynamic regret. We then show an application to the problem of recursive identification of time-varying autoregressive models, in the case when the number of parameters to be estimated is unknown. Numerical results show the practical efficiency of the proposed method.