Non-asymptotic multiplicative bounds for LASSO and matrix compressed sensing

Develop rigorous non-asymptotic multiplicative bounds for the excess risk in LASSO regression and nuclear-norm regularized matrix compressed sensing, demonstrating that performance provably tracks the AMP/state-evolution predictions to within constant factors across finite-sample regimes beyond proportional asymptotics.

Background

While non-asymptotic multiplicative bounds have recently been established for kernel ridge regression, analogous results for LASSO and matrix compressed sensing are absent. The paper’s SE-based predictions match simulations well beyond proven regimes, highlighting a gap between heuristic accuracy and formal guarantees.

Closing this gap would provide the missing theoretical foundation for the phase diagrams and scaling laws reported, and unify non-asymptotic theory across sparse vector and low-rank matrix estimation.

References

Nonetheless, establishing non-asymptotic multiplicative bounds for LASSO and matrix compressed sensing remains a challenging open problem. Our results provide both motivation and supporting evidence for this direction, which we leave for future work.

— Scaling Laws and Spectra of Shallow Neural Networks in the Feature Learning Regime  (2509.24882 - Defilippis et al., 29 Sep 2025) in Section Non-asymptotic state evolution