Dice Question Streamline Icon: https://streamlinehq.com

Computational gaps for other rotationally invariant priors in generalized BSR

Determine whether computational-to-statistical gaps exist for generalized bilinear sequence regression with rotationally invariant priors beyond the factorised Gaussian case, and, if they do, characterize the parameter regimes in which GAMP-RIE fails to attain the Bayes-optimal performance.

Information Square Streamline Icon: https://streamlinehq.com

Background

The authors derive a general GAMP-RIE framework applicable to rotationally invariant priors and observe no hard phase for the specific factorised Gaussian prior with Gaussian label noise. However, they point out that other rotationally invariant priors may lead to computational-to-statistical gaps, which would manifest as multiple fixed points of state evolution where GAMP-RIE converges to a suboptimal solution.

Understanding if and when such gaps appear across priors is essential to chart the algorithmic landscape of generalized BSR and to assess when message-passing methods can provably achieve Bayes-optimal estimation.

References

Other rotationally-invariant priors may still however exhibit computational gaps, but we leave such a study for future work.

Bilinear Sequence Regression: A Model for Learning from Long Sequences of High-dimensional Tokens (2410.18858 - Erba et al., 24 Oct 2024) in Section 2.4 (Message-passing algorithm)