Dice Question Streamline Icon: https://streamlinehq.com

Absence of a hard phase in BSR with Gaussian label noise

Prove that no computational-to-statistical gap (hard phase) occurs in Bayes-optimal inference for the bilinear sequence regression (BSR) model with factorised Gaussian prior and Gaussian label noise; equivalently, show that the GAMP-RIE algorithm converges from zero-overlap initialization to the Bayes-optimal fixed point across the full parameter regime without encountering suboptimal fixed points.

Information Square Streamline Icon: https://streamlinehq.com

Background

The paper develops a GAMP-RIE algorithm and state evolution for the BSR model and shows that for the specific case of factorised Gaussian priors with Gaussian label noise, numerical experiments and free-entropy analysis do not indicate a computational-to-statistical gap. In such gaps, AMP-type algorithms fail to reach the Bayes-optimal estimator despite information-theoretic recoverability.

Establishing the absence of a hard phase would provide a rigorous guarantee that GAMP-RIE achieves the Bayes-optimal mean-squared error for the BSR model with Gaussian label noise, resolving a central algorithmic optimality question raised by the authors’ empirical findings.

References

While not an analytical justification, our observations do not provide any hint to the existence of such a computational-to-statistical gap, allowing us to conjecture that no hard phase is present for this specific choice of prior.

Bilinear Sequence Regression: A Model for Learning from Long Sequences of High-dimensional Tokens (2410.18858 - Erba et al., 24 Oct 2024) in Section 2.4 (Message-passing algorithm)