Dice Question Streamline Icon: https://streamlinehq.com

Threshold and order of replica symmetry breaking under model mismatch

Determine the level of model mismatch between the assumed prior q(x0) and likelihood q(y|z0) and the true distributions p(x0) and p(y|z0) in generalized linear models that compels the free-energy extremum conditions of the postulated posterior estimator to exhibit a replica symmetry breaking structure, and ascertain the necessary order K of replica symmetry breaking required.

Information Square Streamline Icon: https://streamlinehq.com

Background

The paper studies high-dimensional estimation in generalized linear models under model mismatch, where the estimator uses assumed prior and likelihood distributions that may differ from the true generative model. The postulated posterior estimator (PPE) can be analyzed via free energy, and in mismatched settings, the underlying energy landscape may require replica symmetry breaking (RSB).

Existing approximate message passing variants such as VAMP (RS), GASP (1RSB with i.i.d. matrices), and VASP (1RSB with vector surveys) address parts of this landscape, but a fundamental unresolved issue is when mismatch necessitates RSB and what RSB order suffices. The authors introduce KVASP (K-step RSB) and show alignment of its state evolution fixed points with KRSB free-energy saddle points, yet the threshold and required order of RSB remain explicitly unclear.

References

However, many fundamental questions regarding model mismatch remain unanswered. For instance, it is still unclear what level of model mismatch is required for the postulated posterior estimate (PPE) to exhibit a replica symmetry breaking (RSB) structure in the extremum conditions of its free energy, and what order of RSB is necessary.

K-step Vector Approximate Survey Propagation (2410.20902 - Chen et al., 28 Oct 2024) in Abstract