Conditions for provable uniform amortization

Determine conditions under which training a prior-data fitted network provably achieves uniform amortization at a fixed truth, sufficient to transfer Bayesian selection consistency to the amortized median probability model.

Background

The paper proves that frequentist selection consistency of the amortized median probability model follows if the network’s inclusion probabilities have the same threshold signs as the exact posterior with high probability under the fixed truth. This is formalized through the uniform-amortization assumption.

A separate negative result shows that small average training loss under the prior predictive distribution does not by itself imply this fixed-truth uniform approximation. The paper therefore leaves open the identification of architectural, optimization, or approximation conditions that guarantee uniform amortization.

References

On the theory side, the remaining open problems are spurious-inclusion control (assumption \Cref{ass:c6}) for nonparametric priors, an explicit second-order term in the detection threshold, and conditions under which training provably achieves uniform amortization.

— SelectPFN: Amortized Bayesian Variable Selection for Sparse Nonlinear Regression with Prior-Data Fitted Networks  (2610.21950 - Luo, 15 Oct 2026) in Section 4.5, Discussion, paragraph “Extensions”