Bayesian selection consistency for the mixture meta-prior

Prove Bayesian selection consistency with a margin for the mixture meta-prior combining sparse neural-network, additive, interaction, tree, Gaussian-process, and linear function families.

Background

The frequentist consistency result for the amortized median probability model assumes Bayesian selection consistency of the exact posterior, including a uniform margin separating posterior inclusion probabilities from one-half. The paper notes that such results are known for several individual priors but have not been established for the full mixture meta-prior used by SelectPFN.

A possible route is to treat the family and structure as a model index and apply nonparametric posterior-consistency techniques. The authors state that this becomes difficult when multiple nonparametric families approximate the truth at comparable rates, especially because suitable small-ball upper bounds for neural-network priors are unavailable.

References

\Cref{ass:D1 for the mixture meta-prior} is also open. A route through \citet{ghosal2008nonparametric}, with the model index taken as (family, structure), appears feasible for linear and Gaussian-process families. With several nonparametric families approximating $f_0$ at similar rates it is harder, because small-ball upper bounds are not known for neural-network priors.

— SelectPFN: Amortized Bayesian Variable Selection for Sparse Nonlinear Regression with Prior-Data Fitted Networks  (2610.21950 - Luo, 15 Oct 2026) in Appendix, Section “Interpretation and open problems,” immediately after the uniform-amortization conjecture