Principled Selection of Local Posterior Hyperparameters (β, γ)

Determine principled criteria and theoretical guidance for selecting the inverse temperature β and the localization strength γ in the localized Bayesian posterior p_γ(θ | D, θ*) used to define the local Bayesian influence function, so that covariance-based influence estimates are reliable and well-founded across deep neural networks with singular loss landscapes.

Background

The paper introduces the local Bayesian influence function (BIF), which estimates influence via covariances over a localized posterior distribution centered at a trained checkpoint. This localized posterior is parameterized by hyperparameters, notably the inverse temperature β and the localization strength γ, both of which shape the distribution and, consequently, the estimated covariances.

In discussing sources of error, the authors note that choices of β and γ are not yet theoretically grounded. Since these hyperparameters effectively define the geometry of the local posterior for singular models, establishing principled selection methods is essential for robust application of BIF and for minimizing bias in covariance-based influence estimation.