Open-faced Sandwich Adjustment

• Open-faced Sandwich Adjustment is a post-hoc transformation that corrects the covariance of MCMC draws when using surrogate objective functions.
• It employs sandwich asymptotics to scale and rotate quasi-posterior samples, aligning them with the true asymptotic variance.
• The method is applicable in spatial statistics, robust modeling, and complex systems, offering valid frequentist confidence intervals despite non-likelihood data.

The open-faced sandwich (OFS) adjustment is a post-hoc transformation designed for Markov chain Monte Carlo (MCMC) methods when the traditional likelihood function is unavailable, intractable, or undesirable. In scenarios where only surrogate or quasi-likelihoods—objective functions maximized for asymptotically normal estimators—are accessible, standard Bayesian MCMC sampling yields posteriors with the incorrect curvature, resulting in poor frequentist uncertainty quantification. The OFS adjustment uses sandwich asymptotics to rotate and scale the raw MCMC draws such that their empirical distribution matches the correct asymptotic variance, enabling valid confidence sets even with non-likelihood objective functions (Shaby, 2012).

1. Motivation and Context

Frequently, applications in spatial statistics, robust modeling, or extreme value theory encounter intractable, unstable, or unknown likelihoods. Standard alternatives, such as composite or tapered likelihoods, or other estimating-equation criteria, produce consistent and asymptotically normal point estimators, denoted $\hat\theta_M$. Directly substituting these objectives into the MCMC acceptance ratio creates a "quasi-posterior," but this fails to yield draws with the correct spread.

Specifically, when using a non-likelihood surrogate $\ell_M(\theta; y)$ in place of the log-likelihood $\ell(\theta; y)$, the resulting quasi-posterior may concentrate on the correct value, but its covariance misrepresents the estimator's sampling variability. This leads to credible intervals with poor frequentist coverage, undermining Bayesian inferential procedures when the true likelihood is inaccessible.

2. Theoretical Foundation: Extremum Estimators and Sandwich Variance

Consider an objective function $\ell_{M,n}(\theta; y_n)$ formed from $n$ observations, with extremum estimator $\hat\theta_{M,n} = \arg\max \ell_{M,n}$. Under regularity,

$$\sqrt{n} (\hat\theta_{M,n} - \theta_0) \overset{d}{\to} N\left(0, J^{-1}\right),$$

where the sandwich information matrices are

• $$H(\theta) = -\mathbb{E}[\nabla^2_\theta \ell_M(\theta; Y)]$$,
• $$P(\theta) = \mathbb{V}\mathrm{ar}[\nabla_\theta \ell_M(\theta; Y)]$$,
• $J_n = H_n P_n^{-1} H_n$ so the sandwich variance $J_n^{-1} = H_n^{-1} P_n H_n^{-1}$.

For quasi-posterior outputs, the curvature is governed by $H_n^{-1}$, not $J_n^{-1}$. Naïve use of the quasi-posterior yields intervals with the wrong width unless this discrepancy is corrected.

3. Construction and Application of the OFS Adjustment

The OFS adjustment is formalized as a post-hoc linear transformation aligning the quasi-posterior draws’ covariance with the correct sandwich variance.

• Adjustment Matrix:

The goal is to find a matrix $\Omega_n$ such that if $Z \sim N(0, H_n^{-1})$, then $\Omega_n Z \sim N(0, J_n^{-1})$. This is satisfied by setting

$\Omega_n = H_n^{-1} P_n^{1/2} H_n^{1/2},$

where $P_n^{1/2}$ and $H_n^{1/2}$ denote matrix square roots. In practical applications, estimated versions $\hat H$, $\hat P$ are substituted to form $\hat\Omega$.

• Adjusting MCMC Draws:

1. Generate MCMC samples $\theta^{(1)}, \ldots, \theta^{(J)}$ from the quasi-posterior $\pi_M(\theta|y)$.
2. Compute the point estimate $\hat\theta_{QB}$ (e.g., mean or mode).
3. Apply the OFS transformation to each draw:

   $$\theta_{OFS}^{(j)} = \hat\theta_{QB} + \hat\Omega [\theta^{(j)} - \hat\theta_{QB}], \quad j=1, \ldots, J.$$

4. The empirical quantiles of $\{\theta_{OFS}^{(j)}\}$ provide asymptotically valid confidence intervals.

• Integration in Gibbs Samplers:

When $\ell_M$ is present in only some conditional blocks, block-specific adjustments $\Omega_{i| -i}$ can be computed and embedded within the corresponding Gibbs updates. A recommended two-pass approach runs the unadjusted sampler to estimate $\Omega$, then repeats with OFS correction, sufficing in most contexts.

4. Asymptotic Properties and Validity

The OFS adjustment is theoretically justified under regularity conditions ensuring the consistency and asymptotic normality of both the extremum estimator and the quasi-posterior [Chernozhukov & Hong 2003]. The result is that the OFS-adjusted draws share the same first-order asymptotic distribution as the extremum estimator $\hat\theta_M$, yielding quantiles with correct frequentist coverage $1-\alpha+o(1)$ for any fixed $\alpha$. Necessary conditions include the smoothness of $\ell_M$, identifiability, invertibility of $H$ and $P$, and increasing sample size.

5. Estimation of Information Matrices $H$ and $P$

Accurate computation of the OFS adjustment requires estimates of $H$ and $P$:

• Estimating $H$:
  • Option 1: Finite-difference approximation of the Hessian at $\hat\theta_{QB}$, using stored values of $\ell_M$.
  • Option 2: Use the sample covariance of the MCMC draws $\{\theta^{(j)}\}$ as an estimate for $H^{-1}$.
  • Option 3: Insert $\hat\theta_{QB}$ into an analytic formula for $H(\theta)$, if available.
• Estimating $P$:
  • For i.i.d. data $y_i$, the empirical variance of gradients is

  $$\hat P = \frac{1}{n} \sum_i \nabla \ell_M(\hat\theta_{QB}; y_i) \nabla \ell_M(\hat\theta_{QB}; y_i)'.$$ - If closed-form $P(\theta)$ is obtainable, plug in $\hat\theta_{QB}$. - Otherwise, use a parametric bootstrap: simulate $y^{(k)}$ from model$(\hat\theta_{QB})$, compute gradients, and aggregate to estimate variance.

• Computational Considerations:

Hessian and bootstrap calculations introduce overhead, but only need to be computed once per chain (or per block in a Gibbs sampler). Proper tuning (for example, avoiding instability in finite-differenced Hessians) and possible regularization are advised, especially when $H$ or $P$ are ill-conditioned.

6. Application Examples

OFS adjustment has demonstrated empirical efficacy in large-scale and complex models:

Example	Model/Objective Function	Result
Covariance tapering for Gaussian processes	Tapered log-likelihood (sparse Σ⊙T)	OFS achieves near-nominal coverage; naïve intervals severely miscover
Poisson spatio-temporal GLMM for eBird data	Tapered likelihood + Gibbs sampling	OFS corrects under/over-dispersion; credible bands match ecological signals

In spatial Gaussian process applications, direct evaluation of the likelihood incurs $O(n^3)$ computational cost; replacing this with a tapered log-likelihood allows for scalable estimation. Analytical expressions for $H$ and $P$ facilitate OFS computation, with adjusted intervals providing frequentist-valid inference. In spatio-temporal generalized linear mixed models for citizen science counting data (eBird), blockwise OFS correction implemented inside a Gibbs sampler propagates appropriate uncertainty, improving quantile coverage and inference quality (Shaby, 2012).

7. Implications and Summary

The open-faced sandwich adjustment provides a rigorous framework for extending MCMC-based Bayesian procedures to models where only non-likelihood objective functions are available. It enables practitioners to retain hierarchical modeling flexibility and uncertainty propagation, while ensuring that posterior intervals have desirable frequentist properties. The method’s success hinges on accurate estimation of the sandwich matrices, and its structured post-hoc or “block-wise” adjustments are broadly applicable, including to tapered Gaussian processes, composite likelihoods, and spatio-temporal GLMMs (Shaby, 2012). A plausible implication is that as computational and modeling challenges drive increased use of quasi-likelihoods, OFS-style corrections will become essential for valid uncertainty quantification in Bayesian-like workflows.