Sourced Mode (SMD) Estimation

• Sourced Mode (SMD) is a frequentist likelihood‐free estimator that minimizes the weighted discrepancy between observed and simulated auxiliary statistics.
• It leverages simulation averaging to reduce higher-order bias and variance inflation, offering a bootstrap-like improvement over traditional methods.
• It differs from ideal ABC by optimizing average deviations rather than averaging simulation‐specific modes, highlighting the non-commutativity between optimization and averaging.

“Simulation estimation with auxiliary statistics” in this context refers to a class of likelihood-free estimators for structural models whose likelihood is unavailable or too costly to evaluate directly. The paper “The ABC of Simulation Estimation with Auxiliary Statistics” develops a unified account of the frequentist simulated minimum distance (SMD) estimator, Bayesian Approximate Bayesian Computation (ABC), and the paper’s Reverse Sampler (RS), using auxiliary statistics as the common object linking data and model simulation. Its central claim is that SMD and ideal ABC are closely related but not identical: SMD is the mode of the average deviations, whereas ideal ABC, in the RS representation, is a weighted average of simulation-specific modes. The paper further argues that this distinction is asymptotically second-order important because optimization and averaging do not commute, and because posterior means inherit prior-induced and mean-versus-mode terms absent from frequentist minimum-distance estimators (Forneron et al., 2015).

1. Definition of the SMD framework

In the paper, simulated minimum distance (SMD) is the frequentist likelihood-free estimator built from auxiliary statistics when the structural likelihood is unavailable or too costly to evaluate directly. Let $\hat\psi \equiv \hat\psi(\mathbf y(\theta_0))$ denote an $L$-vector of auxiliary statistics computed from the observed data, with binding function

$$\psi(\theta)=\lim_{T\to\infty}\mathbb E[\hat\psi(\theta)],$$

assumed continuously differentiable and locally injective at $\theta_0$. Identification is therefore expressed through the local injectivity of the binding relation between structural parameters and auxiliary statistics.

When $\psi(\theta)$ is analytically intractable, the paper replaces it by simulation from the structural model. For $s=1,\dots,S$, one simulates

$$\mathbf y^s(\theta)\equiv \mathbf y^s(\varepsilon^s,\theta), \qquad \varepsilon^s\sim F_\varepsilon,$$

computes simulated auxiliary statistics

$$\hat\psi^s(\theta)\equiv \hat\psi^s(\mathbf y^s(\theta)),$$

and forms the simulated moment discrepancy

$$\bar g_S(\theta)=\hat\psi-\frac{1}{S}\sum_{s=1}^S \hat\psi^s(\theta).$$

SMD is then defined by the quadratic criterion

$$\hat\theta_{SMD} = \arg\min_\theta J_S(\theta), \qquad J_S(\theta)=\bar g_S(\theta)'W\bar g_S(\theta),$$

where $L$0 is a positive definite weighting matrix. In exactly identified models, the discrepancy can typically be driven numerically to zero up to machine precision. Operationally, SMD chooses the parameter whose simulated auxiliary statistics, averaged over $L$1 simulation draws, are closest to the observed auxiliary statistics in weighted quadratic distance.

The paper’s concise interpretation is

$L$2

so SMD first averages the simulation-specific discrepancies across $L$3, and only then optimizes over $L$4. This is the basis for the description of SMD as the mode of the average deviations.

2. Reverse Sampler and the optimization-based representation of ABC

The paper’s bridge to ABC begins from MCMC-ABC with tolerance $L$5, where accepted draws satisfy

$L$6

In the ideal but infeasible limit $L$7, acceptance requires

$L$8

For fixed simulation shock $L$9, each accepted draw $$\psi(\theta)=\lim_{T\to\infty}\mathbb E[\hat\psi(\theta)],$$0 is therefore the solution to a simulation-specific minimization problem,

$$\psi(\theta)=\lim_{T\to\infty}\mathbb E[\hat\psi(\theta)],$$1

or, in the paper’s reverse sampler notation,

$$\psi(\theta)=\lim_{T\to\infty}\mathbb E[\hat\psi(\theta)],$$2

Each $$\psi(\theta)=\lim_{T\to\infty}\mathbb E[\hat\psi(\theta)],$$3 is therefore an $$\psi(\theta)=\lim_{T\to\infty}\mathbb E[\hat\psi(\theta)],$$4 SMD mode. The Reverse Sampler (RS) converts these simulation-specific solutions into posterior objects by reweighting them with a Jacobian/prior correction: $$\psi(\theta)=\lim_{T\to\infty}\mathbb E[\hat\psi(\theta)],$$5 and then forming

$$\psi(\theta)=\lim_{T\to\infty}\mathbb E[\hat\psi(\theta)],$$6

Under one-to-one mapping and full-rank Jacobian conditions, the RS posterior converges to the ideal posterior $$\psi(\theta)=\lim_{T\to\infty}\mathbb E[\hat\psi(\theta)],$$7. More generally, for any measurable $$\psi(\theta)=\lim_{T\to\infty}\mathbb E[\hat\psi(\theta)],$$8,

$$\psi(\theta)=\lim_{T\to\infty}\mathbb E[\hat\psi(\theta)],$$9

The paper also writes the same logic in generic importance-sampling notation: $\theta_0$0 with ideal importance weights

$\theta_0$1

The paper’s interpretation is therefore exact rather than merely heuristic: an ideal ABC estimate can be viewed as a weighted average of a sequence of SMD modes, each mode solving its own simulation-specific distance minimization problem (Forneron et al., 2015).

3. Non-commutativity of averaging and optimization

The paper repeatedly summarizes the difference between the two procedures as follows:

• SMD: optimize the criterion built from the average simulated discrepancy, i.e. the mode of average deviations.
• Ideal ABC / RS posterior mean: compute a weighted average of simulation-specific modes, i.e. a weighted average of SMD modes.

This distinction is formalized by comparing

$\theta_0$2

with the RS construction that solves many problems of the form $\theta_0$3 and then averages the resulting $\theta_0$4’s using importance weights. Unless the model is very special, the minimizer of the average criterion is not equal to the average of the minimizers.

That statement is the paper’s central conceptual bridge. It connects frequentist minimum-distance logic and Bayesian likelihood-free computation without collapsing them into a single estimator. The first-order relationship is strong, but the order of operations differs:

• averaging first, then optimizing;
• optimizing first, then averaging.

A plausible implication is that many informal comparisons between ABC and SMD can be misleading if they treat the two procedures as interchangeable merely because both use auxiliary statistics and simulation. The paper’s formulation instead makes the distinction structural: the estimators are related through the same simulated discrepancies, but they aggregate those discrepancies in different orders (Forneron et al., 2015).

4. Stochastic expansions, variance, and higher-order bias

A major contribution of the paper is a stochastic expansion for SMD. If the observed auxiliary statistics satisfy

$\theta_0$5

and each simulated statistic has analogous expansion

$\theta_0$6

then

$\theta_0$7

with

$\theta_0$8

and

$\theta_0$9

The first-order term shows simulation noise of order $\psi(\theta)$0. In large samples, the paper states the variance inflation relative to MD/GMM as

$\psi(\theta)$1

Hence SMD is consistent for fixed $\psi(\theta)$2, but less efficient than MD/GMM unless $\psi(\theta)$3.

The higher-order bias comparison is one of the paper’s most distinctive results. For MD,

$\psi(\theta)$4

whereas for SMD,

$\psi(\theta)$5

The $\psi(\theta)$6 term therefore disappears under simulation averaging. In the special case $\psi(\theta)$7, the curvature term also vanishes, and SMD has no $\psi(\theta)$8 bias as $\psi(\theta)$9. This is why the paper describes SMD as having a bootstrap-like bias reduction effect.

5. RS, ideal ABC, and the second-order distinction from frequentist estimators

The paper derives an analogous expansion for the RS/ideal ABC estimator: $s=1,\dots,S$0 with

$s=1,\dots,S$1

and

$s=1,\dots,S$2

Under exact identification and a smooth positive prior, the proposition stated in the paper is

$s=1,\dots,S$3

for arbitrary prior. RS/ideal ABC is therefore first-order equivalent to SMD, but generally has a non-negligible $s=1,\dots,S$4 bias. The same logic is extended to Laplace-type estimators (LT, SLT): posterior means inherit extra bias from the prior and from replacing the mode by the mean.

The paper summarizes a broad class of estimators by

$s=1,\dots,S$5

where $s=1,\dots,S$6 and $s=1,\dots,S$7.

This formulation encapsulates the paper’s general message: Bayesian-computation-based estimators differ from frequentist minimum-distance estimators by prior-induced and mean-vs-mode terms, even when they are first-order equivalent. A common misconception would be to interpret the ABC–SMD connection as implying exact inferential equivalence. The paper’s second-order analysis rejects that interpretation and locates the difference precisely in posterior averaging and prior weighting (Forneron et al., 2015).

6. Assumptions, computation, and the dynamic panel illustration

The assumptions behind the framework are standard but central. The paper requires covariance-stationary data generated by a parametric model $s=1,\dots,S$8; auxiliary statistics satisfying

$s=1,\dots,S$9

a continuously differentiable binding function $$\mathbf y^s(\theta)\equiv \mathbf y^s(\varepsilon^s,\theta), \qquad \varepsilon^s\sim F_\varepsilon,$$0; and local injectivity of $$\mathbf y^s(\theta)\equiv \mathbf y^s(\varepsilon^s,\theta), \qquad \varepsilon^s\sim F_\varepsilon,$$1 at $$\mathbf y^s(\theta)\equiv \mathbf y^s(\varepsilon^s,\theta), \qquad \varepsilon^s\sim F_\varepsilon,$$2. For the RS/ABC links, the simulation-specific maps $$\mathbf y^s(\theta)\equiv \mathbf y^s(\varepsilon^s,\theta), \qquad \varepsilon^s\sim F_\varepsilon,$$3 are required to be continuously differentiable, one-to-one, and to have full-column-rank Jacobian $$\mathbf y^s(\theta)\equiv \mathbf y^s(\varepsilon^s,\theta), \qquad \varepsilon^s\sim F_\varepsilon,$$4. Exact identification simplifies the analysis because the discrepancy can be reduced to zero, whereas overidentification introduces extra weighting and additional higher-order bias terms.

The paper also emphasizes a computational contrast. SMD solves one optimization using $$\mathbf y^s(\theta)\equiv \mathbf y^s(\varepsilon^s,\theta), \qquad \varepsilon^s\sim F_\varepsilon,$$5 fixed simulation streams. ABC samples many candidate $$\mathbf y^s(\theta)\equiv \mathbf y^s(\varepsilon^s,\theta), \qquad \varepsilon^s\sim F_\varepsilon,$$6’s and repeatedly simulates new innovations, suffering from low acceptance when $$\mathbf y^s(\theta)\equiv \mathbf y^s(\varepsilon^s,\theta), \qquad \varepsilon^s\sim F_\varepsilon,$$7 is small. RS avoids rejection by always solving an optimization and then reweighting; this can be attractive when optimization is relatively easy. The practical conclusion is that $$\mathbf y^s(\theta)\equiv \mathbf y^s(\varepsilon^s,\theta), \qquad \varepsilon^s\sim F_\varepsilon,$$8 is a key tuning parameter for ABC, whereas in SMD/RS optimization can make the discrepancy essentially zero up to numerical tolerance.

These points are illustrated with a simulation study of the dynamic panel model. In the short-$$\mathbf y^s(\theta)\equiv \mathbf y^s(\varepsilon^s,\theta), \qquad \varepsilon^s\sim F_\varepsilon,$$9 fixed-effects panel, the MD/LSDV estimator is severely biased. SMD and RS both sharply reduce the bias for $$\hat\psi^s(\theta)\equiv \hat\psi^s(\mathbf y^s(\theta)),$$0, $$\hat\psi^s(\theta)\equiv \hat\psi^s(\mathbf y^s(\theta)),$$1, and $$\hat\psi^s(\theta)\equiv \hat\psi^s(\mathbf y^s(\theta)),$$2, with similar performance. LT behaves much like MD because it uses Bayesian computation to approximate the same extremum problem. MCMC-ABC improves on MD but remains more biased than SMD/RS unless $$\hat\psi^s(\theta)\equiv \hat\psi^s(\mathbf y^s(\theta)),$$3 is made very small. In the reported experiment, $$\hat\psi^s(\theta)\equiv \hat\psi^s(\mathbf y^s(\theta)),$$4 gives a decent acceptance rate but visibly more bias than SMD/RS; reducing $$\hat\psi^s(\theta)\equiv \hat\psi^s(\mathbf y^s(\theta)),$$5 improves bias at the cost of much lower acceptance.

This suggests that the paper’s contribution is not merely classificatory. It gives a comparative theory of likelihood-free estimation in which simulation-based minimum distance, reverse-sampling-based posterior approximation, and Laplace-type procedures can be placed on a common asymptotic scale. Within that comparison, SMD emerges as a frequentist estimator with a distinctive bias-reduction mechanism, while ideal ABC/RS emerges as a posterior mean over simulation-specific modes rather than a single extremum of an averaged discrepancy (Forneron et al., 2015).