Composite Goodness-of-Fit Test

• Composite goodness-of-fit tests are statistical methods that determine whether observed data belong to any member of a specified parametric distribution family, addressing model misspecification.
• They leverage modern techniques such as kernel-based discrepancies (MMD, KSD), empirical characteristic functions, and optimal transport to estimate parameters and evaluate model fit.
• Practical implementations utilize bootstrap methods and invariance principles to accurately estimate critical values while controlling for type I error in diverse applications.

A composite goodness-of-fit (GoF) test is a statistical methodology for assessing whether observed data arise from any member of a specified parametric family of probability distributions, rather than from a fully specified (simple) model. This contrasts with simple GoF tests, which target a single, fully defined null distribution. Composite GoF testing is fundamental for model validation in contemporary statistical, econometric, and machine learning paradigms, where model misspecification or over-fitting is a primary concern. Recent research has advanced the development of robust, general-purpose, and distribution-free methods for composite GoF, leveraging kernel methods, empirical characteristic functions, optimal transport, and probabilistic bootstrap schemes.

1. Formal Definition and Problem Structure

Let $Q$ denote the empirical distribution of the data, and consider a parametric family $\{P_\theta\}_{\theta\in\Theta}$. A composite GoF test seeks to test:

$$\begin{cases} H_0: \exists\, \theta_0 \in \Theta \text{ such that } Q = P_{\theta_0} \ H_1: Q \notin \{P_\theta\}_{\theta\in\Theta} \end{cases}$$

The null hypothesis is composite, as the parameter %%%%2%%%% is unspecified. The tests must maintain validity (correct type I error) uniformly over all admissible parameter values, with power against all fixed alternatives.

2. Classical and Modern Methodologies

2.1 Maximum Mean Discrepancy and Kernel Stein Discrepancy

Composite GoF tests using kernel-based minimum distance estimators were established as general-purpose tools. The workflow (Key et al., 2021) is as follows:

• Define a kernel-based discrepancy $\mathcal{D}$ (e.g., MMD, KSD).
• Estimate the optimal parameter:

$$\hat{\theta}_n = \arg\min_{\theta \in \Theta} \mathcal{D}(P_\theta, Q_n)$$

• Evaluate the test statistic:

$\Delta = n\, \mathcal{D}(P_{\hat{\theta}_n}, Q_n)$

• Appropriately estimate the critical value $c_\alpha$, and reject $H_0$ for $\Delta > c_\alpha$.

Maximum Mean Discrepancy (MMD)

$$\mathrm{MMD}^2(P, Q) = \mathbb{E}[K(X, X')] - 2 \mathbb{E}[K(X, Y)] + \mathbb{E}[K(Y, Y')]$$

Suitable for generative/simulator-based models; only sampling is required from $P_{\hat\theta_n}$.

Kernel Stein Discrepancy (KSD)

$$\mathrm{KSD}^2(P,Q) = \mathbb{E}_{X,X'\sim Q}[h_\mathrm{KSD}(X,X')]$$

Requires a tractable and differentiable model density (possibly unnormalized) and its gradient.

2.2 Empirical Characteristic Function–Based and Other Functional Tests

Tests based on the empirical characteristic function (ECF), Laplace transforms, or comparison curve projections provide flexible, omnibus composite GoF tests. These handle high-dimensional or complicated families, particularly when the probability integral transform or score is not available, or for heavy-tailed and skewed data (Karling et al., 2023, Meintanis et al., 2022, Ducharme et al., 2022, Ebner et al., 2021).

• Weighted $L^2$ distances (using ECF or Laplace transforms) offer robust test statistics.
• Comparison curve methods enable local identification of GoF failures and adaptive test feature selection.

2.3 Wasserstein Distance and Optimal Transport

Multivariate settings can exploit the empirical Wasserstein distance as a GoF measure (Hallin et al., 2020). For group families, invariance-based data reduction yields parameter-free null distributions; otherwise, critical values are inferred via parametric bootstrap.

$$T_{\mathcal{M},n} := W_p^p(\hat{\mathbb{P}_n}, \mathbb{P}_{\hat{\theta}_n})$$

3. Theory: Asymptotics, Consistency, and Bootstrap Calibration

3.1 Null and Alternative Distributions

• Under regularity, composite kernel-based tests (MMD, KSD) yield test statistics with limiting distributions as weighted sums of (possibly infinite) independent $\chi^2$ variables (with parameter estimation introducing additional linear and quadratic disturbances) (Brueck et al., 26 Oct 2025, Key et al., 2021).
• Consistency is established: under the alternative, the statistic diverges ($n\,\mathrm{D}(P_{\hat\theta_n}, Q_n)\to\infty$).
• Some procedures (e.g., Laplace transform-based tests for Lévy) are scale-free under the null, greatly simplifying composite inference (Lukić et al., 2023).

3.2 Bootstrap and Critical Value Estimation

• Parametric bootstrap: Resample data from the fitted model, re-estimate parameters, and recalculate the statistic. This approach correctly accounts for parameter variability, giving valid, high-power composite GoF tests for broad settings (including kernel and ECF-based statistics).
• Wild bootstrap: Offers computational gains but can be conservative or invalid when the U-statistic incorporates parameter estimation (notably, the extra terms in the limiting law under KSD are missed by the wild bootstrap).
• Invariant families: For group models, data transformation yields parameter-free null laws, enabling exact critical values by Monte Carlo.

4. Practical Implementation and Applications

Category	Applicable Methods	Strengths / Limitations
Simulator models	MMD-based kernel tests	Only sampling needed; high-dimensions feasible
Unnormalized models	KSD, linear-time KSD, FSSD	No need for normalization constant; scalable; interpretable
Heavy tails/skewness	ECF & Laplace-based tests	No closed form densities needed; robust to tail aberrations
Multivariate data	Wasserstein-based tests	Natural for location-scale and affine models; optimal transport computation intense in high $d$
Sparse categorical	Modified chi-squared/Kullback tests	Corrects for excess type I under sparse contingency; generalizes ad hoc fixes (Ku)

Notable applications:

• Biophysical simulations: evaluating unnormalised generative models via the MMD (Key et al., 2021).
• Model selection for binary paired organ outcomes: copula-based composite GoF; robust model/GOF coupling (Zhou et al., 27 Jun 2025).
• Markov random fields: conclique-based spatial residuals for composite GoF in spatial models (Kaiser et al., 2012).
• Large sparse categorical tables: corrected statistics stabilize type I error (corrected $Q$/$G$) (Finkler, 2010).

5. Limitations, Nuances, and Current Challenges

• Kernel selection: Imperfect or poorly tuned kernels reduce power or violate nominal error rates; model-dependent tuning is often essential (Key et al., 2021).
• Estimator reliability: Minimum distance estimators used in composite GoF can behave erratically in complex, high-dimensional parameter spaces.
• Bootstrap validity: For Wasserstein-based and high-dimensional GoF statistics, parametric bootstrap is empirically effective, but theoretical guarantees remain conjectural in absence of asymptotic distribution theory (Hallin et al., 2020).
• Dimension curse: All nonparametric methods degrade in power with high dimensionality; exploiting sufficient statistics/invariance can alleviate but not eliminate this.
• Computational burden: Nested or warp-speed bootstraps, especially for L2-functional/ECF tests, are computationally taxing for large $n$, but parallelization can mitigate overhead (Karling et al., 2023).

6. Recent Advances and Future Directions

• Degenerate U-statistics with parameter estimation: Rigorous asymptotic and bootstrap theory for U-statistics in composite GoF (e.g., KSD) now enables valid inference for previously intractable modern tests (Brueck et al., 26 Oct 2025).
• Learning optimal test features: Adaptive selection of test locations (FSSD/linear-time KSD) boosts efficiency and interpretability (Jitkrittum et al., 2017).
• Omnibus local power: Tests based on comparison curves (bar-plots) provide simultaneous local and global diagnostics, supporting iterative model refinement (Ducharme et al., 2022).
• Approximate co-sufficient sampling (aCSS): Exchangeable resampling verified to yield valid, general-purpose GoF tests for virtually any model with an efficient estimator, unattainable directly with the parametric bootstrap (Barber et al., 2020).
• Characterization and scale/family-specific tests: For distributions such as Lévy, tailored functional characterizations (Laplace/empirical transform) yield composite GoF tests with superior Bahadur efficiency and power (Lukić et al., 2023).

7. Mathematical Formulas and Algorithmic Steps Summary

Framework	Test Statistic	Parameter Estimation	Critical Value/Threshold
MMD-based kernel	$$\Delta = n\,\operatorname{MMD}^2(P_{\hat\theta_n}, Q_n)$$	$\arg\min_\theta$	Parametric bootstrap
KSD-based kernel	$$\Delta = n\,\operatorname{KSD}^2(P_{\hat\theta_n}, Q_n)$$	$\arg\min_\theta$	Corrected bootstrap
Wasserstein	$$W_p^p(\hat{\mathbb{P}_n}, \mathbb{P}_{\hat\theta_n})$$	e.g., MLE	Monte Carlo/bootstrap
Laplace/ECF-based	See e.g., $T_{n,m}^{(\Psi)}$ in (Karling et al., 2023)	Canonical/plug-in	Nested/warp-speed bootstrap

8. Conclusion

Composite goodness-of-fit tests are central to principled model assessment in modern statistical practice. Recent advances enable general-purpose, cross-model, and cross-domain applicability, emphasizing kernel-based, empirical characteristic function, and optimal transport techniques, with careful handling of nuisance parameters via parametric bootstrapping or invariance. Rigorous asymptotic theory now underlies degenerate U-statistics with estimated parameters, closing previous gaps in kernel-based GoF. Persistent computational, theoretical, and high-dimensional challenges remain, but the current toolkit allows robust, interpretable, and efficient composite GoF testing across parametric modeling landscapes.