Repro Samples Framework

• Repro Samples Framework is a model-free, likelihood-free method that inverts the data-generating process via artificial noise to construct confidence sets for binary classification.
• It employs a sparsity-constrained working GLM to guide empirical risk minimization without requiring correct model specification or true sparsity.
• Simulation studies and genomic applications demonstrate that the approach yields small candidate model sets with near-nominal coverage and reliable inference.

The repro samples framework is a model-free, likelihood-free statistical inference methodology that constructs confidence sets for high-dimensional binary classification models by generating artificial samples that mimic the underlying data-generating mechanism. This approach leverages a sparsity-constrained working generalized linear model (GLM) to guide inference, but does not require correctness of the model specification or sparsity assumptions for the true data-generating process. Uncertainty is quantified for both the model support (set of influential covariates) and arbitrary linear combinations of the so-called oracle regression coefficients. The method is characterized by inverting the data-generating process via simulated noise vectors and empirical risk minimization, yielding finite-sample valid and computationally tractable procedures for high-dimensional binary models.

1. Model-Free Inferential Foundations

The core principle underlying the framework is the inversion of the data-generating mapping. Observed binary responses $Y$ are modeled as

$$Y = \mathbb{I}\big\{g^{-1}(X_{\tau_0}^\top \beta_{0, \tau_0}) + \epsilon > 0\big\},$$

where $g^{-1}$ is a working inverse link function (e.g., logistic), $\tau_0$ is the true but unknown support set (indices of relevant covariates), $\beta_{0, \tau_0}$ the corresponding coefficients, and $\epsilon$ represents unobserved latent noise generated from a known distribution (such as the logistic distribution, or via $\epsilon = -g(U),\ U \sim \mathrm{Unif}(0,1)$).

A central aspect of the approach is its model-free character: inference targets the optimal fit under a working GLM, but neither correct model specification nor sparsity of the true process is assumed. The population-level "oracle" parameters are those that, under a size constraint $s$, minimize the misclassification risk: $$\tau_0 \in \arg\min_{\tau \subset [p],\, |\tau| \leq s} \mathbb{E}\Big[\mathbb{I}\big\{Y \neq \mathbb{I}\{g^{-1}(X_\tau^\top \beta_\tau) > 1/2\}\big\}\Big].$$ This ensures interpretability of the most influential covariates and coverage even under severe misspecification.

2. Artificial Sample Generation and Model Inversion

Inference proceeds by generating "repro samples": artificial noise vectors $\epsilon^*$ simulating from the known distribution (e.g., sampling $\epsilon_i^* = -g(u_i^*)$ with $u_i^* \sim \mathrm{Unif}(0,1)$), and constructing synthetic responses

$$y_i^* = \mathbb{I}\big\{X_{i, \tau}^\top \beta_{\tau} + \sigma \epsilon_i^* > 0\big\}$$

for candidate supports $\tau$ and coefficient vectors $\beta_\tau$, including a scale parameter $\sigma$. The method then solves, for each artificial sample,

$$\widehat{\tau}(\epsilon^*) = \arg\min_{\tau \subset [p],\, |\tau| \leq s} \min_{\beta_\tau \in \mathbb{R}^{|\tau|},\ \sigma \geq 0} L_n^R(\tau,\beta_\tau,\sigma \mid X^{(obs)}, y^{(obs)}, \epsilon^*),$$

where the empirical risk $L_n^R$ is the $0$-$1$ loss: $$L_n^R(\tau,\beta_\tau,\sigma \mid X^{(obs)}, y^{(obs)}, \epsilon^*) = \frac{1}{n} \sum_{i=1}^n \mathbb{I}\big\{y_i^{(obs)} \neq \mathbb{I}\{X_{i,\tau}^{(obs)\top} \beta_\tau + \sigma \epsilon_i^* > 0\}\big\}.$$ Repeating this process over $d$ independent replicates yields a collection $\mathcal{C}$ of candidate models, each associated with a fitted noise vector.

3. Model Candidate Sets and Coverage Guarantees

A distinctive feature is the construction of a small model candidate set $\mathcal{C}$ by optimizing over all supports of size at most $s$ for each artificial noise vector. The framework provides theoretical guarantees: under a weak signal strength condition, the probability that the true influential support $\tau_0$ is absent from $\mathcal{C}$ is exponentially small in $d$ (number of repro samples), with a bound such as

$$P(\tau_0 \notin \mathcal{C}) \lesssim 2^{- \frac{1}{2} n c_{\min} + O(\log p)} + \left[1- \left|\mathcal{F}_{\log}(\epsilon) - \mathcal{F}_{\log}(-X_{\tau_0}^\top \beta_{0, \tau_0}) \right|^{n}\right]^d,$$

where $c_{\min}$ measures separation and $\mathcal{F}_{\log}$ is the logistic CDF. In high signal regimes, $\mathcal{C}$ typically contains only a handful of candidate models, often just $\tau_0$ itself.

4. Confidence Sets for Regression Coefficients and Case Probabilities

The framework supports construction of confidence sets for arbitrary linear combinations $A\beta_0$ of the oracle coefficients, as well as transformed targets such as case probabilities. For known support $\tau_0$, one computes the MLE $\widehat{\beta}_{\tau_0}$ and constructs (asymptotically valid) Wald-type confidence sets: $$\widetilde{T}(X^{(obs)}, y^{(obs)}, (\tau_0, t)) = n \left\|\widehat{V}(\tau_0)^{-1/2}(D(\tau_0)\widehat{\beta}_{\tau_0} - t)\right\|_2^2,$$ where $\widehat{V}(\tau_0)$ is the plug-in variance and $D(\tau_0)$ derives from a factorization $A_{\cdot, \tau_0} = C(\tau_0)D(\tau_0)$ as in the paper.

When $\tau_0$ is unknown, the confidence set is obtained by profiling over $\mathcal{C}$,

$$\Gamma^{A\beta_0}_\alpha(X^{(obs)}, y^{(obs)}) = \bigcup_{\tau \in \mathcal{C}} \left\{ t:\ \widetilde{T}(X^{(obs)}, y^{(obs)}, (\tau, t_0)) \leq F^{-1}_{\chi^2_r}(\alpha),\ t = C(\tau)t_0 \right\}.$$

This union reflects model selection uncertainty; resulting confidence regions may be disjoint or unions of intervals.

Case probabilities for new instances follow via non-linear transformation of these sets, $g^{-1}(A\beta_0)$.

5. Simulation Studies and Empirical Applications

Extensive simulations under both well-specified (sparse logistic) and misspecified (dense) models demonstrate that the candidate set $\mathcal{C}$ is of small cardinality and yields near-nominal coverage for both support and coefficient inference. Compared to debiasing-based procedures—which typically require correct model specification and strong eigenvalue or $\beta$-min conditions—the repro samples method achieves smaller or comparable confidence set sizes and more reliable nominal coverage.

In a case paper using single-cell RNA-seq data for immune response, the repro samples method recovered known immune-relevant genes (e.g., RSAD2, IFIT1, IFT80, ACTB, HMGN2, IFI47) and identified the gene AK217941 as a significant candidate, indicating potential for discovery in high-dimensional biological contexts.

6. Theoretical and Practical Implications

The method provides finite-sample probability bounds for inclusion of the true model in $\mathcal{C}$, demonstrating rigor without reliance on asymptotic arguments or strong parametric model assumptions. It is robust with respect to model misspecification, accommodates high-dimensionality, and produces interpretably small candidate sets even in the absence of sparsity or precise link function knowledge.

On the practical side, the framework leverages surrogate losses (hinge or logistic) in place of 0–1 loss for computational tractability, and avoids challenging requirements such as sample splitting or estimation of the inverse Fisher information.

7. Summary Table: Key Elements of the Repro Samples Framework

Component	Description	Theoretical Feature
Artificial noise generation ($\epsilon^*$)	Simulate known noise distribution (e.g., Logistic)	No assumption on true noise form
Model candidate set ($\mathcal{C}$)	Collect supports minimizing empirical risk with respect to each $\epsilon^*$	Exponential coverage guarantee
Inference target	Support, any $A\beta_0$, case probabilities	Model-free, no sparsity or link function required
Confidence set construction	Wald test, profiling over candidate models	Valid (finite/asymptotic) under weak signal
Real data application	Gene selection in single-cell RNA-seq datasets	Biological discovery via interpretable sets

In summary, the repro samples framework provides a robust, model-free approach to high-dimensional binary classification inference. Artificial repro samples and empirical inversion are used to construct finite-sample-valid confidence sets for both model support and regression coefficients, with superior flexibility and resilience to model misspecification compared to classical approaches. Simulations and applications in genomics validate its coverage and interpretability, with theoretical guarantees that are explicitly characterized in the presence of weak signal and high dimensionality (Hou et al., 1 Oct 2025).