---
title: 'LOCO CRT: Conditional Randomization Test'
url: https://www.emergentmind.com/topics/loco-conditional-randomization-test-loco-crt
type: topic
---

# LOCO CRT: Conditional Randomization Test

The leave-one-covariate-out conditional randomization test (LOCO–CRT) is a computationally efficient methodology for assessing conditional independence within the model-X framework. It tests the null hypothesis that a response variable $Y$ is independent of a given covariate $X_j$ conditional on all other covariates $X_{-j}$, under the assumption that the marginal distribution of the covariate vector $X$ is known or can be accurately sampled. LOCO–CRT yields valid $p$-values useful for error-rate control in variable selection, minimizing algorithmic randomness and enabling practical application even in high-dimensional settings [2006.08482].

## 1. Statistical Formulation and Model-X Framework

The setup assumes i.i.d. samples $(X_i, Y_i)$, $i=1,\ldots,n$, with $X_i = (X_{i1}, \dots, X_{ip}) \in \mathbb{R}^p$ and arbitrary $Y_i$. The model-X assumption requires that the joint distribution $P_X$ of $X$ is fully known or directly sampleable, while $P_{Y|X}$ remains unrestricted. For each variable $j \in \{1,\dots,p\}$, the test targets
\[
H_{0,j}:~ Y \perp X_j \mid X_{-j},
\]
where $X_{-j}$ denotes all features except $X_j$. This hypothesis asserts that, conditional on $X_{-j}$, $X_j$ carries no information regarding $Y$.

## 2. LOCO Test Statistic

Given a loss function $\ell: \mathbb{R} \times \mathbb{R} \rightarrow \mathbb{R}$, e.g., squared loss for regression or logistic loss for classification, define:
- $\mathcal{F}_{\mathrm{full}}$: any fitted predictor trained on $(X, Y)$,
- $\mathcal{F}_{-j}$: the same model class fitted using $(X_{-j}, Y)$.

The observed LOCO statistic for coordinate $j$ is:
\[
T_j^{\mathrm{obs}} = \frac{1}{n} \sum_{i=1}^n [\ell(\mathcal{F}_{-j}(X_{i,-j}), Y_i) - \ell(\mathcal{F}_{\mathrm{full}}(X_i), Y_i)].
\]
$T_j^{\mathrm{obs}}$ quantifies the change in predictive loss incurred by omitting $X_j$. Under $H_{0,j}$, $T_j^{\mathrm{obs}}$ is expected to be small; under the alternative, it should be large.

## 3. Algorithmic Procedure and P-value Calculation

The LOCO–CRT algorithm uses null randomization to obtain a valid $p$-value for each variable. For each $j$:
1. Fit both $\mathcal{F}_{\mathrm{full}}$ (on $(X, Y)$) and $\mathcal{F}_{-j}$ (on $(X_{-j}, Y)$).
2. Compute $T_j^{\mathrm{obs}}$ as the average loss difference.
3. For $m=1,\dots, M$:
   - For each $i$, sample $X_{i,j}^*(m) \sim P_{X_j | X_{i,-j}}$.
   - Construct $X_i^{*}(m):=(X_{i,1},\dots, X_{i,j-1}, X_{i,j}^*(m), X_{i,j+1},\dots, X_{i,p})$.
   - Compute $T_j^*(m)$, the analogous test statistic using the null-resampled $X_{i, j}^*$.
4. Calculate
\[
p_j = \frac{1 + \sum_{m=1}^M \mathbf{1}\{T_j^*(m) \ge T_j^{\mathrm{obs}}\}}{M + 1}.
\]

Because $\{T_j^{\mathrm{obs}}, T_j^*(1), \dots, T_j^*(M)\}$ are exchangeable under $H_{0,j}$, $p_j$ is valid in finite samples. For simultaneous inference across coordinates, conventional multiplicity corrections (Bonferroni, Holm, Benjamini–Hochberg) can be applied.

## 4. Theoretical Guarantees

The principal theoretical result is finite-sample validity of LOCO–CRT $p$-values:
\[
\Pr\{p_j \le \alpha \} \le \alpha,~\forall~\alpha\in[0,1],
\]
given correct randomization from $P_{X_j|X_{-j}}$ and i.i.d. sampling. Under the null, $p_j$ is a super-uniform $p$-value. Familywise error rate (FWER) can be controlled at level $\alpha$ by rejecting all $j$ with $p_j \le \alpha/p$. The computational efficiency is achieved by fitting $\mathcal{F}_{\mathrm{full}}$ and $\mathcal{F}_{-j}$ only once per variable; null sampling changes only $X_j$, not the fitted models.

## 5. L1ME–CRT Variant for L1-regularized M-Estimators

For L1-regularized estimators (e.g., Lasso, elastic net), refitting after each variable exclusion is computationally intensive. The L1ME–CRT modification capitalizes on the empirical observation that, for coordinates $j$ with $\hat{\beta}_j = 0$ in the full-data fit, the cross-validated penalty parameter $\lambda$ is typically stable after exclusion, under restricted eigenvalue and Lipschitz loss conditions:
\[
\lambda_{\min\mathrm{CV}} \approx \lambda_{\min\mathrm{CV}}^{(-j)}.
\]
Hence, for the “inactive” set $\{j: \hat{\beta}_j=0\}$, the same $\lambda$ can be reused for $\mathcal{F}_{-j}$, obviating additional cross-validations. This reduces computational overhead to near the number of “active” variables, $o(p)$ in sparse regimes.

## 6. Closed-form Solution in the Multivariate Gaussian Covariate Case

Assuming $X \sim N_p(0, \Sigma)$, the conditional law
\[
X_j | X_{-j} \sim \mathcal{N}(\Sigma_{j,-j} \Sigma_{-j,-j}^{-1} X_{-j}, \sigma^2_{j|-j}),
\]
with $\sigma^2_{j|-j} = \Sigma_{jj} - \Sigma_{j,-j} \Sigma_{-j,-j}^{-1} \Sigma_{-j,j}$, enables analytic computation. With squared error loss and ordinary least squares,
\[
T_j^{\mathrm{obs}} = \frac{1}{n}(\Vert Y - \hat Y_{-j}\Vert_2^2 - \Vert Y - \hat Y_{\mathrm{full}}\Vert_2^2),
\]
where $\hat Y_{-j}$ and $\hat Y_{\mathrm{full}}$ are fitted values excluding and including $X_j$. Under $H_{0,j}$, $T_j^{\mathrm{obs}}$ follows a $\chi^2_1$ distribution, and the $p$-value is given by
\[
p_j = 1 - F_{\chi^2_1}\big(T_j^{\mathrm{obs}}\big),
\]
recovering the classical partial $F$-test or $t$-test for a single coefficient in normal linear regression, with no need for Monte Carlo.

## 7. Computational and Practical Considerations

Let $C_{\mathrm{fit}}$ denote the cost of fitting a model and $C_{\mathrm{pred}}$ the cost of scoring $n$ samples:
- A single LOCO–CRT test requires $2C_{\mathrm{fit}}+M\,C_{\mathrm{pred}}$.
- Testing all $p$ features costs $2p\,C_{\mathrm{fit}} + Mp\,C_{\mathrm{pred}}$, but with L1ME–CRT, the actual model refits may be far fewer than $2p$ due to reuse for inactive features.

Typically, $C_{\mathrm{fit}}$ is $O(np^2)$ for OLS or $O(\mathrm{CV} \times np)$ for Lasso; $C_{\mathrm{pred}}$ is $O(np)$. Monte Carlo sample sizes $M=200$–2000 are common, balancing $p$-value granularity and computational burden.

Implementation suggestions include precomputing random seeds for reproducibility, vectorizing null-feature sampling when feasible, and using persistent model objects in languages like R or Python to avoid redundant refitting. These practicalities further enhance runtime efficiency.

---

For foundational and related methodologies, see "Panning for gold: 'model-X' knockoffs for high-dimensional controlled variable selection” (Candès, Fan, Janson & Lv), “Gene hunting with hidden Markov model knockoffs” (Sesia, Candès & Sabatti), and “Multiple testing with the conditional randomization test” (Li & Barber) in addition to the primary development of LOCO–CRT [2006.08482].

Source: https://www.emergentmind.com/topics/loco-conditional-randomization-test-loco-crt