Sample Splitting & Cross Fitting (SSCF)

• SSCF is a method that divides data to separately estimate nuisance functions and evaluate target parameters, ensuring out-of-sample validity.
• Cross fitting rotates the splits so that each observation contributes to both training and evaluation without biasing the results.
• SSCF underlies techniques like doubly robust estimation and orthogonal learning, improving reliability in causal and semiparametric inference.

Sample splitting and cross fitting (SSCF) denotes a family of estimation, inference, and validation strategies in which the data used to learn nuisance functions, tuning parameters, or preliminary models are separated from the data used to evaluate scores, losses, or target functionals. In semiparametric estimation this separation is used to control own-observation bias, nuisance-induced empirical-process terms, and nonlinearity bias; in contemporary formulations it is a key ingredient in double/debiased machine learning, doubly robust estimation, orthogonal learning, and related meta-learning workflows (Peyrot et al., 15 May 2026, Newey et al., 2018). The same logic appears in design-based randomized experiments, weakly dependent time series, correlated-unit causal inference, changepoint detection, post-selection inference, and community recovery, but the relevant split geometry, asymptotic argument, and efficiency tradeoff differ across settings (Lu et al., 21 Aug 2025, Lunde, 2019).

1. Core definitions and conceptual scope

In its simplest form, sample splitting divides the data into distinct parts and assigns different inferential roles to them. A common arrangement is to estimate nuisance objects on one part and estimate the target parameter on another. Cross-fitting strengthens this template by rotating the split so that every observation is used for both training and evaluation, but never on the same fold (Chen et al., 2022). In semiparametric work, the core invariant is that the nuisance predictions entering the target estimating equation are out-of-sample relative to the observations on which that equation is evaluated (Peyrot et al., 15 May 2026).

This basic distinction induces several secondary notions. Repeated sample splitting and repeated cross-fitting average estimators across multiple random partitions; in a general split-sample framework this produces four regimes: sample-splitting, cross-fitting, repeated sample-splitting, and repeated cross-fitting, indexed by the number of folds $K$ and the number of repetitions $M$ (Fava, 7 Nov 2025). In CATE estimation, the fold-specific estimators $\hat\tau_k(x)$ are commonly aggregated as

$$\tilde{\tau}_K(x)=\frac{1}{K}\sum_{k=1}^K \hat{\tau}_k(x),$$

and repeated cross-fit estimates may then be aggregated across repeated random partitions, including by the median (Jacob, 2020).

A recurring source of terminological confusion is the relation between cross-fitting and cross-validation. The correlated-units literature states explicitly that the usual purpose of cross-fitting in causal machine learning is to eliminate an empirical-process bias term, not to perform predictive-model assessment in the cross-validation sense (Balkus et al., 15 Jan 2026). This distinction matters because many SSCF arguments are tied to orthogonal scores, out-of-sample nuisance prediction, and asymptotic linearity rather than to predictive risk minimization.

2. Semiparametric and orthogonal foundations

A standard semiparametric starting point defines the target parameter $\theta_0 \in \mathbb{R}^p$ through a moment restriction

$$M(\theta,g)=\mathbb{E}[m(Z;\theta,g)], \qquad M(\theta_0,g_0)=0.$$

A common two-stage estimator first learns a nuisance $g_0$ by flexible machine learning and then solves the sample moment equation

$$M_n(\hat\theta,\hat g)=o_p(n^{-1/2}), \qquad M_n(\theta,g)=\frac1n\sum_{i=1}^n m(Z_i;\theta,g).$$

The linear-in-$\theta$ case,

$m(Z;\theta,g)=a(Z;g)\,\theta+\nu(Z;g),$

is especially prominent in partially linear regression, partially linear IV, ATE, and policy evaluation (Chen et al., 2022).

The asymptotic rationale for SSCF is that if the same sample is used to estimate $M$0 and to evaluate the second-stage score, terms such as $M$1 are no longer simple averages of i.i.d. variables because $M$2 depends on the sample. Cross-fitting restores conditional independence between the evaluation score and the nuisance learner. Under nuisance RMSE $M$3, Neyman orthogonality,

$M$4

second-order smoothness, and stochastic equicontinuity, the resulting estimator satisfies

$M$5

equivalently $M$6 (Chen et al., 2022).

Newey and Robins analyze the same logic through fast remainder rates for semiparametric functionals $M$7. Their cross-fit plug-in estimator and doubly cross-fit doubly robust estimator remove own-observation bias and, in the doubly cross-fit case, an additional nonlinearity bias that remains when the same sample is used to estimate both nuisance functions. The resulting remainder can attain the fastest known rates in the spline settings they study, and the expected conditional covariance example is semiparametrically efficient under minimal conditions (Newey et al., 2018).

The conditional-effect literature extends these ideas beyond low-dimensional averages. In three-way cross-fitting for conditional effects and other linear functionals, one partition is used to learn $M$8, one to learn the Riesz representer $M$9, and one to fit the pseudo-outcome regression. The pseudo-outcome

$\hat\tau_k(x)$0

has conditional mean equal to the target conditional estimand, and the three-way design is used to reduce the interaction remainder $\hat\tau_k(x)$1 that limits ordinary two-way cross-fitting (Fisher et al., 2023).

3. Fold geometry and strengthened splitting schemes

Although the elementary picture of SSCF is “train on one fold, evaluate on another,” the literature studies a wider geometry. In meta-learner work, the design space includes one-shot 50:50 splitting, 50:50 cross-fit, double split, double split cross-fit, 5-fold split, 5-fold cross-fit, 5-fold combined, and repeated variants that take the median over repeated iterations (Jacob, 2020). These distinctions matter because learners such as the DR-learner, R-learner, and X-learner use several nuisance estimates and can react differently to the amount of data reserved for nuisance learning versus evaluation.

The strongest fold-separation ideas arise when different nuisance branches themselves should not reuse training data. The graph-based crossfit engine in R formalizes this by asking the analyst to specify a target functional and a directed acyclic graph of nuisance models. It then executes an explicit schedule over folds, panels, and repetitions, with node-specific train_fold widths and target-specific evaluation windows (Peyrot et al., 15 May 2026). Within this framework, three fold-allocation modes are distinguished. In overlap, each nuisance is trained out-of-sample relative to the evaluation fold, but different nuisances may still use overlapping or identical training folds. In disjoint, training folds are coordinated so that nuisance components that interact in remainder terms are trained on different data when possible. In independence, reused nodes are duplicated by a tree expansion so that each branch can be trained on disjoint folds (Peyrot et al., 15 May 2026).

This proliferation of geometries reflects a concrete analytical point: many orthogonal or doubly robust estimators contain remainder terms involving products of nuisance estimation errors. If the nuisance branches are trained on the same data, those errors can be correlated. Double cross-fitting, three-way cross-fitting, triple cross-fit, and multiway cross-fitting are all attempts to reduce that correlation structure. A plausible implication is that SSCF is not a single algorithmic object but a design space whose operative parameter is the degree of dependence permitted among nuisance learners.

4. When SSCF is relaxed or omitted

A central development after the consolidation of cross-fitting in DML is the demonstration that SSCF is not universally necessary. One route replaces data partitioning by a stability condition on the nuisance learner. In “Debiased Machine Learning without Sample-Splitting for Stable Estimators,” the nuisance algorithm is trained on the full sample and compared to a leave-one-out perturbation $\hat\tau_k(x)$2. The main theorem assumes leave-one-out stability bounds such as

$\hat\tau_k(x)$3

together with mean-squared continuity and the usual orthogonality and RMSE conditions. Under these assumptions, the no-splitting estimator remains root-$\hat\tau_k(x)$4 consistent and asymptotically normal (Chen et al., 2022). Bagged ensemble estimators built via subsampling without replacement are a principal example; under mild moment assumptions and suitable growth conditions, a typical sufficient regime is $\hat\tau_k(x)$5 with $\hat\tau_k(x)$6 large enough (Chen et al., 2022).

A second route dispenses with SSCF through localization rather than stability. In DML-GMM with general multiway clustered dependence, valid inference can be obtained without sample splitting by combining Neyman-orthogonal moments with a localization-based empirical-process argument. The nuisance estimator is trained on the full sample, localized to a shrinking neighborhood $\hat\tau_k(x)$7, and the centered empirical process is controlled by new maximal inequalities for separately exchangeable arrays. Under an $\hat\tau_k(x)$8-type nuisance rate, VC-type complexity, and separate exchangeability plus dissociation, the resulting estimator is asymptotically linear and asymptotically normal (Chen et al., 11 Feb 2026).

These two lines of work do not deny the usefulness of SSCF; rather, they reclassify it. Cross-fitting remains a convenient device for avoiding difficult same-sample empirical-process arguments, but these papers show that it is not a logical necessity in every orthogonal semiparametric problem. The controversy, therefore, is not whether SSCF works, but whether its convenience should be treated as a requirement.

5. Non-i.i.d. data, experimental design, and dependence-aware splitting

The i.i.d. template is insufficient in several important settings. In design-based randomized experiments, potential outcomes and covariates are fixed and randomization is the sole source of randomness. Ordinary cross-fitting theory assumes i.i.d. sampling and therefore does not directly apply to completely randomized, stratified, or matched-pairs designs. Conditional cross-fitting resolves this by constructing a split such that, conditional on the split,

$\hat\tau_k(x)$9

and each unit retains positive treatment and control probability. Under this condition, the cross-fitted ATE estimator is exactly unbiased for the finite-population ATE,

$$\tilde{\tau}_K(x)=\frac{1}{K}\sum_{k=1}^K \hat{\tau}_k(x),$$0

and the associated variance estimator yields asymptotically valid confidence intervals for BRE, CRE, SRE, and matched-pairs experiments (Lu et al., 21 Aug 2025).

For correlated observational units, the direction of adjustment is more surprising. The paper on cross-fitting with correlated units argues that bespoke fold construction is often unnecessary: one may frequently perform cross-fitting as if units were independent and still eliminate the empirical-process term. The key decomposition is

$$\tilde{\tau}_K(x)=\frac{1}{K}\sum_{k=1}^K \hat{\tau}_k(x),$$1

and cross-fitting targets the last term. If the number of correlated pairs grows no faster than $$\tilde{\tau}_K(x)=\frac{1}{K}\sum_{k=1}^K \hat{\tau}_k(x),$$2, then under an as-IID split

$$\tilde{\tau}_K(x)=\frac{1}{K}\sum_{k=1}^K \hat{\tau}_k(x),$$3

so the cross-fitted estimator retains the same asymptotic rate in clustered, network, and fixed-$$\tilde{\tau}_K(x)=\frac{1}{K}\sum_{k=1}^K \hat{\tau}_k(x),$$4-dependent time-series settings covered by the paper’s corollaries (Balkus et al., 15 Jan 2026).

Weakly dependent time series require a different argument again. Sample splitting after model selection remains asymptotically valid under suitable $$\tilde{\tau}_K(x)=\frac{1}{K}\sum_{k=1}^K \hat{\tau}_k(x),$$5-mixing or $$\tilde{\tau}_K(x)=\frac{1}{K}\sum_{k=1}^K \hat{\tau}_k(x),$$6-dependence assumptions and can be formulated even without stationarity when the target parameter is defined in an $$\tilde{\tau}_K(x)=\frac{1}{K}\sum_{k=1}^K \hat{\tau}_k(x),$$7-indexed way. The paper’s stability theorem replaces exact independence between the selection and inference blocks by asymptotic decoupling, and a block multiplier bootstrap under $$\tilde{\tau}_K(x)=\frac{1}{K}\sum_{k=1}^K \hat{\tau}_k(x),$$8-dependence provides the inferential approximation (Lunde, 2019).

Sample splitting can also simplify goodness-of-fit testing under dependence. In generalized spectral testing for time-series conditional mean models, the parameter is estimated on a fitting subsample and the residual-based Cramér–von Mises statistic is constructed on a checking/testing subsample. Under mild regularity conditions, a score-alignment condition, and a split-ratio condition, the residual-based process has the same limiting null distribution as the infeasible oracle process based on the true errors. Because of this oracle-equivalence property, the bootstrap keeps the estimated parameter fixed and does not require re-estimating the model in each bootstrap replication (Tao et al., 28 May 2026).

6. Finite-sample behavior, aggregation, and software implementation

The applied literature repeatedly emphasizes that SSCF is a finite-sample design choice, not only an asymptotic safeguard. In Monte Carlo work on CATE meta-learners, the performance of all meta-learners heavily depends on the procedure of splitting and averaging. Among the sample-split estimators considered there, the best performance in terms of mean squared error can be achieved when applying cross-fitting plus taking the median over multiple different sample-splitting iterations, and the paper reports that after about 20 repeated iterations results often stabilize for 5-fold cross-fit estimators (Jacob, 2020).

Other simulations reach a more qualified conclusion. In a separate study of meta-learners, sample-splitting and cross-fitting are beneficial in large samples for bias reduction and efficiency of the meta-learners, respectively, whereas full-sample estimation is preferable in small samples (Okasa, 2022). In high-dimensional confounding for ACE estimation with AIPW and TMLE, cross-fitting improves the performance of both methods but is more important for estimation of standard error and coverage than for point estimates; the number of folds is a less important consideration, and TMLE is reported as more stable than AIPW (Ellul et al., 2024). These findings imply that the cost of withholding data from nuisance training is estimator- and regime-specific.

Repeated splitting introduces its own inferential problem because estimates from different splits are dependent. A recent central limit theorem for split-sample estimators addresses this directly. For a broad class of split-sample Z-estimators,

$$\tilde{\tau}_K(x)=\frac{1}{K}\sum_{k=1}^K \hat{\tau}_k(x),$$9

and the variance factor satisfies $\theta_0 \in \mathbb{R}^p$0 whenever $\theta_0 \in \mathbb{R}^p$1, so cross-fitting minimizes the asymptotic variance inflation present in one-shot sample splitting (Fava, 7 Nov 2025). The same paper shows that normal-approximation confidence intervals may undercover in important model-comparison problems and develops an alternative inference procedure that explicitly accounts for dependence across splits, as well as a reproducibility measure for p-values (Fava, 7 Nov 2025).

On the implementation side, the crossfit package turns SSCF into an explicit computational object. It provides a general-purpose, estimator-agnostic engine in R, exposes fold geometry through folds, panels, repetitions, and evaluation windows, validates nuisance dependencies through a DAG representation, and uses reuse-aware caching to avoid redundant refits. The package is available on CRAN, openly developed on GitHub under GPL-3, and is intended as a lightweight, tested foundation for reproducible benchmarking and method development (Peyrot et al., 15 May 2026).

7. Broader applications and recurrent misconceptions

Beyond semiparametric estimation, SSCF appears wherever in-sample reuse distorts the target criterion. In complex changepoint models, minimizing in-sample loss can fail because hyperparameter tuning, model selection, or interpolation induces a severe downward bias in segmentwise losses. The proposed remedy is a cross-fitted segmentation criterion,

$\theta_0 \in \mathbb{R}^p$2

which evaluates each segment on observations excluded from the fit used to score them. Under predictive-accuracy and signal-spacing conditions, the procedure consistently recovers both the number and locations of changepoints (Qian et al., 2024).

In network models, sample splitting plays a different role: it creates independence between a rough first-stage estimator and the second-stage refinement statistics. In stochastic block models and degree-corrected block models, SplitClust randomly partitions the nodes, obtains an approximately correct partition on one half, and then refines the other half by cross clustering based on average adjacency profiles into the estimated communities. Under expected degree at least order $\theta_0 \in \mathbb{R}^p$3, balanced communities, and sufficient separation of community profiles, the method exactly recovers the communities with high probability (Lei et al., 2014).

In post-selection and score-construction problems, repeated splitting stabilizes inference. Physical-activity scoring uses one sample to build a score and the other to estimate its association with mortality; with multiple random splits, the averaged estimator becomes asymptotically equivalent to solving a stacked set of estimating equations (Kravitz et al., 2019). In high-dimensional generalized linear models, one subsample is used for lasso variable selection and the held-out subsample for debiased lasso estimation; multiple sample splitting is then used to address the loss of efficiency associated with a single split and to produce asymptotically normal estimates for prespecified coefficients (Vazquez et al., 2023).

Several misconceptions recur across these literatures. One is that cross-fitting is always required for valid orthogonal inference; the stability-based and localization-based no-splitting results show that this is false under additional assumptions (Chen et al., 2022, Chen et al., 11 Feb 2026). A second is that folds must be independent in the predictive-modeling sense; the correlated-units results show that ordinary as-IID cross-fitting often still eliminates the relevant bias term when dependence is weak enough (Balkus et al., 15 Jan 2026). A third is that stronger separation is uniformly better; the finite-sample studies instead show a bias-variance tradeoff in which full-sample estimation may dominate in small samples, while cross-fitting or repeated cross-fitting becomes preferable as nuisance complexity and sample size grow (Okasa, 2022, Jacob, 2020).