---
title: Predict-Then-Debias Techniques
url: https://www.emergentmind.com/topics/predict-then-debias
type: topic
---

# Predict-Then-Debias Techniques

Predict-Then-Debias denotes a family of two-stage workflows in which a first-stage predictor, proxy, or intentionally biased model is constructed, and a second stage removes or attenuates the bias induced by spurious correlations, nuisance structure, measurement error, or prediction error. The label has been used across several research programs rather than for a single canonical algorithm. In representation learning and dataset-bias mitigation, it refers to training a biased classifier and then using its failures to train a debiased classifier [2007.02561]. In semiparametric and high-dimensional inference, it refers to learning nuisance functions or external predictions first and then debiasing the downstream estimator by orthogonal moments, residualization, bootstrap correction, or debiased regularization [2011.00417], [2502.12323], [2507.09119], [2501.18577], [2606.15602]. This suggests a unifying template: prediction is treated as an intermediate object, not as the final estimand.

## 1. Conceptual scope and recurring architecture

Across the literature, Predict-Then-Debias is organized around a division between a prediction stage and a correction stage. The prediction stage may produce a biased classifier, a nuisance-function approximation, a machine-learned outcome proxy, imputed covariates, or an external labeler. The debiasing stage then modifies optimization, moment conditions, or inferential formulas so that the final predictor or estimator is less sensitive to the bias carried by the first stage.

The recurring distinction is between predictive performance and downstream validity. In classifier debiasing, the objective is robustness to dataset bias without explicit bias labels [2007.02561]. In semiparametric and post-prediction inference, the objective is valid estimation of a target parameter such as \(\beta\), a regression slope, or a general functional \(\theta=\phi(\mathbb P_X)\), even when the available predictions are noisy or differentially biased [2011.00417], [2502.12323], [2501.18577].

| Framework | Predict stage | Debias stage |
| --- | --- | --- |
| Learning from Failure | Train biased network \(f_b(x;\theta_b)\) with GCE | Train debiased network \(f_d(x;\theta_d)\) with weighted CE |
| DebiNet | Fit wide ReLU network to \((\mathbb E[Y\mid Z],\mathbb E[D\mid Z])\) | Residualize and run OLS or cross-fitting |
| Adversarial debiasing | Fit \(\hat y=f(x;\theta)\) | Penalize information that residuals \(\nu\) carry about \(z\) |
| Moment-based post-prediction inference | Fit black-box \(\hat Y=f(\boldsymbol Z)\) and calibration model | Use moment-corrected estimator with \(c=N/n\) |
| PTD with imputed covariates | Use proxy covariates \(\tilde X\) on a large incomplete sample | Correct by \(\hat\theta_{\rm cl}+(\hat\gamma_{\rm big}-\hat\gamma_{\rm sml})\) and bootstrap |
| DEAL | Use external predictor \(\hat f(x)\) or \(\hat\beta_{\rm ext}\) | Bias-aware shrinkage, stacked Lasso refit, final debiasing |

A central implication of these formulations is that the first-stage model is often allowed to be imperfect. Some methods exploit its imperfection directly: Learning from Failure amplifies the prejudice of a biased network so that its failures reveal bias-conflicting samples [2007.02561], while adversarial post-prediction regression explicitly models the bias induced by residuals correlated with regressors [2502.12323].

## 2. Failure-based classifier debiasing

In "Learning from Failure," Predict-Then-Debias is implemented as a two-network architecture with identical backbones, such as an MLP for Colored MNIST or ResNet-20/18 for CIFAR-10, CelebA, and BAR [2007.02561]. The biased network \(f_b(x;\theta_b)\) is encouraged to learn spurious correlations, while the debiased network \(f_d(x;\theta_d)\) is trained to focus on samples that the biased network finds difficult.

The biased-network loss is the Generalized Cross-Entropy loss
\[
L_b(x,y;\theta_b)=\mathrm{GCE}(p_b(x),y)=\frac{1-p_{b,y}(x)^q}{q},
\]
with gradient
\[
\frac{\partial L_b}{\partial \theta_b}=p_{b,y}(x)^q\cdot \frac{\partial \mathrm{CE}(p_b(x),y)}{\partial \theta_b}.
\]
Because this gradient up-weights easy examples with high \(p_{b,y}(x)\), the biased network quickly memorizes bias-aligned samples. The debiased network uses weighted standard cross-entropy with per-example weight
\[
w(x)=\frac{\mathrm{CE}(p_b(x),y)}{\mathrm{CE}(p_b(x),y)+\mathrm{CE}(p_d(x),y)},
\]
so that examples easy for \(f_b\) receive low weight and examples difficult for \(f_b\) receive weight near \(1\).

The theoretical motivation is an empirical observation about malignant bias: when the bias attribute is easier than the target attribute, a network trained with standard cross-entropy learns bias-aligned samples early and only later fits bias-conflicting samples. By amplifying that easy bias pattern in \(f_b\) via GCE, the method exaggerates failures on bias-conflicting examples; \(f_d\) then focuses on those failures. The paper explicitly connects this behavior to the "small-loss first" learning dynamics in deep nets [2007.02561].

The joint training algorithm samples a minibatch, updates the biased network with GCE, optionally maintains an exponential moving average of each cross-entropy for stable \(w(x)\) estimates, and updates the debiased network with \(w(x)\cdot \mathrm{CE}(p_d(x),y)\). Typical hyperparameters are \(q=0.7\), \(\eta=10^{-3}\) for small datasets, \(\eta=10^{-4}\) for large datasets, batch size \(256\), optimizer Adam, and training epochs \(100\)–\(200\) for synthetics and \(50\)–\(90\) for real-world benchmarks [2007.02561].

The empirical protocol uses both controlled and real-world datasets. On Colored MNIST at \(p_{\rm align}=99.5\%\), vanilla unbiased accuracy increases from \(35.34\%\) to \(63.39\%\), and bias-conflicting accuracy from \(28.15\%\) to \(63.49\%\). On Corrupted CIFAR-10\(^1\), vanilla unbiased accuracy increases from \(17.93\%\) to \(31.66\%\), and bias-conflicting accuracy from \(10.50\%\) to \(28.61\%\). On CelebA HairColor, vanilla accuracy increases from \(70.25\%\) to \(84.24\%\), with bias-conflicting accuracy increasing from \(52.52\%\) to \(81.24\%\). On BAR, average evaluation accuracy increases from \(51.85\%\) to \(62.98\%\), compared with \(59.74\%\) for ReBias. The reported summary is that LfF significantly improves robustness to dataset bias without explicit bias labels or architecture changes, and in some cases outperforms debiasing methods that require explicit supervision of spuriously correlated attributes [2007.02561].

## 3. Semiparametric residualization and DebiNet

DebiNet uses Predict-Then-Debias in a semiparametric partially linear model,
\[
Y_i=D_i^\top \beta + f(Z_i)+\varepsilon_i,\qquad \mathbb E[\varepsilon_i\mid D_i,Z_i]=0,
\]
where \(\beta\in\mathbb R^{p_L}\) is the low-dimensional target parameter and \(f:\mathbb R^{p_N}\to\mathbb R\) is a nuisance function [2011.00417]. The prediction stage is the joint approximation of \(\mathbb E[Y\mid Z]\) and \(\mathbb E[D\mid Z]\), written as
\[
M(Z)=\bigl(\mathbb E[Y\mid Z],\mathbb E[D\mid Z]\bigr)\in\mathbb R^{1+p_L}.
\]

To estimate \(M(Z)\), DebiNet fits a wide two-layer ReLU network
\[
F(W,A;z)=\frac1{\sqrt m}\sum_{r=1}^m A_r\,\sigma(W_r^\top z)\in\mathbb R^{1+p_L},
\]
with training objective
\[
L(W,A)=\sum_{i=1}^n \frac12\left\|F(W,A;Z_i)-M_i\right\|_2^2.
\]
Under NTK-type over-parameterization conditions, with width \(m\gtrsim n^6\cdot \mathrm{poly}(p_L)/\lambda_0^4\), gradient descent on \(W\) with random fixed \(A\) converges exponentially fast to zero training loss. The paper notes that in practice one may train both layers jointly and use Adam or SGD, but the theory is easiest with lazy first-layer training [2011.00417].

The debiasing stage is residualization followed by orthogonal-moment estimation. With
\[
\widehat M_i=\bigl(\widehat g_y(Z_i),\widehat g_D(Z_i)\bigr),
\]
the residuals are
\[
\widetilde Y_i=Y_i-\widehat g_y(Z_i),\qquad \widetilde D_i=D_i-\widehat g_D(Z_i).
\]
The orthogonal moment leads to
\[
\frac1n\sum_i \widetilde D_i\bigl(\widetilde Y_i-\widetilde D_i^\top\beta\bigr)=0,
\]
which is equivalent to OLS of \(\widetilde Y\) on \(\widetilde D\), giving
\[
\hat\beta=\left(\frac1n\sum_i \widetilde D_i\widetilde D_i^\top\right)^{-1}\left(\frac1n\sum_i \widetilde D_i\widetilde Y_i\right).
\]
Cross-fitting is optional: partition the data into \(K\) folds, train on all data except fold \(k\), predict on fold \(k\), form residuals there, compute \(\hat\beta^{(k)}\), and average [2011.00417].

The theoretical guarantees state that if \(\|\widehat g-\mathbb E[g\mid Z]\|_{L^2}=o_p(n^{-1/4})\), then \(\hat\beta\) is \(\sqrt n\)-consistent,
\[
\sqrt n(\hat\beta-\beta)\xrightarrow{d}N(0,\Sigma),
\]
with asymptotic variance
\[
\Sigma=\sigma^2\Bigl(\mathbb E[D-\mathbb E[D\mid Z]\,(D-\mathbb E[D\mid Z])^\top]\Bigr)^{-1},
\]
and a Wald-type \(95\%\) confidence interval based on
\[
\hat\beta_j \pm 1.96\,\hat\sigma\,\sqrt{\left[\left(\frac1n\sum \widetilde D_i\widetilde D_i^\top\right)^{-1}\right]_{jj}/n}.
\]

Empirically, synthetic PLM experiments compare PLM-NN with NW-kernel PLM, DML-Lasso, DML-RF, and related alternatives using estimation MSE \(\|\hat\beta-\beta\|^2\) and train/test MSE; the summary is that PLM-NN matches or beats alternatives, with far fewer nuisance-fits. In high-dimensional linear data, DebiNet is reported to attain the lowest MSE, valid \(>90\%\) coverage even when debiased-Lasso fails, and fast runtime. On the 401(k) treatment-effect example, PLM-NN gives \(\mathrm{ATE}\approx 0.59\) with \(\mathrm{SE}\approx 0.19\), versus kernels \(\approx 0.91\) with \(\mathrm{SE}\,0.35\) and DML-Lasso \(\approx 0.89\) with \(\mathrm{SE}\,0.42\) [2011.00417].

## 4. Post-prediction inference with predicted outcomes

A different Predict-Then-Debias line studies downstream regression when the dependent variable is a machine-learned prediction. The basic workflow is explicit: first fit an ML predictor \(\hat y=f(x;\theta)\approx y\), then plug \(\hat y\) into an OLS regression on covariates \(z\) [2502.12323]. If the true model is
\[
y=z^\top\beta^\ast+\eta,\qquad \mathbb E[\eta\mid z]=0,
\]
and the prediction error is \(\nu=y-\hat y\), then OLS on \(\hat y\) yields
\[
\hat\beta=\beta^\ast+(Z^\top Z)^{-1}Z^\top\eta+(Z^\top Z)^{-1}Z^\top\nu.
\]
The only bias term is
\[
\mathbb E[\hat\beta]-\beta^\ast=(Z^\top Z)^{-1}Z^\top \mathbb E[\nu],
\]
or, in the scalar case,
\[
\mathbb E[\hat\beta]=\beta^\ast+\frac{\mathrm{Cov}(X,\nu)}{\mathrm{Var}(X)}.
\]
The operative issue is differential error, \(\mathrm{Cov}(\nu,z)\neq 0\), rather than merely low predictive accuracy [2502.12323].

The adversarial debiasing response is to learn \(\theta\) so that residuals \(\nu=y-f(x;\theta)\) carry no linear information about \(z\). The min-max objective is
\[
\min_\theta \left\{L_{\rm pred}(f(x;\theta),y)+\lambda\cdot \max_\psi L_{\rm adv}(f(x;\theta),z;\psi)\right\}.
\]
For a linear adversary, \(\nu_i\approx z_i^\top \psi+\epsilon_i\), with
\[
L_{\rm adv}=\frac1n\|\nu-Z(Z^\top Z)^{-1}Z^\top \nu\|^2.
\]
Training alternates between updating \(\psi\) and updating \(\theta\) using \(L_{\rm pred}-\lambda L_{\rm adv}\). The paper states that as \(\lambda\to\infty\), if prediction accuracy is held constant, \(\mathrm{Cov}(z,\nu)\to 0\), so the bias term in \(\hat\beta\) shrinks. A diagnostic measurement-error test regresses \(\nu\) on \(z\) in a small labeled sample and tests \(H_0:\gamma=0\) using standard or bootstrapped standard errors [2502.12323].

The reported simulations use \(n=20{,}000\) points and labeled sample sizes \(J\) from \(100\) to \(10{,}000\). Baseline ML predictions yield a large negative bias in \(\hat\beta\), while both bias-correction and adversarial methods recover \(\beta^\ast=0\) on average. With true bias \(0.025\), about \(2{,}500\) labels are needed to detect bias at \(80\%\) power. In the West Africa road-and-forest-cover case study, the true slope is approximately \(-0.15\), the baseline overestimates the road effect at approximately \(-0.25\), and adversarial and bias-correction nearly recover the true slope with valid standard errors. Hyperparameter tuning shows that \(\lambda\sim 1\) suffices, and the DNN primary model can gain slight accuracy improvement because the adversary acts as a regularizer [2502.12323].

A related contribution generalizes post-prediction inference with a moment correction. The setup uses a labeled sample
\[
\mathcal L=\{(Y_i,\boldsymbol X_i,\boldsymbol Z_i)\}_{i=1}^n
\]
and an unlabeled sample
\[
\mathcal U=\{(\boldsymbol X_i,\boldsymbol Z_i)\}_{i=n+1}^{n+N},
\]
with \(\hat Y=f(\boldsymbol Z)\) and downstream target
\[
Y=\boldsymbol X^\top \beta+\varepsilon,\qquad \mathbb E[\varepsilon\mid \boldsymbol X]=0.
\]
Naive regression of \(\hat Y\) on \(\boldsymbol X\) is biased whenever
\[
\eta=Y-\mathbb E[Y\mid \hat Y]
\]
satisfies \(\mathrm{Cov}(\boldsymbol X,\eta)\neq 0\) [2507.09119].

Wang et al. (2020) are reviewed through the calibration model
\[
Y_i=\gamma_0+\gamma_1\hat Y_i+\eta_i,\qquad \mathbb E[\eta_i\mid \hat Y_i]=0,
\]
with key assumption \(\eta\perp\!\!\!\perp \boldsymbol X\). The moment-based extension relaxes that assumption and uses
\[
\hat\beta=
\bigl(\hat M_{XX}^{(u)}\bigr)^{-1}
\Bigl(\hat\gamma_1\,\hat C_{X\hat Y}^{(u)}+\hat C_{X\eta}^{(\ell)}\Bigr),
\]
where \(\hat C_{X\eta}^{(\ell)}\) is estimated in the labeled sample and \(\hat C_{X\hat Y}^{(u)}\), \(\hat M_{XX}^{(u)}\) are estimated in the unlabeled sample. To preserve calibration variability when \(N\) is large, the method introduces the scaling factor
\[
c=\frac{N}{n},
\]
so that
\[
\widehat{\mathrm{Var}(\hat\beta)}=
\frac1N\bigl(\hat M_{XX}^{(u)}\bigr)^{-1}
\Bigl(\hat\gamma_1^2\hat S_1^{(u)}+c\,\hat S_2^{(\ell)}\Bigr)
\bigl(\hat M_{XX}^{(u)}\bigr)^{-1}.
\]
Under i.i.d. sampling, correct model specification, and moment consistency, the estimator is unbiased; by a multivariate CLT and the delta method, it is asymptotically normal and yields asymptotically correct nominal Type I error and coverage. The simulation summary is that the estimator is unbiased, controls Type I error at \(\alpha\), and achieves near-nominal coverage in all settings, while naive regression and original PostPI fail when \(\mathrm{Cov}(\boldsymbol X,\eta)\neq 0\) or when \(N\gg n\) [2507.09119].

## 5. Imputed covariates, complex sampling, and external-model-assisted high-dimensional regression

Another Predict-Then-Debias formulation treats machine learning as an imputation device for missing covariates rather than for outcomes. In the two-phase sampling setup, the true covariates are \(X\in\mathbb R^p\), proxy covariates are \(\tilde X\in\mathbb R^p\), and the target is a \(d\)-dimensional functional
\[
\theta=\phi(\mathbb P_X),
\qquad
\gamma=\phi(\mathbb P_{\tilde X}).
\]
A label indicator \(I_i\in\{0,1\}\) induces weights
\[
W_i=\frac{I_i}{\pi(\tilde X_i)},
\qquad
\bar W_i=\frac{1-I_i}{1-\pi(\tilde X_i)},
\]
where \(\pi(\tilde X)=\mathbb P(I=1\mid \tilde X)\) is known and bounded away from \(0\) and \(1\) [2501.18577].

Given a black-box routine \(A(D,w)\in\mathbb R^d\), the paper defines
\[
\hat\theta_{\rm cl}=A(X_{1:N};W_{1:N}),
\qquad
\hat\gamma_{\rm sml}=A(\tilde X_{1:N};W_{1:N}),
\qquad
\hat\gamma_{\rm big}=A(\tilde X_{1:N};\bar W_{1:N}),
\]
and the PTD estimator
\[
\hat\theta^{\rm PTD}=\hat\theta_{\rm cl}+(\hat\gamma_{\rm big}-\hat\gamma_{\rm sml}).
\]
A more general version uses a tuning matrix \(\Omega\),
\[
\hat\theta_\Omega=\Omega\,\hat\gamma_{\rm big}+\bigl(\hat\theta_{\rm cl}-\Omega\,\hat\gamma_{\rm sml}\bigr),
\]
with asymptotic normality
\[
\sqrt N(\hat\theta_\Omega-\theta)\dto \mathcal N(0,\Sigma(\Omega)).
\]
The optimal tuning matrix is
\[
\Omega_{\rm opt}=(\Sigma_{\tilde X,W}+\Sigma_{\tilde X,\bar W})^{-1}\Sigma_{X,\tilde X}.
\]
In the uniform subsampling case, the variance identity
\[
\mathrm{Var}(\hat\theta_{{\rm opt},j})
=
\mathrm{Var}(\hat\theta_{{\rm cl},j})
\Bigl[1-(1-\pi_L)\,\mathrm{Corr}^2\bigl(\Psi_j(X),\tilde\Psi_j(\tilde X)\bigr)\Bigr]
\]
shows that PTD is always at least as efficient as the classical estimator [2501.18577].

Confidence intervals are produced by a percentile bootstrap that resamples weighted two-phase data and recomputes \(\hat\theta_{\rm cl}^{\ast(b)}\), \(\hat\gamma_{\rm sml}^{\ast(b)}\), and \(\hat\gamma_{\rm big}^{\ast(b)}\), or by a faster convolution bootstrap. Stratified and cluster-bootstrap variants are also given. The key theoretical statements are asymptotic normality, PTD no less efficient than the classical estimator, bootstrap consistency, validity independent of ML quality, and confidence intervals no wider than those that ignore the proxy. Real-data examples include remote-sensing of housing prices, tree-cover regression with clustered sampling by \(0.5^\circ\) grid, and census disability with stratified sampling by age group [2501.18577].

In high-dimensional semi-supervised linear regression, DEAL extends the same logic to an external predictor \(f\) or \(\hat\beta_{\rm ext}\). The target model is
\[
y_0=X_0\beta^\star+\varepsilon_0,\qquad \mathbb E[\varepsilon_0\mid X_0]=0,\qquad \mathrm V(\varepsilon_0\mid X_0)=\sigma^2 I_{n_0},
\]
with abundant unlabeled covariates \(\tilde X\) and sparse \(\beta^\star\) [2606.15602]. A central observation is the rectifier-cancellation proposition: if \(f(X)=X\beta_f\), then
\[
\hat\beta^{\rm PPI}=\hat\beta_{\rm OLS}
\qquad\text{and}\qquad
\hat\beta^{\rm PPI++}=\hat\beta_{\rm OLS}.
\]
The paper concludes that PPI and PPI++ cannot improve on labeled-only OLS, or its high-dimensional analog, when the labeler is linear or nearly oracle.

DEAL avoids mean-based rectification and instead routes the external estimator and the unlabeled covariates into the variance of a debiased estimator. Its four stages are a bias-aware initializer, pseudo-label imputation, stacked Lasso refit, and final debiasing. The initializer is
\[
\tilde\beta^{\rm init}=\hat\beta_{\rm ext}+\hat t\,C,
\]
where \(C=\tfrac1{n_0}M_1X_0^\top(y_0-X_0\hat\beta_{\rm ext})\) and \(\hat t\in[0,1]\) is a cross-fitted shrinkage estimate. The final debiased estimator is
\[
\tilde\beta=\hat\beta+\frac1{n_0+N}M_2[X_{\rm stk}]^\top([y_0;\tilde f]-X_{\rm stk}\hat\beta),
\]
with coordinatewise confidence intervals
\[
\tilde\beta_j\pm z_{1-\alpha/2}\,\hat\tau_j/\sqrt{n_0+N}.
\]
Under the paper’s assumptions, \(\sqrt{n_0+N}(\tilde\beta_j-\beta_j^\star)\xrightarrow{d}\mathcal N(0,\tau_j^2)\). Under misspecification, validity extends to the best-linear projection \(\beta^\star_{\rm proj}\). The interval-length comparison states that, at the same \(N\), DEAL intervals are shorter than those of debiased Lasso, PPI, and PPI++; in simulations, DEAL median confidence-interval length is \(0.49\)–\(0.87\) of debiased Lasso, and in six real-data applications the median ratio is \(0.23\)–\(0.53\). A shift-aware variant restores unbiasedness and the CLT under covariate shift [2606.15602].

## 6. Common structure, limitations, and adjacent post-processing methods

Several misconceptions are corrected by this literature. One is that a highly accurate upstream predictor can be treated as ground truth in downstream analysis. Papers on adversarial debiasing, moment-based post-prediction inference, and PTD with imputed covariates all show that naive use of predicted data can produce biased coefficients, invalid standard errors, or incorrect coverage when residual prediction error is correlated with downstream covariates or when calibration uncertainty is ignored [2502.12323], [2507.09119], [2501.18577]. Another is that debiasing necessarily requires explicit labels for the bias attribute: Learning from Failure improves robustness without explicit bias labels by exploiting training dynamics instead [2007.02561].

The methods also expose distinct bias-variance trade-offs. In Learning from Failure, the debiasing mechanism depends on early-phase preference for easy, bias-aligned samples and on amplifying that preference with GCE [2007.02561]. In adversarial post-prediction regression, if \(\lambda\) is too small there is no debiasing, while if \(\lambda\) is too large the model may randomize \(\hat y\) to thwart the adversary at the expense of accuracy; the algorithmic stability issue is attributed to the non-convexity of adversarial loss [2502.12323]. In moment-based post-prediction inference, larger labeled-sample size \(n\) reduces the variance term \(c\,S_2^{(\ell)}\), and the recommendation is that \(n\) be at least a few times \(p\) so that the relationship model and moment estimates are stable [2507.09119]. In PTD with imputed covariates, efficiency gains depend on \(\mathrm{Corr}(\Psi(X),\tilde\Psi(\tilde X))\), but validity does not require \(\tilde X\approx X\) [2501.18577].

A related but distinct post-processing line is TowerDebias, which addresses unfairness in black-box predictions rather than regression bias from predicted data. It uses the Tower Property
\[
\mathbb E[Y\mid X]=\mathbb E[\mathbb E(Y\mid X,S)\mid X]
\]
and replaces \(f(X,S)\) by
\[
\widetilde f(X)=\mathbb E[f(X,S)\mid X].
\]
The paper states an informal theorem that
\[
|\mathrm{Cov}(\widetilde f,S)|\le |\mathrm{Cov}(f,S)|
\quad\Longrightarrow\quad
|\rho(\widetilde f,S)|\le |\rho(f,S)|.
\]
Empirically, tDB cuts \(|\tau|\) or \(|\rho|\) by \(30\)–\(60\%\) at modest \(k\), often around \(k=10\), with utility loss typically below a \(5\%\) rise in MAPE or below a \(10\%\) relative increase in misclassification rate [2411.08297]. This is not framed as Predict-Then-Debias in the same inferential sense, but it belongs to the same post-hoc debiasing family in which a black-box model is left intact and the correction is applied to its outputs.

Taken together, the literature uses Predict-Then-Debias to denote a class of procedures in which debiasing is downstream of prediction rather than upstream of model design. The technical realization varies—failure-based weighting, orthogonal moments, adversarial residual decorrelation, bootstrap correction, or bias-aware debiasing of regularized estimators—but the common principle is stable: prediction is used as a first-stage surrogate, and the inferential or fairness target is recovered only after an explicit debiasing step [2007.02561], [2011.00417], [2502.12323], [2507.09119], [2501.18577], [2606.15602].

Source: https://www.emergentmind.com/topics/predict-then-debias