---
title: DNN-Assisted Inverse-Probability Weighting
url: https://www.emergentmind.com/topics/dnn-assisted-inverse-probability-weighted-estimator
type: topic
---

# DNN-Assisted Inverse-Probability Weighting

A DNN-assisted inverse-probability weighted estimator is an inverse-probability weighted estimator in which the unknown sampling, treatment, or labeling probabilities are estimated with a deep neural network rather than a restrictive parametric model. In the finite-population survey-integration formulation developed in "Deep Neural Networks for Doubly Robust Estimation with Nonprobability Survey Samples" [2605.28762], the method targets the finite population mean by combining a nonprobability sample that contains the study variable with a reference probability sample that provides design-based auxiliary information. Closely related formulations use deep sequence models for inverse probability of treatment weighting with longitudinal claims records [2406.08851], impose local balance and local calibration conditions in nonparametric propensity-score estimation [2404.04794], or combine deep prediction models with Horvitz–Thompson or Hájek IPW rectification under informative labeling [2508.10149]. This suggests that “DNN-assisted IPW” is best understood as a family of reweighting procedures whose common structure is unchanged—estimate a probability of observation or treatment, invert it, and use the resulting weights to recover the target estimand—but whose score model, loss, and asymptotic analysis depend on the sampling regime.

## 1. Finite-population formulation and the classical IPW construction

In the survey-sampling setting, let $U=\{1,2,\ldots,N\}$ be the finite population and let $y_i$ denote the study variable. The target parameter is the finite population mean
$$
\mu_y = N^{-1}\sum_{i=1}^N y_i.
$$
A nonprobability sample $S_A$ of size $n_A$ is observed, with inclusion indicator $R_i=1\{i\in S_A\}$. Under the ignorability assumptions $(A1$–$A3)$,
$$
\pi_i^A := P(R_i=1\mid x_i) > 0.
$$
The classical IPW estimator replaces the unknown $\pi_i^A$ by an estimate $\hat\pi_i^A$ and reweights the observed outcomes:
$$
\hat N^A = \sum_{i\in S_A} \frac{1}{\hat\pi_i^A},
\qquad
\hat\mu_{\mathrm{IPW}} = \frac{1}{\hat N^A}\sum_{i\in S_A}\frac{y_i}{\hat\pi_i^A}.
$$
Units with low estimated selection probability $\hat\pi_i^A$ receive large weight $1/\hat\pi_i^A$ so as to recover representativeness [2605.28762].

The DNN-assisted version preserves this Horvitz–Thompson/Hájek-style logic but changes the score-estimation step. Rather than assuming a linear-logit propensity model, it treats the logit sampling score as an unknown nonparametric function and approximates that function with a deep network. The estimator is therefore “assisted” by a DNN in the sense that the weighting formula remains IPW, while the neural network is used to estimate the probabilities that define the weights.

## 2. Neural estimation of sampling or propensity scores

In the survey formulation, the unknown logit score is written as
$$
\operatorname{logit}\pi^A(x) := \log\left\{\frac{\pi^A(x)}{1-\pi^A(x)}\right\} = g_0(x),
$$
where $g_0$ is an unknown smooth function. The proposed method approximates $g_0$ by a deep ReLU network $g(x\mid\Theta)$ and estimates $\Theta$ by maximizing a pseudo-log-likelihood that combines the nonprobability sample $S_A$ and a reference probability sample $S_B$ of size $n_B$, with design weights $d_i^B=1/\pi_i^B$:
$$
\ell^*(g)
=
\sum_{i\in S_A} g(x_i)
-
\sum_{i\in S_B} d_i^B \log\!\bigl[1+\exp(g(x_i))\bigr].
$$
Equivalently, one minimizes the negative pseudo-likelihood
$$
L(\Theta) = -\ell^*(g(\cdot\mid\Theta)).
$$
No explicit $L_2$ or $L_1$ penalty appears in the objective, but the parameter space is constrained to a bounded, sparse-weight DNN class $\mathcal{G}(K,s,p,D)$, and overfitting is further controlled by early stopping [2605.28762].

The survey paper uses a $(K+1)$-layer feedforward architecture,
$$
g(x) = W_K \sigma\!\bigl(W_{K-1}\sigma(\cdots \sigma(W_0x+v_0)\cdots)+v_{K-1}\bigr)+v_K,
$$
with $\sigma(\cdot)=\operatorname{ReLU}(\cdot)=\max\{\cdot,0\}$, depth $K=O(\log n)$, layer widths $p_0,\ldots,p_K$, output dimension $p_{K+1}=1$, bounded entries $\lVert W_k\rVert_\infty,\lVert v_k\rVert_\infty\le 1$, and total nonzeros bounded by $s$. The DNN parameters are optimized by ADAM, with a learning rate $\gamma$ such as $10^{-3}$, first- and second-moment decay parameters $r_1\approx 0.9$ and $r_2\approx 0.999$, $\epsilon_0\approx 10^{-8}$, Xavier initialization for $W_k$, zero initialization for $v_k$, and early stopping when $\lVert\Theta^{(t)}-\Theta^{(t-1)}\rVert_2\le \ell$ [2605.28762].

Related work shows that the DNN component is highly setting-dependent. For longitudinal claims records, inverse probability of treatment weighting has been implemented with LSTM, Transformer encoder (“BERT_code”), and Transformer encoder on record embeddings (“BERT_record”) architectures, all trained with binary cross-entropy to estimate $e(x)=P(A=1\mid x)$ directly from raw claims histories [2406.08851]. In a different nonparametric propensity-score formulation, a three-layer feed-forward network with batch-normalization, ReLU activations, and a residual connection is trained not by cross-entropy but by minimizing a loss that enforces “local balance” and “local calibration” across a dense grid of score values [2404.04794]. These variations do not alter the IPW principle; they alter the way the score function is learned.

## 3. The DNN-assisted inverse-probability weighted estimator

After optimizing the network parameters, the survey estimator sets
$$
\hat g(x)=g(x\mid\hat\Theta),
\qquad
\hat\pi_i^A = \bigl[1+\exp\{-\hat g(x_i)\}\bigr]^{-1},
\qquad
\hat N^A = \sum_{i\in S_A}\frac{1}{\hat\pi_i^A}.
$$
The DNN-assisted inverse-probability weighted estimator, denoted DIPW in the paper, is
$$
\hat\mu_{\mathrm{DIPW}}
=
\frac{1}{\hat N^A}
\sum_{i\in S_A}
\frac{y_i}{\hat\pi_i^A}.
$$
The same DNN-estimated sampling scores are also incorporated into a deep doubly robust estimator, denoted DDR, within the same framework [2605.28762].

The defining feature of DIPW is therefore not a new weighting formula but the replacement of a parametric score model by a deep nonparametric approximation. In the paper’s narrative exposition, the contrast is explicit: instead of positing a logistic regression $\pi^A(x)=\sigma(x^\top\theta)$, the method allows $\operatorname{logit}\pi^A(x)$ to be an unknown smooth function. A plausible implication is that the estimator is designed for regimes in which the true selection mechanism contains nonlinear structure that a linear-logit model omits.

## 4. Assumptions, consistency, and convergence rates

The asymptotic analysis of DIPW is developed under Assumptions $A1$–$A3$ for ignorability, positivity, and independence; $B1$–$B2$ for DNN complexity relative to a composite Hölder class $g_0\in\mathcal{H}(q,\gamma,d,\tilde d)$; and design-consistency Conditions $C1$–$C9$ [2605.28762]. Under these conditions, the paper establishes two central rate results.

First, Theorem 1 states that
$$
\lVert \hat g-g_0\rVert_{L^2} = O_p(\gamma_n\log^2 n),
$$
where
$$
\gamma_n = \max_{i=0,\ldots,q} n^{-\tilde\gamma_i/(2\tilde\gamma_i+\tilde d_i)}
$$
captures the intrinsic smoothness and dimension. Second, Theorem 2 states that
$$
\bigl|\hat\mu_{\mathrm{DIPW}}-\mu_y\bigr| = O_p(\gamma_n\log^2 n).
$$
The proof sketch reported in the supplied material combines uniform convergence of the pseudo-log-likelihood over the sparse DNN class, empirical-process bounds, design consistency, approximation error for composite Hölder $g_0$, and then standard $M$-estimation and Taylor-expansion arguments to transfer the convergence of $\hat g$ to that of the IPW estimator [2605.28762].

A common misconception is that a DNN score model removes the need for identification assumptions. The survey formulation does not claim this. The estimator is still derived under ignorability, positivity, and independence, and its large-sample properties are stated only under the specified regularity and design-consistency conditions. Likewise, the DNN itself is not presented as unconstrained universal flexibility; the theory explicitly ties the estimator to a bounded, sparse-weight network class and to early stopping.

## 5. Finite-sample behavior and robustness to misspecification

The finite-population simulation in the survey paper uses $N=20{,}000$, $n_A=500$, and $n_B=1{,}000$. The true logit includes nonlinear terms—products, powers, sine, and log—that are omitted by a linear “parametric” logistic model. Two scenarios are considered: TF, in which the outcome regression is correct and the propensity parametric model is misspecified, and FF, in which both the outcome and parametric propensity models are misspecified [2605.28762].

The reported results are specific. Standard IPW with a linear-logit model has large bias of approximately $-6\%$ and high MSE. DIPW reduces bias to approximately $+2\%$ and cuts MSE by a factor of greater than $5$. The deep doubly robust DDR estimator further shrinks bias to below $1\%$ and further reduces MSE. Under FF, conventional doubly robust DR breaks down, with bias of approximately $-24\%$, while DIPW and DDR remain accurate. The paper summarizes these findings as evidence that the proposed estimators can improve robustness to parametric propensity-score misspecification, especially when the true selection mechanism is nonlinear [2605.28762].

Beyond simulation, the same study evaluates the proposed estimators in an empirical application using Pew Research Center and Behavioral Risk Factor Surveillance System data. The supplied abstract does not provide numerical results for that application, but it places the estimator in the broader problem of integrating nonprobability and probability survey samples, where the nonprobability sample may contain rich outcome information and the probability sample may provide design-based auxiliary information.

## 6. Related DNN-assisted IPW variants in adjacent literatures

The broader literature represented in the supplied sources places DNN-assisted IPW estimators in several adjacent regimes.

| Setting | DNN component | Weighted estimand |
|---|---|---|
| Nonprobability survey integration [2605.28762] | Deep ReLU network for $\operatorname{logit}\pi^A(x)$ via pseudo-likelihood | Finite population mean |
| Longitudinal claims or EHRs [2406.08851] | LSTM, BERT_code, or BERT_record for $e(x)=P(A=1\mid x)$ | Average treatment effect via IPTW |
| Informative labeling in PPI [2508.10149] | DNN predictor $\psi(x;\theta)$ plus estimated labeling probabilities $\pi_i$ | Population mean with HT or Hájek rectifier |

In treatment-effect estimation from claims records, inverse probability of treatment weighting is used to address time-dependent confounding. The deep-sequence formulation estimates propensity scores directly from claims histories without feature processing, then constructs either unstabilized weights,
$$
w_i = \frac{A_i}{\hat e_i} + \frac{1-A_i}{1-\hat e_i},
$$
or stabilized weights,
$$
SW_i = \frac{A_i\,\pi}{\hat e_i} + \frac{(1-A_i)(1-\pi)}{1-\hat e_i},
$$
and estimates the average treatment effect by
$$
\hat\Delta_{\mathrm{IPTW}}
=
\frac{1}{N}\sum_{i=1}^N\left(
\frac{A_iY_i}{\hat e_i}
-
\frac{(1-A_i)Y_i}{1-\hat e_i}
\right).
$$
The paper reports that deep sequence models outperform logistic regression and multilayer perceptron baselines, with and without High-Dimensional Propensity Score adjustment, in PS-MAE and ATE-MAE across all scenarios; it also reports that trimming or clipping at $\alpha=0.05$ induces negligible changes in ATE-MAE [2406.08851].

In nonparametric propensity-score estimation with optimized covariate balance, the DNN is trained to satisfy two sufficient and necessary conditions for a score $S(X)$ to equal the true propensity score: local balance, $A\perp X\mid S(X)$, and local calibration, $S(X)=E[A\mid S(X)]$. The corresponding loss function combines a local-balance criterion $Q_1(\theta)$ and a local-calibration criterion $Q_2(\theta)$ through
$$
Q(\theta)=Q_1(\theta)+\lambda Q_2(\theta),
\qquad \lambda=1,
$$
and the resulting IPW estimator targets
$$
\hat\tau_{\mathrm{IPW}}
=
\frac{1}{n}\sum_{i=1}^n
\left(
\frac{A_iY_i}{\hat e(X_i)}
-
\frac{(1-A_i)Y_i}{1-\hat e(X_i)}
\right).
$$
The paper states that LBC-Net attains the lowest GSD and LSD and the smallest RMSE and variance in Kang–Schafer simulations, and in real data from the EQLS European well-being survey achieves GSD below $0.1\%$ and LSD below $0.1\%$ across all covariates and propensity-score levels [2404.04794].

In prediction-powered inference with informative labeling, the deep network predicts outcomes on the large unlabeled set, while IPW is used only in the bias-correction term. If $\psi(x_i;\hat\theta)$ is the DNN prediction and $\pi_i=P(L_i=1\mid x_i)$ is the labeling probability, the Horvitz–Thompson and Hájek rectifiers are
$$
\widehat\Delta_{\rm HT}
=
\frac{1}{N}\sum_{i=1}^N
\frac{L_i}{\hat\pi_i}\bigl(\psi(x_i;\hat\theta)-y_i\bigr),
$$
and
$$
\widehat\Delta_{\rm H\acute{a}jek}
=
\frac{\sum_{i=1}^N \frac{L_i}{\hat\pi_i}\bigl(\psi(x_i;\hat\theta)-y_i\bigr)}
{\sum_{i=1}^N \frac{L_i}{\hat\pi_i}}.
$$
The combined estimator of the population mean is
$$
\hat\mu
=
\frac{1}{N}\sum_{i=1}^N \psi(x_i;\hat\theta)
-
\widehat\Delta,
$$
and the paper reports that, in simulations, IPW-adjusted PPI with estimated propensities closely matches the known-probability case while retaining nominal coverage and the variance-reduction benefits of PPI [2508.10149].

Taken together, these results indicate that DNN assistance is not a single implementation recipe. In some settings the network estimates the sampling score directly by pseudo-likelihood; in others it estimates the treatment propensity by binary cross-entropy; in still others it enters through a prediction model while the IPW component corrects informative labeling. What remains invariant is the inverse-probability weighting principle: probability estimates generated by a learned model are inverted and used to reweight observed information so that the resulting estimator targets a population quantity under the stated assumptions.

Source: https://www.emergentmind.com/topics/dnn-assisted-inverse-probability-weighted-estimator