Semi-Supervised Generalized Riesz Regression

• The paper introduces a semi-supervised extension of generalized Riesz regression that estimates the Riesz representer by minimizing empirical Bregman-divergence objectives while leveraging unlabeled covariates.
• It unifies debiased machine learning, covariate balancing, and density-ratio estimation under a framework that constructs efficient influence-function based estimators.
• The approach exploits dual formulations and semi-supervised constructions to reduce variance and improve estimation in settings like covariate shift and treatment-effect analysis.

Semi-Supervised Generalized Riesz Regression is a semi-supervised extension of generalized Riesz regression in which the Riesz representer is estimated by minimizing empirical Bregman-divergence objectives while incorporating unlabeled covariates into the terms that depend only on the marginal law of $X$. It belongs to the debiased machine learning and automatic debiased machine learning literature, where the parameter of interest is written as $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$, the regression function is $\gamma_0(x)=\mathbb{E}[Y\mid X=x]$, and the Riesz representer $\alpha_0(X)$ is defined by the identity $$\mathbb{E}[m(W,\gamma)] = \mathbb{E}[\alpha_0(X)\gamma(X)]$$ for all admissible $\gamma$. In semi-supervised settings, unlabeled covariates strengthen estimation of $\alpha_0$ through density-ratio components, covariate moment constraints, positive-unlabeled losses, or score-matching constructions, while the final estimator remains an orthogonal or efficient influence-function estimator built from $(\widehat\gamma,\widehat\alpha)$ (Kato, 12 Jan 2026, Kato, 11 Nov 2025, Kato, 11 Jun 2026).

1. Riesz representers, orthogonal scores, and target functionals

The central object is the Riesz representer associated with a linear functional of a regression. In the basic setup, $W=(X,Y)$ is generated by $P_0$, $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$0, and the target parameter has the form

$\theta_0 = \mathbb{E}[m(W,\gamma_0)]$1

When $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$2 is linear in $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$3 and continuous in $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$4, the Riesz representation theorem yields a unique $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$5 such that

$\theta_0 = \mathbb{E}[m(W,\gamma_0)]$6

The associated debiased or Neyman-orthogonal score is

$\theta_0 = \mathbb{E}[m(W,\gamma_0)]$7

and at $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$8 it has mean zero with vanishing Gateaux derivative with respect to $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$9 (Kato, 12 Jan 2026).

Several canonical targets fit this template. For the average treatment effect (ATE), with $\gamma_0(x)=\mathbb{E}[Y\mid X=x]$0 and $\gamma_0(x)=\mathbb{E}[Y\mid X=x]$1, the functional is $\gamma_0(x)=\mathbb{E}[Y\mid X=x]$2 and the representer is

$\gamma_0(x)=\mathbb{E}[Y\mid X=x]$3

For the average marginal effect (AME), with continuous $\gamma_0(x)=\mathbb{E}[Y\mid X=x]$4, the functional is $\gamma_0(x)=\mathbb{E}[Y\mid X=x]$5 and the representer is the negative score

$\gamma_0(x)=\mathbb{E}[Y\mid X=x]$6

For the average policy effect (APE), if $\gamma_0(x)=\mathbb{E}[Y\mid X=x]$7 with $\gamma_0(x)=\mathbb{E}[Y\mid X=x]$8, then

$\gamma_0(x)=\mathbb{E}[Y\mid X=x]$9

Under covariate shift adaptation, with labeled source $\alpha_0(X)$0 and unlabeled target $\alpha_0(X)$1, the representer is the density ratio

$\alpha_0(X)$2

These identities make the Riesz representer simultaneously a debiasing weight, a density ratio, or a score function, depending on the target (Kato, 12 Jan 2026, Kato, 23 Dec 2025).

Target	Linear functional $\alpha_0(X)$3	Riesz representer
ATE	$\alpha_0(X)$4	$\alpha_0(X)$5
AME	$\alpha_0(X)$6	$\alpha_0(X)$7
APE	$\alpha_0(X)$8	$\alpha_0(X)$9
Covariate shift	$$\mathbb{E}[m(W,\gamma)] = \mathbb{E}[\alpha_0(X)\gamma(X)]$$0	$$\mathbb{E}[m(W,\gamma)] = \mathbb{E}[\alpha_0(X)\gamma(X)]$$1

This representation is the basis of automatic debiased machine learning. A foundational formulation estimates the representer directly by minimizing

$$\mathbb{E}[m(W,\gamma)] = \mathbb{E}[\alpha_0(X)\gamma(X)]$$2

or its weighted analogue in generalized regressions, instead of relying on an explicit closed-form formula for $$\mathbb{E}[m(W,\gamma)] = \mathbb{E}[\alpha_0(X)\gamma(X)]$$3; this is the original Riesz regression perspective (Chernozhukov et al., 2021).

2. Generalized Riesz regression under Bregman divergence

Generalized Riesz regression extends squared-loss Riesz regression to a Bregman-divergence family. Let $$\mathbb{E}[m(W,\gamma)] = \mathbb{E}[\alpha_0(X)\gamma(X)]$$4 be a strictly convex differentiable scalar generator. The pointwise divergence is

$$\mathbb{E}[m(W,\gamma)] = \mathbb{E}[\alpha_0(X)\gamma(X)]$$5

and averaging over $$\mathbb{E}[m(W,\gamma)] = \mathbb{E}[\alpha_0(X)\gamma(X)]$$6 gives the population criterion. Because $$\mathbb{E}[m(W,\gamma)] = \mathbb{E}[\alpha_0(X)\gamma(X)]$$7 is unknown, the identity

$$\mathbb{E}[m(W,\gamma)] = \mathbb{E}[\alpha_0(X)\gamma(X)]$$8

eliminates $$\mathbb{E}[m(W,\gamma)] = \mathbb{E}[\alpha_0(X)\gamma(X)]$$9 and yields the population objective

$\gamma$0

The empirical regularized estimator solves

$\gamma$1

This framework unifies Riesz regression, tailored loss minimization, covariate balancing, and density-ratio fitting (Kato, 12 Jan 2026).

The squared-loss special case is obtained with $\gamma$2, which reduces the objective to

$\gamma$3

This is exactly Riesz regression and coincides with LSIF in density-ratio estimation. KL-type generators produce different primal and dual forms. For the unnormalized KL (UKL) generator, the primal corresponds to tailored loss minimization and the dual corresponds to entropy balancing weights in ATE under specific model specifications. For the binary KL (BKL) generator, the method recovers logistic-likelihood based propensity modeling. In the density-ratio view, $\gamma$4 recovers LSIF, $\gamma$5 recovers KLIEP, and $\gamma$6 recovers classification-based density ratio estimation (Kato, 12 Jan 2026, Kato, 6 Nov 2025).

A generalized linear parameterization is often used:

$\gamma$7

with basis functions $\gamma$8 and a link $\gamma$9 chosen so that $\alpha_0$0 becomes linear in $\alpha_0$1. This choice is the basis of the automatic balancing results and of the dual formulation. A software implementation of this design principle appears in the Python package "genriesz" (Kato, 19 Feb 2026).

3. Duality, automatic balancing, and the density-ratio viewpoint

A central structural result is that generalized Riesz regression is dual to covariate balancing when $\alpha_0$2 is linear in the basis. In that case the dual program imposes moment conditions of the form

$\alpha_0$3

The primal therefore minimizes a Bregman risk, while the dual enforces the covariate moments required by $\alpha_0$4. This duality is described as automatic covariate balancing (Kato, 12 Jan 2026).

Under squared loss with linear link, the dual corresponds to stable balancing weights. In the ATE case with $\alpha_0$5, the balance equations become

$\alpha_0$6

Under UKL with log or exponential link, the dual corresponds to entropy balancing weights. In density-ratio estimation with $\alpha_0$7, the normalization constraints are Silverman’s trick or KLIEP (Kato, 12 Jan 2026).

The density-ratio interpretation is especially important in semi-supervised settings. In the ATE case, the representer can be decomposed through density ratios

$\alpha_0$8

so that

$\alpha_0$9

The squared-loss Bregman objective then decomposes into two LSIF objectives. More generally, when $(\widehat\gamma,\widehat\alpha)$0 is a linear combination of density ratios, Riesz regression is exactly direct density-ratio estimation, which imports LSIF, KLIEP, non-negative Bregman corrections, and telescoping ratios into the Riesz-estimation problem (Kato, 6 Nov 2025).

This equivalence also clarifies the role of unlabeled covariates. The terms corresponding to expectations under the marginal law of $(\widehat\gamma,\widehat\alpha)$1 can be estimated from $(\widehat\gamma,\widehat\alpha)$2-only data, while the terms involving the joint law of $(\widehat\gamma,\widehat\alpha)$3 or $(\widehat\gamma,\widehat\alpha)$4 still require the labeled sample. A plausible implication is that the semi-supervised gain is largest when the representer is close to a density-ratio problem, such as covariate shift, APE, or ATE with abundant covariate information (Kato, 6 Nov 2025, Chernozhukov et al., 2021).

4. Semi-supervised constructions and algorithmic variants

Semi-supervised generalized Riesz regression uses the fact that many Bregman-Riesz objectives split naturally into labeled and unlabeled components. A generic template is

$(\widehat\gamma,\widehat\alpha)$5

where the unsupervised term may be a density-ratio component or a balancing component (Kato, 12 Jan 2026).

A basic example is semi-supervised covariate shift, with labeled source $(\widehat\gamma,\widehat\alpha)$6 and unlabeled target $(\widehat\gamma,\widehat\alpha)$7. The population objective is

$(\widehat\gamma,\widehat\alpha)$8

and the empirical objective is

$(\widehat\gamma,\widehat\alpha)$9

For squared loss this becomes LSIF; for UKL it becomes KLIEP under normalization. The associated AIPW estimator is

$W=(X,Y)$0

and the stated benefit is that unlabeled target covariates directly improve $W=(X,Y)$1 estimation through $W=(X,Y)$2, reducing variance and stabilizing weights (Kato, 12 Jan 2026).

For treatment-effect estimation with unlabeled covariates, one line of work studies one-sample and two-sample semi-supervised regimes. In the one-sample setting, the data contain an observation indicator $W=(X,Y)$3, and unlabeled observations contribute through the $W=(X,Y)$4-only expectation in the Bregman objective

$W=(X,Y)$5

so that unlabeled covariates sharpen estimation of the representer or of the corresponding propensity component (Kato, 11 Nov 2025).

A distinct semi-supervised construction is positive-unlabeled Riesz estimation for ATE. Here some labeled $W=(X,Y)$6 positives and unlabeled covariates $W=(X,Y)$7 with mixed class are available, together with a class prior $W=(X,Y)$8. The PU generator is

$W=(X,Y)$9

with derivative

$P_0$0

An EM-like scheme alternates an E-step computing posteriors $P_0$1 and an M-step updating $P_0$2 by minimizing a PU-like Bregman objective; then

$P_0$3

is used in AIPW (Kato, 12 Jan 2026).

A further extension replaces direct ratio fitting by score matching. Time-score or diffusion-inspired constructions estimate $P_0$4 along bridges between endpoint distributions using only $P_0$5-only data. For ATE and APE they use endpoint identities; for AME they can use denoising score matching without $P_0$6. The core identity is

$P_0$7

which converts global density-ratio estimation into infinitesimal classification. This approach is motivated by overfitting pathologies in flexible direct density-ratio estimation and is presented as particularly well-suited to semi-supervised APE and ratio tasks (Kato, 23 Dec 2025).

5. Estimation workflows, efficiency bounds, and convergence theory

The standard workflow combines nuisance estimation of $P_0$8 and $P_0$9 with cross-fitting. A representative sequence is: choose a loss $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$00 and a link $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$01; specify a model class for $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$02 such as linear bases, RKHS, or neural network; minimize the empirical Bregman-Riesz objective with regularization; fit $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$03; and solve the orthogonal estimating equation

$\theta_0 = \mathbb{E}[m(W,\gamma_0)]$04

Diagnostics include dual balancing residuals

$\theta_0 = \mathbb{E}[m(W,\gamma_0)]$05

inspection of tails of $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$06, and train-versus-test loss, especially for KL-type objectives (Kato, 12 Jan 2026).

In treatment-effect settings with auxiliary unlabeled covariates, the efficiency-theoretic contribution is explicit. One study develops efficiency bounds and efficient estimators in one-sample and two-sample semi-supervised designs and states that auxiliary covariates can lower the efficiency bound and yield an asymptotic variance smaller than that without such covariates (Kato, 11 Nov 2025). In the two-sample prediction-powered formulation, with labeled sample proportion $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$07 and evaluation density $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$08, the semiparametric efficiency bound is

$\theta_0 = \mathbb{E}[m(W,\gamma_0)]$09

The first term is a conditional-noise term, while the second and third terms are regressor-averaging terms; the latter are the terms reduced by unlabeled regressors (Kato, 11 Jun 2026).

The same paper defines two efficient estimators. The estimating-equation DML-PPCI estimator is

$\theta_0 = \mathbb{E}[m(W,\gamma_0)]$10

and the TMLE-DML-PPCI estimator updates

$\theta_0 = \mathbb{E}[m(W,\gamma_0)]$11

with

$\theta_0 = \mathbb{E}[m(W,\gamma_0)]$12

Both are asymptotically linear with the efficient influence function and attain the two-sample efficiency bound (Kato, 11 Jun 2026).

The key nuisance-rate condition remains the familiar product-rate condition

$\theta_0 = \mathbb{E}[m(W,\gamma_0)]$13

under cross-fitting or Donsker conditions, which yields

$\theta_0 = \mathbb{E}[m(W,\gamma_0)]$14

For generalized regressions and for automatic DML more broadly, the same orthogonal form extends to weighted Riesz regression and generalized residuals (Kato, 12 Jan 2026, Chernozhukov et al., 2021).

Convergence guarantees are available for several representer classes. In the unified Bregman framework, under bounded $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$15, strong convexity of $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$16, Lipschitz $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$17, and bracketing entropy conditions, the RKHS estimator satisfies

$\theta_0 = \mathbb{E}[m(W,\gamma_0)]$18

while a neural-network estimator satisfies

$\theta_0 = \mathbb{E}[m(W,\gamma_0)]$19

In the semi-supervised prediction-powered setting, a deep ReLU sieve bound takes the form

$\theta_0 = \mathbb{E}[m(W,\gamma_0)]$20

with $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$21 (Kato, 12 Jan 2026, Kato, 11 Jun 2026).

6. Relations, misconceptions, limitations, and implementations

Semi-Supervised Generalized Riesz Regression sits at the intersection of debiased machine learning, covariate balancing, density-ratio estimation, score matching, and semi-supervised efficiency theory. One comparison stated explicitly is that R-learner and DR-learner primarily focus on $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$22, whereas generalized Riesz regression focuses on $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$23 directly via empirical risk minimization and its dual balance conditions (Kato, 12 Jan 2026). Another is that adversarial estimation provides a different route to Riesz estimation, based on a min-max moment-matching problem over functions of $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$24; because that objective does not require $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$25, it is naturally compatible with semi-supervised use of unlabeled covariates (Chernozhukov et al., 2020).

A recurring misconception is that unlabeled data automatically improve every nuisance. The supplied results are more specific. Unlabeled $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$26 directly improves the estimation of $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$27 when the Riesz objective contains $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$28-only terms such as $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$29, density-ratio expectations, or evaluation-function moments, but $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$30 remains an outcome regression and does not automatically improve unless additional semi-supervised regression techniques are introduced. In the automatic DML synthesis, this point is stated directly for both the conditional-mean and generalized-regression cases (Chernozhukov et al., 2021).

Another misconception is that any flexible loss is benign. The literature cited here notes train-loss hacking in KL-type objectives, density-chasm or support-chasm problems in direct density-ratio estimation, and instability from extreme weights or deconvolution in high dimension. Recommended remedies include robust BP losses, non-negative Bregman corrections, telescoping ratios, score matching, weight clipping, regularization, and cross-fitting (Kato, 12 Jan 2026, Kato, 6 Nov 2025, Kato, 23 Dec 2025). Fully unsupervised covariate-only learning without endpoint tags is also not covered for ATE or APE in the score-matching formulation; that approach still needs endpoint identities such as treatment labels or policy indices (Kato, 23 Dec 2025).

Identification and regularity conditions remain stringent. For ATE, the assumptions include unconfoundedness and positivity; for covariate shift, common support; for AME and APE, integration-by-parts regularity and controlled tails. In prediction-powered and two-sample formulations, one also needs dominance conditions such as $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$31 and bounded density ratios; when $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$32, the parameter-defining mixture weight $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$33 should not be chosen purely to optimize asymptotic variance unless the estimand is invariant to $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$34 (Kato, 11 Nov 2025, Kato, 11 Jun 2026).

Software infrastructure has begun to reflect the generalized Riesz perspective. The package "genriesz" implements automatic DML and generalized Riesz regression by minimizing empirical Bregman divergences, constructs a compatible link through automatic regressor balancing, and returns RA, RW, ARW, and TMLE-style estimators with cross-fitting, confidence intervals, and $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$35-values. The implementation supports bases such as polynomial features, RKHS approximations, random forest leaf encodings, neural embeddings, and a nearest-neighbor catchment basis, although the supplied description also notes that the core API does not natively expose separate unlabeled inputs (Kato, 19 Feb 2026).

Taken together, these developments define Semi-Supervised Generalized Riesz Regression as a unified method for estimating the score-relevant representer in orthogonal semiparametric estimation by exploiting unlabeled covariates wherever the target functional depends on marginal or mixture structure of $\theta_0 = \mathbb{E}[m(W,\gamma_0)]$36. The framework encompasses squared-loss Riesz regression, KL-type calibrated fitting, density-ratio estimation, positive-unlabeled learning, and score-matching variants, while preserving the efficient influence-function logic of debiased machine learning (Kato, 12 Jan 2026, Kato, 11 Jun 2026).