Target Linearity in Distribution-Shift Regression

• Target linearity is defined as the invariant linear conditional mean relationship (Y = β₀ + Xᵀβ) that holds across source and target environments under distribution shift.
• It leverages known target marginal information to impose algebraic constraints on the intercept, slope, and noise variance, differentiating it from traditional covariate shift models.
• Practical methods such as hybrid loss estimation, constrained moment matching, and two-stage calibration are introduced to improve performance across varying signal-to-noise regimes.

Target linearity denotes the assumption that, under distribution shift, the conditional mean relationship remains linear and stable between source and target environments. In the multivariate linear regression setting studied in "Target-Aware Linear Regression Under Distribution Shift" (Hou et al., 22 Jun 2026), this assumption is written as $E[Y \mid X] = \beta_0 + X^\top \beta$, with target-side information entering through known or specified marginals of $X$ and $Y$. The resulting problem is not merely linear regression under covariate shift; it is regression with deployment-time moment information, where target marginals impose algebraic constraints on the admissible intercept, slope, and noise variance. This makes target linearity a distribution-shift concept tied to stable conditional structure, distinct from other uses of linearity in testing, electronics, graph encoding, group theory, or type systems.

1. Definition and statistical setting

The basic model is

$$Y = \beta_0 + X^\top \beta + \varepsilon,\qquad \varepsilon \sim N(0,\sigma_e^2),\qquad X \perp \varepsilon,$$

so that $E[Y \mid X] = \beta_0 + X^\top \beta$ (Hou et al., 22 Jun 2026). In this setting, the source or training distribution supplies observations $\{(X_i,Y_i)\}_{i=1}^n$, while the target or deployment distribution supplies marginals rather than paired samples. Specifically, the target-side information is

$$X \sim N(\mu_{k,x}, \Sigma_{k,x}),\qquad Y \sim N(\mu_{k,y}, \sigma_{k,y}^2),$$

subject to the stability constraints

$$\mu_{k,y} = \beta_0 + \mu_{k,x}^\top \beta,\qquad \sigma_{k,y}^2 = \beta^\top \Sigma_{k,x}\beta + \sigma_e^2$$

(Hou et al., 22 Jun 2026).

These constraints formalize what target linearity means operationally: the target marginals are compatible with the same linear conditional mean that generated the source data. The framework also admits a normalization step. Without loss of generality, one may whiten the target $X$-marginal so that $\mu_{k,x}=0$ and $X$0, absorbing a linear transform into $X$1 (Hou et al., 22 Jun 2026). This reparameterization simplifies both asymptotic formulas and estimator comparison.

A plausible implication is that target linearity is best viewed as a structural invariance assumption plus side information about the deployment distribution. It is therefore stronger than ordinary source-only linear modeling, but weaker than full target supervision.

2. Target constraints and the hybrid-loss benchmark

The benchmark target-aware estimator jointly uses source data and both target marginals through a hybrid loss (Hou et al., 22 Jun 2026). Writing $X$2,

$X$3

$X$4

with

$X$5

the hybrid loss is

$X$6

The tuning parameter $X$7 trades off source-data fidelity against enforcement of the target marginals (Hou et al., 22 Jun 2026).

This construction uses two deployment constraints simultaneously: one on the target mean of $X$8, and one on its target variance. The first is affine in $X$9, while the second is nonlinear because it couples $Y$0 and $Y$1 through $Y$2. For this reason, direct computation of the hybrid estimator requires solving a coupled nonlinear optimization, which the paper identifies as expensive at scale (Hou et al., 22 Jun 2026).

The asymptotic theory is expressed through

$Y$3

$Y$4

where $Y$5 and $Y$6 (Hou et al., 22 Jun 2026). Under Theorem 3.1,

$Y$7

and Corollary 3.2 gives

$Y$8

with $Y$9 and $$Y = \beta_0 + X^\top \beta + \varepsilon,\qquad \varepsilon \sim N(0,\sigma_e^2),\qquad X \perp \varepsilon,$$0 (Hou et al., 22 Jun 2026).

3. Tractable alternatives: moment matching and two-stage calibration

Because the hybrid estimator is the benchmark but not the cheapest procedure computationally, the paper develops two tractable alternatives (Hou et al., 22 Jun 2026).

The first is the constrained moment-matching estimator. Rather than softly penalizing target mean and variance discrepancies, it enforces them exactly:

$$Y = \beta_0 + X^\top \beta + \varepsilon,\qquad \varepsilon \sim N(0,\sigma_e^2),\qquad X \perp \varepsilon,$$1

After eliminating $$Y = \beta_0 + X^\top \beta + \varepsilon,\qquad \varepsilon \sim N(0,\sigma_e^2),\qquad X \perp \varepsilon,$$2 using the mean constraint and centering the data as $$Y = \beta_0 + X^\top \beta + \varepsilon,\qquad \varepsilon \sim N(0,\sigma_e^2),\qquad X \perp \varepsilon,$$3 and $$Y = \beta_0 + X^\top \beta + \varepsilon,\qquad \varepsilon \sim N(0,\sigma_e^2),\qquad X \perp \varepsilon,$$4, define

$$Y = \beta_0 + X^\top \beta + \varepsilon,\qquad \varepsilon \sim N(0,\sigma_e^2),\qquad X \perp \varepsilon,$$5

$$Y = \beta_0 + X^\top \beta + \varepsilon,\qquad \varepsilon \sim N(0,\sigma_e^2),\qquad X \perp \varepsilon,$$6

The constraint becomes

$$Y = \beta_0 + X^\top \beta + \varepsilon,\qquad \varepsilon \sim N(0,\sigma_e^2),\qquad X \perp \varepsilon,$$7

(Hou et al., 22 Jun 2026). If $$Y = \beta_0 + X^\top \beta + \varepsilon,\qquad \varepsilon \sim N(0,\sigma_e^2),\qquad X \perp \varepsilon,$$8 has real solutions, the zero-loss set is an affine ellipsoid, and the chosen estimator is

$$Y = \beta_0 + X^\top \beta + \varepsilon,\qquad \varepsilon \sim N(0,\sigma_e^2),\qquad X \perp \varepsilon,$$9

Proposition 3.3 shows that there is a scalar $E[Y \mid X] = \beta_0 + X^\top \beta$0 such that

$E[Y \mid X] = \beta_0 + X^\top \beta$1

and

$E[Y \mid X] = \beta_0 + X^\top \beta$2

(Hou et al., 22 Jun 2026).

The second alternative is a two-stage calibration estimator. Stage 1 is ordinary least squares:

$E[Y \mid X] = \beta_0 + X^\top \beta$3

Stage 2 uses an independent test set, or assumes one, whose empirical mean and covariance exactly match $E[Y \mid X] = \beta_0 + X^\top \beta$4. Raw predictions are formed as $E[Y \mid X] = \beta_0 + X^\top \beta$5, and then rescaled so that $E[Y \mid X] = \beta_0 + X^\top \beta$6 has mean $E[Y \mid X] = \beta_0 + X^\top \beta$7 and variance $E[Y \mid X] = \beta_0 + X^\top \beta$8:

$E[Y \mid X] = \beta_0 + X^\top \beta$9

The calibrated slope is

$\{(X_i,Y_i)\}_{i=1}^n$0

(Hou et al., 22 Jun 2026).

These two estimators represent two distinct uses of target information. Moment matching treats the target moments as hard feasibility conditions. Two-stage calibration treats OLS as the primary fit and then adjusts it to satisfy target mean-variance structure.

4. Asymptotic risk and improvement conditions

The paper gives closed-form asymptotic mean squared errors for the hybrid benchmark, constrained moment matching, and two-stage calibration, making their comparison explicit (Hou et al., 22 Jun 2026). The OLS baseline is

$\{(X_i,Y_i)\}_{i=1}^n$1

For moment matching, Corollary 3.5 states

$\{(X_i,Y_i)\}_{i=1}^n$2

where

$\{(X_i,Y_i)\}_{i=1}^n$3

(Hou et al., 22 Jun 2026). Improvement over OLS occurs if and only if

$\{(X_i,Y_i)\}_{i=1}^n$4

(Hou et al., 22 Jun 2026).

For the two-stage estimator, under $\{(X_i,Y_i)\}_{i=1}^n$5, $\{(X_i,Y_i)\}_{i=1}^n$6, and $\{(X_i,Y_i)\}_{i=1}^n$7, Theorem 3.6 gives

$\{(X_i,Y_i)\}_{i=1}^n$8

where

$\{(X_i,Y_i)\}_{i=1}^n$9

Its improvement condition over OLS is

$$X \sim N(\mu_{k,x}, \Sigma_{k,x}),\qquad Y \sim N(\mu_{k,y}, \sigma_{k,y}^2),$$0

(Hou et al., 22 Jun 2026).

In the high signal-to-noise regime, both moment matching and two-stage calibration nearly match the hybrid benchmark, and the paper emphasizes that the two-stage estimator does so at essentially no additional cost (Hou et al., 22 Jun 2026). In moderate-to-low signal-to-noise regimes, the variance constraint can become noisy or uninformative; under those conditions, both tractable alternatives can underperform OLS if their respective threshold conditions fail (Hou et al., 22 Jun 2026).

Estimator	Mechanism	Leading asymptotic slope-MSE
OLS	Source-only fit	$$X \sim N(\mu_{k,x}, \Sigma_{k,x}),\qquad Y \sim N(\mu_{k,y}, \sigma_{k,y}^2),$$1
Hybrid	Soft mean and variance penalties	$$X \sim N(\mu_{k,x}, \Sigma_{k,x}),\qquad Y \sim N(\mu_{k,y}, \sigma_{k,y}^2),$$2
Moment matching	Exact mean and variance constraints	$$X \sim N(\mu_{k,x}, \Sigma_{k,x}),\qquad Y \sim N(\mu_{k,y}, \sigma_{k,y}^2),$$3
Two-stage calibration	OLS followed by target calibration	$$X \sim N(\mu_{k,x}, \Sigma_{k,x}),\qquad Y \sim N(\mu_{k,y}, \sigma_{k,y}^2),$$4

A plausible implication is that target linearity contributes roughly one effective additional constraint on the slope vector through the target variance identity. The paper makes this intuition explicit by noting that known response marginals yield one scalar constraint $$X \sim N(\mu_{k,x}, \Sigma_{k,x}),\qquad Y \sim N(\mu_{k,y}, \sigma_{k,y}^2),$$5 involving $$X \sim N(\mu_{k,x}, \Sigma_{k,x}),\qquad Y \sim N(\mu_{k,y}, \sigma_{k,y}^2),$$6 unknown slope parameters, thereby shrinking sampling variability in approximately one effective degree of freedom (Hou et al., 22 Jun 2026).

5. Geometry, dimension, and estimator choice

The benefit of target-aware estimation depends not only on signal-to-noise ratio but also on the geometry of the source-target shift (Hou et al., 22 Jun 2026). Under anisotropic $$X \sim N(\mu_{k,x}, \Sigma_{k,x}),\qquad Y \sim N(\mu_{k,y}, \sigma_{k,y}^2),$$7 or nonzero mean shift $$X \sim N(\mu_{k,x}, \Sigma_{k,x}),\qquad Y \sim N(\mu_{k,y}, \sigma_{k,y}^2),$$8, the moment-matching estimator exploits the full geometry of $$X \sim N(\mu_{k,x}, \Sigma_{k,x}),\qquad Y \sim N(\mu_{k,y}, \sigma_{k,y}^2),$$9 and can outperform the two-stage estimator when the mean or covariance shifts lie in directions orthogonal to $$\mu_{k,y} = \beta_0 + \mu_{k,x}^\top \beta,\qquad \sigma_{k,y}^2 = \beta^\top \Sigma_{k,x}\beta + \sigma_e^2$$0 (Hou et al., 22 Jun 2026). By contrast, the two-stage estimator depends only on the projection along $$\mu_{k,y} = \beta_0 + \mu_{k,x}^\top \beta,\qquad \sigma_{k,y}^2 = \beta^\top \Sigma_{k,x}\beta + \sigma_e^2$$1, and is correspondingly less sensitive to those shifts (Hou et al., 22 Jun 2026).

The dimensional regime also matters. In very high dimensions, the paper states that the absolute effect of one variance constraint on $$\mu_{k,y} = \beta_0 + \mu_{k,x}^\top \beta,\qquad \sigma_{k,y}^2 = \beta^\top \Sigma_{k,x}\beta + \sigma_e^2$$2 parameters is $$\mu_{k,y} = \beta_0 + \mu_{k,x}^\top \beta,\qquad \sigma_{k,y}^2 = \beta^\top \Sigma_{k,x}\beta + \sigma_e^2$$3, so relative gains shrink as $$\mu_{k,y} = \beta_0 + \mu_{k,x}^\top \beta,\qquad \sigma_{k,y}^2 = \beta^\top \Sigma_{k,x}\beta + \sigma_e^2$$4 (Hou et al., 22 Jun 2026). This does not make target linearity irrelevant, but it limits its asymptotic leverage when the ambient parameter dimension is large.

The empirical guidance follows directly from the asymptotic analysis and Monte Carlo results. Figure 1 is described as a Pareto frontier in which the two-stage calibration estimator sits at essentially the same cost as OLS but achieves substantial gains whenever the base OLS model has reasonable $$\mu_{k,y} = \beta_0 + \mu_{k,x}^\top \beta,\qquad \sigma_{k,y}^2 = \beta^\top \Sigma_{k,x}\beta + \sigma_e^2$$5 (Hou et al., 22 Jun 2026). The hybrid estimator requires iterative optimization and yields only marginal improvements over two-stage calibration when signal-to-noise ratio is high (Hou et al., 22 Jun 2026). In low- to moderate-SNR regimes, hybrid or moment matching can produce noticeably lower MSE than two-stage calibration, at modest additional computational cost (Hou et al., 22 Jun 2026).

This suggests an estimator hierarchy rather than a single universally dominant procedure. When the linear model is already strong, calibration is sufficient. When the target constraints carry substantial information relative to source uncertainty, exact or softly penalized target-aware estimators become more attractive.

6. Relation to other meanings of linearity

The broader arXiv literature uses linearity in several technically distinct senses, and target linearity should not be conflated with them. In property testing, "On Optimal Testing of Linearity" (Arora et al., 2024) studies Boolean linearity over $$\mu_{k,y} = \beta_0 + \mu_{k,x}^\top \beta,\qquad \sigma_{k,y}^2 = \beta^\top \Sigma_{k,x}\beta + \sigma_e^2$$6 under online manipulations and sample-based models, and real-valued additivity and linearity over $$\mu_{k,y} = \beta_0 + \mu_{k,x}^\top \beta,\qquad \sigma_{k,y}^2 = \beta^\top \Sigma_{k,x}\beta + \sigma_e^2$$7 with $$\mu_{k,y} = \beta_0 + \mu_{k,x}^\top \beta,\qquad \sigma_{k,y}^2 = \beta^\top \Sigma_{k,x}\beta + \sigma_e^2$$8 query complexity. In semiparametric panel models, "Statistical Inference on Partially Linear Panel Model under Unobserved Linearity" (Liu et al., 2019) treats linearity as the problem of identifying which additive components are exactly linear, using a modified spline basis and folded-concave penalties. In functional regression, "Restricted Likelihood Ratio Tests for Linearity in Scalar-on-Function Regression" (McLean et al., 2013) frames linearity as the null hypothesis that the functional linear model is adequate inside a broader FGAM class.

In electronics and instrumentation, linearity commonly means local fidelity of a transfer function to an affine law. "Seeking Maximum Linearity of Transfer Functions" (Silva et al., 2016) identifies the most linear region of fixed width by least-squares scanning of subintervals, while "Negative Feedback, Linearity and Parameter Invariance in Linear Electronics" (Costa et al., 2016) analyzes how negative feedback attenuates nonlinear deviations and shows that the induced linearization in the reported experiments is relatively modest. These are notions of operating-range linearity, not distribution-shift invariance.

Other domains attach still different meanings to the word. "Linearity is Strictly More Powerful than Contiguity for Encoding Graphs" (Crespelle et al., 2018) defines graph linearity through $$\mu_{k,y} = \beta_0 + \mu_{k,x}^\top \beta,\qquad \sigma_{k,y}^2 = \beta^\top \Sigma_{k,x}\beta + \sigma_e^2$$9 linear orders whose intervals cover closed neighborhoods. "Linearity and Non-linearity of Groups of Polynomial Automorphisms of $X$0" (Mathieu, 2022) uses linearity in the representation-theoretic sense of faithful finite-dimensional embeddings into $X$1. Session-typed systems use linearity to mean single-use discipline for channels, even when that discipline is enforced without linear contexts (Sano et al., 2023) or via static type-level encodings (King et al., 2019).

This cross-disciplinary variation suggests that target linearity is a specialized term rather than a universal one. Within distribution-shift regression, however, it has a precise meaning: stability of the linear conditional mean together with exploitable target-marginal information (Hou et al., 22 Jun 2026).