Semiparametric Density Ratio Model

• Semiparametric density ratio models define multiple distributions through an exponential tilting function relative to a baseline density.
• The estimation process uses constrained empirical likelihood maximization to pool data and enforce density ratio constraints efficiently.
• These models offer efficiency gains in regression, quantile estimation, and diagnostic testing compared to fully nonparametric alternatives.

A semiparametric density ratio model specifies that multiple probability distributions within a set are mutually absolutely continuous, with all but one related to a reference “baseline” distribution through an exponential tilting function. This framework serves as an intermediate between parametric and fully nonparametric models, allowing for the parametric modeling of density ratios (through a finite-dimensional tilting parameter) while leaving the baseline or reference density unspecified. Broadly, this modeling approach is motivated by applications demanding the pooling of information from multiple samples or populations (e.g., case-control studies, semi-supervised learning, diagnostic test analysis, or semiparametric two-sample tests), where it offers efficiency gains by exploiting structural relationships across populations.

1. Model Definition and Theoretical Structure

Let $m$ distinct populations have densities %%%%1%%%% with respect to a common dominating measure on $\mathbb{R}^p$. In semiparametric density ratio models, these densities are linked via

$$g_i(\mathbf{x}) = w(\mathbf{x}; \theta_i) \, g_m(\mathbf{x}), \quad i = 1, ..., m-1,$$

$$\text{or, equivalently,} \quad \frac{g_i(\mathbf{x})}{g_m(\mathbf{x})} = w(\mathbf{x}; \theta_i)$$

where $g_m(\mathbf{x})$ is the reference (“baseline”) density, and $w(\mathbf{x}; \theta_i)$ is a parametrically specified tilt function. A canonical and widely adopted choice is the exponential tilt,

$$w(\mathbf{x}; \theta_i) = \exp(\alpha_i + \beta_i^\top \mathbf{x}),$$

with tilting parameters $\theta_i = (\alpha_i, \beta_i)$. The model is termed “semiparametric” because the functional form of $g_m$ is left unrestricted (infinite-dimensional), whereas the tilting parameters are finite-dimensional and parametric.

This construction directly generalizes to multi-sample settings (e.g., $q$ cases, 1 control) and to higher-dimensional contexts where $\mathbf{x}$ may include both covariates and dependent variables, enabling joint modeling and efficient use of information from different sources.

2. Estimation: Empirical Likelihood and Constraints

Estimation under the semiparametric density ratio model is typically achieved via constrained empirical likelihood maximization. Suppose $n_i$ independent observations $\mathbf{x}_{ij}$ from population $i$ ($i = 1,...,m$). Assign probability masses $p_{ij}$ to each observed point, and maximize the empirical likelihood

$$L(\theta, G_m) = \prod_{i=1}^{m} \prod_{j=1}^{n_i} \left\{ p_{ij} \cdot [w(\mathbf{x}_{ij}; \theta_i)]^{I(i \neq m)} \right\}$$

subject to

• $p_{ij} \geq 0$, $\sum_{i=1}^m \sum_{j=1}^{n_i} p_{ij} = 1$,
• $$\sum_{i=1}^m \sum_{j=1}^{n_i} p_{ij} \, w(\mathbf{x}_{ij}; \theta_k) = 1$$ for each $k=1,...,m-1$.

The resulting estimator for the reference cumulative distribution is

$$\widehat{G}_m(\mathbf{x}) = \sum_{i=1}^m \sum_{j=1}^{n_i} \widehat{p}_{ij} \, I(\mathbf{x}_{ij} \leq \mathbf{x}),$$

where

$$\widehat{p}_{ij} = \frac{1}{n} \left\{ 1 + \sum_{k=1}^{m-1} \widehat{\mu}_k [w(\mathbf{x}_{ij}; \widehat{\theta}_k) - 1] \right\}^{-1}$$

and $\widehat{\mu}_k$ are Lagrange multipliers enforcing the normalization constraints.

This approach leverages all data collectively, “borrowing strength” from the reference sample and enforcing the density ratio structure through the constraints.

3. Application to Regression and Multivariate Analysis

In regression contexts with random covariates, the joint density of covariates and response is modeled using the semiparametric density ratio model. Kernel density estimates constructed from the “pooled” sample—together with the density ratio tilts—enable efficient estimation of conditional expectations of the form $E[y \mid \mathbf{x}]$. Specifically, for discrete observed $y_i$,

$$\widehat{E}[y \mid x_1, ..., x_{p-1}] = \frac{\sum_{i: y_i \text{ observed}} y_i \, \widehat{g}(x_1, ..., x_{p-1}, y_i)}{\sum_{i: y_i \text{ observed}} \widehat{g}(x_1, ..., x_{p-1}, y_i)}$$

where $\widehat{g}$ is the estimated joint density. This enables semiparametric estimation of regression surfaces, as implemented in the analysis of testicular germ cell tumor data (Voulgaraki et al., 2010), where effects of height and age on weight are quantified for cases and controls, and nonlinear conditional expectation relationships are directly recovered.

Semiparametric density ratio models are also employed in quantile estimation, providing Bahadur representations for quantile estimators derived from empirical likelihood under the density ratio model (Chen et al., 2013), and in the efficient pooling of information for estimation of linear functionals (moments, ratios, coefficients of variation, Gini indices) across semicontinuous populations (Yuan et al., 2020, Yuan et al., 2021).

4. Efficiency, Diagnostic Techniques, and Model Assessment

A principal advantage of the semiparametric density ratio model is efficiency gain over fully nonparametric estimators. Pooling data through a common reference and exploiting the density ratio constraints yields kernel density estimators with lower mean integrated squared error (MISE) than those based on a single sample. Theoretical analysis demonstrates that “borrowing strength” significantly reduces estimator variance, particularly when the reference sample size is large (Zhang et al., 2023).

Model checking is essential:

• Graphical diagnostics: Plotting $\widehat{G}_i$ (the fitted CDFs under the density ratio model) against empirical CDFs at selected points; alignment near the identity line indicates adequacy.
• Goodness-of-fit measures: Summary metrics such as $R^2_{\alpha,k}$ and $R^2_3$ (which compare fitted and empirical CDFs within confidence intervals) offer quantitative assessment. High $R^2_{\alpha,k}$ values indicate model appropriateness for the data.
• Residual analysis: Evaluating the difference between observed and fitted conditional expectation values, checking for centering around zero, to confirm the regression structure’s adequacy.

In simulation studies and applied analyses, these diagnostics distinguish cases where the density ratio model is appropriate (e.g., under normal or log-linear distributions) from those where it is not (e.g., Cauchy or uniform situations), aiding in model selection and validation.

5. Comparative Advantages over Other Approaches

Relative to:

• Multiple regression and GAM: The semiparametric density ratio model directly estimates joint densities and associated conditional expectations, offering increased flexibility and accommodating nonlinearity without requiring additive or normality assumptions.
• Nonparametric kernel regression: The joint modeling and “pooling” in the semiparametric density ratio approach are more efficient, particularly in multivariate settings; optimal bandwidths can be derived, and high-dimensional regression is more tractable.
• Empirical quantile estimation: Pooling via the density ratio model leads to quantile estimators that are root-$n$ consistent, have shorter confidence intervals, and maintain better coverage under mild model misspecification (Chen et al., 2013).
• Classical two-sample tests: For divergence and homogeneity testing, f-divergence estimators built upon semiparametric density ratio models achieve minimum asymptotic variance, match empirical likelihood-based tests in power, and handle model misspecification more robustly (Kanamori et al., 2010).

6. Practical Implementation and Extensions

Implementation generally requires:

• Optimization under nonlinear constraints, usually via Lagrange multipliers and empirical likelihood maximization.
• Selection of functional forms for basis vectors $q(x)$ or tilt functions, either via domain knowledge or data-driven procedures such as functional principal component analysis when structural cues are weak (Zhang et al., 2021).
• Computation of density estimates (for regression or estimation of joint probabilities), often using pooled kernel density estimators and profile likelihood approaches.

Method extensions include:

• Incorporation of auxiliary information expressed as estimating equations, further improving estimator efficiency when valid external summaries or constraints are available (Yuan et al., 2021).
• Handling populations with semicontinuous distributions (point mass at zero and continuous positive components) via integrated modeling for both parts (Yuan et al., 2020, Yuan et al., 2021).
• Generalization to semiparametric frameworks with nonparametric components in reproducing kernel Hilbert spaces, enabling rich, penalized-likelihood–based density modeling (Shi et al., 2019).

Recent theoretical results establish optimality properties: when combining samples from multiple populations of disparate sizes, estimation of the distribution function (or functionals) for a small sample can, through the density ratio model, achieve efficiency equivalent to the best possible under a correct parametric model if the reference sample is large (Zhang et al., 2023).

7. Applications and Impact

Semiparametric density ratio models have been applied to:

• Multivariate regression with random covariates (e.g., effects of clinical covariates on patient outcomes in oncology (Voulgaraki et al., 2010))
• Quantile estimation for forestry and material strength data (Chen et al., 2013)
• Diagnostic marker analysis (e.g., estimation of the Youden index and optimal cutoff in biomarker studies (Yuan et al., 2020))
• Socioeconomic inequality measurement (e.g., Gini index estimation for income data (Yuan et al., 2021))
• Modern two-sample testing procedures and divergence estimation (Kanamori et al., 2010)
• Semi-supervised learning frameworks where density ratio estimation is used to minimize variance under model misspecification (Kawakita et al., 2012)

The estimators and associated inference procedures demonstrate superior efficiency and interpretability compared with classical nonparametric procedures, while maintaining flexibility and robustness not present in parametric alternatives. Empirical findings consistently show improved mean squared error, shorter and valid confidence intervals, and enhanced statistical power in both simulated and real data scenarios.

---

This article provides a comprehensive summary of the definition, theoretical foundations, estimation, diagnostic evaluation, advantages, implementation considerations, and the range of applications for semiparametric density ratio models, as developed and applied across the referenced literature.