--- title: Weighted Empirical Distribution Methods url: https://www.emergentmind.com/topics/weighted-empirical-distribution type: topic --- # Weighted Empirical Distribution Methods A weighted empirical distribution generalizes the classical empirical distribution by assigning non-uniform, possibly data-dependent, and potentially predetermined weights to individual observations. This concept is crucial in contemporary statistical inference, nonparametric theory, large-scale resampling, importance sampling, transfer learning, optimal risk minimization under dataset shift, and robust and distributionally-constrained modeling. The weighted empirical distribution appears both as an explicit object of inference—enabling corrections for heterogeneity or bias—and as an implicit tool for constructing statistics, likelihoods, and loss functions in high-dimensional and complex data scenarios. ## 1. Definition and Fundamental Properties Given independent observations $X_1, \dotsc, X_n$ (not necessarily identically distributed) and nonnegative weights $w_1, \dotsc, w_n$ with normalization $\sum_{i=1}^n w_i = n$ (or $\sum_{i=1}^n w_i = 1$ in some conventions), the weighted empirical distribution $\hat{F}$ is \[ \hat{F}(x) = \frac{1}{n} \sum_{i=1}^n w_i \, \mathbb{I}(X_i < x) \] When $w_i \equiv 1$, $\hat{F}$ reduces to the classical empirical cumulative distribution function (ECDF). In general, $\hat{F}$ is a discrete probability measure supported on the sample points but with non-uniform masses. The expectation of $\hat{F}(x)$ under sampling is \[ \mathbb{E}\,\hat{F}(x) = \frac{1}{n} \sum_{i=1}^n w_i F_i(x) \] where $F_i$ is the true distribution function of $X_i$. Weighted empirical distributions facilitate correct representation of data under non-identically distributed samples, enable efficiency improvements through deterministic or data-adaptive reweighting, and underpin modern procedures for robust modeling and inference in heterogeneous data environments [1002.4338]. ## 2. Edgeworth, Cornish–Fisher, and Higher Order Expansions Weighted empirical distributions enable not only classical Central Limit Theorem (CLT) results for plug-in estimators $T(\hat{F})$, but also refined, higher-order distributional approximations. Any smooth functional $T(\cdot)$—e.g., mean, variance, quantiles, or functionals arising in statistical estimation—can have its sampling distribution expanded to third order as \[ P(Y_n < x) \approx \Phi(x) - n^{-1/2}h_1(x)\varphi(x) - n^{-1} h_2(x)\varphi(x) - \cdots \] where $Y_n = n^{1/2}[T(\hat{F}) - T(F)] / a_{21}^{1/2}$, $\Phi$ and $\varphi$ are the standard normal CDF and density, and $h_1(x), h_2(x),\dots$ are Hermite-polynomial-based correction terms determined by cumulants involving von Mises derivatives and the weight structure. These expansions provide accurate quantile inference and coverage for confidence intervals beyond first-order CLT normality, correcting both bias and variance due to heterogeneity and weighting, and yielding meaningful improvements in classical statistical inferential tasks [1002.4338]. ## 3. Cumulant Expansions and von Mises Derivatives To obtain higher order expansions for $T(\hat{F})$, the paper employs von Mises–Taylor expansions for functionals defined not only on probability measures but on signed measures (of total measure $1$). The general expansion is \[ T(G) - T(F) = \sum_{r=1}^\infty \frac{1}{r!} T_{(r)}(F)\bigl( G-F \bigr)^r \] where $T_{(r)}$ is the $r$-th order von Mises derivative, defined so $\int T_{(r)}(x_1, \ldots, x_r) \, dF(x_1) = 0$ for all $r \geq 1$. Cumulant expansions for $T(\hat{F})$ take the form \[ \kappa_r = \sum_{j=r-1}^\infty a_{rj}\, n^{-j/2} \] with explicit, weight-dependent terms; e.g., the coefficients involve moments and combinations such as \[ a_{21} = \{12\},\qquad a_{32} = \{13\} + 3\{1,2,12\},\qquad \text{etc.} \] Here, $\{12\}$ denotes an expectation of a product of first and second von Mises derivatives, averaged against the weight profile $w_1, ..., w_n$. The $r$-th order moment structure further involves cross-product terms coupling distinct sample points, reflecting both heterogeneity and weighting [1002.4338]. ## 4. Practical Applications: Estimators and Edge Cases ### a) Sample Mean and Variance For the mean, the influence function is $x-p$ and cumulant coefficients reduce to scaled moments of the (mean) distribution convolved with the $r$-th moment of the weights. For the sample variance $T(F) = \mu_2 - (\mu_1)^2$, higher order derivatives and mixed moments appear, essential under non-identical distributions; e.g., \[ a_{21} = \{12\},\qquad a_{32} = M_3 - 3 M_{24} + 2M_{222} - 6(M_3 - M_{22})^2 \] with $M_k$ denoting empirical moments. ### b) Studentized Mean, Coefficient of Variation The framework extends directly to plug-in functionals such as the Studentized mean and sample coefficient of variation, where the higher order corrections account for weighted, heterogeneous sampling. In each case, practical inference—confidence intervals, hypothesis tests—benefits from improved asymptotics reflecting both the weights and the varying sampling distributions [1002.4338]. ## 5. Edgeworth–Cornish–Fisher Expansions: Quantiles and Distributional Approximations The explicit cumulant expansion allows the use of Edgeworth–Cornish–Fisher (ECF) formulas to correct the estimated distribution and quantiles for $T(\hat{F})$: \[ P(Y_n < x) \sim \Phi(x) + n^{-1/2} P_1(x) + n^{-1} P_2(x) + \text{higher terms} \] where $P_1(x), P_2(x), \dots$ are determined via Hermite expansions in terms of the standardized cumulant coefficients. This sequence yields more accurate p-values and critical values for functionals under both heterogeneity and weighting, and directly extends classical asymptotic inference. ## 6. Methodological Extensions: Non-i.i.d. Data and Signed Measures A crucial methodological development is the extension of von Mises functional derivatives to signed measures of total measure $1$—a necessary move since the weighted empirical distribution may involve negative or non-uniform weighting (e.g., in leave-one-out or importance resampling). The normalization $\int T_{(r)}(x_1,\ldots,x_r)\,dF(x_1) = 0$ for all $r \ge 1$ ensures the expansion is valid in generality. This approach is essential for robust inference under - Non-identical distributions (distinct $F_i$ for each $X_i$), - Preassigned or data-adaptive weighting (e.g., regression, survey sampling, sandwich estimation), - Settings where classical plug-in theory fails. Thus, the framework generalizes both the sample measures themselves and the associated inferential expansions [1002.4338]. ## 7. Implications for Nonparametric and Robust Inference The explicit third order expansions and cumulant-based inferential methodology provide a substantial improvement over traditional CLT-level inference. The ability to model and correct for both heterogeneity and sampling design is fundamental for - Nonparametric inference with complex surveys or regression residuals, - Robust estimation, model validation, and hypothesis testing in high-variance environments, - Bayesian and likelihood-based procedures requiring accurate quantile or tail approximations. By integrating weighting into both the empirical measure and its higher order derivative structure, the approach provides a rigorous, systematic framework for precision inference in real-world, heterogeneous data scenarios. --- **Summary Table: Core Components of Weighted Empirical Distribution Results** | Component | Mathematical Expression / Role | Key Implications | |---------------------------|------------------------------------------------------------|--------------------------| | Weighted empirical CDF | $\hat{F}(x) = \frac{1}{n} \sum_i w_i\, \mathbb{I}(X_i