Weighted Exponentiated Functions in Loss Analysis

• Weighted exponentiated functions are loss metrics defined by applying a weight and exponent to the absolute error, capturing prediction deviations.
• They bridge level-based losses (comparing direct magnitudes) and share-based losses (comparing normalized proportions), linking to GLMs and MLE.
• Their formulation supports both aggregate and instance-level analyses, enabling bias-variance trade-offs and robust, privacy-aware learning.

Level-based and share-based loss functions constitute two broad frameworks for quantifying prediction error in learning systems, especially neural networks and aggregate data regression. Each defines the error structure with respect to different normalization choices, data types, and target output domains. The paper of their mathematical and statistical foundations reveals strong links to generalized linear models (GLMs), maximum likelihood estimation (MLE), and fundamental statistical properties such as Bayes-optimality and robustness. Recent results establish their asymptotic equivalence for a broad family of losses, underscoring their interchangeable utility in large-sample regimes.

1. Formal Definitions and Output Domains

Level-based losses directly compare the “level” (i.e., magnitude) of each target value and its realization. For a vector of targets $t = (t_1, \dots, t_k)$ and corresponding realizations $y = (y_1, \dots, y_k)$, the level-based loss for unit $i$ is typically

$L_{\mathrm{level}}(t_i, y_i) = \ell(t_i, y_i)$

where $\ell(\cdot, \cdot)$ is a convex loss such as squared error or absolute error (Berzal, 7 Nov 2025, Coleman, 17 Nov 2025).

Share-based losses, in contrast, assess error between normalized proportions (“shares”) of each unit. With total sums $S_t = \sum_{i=1}^k t_i$, $S_y = \sum_{i=1}^k y_i$, the associated shares for unit $i$ are $x_i = t_i/S_t$, $s_i = y_i/S_y$, and the share-based loss is

$L_{\mathrm{share}}(t_i, y_i) = \ell(x_i, s_i)$

This formulation is essential in compositional data and when overall sum constraints are present (Coleman, 17 Nov 2025).

The output domain, activation function choice, and statistical assumptions differ for each approach:

Loss Type	Output Domain	Activation	Typical Statistical Model
Level-based	$\hat y \in \mathbb{R}$	Linear ($f(z)=z$)	Additive (Gaussian / Laplace) noise
Share-based	$\hat p \in \Delta^K$	Softmax or Sigmoid	Categorical / Multinomial outcomes

2. Statistical Justification and GLM Connections

Both classes of loss functions can be derived as negative log-likelihoods under suitable probabilistic observation models, linking them to GLMs (Berzal, 7 Nov 2025).

Level-Based Losses

• MSE ($(y - \hat y)^2$): arises from an additive Gaussian noise model; negative log-likelihood of the normal.
• MAE ($|y - \hat y|$): arises from additive Laplace (double-exponential) noise; negative log-likelihood of the Laplace.
• Both loss types utilize a linear identity link, mapping directly to the continuous outcome domain.

Share-Based Losses

• Categorical Cross-Entropy (CCE): For multi-class classification, assumes outcomes sampled from a multinomial distribution, using softmax activation.
• Binary Cross-Entropy (BCE): For binary classification, assumes Bernoulli outcomes, using sigmoid activation.
• Both are tied to the canonical logit or generalized-logit links of categorical GLMs.

The Bayes-optimal decision rule and output interpretation are dictated by the loss type:

• MSE: conditional mean, MAE: conditional median
• CCE/BCE: true posterior class probabilities

3. Canonical Losses and Their Mathematical Form

A selection of canonical forms for each class is as follows (Berzal, 7 Nov 2025):

Level-Based:

• Mean Squared Error (MSE):

$$L_{\rm MSE}(y, \hat y) = \frac{1}{n} \sum_{i=1}^n (y_i - \hat y_i)^2$$

• Mean Absolute Error (MAE):

$$L_{\rm MAE}(y, \hat y) = \frac{1}{n} \sum_{i=1}^n |y_i - \hat y_i|$$

Share-Based:

• Categorical Cross-Entropy (CCE):

$$L_{\rm CCE}(T, \hat P) = -\frac{1}{n} \sum_{i=1}^n \sum_{k=1}^K t_{ik} \log \hat p_{ik}$$

• Binary Cross-Entropy (BCE):

$$L_{\rm BCE}(t, \hat y) = -\left[t \log \hat y + (1-t) \log(1-\hat y)\right]$$

• Kullback–Leibler Divergence:

$$L_{\rm KL}(p, \hat p) = \sum_{k=1}^K p_k \log \frac{p_k}{\hat p_k}$$

For a general “weighted exponentiated” loss,

$L_{w,a}(t, y) = w(t, y) |t - y|^a,\quad a > 0$

the paper (Coleman, 17 Nov 2025) demonstrates that such losses, with product-decomposable weights, admit a parallel share-based formulation and are suitable for large-sample equivalence analysis.

4. Asymptotic Equivalence of Level-Based and Share-Based Losses

Recent results establish that, within the broad family of weighted exponentiated losses with decomposable weights, level-based and share-based losses are asymptotically proportional as the number of units increases. Specifically, with targets $\{T_i\}$ and realizations $\{Y_i\}$, if $X_i = T_i/S_{T,n}$, $S_i = Y_i/S_{Y,n}$, and the totals satisfy regularity ($S_{T,n}/n \to \mu_T$, $S_{Y,n}/n \to \mu_Y$), then for

$$A_n = \sum_{i=1}^n L_i^{(\ell)},\quad B_n = \sum_{i=1}^n \ell_i^{(\ell)},\quad c_n = \frac{S_{T,n}}{S_{Y,n}}$$

the ratio $A_n/B_n$ converges almost surely to $(\mu_T/\mu_Y)^a$ (Coleman, 17 Nov 2025): $$\frac{A_n}{B_n} \xrightarrow{a.s.} K = \left(\frac{\mu_T}{\mu_Y}\right)^a$$ When $\mu_T = \mu_Y$, this constant $K = 1$, meaning both losses yield asymptotically identical relative scores and rankings.

A corollary: Whether one evaluates or optimizes using level-based or share-based losses, the resulting ordering of outcomes converges in large samples for this class, explaining the practical equivalence of metrics such as mean absolute error and the index of dissimilarity on large cross-sections (Coleman, 17 Nov 2025).

5. Aggregate Learning: Instance-Level vs. Bag-Level Losses

In scenarios where only aggregate (bag-level) observations are available, such as in privacy-preserving machine learning, the distinction between level-based and share-based (or aggregate-based) losses emerges in a modified form (Javanmard et al., 20 Jan 2024).

• Bag-Level Loss: Compares aggregate (mean) prediction and actual response for each group:

$$L_{\rm bag}(\theta) = \frac{1}{m} \sum_{a=1}^m \ell\left(\bar y_a, \frac{1}{k} \sum_{i \in B_a} f_\theta(x_i)\right)$$

• Instance-Level Loss: Compares each instance's prediction to the bag aggregate:

$$L_{\rm inst}(\theta) = \frac{1}{mk} \sum_{a=1}^m \sum_{i \in B_a} \ell(\bar y_a, f_\theta(x_i))$$

For quadratic $\ell(u, v) = (u-v)^2$, instance-level loss regularizes the bag-level objective via within-bag prediction variance: $$L_{\rm inst}(\theta) = L_{\rm bag}(\theta) + \frac{1}{mk} \sum_{a=1}^m \sum_{i,j \in B_a} (f_\theta(x_i) - f_\theta(x_j))^2$$ Bias-variance analysis in high-dimensional linear regression reveals that pure bag-level losses yield unbiased but higher-variance estimators, while instance-level losses introduce bias but reduce variance. An interpolating estimator,

$$L_\rho(\theta) = (1-\rho) L_{\rm bag}(\theta) + \rho L_{\rm inst}(\theta)$$

enables practitioners to tune the bias-variance trade-off as a function of bag size and data regime (Javanmard et al., 20 Jan 2024).

6. Practical Guidelines and Use Cases

Selection between level-based and share-based losses—or their aggregate analogs—should be grounded in the output structure, statistical characteristics of the data, and modeling objective (Berzal, 7 Nov 2025, Javanmard et al., 20 Jan 2024).

• Use level-based losses (with linear/identity output) when targets are continuous and noise is symmetric and light-tailed.
• Employ share-based losses (softmax or sigmoid activation) for classification, compositional targets, or distribution-matching, where outputs are constrained to a simplex or [0,1].
• Prefer MAE for robust regression, MSE for standard regression, CCE/BCE for classification, and KL-divergence for soft target matching.
• In privacy-constrained or aggregate data settings, adjust between bag- and instance-level approaches to target optimal risk—selecting trade-off parameter $\rho$ and bag size $k$ as dictated by asymptotic risk formulas and privacy requirements (Javanmard et al., 20 Jan 2024).

7. Broader Implications and Theoretical Significance

The asymptotic equivalence result generalizes to any loss of the form $L_{w,a}(t, y) = w(t, y) |t - y|^a$ with decomposable weights, and suggests that for large-scale applications, the choice between level-based and share-based evaluations is asymptotically unimportant for model ranking, provided the loss structure conforms to the class analyzed (Coleman, 17 Nov 2025).

This convergence also explains empirical findings where numerical and distributive accuracy measures yield nearly indistinguishable conclusions on large datasets, making debates about their respective adequacy asymptotically moot in such regimes. Nevertheless, in moderate or small samples, and especially in the presence of population-level constraints, the choice may still matter and should be justified by the modeling context and statistical properties of the data (Coleman, 17 Nov 2025, Berzal, 7 Nov 2025).