Balanced Loss Metrics: B-MSE and B-MAE

• Balanced loss metrics are regression loss functions that adjust traditional MSE and MAE by incorporating rank-based normalization or sample-specific density re-weighting.
• They offer bounded evaluation measures for ordinal data and heightened sensitivity to errors in rare or extreme target regions.
• Applications span statistical disclosure control and imbalanced forecasting, demonstrating improved interpretability and performance in preserving data quality and predicting rare events.

Balanced loss metrics—specifically Balanced Mean Squared Error (B-MSE) and Balanced Mean Absolute Error (B-MAE)—are a family of regression loss functions designed to mitigate issues of target imbalance and comparability in quantitative modeling. These metrics have found influential application both in information loss quantification for statistical disclosure control and in forecasting rare events under regression settings. B-MSE and B-MAE modify conventional MSE and MAE by either normalization or sample re-weighting, achieving losses with interpretable bounds or distributional equity that aligns with the risk or class distribution at hand (Neto, 2023, Ning et al., 19 Feb 2025).

1. Formal Definitions and Loss Construction

Balanced loss metrics are constructed via two principal strategies:

1.1. Bounded Rank-Based Metrics (brMAE, brMSE)

For statistical disclosure control, the bounded rank-MAE (brMAE) and bounded rank-MSE (brMSE) operate on the rank representation of original and masked numeric microdata. Given $X = [x_{ij}] \in \mathbb{R}^{n \times p}$ and masked $\widetilde X = [\tilde x_{ij}]$, ranks $r_{ij}$ and $\tilde r_{ij}$ are evaluated per column with unique tie-breaking. The metrics are: $$\mathrm{brMAE} = \frac{ \sum_{j=1}^p \sum_{i=1}^n |r_{ij}-\tilde r_{ij}| }{ 2p \sum_{k=1}^K (n-2k+1) }, \quad K = \lfloor n/2 \rfloor$$

$$\mathrm{brMSE} = \frac{ \sum_{j=1}^p \sum_{i=1}^n (r_{ij}-\tilde r_{ij})^2 }{ 2p \sum_{k=1}^K (n-2k+1)^2 }$$

Both are within $[0,1]$, 0 indicating perfect ordinal preservation and 1 indicating maximal inversion (Neto, 2023).

1.2. Weighted (Density-Rebalanced) Metrics (B-MSE, B-MAE)

For imbalanced regression tasks, B-MSE (and by extension, B-MAE) adopt sample-specific weights inversely proportional to the data density, up-weighting rare but critical outcomes. Denoting regression targets $y = (y_1, ..., y_N)$ and predictions $\hat y$, and letting $\hat p(y_i)$ be a kernel/histogram density estimate with smoothing parameter $\sigma$,

$$L_\mathrm{BMSE}(y, \hat y) = \frac{1}{N} \sum_{i=1}^N w(y_i) (\hat y_i - y_i)^2,\quad w(y_i) = \frac{1}{\hat p(y_i) + \sigma}$$

$$L_\mathrm{BMAE}(y, \hat y) = \frac{1}{N} \sum_{i=1}^N w(y_i) |\hat y_i - y_i|$$

For event-focused weighting (e.g. extreme percentiles), $w(y_i)$ can also be thresholded using percentile logic (e.g. increasing $w$ for $y_i$ above the 80th/90th percentile) (Ning et al., 19 Feb 2025).

2. Theoretical Properties and Interpretation

Balanced loss metrics adjust key theoretical properties:

• Range and Boundedness: Bounded forms (brMAE, brMSE) are strictly within $[0,1]$, facilitating direct comparability with bounded disclosure risk scores. Weighted forms adapt to the empirically induced class/target imbalance but are not a priori bounded.
• Error Sensitivity: brMAE penalizes average rank displacement; brMSE penalizes large rank changes more heavily (quadratic). B-MSE and B-MAE penalize errors more when they occur on rare (under-represented) targets.
• Interpretive Consistency: brMAE $\approx 0.3$ implies dataset ranks are $30\%$ of the possible shift away from truth, given the maximal possible permutation distance. For B-MSE/B-MAE, high values indicate increased forecasting error, particularly for rare targets, making them sensitive proxies for tail event prediction performance (Neto, 2023, Ning et al., 19 Feb 2025).
• Comparative Advantage: Bounded metrics ease DR–IL (disclosure risk–information loss) tradeoff analysis, while re-weighted (balanced) metrics ensure minority outcomes (e.g., extreme events) are not marginalized by typical-case optimization (Neto, 2023, Ning et al., 19 Feb 2025).

3. Computational Procedure and Implementation

Balanced loss evaluation consists of the following steps:

Rank-based (brMAE/brMSE):

1. Rank original and perturbed data independently per feature.
2. Accumulate absolute or squared rank differences across the $n \times p$ data matrix.
3. Normalize by the theoretical maximum (full inversion), calculated as $2 \sum_{k=1}^K (n - 2k + 1)$ (MAE) or $2 \sum_{k=1}^K (n - 2k + 1)^2$ (MSE).
4. Return the normalized value(s) (Neto, 2023).

Re-weighted (B-MSE/B-MAE):

1. Estimate the marginal density of each target $y_i$ or partition targets by percentile.
2. Assign $w(y_i)$ per sample according to density or event-centric logic.
3. Compute loss: sum of per-sample weighted errors (squared or absolute).
4. Average over all $N$ samples.

A minimal PyTorch implementation for B-MSE (with optional B-MAE) involves construction of the weight vector for each batch based on density or thresholds, pointwise multiplication of errors by weights, and standard backward optimization (Ning et al., 19 Feb 2025).

4. Empirical Performance and Application Contexts

Balanced loss metrics have been empirically validated in two principal domains:

Disclosure Control (brMAE/brMSE):

• Tested on Tarragona and Census datasets with masking schemes (microaggregation, additive noise, rank-swapping).
• All metrics (raw, bounded, IL1, IL1s) increased monotonically with perturbation.
• Very high Spearman’s $\rho$ ($\approx 0.95$–0.99) between brMAE/brMSE and respective unbounded metrics.
• brMAE/brMSE outperformed IL1 under rank-swapping; IL1s occasionally superior for noise addition but differences marginal.
• Bounded metrics facilitate fair comparisons with bounded risk measures, and preserve dataset quality ordering under typical perturbation scenarios (Neto, 2023).

Imbalanced Event Forecasting (B-MSE/B-MAE):

• Applied to marine heatwave prediction for New Zealand SSTAs, where extreme (e.g., 90th percentile) MHW events are rare.
• FCNs with B-MSE achieved the best CSI80 (0.5002), outperforming MAE, MSE, Huber, weighted MSE, and focal losses for extreme event detection.
• B-MSE/B-MAE-focused models substantially improved rare event forecasting without degrading average-case performance.
• A Friedman test confirmed the superior ranking of balanced/scaling-weighted MSE for extreme-regime metrics (CSI80), as opposed to vanilla MSE for general SSTA (Ning et al., 19 Feb 2025).

5. Practical Recommendations, Limitations, and Design Choices

• Implementation Recommendations: Use built-in rank functions with unique tie-breaking for br* metrics. For weighted metrics, precompute or batch compute density estimates or event thresholds. Precompute normalization sums for bounded metrics once per $n$.
• Computational Complexity: $O(np)$ for br* metrics (rank-based); weighted losses add per-sample re-weighting with minimal overhead.
• Use Cases: Favor bounded metrics to align information loss with bounded risk analysis in microdata masking. Deploy balanced (weighted) metrics in forecasting settings where rare target regimes (extremes, tail events) are crucial for deployment.
• Limitations:
  • Bounded metrics: Disregard numeric magnitude shifts beyond their ordinal impact.
  • Re-weighted losses: Sensitive to choice and estimation of density, particularly in heavy-tailed or multi-modal distributions.
  • Discrete Levels: For small $n$, bounded metrics may possess reduced granularity.
  • Consistency: Tie-breaking and normalization must be held consistent across iterations or data splits.
• Design Choices: Select brMAE for linearly penalizing rank errors; brMSE for emphasizing large deviations. Report both br* and raw metrics when both numeric and ordinal distances matter. For B-MSE/B-MAE, tune weight calculation and smoothing parameters for application specificity (Neto, 2023, Ning et al., 19 Feb 2025).

6. Connections to Broader Methodological Trends

Balanced loss metrics embody two methodological trends:

• Standardization of Evaluation: Bounded and comparable scales are critical where model performance must be interpreted relative to jointly bounded risk or utility measures, as in statistical disclosure control.
• Imbalanced Learning: Density-weighted (balanced) loss functions directly address the well-recognized issue of regression/classification with skewed target distributions, making them essential for rare event modeling in domains such as climate extremes or finance.

From disclosure risk management to neural models for environmental event prediction, B-MSE and B-MAE represent key innovations in tailoring loss landscapes to the realities of data distribution and stakeholder priorities (Neto, 2023, Ning et al., 19 Feb 2025).