Year-Weighted Loss Function

• Year-weighted loss function is a variant of weighted classification loss that assigns scalar weights based on observation years to capture temporal importance.
• It integrates expected confusion matrix components and smooth surrogate losses, such as cross-entropy, to optimize time-sensitive performance metrics.
• The approach is implemented in neural network training to align cost-sensitive learning with evaluation metrics, addressing challenges like overfitting and instability.

A year-weighted loss function is a variant of weighted classification loss designed to emphasize the relative importance of data samples according to their temporal provenance, specifically by assigning scalar weights based on the "year" attribute associated with each observation. This methodology provides a formal mechanism for optimizing neural network classification performance with respect to evaluation metrics that account for temporal heterogeneity, as delineated in a comprehensive theoretical framework for weighted metrics (Marchetti et al., 2023).

1. Definition and Formalization

Consider a supervised binary classification task with training data $\{(x_i, y_i)\}_{i=1}^n$, where $y_i \in \{0, 1\}$ and $p_i \equiv p_\theta(x_i) = P(y_i=1 | x_i; \theta)$ denotes the predicted probability. The year-weight for each instance is denoted as $w_i = w(\mathrm{year}_i)$ with $w_i > 0$. The weighted accuracy-type metric over threshold $\tau$ is expressed as:

$$M_w(\theta) = \frac{\sum_{i=1}^n w_i\bigl[y_i\,I\{p_i>\tau\} + (1-y_i)(1-I\{p_i>\tau\})\bigr]}{\sum_{i=1}^n w_i}$$

where $I\{\cdot\}$ is the indicator function and $\tau$ is a classifier decision threshold. For threshold-integrated score-oriented losses with $\tau \sim \mathrm{Uniform}[0, 1]$, the indicator expectation reduces to $E_\tau[I\{p_i > \tau\}] = p_i$.

2. Weighted Confusion Matrix and Score Construction

Replacing hard-thresholded decisions with expectations over $\tau$ yields "expected" weighted confusion-matrix components:

• $TP_w = \sum_{i} w_i y_i p_i$ (weighted true positives)
• $TN_w = \sum_{i} w_i (1-y_i)(1-p_i)$ (weighted true negatives)
• $FP_w = \sum_{i} w_i (1-y_i) p_i$ (weighted false positives)
• $FN_w = \sum_{i} w_i y_i (1-p_i)$ (weighted false negatives)

Score-oriented metrics such as the weighted True Skill Statistic (TSS) can be written as:

$$M_w(\theta) = \frac{TP_w}{TP_w + FN_w} - \frac{FP_w}{FP_w + TN_w}$$

More generally, any metric $s_w$ expressible as a function of weighted confusion matrix entries is admissible, provided monotonicity in TP, TN and anti-monotonicity in FP, FN (see Def. 2.2 in (Marchetti et al., 2023)).

3. Construction of Differentiable Score-Oriented Losses

The design of a surrogate, differentiable loss proceeds as:

1. Definition of $w_i$ and construction of $wCM(\tau, \theta)$, e.g., $FP \to \sum_i w_i (1-y_i) I\{p_i > \tau\}$.
2. Introduction of a threshold density $f$ on $[0,1]$ (commonly uniform), replacing indicators with their expectation $E_\tau[I\{p_i > \tau\}] = F(p_i)$.
3. Selection of a smooth metric $s_w(\theta) = s(E_\tau[wCM(\tau, \theta)])$.
4. Definition of score-oriented loss:

$L(\theta) = -s_w(\theta)$

If the metric $s$ is linear in confusion matrix entries, minimization of $L(\theta)$ precisely aligns with maximization of $E_\tau[s(wCM(\tau, \theta))]$. Taylor expansion yields an approximation for non-linear cases.

4. Year-Weighted Cross-Entropy Loss Derivation

A prevalent choice sets $s_w(\theta) = -(FP_w + FN_w)$, yielding:

$$L_w(\theta) = E_\tau[FP_w + FN_w] = \sum_{i=1}^n w_i[(1-y_i)F(p_i) + y_i(1-F(p_i))]$$

For uniform $f(\tau)$, $F(p) = p$ and this simplifies to

$$L_w(\theta) = \sum_{i=1}^n w_i[(1-y_i)p_i + y_i(1-p_i)]$$

To recover the canonical cross-entropy, the framework generalizes to a weight function $w_i(y_i,p_i)$:

$$w_i(y_i,p_i) = -\omega_0(1-y_i)\frac{\log(1-p_i)}{p_i} - \omega_1 y_i \frac{\log(p_i)}{1-p_i}$$

Choosing positive constants $\omega_0, \omega_1$, and integrating on $\tau$ leads to

$$L_{\mathrm{wCE}}(\theta) = -\sum_{i=1}^n [\omega_1 y_i \log p_i + \omega_0(1-y_i)\log(1-p_i)]$$

5. Practical Year Weight Schemes

A canonical example employs a linearly increasing year-weight:

$$w_i = 1 + \lambda(\mathrm{year}_i - 2000),\quad \lambda > 0$$

Setting $\omega_0 = \omega_1 = 1$, the training loss is:

$$\mathcal{L}(\theta) = -\frac{1}{n}\sum_{i=1}^n \left(1 + \lambda(\mathrm{year}_i - 2000)\right)\left[y_i \log p_\theta(x_i) + (1-y_i)\log(1 - p_\theta(x_i)) \right]$$

6. Assumptions, Limitations, and Implementation

The theoretical framework assumes:

• $\tau$ is uniform on $[0,1]$ (so $F(p) = p$), alternative densities would distort $p \mapsto F(p)$.
• Year-weights $w_i > 0$ are necessary for maintaining loss convexity.
• Extreme or rapidly varying $w_i$ can cause instability or overfitting, with potential detriment to generalization when heavily weighting recent/early years with sparse data.
• Weights are generally held fixed a priori; selection or validation-based tuning of $\lambda$ or the weight function family may be necessary if year importance is uncertain.
• Expectation over $\tau$ produces a smoothed loss, but downstream deployment commonly selects a fixed threshold $\tau^* \neq 0.5$; the framework aligns with metrics only in expectation over $\tau$.
• Direct applicability is to binary classification (or multilabel via one-vs-rest); multiclass settings require additional care (see Sec 7 in (Marchetti et al., 2023)).
• Weighted cross-entropy is proper (Fisher-consistent) and convex in $p_i$.

Implementing the year-weighted loss in neural network training involves constructing the loss above with appropriate $w_i$ and using a standard optimizer such as Adam. The actual choice of $w_i$ encodes model bias toward samples from years considered more important.

7. Relationship to Broader Weighted Metric Optimization

The year-weighted loss is encompassed within the more general theoretical construct for optimizing neural networks against weighted metrics, unifying cost-sensitive learning, weighted cross-entropy, and skill score maximization. This approach resolves the classical misalignment between the metric of interest—potentially weighted for temporal or other domain-specific relevance—and the minimization of the surrogate loss during neural network training, aligning learning objectives with downstream evaluation protocols.