When does a predictor know its own loss? (2502.20375v1)

Abstract: Given a predictor and a loss function, how well can we predict the loss that the predictor will incur on an input? This is the problem of loss prediction, a key computational task associated with uncertainty estimation for a predictor. In a classification setting, a predictor will typically predict a distribution over labels and hence have its own estimate of the loss that it will incur, given by the entropy of the predicted distribution. Should we trust this estimate? In other words, when does the predictor know what it knows and what it does not know? In this work we study the theoretical foundations of loss prediction. Our main contribution is to establish tight connections between nontrivial loss prediction and certain forms of multicalibration, a multigroup fairness notion that asks for calibrated predictions across computationally identifiable subgroups. Formally, we show that a loss predictor that is able to improve on the self-estimate of a predictor yields a witness to a failure of multicalibration, and vice versa. This has the implication that nontrivial loss prediction is in effect no easier or harder than auditing for multicalibration. We support our theoretical results with experiments that show a robust positive correlation between the multicalibration error of a predictor and the efficacy of training a loss predictor.

• The paper establishes a theoretical framework connecting a predictor's ability to estimate its own loss on specific inputs to its multicalibration status across data subgroups.
• It introduces a hierarchy of loss predictors, using the model's self-entropy as a baseline to measure whether loss can be predicted better than the model expects.
• The work demonstrates that improved loss prediction identifies multicalibration failures, offering a practical method to audit machine learning models for fairness and reliability.

This paper explores how well a predictor can estimate the loss it will incur on a given input. This is called loss prediction, and it's an important part of figuring out how certain a predictor is about its own predictions. In simple terms, the paper asks: "When does a predictor know what it knows and what it doesn't know?"

Here's a breakdown of the key ideas:

1. Background and Motivation

The paper starts by pointing out that machine learning models are often used in situations where the people using them (downstream consumers) have different tasks, data, and ways of measuring success than the people who created the model. This is especially true in cases like:

• Zero-shot classification: Where a model has to classify things it wasn't specifically trained on.
• Medical classification: Where a model trained on, say, lab reports is used with extra information like a patient's history.

In these situations, it's useful to know how well the model is likely to perform on a specific task or for a specific user. This information can help with:

• Collecting more data to improve the model in areas where it struggles.
• Identifying biases or unfairness in the model's performance.
• Deciding when to trust the model's predictions and when to get a second opinion.

2. The Loss Prediction Problem

The authors focus on binary classification (where the answer is either 0 or 1) to keep things simple, but many of the ideas can be applied to situations with more than two options.

The goal is to estimate the loss (how wrong the prediction is) at a specific input using a loss predictor (LP). The quality of the loss predictor is measured by how close its predictions are to the actual loss.

3. The Self-Entropy Predictor

To measure how good a loss predictor is, the authors introduce a baseline called the self-entropy predictor. Here's the idea:

• A predictor $p$ outputs a probability between 0 and 1 for each input $x$, denoted $p(x)$.
• The predictor acts as if the labels for $x$ are drawn randomly based on the probability $p(x)$.
• The self-entropy predictor calculates the expected loss based on this assumption.

In other words, the self-entropy predictor is the predictor's own estimate of how well it should be doing. The paper then asks: can we create a loss predictor that does better than the model's own self-assessment?

4. A Hierarchy of Loss Prediction Models

The paper defines different types of loss predictors, based on what information they have access to:

• Prediction-only loss predictors: These only know the predictor's output $p(x)$. The self-entropy predictor is an example.
• Input-aware loss predictors: These also know the input features $i(x)$ that were used to train the original model.
• Representation-aware loss predictors: These have access to even more information, such as $r(x)$ some representation of the input. This representation can be:
  • Internal: Computed by the original predictor itself (e.g., the output of one of the layers in a neural network).
  • External: Obtained from a different source (e.g., a different model or human experts).

5. Connecting Loss Prediction to Multicalibration

The core of the paper is connecting loss prediction to a concept called multicalibration. Multicalibration is a way of measuring fairness in machine learning models. A multicalibrated model is well-calibrated across different subgroups of the data.

The authors show that there's a close relationship between:

• Finding a good loss predictor (one that beats the self-entropy predictor).
• Finding a failure of multicalibration in the original predictor.

In other words, if you can predict the loss better than the model itself expects, it means the model isn't properly calibrated across all subgroups of the data.

6. Key Theorems

The paper formalizes the connection between loss prediction and multicalibration using a theorem. The theorem essentially states:

• If you can find a loss predictor that significantly improves upon the self-entropy predictor, then you can use that loss predictor to identify a violation of multicalibration.
• Conversely, if you can find a violation of multicalibration, you can use that to build a better loss predictor.

7. Calibration Blind Spots

The paper also points out that there are situations where calibration isn't necessary for good loss estimation. For example, a predictor that always outputs 0.5 will have a squared loss of 0.25, matching its self-entropy predictor regardless of the true labels. The paper refers to this as "blind spots" of the loss function.

8. Experiments

The authors tested their theoretical ideas with experiments on real-world datasets. The experiments show:

• As the multicalibration error of a model increases, the advantage of using a loss predictor also increases.
• Loss predictors are more helpful for subgroups of data that have higher calibration error.

9. Practical Implications

The paper suggests that loss prediction can be a useful tool for:

• Auditing machine learning models for fairness and reliability.
• Improving models by identifying areas where they are poorly calibrated.
• Deciding when to trust a model's predictions and when to seek external expertise.

In short, this paper provides a theoretical framework for understanding loss prediction and its relationship to multicalibration. It shows that the ability to predict a model's errors is closely tied to the model's calibration across different subgroups of data, with the practical implication that loss prediction can be used to audit the predictor.