---
title: Area Under the Performance-Cost Curve (AUC-PCC)
url: https://www.emergentmind.com/topics/area-under-the-performance-cost-curve-auc-pcc
type: topic
---

# Area Under the Performance-Cost Curve (AUC-PCC)

Searching arXiv for the cited papers to ground the article and verify identifiers.
Area Under the Performance-Cost Curve (AUC-PCC) denotes an area-under-curve functional in cost space in which classifier performance is evaluated as a function of an operating-condition or cost parameter rather than solely in ROC space. In "Technical Note: Towards ROC Curves in Cost Space" [1107.5930], AUC-PCC is the area under the ROC Cost Curve generated by a rate-driven threshold choice method under a uniform distribution of cost proportions, and it is linearly related to ROC AUC. In "Partial VOROS: A Cost-aware Performance Metric for Binary Classifiers with Precision and Capacity Constraints" [2510.18520], AUC-PCC is the area under a performance-cost curve whose ordinate is the normalized partial area of lesser classifiers over a feasible ROC region, and this area is exactly Partial VOROS. This suggests that AUC-PCC is best understood as a family of expected-performance or expected-loss summaries indexed by the choice of cost parameterization, threshold-selection rule, and feasible operating set.

## 1. Cost-space formulation and operating conditions

For binary classification with class-dependent costs only, the cost-space formulation in [1107.5930] uses misclassification costs \(c_0 \ge 0\) and \(c_1 \ge 0\), class proportions \(\pi_0\) and \(\pi_1\) with \(\pi_0+\pi_1=1\), and a threshold \(t\). With total cost normalized by \(b=c_0+c_1\) and cost proportion \(c=c_0/b\), the loss at threshold \(t\) and cost proportion \(c\) is
\[
Q_c(t; c) \triangleq c_{0} \pi_0 (1 -F_0(t)) + c_{1} \pi_1 F_1(t)
= 2\{c \pi_0 (1 -F_0(t)) + (1-c) \pi_1 F_1(t)\},
\]
where \(F_0(t)\) is sensitivity and \(F_1(t)\) is false positive rate for class \(1\). The same note also uses skew,
\[
z \triangleq \frac{c_0\pi_0}{c_0\pi_0 + c_1\pi_1}
= \frac{c\pi_0}{c\pi_0 + (1-c)(1-\pi_0)},
\]
and defines the normalized loss in skew parameterization as
\[
\frac{Q_c(t; c)}{c_0\pi_0 + c_1\pi_1}
= z (1 -F_0(t)) + (1-z) F_1(t) \triangleq Q_(t; z).
\]
Balanced classes, \(\pi_0=\pi_1\), imply \(z=c\), and after setting \(b=2\), \(Q_\) and \(Q_c\) coincide up to the normalization [1107.5930].

This formulation places ROC analysis and cost analysis in a common expected-loss framework. Following Adams and Hand (1999), the note defines expected loss over a distribution of operating conditions by
\[
L_c \triangleq \int^1_0 Q_c(T_c(c); c)\, w_c(c)\, dc,
\qquad
L_ \triangleq \int^1_0 Q_(T_(z); z)\, w_(z)\, dz,
\]
so any AUC-PCC interpretation is necessarily tied to a threshold-choice method \(T\) and a weighting over operating conditions. A plausible implication is that the phrase “area under the performance-cost curve” is not meaningful without specifying both ingredients.

## 2. ROC Cost Curves and threshold-choice methodology

Traditional cost curves in [1107.5930] are defined in cost space by linear “cost lines” for each fixed threshold. In skew space,
\[
Q_(t; z) = z (1 -F_0(t)) + (1-z) F_1(t),
\]
with intercept \(a_0 = F_1(t)\) at \(z=0\) and slope \(a_1 = 1-F_0(t)-F_1(t)\). The lower envelope of these lines gives the optimal cost curve, obtained by the cost-minimizing selection criterion
\[
T^o_c(c) \triangleq \argmin_t \{Q_c(t;c)\}
= \argmin_t 2\{c\pi_0(1-F_0(t)) + (1-c)\pi_1 F_1(t)\},
\]
and equivalently
\[
T^o_(z) \triangleq \argmin_t \{Q_(t;z)\} = T^o_c(c).
\]

The distinctive construction introduced in [1107.5930] is the ROC Cost Curve. The note first defines a rate-fixed threshold choice method,
\[
T^q_c[r](c) \triangleq \{ t : P(s_i < t) = r \},
\]
with
\[
T^q_c[r](c) = \{ t: F_0(t)\pi_0 + F_1(t)\pi_1 = r \},
\]
and then specializes it to the rate-driven threshold choice method
\[
T^n_c(c) \triangleq T^q_c[c](c) = \{ t : P(s_i < t) = c \}.
\]
Under this rule, the threshold is chosen so that the proportion of positive predictions equals the operating condition. The ROC Cost Curve in cost space is then
\[
c(c) \triangleq Q_c(T^n_c(c); c).
\]

For a scored classifier with \(n\) examples, construction proceeds by sorting examples by score ascending, considering the \(n+1\) cutpoints between successive scores and at the extremes, forming the \(n+1\) cost lines \(CL_0,\ldots,CL_n\), and then following each line over an interval of width \(1/(n+1)\) on the \(x\)-axis. The note states: “From left to right, draw the curve following each cost line (from \(CL_0\) to \(CL_n\)) with a width on the x-axis of \(1/(n+1)\).” Each segment corresponds to a point on the ROC curve; segments that coincide with the optimal cost curve correspond to ROCCH points, and other segments correspond to suboptimal thresholds [1107.5930].

## 3. Area, expected loss, and the linear relation to ROC AUC

With the ROC Cost Curve threshold rule, the expected loss over uniform cost proportion is
\[
L^n_c \triangleq \int^1_0 Q_c(T^n_c(c); c)\, w_c(c)\, dc.
\]
When \(w_c(c)=U(c)\), this quantity is the area under the ROC Cost Curve. The main result in [1107.5930] is the exact linear transformation
\[
U(c) = 2 \pi_0 \pi_1 (1-\mathrm{AUC}) + \frac{1}{3} - \pi_0\pi_1.
\]
If AUC-PCC is defined as the area under the ROC Cost Curve generated by the rate-driven threshold choice method under uniformly distributed cost proportions, then
\[
\mathrm{AUC\text{-}PCC} = U(c) = 2\pi_0\pi_1 (1-\mathrm{AUC}) + \frac{1}{3} - \pi_0\pi_1.
\]
For balanced classes, \(\pi_0=\pi_1=1/2\), the skew version becomes
\[
U(z) = \frac{1-\mathrm{AUC}}{2} + \frac{1}{12}.
\]

This relation makes AUC-PCC an expected-loss summary that is linearly coupled to AUC only under the specific operating-condition distribution and threshold rule above. The inverse mapping is also given:
\[
\mathrm{AUC}
= 1 - \frac{U(c) - (1/3 - \pi_0 \pi_1)}{2\pi_0\pi_1},
\]
and for balanced classes,
\[
\mathrm{AUC} = 1 - 2\left(U(z)-\frac{1}{12}\right).
\]

Several limiting values are reported for balanced classes. For a random classifier with \(\mathrm{AUC}\approx 0.5\), expected loss is \(1/3\). For a perfect classifier with \(\mathrm{AUC}=1\), expected loss is \(1/12\). For a worst classifier with \(\mathrm{AUC}=0\), expected loss is \(7/12\). The note also states that, in the limit \(n\to\infty\), the ROC Cost Curve for a random classifier is \(y=2c(1-c)\), identified as Gini impurity. In the example of Figure 2, \(\mathrm{AUC}=0.83333\), the empirical area under the ROC Cost Curve is \(0.1695\) for cost proportions, and the theoretical transformation gives \(0.1701\) [1107.5930].

## 4. Distinction from optimal cost curves and relation to the Brier score

A recurrent point in [1107.5930] is that the area under the optimal cost curve and the area under the ROC Cost Curve are different quantities. The area under the optimal cost curve is
\[
U( \triangleq \int^1_0 Q_(T^o_(z); z)\, U(z)\, dz,
\]
and it is computed by assuming perfect knowledge of the operating condition and choosing the cost-minimizing threshold at each \(z\). By contrast, AUC-PCC in the ROC Cost Curve sense uses the rate-driven rule \(T^n_c(c)\), not the cost-minimizing rule. The note states explicitly that the area under the optimal cost curve is not linearly coupled to AUC in general; it depends on the ROC convex hull and reflects cost-minimizing thresholds.

The same note establishes a second connection through the probabilistic threshold choice method
\[
T^p_c(c) \triangleq c,
\qquad
T^p_(z) \triangleq z.
\]
Under uniform \(c\), expected loss becomes
\[
U(c) \triangleq \int^1_0 Q_c(c;c)\, U(c)\, dc,
\]
and this is proved to equal the Brier score,
\[
 \triangleq \frac{1}{n} \sum_{i=1}^{n}(s_i-y_i)^2 = \pi_0 _0 + \pi_1 _1.
\]
If scores are evenly spaced, the probabilistic threshold choice method reduces to the rate-driven method, ROC Cost Curves equal Brier Curves, and the corollary is
\[
2 \pi_1\pi_0 (1-\mathrm{AUC}) + \frac{1}{3} - \pi_1\pi_0 = \mathrm{BS}.
\]
Thus, under evenly-spaced scores, AUC-PCC equals the Brier score exactly. Without the evenly-spaced assumption, the area under the ROC Cost Curve remains linearly related to AUC, but it will not generally equal the Brier score. The note further states that, for perfectly calibrated classifiers, the optimal cost curve may coincide with the Brier curve, but the ROC Cost Curve may still make suboptimal choices at some \(c\) [1107.5930].

## 5. Partial VOROS as a constrained AUC-PCC

The framework of [2510.18520] introduces a different performance-cost construction for binary classifiers subject to a minimum precision constraint and a capacity constraint. With data \(D=\{(X_i,Y_i)\}\), positives \(P\), negatives \(N\), prevalence \(p=|P|/|D|\), and a thresholded score function yielding ROC point \((h,k)=(\mathrm{FPR},\mathrm{TPR})\), the feasible ROC region is determined by
\[
\mathrm{Prec} = \frac{pk}{pk+(1-p)h} \ge \alpha
\quad\Rightarrow\quad
k \ge \frac{\alpha(1-p)}{(1-\alpha)p} h,
\]
and by the capacity bound
\[
|P|k + |N|h \le \kappa,
\qquad\text{equivalently}\qquad
y \le \frac{\kappa - |N|x}{|P|}.
\]
The feasible region is therefore the intersection of the unit square with the half-planes \(y \ge m_\alpha x\) and \(y \le (\kappa-|N|x)/|P|\). Under the paper’s practical assumptions, this feasible polygon falls into one of three non-degenerate cases.

Costs are parameterized by unit costs \(c_{FP}\) and \(c_{FN}\), cost ratio \(\lambda=c_{FP}/c_{FN}\), and the fractional cost parameter
\[
t = \frac{c_{FP}|N|}{c_{FP}|N| + c_{FN}|P|}
= \frac{\lambda(1-p)}{\lambda(1-p)+p}.
\]
The normalized cost in ROC space is
\[
\mathrm{Cost}_t(h,k) = t h + (1-t)(1-k),
\]
and iso-cost lines through \((h,k)\) satisfy
\[
y = \frac{t}{1-t}(x-h) + k.
\]
A feasible classifier \(F_1\) is lesser than \(F_2\) at cost parameter \(t\) if \(\mathrm{Cost}_t(h_1,k_1) > \mathrm{Cost}_t(h_2,k_2)\). For a feasible classifier \(F_\tau\), \(A_t^*(F_\tau)\) is the area of the subset of the feasible region consisting of feasible lesser classifiers; geometrically, it is the polygonal area under the iso-cost line within the feasible polygon. The paper proves that \(A_t^*(F_\tau)\) is cost monotone and is a rational linear function of \(t\) [2510.18520].

After normalization by the feasible-region area \(A^*\), performance at cost \(t\) is
\[
f(t) = \max_\tau \frac{A_t^*(F_\tau)}{A^*}.
\]
Partial VOROS is then
\[
PV(F) = \frac{1}{(b-a)A^*}\int_{t=a}^{b}\max_\tau A_t^*(F_\tau)\, dt
= \frac{1}{b-a}\int_a^b f(t)\, dt.
\]
The paper states that AUC-PCC over a cost range \([a,b]\) with uniform weighting \(w(t)=1/(b-a)\) is
\[
\mathrm{AUC\text{-}PCC} = \int_{t=a}^b w(t)f(t)\, dt
= \frac{1}{b-a}\int_a^b f(t)\, dt,
\]
and therefore Partial VOROS equals the area under the performance-cost curve when performance is defined as the normalized partial area function \(f(t)\). If stakeholders specify a distribution over \(\lambda\) rather than \(t\), the generalized form is
\[
PV(F) = \int_{\lambda_{\min}}^{\lambda_{\max}} f(t(\lambda))\, w(\lambda)\, d\lambda.
\]

## 6. Computation, assumptions, and interpretation

For ROC Cost Curves in [1107.5930], AUC-PCC can be computed directly from scores and labels by sorting examples by score ascending, evaluating \(F_0(t_j)\) and \(F_1(t_j)\) at the \(n+1\) cutpoints, forming the cost lines
\[
Q_c(t_j;c) = 2\{c\pi_0(1-F_0(t_j)) + (1-c)\pi_1F_1(t_j)\},
\]
assembling the piecewise linear curve with segment width \(1/(n+1)\), and numerically integrating over \(c\in[0,1]\). Because each segment is linear in \(c\), exact segment area is straightforward. The note also gives a more robust route: compute empirical AUC from the ROC curve and then apply
\[
U(c)=2\pi_0\pi_1(1-\mathrm{AUC})+\frac{1}{3}-\pi_0\pi_1.
\]
The exact linear form depends on the uniform operating-condition distribution; different distributions change the coefficients in expected loss and break the exact linear relation. The formula also depends explicitly on class proportions \(\pi_0,\pi_1\), so interpretation is tied to the evaluation dataset. Ties may lead to fewer unique thresholds, small discrepancies in implementations, and exact agreement only in the continuous limit as \(n\to\infty\) [1107.5930].

For the constrained framework of [2510.18520], computation starts by restricting ROC points to those feasible under both bounds, computing the convex hull of feasible ROC points, and then, for each \(t\), selecting the feasible hull point with minimum \(\mathrm{Cost}_t(h,k)\). The corresponding polygon area \(A_t^*(F_\tau)\) is obtained from the relevant canonical polygon by the shoelace formula, normalized by \(A^*\), and integrated over \([a,b]\) or under a distribution \(w(t)\). The paper reports experiments on a MIMIC-IV cohort of ICU patient-stays with train/val/test sizes \(15474/7802/7861\) and \(p\approx .10\), using logistic regression, MLP, and random forest with 250+ ROC curves. In Scenario 1, \(\alpha=0.15\), \(\kappa=0.5|D|\), and \(\lambda\sim \mathrm{Unif}(1/9,1/6)\), max PV achieved the lowest average test cost \(0.261\), while alternatives were \(\ge 0.305\)–\(0.306\). In Scenario 2, \(\alpha=0.5\), \(\kappa=0.1|D|\), and \(\lambda\sim \mathrm{Unif}(1/40,1/20)\), most strategies were similar at \(0.535\), while VOROS was worse at \(0.636\). The paper’s interpretation is that PV better ranks and selects operating points under realistic precision and capacity constraints and cost ranges [2510.18520].

Across both constructions, AUC-PCC is not the minimum achievable cost unless the thresholding rule itself is cost minimizing. In the ROC Cost Curve setting it is the expected loss of the rate-driven threshold rule under uniformly distributed operating conditions. In the Partial VOROS setting it is the expected normalized partial area over a feasible ROC region. Common misunderstandings arise when these are conflated with the area under the optimal cost curve, with unconstrained ROC AUC, or with the Brier score outside the evenly-spaced-score condition. The sources instead support a narrower conclusion: AUC-PCC is a cost-space summary whose semantics are determined by the chosen threshold rule, the weighting over cost parameters, and, where applicable, the precision and capacity constraints that define the feasible operating region.

Source: https://www.emergentmind.com/topics/area-under-the-performance-cost-curve-auc-pcc