Unlearning Efficiency Score (UES)

• Unlearning Efficiency Score (UES) defines a family of metrics that evaluate the computational side of machine unlearning through runtime, forgetting-utility trade-offs, and fairness measures.
• Different formulations of UES capture raw runtime efficiency, composite benchmark objectives, and fairness-aware costs in federated and decentralized settings.
• Theoretical and sample-level approaches further contrast unlearning with retraining complexity, offering insights for real-time monitoring and standardization of unlearning efficacy.

Searching arXiv for papers on unlearning efficiency metrics and related evaluation frameworks. Unlearning Efficiency Score (UES) denotes a class of evaluation constructs for machine unlearning rather than a single universally adopted statistic. Across the recent literature, the term is used for at least four distinct purposes: direct runtime accounting, composite forgetting–utility benchmarking, fairness-aware measurement of unlearning cost in federated settings, and theory-oriented complexity ratios comparing unlearning with retraining. Taken together, these works suggest that UES is best understood as a family of operationalizations for the computational side of unlearning, with different formulations reflecting different desiderata: wall-clock speed, indistinguishability from retraining, client-level equity, or sample-level auditability (Triantafillou et al., 2024, Wen et al., 13 Aug 2025, Waerebeke et al., 24 Feb 2025).

1. Terminological scope and representative formulations

The literature does not present a canonical UES definition. In some papers, efficiency is reported directly as runtime or compute cost and no explicit UES formula is given; in others, a named or UES-like score combines efficiency with forgetting or fairness objectives. This suggests that the semantic content of UES depends on the evaluation regime being optimized (Liu et al., 2023, Wang et al., 2024, Wen et al., 13 Aug 2025).

The main formulations can be organized as follows.

Formulation family	Representative definition	Primary emphasis
Runtime-centered	RTE in seconds	Wall-clock unlearning cost
Composite benchmark score	$\mathcal{F}$ times utility ratios	Forgetting plus utility
Fairness-aware federated metric	$M_e$	Contribution-weighted cost variance
Theoretical complexity metric	$T_e^U/T_e^S$ or $\rho$	Unlearning vs retraining complexity
Sample-level auditing metric	UnleScore$(z)$	Efficient completeness monitoring

A useful distinction is between efficiency as raw cost and efficiency as a trade-off object. Runtime-centered works measure how fast an unlearning request is processed. Composite benchmark works measure whether the method forgets well while preserving model utility, with speed either entering as a hard eligibility constraint or as one factor in an aggregate score. Federated and theoretical works go further by asking whether cost is equitably distributed or whether speedup is even possible in a minimax sense.

2. Runtime-centered interpretations

A large part of the machine unlearning literature operationalizes efficiency directly through elapsed time or closely related resource proxies. In Controllable Machine Unlearning (ConMU), unlearning efficiency is quantitatively measured using Runtime Efficiency (RTE), defined as the total time in seconds taken by the unlearning algorithm to complete the unlearning request and produce the unlearned model. The paper states explicitly that no explicit “Unlearning Efficiency Score (UES)” formula is provided, and uses RTE as the practical efficiency metric throughout. ConMU’s important data selection module is the principal efficiency lever: selecting fewer retained and forgetting samples reduces fine-tuning volume and thus runtime, while the bounds are controlled by $z_1$ and $z_2$,

$$[\mu_\theta(x) - z_1 \cdot \sigma_\theta(x),\ \mu_\theta(x) + z_2 \cdot \sigma_\theta(x)].$$

On CIFAR-10 with ResNet18 under random forgetting, the reported RTE values are 933.51 for Retrain, 33.69 for IU + Pruning, 28.15 for GA + Pruning, 475.99 for FT + Pruning, and 59.59 for ConMU (Liu et al., 2023).

Other systems follow the same runtime-first pattern. GraphEraser measures unlearning efficiency by average unlearning time over 100 independent requests and reports speedups from $2.06\times$ to $35.94\times$ relative to retraining from scratch, depending on dataset scale (Chen et al., 2021). CAUSE, an exact unlearning system for resource-constrained devices, does not define a UES either; instead it evaluates unlearning speed, energy consumption, and post-unlearning accuracy, with retrained sample number used as a proxy because retraining time and energy are reported to be linearly related to the number of retraining samples (Xia et al., 2024). IAU likewise treats unlearning time in seconds as the efficiency measure and combines it with model-utility and efficacy gaps through an average-rank comparison, rather than through a closed-form UES (Liu et al., 31 Jul 2025). UGradSL uses the language of unlearning efficiency but again gives no explicit UES formula; efficiency is judged comparatively as improved unlearning accuracy at only a marginal additional computational cost (Di et al., 2024).

The significance of this runtime-centered tradition is methodological clarity. It preserves a direct comparison to retraining and aligns with deployment constraints. Its limitation is equally clear: runtime alone does not certify high-quality forgetting, nor does it distinguish between globally low cost and unevenly distributed cost across clients or requests.

3. Composite benchmark scores in formal evaluation

A different interpretation appears in the analysis of the first NeurIPS unlearning competition, where the final evaluation score was also referred to as an “Unlearning Efficiency Score.” Here the score is not a pure time measure. Instead, it composes forgetting quality and utility retention into a single benchmark objective, while computational efficiency is enforced by a runtime cut-off: only algorithms significantly faster than retraining, specifically at most $M_e$0 of retraining wall-clock time, were considered (Triantafillou et al., 2024).

The competition formalized forgetting through an operational unlearning notion based on distributional closeness between retrained parameters $M_e$1 and unlearned parameters $M_e$2. Forgetting quality is aggregated as

$M_e$3

with

$M_e$4

where the bin-width is typically $M_e$5. Utility is measured as the retain-set and held-out test-set accuracies of the unlearned model relative to retraining. The final score is

$M_e$6

This formulation is important because it separates two issues that are often conflated. First, forgetting quality is not approximated by a simple forget-set accuracy gap; the paper states that “accuracy gap” is a poor proxy for forgetting quality under its formal definition. Second, efficiency may enter evaluation as an eligibility condition rather than as a multiplicative term. In this sense, the competition’s UES is holistic but not purely computational.

4. Fairness-aware UES in federated unlearning

In federated unlearning, FedShard introduces a distinct efficiency-fairness metric $M_e$7, presented as the UES counterpart for decentralized settings. The metric is defined over the client set $M_e$8 by

$M_e$9

where $T_e^U/T_e^S$0 is client $T_e^U/T_e^S$1’s unlearning cost, $T_e^U/T_e^S$2 is the absolute value of the client’s contribution factor to the final global model, and $T_e^U/T_e^S$3 is average unlearning cost across clients. A lower $T_e^U/T_e^S$4 indicates better efficiency fairness: clients with similar contribution should incur similar unlearning costs, while higher costs may be acceptable for more influential clients (Wen et al., 13 Aug 2025).

This definition differs fundamentally from raw runtime. The numerator measures deviation from average cost, while the denominator normalizes by model contribution. The paper further states that the metric satisfies reduction to equality, continuity, partition, and saturation, thereby aligning it with established fairness-measurement properties. It also emphasizes that prior works generally reported average or maximum unlearning cost rather than fairness-aware cost dispersion.

Empirically, the paper reports the following $T_e^U/T_e^S$5 values on CIFAR-10 with 512 clients: Retrain 75.64, FedEraser 41.32, RapidRetrain 39.30, FATS 1081.73, FedRecovery 35.92, and FedShard 10.90. The abstract additionally states that FedShard accelerates the data-unlearning process $T_e^U/T_e^S$6–$T_e^U/T_e^S$7 times faster than retraining from scratch and $T_e^U/T_e^S$8 times faster than state-of-the-art exact unlearning methods (Wen et al., 13 Aug 2025).

The broader implication is that a low average runtime does not guarantee equitable deployment cost. In federated systems, efficiency may need to be audited at the client level rather than only at the system average.

5. Theoretical efficiency scores and complexity trade-offs

A theory-oriented line of work replaces empirical runtime with asymptotic comparison to retraining. “When to Forget? Complexity Trade-offs in Machine Unlearning” introduces the unlearning complexity ratio,

$T_e^U/T_e^S$9

where $\rho$0 is the minimal expected number of gradient steps required to unlearn to excess risk below $\rho$1 under an $\rho$2-unlearning guarantee, and $\rho$3 is the minimal expected number of gradient steps needed to retrain from scratch on the retained data to the same excess-risk target. This ratio is the paper’s central efficiency metric and is analyzed under a minimax perspective (Waerebeke et al., 24 Feb 2025).

The paper’s phase diagram partitions machine unlearning into three regimes. In the impossible regime, unlearning cannot asymptotically outperform retraining:

$\rho$4

In the trivial regime, unlearning can be free in the sense of requiring no gradient steps:

$\rho$5

In the efficient regime, a noise-and-fine-tune method achieves

$\rho$6

A related but distinct theoretical construct appears in “From Adaptive Query Release to Machine Unlearning,” which defines relative unlearning complexity

$\rho$7

and derives excess-population-risk bounds for smooth and non-smooth stochastic convex optimization and generalized linear models. The paper treats $\rho$8 and its associated error as the basis for a practical UES-style benchmark, and extends the framework to dynamic streams of insertions and deletions, with at most $\rho$9 retraining triggers for $(z)$0 unlearning requests (Ullah et al., 2023).

These works shift the meaning of efficiency from “seconds on hardware” to “algorithmic difficulty under privacy and utility constraints.” This suggests that any empirical UES that ignores target risk, forget ratio, dimensionality, or privacy level may conflate intrinsically different regimes.

6. Sample-level auditing, application-specific scores, and standardization issues

A further branch of the literature studies efficient scoring at the level of individual samples or application-specific workloads. For approximate unlearning, “Towards Lifecycle Unlearning Commitment Management” introduces UnleScore, a sample-level, black-box metric for unlearning completeness. It combines likelihood-difference and confidence-change statistics:

$(z)$1

with values in $(z)$2, where $(z)$3 indicates no unlearning and $(z)$4 indicates that the sample matches non-member status. The paper reports that UnleScore is strongly correlated with degrees of actual unlearning, with $(z)$5, and that its runtimes are $(z)$6–$(z)$7 shorter than membership-inference-based alternatives, making it suitable for real-time monitoring (Wang et al., 2024).

Large-model unlearning papers also adopt task-specific efficiency surrogates. RapidUn does not define a formal UES, but reports ASR reduction per hour as a direct efficiency metric,

$(z)$8

alongside clean perplexity and seen/OOD attack success rate. On Alpaca-57k, the reported seen-trigger efficiency values are 231.24 for RapidUn, 142.04 for LoReUn, 98.67 for GA Unlearn, 11.28 for Fisher Unlearn, 9.66 for Retrain, and 3.40 for Retain Only (Zhao et al., 4 Dec 2025). In another direction, the low-influence-point framework of “When unlearning is free” argues that pruning negligible-impact points before unlearning can reduce computational cost by up to approximately $(z)$9, and frames that reduction as improving the efficiency component of UES while preserving privacy and performance (Kleiman et al., 4 Dec 2025). For IoT security, DiRLU quantifies efficiency via FLOPS and before/after unlearning accuracy and F1 score rather than through a UES formula, reporting 2,370 FLOPS and approximately $z_1$0 greater efficiency than the state-of-the-art model (Hasan et al., 8 Jul 2026).

Taken together, these works suggest two common misconceptions. The first is that UES is a settled term with a single formula; the literature instead uses the label for multiple non-equivalent constructs. The second is that efficiency can be inferred from runtime alone. Competition benchmarks require high forgetting quality and retained utility; federated settings require fairness; lifecycle monitoring requires per-sample completeness; theory papers show that speedup may be impossible under strict privacy or utility requirements. A plausible implication is that future standardization will need to specify, at minimum, whether UES refers to raw computational cost, a constrained benchmark score, a fairness-aware cost functional, or a complexity ratio against retraining.