Smooth Partial Lotteries for Stable Randomized Selection

Abstract: Competitive selection processes, from scientific funding to admissions and hiring, use evaluations to score candidates, and eventually choose a subset of them based on those scores. Recently, many organizations have adopted partial lotteries, which randomize selection based on evaluation scores. However, existing lottery designs are inherently unstable, as a small change to a single candidate's score can cause large shifts in their selection probabilities. This instability undermines a key goal of lotteries: reducing the influence of fine-grained score distinctions near the decision boundary. We propose smoothness as a design principle for partial lotteries, formalizing it as a Lipschitz condition on the mapping from review scores over candidates to selection probabilities. We introduce the Clipped Linear Lottery, a simple mechanism in which selection probabilities scale linearly with estimated quality between an upper threshold, above which we always accept, and a lower threshold, below which we always reject. We prove that the Clipped Linear Lottery's worst-case regret matches a lower bound for any smooth selection rule up to a factor of $(1 - k/n)$, where $k/n$ is the acceptance rate. We compare smooth selection to other stability notions like Individual Fairness and Differential Privacy, showing that the Clipped Linear Lottery achieves a better smoothness-regret tradeoff than alternatives. Experiments on real peer review data from ICLR 2025, NeurIPS 2024, and the Swiss National Science Foundation demonstrate that existing lottery designs are highly unstable in practice even under perturbations to a single score. Our experiments also confirm the tightness of our theoretical analysis and show that our proposed Clipped Linear Lottery achieves a better smoothness-utility tradeoff than alternatives in practice.

• The paper introduces smoothness in partial lotteries by formalizing an L-smooth selection rule that bounds probability changes relative to score perturbations.
• It presents the Clipped Linear Lottery mechanism, which achieves near-optimal regret while offering transparent and interpretable probability mappings under individual fairness constraints.
• Empirical evaluations on datasets from ICLR, NeurIPS, and Swiss NSF demonstrate that the new approach significantly mitigates instability inherent in traditional threshold-based lottery methods.

Smooth Partial Lotteries for Stable Randomized Selection: Technical Summary

Problem Context and Instability in Partial Lotteries

Randomized selection mechanisms, or partial lotteries, have gained traction in high-stakes evaluative environments such as scientific funding, hiring, and admissions. These mechanisms are motivated by the desire to mitigate excessive focus on irrelevant fine-grained score distinctions and to reduce the risk of arbitrariness near selection thresholds. Traditional methodologies involve three-tier structures with sharp accept/lottery/reject thresholds; however, these rules are fundamentally unstable. The paper demonstrates that minimal score perturbations can induce large, discontinuous shifts in a candidate's selection probability—a property explicitly contradicted by the intended smoothing effect of randomization mechanisms.

Empirically, this instability is not just theoretical: the authors demonstrate using data from ICLR, NeurIPS, and the Swiss NSF that a single-point change in a review can alter a proposal's selection probability by more than $0.3$ under existing lottery designs.

Formalization of Smoothness and the Clipped Linear Lottery

The core contribution is the formalization of smoothness in partial lotteries. The paper defines an $L$-smooth selection rule as one whose mapping from review scores to selection probabilities is Lipschitz: small changes in the aggregate review matrix $X$ ($\ell_{1,1}$ norm) translate to bounded changes in marginal selection probabilities ($\ell_1$ norm) with coefficient $L$.

The authors propose the Clipped Linear Lottery (CLL) as a mechanism that assigns selection probabilities by linearly scaling utility scores (which are assumed to be Lipschitz functions of raw reviews), shifting by a common intercept to meet the selection budget, and clipping the results at $[0,1]$. The CLL has the following formal guarantees:

• Exact $L$-smoothness: Any review perturbation changes selection probabilities by at most $L$ times the review change magnitude.
• Near-optimal regret: Worst-case expected utility loss (regret) is within a $(1 - k/n)$ factor of the theoretical lower bound for any $L$0-smooth rule, where $L$1 is the acceptance rate.
• Transparent, easily interpretable rule: The probability mapping is monotonic and sparse (auto-accept, auto-reject, and lottery pools).
• Instance-optimal under individual fairness (IF): For a fixed review matrix, CLL maximizes utility subject to a Lipschitz constraint on the difference between selection probabilities and candidate utilities.

Theoretical Analysis: Smoothness-Utility Tradeoff

The paper delivers tight upper and lower bounds on the fundamental tradeoff between smoothness and utility. By using utility functions with provable Lipschitz properties (e.g., mean, min, max of reviews), the analysis shows:

• The CLL's regret scales as $L$2, where $L$3 is the utility function's Lipschitz constant.
• No $L$4-smooth rule can outperform the CLL by more than a $L$5 multiplicative factor in worst-case regret for a given $L$6.
• As $L$7 (perfect smoothness), the bound matches that of uniform random selection; as $L$8 becomes large, CLL approaches deterministic top-$L$9 selection.

Comparison to Other Stability and Fairness Notions

The CLL is positioned relative to alternative stability definitions:

• Individual Fairness (IF): CLL is instance-optimal under IF, but IF and smoothness are formally incomparable (the former is per-instance, the latter is across-instances).
• Standard Differential Privacy (DP): DP constraints are neither necessary nor sufficient for marginal smoothness. DP can allow mechanisms with discontinuous changes in marginals upon infinitesimal input perturbations.
• Metric DP: Implies smoothness, but enforces it less efficiently, incurring a $X$0-fold utility regret overhead relative to CLL.
• Softmax/Exponential Mechanism: Achieves tunable smoothness but always higher regret scaling as $X$1 compared to linear $X$2 in CLL.

Empirical Evaluation

Experiments were conducted on ICLR 2025, NeurIPS 2024, Swiss NSF, and synthetic datasets, focusing on the regret-smoothness frontier:

• Threshold-based lotteries (MERIT, Swiss NSF) show high instability; single small review perturbations yield large ($X$3) probability jumps.
• Clipped Linear Lottery achieves strictly superior smoothness–utility tradeoffs, consistently dominating both top-$X$4 softmax and interval-based mechanisms at all tested acceptance rates.
• Theoretical smoothness and regret bounds are empirically tight, with both CLL and softmax observed within 1–5% of the predicted limits.

Practical and Theoretical Implications

The results have direct implications for randomized selection policy design:

• Partial lotteries can and should be designed with global stability guarantees to prevent reversal of the key motivation for randomization: mitigating arbitrary decision jumps near the selection threshold.
• The CLL's transparency and interpretable mapping enable robust institutional adoption and clear communication with stakeholders.
• The theoretical machinery for smoothness is compatible with other constraints (e.g., ex post validity via interval dominance) but enforcing both jointly may require compromises, as discussed in the appendices.

Theoretically, the work clarifies the fundamental tradeoff landscape for randomization-induced fairness and stability, setting a benchmark for future mechanistic innovations.

Limitations and Future Directions

The paper highlights several open issues:

• The global Lipschitz assumption on utility functions may be optimistic for noisy, context-dependent evaluations.
• The theoretical optimality gap—the $X$5 factor—remains to be closed.
• Broader institutional and behaviorally informed studies are needed to assess if smooth lotteries reduce reviewer deliberation and improve perceptions of legitimacy.

Further research should explore instance-adaptive lotteries and locally Lipschitz or stochastic utility functions, possibly integrating advanced privacy techniques for enhanced adaptivity.

Conclusion

This work rigorously defines smoothness as a principle for partial lottery design, introduces and analyzes the Clipped Linear Lottery as a near-optimal mechanism on the regret–smoothness frontier, and provides tight theoretical and empirical evidence for its superiority over established alternatives. The framework and results have immediate application in randomized selection for peer review, funding allocation, and other evaluative processes where both utility-maximization and stability to evaluation noise are paramount.