Confidence-Level Allocation (COLA)

• COLA is a framework that allocates a specified risk level (α) to balance judgment and statistical evidence, controlling miscoverage across procedures.
• In statistical decision theory, COLA integrates judgmental actions with data-driven updates to ensure optimality and tunable risk aversion.
• In conformal prediction, COLA optimizes the allocation of miscoverage over multiple nonconformity scores, reducing prediction set sizes while guaranteeing coverage.

Confidence-Level Allocation (COLA) refers to a class of decision rules and statistical frameworks in which a global or local confidence level parameter $\alpha$ is allocated, either for the purposes of decision-making under uncertainty or for controlling statistical miscoverage across multiple procedures. COLA methodologies appear in two principal domains: (1) statistical decision theory, where it encodes statistical risk aversion when departing from a judgmental anchor, and (2) conformal prediction, where it governs the allocation of miscoverage across prediction sets induced by multiple nonconformity scores. Both classes of COLA strategies provide optimality and admissibility guarantees, with $\alpha$ having a direct interpretation as a measure of tolerated statistical error or uncertainty.

1. COLA in Statistical Decision Theory

In the context of statistical decision theory, the Confidence-Level Allocation (COLA) methodology formalizes how a decision-maker merges judgmental and statistical inputs. Consider a parameter $\theta \in \mathbb{R}$, a sample $X \sim N(\theta, \sigma^2)$ (with known $\sigma^2$), and a quadratic loss function $L(\theta, a) = -a\theta + \frac{1}{2} a^2$ governing the quality of action $a$ (e.g., a portfolio weight). The decision-maker specifies a status-quo action $\tilde a$ and a confidence level $\alpha \in [0,1]$. The COLA rule is as follows:

• If the data do not provide statistically significant evidence against $\tilde a$ at level $\alpha$, retain $\tilde a$.
• Otherwise, move from $\tilde a$ to the nearest endpoint of the $(1-\alpha)$ confidence interval for the optimality condition.

This rule admits an explicit form: for $a^* = \theta$ (the "informed" optimum), the action

$$\hat d_\alpha(x) = \begin{cases} \tilde a, & \text{if } \tilde a \in [x + c_{\alpha/2},\,x + c_{1-\alpha/2}] \ x + c_{\alpha/2}, & \text{if } x + c_{\alpha/2} > \tilde a \ x + c_{1-\alpha/2}, & \text{if } x + c_{1-\alpha/2} < \tilde a \end{cases}$$

where $c_q = \Phi^{-1}(q)$ and $\Phi$ is the standard normal CDF (Manganelli, 2019). For the special case $\tilde a = 0$, this reduces to soft-thresholding:

$$\hat d_\alpha(x) = \operatorname{sign}(x)\,\max\{|x| - z_{1-\alpha/2}, 0\}, \quad z_{1-\alpha/2} = \Phi^{-1}(1 - \tfrac{\alpha}{2}).$$

2. Admissibility, Performance Guarantees, and Statistical Risk Aversion

COLA rules in decision theory are admissible in the sense of minimal risk—no other rule yields uniformly lower expected loss under the quadratic objective. The central guarantee: with probability at least $1-\alpha$, the data-driven action $\hat d_\alpha(X)$ does not incur greater loss than $\tilde a$. This formalizes $\alpha$ as the maximum probability of performing strictly worse than the judgmental action (Manganelli, 2019).

The parameter $\alpha$ quantifies statistical risk aversion:

• As $\alpha \to 0$, the rule never abandons $\tilde a$ (maximum aversion).
• As $\alpha \to 1$, it always sets $a = x$ (plug-in maximum likelihood, risk-neutral).

Urn-based elicitation experiments, modeled after Ellsberg, have been proposed to operationalize the choice of $\alpha$. Here, a participant's chosen "bet number" $b$ (count of adverse outcomes tolerated in repeated urn draws) translates to $\alpha = b/100$ and thus codifies their statistical risk aversion.

3. COLA in Conformal Prediction: Multi-Score Aggregation

In predictive inference, Confidence-Level Allocation (COLA) addresses the challenge of aggregating multiple conformal prediction sets induced by $K$ distinct nonconformity score functions. Each $S_k: \mathcal{X}\times\mathcal{Y}\to\mathbb{R}$ generates a split-conformal prediction set at nominal coverage $1-\alpha_k$, with the allocation vector $\alpha_{\vec} \in \mathbb{R}^K$ satisfying $\sum_{k=1}^K \alpha_k = \alpha$.

Given $K$ candidate split-conformal sets, the COLA framework searches for the allocation of miscoverage $(\alpha_1, \ldots, \alpha_K)$ that minimizes the expected (or empirical) size of the intersection set:

$\bigcap_{k=1}^{K} \hat C_k(x; \alpha_k)$

while preserving the overall marginal coverage guarantee $1-\alpha$ via a union bound. The corresponding optimization is:

$\min_{\alpha_{\vec} \in \Theta} \frac{1}{n} \sum_{i=1}^n \left| \bigcap_{k=1}^K \hat C_k(X_i; \alpha_k) \right|,\quad \text{s.t.}\ \alpha_k \geq 0,\ \sum_k \alpha_k = \alpha.$

where $\Theta$ is the simplex of feasible allocations.

4. COLA Algorithmic Variants: COLA-e, COLA-s, COLA-f, COLA-l

Distinct algorithmic instantiations of COLA provide performance-efficiency tradeoffs and adapt to application constraints (Xu et al., 15 Nov 2025):

• COLA-e (Empirical Allocation): Minimizes the empirical average prediction set size over the training set, achieving asymptotic marginal coverage with a rate gap of $O(\sqrt{\log(Kn)/n})$.
• COLA-s (Sample Splitting): Ensures finite-sample marginal coverage by dividing data into train/validation (fit allocation on split, deploy on held-out). Coverage is exact due to exchangeability; set sizes are modestly larger than COLA-e.
• COLA-f (Full Conformalization): Grants exact finite-sample coverage by re-computing allocations for every test point/label under augmented exchangeability. Computationally intensive, but minimizes conservatism relative to splitting.
• COLA-l (Local Allocation): Individualized, data-adaptive allocation based on kernel-weighted quantiles. Minimizes local prediction set size at $X_{n+1}$; achieves asymptotic conditional coverage.

The underlying optimization remains piecewise-constant and combinatorial. Grid and stepwise search routines over the allocation simplex are used to identify near-optimal allocations, with computational complexity scaling as $O((\alpha n)^{K-1})$ or better when exploiting sparsity.

5. Theoretical Guarantees and Efficiency

Theoretical performance of COLA variants is governed by the properties of empirical quantiles and the Bonferroni union bound. COLA-s and COLA-f yield exact finite-sample marginal coverage; COLA-e attains asymptotic coverage with explicit convergence rates. Efficiency (measured by set size) is bounded above by terms involving the Lipschitz constants of empirical quantile functions, with excess size over oracle allocation vanishing as sample size increases.

COLA-l's conditional validity result demonstrates that under appropriate kernel smoothness conditions, its local prediction set attains asymptotic conditional coverage at the specified level, modulo rates depending on kernel bandwidth and support overlap.

6. Empirical Benchmarks and Applications

Empirical evaluations utilize both synthetic and real-world regression datasets (e.g., UCI BlogFeedback, Concrete, and Superconductivity) to benchmark COLA against baselines such as EFCP/VFCP (single-score selection), majority vote, score/model-level aggregation, and SAT (p-value merging) (Xu et al., 15 Nov 2025). Across scenarios, COLA-e achieves the smallest average set sizes for moderate to large $n$, while COLA-s and COLA-f guarantee exact coverage with limited cost in size. COLA-l adapts set sizes to local complexity, yielding smallest sets where the conformity scores agree or data density is high.

In the statistical decision-theoretic setting, applications center on mean–variance portfolio allocation, where COLA offers tunable risk guarantees relative to a judgmental "cash only" baseline. For low $\alpha$, portfolios remain in cash; for high $\alpha$, they aggressively chase mean estimates (with empirical evidence of higher drawdowns in adverse regimes) (Manganelli, 2019).

7. Practical Recommendations and Extensions

Choice among COLA variants depends on validity-efficiency-computation tradeoffs:

• For exact finite-sample validity with manageable computation, use COLA-s.
• For minimal set size with large $n$ and acceptable approximate coverage, COLA-e is preferred.
• For individualized coverage demands and heterogeneity, COLA-l leverages data-adaptive kernel weighting.
• For scenarios tolerating high computation to minimize conservatism, COLA-f is applicable.

Hyperparameters, e.g., train/cal split proportion, kernel and bandwidth in COLA-l, require cross-validation or plug-in tuning. Extensions under discussion include regularized allocation to enforce structure (e.g., sparsity) and end-to-end optimization of models and $\alpha$-allocation in conformal pipelines, as well as utility-driven allocations trading off set size for downstream decision value (Xu et al., 15 Nov 2025).