---
title: 'CTHS: Risk-Aware Error Allocation Strategy'
url: https://www.emergentmind.com/topics/confirm-triggered-harmonic-spending-cths
type: topic
---

# CTHS: Risk-Aware Error Allocation Strategy

Confirm-Triggered Harmonic Spending (CTHS) is a statistical error-budget allocation strategy for controlling cumulative risk in recursive self-modification frameworks. First introduced in the context of the Statistical Gödel Machine (SGM) paradigm, CTHS ensures that familywise error rate (FWER) is tightly bounded during iterative, proof-less modification of code or hyperparameters. By allocating α only at confirmation events—rather than every iteration or proposal—CTHS concentrates statistical power on genuinely promising candidates while preserving global error guarantees across potentially unbounded self-modification trajectories [2510.10232].

## 1. Foundational Motivation and Formalism

In recursive self-modification settings such as SGM, each outer round proposes a modification to system code or hyperparameters. This proposal is initially subjected to a lightweight screening procedure, intended to filter out obviously inferior options without committing error budget. Only if the screening is sufficiently promising does the candidate escalate to a confirmation event, where a thorough statistical test is conducted to certify strict improvement. 

To maintain a global upper bound δ on the probability of ever accepting a harmful modification, standard approaches split δ among tests a priori, either uniformly or harmonically across all rounds. The naïve split δ/T or harmonic per-round rule
$$
\delta_t = \frac{\delta}{t H_T},\quad H_T = \sum_{i=1}^T \frac{1}{i}
$$
spends α regardless of whether a true confirmation occurs, thus dissipating power by allocating to rounds that never escalate beyond screening.

CTHS addresses this inefficiency by indexing the error budget not by attempted rounds t but by the actual sequence of confirmation events k. At the k-th confirmation event (out of total at most T), the test is conducted at level
$$
\alpha_k = \frac{\delta}{k H_T},\quad k = 1,2,\ldots,T.
$$
As a consequence,
$$
\sum_{k=1}^T \alpha_k = \delta \frac{(\sum_{k=1}^T \frac{1}{k})}{H_T} = \delta,
$$
so the global FWER remains controlled at the target δ [2510.10232].

## 2. Operational Algorithm and Integration

The CTHS protocol is implemented as a wrapper for the outer self-modification loop:
1. **Initialization:** Set global δ, fix T (maximum rounds), compute $H_T = \sum_{i=1}^T 1/i$, set confirmation index $k \gets 0$.
2. **Proposal and Screening:** For each round t, proposer $\Pi$ suggests a candidate $\theta_t'$. A screening evaluation is performed. 
   - If screening fails, reject instantly; no error budget is spent.
   - If screening passes, escalate to confirmation.
3. **Confirmation:** Increment $k \gets k+1$, allocate $\alpha_k = \delta/(k H_T)$.
   - Gather n paired improvement samples $(\Delta_1, \ldots, \Delta_n)$ via thorough evaluation.
   - Compute one-sided lower confidence bound $LCB_{1-\alpha_k}$ for the mean improvement $\mu$.
   - If $LCB_{1-\alpha_k} > 0$, accept the edit and update incumbent parameters.
   - Else, reject.
4. **Termination:** Halt at T rounds or upon external stopping criteria.

This approach strictly requires that no portion of δ is spent unless a confirmation event occurs, focusing expenditure on decisions with real risk [2510.10232].

## 3. Mathematical Guarantees: Confidence Bounds and FWER Control

Given bounded improvement samples $\Delta_i \in [a, b]$ and normalized $X_i = \Delta_i/R$ for $R = \max\{|a|,|b|\}$, the empirical mean is
$$
\hat{\mu} = \frac{1}{n} \sum_{i=1}^n X_i.
$$
Hoeffding’s inequality yields the bound
$$
\mathbb{P}(\mu < \hat{\mu} - \epsilon) \leq \exp\left(-\frac{2n\epsilon^2}{(b-a)^2}\right),
$$
and for chosen $\alpha_k$, solving for $\epsilon_k$ gives
$$
\epsilon_k = (b-a)\sqrt{\frac{1}{2n} \ln \frac{1}{\alpha_k}}.
$$
The acceptance criterion is $LCB_{1-\alpha_k} := \hat{\mu} - \epsilon_k > 0$.  

The total probability of a harmful acceptance is controlled by the union bound:
$$
\mathbb{P}(\exists \text{ harmful accept}) \leq \sum_{k=1}^T \alpha_k = \delta,
$$
thus proving FWER control. Substitution of anytime e-value tests with Ville’s inequality yields equivalent guarantees [2510.10232].

## 4. Comparison to Traditional α-Spending Rules

| Scheme                      | Allocation Index   | α Spent per Test            | Typical Outcome                    |
|-----------------------------|-------------------|-----------------------------|------------------------------------|
| Fixed per-round             | $t$               | $\delta / T$                | Small α per round, power diluted   |
| Harmonic per-round          | $t$               | $\delta / (t H_T)$          | Somewhat larger α early, but waste |
| CTHS                        | $k$ (event count) | $\delta / (k H_T)$          | Highest α at first confirmations   |

Traditional methods spend α at every round, regardless of screening outcome, leading to "wasted" budget when many proposals are rejected early. Even harmonic per-round schedules suffer if confirmations are infrequent or delayed. CTHS, indexed by actual event count k, spends budget only when a full statistical test is required, concentrating resources on proposals that may genuinely yield improvement. This maximizes statistical power, particularly in the first few confirmations, while exactly preserving the global error threshold [2510.10232].

## 5. Pseudocode Template for CTHS Integration

```python
# Initialization
H_T = sum(1/i for i in range(1, T+1))
k = 0        # confirmation event counter
theta = theta_0

for t in range(1, T+1):
    theta_prime = Pi.propose(theta)
    Delta_screen = H.screen(theta, theta_prime)
    if mean(Delta_screen) < screening_threshold:
        continue   # reject without spending α

    # Confirmation event
    k += 1
    alpha_k = delta / (k * H_T)
    Delta_confirm = H.confirm(theta, theta_prime, n_samples)
    LCB = compute_LCB(Delta_confirm, alpha_k)
    if LCB > 0:
        theta = theta_prime   # accept edit
    else:
        # reject
        continue
```

In this scheme, α is only decremented at actual confirmation events, and each acceptance is validated via a rigorous confidence bound tied to the current α allocation.

## 6. Empirical Evidence and Experimental Implications

Experimental results from SGM stress tests confirm the superior power of CTHS. In a synthetic power analysis (CIFAR-100), a +4 percentage point true gain was injected at confirmation. Under the harmonic per-round rule with $\delta=0.10$, total α spent over four confirmations ($t=3,4,5,6$) was 0.0388, insufficient for acceptance (zero accepts). With CTHS ($k=1,2,3$ confirmations), allocation was more aggressive: $α_1 \approx 0.0408, α_2 \approx 0.0204, α_3 \approx 0.0136$; the first confirmation yielded an immediate accept [2510.10232]. 

| Schedule     | Confirm. Rounds | Total Spend | Accepts | Outcome                                      |
|--------------|-----------------|-------------|---------|----------------------------------------------|
| CTHS         | 1,5,6           | 0.0748      | 1       | Early accept; later rejects                   |
| Harmonic     | 3,4,5,6         | 0.0388      | 0       | No acceptance; small αₜ                      |

On real CIFAR-100 hyperparameter optimization (Table 2), CTHS certified a genuine +5.51 pp gain under $\delta=0.1$ at iteration 6 (30-seed confirmation), while rejecting spurious tweaks and never exceeding the global error rate [2510.10232].

## 7. Context, Applicability, and Significance

CTHS formalizes a principled and computationally parsimonious approach to α-spending in recursive self-modification and continual learning systems where risk control is critical. By linking the error budget to actual confirmation events, CTHS not only preserves familywise validity but also materially increases power relative to traditional splits. Its impact is most pronounced in scenarios with frequent screening rejection, highly variable confirmation timing, or open-ended search processes as seen in AutoML and neural architecture search pipelines. 

A plausible implication is that CTHS could generalize to other statistical decision processes with sequential filtering, wherever error budget must be tightly controlled over dynamically triggered, rather than deterministic, test sequences. The method’s adoption in SGM positions it as a foundational building block for scalable, risk-aware self-modifying machine learning [2510.10232].

Source: https://www.emergentmind.com/topics/confirm-triggered-harmonic-spending-cths