AObench: Benchmarking Activation Oracles

• AObench is a modular, extensible evaluation suite that benchmarks Activation Oracles by targeting challenges such as hallucination, vagueness, and text inversion.
• It integrates a diverse task battery—including binary, diffing, and free-form QA tasks—with rigorously defined metrics like ROC AUC and chance-adjusted scores.
• The evaluation pipeline employs consensus sampling and container-friendly software to ensure reproducible, end-to-end interpretability research.

AObench is a modular, extensible evaluation suite that systematically measures the quality of Activation Oracles (AOs)—specialized LLMs designed to interpret residual-stream activations from LLMs. It targets fundamental limitations identified in AO research, namely hallucination, vagueness, and text-inversion confounds, by providing a diverse task battery and formally defined metrics. AObench is widely referenced as the first public benchmark aimed at principled, chance-adjusted, and realistic evaluation of the interpretation capacities of AOs, enabling end-to-end interpretability research grounded in observable activation phenomena (Bauer et al., 23 May 2026).

1. Motivation and Problem Scope

AObench was introduced to address known weaknesses in Activation Oracles, including their tendency to hallucinate unsupported facts, generate vague or generic responses, and exploit superficial correlations arising from text-inversion. Prior evaluations of AOs were ad-hoc, lacked extensibility to new tasks or architectures, and often conflated genuine interpretive success with trivial surface-form reconstruction. AObench explicitly targets these deficiencies by operationalizing a suite of binary, model-diffing, and free-form QA tasks relevant to AO use cases, and by implementing design choices that mitigate confounding factors such as text-inversion (Bauer et al., 23 May 2026).

2. Architectural Overview

AObench’s architecture is structured around three principal components to yield reliable, extensible assessment:

Component	Description	Examples
Task Suite	Battery of binary, diffing, and free-form QA tasks covering all major AO failure modes	Sycophancy-from-CoT, Not Just Reading Tokens, Taboo Extraction
Benchmark Dataset	Includes “model organisms” and stress tests for text-inversion and solvability	Context-differentiated tokens, on-policy rollouts
Evaluation Pipeline	End-to-end scriptable process for loading activations, constructing inputs, sampling, and scoring	Consensus-based sampling, ROC AUC, specificity metrics

Tasks are intentionally varied to isolate different behavioral axes: binary tasks (e.g., sycophancy detection), model-diffing (assessing contextual dependence), and open-ended question answering about model latent reasoning. Datasets include not only hand-crafted examples but also challenging confounds and solvability controls, ensuring that a given AO checkpoint is evaluated beyond trivial inference mechanisms (Bauer et al., 23 May 2026).

3. Evaluation Protocol and Formal Metrics

AObench computes standardized, chance-adjusted metrics for each task class:

• Binary-classification tasks: evaluated using ROC AUC for AO logit margins $\ell_e =$ logits(Yes) $-$ logits(No) against ground truth $y_e \in \{0, 1\}$. ROC AUC is preferred over accuracy due to class imbalance and checkpoint bias.
• Open-ended QA tasks: LLM judges or gold references score each oracle answer $r$ for correctness ($s_{correct} \in [0, 1]$) and specificity ($s_{spec} \in [0, 1]$). Hallucination rate is computed as $1 - s_{correct}$; vagueness as $1 - s_{spec}$.
• Chance-adjusted scores correct for baseline performance: $$m_T^{adj} = (m_T - m_T^{chance}) / (1 - m_T^{chance})$$.
• Consensus sampling: For free-form questions, AObench samples $k$ completions and retains the majority answer $-$0 only if its frequency exceeds a threshold $-$1 (e.g., 0.8), trading recall for higher precision against hallucination.

This protocol enables robust pipeline-based benchmarking and supports rapid integration of new tasks or model checkpoints (Bauer et al., 23 May 2026).

4. Dataset Construction and Task Design

AObench moves beyond the LatentQA paradigm—which was prone to text inversion—by introducing an on-policy, chain-of-thought–based QA dataset. The process includes:

1. Generating chain-of-thought rollouts from a target model.
2. Selecting random indices in the token stream and partitioning prefix/suffix.
3. Employing a strong LLM to generate questions about the suffix ($-$2) that cannot be answered by the prefix ($-$3) alone, enforcing that questions are latent-activation dependent and not mere restatements of the text.

Layer selection and feeding strategies are calibrated empirically (e.g., layers 21–25 yield best aggregate scores for the Qwen3-8B target model). The dataset incorporates on-policy lens data, aligning the pre-training objective (predicting tokens adjacent to the activation window) with the actual AO deployment context. These choices directly uplift the chance-adjusted AObench score and reduce confounds (Bauer et al., 23 May 2026).

5. Key Benchmarking Results and Ablations

Successive enhancements to AO training and evaluation are quantified on AObench through an ablation ladder:

1. Conversational dataset integration: +0.066 uplift (chance-adjusted)
2. On-policy lens data: +0.030
3. Multi-layer activation feeding: +0.038
4. 2× steering strength in injection formula: +0.003

The full improved recipe achieves a +0.435 chance-adjusted AObench score, compared to +0.244 for the baseline (Karvonen et al., 2025). Hallucination recall at 95% precision rises from 68.8% to 84.6%, and vagueness scores improve from 0.076 to 0.205 (chance-adjusted). No singular task dominates the improvement; uplift is observed broadly across the suite. Notably, consensus sampling at inference serves as an effective, training-free defense against open-ended hallucinations, providing finely controllable precision-recall trade-offs (Bauer et al., 23 May 2026).

6. Software Structure and Usage

AObench is open-source and container-friendly, enabling reproducible benchmarking:

• Directory aobench/ contains tasks, metrics, and the main runner.
• Example usage: $-$4
• Dependencies: python≥3.9, PyTorch, transformers, accelerate, datasets.
• Models and evaluation sets are released via Hugging Face, allowing for rapid checkpoint swapping and task extension.

This modularity underpins its adoption as a de facto standard for end-to-end interpretability evaluation via activation-based question answering (Bauer et al., 23 May 2026).

7. Significance and Future Directions

AObench formalizes the empirical study of Activation Oracles by rigorously controlling for confounds and guiding training and evaluation towards genuine interpretive success. The primary leverage for improved AO performance is the use of a conversation-centric, on-policy, activation-conditioned QA dataset; multi-layer feeding and consensus sampling yield additional practical gains. The benchmark directly informs the development of AOs, reference datasets, injection strategies, and holds promise for evaluating future architectures centered on LLM interpretability. A plausible implication is that techniques validated on AObench will improve both the transparency and reliability of mechanistic interpretability pipelines based on learned oracles (Bauer et al., 23 May 2026).