Overlap-Aware Sheaf Repulsion in LLMs

• The paper introduces an OASR framework that systematically reduces overlap between discovered sheaves, challenging the assumption of a unique circuit in LLMs.
• OASR augments standard circuit discovery loss functions with an overlap penalty using techniques like Gumbel-Sigmoid optimization to isolate distinct, low-overlap circuits.
• Empirical results demonstrate that OASR maintains high task accuracy while revealing multiple non-indispensable, sparse circuits, reshaping mechanistic interpretability in LLMs.

Overlap-Aware Sheaf Repulsion (OASR) is a methodological framework for the discovery of multiple, structurally distinct mechanistic explanations—termed “sheaves” or “circuits”—for the same behavior in LLMs. Standard circuit/sheaf discovery (CSD) procedures often assume the existence of a unique or near-unique underlying circuit responsible for a given function. OASR introduces a mechanism for repelling overlap between discovered sheaves across multiple runs, providing systematic evidence against the uniqueness assumption and offering empirical and theoretical tools for the characterization of functionally-equivalent, low-overlap circuits (Chen et al., 12 May 2026).

1. Formal Definition and Objective

OASR augments existing CSD loss functions (such as those used in DiscoGP) with a penalty term that explicitly discourages structural overlap between repeatedly discovered sheaves. Let $L_{\mathrm{CSD}}(\ell)$ denote the base circuit/sheaf discovery loss, which typically comprises:

• $L_{\mathrm{fidelity}}$: task-specific loss (e.g., cross-entropy on masked graphs)
• $L_{\mathrm{sparsity}}$: regularizer penalizing expected number of kept edges
• $L_{\mathrm{complete}}$: (optional) regularizer promoting completeness

The OASR objective for each run, parameterized by mask logits $\{\ell_e\}_{e\in E}$, is

$$L_{\mathrm{OASR}}(\ell) = L_{\mathrm{CSD}}(\ell) + \lambda_{\mathrm{overlap}} R_{\mathrm{overlap}}(\ell; \mathcal{R})$$

with $\lambda_{\mathrm{overlap}}$ controlling the strength of the overlap penalty, and $\mathcal{R} = \{R^{(1)}, \dots, R^{(k-1)}\}$ being the edge-sets of prior discovered sheaves. The expanded formulation is

$$L_{\mathrm{OASR}} = L_{\mathrm{fidelity}} + \alpha L_{\mathrm{sparsity}} + \beta L_{\mathrm{complete}} + \lambda_{\mathrm{overlap}} R_{\mathrm{overlap}}(\ell; \mathcal{R})$$

where $\alpha$ and $L_{\mathrm{fidelity}}$0 modulate sparsity and completeness.

2. Overlap Penalty: Definition and Implementation

Structural overlap is quantified at the edge level. Each edge $L_{\mathrm{fidelity}}$1 is associated with a continuous “keep” probability $L_{\mathrm{fidelity}}$2 (via Gumbel-Sigmoid/Straight-Through estimators). Given $L_{\mathrm{fidelity}}$3, with $L_{\mathrm{fidelity}}$4 the edge-set of the $L_{\mathrm{fidelity}}$5th discovered sheaf, the overlap penalty is defined as

$L_{\mathrm{fidelity}}$6

This penalty discourages reuse of edges activated in prior runs, thereby minimizing the intersection over union (IoU) between the current and previous sheaves. High $L_{\mathrm{fidelity}}$7 enforces repulsion more strongly, potentially at the expense of task fidelity.

3. OASR Optimization Procedure

The OASR discovery routine trains a sequence of mask parameters to recover $L_{\mathrm{fidelity}}$8 sheaves with minimized structural overlap. The process for each run (indexed by $L_{\mathrm{fidelity}}$9) is as follows:

1. Initialize mask logits $L_{\mathrm{sparsity}}$0 for all $L_{\mathrm{sparsity}}$1.
2. Iteratively optimize the loss $L_{\mathrm{sparsity}}$2:
   • Sample binary masks $L_{\mathrm{sparsity}}$3 using Gumbel-Sigmoid$L_{\mathrm{sparsity}}$4.
   • Run the model with the subset of active edges and compute model logits.
   • Evaluate $L_{\mathrm{sparsity}}$5 on the task.
   • Compute $L_{\mathrm{sparsity}}$6 and $L_{\mathrm{sparsity}}$7 (if enabled).
   • Compute $L_{\mathrm{sparsity}}$8 given all previously discovered sheaves.
   • Aggregate the total loss and perform gradient descent on $L_{\mathrm{sparsity}}$9.
3. After convergence, define $L_{\mathrm{complete}}$0 as the $L_{\mathrm{complete}}$1th discovered sheaf.
4. Append $L_{\mathrm{complete}}$2 to $L_{\mathrm{complete}}$3 and repeat until $L_{\mathrm{complete}}$4 sheaves have been discovered.

Key points include the use of gradients with respect to $L_{\mathrm{complete}}$5 to push probabilities downward for previously used edges and the guarantee that each run’s loss function encodes repulsion from all prior sheaves.

4. Hyperparameter Regimes and Empirical Effects

Key hyperparameters in OASR include:

Hyperparameter	Role	Empirical Setting
$L_{\mathrm{complete}}$6	Overlap repulsion strength	Range [0.1, 1.0] halves IoU with ≤1 pt accuracy loss
$L_{\mathrm{complete}}$7	Sparsity, completeness balancers (as in DiscoGP)	Chosen as in the base CSD method
$L_{\mathrm{complete}}$8	Gumbel-Sigmoid temperature	Typically 0.5–1.0

$L_{\mathrm{complete}}$9 reduces to standard CSD, resulting in high overlap across seeds. Excessive $\{\ell_e\}_{e\in E}$0 can induce nearly disjoint circuits but may compromise fidelity. Practical selection of $\{\ell_e\}_{e\in E}$1 targets a fidelity loss below 1–2 percentage points.

5. Empirical Findings and Metrics

OASR has been evaluated on the following models and benchmarks:

• Models: GPT-2 Small (12 layers, 12 heads), Pythia-160M
• Benchmarks: IOI (Indirect Object Identification), BLiMP minimal pairs (six syntax paradigms), AGA/ANA (pronoun agreement), DNAvocab (number/gender agreement), Python docstring completion

Metrics used:

• Fidelity: Task accuracy on masked model
• Sparsity: Edge density $\{\ell_e\}_{e\in E}$2selected edges$\{\ell_e\}_{e\in E}$3all edges$\{\ell_e\}_{e\in E}$4
• Completeness: Accuracy of removed edges’ complement
• Overlap: Pairwise IoU of edge sets

Salient results include:

• On IOI, two OASR-discovered sheaves each achieve 100% accuracy, edge density $\{\ell_e\}_{e\in E}$5 3.5%, and IoU $\{\ell_e\}_{e\in E}$6 4%.
• For $\{\ell_e\}_{e\in E}$7 runs, mutual IoU falls below 1% (versus $\{\ell_e\}_{e\in E}$81–3% by random initialization).
• An 86.7%-accurate three-edge sheaf for IOI is found (edges: Input$\{\ell_e\}_{e\in E}$9MLP$$L_{\mathrm{OASR}}(\ell) = L_{\mathrm{CSD}}(\ell) + \lambda_{\mathrm{overlap}} R_{\mathrm{overlap}}(\ell; \mathcal{R})$$0, MLP$$L_{\mathrm{OASR}}(\ell) = L_{\mathrm{CSD}}(\ell) + \lambda_{\mathrm{overlap}} R_{\mathrm{overlap}}(\ell; \mathcal{R})$$1Attn$$L_{\mathrm{OASR}}(\ell) = L_{\mathrm{CSD}}(\ell) + \lambda_{\mathrm{overlap}} R_{\mathrm{overlap}}(\ell; \mathcal{R})$$2.H$$L_{\mathrm{OASR}}(\ell) = L_{\mathrm{CSD}}(\ell) + \lambda_{\mathrm{overlap}} R_{\mathrm{overlap}}(\ell; \mathcal{R})$$3.V, Attn$$L_{\mathrm{OASR}}(\ell) = L_{\mathrm{CSD}}(\ell) + \lambda_{\mathrm{overlap}} R_{\mathrm{overlap}}(\ell; \mathcal{R})$$4.H$$L_{\mathrm{OASR}}(\ell) = L_{\mathrm{CSD}}(\ell) + \lambda_{\mathrm{overlap}} R_{\mathrm{overlap}}(\ell; \mathcal{R})$$5Output), none of whose edges is individually indispensable (removal of any edge yields $$L_{\mathrm{OASR}}(\ell) = L_{\mathrm{CSD}}(\ell) + \lambda_{\mathrm{overlap}} R_{\mathrm{overlap}}(\ell; \mathcal{R})$$699.8% accuracy).
• The phenomenon persists across major CSD methods (ACDC, EAP, EP, DiscoGP + OASR).

6. Theoretical Existence: Distributive Dense Circuit Hypothesis

The Distributive Dense Circuit Hypothesis asserts that, for any LLM task and under mild local-linearity assumptions, there exist multiple, structurally distinct, low-overlap circuits that are simultaneously faithful—within an $$L_{\mathrm{OASR}}(\ell) = L_{\mathrm{CSD}}(\ell) + \lambda_{\mathrm{overlap}} R_{\mathrm{overlap}}(\ell; \mathcal{R})$$7 tolerance—to the same task behavior. The proof sketch is as follows:

1. Express circuit-pruned logits as a subset sum of edge contributions plus a small residual, per local linearization.
2. The number of size-$$L_{\mathrm{OASR}}(\ell) = L_{\mathrm{CSD}}(\ell) + \lambda_{\mathrm{overlap}} R_{\mathrm{overlap}}(\ell; \mathcal{R})$$8 edge subsets scales combinatorially ($$L_{\mathrm{OASR}}(\ell) = L_{\mathrm{CSD}}(\ell) + \lambda_{\mathrm{overlap}} R_{\mathrm{overlap}}(\ell; \mathcal{R})$$9), whereas the output logit space is only polynomially bounded. By the pigeonhole principle, distinct subsets must collide within a small logit margin $\lambda_{\mathrm{overlap}}$0.
3. If the full-model logit margin is at least $\lambda_{\mathrm{overlap}}$1 and $\lambda_{\mathrm{overlap}}$2, the colliding subsets yield identical top-1 predictions and thus perfect fidelity.
4. There exist colliding pairs with arbitrary low overlap, controlled via subset selection.

This establishes the non-uniqueness of sparse, faithful explanations as a consequence of high-dimensional superposition, not merely a feature of discovery algorithms.

7. Implications for Mechanistic Interpretability

OASR demonstrates that mechanistic explanations in LLMs are inherently non-canonical: no single “the circuit” exists. Instead, the outcome of circuit or sheaf discovery must be interpreted as a sample from a manifold of functionally-equivalent mechanisms. Circuit-discovery studies should report overlap and variability across runs, in addition to standard metrics of sparsity and fidelity. The presence of ultra-sparse, non-indispensable circuits challenges established notions of canonical structure in LLMs.

Future research directions include characterizing the full space of competing circuits, devising evaluation protocols robust to non-uniqueness, and exploring the dynamics of circuit multiplicity under distribution shift and fine-tuning (Chen et al., 12 May 2026). A plausible implication is that mechanistic interpretability will need to accommodate the plurality and context-dependence of circuit realizations in complex deep learning systems.