---
title: Overlap-Aware Sheaf Repulsion in LLMs
url: https://www.emergentmind.com/topics/overlap-aware-sheaf-repulsion-oasr
type: topic
---

# Overlap-Aware Sheaf Repulsion in LLMs

Overlap-Aware Sheaf Repulsion (OASR) is a methodological framework for the discovery of multiple, structurally distinct mechanistic explanations—termed “sheaves” or “circuits”—for the same behavior in large language models (LLMs). Standard circuit/sheaf discovery (CSD) procedures often assume the existence of a unique or near-unique underlying circuit responsible for a given function. OASR introduces a mechanism for repelling overlap between discovered sheaves across multiple runs, providing systematic evidence against the uniqueness assumption and offering empirical and theoretical tools for the characterization of functionally-equivalent, low-overlap circuits [2605.12671].

## 1. Formal Definition and Objective

OASR augments existing CSD loss functions (such as those used in DiscoGP) with a penalty term that explicitly discourages structural overlap between repeatedly discovered sheaves. Let $L_{\mathrm{CSD}}(\ell)$ denote the base circuit/sheaf discovery loss, which typically comprises:

- $L_{\mathrm{fidelity}}$: task-specific loss (e.g., cross-entropy on masked graphs)
- $L_{\mathrm{sparsity}}$: regularizer penalizing expected number of kept edges
- $L_{\mathrm{complete}}$: (optional) regularizer promoting completeness

The OASR objective for each run, parameterized by mask logits $\{\ell_e\}_{e\in E}$, is

\[
L_{\mathrm{OASR}}(\ell) = L_{\mathrm{CSD}}(\ell) + \lambda_{\mathrm{overlap}} R_{\mathrm{overlap}}(\ell; \mathcal{R})
\]

with $\lambda_{\mathrm{overlap}}$ controlling the strength of the overlap penalty, and $\mathcal{R} = \{R^{(1)}, \dots, R^{(k-1)}\}$ being the edge-sets of prior discovered sheaves. The expanded formulation is

\[
L_{\mathrm{OASR}} = L_{\mathrm{fidelity}} + \alpha L_{\mathrm{sparsity}} + \beta L_{\mathrm{complete}} + \lambda_{\mathrm{overlap}} R_{\mathrm{overlap}}(\ell; \mathcal{R})
\]

where $\alpha$ and $\beta$ modulate sparsity and completeness.

## 2. Overlap Penalty: Definition and Implementation

Structural overlap is quantified at the edge level. Each edge $e$ is associated with a continuous “keep” probability $\sigma(\ell_e)$ (via Gumbel-Sigmoid/Straight-Through estimators). Given $\mathcal{R} = \{R^{(i)}\}$, with $R^{(i)} \subseteq E$ the edge-set of the $i$th discovered sheaf, the overlap penalty is defined as

\[
R_{\mathrm{overlap}}(\ell; \mathcal{R}) = \sum_{i=1}^{k-1} \sum_{e \in R^{(i)}} \sigma(\ell_e) = \sum_{e \in E} \left| \{i : e \in R^{(i)}\} \right| \sigma(\ell_e)
\]

This penalty discourages reuse of edges activated in prior runs, thereby minimizing the intersection over union (IoU) between the current and previous sheaves. High $\lambda_{\mathrm{overlap}}$ enforces repulsion more strongly, potentially at the expense of task fidelity.

## 3. OASR Optimization Procedure

The OASR discovery routine trains a sequence of mask parameters to recover $M$ sheaves with minimized structural overlap. The process for each run (indexed by $k$) is as follows:

1. Initialize mask logits $\ell_e \sim \mathcal{N}(0, 1)$ for all $e \in E$.
2. Iteratively optimize the loss $L_{\mathrm{OASR}}$:
   - Sample binary masks $m_e$ using Gumbel-Sigmoid$(\ell_e)$.
   - Run the model with the subset of active edges and compute model logits.
   - Evaluate $L_{\mathrm{fidelity}}$ on the task.
   - Compute $L_{\mathrm{sparsity}}$ and $L_{\mathrm{complete}}$ (if enabled).
   - Compute $R_{\mathrm{overlap}}$ given all previously discovered sheaves.
   - Aggregate the total loss and perform gradient descent on $\ell_e$.
3. After convergence, define $R^{(k)} = \{e : \sigma(\ell_e) > 0.5\}$ as the $k$th discovered sheaf.
4. Append $R^{(k)}$ to $\mathcal{R}$ and repeat until $M$ sheaves have been discovered.

Key points include the use of gradients with respect to $\ell_e$ to push probabilities downward for previously used edges and the guarantee that each run’s loss function encodes repulsion from all prior sheaves.

## 4. Hyperparameter Regimes and Empirical Effects

Key hyperparameters in OASR include:

| Hyperparameter         | Role                                         | Empirical Setting                                              |
|-----------------------|----------------------------------------------|---------------------------------------------------------------|
| $\lambda_{\mathrm{overlap}}$ | Overlap repulsion strength                   | Range [0.1, 1.0] halves IoU with ≤1 pt accuracy loss           |
| $\alpha, \beta$         | Sparsity, completeness balancers (as in DiscoGP) | Chosen as in the base CSD method                              |
| $\tau$                | Gumbel-Sigmoid temperature                   | Typically 0.5–1.0                                             |

$\lambda_{\mathrm{overlap}} = 0$ reduces to standard CSD, resulting in high overlap across seeds. Excessive $\lambda_{\mathrm{overlap}}$ can induce nearly disjoint circuits but may compromise fidelity. Practical selection of $\lambda_{\mathrm{overlap}}$ targets a fidelity loss below 1–2 percentage points.

## 5. Empirical Findings and Metrics

OASR has been evaluated on the following models and benchmarks:

- **Models:** GPT-2 Small (12 layers, 12 heads), Pythia-160M
- **Benchmarks:** IOI (Indirect Object Identification), BLiMP minimal pairs (six syntax paradigms), AGA/ANA (pronoun agreement), DNAvocab (number/gender agreement), Python docstring completion

**Metrics used:**
- **Fidelity:** Task accuracy on masked model
- **Sparsity:** Edge density $= |$selected edges$| / |$all edges$|$
- **Completeness:** Accuracy of removed edges’ complement
- **Overlap:** Pairwise IoU of edge sets

Salient results include:
- On IOI, two OASR-discovered sheaves each achieve 100% accuracy, edge density $\sim$ 3.5%, and IoU $\approx$ 4%.
- For $M = 20$ runs, mutual IoU falls below 1% (versus $\sim$1–3% by random initialization).
- An 86.7%-accurate three-edge sheaf for IOI is found (edges: Input$\to$MLP$_0$, MLP$_0\to$Attn$_{10}$.H$_7$.V, Attn$_{10}$.H$_7\to$Output), none of whose edges is individually indispensable (removal of any edge yields $>$99.8% accuracy).
- The phenomenon persists across major CSD methods (ACDC, EAP, EP, DiscoGP + OASR).

## 6. Theoretical Existence: Distributive Dense Circuit Hypothesis

The Distributive Dense Circuit Hypothesis asserts that, for any LLM task and under mild local-linearity assumptions, there exist multiple, structurally distinct, low-overlap circuits that are simultaneously faithful—within an $\epsilon$ tolerance—to the same task behavior. The proof sketch is as follows:

1. Express circuit-pruned logits as a subset sum of edge contributions plus a small residual, per local linearization.
2. The number of size-$s$ edge subsets scales combinatorially ($\binom{E}{s}$), whereas the output logit space is only polynomially bounded. By the pigeonhole principle, distinct subsets must collide within a small logit margin $\epsilon$.
3. If the full-model logit margin is at least $\delta$ and $\epsilon<\delta/2$, the colliding subsets yield identical top-1 predictions and thus perfect fidelity.
4. There exist colliding pairs with arbitrary low overlap, controlled via subset selection.

This establishes the non-uniqueness of sparse, faithful explanations as a consequence of high-dimensional superposition, not merely a feature of discovery algorithms.

## 7. Implications for Mechanistic Interpretability

OASR demonstrates that mechanistic explanations in LLMs are inherently non-canonical: no single “the circuit” exists. Instead, the outcome of circuit or sheaf discovery must be interpreted as a sample from a manifold of functionally-equivalent mechanisms. Circuit-discovery studies should report overlap and variability across runs, in addition to standard metrics of sparsity and fidelity. The presence of ultra-sparse, non-indispensable circuits challenges established notions of canonical structure in LLMs.

Future research directions include characterizing the full space of competing circuits, devising evaluation protocols robust to non-uniqueness, and exploring the dynamics of circuit multiplicity under distribution shift and fine-tuning [2605.12671]. A plausible implication is that mechanistic interpretability will need to accommodate the plurality and context-dependence of circuit realizations in complex deep learning systems.

Source: https://www.emergentmind.com/topics/overlap-aware-sheaf-repulsion-oasr