---
title: Overlap-Aware Sheaf Repulsion in LLMs
url: https://www.emergentmind.com/topics/overlap-aware-sheaf-repulsion
type: topic
---

# Overlap-Aware Sheaf Repulsion in LLMs

Overlap-Aware Sheaf Repulsion (OASR) is a method for systematically uncovering multiple, structurally distinct, yet functionally faithful and sparse mechanistic explanations—termed “sheaves”—for tasks performed by large language models (LLMs). OASR augments standard circuit and sheaf discovery (CSD) objectives with an explicit penalty on structural overlap between successively found sheaves, enabling the empirical extraction of numerous low-overlap, high-fidelity circuits. The resulting findings challenge the Functional Anisotropy Hypothesis: the long-standing, often implicit assumption that such tasks are implemented by unique or near-unique internal mechanisms within LLMs [2605.12671].

## 1. Foundations: Definitions and Notations

Within Transformer architectures, let the residual-stream computation graph be denoted as $G=(V,E)$, where $V$ is the set of components—typically attention heads ($h$), MLP blocks ($m$), input, and output nodes—and $E \subseteq V \times V$ are directed edges representing information flow.

- **Circuit**: A subgraph $C \subseteq E$ functions as a “circuit” for task $T$ if ablating all edges in $E \setminus C$ causes only mild degradation in performance, i.e., the masked model’s fidelity $f_C \approx f_0$.
- **Sheaf**: A sheaf $S \subseteq E$ is a circuit endowed with standalone faithfulness: when only edges in $S$ are active, with all others set to zero, task performance remains high.
- **Edge Scores/Masks**: Each edge $e \in E$ has a continuous score $s_e \in [0,1]$ (obtained via a sigmoid of a learnable logit $l_e$) and a binary mask $m_e \in \{0,1\}$, sampled via the straight-through Gumbel-Sigmoid estimator.
- **Structural Overlap**: For two edge sets $S_i, S_j$, overlap is quantified as $\operatorname{IoU}(S_i, S_j) = \frac{|S_i\cap S_j|}{|S_i \cup S_j|}$. Alternative measures include $\Omega(S_i,S_j) = \frac{|S_i\cap S_j|}{|E|}$ and continuous-score-based $\Omega = \sum_{e} s_e^{(i)} s_e^{(j)}$.

## 2. Overlap-Aware Objective Formulation

The standard objective for sheaf discovery, inspired by DiscoGP, seeks to minimize:

\[
L_{\text{orig}} = L_{\text{fidelity}} + \lambda_s L_{\text{sparsity}} + \lambda_c L_{\text{complete}}
\]

where
- $L_{\text{fidelity}} = CE(\text{model}_S(x), y)$ evaluates the cross-entropy loss of the masked model on task labels.
- $L_{\text{sparsity}} = (1/|E|) \sum_{e\in E} \sigma(l_e)$ enforces edge-level sparsity.
- $L_{\text{complete}} = CE(\text{model}_{E\setminus S}(x), y)$ penalizes completeness, ensuring the complement does not solve the task.

OASR introduces an overlap penalty:

\[
L_{\text{overlap}}(R_{<j}) = \frac{1}{|E|} \sum_{i<j}\sum_{e\in R_i} \sigma(l_e)
\]

For the $j$th discovery run, the new objective becomes

\[
L_{\text{repel}} = L_{\text{fidelity}} + \lambda_s L_{\text{sparsity}} + \lambda_c L_{\text{complete}} + \lambda_o L_{\text{overlap}}
\]

where $\lambda_o$ governs the strength of repulsion from prior sheaves $R_1,\ldots,R_{j-1}$.

## 3. Algorithmic Protocol

The OASR methodology operates by iteratively discovering distinct sheaves through gradient-based optimization:

- **Initialization**: For each run $j\in 1,\ldots,K$, initialize $\{l_e\}$ and Gumbel sampling seeds randomly.
- **Optimization**: Minimize $L_{\text{repel}}$ using gradient descent (e.g., Adam), computing hard masks $m_e$ via the straight-through Gumbel-Sigmoid.
- **Extraction**: At convergence, extract $S_j = \{e : s_e > 0.5\}$ as the discovered sheaf.
- **Coordination**: After run $j$, append $S_j$ to the set of prior sheaves for overlap penalization in subsequent runs.

In practice, overlap penalties are efficiently computed using stored binary masks $R_i = \{e : m_e = 1\}$; penalties take the form $\sum_{e \in \cup R_i} s_e$ or via score–mask cross-terms $\sum_{i<j} \sum_e s_e^{(j)} m_e^{(i)}$.

## 4. Theoretical Insights: Distributive Dense Circuit Hypothesis

Under assumed local linearity and high-dimensional superposition within the model, the class logit $z_c(x)$ can be locally linearized:

\[
z_c(x) \approx \sum_{e \in E} \Delta_e(x)
\]

where $\Delta_e(x)$ is the contribution of edge $e$.

Due to the exponentially large number of edge subsets and only polynomially many distinguishable readout vectors (given a margin $\delta$), a subset-sum pigeonhole argument guarantees the existence of many structurally distinct edge sets $S_i \neq S_j$ such that

\[
\sum_{e \in S_i} \Delta_e \approx \sum_{e \in S_j} \Delta_e
\]

This result demonstrates that multiple low-overlap sheaves can achieve equivalently high fidelity for the same model and task, providing theoretical support for the empirical findings imposed by OASR.

## 5. Empirical Characterization

OASR has been extensively evaluated on a range of LLM circuit discovery benchmarks, including IOI (Indirect Object Identification), BLiMP (English grammaticality), AGA/ANA (agreement tasks), DNA variants (Dependency-Number Agreement), and Docstring (code completion).

### Benchmark Results

| Task                          | Edge Densities (%) | Completeness (%) | $\operatorname{IoU}(S_A, S_B)$ (%) |
|-------------------------------|--------------------|------------------|------------------------------------|
| IOI                           | 3.5 vs 4.0        | 45 vs 46         | 4–11                               |

Findings include:
- For each task, two sheaves $S_A$, $S_B$ can be discovered that both attain high accuracy (e.g., IOI: 100% vs 100%), with low edge overlap ($\operatorname{IoU}$ in the 4–11% range).
- Increasing the number of OASR-driven runs ($K = 20$) further shrinks the mutual intersection $|\cap_j S_j|$ to negligible values (tens of edges or less) while maintaining accuracy and sparsity.

### Ultra-Sparse Sheaf Discovery

By intersecting 20 OASR runs on IOI, an 11-edge sheaf core retains $>90\%$ accuracy; exhaustive search within this core yields a 3-edge sheaf $S_* = \{e_1, e_2, e_3\}$ achieving $86.7\%$ IOI accuracy on its own, with the complement performing at chance.

Indispensability analysis further demonstrates that ablating any single edge $e_k$ in $S_*$ does not significantly impact total task accuracy (leaving IOI accuracy $\approx$ 99.9%), and alternative discovery runs excluding $S_*$ still yield sparse, high-accuracy sheaves. This strongly undermines arguments for edge-level canonicity or uniqueness.

## 6. Implications and Reevaluation of Mechanistic Interpretability

The aggregate theoretical and experimental evidence provided by OASR demonstrates a “plethora” of alternative, low-overlap, causally effective sheaves underpinning LLM task performance. These results invalidate any strong or weak form of the Functional Anisotropy Hypothesis—the view that LLM functions are realized through unique or essential internal circuits. As different sheaves can be found that are both sparse and faithful, with little or no overlap and no indispensable components, mechanistic explanations in LLMs are inherently non-canonical.

A plausible implication is the need to fundamentally reconsider how circuit and sheaf discovery results are interpreted: rather than treating discovered explanations as unique or essential, future methodologies must acknowledge the inherent multiplicity and non-uniqueness illuminated by OASR [2605.12671].

Source: https://www.emergentmind.com/topics/overlap-aware-sheaf-repulsion