Functional Anisotropy Hypothesis in Complex Systems

• The Functional Anisotropy Hypothesis is a framework describing how task-relevant function emerges from distributed, directionally preferential organization in high-dimensional systems.
• It is mathematically formalized by demonstrating that multiple, sparse, and faithful circuits can coexist, with overlap measures as low as 4–11% across independent discoveries.
• The hypothesis has broad applications, elucidating energy landscapes in crystalline materials and guiding dual-metric evaluations of tissue mechanics in developmental biology.

The Functional Anisotropy Hypothesis (FAH) encompasses a set of domain-specific propositions regarding the distribution and significance of anisotropy in complex systems—ranging from mechanistic explanations in LLMs, to the energetics of grain boundaries in crystalline materials, to the mechanical transitions in developing epithelial tissues. Though articulated in diverse scientific contexts, these hypotheses share a unifying theme: that task-relevant function or energy cannot be fully understood by reference to isotropic, uniquely localised mechanisms, but rather emerges through structured, directionally preferential organization in their respective high-dimensional spaces.

1. Functional Anisotropy in Mechanistic Interpretability of Neural Networks

FAH has been most rigorously interrogated in the mechanistic interpretability of LLMs, where it originally posited that each function or capability—such as indirect object identification (IOI)—is realized by a unique or near-unique internal sub-mechanism, often conceptualized as a "circuit" or "sheaf." This form of anisotropy asserts that capability is localized and non-redundant: the discovery of a single, sparse subgraph is held to constitute the one true mechanistic explanation for the behavior in question. This assumption is embedded in the design and evaluation of circuit and sheaf discovery (CSD) benchmarks, which reward minimal, high-fidelity subgraphs as canonical explanations.

Recent work has comprehensively falsified this strong FAH by demonstrating that not only can multiple, structurally distinct circuits or sheaves be uncovered for the same task, but that these can be simultaneously sparse, faithful, and complete. Across common interpretability benchmarks (IOI, BLiMP, AGA, ANA, various number agreement tasks, Docstring), the mutual intersection of independently discovered sheaves is often below 1% of the constituent edges, even when each sheaf achieves close to perfect task accuracy. Experimentally, the introduction of Overlap-Aware Sheaf Repulsion (OASR), wherein an explicit penalty is placed upon structural overlap between circuits discovered in multiple independent runs, reliably yields collections of circuits with pairwise Intersection-over-Union (IoU) values in the 4–11% range, supporting the proposition that mechanisms are not unique but plural (Chen et al., 12 May 2026).

2. Mathematical and Algorithmic Formalization of Non-Uniqueness

The empirical refutation of the unique-circuit variant of FAH is formalized via the Distributive Dense Circuit Hypothesis. Under a residual-additive edge model—where an LLM’s forward computation is a directed acyclic graph of edges each injecting a vector additive to the residual stream—the space of task-faithful circuits is combinatorially vast. Theoretical analysis demonstrates, under broad assumptions (local linearisation on-distribution, margin-stability for accuracy), that for any sufficiently overcomplete set of circuit candidates, there exist numerous, arbitrarily low-overlap subgraphs that yield indistinguishable outputs on the task of interest. The proof leverages a counting argument: if the number of possible size-$s$ circuits exceeds the number of coarse-grid bins in the quantized output space, two structurally distinct circuits must coincide in behavior to within any arbitrarily small threshold, and the overlap between their edge sets can be driven arbitrarily low. Thus, non-uniqueness and distributive realization of function are mathematical inevitabilities in sufficiently high-dimensional, overparameterized models (Chen et al., 12 May 2026).

The OASR method operationalizes this idea in discovery algorithms: given a CSD objective $\mathcal{L}_{\mathrm{CSD}}$, the loss is augmented with a repulsion term summing $\operatorname{IoU}(S_i, S_j)$ over all pairs of sheaves across $R$ runs,

$$\mathcal{L}_{\mathrm{OASR}} = \sum_{r=1}^R \mathcal{L}_{\mathrm{CSD}}(S_r) + \lambda \sum_{1\le i<j\le R} \operatorname{IoU}(S_i, S_j).$$

This approach systematically drives discovery towards sets of high-fidelity, sparse, minimally overlapping sheaves, substantiating the theoretical non-uniqueness with concrete algorithmic output.

3. Refutations and Limitations of Canonical Explanations

Evidence against FAH extends beyond OASR. Multiple circuit discovery algorithms—including ablation-based pruning (ACDC), gradient-attribution pruning (EAP), and optimization-based pruning (EP)—demonstrate radical sensitivity of the discovered circuit to minor method-irrelevant perturbations, with disparate circuits yielding equivalent performance. The intersection of twenty independently discovered IOI circuits produces an ultra-sparse three-edge sheaf which attains 87% accuracy in isolation; yet none of its component edges are individually indispensable, as ablating any leaves task accuracy at ~99%, and forbidding all three does not preclude new high-accuracy, sparse circuits from emerging. This systematically repudiates even weakened notions of canonical, essential components and demonstrates that the search for unique mechanistic causes is fundamentally ill-posed in overcomplete neural systems (Chen et al., 12 May 2026).

4. Functional Anisotropy in Condensed Matter: Grain Boundary Energy Landscapes

FAH also arises in condensed matter physics, notably in the description of grain boundary (GB) energies in face-centered cubic (FCC) metals. Here, the hypothesis asserts that the five-dimensional landscape of GB energy, $\sigma$, is characterized by low-dimensional, locally minimal structures—termed "grofs"—imbedded within the space of macroscopic crystallographic degrees of freedom (three for misorientation, two for boundary-plane inclination). Each k-grof is a contiguous $(N-k)$-dimensional subset of this space, smooth along those directions but exhibiting cusp-like minima orthogonal to the subset. The intersection of grofs (e.g., at high-coincidence misorientations or coherent twins) produces true cusps in the energy landscape.

A closed-form function, parameterized by just two material-specific parameters and informed by symmetry and group-theoretic topology, quantitatively reproduces the anisotropic structure of this energy landscape to within 5% RMS accuracy across FCC metals. This universality and the hierarchical construction of the interpolation function from grofs, coupled with the dominance of low-dimensional, directionally distinct minima, embody the FAH in the energetic context (Bulatov et al., 2013).

5. FAH in Developmental Biology: Tissue Mechanics and Solid-Fluid Transitions

In the context of morphogenesis during embryonic development, specifically convergent extension in the Drosophila germband, FAH is articulated as a two-metric rule for predicting transitions between solid-like (jammed) and fluid-like (unjammed) tissue behavior. Contrary to isotropic vertex models, where a single cell shape index $p$ (mean perimeter normalized by area) dictates the rigidity transition, FAH asserts that in anisotropic tissues, transitions are governed by both $p$ and a nematic cell alignment index $Q$. Here, $\mathcal{L}_{\mathrm{CSD}}$0 quantifies the extent of alignment of cell elongations across the tissue, thus capturing emergent collective anisotropy.

The transition threshold is given by

$\mathcal{L}_{\mathrm{CSD}}$1

where $\mathcal{L}_{\mathrm{CSD}}$2 (depending on disorder) and $\mathcal{L}_{\mathrm{CSD}}$3 is derived empirically. This theory, validated by time-lapse imaging and quantitative analysis, shows that neither $\mathcal{L}_{\mathrm{CSD}}$4 nor $\mathcal{L}_{\mathrm{CSD}}$5 alone suffices to predict the onset of rapid cell rearrangement in anisotropic developmental contexts. Thus, FAH identifies functional anisotropy in both quantitative metrics and underlying mechanical principles, necessitating a move from one-dimensional to two-dimensional phase diagrams in tissue mechanobiology (Wang et al., 2020).

6. Implications for Model Evaluation, Interpretation, and Universality

The collective refutation of unique-mechanism variants of FAH in neural systems establishes major implications for methodology and epistemology in mechanistic interpretability. Discovered circuits and sheaves should no longer be interpreted as singularly privileged explanations, but as points in a vast—and necessarily plural—space of faithful, sparse, and complete computational mechanisms. Evaluation metrics in circuit and sheaf discovery should be reoriented away from minimality and uniqueness towards diversity, overlap-aware quantification, and class-level recurrence of functional components.

In the materials context, universality and predictive accuracy arise not from exhaustive parameterization, but from the alignment of the model structure with the symmetry-constrained, anisotropic landscapes codified by grofs. For developmental processes, predictive understanding of mechanical transitions necessitates the joint consideration of cell shape and alignment, generalizing the isotropic paradigm to anisotropic, collectively ordered tissues.

A plausible implication is that FAH—in its plural, structural, and metric formulations—serves not as a universal law but as a lens revealing the necessity of directionally preferential, distributed organization in high-dimensional, overcomplete, or complexly constrained systems. The practical consequence is a shift from reductionist, uniquely localized explanations to an integrative recognition of distributed, overlapping, and context-dependent function.