Group Crosscoders for Mechanistic Analysis of Symmetry (2410.24184v2)

Abstract: We introduce group crosscoders, an extension of crosscoders that systematically discover and analyse symmetrical features in neural networks. While neural networks often develop equivariant representations without explicit architectural constraints, understanding these emergent symmetries has traditionally relied on manual analysis. Group crosscoders automate this process by performing dictionary learning across transformed versions of inputs under a symmetry group. Applied to InceptionV1's mixed3b layer using the dihedral group $\mathrm{D}_{32}$, our method reveals several key insights: First, it naturally clusters features into interpretable families that correspond to previously hypothesised feature types, providing more precise separation than standard sparse autoencoders. Second, our transform block analysis enables the automatic characterisation of feature symmetries, revealing how different geometric features (such as curves versus lines) exhibit distinct patterns of invariance and equivariance. These results demonstrate that group crosscoders can provide systematic insights into how neural networks represent symmetry, offering a promising new tool for mechanistic interpretability.

• The paper’s main contribution is introducing group crosscoders to automatically extract and cluster symmetrical features in neural networks.
• The methodology uses dictionary learning on activation vectors from transformed inputs and cosine similarity to reveal distinct feature clusters.
• Experimental results on InceptionV1’s mixed3b layer clearly separate curvilinear and angular features, enhancing network interpretability.

Analysis of "Group Crosscoders for Mechanistic Analysis of Symmetry"

The paper "Group Crosscoders for Mechanistic Analysis of Symmetry" by Liv Gorton contributes to the understanding of symmetry in neural networks by introducing the concept of group crosscoders. This novel approach extends the traditional crosscoders, which were initially designed to find analogous features across neural network layers, enabling them to systematically explore and analyse the symmetry within neural networks.

Key Contributions

The core contribution of the paper is the introduction of group crosscoders. These are applied to neural networks to identify and cluster symmetrical features, providing a new layer of interpretability into how neural networks inherently develop certain symmetry properties even when these are not explicitly enforced. The approach is demonstrated on the InceptionV1 architecture's mixed3b layer using the dihedral group $\mathrm{D}_{32}$, a structured group with operations including rotations and reflections.

The novelty of group crosscoders lies in their ability to automatically extract and analyze equivariant features, without the need for predefined architectural constraints. By incorporating dictionary learning across transformed input versions under a symmetry group, these crosscoders not only reveal inherent symmetry but also cluster features into interpretable families. This clustering facilitates a more granular understanding of feature types, surpassing the resolution provided by standard sparse autoencoders.

Methodology

Group crosscoders build on the existing framework of crosscoders by learning from activation patterns of transformed inputs under group actions, instead of relying on multilayer or multimodel parallels. The paper conducts rigorous dictionary learning on vectors containing activations from multiple transformations of the same input image, a departure from the conventional vector representations in this context.

The group crosscoder's ability to predict transformed vector activations from untransformed inputs introduces a methodological innovation that aids interpretability. The training process involves constructing datasets from the ImageNet and adopting cosine similarity as a metric to measure feature symmetry through a distance matrix. A UMAP (Uniform Manifold Approximation and Projection) visualization was employed to represent the clustering of feature families effectively.

Experimental Results

The results demonstrate that group crosscoders can discern and classify features into distinct clusters corresponding to previously hypothesized feature families. The experiment with InceptionV1's mixed3b layer shows clear separation and interpretability across these clusters, distinguishing between, for example, curvilinear and angular features in neural network representations. The features’ symmetry analysis reveals how different geometric traits exhibit distinct transformation patterns—such as curves requiring a full 360\textdegree rotation for equivariance compared to lines’ 180\textdegree rotational symmetry.

Implications and Future Directions

The introduction of group crosscoders marks an advancement in mechanistic interpretability approaches. By automating the discovery and clustering of symmetrical features, this method reduces the ambiguity in understanding neural network feature representation, especially concerning innate equivariant properties. This advancement adds depth to ongoing research in vision interpretability by providing a structured approach to symmetry analysis, potentially applicable across modalities beyond image processing or for groups with more complex symmetries.

Future research can explore extending this methodology to groups beyond the dihedral, perhaps incorporating scaling transformations or colour transformations, thereby broadening the applicability and robustness of group crosscoders. Additionally, cross-examination with other architectures could compare the prevalence and nature of symmetrical feature representation across models, potentially influencing design principles in neural network development.

In conclusion, the development of group crosscoders offers an insightful tool for AI researchers looking to explore and quantify neural network symmetries. The methodology sets a precedent for future explorations in mechanistic interpretability, offering potential pathways for rich academic inquiry.