TopoNets: High Performing Vision and Language Models with Brain-Like Topography (2501.16396v1)

Abstract: Neurons in the brain are organized such that nearby cells tend to share similar functions. AI models lack this organization, and past efforts to introduce topography have often led to trade-offs between topography and task performance. In this work, we present TopoLoss, a new loss function that promotes spatially organized topographic representations in AI models without significantly sacrificing task performance. TopoLoss is highly adaptable and can be seamlessly integrated into the training of leading model architectures. We validate our method on both vision (ResNet-18, ResNet-50, ViT) and LLMs (GPT-Neo-125M, NanoGPT), collectively TopoNets. TopoNets are the highest-performing supervised topographic models to date, exhibiting brain-like properties such as localized feature processing, lower dimensionality, and increased efficiency. TopoNets also predict responses in the brain and replicate the key topographic signatures observed in the brain's visual and language cortices. Together, this work establishes a robust and generalizable framework for integrating topography into leading model architectures, advancing the development of high-performing models that more closely emulate the computational strategies of the human brain.

• The paper introduces TopoLoss, a novel loss function that induces brain-like topographic organization in both convolutional and transformer-based neural networks.
• It validates TopoNets on state-of-the-art architectures for vision and language, achieving superior performance on benchmarks like ImageNet classification and BLiMP evaluation with minimal trade-offs.
• By reducing effective dimensionality and enhancing parameter efficiency, the approach mirrors synaptic pruning, paving the way for scalable, brain-inspired artificial neural networks.

The paper "TopoNets: High performing vision and LLMs with brain-like topography" presents a novel approach to infusing topographic organization into artificial neural networks (ANNs) by leveraging a newly proposed loss function, termed TopoLoss. Drawing inspiration from neurobiological systems, the authors aim to replicate the brain's efficiency and localized processing properties in both vision and LLMs without compromising task performance.

Key Contributions

1. TopoLoss Introduction: The paper introduces TopoLoss, a novel loss function that can be effectively integrated into the training procedures of both convolutional neural networks and transformers. This loss promotes the formation of topographically organized representations—akin to the functional organization seen in brain cortices—by utilizing spatial constraints derived from brain-like synaptic pruning principles.
2. Model Architectures and Domains: The authors validate the effectiveness of TopoLoss by developing TopoNets, which are applied to several state-of-the-art model architectures for vision (ResNet-18, ResNet-50, and ViT-b32) and language (GPT-Neo-125M and NanoGPT). These models are claimed to be the highest performing supervised topographic models to date.
3. Performance with Minimal Trade-off: The paper shows that TopoNets achieve superior performance on benchmark tasks (e.g., ImageNet classification and BLiMP evaluation) while exhibiting comparable levels of topographic organization to previous topographic models. Specifically, TopoNets maintain similar or improved task accuracies compared to baseline models, such as ResNet-18 for vision and GPT-Neo-125M for language, thereby setting a new standard for supervised topographic models.
4. Inductive Bias Across Domains: TopoLoss is demonstrated to be highly adaptable, as it generalizes well across diverse domains and neural architectures, establishing a unified method for inducing topography in ANNs. This adaptability is a pivotal advancement over traditional, domain-specific approaches to topographic organization in machine learning models.
5. Dimensionality and Efficiency: Empirical analyses reveal that topography, not merely task performance, drives reductions in the effective dimensionality of the learned representations. TopoNets lead to more parameter-efficient models, as demonstrated through resilience to unstructured L1 pruning and improved performance retention when subjected to downsampling procedures.
6. Replicating Brain-Like Topography: The paper emphasizes that TopoNets successfully replicate various brain-like topographic signatures, such as category selectivity maps in the ventral visual cortex and temporal clustering in the language cortex. These models offer insights into potential computational parallels between the brain and ANNs.

Methodological Details

• Cortical Sheet Definition: The process begins by defining a cortical sheet for ANNs where each neural layer is reshaped into a multidimensional grid that simulates the topographic map found in brain tissues. This restructuring is crucial for applying TopoLoss effectively.
• Topographic Influence via Cosine Similarity: TopoLoss maximizes the similarity between the original cortical maps and their smoothed, or blurred, versions. This operation mirrors synaptic pruning's effect in reducing noisy neural connections to retain meaningful patterns.
• Scalability: The framework and associated models are scalable and can be adapted to more complex tasks and architectures with minimal adjustments to existing code bases. The optimization incorporates a minimal computational overhead, showcasing potential for broad application in the development of efficient AI systems.

Broader Implications

The research provides a compelling argument for the utility of brain-inspired topography in designing efficient and adaptable neural networks. By grounding their approach in neuroscience principles, the authors offer a bridge between biological cognitive functions and artificial intelligence, potentially enhancing the interpretability and computational efficiency of AI systems. The generalizability of TopoLoss across different architectures suggests a promising avenue for future research focused on leveraging biologically motivated techniques to improve machine learning models.