NeuroCogMap Reveals Cognitive Organization of Large Language Models
This presentation introduces NeuroCogMap, a cognitive-neuroscience-inspired framework that systematically maps the internal representations of large language models to interpretable cognitive functions. We explore how this multi-level atlas construction enables precise attribution of failure modes like hallucination and refusal, demonstrate its remarkable alignment with human cortical organization during language comprehension, and show how mechanism-guided interventions can robustly correct pathological behaviors across different model architectures.Script
Most language model interpretability research operates at extremes: either dissecting tiny circuits or describing high-level behaviors, with no systematic map connecting the two. NeuroCogMap bridges this gap by building the first comprehensive cognitive atlas of large language models, revealing how distributed internal representations organize into interpretable functional systems.
The framework organizes sparse autoencoder features into 270 functionally coherent parcels, each mapped to specific cognitive capabilities like arithmetic reasoning or entity retrieval. These capabilities are further structured into four hierarchical layers inspired by cognitive complexity: perception, representation, abstraction, and application. This multi-level organization is not random: it replicates across different model architectures and predicts activation patterns with statistical significance far exceeding baseline methods.
Hallucination emerges as a multi-level circuit disruption: in question-answering tasks, retrieval parcels fail to couple with evaluative control mechanisms, allowing misleading representations to dominate. NeuroCogMap detects these failures with 0.84 AUROC, outperforming entropy, perplexity, and self-consistency baselines. Targeted interventions that rebalance under-recruited and over-recruited parcels restore factual accuracy where generic methods fail.
Perhaps most striking is the partial isomorphism with human cortex: NeuroCogMap parcels predict brain activity during naturalistic language comprehension better than conventional language features. The strongest correspondences appear in association cortex, particularly in default mode, frontoparietal, and salience networks, suggesting these models have converged on representational strategies that echo human cognitive architecture.
The real power lies in mechanistic intervention: refusal failure, where models incorrectly comply with harmful requests, stems from a shift from negation circuitry to procedural execution modules. NeuroCogMap detects these failures with near-perfect accuracy and steers refusal rates from 38% to 99% in some settings. These improvements generalize across model families, including Llama 3.1, validating the framework's architectural independence.
NeuroCogMap establishes a new interpretability regime: coarse enough for cross-model generalization, granular enough for precise causal manipulation, and structured enough for principled comparison with human cognition. By moving beyond circuit minutiae and behavioral description to system-level cognitive organization, it opens paths for both safer artificial intelligence and deeper understanding of intelligence itself. To explore this framework further and create your own research videos, visit EmergentMind.com.
