Neural Networks Secretly Build Symbolic Structures
This presentation reveals groundbreaking evidence that neural networks encode symbolic structure in their continuous vector representations. Through a novel method called DISCOVER, researchers demonstrate that networks trained on sequences, language, arithmetic, logic, and code construct internal representations that closely approximate symbolic role-filler bindings. By replacing entire representation processes with interpretable symbolic approximations and performing targeted causal interventions, the work shows these structures are not merely correlational but functionally drive model behavior across diverse symbolic tasks.Script
Neural networks dominate language, arithmetic, and logic, yet they represent everything as continuous vectors while these domains seem to demand discrete symbolic trees. How can floating-point coordinates possibly encode the hierarchical bindings that reverse a sequence or parse a sentence?
The authors propose that networks approximate tensor product representations. Each piece of content binds to a structural role through a tensor product, these bound pairs sum together, and a learned affine transformation maps the result into the network's actual vector space.
Their method, DISCOVER, fits this tensor product structure to a trained network's hidden states and then replaces the network's own representations with the fitted symbolic approximation. The test is behavioral: does the original decoder still produce the right answer when given the reconstructed symbolic structure instead of the network's native computation?
For models trained to reverse or interleave letter sequences, bidirectional tensor product approximations achieve reconstruction accuracies above 99% across multi-layer perceptrons, recurrent networks, and transformers. Crucially, structureless bag-of-words approximations fail completely, while unidirectional role schemes show task-dependent asymmetries: models solving reversal prefer right-to-left roles; models solving copying prefer left-to-right roles.
When large language models encode the period at the end of a sentence, that single vector contains enough information to reconstruct the entire preceding context with over 85% accuracy for lists and 50% for complex sentences. Remarkably, tensor product approximations sometimes decode more accurately than the original period vectors themselves, suggesting the networks build approximately symbolic but noisy representations that DISCOVER clarifies.
The strongest evidence comes from causal surgery: the authors subtract one role-filler binding from a network's representation and add another, then measure whether behavior changes as predicted. Across arithmetic, code execution, and syntax transformation, interventions achieve 90% accuracy on average. Swapping numbers in expressions, moving variables between function arguments, and relocating syntactic modifiers all produce the targeted behavioral changes, confirming these symbolic structures don't just correlate with behavior but causally drive it. Visit EmergentMind.com to explore more research that reveals the hidden mechanisms behind neural intelligence, and create your own videos explaining the work that matters to you.
