- The paper demonstrates that semantic role understanding emerges during pre-training with significant F1 improvements from frozen representations.
- Key experiments using GPT-style models reveal that fine-tuning increasingly outperforms pre-training in larger scales, highlighting distributed neural encodings.
- Ablation studies indicate a shift from role-selective neurons in smaller models to redundant, interfering units in larger ones, motivating advanced interpretability methods.
Emergent Semantic Role Understanding in LLMs – An Expert Analysis
Motivation and Problem Statement
Understanding the implicit linguistic competencies of LLMs is pivotal for both interpretability and reliability. Semantic role understanding—the ability to parse predicate-argument structure or to answer "who did what to whom"—is foundational for meaning representation. Despite the well-documented ability of LLMs to perform downstream tasks with minimal supervision, it remains an open question to what extent semantic role encoding emerges during language modeling pre-training, as opposed to being acquired through task-specific fine-tuning. This work systematically investigates the emergence and structure of semantic role information in the latent representations of decoder-only transformers of varying scale, using QA-SRL as a probing paradigm.
Experimental Framework
The paper presents a rigorous methodology:
- Model Suite: Four GPT-style decoder-only transformers are trained from scratch on WikiText-103, encompassing scales from 0.4M to 56.7M parameters. The tokenizer and training data distribution are held fixed, isolating model size as the only variable.
- Probing Protocols: The core experiment involves freezing the entire pre-trained model (save for the token embeddings and a QA-specific output head) and training only the QA head (and, as needed, the embedding for separator tokens) on the QA-SRL task. Comparison is made to full fine-tuning and random-initialization baselines.
- Analytic Tools: Emergence is disentangled from adaptation by isolating transformer weights versus embedding adaptation, and further dissected via per-layer linear probes, CKA, PCA/t-SNE, and neuron- and component-level ablations.
Notably, the experimental protocol precisely measures the “emergence” of semantic role information as the linear decodability from frozen representations, beyond what is possible through random initialization and independent of trainable embeddings.
Principal Results
Emergence and Scale
- Robust Emergence Across Scales: Across all model sizes, frozen-probe performance on QA-SRL is significantly above the random baseline, with a 12–19 F1 improvement due to pre-training alone. Thus, semantic role information is present in the transformer weights of even small LMs without fine-tuning.
- Scale-Dependent Mediation: The normalized emergence score (fraction of fine-tuned performance achieved by the frozen probe above baseline) declines monotonically with model scale—pre-training’s absolute contribution is approximately constant, while fine-tuning’s contribution increases with scale. Thus, fine-tuning becomes increasingly critical for realizing full task performance in larger models.
Representational Structure and Causality
- Role-Selective Neurons and Distributed Representations: In smaller models, role-selective neurons are causally important—e.g., ablating "Agent" neurons reduces role-specific F1. At larger scales, these same ablations improve Agent-role performance, suggesting interference from redundant or miscoordinated units. Time-selective neurons, however, remain causally contributive to temporal role decoding across all scales.
- Distributed/Overlapping Encodings: PCA and t-SNE visualizations show only modest clustering of representations by role—semantic role information exists in a distributed subspace with weak but statistically significant increases in same-role neuron correlations with depth.
- Component-Level Causal Attribution: As model scale increases, the number of MLP or attention components contributing to semantic role information increases and concentration ratio decreases—again supporting a more distributed, less monosemantic representation at scale.
Comparative Analyses
- Replicability: The sign-flip in causal importance of Agent-selective neurons, and general trends in distributed representation, replicate in publicly available GPT-2 and Pythia models at comparable and larger scales, supporting the generality of these findings beyond custom-trained models.
- Emergence During Pre-training: Layer-wise probing of intermediate pre-training checkpoints shows that a significant fraction of semantic role information arises early and persists throughout training.
Theoretical and Practical Implications
Implications for Model Interpretability
Identification of role-selective units in small models demonstrates that neural circuits corresponding to emergent linguistic functionality can be found and causally validated. However, the transition to distributed redundant encodings with increased scale undermines the applicability of single-unit mechanistic attributions for large models—interpretability strategies must thus move towards subspace- or circuit-level analyses.
Implications for Scalability
The distinct finding that fine-tuning increasingly outperforms emergence from pre-training at larger scales points to a fundamental shift in the locus of semantic knowledge in LLMs. Pre-training confers a meaningful but ultimately bounded foundation for semantic role reasoning; hierarchy and sparsity in neural codes, as well as potential interference effects, emerge with increased width and depth.
The use of more continuous (F1) metrics and careful separation of decodability from emergence addresses open critiques about the artifact-driven nature of observed LLM "emergent" behaviors. These results argue for nuanced evaluation protocols when probing the latent linguistic capacities of LMs.
Speculation on Future Directions
- Scalable Interpretability: As models scale, monosemantic neuron discovery becomes less tractable, indicating the need for higher-order interpretability tools (e.g., subspace clustering, circuit-level interventions, or dictionary learning).
- Task Generality vs. Specialization: The scale-induced reorganization in neural locus of semantics may underlie both generalization phenomena (capabilities unlocked by fine-tuning) and robustness failures (interference due to overlapping encodings).
- Supervision-Efficient Adaptation: The robust emergence of significant semantic role information from pre-training alone supports the continual scaling of unsupervised objectives, but also signals the enduring value of minimal supervision, particularly in extracting highly structured knowledge.
Conclusion
This paper presents the strongest evidence to date that core aspects of semantic role understanding are an emergent property of autoregressive language modeling pre-training, detectable via linear probe even at low-to-mid parameter scales. However, it also demonstrates that full semantic role understanding—and the decodability of that information—depends increasingly on model-specific fine-tuning as scale grows, with distributed and sometimes interfering representations supplanting the monosemantic neuron-level coding seen in smaller models. These results redefine the boundaries between mechanistic interpretability and distributed representation in LLMs, and offer precise tools and metrics for dissecting where and how linguistic abstraction arises. The findings are directly relevant both for the development of more interpretable architectures and for understanding which aspects of meaning representation are directly available from the pre-trained weights of modern LLMs.
Citation:
"Emergent Semantic Role Understanding in LLMs" (2605.09187)
