Analyzing the Factual Knowledge Representation in Pretrained LLMs Through Negated and Misprimed Probes
This paper presents a critical evaluation of the abilities of Pretrained LLMs (PLMs) to understand and recall factual knowledge, specifically under conditions of negation and mispriming. The paper extends upon prior work by Petroni et al., leveraging LAMA (LLM Analysis) to formulate probing tasks that challenge a PLM's capacity for nuanced comprehension. Two novel tasks are introduced: negation and mispriming, effectively testing the robustness of PLMs like Transformer-XL, ELMo, and BERT.
Summary of Findings
- Negation: The introduction of the "negated LAMA dataset" allows for probing the effect of negation. The results reveal a significant overlap in model predictions for positive (e.g., "Birds can fly") and negated (e.g., "Birds cannot fly") statements. Even BERT, known to handle negation comparatively well, struggles, illustrating a clear failure to differentiate between affirming and negating factual assertions. It was observed that BERT can memorize individual instances of negation if encountered during training but lacks generalization on unseen negated statements. However, finetuning improves its ability to correctly discern true and false statements, indicating that supervised learning can ameliorate some of the deficiencies observed during unsupervised pretraining.
- Mispriming: The application of psychological priming in a novel context for PLMs is another core contribution. The paper introduces "misprimes" (e.g., "Talk? Birds can [MASK]") to challenge BERT’s processing, simulating scenarios where human semantics would not normally be misled. Results show that PLMs are significantly misled by misprimes; BERT frequently substitutes the prime into the masked position instead of the expected value. Even increased distance between the misprime and the mask within a sentence doesn’t significantly mitigate this effect, suggesting an underlying tendency to rely on immediate contextual proximity over stored factual knowledge.
Implications and Future Directions
The findings have notable implications for the interpretation of PLMs’ performance in question-answering tasks and their potential application in real-world scenarios. The research raises questions about the robustness of inferred "knowledge" in PLMs, which might be heavily reliant on patterns and co-occurrences in training data rather than a deep understanding of semantics and logical constructs such as negation.
For practical applications, this means that there is still a significant gap between human-level understanding and the mimicry of such understanding seen in models like BERT. The paper argues this gap could hinder advancements where fine-grained and contextual comprehension is necessary. The research also calls attention to the fact that training data lacking sufficient examples of negation or varied context might lead to brittle model performance, suggesting a need for more systematic incorporation of such phenomena in training datasets.
Theoretically, the findings suggest avenues for architectural developments in PLMs that could better handle discrete phenomena like factuality and negation. Future research might need to focus on enhancing the model architecture or training paradigms to more closely simulate human levels of language understanding, possibly through hybrid approaches that integrate explicit logic or semantic reasoning capabilities with conventional deep learning techniques.
Overall, this paper contributes to a growing body of literature that challenges the surface-level proficiency of PLMs in handling complex linguistic phenomena, underscoring the necessity for continued innovation as these models are further developed for natural language understanding tasks.