Designing and Interpreting Probes with Control Tasks (1909.03368v1)

Abstract: Probes, supervised models trained to predict properties (like parts-of-speech) from representations (like ELMo), have achieved high accuracy on a range of linguistic tasks. But does this mean that the representations encode linguistic structure or just that the probe has learned the linguistic task? In this paper, we propose control tasks, which associate word types with random outputs, to complement linguistic tasks. By construction, these tasks can only be learned by the probe itself. So a good probe, (one that reflects the representation), should be selective, achieving high linguistic task accuracy and low control task accuracy. The selectivity of a probe puts linguistic task accuracy in context with the probe's capacity to memorize from word types. We construct control tasks for English part-of-speech tagging and dependency edge prediction, and show that popular probes on ELMo representations are not selective. We also find that dropout, commonly used to control probe complexity, is ineffective for improving selectivity of MLPs, but that other forms of regularization are effective. Finally, we find that while probes on the first layer of ELMo yield slightly better part-of-speech tagging accuracy than the second, probes on the second layer are substantially more selective, which raises the question of which layer better represents parts-of-speech.

• The paper introduces control tasks to discern whether high probe accuracies reflect genuine linguistic structure or mere memorization.
• It defines selectivity as the accuracy difference between linguistic and control tasks, revealing that linear probes are more selective than MLP variants.
• The findings challenge conventional views by demonstrating that deeper ELMo layers yield richer linguistic representations despite lower raw accuracy.

Designing and Interpreting Probes with Control Tasks

The paper "Designing and Interpreting Probes with Control Tasks" by John Hewitt and Percy Liang addresses an important question in evaluating the linguistic capabilities of neural representations like ELMo: Do high probing accuracies indicate linguistic structure within the representations, or are they a result of the high capacity of the probe itself? The authors introduce "control tasks" to isolate these factors and propose the notion of "selectivity" to measure the reliability of probes.

Core Contributions

1. Control Tasks: The paper introduces control tasks, which associate word types with random outputs. By construction, the randomness ensures that these tasks can only be learned by memorization through the probe. The probe should therefore achieve high accuracy on linguistic tasks and low accuracy on control tasks to be deemed selective, indicating its ability to reflect the representation's properties rather than memorize data.
2. Selectivity as a Metric: Selectivity is defined as the difference between linguistic task accuracy and control task accuracy. This metric helps interpret probing results and offers insights into the interaction between probes and representations.
3. Probing Different Architectures: The paper explores various probing architectures, such as linear, MLP-1, and MLP-2, on tasks like part-of-speech tagging and dependency edge prediction, using ELMo representations. It examines these probes' selectivity and accuracy under various complexity control methods, including dropout and weight decay.

Key Findings

• Linear vs. MLP Probes: Linear probes exhibit higher selectivity compared to MLP probes, suggesting that the latter's minor gains in accuracy may be attributed to their expressiveness, which includes memorization.
• Regularization: Dropout, commonly used for regularizing MLPs, does not consistently improve selectivity, highlighting a gap in current probing methodologies. Other regularization techniques, like constraining hidden states or using weight decay, prove more effective.
• Layer Selectivity in ELMo: ELMo's second layer shows higher selectivity compared to the first, challenging the assumption that the first layer's higher accuracy in part-of-speech tasks directly correlates to better linguistic encoding.

Implications and Future Directions

The introduction of control tasks presents a novel approach to disentangling a probe's capacity to memorize from its ability to reveal the linguistic properties of neural representations. This contributes to the broader understanding of what these models learn beyond high task accuracy.

The paper's findings have implications for developing future methodologies that incorporate selectivity for more insightful probing. This could lead to better-designed probes in evaluating representations like BERT or newer transformers, driving further advances in natural language processing.

Furthermore, distinguishing between memorization and learned representation properties may inspire new architectures or training paradigms focused on encoding generalized linguistic structures.

In conclusion, this paper's methodology offers a robust framework for interpreting probing results, providing guidance in the continued exploration of the linguistic characteristics of neural representations in AI.