Changing the World by Changing the Data (2105.13947v1)

Abstract: NLP community is currently investing a lot more research and resources into development of deep learning models than training data. While we have made a lot of progress, it is now clear that our models learn all kinds of spurious patterns, social biases, and annotation artifacts. Algorithmic solutions have so far had limited success. An alternative that is being actively discussed is more careful design of datasets so as to deliver specific signals. This position paper maps out the arguments for and against data curation, and argues that fundamentally the point is moot: curation already is and will be happening, and it is changing the world. The question is only how much thought we want to invest into that process.

• The paper argues that deliberate data curation is essential for building robust and ethical NLP systems, highlighting its necessity over an exclusive focus on algorithmic improvements.
• Data curation directly addresses issues like amplifying social biases, security vulnerabilities, superficial language understanding, and susceptibility to adversarial attacks inherent in models trained on current datasets.
• The paper advocates for policy changes, including academic incentives for data work, interdisciplinary collaboration, and integrating ethics and linguistics into NLP education to prioritize data practices.

Changing the World by Changing the Data

Anna Rogers presents a compelling argument in her paper titled Changing the World by Changing the Data regarding the current state of data utilization within the NLP community, highlighting the emphasis placed almost exclusively on algorithm development and the necessity of data curation. The paper posits the inevitable need for reconsideration of data practices in NLP systems, addressing the ongoing tension between data-driven and qualitative approaches.

Core Arguments

The paper starts by recognizing transformative successes attributed to Transformer-based models such as BERT and its successors. These models marked substantial progress in surpassing human baselines on established benchmarks like SuperGLUE. However, it underscores a glaring issue: models are inherently predisposed to learning and amplifying biases present in training datasets, including gender, race, and other social biases.

Additionally, Rogers points out the failures of deep learning models in tackling spurious patterns and annotation artifacts embedded in datasets. These originate from shortcuts and heuristics often present in data preparation processes, which can lead to superficial solutions that lack genuine sophistication required in NLP tasks.

The Case for Data Curation

The crux of Rogers' arguments centers around the need for deliberate data curation—a practice she argues is already in motion, consciously or otherwise. She offers multiple reasons supporting this position:

1. Mitigating Social Biases: The perpetuation of racial or gender biases by NLP models reflects societal injustices—making careful selection of dataset attributes crucial for fair representation.
2. Addressing Security and Privacy Concerns: One crucial issue is models memorizing specific data points; consideration must be given to privacy-sensitive information which should not be retained—emphasizing a curation approach to address potential vulnerabilities.
3. Advancing Natural Language Understanding (NLU): Current data aggregation methods prioritize large-scale, easily accessible sources, often ignoring specific linguistic nuances that are pivotal in achieving true NLU.
4. Vulnerability to Adversarial Attacks: The susceptibility of LLMs to crafted adversarial triggers requires preventive measures through controlled dataset configurations.
5. Evaluation Practices: The conventional practice of in-distribution evaluation often overstates model capabilities and doesn't consider stress-testing against diverse language patterns—making out-of-distribution testing a potential necessity.

Counterarguments

Acknowledging opposing views, Rogers delineates arguments against data curation, including the belief that models should represent language as naturally expressed and that current large-scale data efforts potentially encapsulate the entirety of language diversity. She counters these objections by emphasizing the lack of inherent representativeness in naturally occurring datasets due to societal technological disparities, the empirical question of whether algorithmic methods alone are sufficient, and ultimately challenges the notion that exhaustive language data exists.

Implications and Future Directions

Rogers' paper articulates an inescapable reality: NLP practices and outcomes shape the world by influencing human interactions and learning. She advocates for interdisciplinary collaboration, integrating computational methodologies with linguistic theory and ethical practices to address the consequences deterministically imposed by LLMs.

Policy Recommendations

The research suggests actionable steps for the community:

• Incentive Structuring: Proposing greater recognition and publications for quality data work at conferences to elevate its prestige.
• Educational Integration: Advocates for holistic NLP curricula encompassing engineering, linguistic theory, and AI ethics.
• Interdisciplinary Collaboration: Encourages joint efforts among computational linguists, ethicists, and engineers for well-rounded research initiatives.
• Innovation and Governance: Calls for established mechanisms to assess and balance benefits with potential harms in real-world applications of NLP models.

In conclusion, Changing the World by Changing the Data argues that impactful and conscientious data curation is not only critical but unavoidable in advancing the robustness and fairness of NLP systems. The field must converge on inclusive practices that ensure LLMs accurately reflect and serve diverse human conditions, setting a paradigm that transcends mere algorithmic prowess.