Troubling Trends in Machine Learning Scholarship (1807.03341v2)

Abstract: Collectively, ML researchers are engaged in the creation and dissemination of knowledge about data-driven algorithms. In a given paper, researchers might aspire to any subset of the following goals, among others: to theoretically characterize what is learnable, to obtain understanding through empirically rigorous experiments, or to build a working system that has high predictive accuracy. While determining which knowledge warrants inquiry may be subjective, once the topic is fixed, papers are most valuable to the community when they act in service of the reader, creating foundational knowledge and communicating as clearly as possible. Recent progress in machine learning comes despite frequent departures from these ideals. In this paper, we focus on the following four patterns that appear to us to be trending in ML scholarship: (i) failure to distinguish between explanation and speculation; (ii) failure to identify the sources of empirical gains, e.g., emphasizing unnecessary modifications to neural architectures when gains actually stem from hyper-parameter tuning; (iii) mathiness: the use of mathematics that obfuscates or impresses rather than clarifies, e.g., by confusing technical and non-technical concepts; and (iv) misuse of language, e.g., by choosing terms of art with colloquial connotations or by overloading established technical terms. While the causes behind these patterns are uncertain, possibilities include the rapid expansion of the community, the consequent thinness of the reviewer pool, and the often-misaligned incentives between scholarship and short-term measures of success (e.g., bibliometrics, attention, and entrepreneurial opportunity). While each pattern offers a corresponding remedy (don't do it), we also discuss some speculative suggestions for how the community might combat these trends.

• The paper exposes how ML research blurs the line between speculation and evidence, leading to widespread misconceptions.
• The paper reveals that superficial architectural tweaks often mask basic hyper-parameter tuning as innovative breakthroughs.
• The paper warns that the misuse of mathematics and imprecise language hinders clear communication and rigorous evaluation.

An Expert Analysis on "Troubling Trends in Machine Learning Scholarship"

The paper "Troubling Trends in Machine Learning Scholarship" by Zachary C. Lipton and Jacob Steinhardt offers an incisive critique of prevailing practices in the ML research community. This work identifies several patterns in ML scholarship that potentially undermine research quality and offers both reflections and suggestions for improvement. Key trends discussed include the blurring of explanation with speculation, misattribution of empirical advances, "mathiness" in presentation, and the misuse of language, which all contribute to issues in communication and comprehension within the community.

The authors argue that ML papers often mislead by failing to distinguish between speculative ideas and verified explanations. This issue can propagate misconceptions when anecdotal or intuitive claims are presented with the veneer of rigorous scientific inquiry. For instance, they critique the characterization of the "internal covariate shift" in batch normalization, a concept that was inadequately defined and thus led to widespread misinterpretation and citation as fact.

A second highlighted trend is the tendency to obscure the true origins of empirical gains by emphasizing superficial modifications to architectures rather than rigorous analysis of the underlying causes. This is exemplified in the practice of presenting multiple architectural tweaks without clear ablations to assess their specific contributions. By contrast, retrospective studies often reveal that gains were attributed to simpler factors, such as hyper-parameter tuning, rather than the proposed innovative models.

The paper also discusses "mathiness," the inappropriate use of mathematics intended to impress rather than inform. This phenomenon creates barriers to understanding when informal reasoning is cloaked in the guise of formalism, without clear exposition of assumptions and implications. Such practices can obscure the actual contributions and significance of theoretical results.

Misuse of language, particularly through suggestive definitions and the overloading of terms, further complicates the discourse. Terminology borrowed from other fields or colloquial usage often carries misleading connotations, leading to confusion or inflated expectations. The term "human-level" performance, for example, can be misleading when applied to specific constrained tasks, thus distorting the field's actual achievements and potential.

The paper speculatively attributes these trends to factors such as the community's rapid growth, reviewer inexperience, and misaligned incentives toward sensational outcomes rather than solid science. The authors contend that the swift expansion of the ML field exacerbates the reviewer burden, potentially lowering standards and frequency of critical assessments.

In response to these challenges, the authors propose strategies for researchers and reviewers to improve scholarly rigor. Encouragement is given to adopt more robust empirical analysis techniques such as error analysis, ablation studies, and robustness checks to better understand and report the source of empirical findings. They argue for clarity in distinguishing speculation from verified results and for using mathematical tools judiciously to enhance rather than obfuscate understanding. The suggestion to promote authoritative surveys could help distill genuine progress from the noise of flashy but shallow contributions.

Overall, this paper provides a thoughtful examination of self-perceived deficiencies in machine learning scholarship—an introspection reflecting ongoing dialogues to enhance scientific standards. Its observations and recommendations, while not novel to scientific discourse at large, are crucial for guiding the community through a period of rapid change and ensuring a firm foundation for future advancements. The work is a call to action for more disciplined practices that can sustain and nurture the growth and maturity of machine learning research within both academic and public spheres.