- The paper shows that superficial participation in ML can legitimize exploitative practices that reinforce systemic inequities.
- It differentiates participation as work, consultation, and justice, analyzing how each form can either challenge or perpetuate power imbalances.
- The study recommends compensating labor, applying context-specific consultation, and fostering long-term justice partnerships to enhance equity in ML design.
Participation is not a Design Fix for Machine Learning
The paper "Participation is not a Design Fix for Machine Learning" by Mona Sloane, Emanuel Moss, Olaitan Awomolo, and Laura Forlano offers a critical evaluation of participatory methods in the intersection of design and ML. The authors argue against what they term "participation-washing," urging the ML community to be more discerning of extractive and exploitative participation practices that prioritize scalability over genuine community involvement.
Participation in ML is categorized into three forms: participation as work, participation as consultation, and participation as justice. The authors stress that participation is not inherently benign; if executed poorly, it can reinforce existing power dynamics and further marginalize vulnerable groups.
Participation as Work
Participation as work acknowledges the often-invisible labor that supports ML architecture. Datasets like ImageNet, fundamental for image recognition technology, are built through the work of many anonymous individuals, such as mTurk workers and millions of internet users. This category underscores the integrated, often unconscious user contributions essential for ML systems. The authors highlight that this labor generally lacks consent and compensation, thereby challenging the notion of "designing for," which omits real user engagement.
Participation as Consultation
Participation as consultation sees diverse stakeholders being engaged in episodic, short-term projects. Frequently used in architecture and urban planning, this form often suffers from the limitations of being cost-prohibitive, context-independent, and potentially performative. Without a long-term engagement and an understanding of effective participatory processes, this mode can entrench systemic inequalities into design protocols.
Participation as Justice
Participation as justice emphasizes long-term, equitable, and mutually beneficial partnerships with stakeholders. It draws on frameworks such as participatory action research, design justice, and data feminism. However, achieving genuine participation as justice is particularly challenging in corporate environments dominated by extraction logics. Despite the theoretical benefits, implementing justice-oriented participation in ML can seem contradictory, given the prevailing capitalist constraints.
Critiques and Recommendations
The authors critique the centrality of scale in ML, where the scalability promises of ML inadvertently overlook the contextual intricacies embedded in datasets. They recognize that true scaling demands perpetual updates from user-generated inputs, yet current business models often discourage maintaining consultative and justice-oriented participation as products grow.
To address these issues, the authors provide three recommendations:
- Recognizing participation as work by obtaining consent and offering compensation.
- Ensuring context-specific participation rather than a generic one-size-fits-all approach, particularly for consultation.
- Engaging in genuine, long-term partnerships that are transparent and embedded in justice.
The importance of expanding value assessments beyond the monetary, and incorporating frameworks such as Indigenous data sovereignty, are also highlighted as means of addressing the imbalances within ML design and deployment.
Implications and Future Work
The paper calls for an expansion of the participatory design notion to acknowledge subtler and potentially exploitative forms of participation. It suggests a move towards holistic and interdisciplinary approaches to foresee and address the social consequences of ML. By proposing a cross-sectoral database of participation failures, the authors aim to create resources that acknowledge design injustices and socioeconomic dimensions.
The framework laid out can guide future endeavors in making ML systems more equitable and informed by those whom they impact most. Researchers are encouraged to engage in lateral thinking and cross-disciplinary collaboration to identify and remedy harms perpetuated by ML technology.
