Preserving depth of insight in scalable participation models

Determine the extent to which lower-touch or asynchronous participatory methods can preserve the depth of insights into participants’ normative preferences for cultural representation and the level of participant buy-in achieved by synchronous, in-depth engagements when systematizing community-informed rubrics for evaluating the cultural appropriateness of AI-generated images of cultural artifacts.

Background

The paper engages three communities (blind and low-vision individuals in the UK, and residents of Kerala and Tamil Nadu) to co-design rubrics that systematize what constitutes culturally appropriate depictions of salient artifacts in AI-generated images. These rubrics are intended for later operationalization in MLLM-as-a-judge pipelines.

While the study demonstrates the benefits of synchronous, in-depth workshops for eliciting lived and normative perspectives (including participant buy-in and nuanced criteria), the authors also discuss pragmatic pressures to scale participation. They outline lighter-weight approaches (e.g., asynchronous elicitation methods, interfaces to help users design LLM-as-a-judge criteria, or starting from LLM-generated rubrics and refining with communities), and raise an explicit open question about whether such approaches can preserve the depth achieved by synchronous engagements.