Perspectives from India: Opportunities and Challenges for AI Replication Prediction to Improve Confidence in Published Research (2310.19158v3)

Abstract: Over the past decade, a crisis of confidence in scientific literature has gained attention, particularly in the West. In response, we have seen changes in policy and practice amongst individual researchers and institutions. Greater attention is given to the transparency of workflows and the appropriate use of statistical methods. Advances in scholarly big data and machine learning have led to the development of AI-driven tools for the evaluation of published findings. In this study, we conduct 19 semi-structured interviews with Indian researchers to understand their perspectives on challenges and opportunities for AI technologies to improve confidence in published research. Our findings highlight the importance of social and cultural context for the design and deployment of AI tools for research assessment. Our work suggests that such technologies must work alongside rather than replace human research assessment mechanisms. They must be explainable and situated within well-functioning human-centered peer review processes.