Can LLMs detect originality, rigour, and impact across fields?

Evaluate the capability, accuracy, and biases of large language models to detect and assess originality, methodological rigour, and impact in journal articles across diverse disciplines, relative to expert judgements.

Background

Traditional machine learning using bibliometrics has shown some promise for predicting article quality, especially in certain fields. Whether LLMs can meaningfully appraise key quality dimensions from text remains unclear and may depend on complex field norms and tacit knowledge.

Robust evaluation requires field-stratified benchmarks and careful bias analysis.