- The paper introduces GIScholarBench, a benchmark using 10,865 GIScience papers and three scholarly tasks to measure behavioral overconfidence in Claude, Gemini, and ChatGPT.
- The evaluation finds that models retrieve some correct citations but perform poorly on longer lists, with no system exceeding 0.55 precision at 20 references and frequent bibliographic errors.
- The results show overconfidence in every task—factual overgeneration, unsupported citation expansion, and incomplete research agendas—supporting source verification and cautious use of LLMs for research workflows.
Overview and motivation
GIScholarBench is a benchmark for evaluating LLMs on scholarly workflow tasks in Geographic Information Science (GIScience), with a specific focus on overconfidence. The authors define overconfidence behaviorally rather than in the calibration sense: the tendency to produce complete, assertive, well-formatted outputs even when the underlying knowledge is incorrect or unverifiable. Because their protocol does not elicit explicit confidence scores, they do not claim to measure calibration error directly, a distinction they state plainly.
The motivation rests on two observations. First, LLMs are increasingly used in literature retrieval, citation generation, and research ideation, yet fabricated DOIs or citations propagate through manuscripts and databases in ways that are costly to correct. Second, existing benchmarks such as HELM (Liang et al., 2022) and SciEval (Fang et al., 2024) target factual question answering under controlled settings rather than open-ended academic workflows under real user-facing conditions. Prior work has documented citation fabrication by ChatGPT (Jeong et al., 2023), quantified bibliographic errors at scale (Liu et al., 2023), and shown that prompting models to express uncertainty does not reliably improve calibration (Xiong et al., 2023), but systematic multi-task evaluation of overconfidence in deployed systems has been lacking.
Benchmark construction
The corpus comprises 10,865 papers drawn from 25 core GIScience journals published between January 2020 and December 2025, retrieved via the Scopus API after deduplication and removal of records without resolvable DOIs or abstracts. Ten structured metadata fields are collected per article, including title, DOI, keywords, abstract, citation count, and reference list. Journal selection follows prior bibliometric surveys of the field [13658816.(2015.11308)31, 13658816.(2024.23473)06].
Three tasks of increasing cognitive complexity are defined:
- Metadata retrieval: bidirectional title–DOI lookup (DOI→Title and Title→DOI), each under Strong (rich context plus verification instructions) and Weak (minimal context) prompt conditions.
- Literature linking: given a seed paper's metadata, generate its citation neighborhood; ground truth is built from Scopus records for 816 seed papers, with neighborhoods averaging 47.3 references (range 1–352). Ground truth is bounded by the corpus itself, so hit metrics are conservative lower bounds on true recall.
- Research direction generation: models generate 8–10 concrete research directions extending a seed paper; ground truth is derived from the titles and keywords of all future-citing papers for 335 seed papers.
Matching uses content-based criteria: normalized title overlap with a 0.6 content-word threshold for Task 1; hybrid word-level and character-level TF-IDF cosine similarity with threshold τ=0.20 for Task 2; and a shared topic space built from TF-IDF features with K-Means clustering (k=20, chosen by elbow method) for Task 3. Accuracy denominators are fixed to the full corpus size, penalizing failed or incomplete response collection.
A notable methodological choice is that all responses are collected through the native web interfaces of Claude Sonnet 4.5, Gemini 3, and ChatGPT 5.3 using an automated browser extension, rather than through APIs. This captures product-level behaviors such as retrieval augmentation and refusal patterns as experienced by real users, at the cost of reproducibility, since web deployments change without documentation.
ChatGPT 5.3 achieves the highest accuracy, reaching 0.87 in the Title→DOI Strong setting, with reported accuracy ranging from 0.61 to 0.87 across settings. Two findings stand out. First, additional context helps Title→DOI retrieval for all models, confirming that abstracts and keywords aid disambiguation. Second, verification instructions do not consistently help: in the DOI→Title task, Claude's accuracy decreases from 0.48 to 0.28 when the Strong condition adds an explicit consistency-verification instruction, while ChatGPT remains nearly unchanged. Models frequently generate titles that are semantically compatible with the supplied abstract but differ from the actual paper title, and none of the three systems meaningfully abstains when retrieval fails. The implication is that even the best-performing model returns incorrect bibliographic entries at nontrivial rates, so LLM-generated metadata should be verified against authoritative sources such as Crossref before use in manuscripts or databases.
Literature linking results
All models reliably identify at least one relevant citation—Hit@1 ranges from 0.70 to 0.97—but performance collapses on longer lists. Claude Sonnet 4.5 leads overall (Avg Hits = 10.3, P@20 = 0.53), whereas Gemini Full reaches Hit@10 of only 0.20 and ChatGPT Full only 0.08. No model exceeds P@20 = 0.55 in any setting, meaning most generated references beyond the first few cannot be matched to verified citation records despite fluent formatting. ChatGPT's comparatively higher precision (~0.52) is partly an artifact of generating shorter reference lists rather than fully populating 20 slots.
The sharp Hit@1/Hit@10 divergence is the central result here: models continue producing plausible references well past the point where reliable retrieval capacity is exhausted. Practically, this supports using LLMs to find anchor papers when entering an unfamiliar topic, while treating complete generated reference lists as preliminary leads rather than bibliographic outputs.
Research direction generation results
Generated directions cover far less of the actual future research space than the trajectories realized in later citing literature:
| Metric |
Ground Truth |
Claude |
Gemini |
ChatGPT |
| Topic Coverage |
~0.22 |
0.12 |
~0.09 |
~0.09 |
| Novel Miss Rate |
— |
0.88 |
~0.91 |
~0.91 |
| HHI (concentration) |
~0.64 |
0.70 |
~0.87 |
~0.87 |
| Kernel entropy |
0.308 |
0.196 |
0.183 |
0.179 |
Gemini and ChatGPT fail to capture roughly 91% of the topic space occupied by real downstream research, and their generated distributions are markedly more concentrated than the ground-truth baseline, both in cluster concentration (HHI) and in semantic-space dispersion (kernel entropy). Importantly, individual directions are not factually wrong; the overconfidence here is meta-cognitive—models present 8–10 "actionable" directions with no hedging about what may be missing, implicitly claiming completeness they do not possess. This echoes the self-knowledge limitation identified by Kadavath et al. (Kadavath et al., 2022): models do not know what they do not know. The practical implication is that AI-generated research agendas risk steering researchers toward mainstream themes while systematically underrepresenting emerging, interdisciplinary, or unconventional directions—a pattern consistent with evidence that AI tools can contract science's focus [Nature, doi:10.1038/s41586-025-09922-y].
Synthesizing across tasks, the authors characterize overconfidence as task-invariant but form-varying: it appears in every evaluated model and task, but takes different forms as cognitive demands increase. In metadata retrieval it manifests as factual overgeneration (definitive titles and DOIs despite errors); in literature linking as unreliable citation expansion (fluent references beyond verifiable capacity); and in research ideation as overconfidence in output completeness (unhedged direction lists covering only a fraction of the real topic space). This framing is the paper's main conceptual contribution, suggesting that mitigation strategies must be form-specific rather than generic.
Limitations and open questions
The authors acknowledge three limitations explicitly. First, web-interface collection improves ecological validity but undermines reproducibility: deployed models are updated continuously, so observed behaviors reflect specific deployment versions during the collection window, and identical prompts may yield different outputs upon replication. Second, the benchmark is entirely GIScience-specific; generalization to other disciplines—with different terminological density and citation structures—is untested. Third, because no confidence scores are elicited, the study measures behavioral overconfidence only, leaving the relationship between self-reported confidence and empirical accuracy (e.g., expected calibration error) unquantified. Open questions include whether explicit "Not Found" constraints, retrieval augmentation, or confidence scoring can reduce factual overgeneration, and whether the observed mainstream bias in ideation persists in other scientific domains.
Conclusion
GIScholarBench provides a large-scale, domain-grounded evaluation of LLM reliability in scholarly workflows, combining 10,865 Scopus-indexed papers with three tasks spanning factual retrieval, relational linking, and generative ideation, evaluated through real user-facing interfaces. Its consistent finding is that Claude Sonnet 4.5, Gemini 3, and ChatGPT 5.3 all produce authoritative, well-formatted outputs beyond their verifiable knowledge—whether fabricating plausible metadata, extending citation lists past P@20 = 0.55, or presenting narrow research agendas as comprehensive. The benchmark supports treating LLMs as orientation and brainstorming aids rather than authoritative sources of scholarly knowledge, and offers a foundation for evaluating epistemic reliability in AI-assisted research.