Reliable Evaluation of Large Language Models

Develop reliable evaluation methodologies for large language models that effectively assess model performance, including helpfulness and harmlessness, addressing the acknowledged unresolved problem of evaluating such models.

Background

The paper evaluates Safe RLHF across three iterations and notes that assessing LLMs remains difficult. To proceed despite this difficulty, the authors employ two practical methods: fast model-based evaluation using unified reward and cost models trained on balanced preference data, and Elo-style pairwise comparisons of model outputs judged by GPT-4 and humans.

They further construct their own evaluation prompt dataset due to gaps in existing benchmarks, underscoring that current evaluation standards for alignment (helpfulness and harmlessness) are inconsistent and costly when relying solely on human judgments. This context motivates the need for more reliable, comprehensive evaluation approaches.

References

However, evaluating LLMs has consistently been a challenging and unresolved problem.

— Safe RLHF: Safe Reinforcement Learning from Human Feedback  (2310.12773 - Dai et al., 2023) in Section 4.1, Helpfulness and Harmlessness Evaluation

Second, declared evidence markers give verifiable ground truth for an engagement whose author can state them, and open-ended harm still depends on a model's judgement.

— ACEA: An Adversarial Co-Evolution Arena for Head-to-Head Red-Team and Blue-Team LLM Testing  (2609.08256 - Shen et al., 8 Sep 2026) in Conclusion

Some benchmarks include human studies, but these remain contentious, as designing sound human-model evaluations is an open question and a direction for future work.

— Not Another Text Benchmark: Putting the "Visual" Back in Visual Question Answering for Large Video Models  (2609.17112 - Chakraborty et al., 15 Sep 2026) in Section 4, subsection “Human baseline”

The implementation running on a device, with its leakage, faults, and countermeasures, remains open.

— CESBench: Benchmarking Large Language Models on Cryptographic Engineering Security for IoT Devices  (2609.21344 - Zhou et al., 18 Sep 2026) in Section 2, subsection “LLM Benchmarks in Cryptography” (Related Work)

The cryptographic module inside the device remains open as an object of evaluation.

— CESBench: Benchmarking Large Language Models on Cryptographic Engineering Security for IoT Devices  (2609.21344 - Zhou et al., 18 Sep 2026) in Section 2, subsection “LLM Benchmarks in Cybersecurity and IoT” (Related Work)

However, open questions remain regarding stochastic variability, error behavior, and cross-model performance.

— Two Truths and A Lie? Benchmarking Off-the-Shelf LLMs for Requirements Quality Assessment: Performance, False Alarms, and Misses  (2609.03230 - Shefa et al., 3 Sep 2026) in Section 2.2, AI for Requirements Quality Assessment, p. 3

As a result, despite growing interest in deployment, robust evaluation of clinical information retrieval remains an open challenge [16, 17].

— A Living Benchmark for Information Retrieval from Electronic Health Records  (2609.30205 - Cahoon et al., 24 Sep 2026) in Section 1, Introduction

Its reliability for assessing LLM outputs remains to be established.

The open frontier is outputs that execute cleanly yet are semantically wrong.

— Who Verifies the Verifier? Co-Evolving Inspectable Graders with Self-Improving Agents  (2610.11464 - Zhang et al., 8 Oct 2026) in Section 6, Discussion and conclusion

Evaluation on general-purpose frontier LLMs as judges is left to future work.

— Judges Follow the Reference: The Derivation Gap in Verifying Tool-Call Arguments  (2610.18497 - Ma et al., 13 Oct 2026) in Limitations section

Our LLM-as-judge achieves 100\% agreement with human evaluation on 200 sampled cases, but some errors may still exist in the remaining samples. We will further explore more robust and programmatic evaluation procedures to improve evaluation accuracy and reliability.

— Plan-Bench: Can Existing VLMs Go Well Beyond Perception to Reason and Plan in Embodied Worlds?  (2610.18836 - Feng et al., 13 Oct 2026) in Section 6, Limitations and Discussion

Inter-LLM disagreement may therefore be a useful label-free signal for which phenotypes most need prompt refinement or human review, although the present spot-check is too small to validate it.

— Documenting Trofinetide Treatment Phenotypes in Rett Syndrome with a Dual-LLM Consensus Pipeline  (2610.22062 - Palacios et al., 15 Oct 2026) in Supplementary Note S1, paragraph “Prompt wording drives agreement”

As agents move toward real-world deployment, how to evaluate them and when to trust them remain open challenges~\citep{kapoor2025agents, kirchhof2025position}.

— GraphACM: Graph-based Action Consistency Measurement for Uncertainty Quantification in LLM Agents  (2610.22031 - Ma et al., 15 Oct 2026) in Section 1, Introduction