Determine the recall of multimodal false-claim extraction

Determine whether the multimodal false-claim extraction agent misses additional valid false claims within disaster-related social media posts, thereby establishing its recall through comprehensive evaluation.

Background

The study validates the correctness of claims extracted by its multimodal false-claim extraction agent through human verification, but this validation does not establish whether the agent identifies all false claims present in a post. Exhaustively annotating every possible false claim in multimodal disaster-related social media posts is described as prohibitively expensive. Consequently, the recall of the extraction process remains unresolved, and the authors identify comprehensive recall-oriented evaluation as a problem for future investigation.

References

Embedding-based retrieval makes semantic search feasible at this scale, but it is only one possible strategy for identifying potential false negatives on a live platform. Alternative retrieval or sampling approaches may surface different candidate posts. Analyzing how VLMs capture such potential harms is left for future work.

— Can Foundation Models Moderate Online Content? Evaluating Instruction- vs. Example-Driven Policy Operationalization  (2609.10410 - Majumdar et al., 9 Sep 2026) in Appendix, Section “Data Gathering and Curation”, subsection “Near-moderated Posts”

Firstly, although human verification confirms that the extracted false claims are highly accurate, this validation only assesses the correctness of the extracted claims rather than the recall of the extraction. Manually identifying all possible false claims within multimodal social media posts is prohibitively expensive, we cannot determine whether additional valid claims were missed.

— Beyond Truth Discovery: A Two-Stage Framework to Assess the Severity of False Claim during Disasters  (2608.20983 - Yao et al., 21 Aug 2026) in Section Limitations