PFQABench: Multi-Domain Benchmark Suite

• PFQABench is a multi-domain benchmark suite that rigorously evaluates LLM personalization, factuality robustness, human preference in AI-generated faces, and quantum chemistry accuracy.
• It employs curated datasets and protocols—from realistic chat logs and knowledge graph edits to human annotation and quantum MC benchmarks—to diagnose model behaviors.
• Its evaluations reveal key trade-offs between personalization and factual accuracy, highlight risks of hallucination, and inform improved methodologies across diverse fields.

PFQABench is a designation that has appeared in the literature for distinct, domain-specific benchmark suites in LLM personalization safety, factuality robustness under false premises, human preference evaluation for AI-generated face images, and quantum chemistry accuracy benchmarking. The PFQABench identifier thus denotes rigorous, focused datasets or protocols serving as standards for quantitative comparison and diagnostic evaluation in their respective fields. The following sections provide a detailed synthesis of the various PFQABench applications as documented in recent academic literature.

1. PFQABench for Personalized LLM Factuality Robustness

The most recently introduced and widely referenced incarnation of PFQABench is as the first benchmark expressly designed to jointly evaluate factual and personalized question answering (QA) in LLMs subjected to personalization conditioning (Sun et al., 16 Jan 2026). This benchmark addresses personalization-induced hallucinations: instances where personalization signals (user history, preferences) entangle with factual representations and cause LLMs to output persona-aligned but factually incorrect answers in response to purely factual queries.

PFQABench embeds both personalized and factual queries into realistic user session contexts, constructed from 50,000 chat logs (extracted from LongMemEval) and a merged multi-hop QA set (FactQA: HotpotQA + 2WikiMultiHopQA). It comprises 1,000 examples (500 users × 2 queries each), balanced between personalized QA (requiring correct use of persona) and factual QA under persona (requiring persona-invariant, world-knowledge answers).

Input formats cover multiple personalization paradigms—retrieval-augmented (RAG) and three profile-augmented methods (PAG, DPL, LLM-TRSR)—all concatenating user signal/context with the prompt. Evaluation is automated via an LLM-as-Judge protocol (Qwen2.5-32B-Instruct), reporting Personalization Accuracy (P-Score), Factuality Accuracy (F-Score), and Overall Score (average of the two), along with formal metrics:

$$\mathrm{FAcc} = \frac1N\sum_{i=1}^N \mathbf{1}(\hat a_i = a_i),\quad \mathrm{PAcc} = \frac1N\sum_{i=1}^N \mathrm{sim}(\hat a_i, \mathrm{profile}_i),\quad \mathrm{Joint} = 2\frac{\mathrm{FAcc}\cdot\mathrm{PAcc}}{\mathrm{FAcc}+\mathrm{PAcc}}$$

PFQABench exposes the tradeoff between factuality and personalization: baseline models achieve P-Scores of 30–50% but F-Scores drop to 10–30%. The Factuality-Preserving Personalized Steering (FPPS) framework introduced in the same work leverages these benchmarks, with FPPS-H (hard removal) raising F-Scores to 75–85% at the expense of P-Score, and the mixed version FPPS-M achieving Overall scores of ~55–65%, simultaneously recovering factual accuracy while retaining personalized performance.

PFQABench's unique structure enables:

• Quantification of persona-induced hallucinations that standard QA or personalization-only benchmarks miss
• Fine-grained evaluation of personalized LLM protocols for safe deployment, e.g., in education/health
• Investigations into history-length sensitivity and longitudinal belief formation under personalization

2. Knowledge-Graph-Based False Premise Question Benchmark ("KG-FPQ", sometimes referenced as PFQABench)

In a related but technically distinct domain, KG-FPQ refers to a large-scale, knowledge graph–driven benchmark designed to evaluate LLM vulnerability to factuality hallucination induced by false premise questions (FPQs) (Zhu et al., 2024). These FPQs systematically manipulate true subject–relation–object triplets from Wikidata (KoPL subset), applying six controlled editing methods representing axes of confusability (conceptual or relational proximity and hop distance) to generate plausible but incorrect premises.

The automated pipeline produces 178,320 FPQs across Art, People, and Place domains, with both Yes–No (discriminative) and WH-style (generative) formats. Accuracy is reported per domain and edit class, revealing that confusability (e.g., Neighbor-Same-Concept edits) and question format strongly influence hallucination rates, which are not solely a function of domain proficiency or model scale. Tools such as the FPQ-Judge (a classifier trained on both GPT and human-annotated answers) enable scalable, objective labeling of generative responses.

Key findings include:

• LLM accuracy in rejecting FPQs rises with model size (from ~60% at 6B–8B to ~85–95% at GPT-4), but sophisticated FPQs (close edits) remain highly confounding
• FPQ benchmarks generated at KG-scale enable fine-grained, robust assessment unobtainable from prior hand-crafted sets

3. PFQABench ("F-Bench") for Human Preference Evaluation of AI-Generated Face Images

In the context of AI-generated faces, PFQABench (also denoted "F-Bench") is a large-scale, human-annotated benchmark specifically designed for comprehensive assessment of face generation, customization, and restoration (Liu et al., 2024). Leveraging the FaceQ database (12,255 images, 29 models, three task categories), it collects 32,742 mean opinion scores (MOSs) from 180 annotators, normalized and aggregated as follows:

$$z_{ij} = \frac{r_{ij}-\mu_j}{\sigma_j}, \quad z'_{ij} = \frac{100(z_{ij}+3)}{6},\quad \mathrm{MOS}_j = \frac1N\sum_{i=1}^N z'_{ij}$$

Preference dimensions include Overall Quality, Authenticity, Identity Fidelity, and Text-Image Correspondence, with protocols ensuring high inter-annotator agreement (Fleiss’ $\kappa$).

Multiple categories of no-reference and deep learning–based image quality assessment (IQA), face-specific IQA/FQA, and vision–language metrics are benchmarked for alignment with human preferences. Deep IQA metrics (MINTIQA, MANIQA, TReS) correlate best with MOS in quality, authenticity, and correspondence; face-specific FQA metrics (DSL-FIQA, ArcFace) capture identity fidelity but not semantic or perceptual quality.

PFQABench demonstrates:

• No single metric captures all preference axes; hybrid approaches are required
• Classical IQA and current vision–LLMs are insufficient for high-level perceptual alignment or artifact detection in AIGF evaluation
• The dataset enables development of new predictors matching human perceptual judgments across all key axes

4. PFQABench for Quantum Chemistry: Phaseless AFQMC Benchmarks

In computational quantum chemistry, PFQABench denotes a rigorously curated set of small-molecule and cluster benchmarks formulated to quantify the systematic and statistical errors of the phaseless auxiliary-field quantum Monte Carlo (ph-AFQMC) method (Sukurma et al., 2023). It comprises:

• 26 HEAT set molecules (cc-pVDZ, frozen core)
• Benzene (cc-pVDZ, frozen core)
• Four small water clusters (heavy-augmented cc-pVDZ, all-electron basis)

The ph-AFQMC propagator, constrained by the phaseless approximation, projects out ground-state energies with cubic to quartic scaling. PFQABench also introduces a refined constraint (ph$^*$-AFQMC) retaining the full complex weight in energy estimators, reducing overcorrelation biases inherent in the original phaseless formalism.

Systematic benchmarking against CCSDTQP, CCSD(T), and FCI (benzene) references yields mean absolute deviations of 1.15 kcal/mol (MAD for HEAT set) and water-cluster binding energies accurate to within 0.5 kcal/mol. Notably, the ph$^*$ version matches the accuracy of very large trial wavefunction expansions at far lower cost.

Emphasis is placed on rigorous methodology:

• Single-determinant trials suffice for routine systems; multi-determinant or symmetry-adapted trials are needed for open-shell/strongly correlated outliers
• Parameter settings: Cholesky threshold $\leq 10^{-8}$ Hartree, $\tau \sim 0.002~E_h^{-1}$, walker populations $\geq 5,000$, equilibration $\geq$40,000 steps
• Both ph-AFQMC and ph$^*$-AFQMC results should be reported to document overcorrelation trends community-wide

5. Comparative Structure and Unique Features

While the disparate PFQABench variants address different domains, each advances its field through methodological rigor, reproducible data splits, and a focus on failure modes neglected by existing benchmarks.

PFQABench Application	Domain	Distinctive Features
Personalized LLM Factuality	NLP, personal assistants	Joint factual/persona QA, exposes hallucinations
KG-FPQ (as PFQABench)	NLP, factual QA	Large-scale FPQ generation; finetuned model judges
F-Bench (AI Faces)	Generative vision	Large human MOS dataset; multi-axis preference eval
AFQMC (Quantum Chemistry)	Theoretical chemistry	Systematic error benchmarks; phaseless/proposed mod.

A plausible implication is that future benchmarks are trending multi-axis, with synthetic data pipelines and expert-labeled outcomes to facilitate both diagnostic and comparative methodological research across subfields.

6. Implications and Future Trajectories

PFQABench establishes the standard for next-generation benchmarking across model personalization, factual robustness, human-centric evaluation, and scientific computing. Key extrapolations include:

• Multi-modal or longitudinal extensions (particularly for personalized LLMs) are suggested as natural next steps (Sun et al., 16 Jan 2026).
• Automated synthetic data generation (as seen in KG-FPQ) supports ongoing scalability and fine-grained control of difficulty (Zhu et al., 2024).
• Hybrid reference-based evaluation, unifying deep semantic metrics with perceptual and preference fidelity, will likely become the benchmark norm for generative applications (Liu et al., 2024).
• In quantum chemistry, adoption of improved trial wavefunction protocols—potentially borrowing from recent developments in selected CI or machine-learned ansätze—could further minimize remaining systematic biases (Sukurma et al., 2023).

The adoption of PFQABench across domains is emblematic of a pivot toward benchmarks robust to confounding factors, grounded in both expert and user-derived evaluations, and transparent in probabilistic/statistical methodology.