PeopleSearchBench: Multi-Dimensional Benchmark

• PeopleSearchBench is an open-source, multi-dimensional benchmark for evaluating AI-driven people search systems using realistic queries and explicit, verifiable criteria.
• It supports four key use cases—corporate recruiting, B2B sales prospecting, expert search, and influencer discovery—with multilingual and domain-specific queries.
• It employs a three-stage Criteria-Grounded Verification pipeline to assess factual relevance, effective coverage, and information utility, providing an overall performance score.

Searching arXiv for papers on “PeopleSearchBench” and closely related people-search benchmarks to ground the article in current literature. PeopleSearchBench is an open-source, multi-dimensional benchmark designed to evaluate and compare AI-powered people-search platforms in realistic, commercial settings. It is based on 119 real-world queries spanning four use cases, employs a Criteria-Grounded Verification pipeline for factual relevance assessment, and measures system performance along three orthogonal dimensions—Relevance Precision, Effective Coverage, and Information Utility—before combining them into an overall score (Wang et al., 29 Mar 2026). Its central methodological premise is that people-search evaluation should be grounded in explicit, verifiable criteria rather than subjective holistic judging.

1. Scope, task formulation, and supported use cases

PeopleSearchBench evaluates people-search systems on four supported use cases: Corporate Recruiting with 30 queries, B2B Sales Prospecting with 32 queries, Expert Search with Deterministic Answers with 28 queries, and Influencer/KOL Discovery with 29 queries (Wang et al., 29 Mar 2026). The benchmark is multilingual, covering English, Portuguese, Spanish, and Dutch, and its query definitions were crafted with domain experts to reflect realistic constraints such as roles, skills, locations, follower-counts, and known entity lists.

Use case	Queries	Characterization
Corporate Recruiting	30	Skills, seniority, geography
B2B Sales Prospecting	32	Role, industry, thought-leadership
Expert Search with Deterministic Answers	28	Correct answers known or exhaustively verifiable
Influencer/KOL Discovery	29	Platform-specific social metrics

The query classes are intentionally heterogeneous. Corporate Recruiting seeks candidates with specified combinations of skills, seniority, and geographic preference. B2B Sales Prospecting targets decision-makers at enterprises based on role, industry, and thought-leadership signals. Expert Search with Deterministic Answers covers queries for which correct answers are known or verifiable exhaustively, making them useful for validating factual accuracy. Influencer/KOL Discovery identifies social-media thought leaders by platform-specific metrics and tends to produce the largest performance variance across platforms (Wang et al., 29 Mar 2026).

This design places PeopleSearchBench at the intersection of information retrieval, entity resolution, and agentic web search. A plausible implication is that it tests not only ranking quality but also the ability of a system to operationalize complex structured constraints from natural-language requests.

2. Criteria-Grounded Verification

A defining feature of PeopleSearchBench is Criteria-Grounded Verification, a factual relevance pipeline that decomposes each query into explicit checks and then verifies returned people against those checks with live web evidence (Wang et al., 29 Mar 2026). This replaces subjective holistic LLM-as-judge scoring with a criterion-wise procedure.

The pipeline has three stages. In Stage 1, an LLM performs criteria extraction and outputs a set of independently checkable criteria such as role, employer, and location. In Stage 2, each returned person is checked against every criterion using an external web API, Tavily Search. For each criterion, the judge assigns one of three labels: met with score 1.0, partially_met with score 0.5, or not_met with score 0.0. The overall relevance grade for a returned person is the arithmetic mean across criteria: $$\mathrm{rel}(p_i)=\frac{1}{N}\sum_{j=1}^N \mathrm{score}(c_j,p_i).$$

In Stage 3, the graded relevance is converted into a binary qualification decision. A person is considered qualified if $\mathrm{rel}(p_i)\ge 0.5$. All downstream metrics that depend on qualification, including coverage and qualified yield, are defined from this threshold (Wang et al., 29 Mar 2026).

Methodologically, this framework separates semantic interpretation of the query from factual verification of candidates. This suggests that PeopleSearchBench is structured to penalize systems that produce plausible-sounding but weakly substantiated matches, even if those matches might receive favorable holistic judgments from a generic judge model.

3. Evaluation dimensions and score aggregation

PeopleSearchBench evaluates systems along three dimensions, each scaled to $[0,100]$, and computes the overall score as their equal-weight average (Wang et al., 29 Mar 2026). The dimensions are Relevance Precision, Effective Coverage, and Information Utility.

Relevance Precision is defined through padded nDCG@10. The benchmark uses graded relevance scores rather than purely binary judgments at this stage, with: $$\mathrm{DCG@}K=\sum_{i=1}^{K}\frac{\mathrm{rel}(p_i)}{\log_{2}(i+1)}.$$ The ideal denominator is the padded $\mathrm{IDCG@10}$ obtained by assuming relevance $1.0$ at all top-10 ranks. Query-level $\mathrm{nDCG@10}$ is then averaged over all queries and multiplied by 100 (Wang et al., 29 Mar 2026).

Effective Coverage combines two components. The first is Task Completion Rate, defined as the fraction of queries with at least one qualified result. The second is a yield term, computed per query as $\min(\mathrm{qualified}(q)/K,1.0)$. Effective Coverage multiplies Task Completion Rate by the average of this capped yield term and scales the result to $[0,100]$ (Wang et al., 29 Mar 2026).

Information Utility measures the quality of returned profiles rather than their relevance alone. It averages three sub-dimensions scored in $[0,1]$: structural completeness, query-specific evidence, and actionability. Structural completeness concerns the presence of fields such as name, title, employer, contact, education, and experience. Query-specific evidence asks whether the result explains why it matches each criterion, with sources. Actionability measures whether sufficient information is present for outreach (Wang et al., 29 Mar 2026).

The overall score is the equal-weight average of these three dimensions. The benchmark therefore does not reduce people search to ranked relevance only; it treats discoverability, usable output quality, and query completion as co-equal properties.

4. Benchmark construction and normalization procedures

The benchmark construction process includes a unified result schema with fields person_id, name, title, company, location, social URLs, contact, skills, match_explanation, and source URLs (Wang et al., 29 Mar 2026). Platform outputs are normalized either through mapping rules or regex parsing for text responses. Deduplication is performed by normalized name plus company, keeping the most complete profile.

All code, prompts, and full query metadata are publicly released for reproducibility (Wang et al., 29 Mar 2026). The documentation includes prompt templates for criteria extraction, verification, and utility scoring. The availability of query definitions, prompts, and aggregated results is significant because prompt-induced variance and schema-mapping variance are major confounds in benchmarking agentic retrieval systems.

The benchmark evaluates four systems under identical conditions during January 15–22, 2025: Lessie (v2.1.0), described as a specialized AI agent; Exa (API v1), described as a structured search entity database; Juicebox (PeopleGPT v3.2), described as a recruiting platform with 800M+ profiles; and Claude Code (claude-sonnet-4-6), described as a general AI agent (Wang et al., 29 Mar 2026). The inclusion of both specialized and general systems makes the setup comparative across product types rather than within a single model family.

5. Reported empirical results

Across all 119 queries, Lessie attains the highest reported scores on all three dimensions as well as the overall score, and its lead over the second-ranked system is marked as statistically significant at $\mathrm{rel}(p_i)\ge 0.5$0 (Wang et al., 29 Mar 2026).

Platform	Overall score	Task completion
Lessie	65.2 ± 1.5	100%
Exa	55.0 ± 1.8	96.6 ± 1.8%
Claude Code	46.0 ± 2.1	86.5 ± 3.1%
Juicebox	45.8 ± 1.9	84.0 ± 3.3%

The dimension-wise scores are also reported. For Relevance Precision, Lessie scores $\mathrm{rel}(p_i)\ge 0.5$1, Exa $\mathrm{rel}(p_i)\ge 0.5$2, Claude Code $\mathrm{rel}(p_i)\ge 0.5$3, and Juicebox $\mathrm{rel}(p_i)\ge 0.5$4. For Effective Coverage, Lessie scores $\mathrm{rel}(p_i)\ge 0.5$5, Exa $\mathrm{rel}(p_i)\ge 0.5$6, Claude Code $\mathrm{rel}(p_i)\ge 0.5$7, and Juicebox $\mathrm{rel}(p_i)\ge 0.5$8. For Information Utility, Lessie scores $\mathrm{rel}(p_i)\ge 0.5$9, Exa $[0,100]$0, Claude Code $[0,100]$1, and Juicebox $[0,100]$2 (Wang et al., 29 Mar 2026).

Qualified-result yield is reported separately from the composite coverage metric. Lessie achieves 100% task completion with $[0,100]$3 qualified results per query. Exa achieves $[0,100]$4 qualified results per query, Claude Code $[0,100]$5, and Juicebox $[0,100]$6 (Wang et al., 29 Mar 2026).

Scenario-level results show nonuniform difficulty. In Recruiting, all platforms are above 64 overall, and Juicebox leads in Information Utility with $[0,100]$7. In B2B Prospecting, Lessie leads with $[0,100]$8 and holds a 12.8-point margin in Relevance Precision. In Expert Search with Deterministic Answers, Lessie scores $[0,100]$9, while Claude Code is second in Precision with $$\mathrm{DCG@}K=\sum_{i=1}^{K}\frac{\mathrm{rel}(p_i)}{\log_{2}(i+1)}.$$0. In Influencer/KOL Discovery, the spread is widest, and Lessie leads by more than 27 points in Precision over Juicebox (Wang et al., 29 Mar 2026).

These results indicate that the benchmark differentiates sharply among systems in query classes that depend on external web evidence and platform-specific social signals, rather than only on corporate profile retrieval.

6. Validation, robustness, and ablation studies

PeopleSearchBench includes several validation and ablation analyses intended to test whether its scoring pipeline is stable and aligned with human judgment (Wang et al., 29 Mar 2026). In a human validation study over 200 person-query pairs, criterion labels assigned by the LLM judge versus human consensus achieve Cohen’s $$\mathrm{DCG@}K=\sum_{i=1}^{K}\frac{\mathrm{rel}(p_i)}{\log_{2}(i+1)}.$$1 with 95% CI $$\mathrm{DCG@}K=\sum_{i=1}^{K}\frac{\mathrm{rel}(p_i)}{\log_{2}(i+1)}.$$2. The corresponding binary qualified-status decision achieves $$\mathrm{DCG@}K=\sum_{i=1}^{K}\frac{\mathrm{rel}(p_i)}{\log_{2}(i+1)}.$$3 with 95% CI $$\mathrm{DCG@}K=\sum_{i=1}^{K}\frac{\mathrm{rel}(p_i)}{\log_{2}(i+1)}.$$4. Relevance grade correlation reaches Pearson’s $$\mathrm{DCG@}K=\sum_{i=1}^{K}\frac{\mathrm{rel}(p_i)}{\log_{2}(i+1)}.$$5 with 95% CI $$\mathrm{DCG@}K=\sum_{i=1}^{K}\frac{\mathrm{rel}(p_i)}{\log_{2}(i+1)}.$$6.

Criteria extraction stability is evaluated over 150 runs at temperature 0.7. The study reports semantic equivalence in 94.7% of extractions and a mean of 2.73 criteria per query with $$\mathrm{DCG@}K=\sum_{i=1}^{K}\frac{\mathrm{rel}(p_i)}{\log_{2}(i+1)}.$$7 (Wang et al., 29 Mar 2026). Alternative judges, specifically GPT-4o and Claude 3.5, agree with the primary judge, Gemini 3 Flash, at $$\mathrm{DCG@}K=\sum_{i=1}^{K}\frac{\mathrm{rel}(p_i)}{\log_{2}(i+1)}.$$8. Prompt variants described as simplified and chain-of-thought enhanced yield 88.5%–93.2% agreement with the original prompts, with latency trade-offs.

The ablations report stable platform rankings under several perturbations: qualified threshold at 0.3, 0.5, and 0.7; top-$$\mathrm{DCG@}K=\sum_{i=1}^{K}\frac{\mathrm{rel}(p_i)}{\log_{2}(i+1)}.$$9 padding at $\mathrm{IDCG@10}$0; precision-heavy, coverage-heavy, utility-heavy, and optimized weighting schemes; and removal of partial credit in favor of binary-only scoring (Wang et al., 29 Mar 2026). Removing Information Utility from the overall score causes Juicebox to drop below Claude Code, which the paper interprets as validating the importance of the utility dimension.

The benchmark also reports a latency breakdown: web verification at approximately 54 seconds per query dominates runtime, and the full pipeline can be parallelized to complete all 476 runs in approximately 1.2 hours on eight workers (Wang et al., 29 Mar 2026). This makes the methodology operationally feasible, but it also indicates that live evidence gathering rather than ranking itself is the primary computational bottleneck.

7. Relation to other “people search” literatures and uses of the name

The term people search is not uniform across the literature. In computer vision, person search typically denotes the joint solution of person detection and person re-identification, as stated explicitly in "Sequential End-to-end Network for Efficient Person Search" (Li et al., 2021) and "Multi-Attribute Enhancement Network for Person Search" (Chen et al., 2021). That line of work is evaluated on benchmarks such as CUHK-SYSU and PRW, and has expanded to settings such as multi-scale matching (Lan et al., 2018), partial matching with LSPS (Zhong et al., 2020), and cross-platform ground-to-aerial search with G2APS (Zhang et al., 2023).

The label PeopleSearchBench has also appeared in other contexts. "Bootstrapped Grouping of Results to Ambiguous Person Name Queries" describes a benchmark for ambiguous person-name search built from 50 Wikipedia names and approximately 5,000 Google pages, framing the task as mapping retrieved documents to bootstrapped entity profiles plus a noise class (Gruetze et al., 2013). Separately, "Person Search in Videos with One Portrait Through Visual and Temporal Links" presents a large-scale video person-search dataset of 127K manually annotated tracklets from 192 movies; in the supplied summary, that dataset is referred to as PeopleSearchBench and also as CSM in the original paper (Huang et al., 2018).

The 2026 PeopleSearchBench therefore occupies a distinct niche. Rather than benchmarking detector–re-ID pipelines or ambiguous-name clustering, it evaluates AI-powered people-search platforms used in recruiting, sales prospecting, expert search, and influencer discovery, and it does so through factual verification, coverage analysis, and utility assessment (Wang et al., 29 Mar 2026). This distinction is important because the benchmarked object is not a visual recognition model or a name-disambiguation classifier, but an end-to-end system that must interpret natural-language constraints, surface candidate people, and provide enough evidence and structure for downstream use.