Reviewer Recommendation and Assignment

• Reviewer recommendation and assignment is a process that matches submissions with appropriate experts by estimating suitability and solving combinatorial optimization problems.
• It combines information-retrieval techniques with optimization-based methods to compute similarity scores and enforce constraints such as load balance, fairness, and diversity.
• Recent advances integrate neural models, multi-objective optimization, and robust conflict management to enhance accuracy and mitigate manipulation in the reviewer assignment process.

Peer review is a crucial mechanism underpinning scientific quality and researcher advancement, with the efficacy of the process fundamentally contingent on the optimal assignment of submissions to suitable reviewers. Reviewer recommendation and assignment encompasses a diverse set of algorithmic, workflow, and community-driven methodologies aimed at ensuring this match is accurate, fair, diverse, and robust to practical constraints such as conflicts of interest and manipulation. Advances in this area address both the calculation of reviewer-paper similarity and the assignment optimization process and increasingly integrate multiple objectives, including diversity, fairness, and workload balance.

1. Taxonomies and Fundamental Problem Formulations

The reviewer assignment problem (RAP) decomposes into two principal aspects: estimation of reviewer–submission suitability and combinatorial assignment optimization subject to constraints. According to recent survey literature, approaches fall into two broad categories (Aksoy et al., 2023):

• Information-Retrieval–Based RAP (IRRAP): Includes manual (bidding, keywords), content-based (TF–IDF, topic models like LDA, semantic similarity), neural LLMs (static and contextual embeddings), and network-based techniques (co-authorship/citation graphs).
• Optimization-Based RAP (ORAP): Framed as linear or integer programs maximizing total suitability, subject to capacity, load, and conflict constraints, and further extended to fairness (max-min, leximin), diversity (entropy or attribute constraints), or robustness.

The generic assignment problem is formalized as: $$\begin{align*} & \max_{x \in \{0,1\}^{N \times M}} \sum_{i=1}^N \sum_{j=1}^M S_{ij} x_{ij} \ \text{s.t.}\quad & \sum_{j=1}^M x_{ij} = k, \quad \forall i \quad (\text{each submission assigned } k \text{ reviewers})\ & L_j \leq \sum_{i=1}^N x_{ij} \leq U_j, \quad \forall j \quad (\text{reviewer load bounds}) \ & x_{ij} = 0, \quad \forall (i,j) \in \mathcal{F} \quad (\text{conflicts}) \end{align*}$$ where $S_{ij}$ is the suitability or affinity score.

2. Suitability Modeling: Signals, Algorithms, and Evaluation

A diversity of signals and machine learning methods underpins the computation of reviewer–paper suitability scores, with performance evaluated by empirical precision and coverage metrics.

• Classical approaches:
  • Keywords/Taxonomies: Both authors and reviewers select from taxonomies, with semantic distances computed via hierarchical trees or ontologies (Kalmukov, 2011).
  • Bidding systems: Direct relevance but sparse and vulnerable to gaming (Aksoy et al., 2023).
• Text-based and Embedding Models:
  • Vector-Space/TF–IDF: Cosine similarity over (title+abstract or full text) vectors (Stelmakh et al., 2023, Aksoy et al., 2023).
  • Topic models: LDA, PLSA compute topic-proportion vectors for similarity measures (Carpenter et al., 2024, Aksoy et al., 2023).
  • Neural models: Static (Word2Vec, Doc2Vec) and contextualized (BERT, SciBERT, SPECTER) embeddings for sophisticated semantic representation. Dense retrievers (e.g., SPECTER) yield 3–5 points higher nDCG@10 than BM25 baselines in large-scale benchmarks (Ebrahimi et al., 11 Feb 2025).
• Community-grounded/data-driven learning:
  • Multi-label learning over hierarchical field taxonomies (e.g., Hiepar-MLC with attention-based Bi-GRU) outperforms LDA/BERT for label recovery and interpretable assignment by significant margins (Zhang et al., 2019).
• Role/context–aware semantic profiling:
  • Submission–aware clique-based methods enhance explainability and top-k reviewer precision, improving over embedding and standard topic-modeling baselines (Anjum et al., 2022).
  • Graph/hypergraph and social-network–based approaches capture non-local collaborative or knowledge signals (e.g., multiplex hyperedges in code review (Qiao et al., 2024)).

Evaluation of suitability models is increasingly empirical, leveraging “gold standard” datasets with self-assessed expertise labels (Stelmakh et al., 2023) and IR metrics (nDCG, MAP, Recall@k, set overlap). Notably, there remains heavy tail error: even best-in-class similarity models misrank 12–30% of easy, and 36–43% of hard, reviewer–paper pairs (Stelmakh et al., 2023).

3. Assignment Algorithms: Optimization, Diversity, and Fairness

Assignment algorithms map computed suitability matrices to reviewer–submission pairings while enforcing institutional, fairness, and operational constraints.

• Flow and Linear Programming Approaches:
  • Classical maximum-weight matching reduces to min-cost flow or assignment LP for binary variables. Advanced approaches integrate fairness (max-min, leximin), diversity, and strategy-proofing through side-constraints and multi-objective formulations (0906.4044, Carpenter et al., 2024, Aksoy et al., 2023).
  • Algorithms such as PeerReview4All and FairSequence (the latter in OpenReview) guarantee fairness via envy-freeness up to one item (EF1/WEF1) while maintaining ≥92% of utilitarian welfare (Payan et al., 2021).
• Diversity-Enriched Assignment:
  • DiveRS augments suitability-based assignments with entropy-based diversity metrics across reviewer attributes (background, region, seniority), solved iteratively as a min-cost flow with marginal diversity linearization. Moderate trade-off yields up to +14% diversity (entropy) at only 5% topical-fit loss; 85% of newly suggested reviewers judged “highly relevant” by human panels (Kreutz et al., 2022).
• Robustness and Randomization:
  • Perturbed maximization frameworks (PM, RAMP) inject controlled randomness into assignments to mitigate manipulation, improve anonymity, and enhance diversity, via concave rewards and attribute-aware sampling over the Birkhoff–von Neumann polytope. This yields higher assignment entropy, larger support size, and dramatically fewer coassignments among coauthors or reciprocating duos (Xu et al., 2023, Cui et al., 20 Jan 2026).
  • Robust max-min assignment under affinity uncertainty (RRA) leverages adversarial optimization over specified uncertainty sets (box, budgeted, ellipsoidal), yielding assignments that guarantee worst-case utilitarian social welfare within 1–2% of optimal even under high affinity noise (Cousins et al., 2023).
• Reviewer Set Construction:
  • Multi-criteria algorithmic frameworks (RevASIDE) explicitly multiply five normalized component scores (expertise, authority, interest, diversity, seniority) to optimality, subject to conflict and coauthorship constraints, using exact enumeration within ranked candidate lists. This consistently outperforms both pure expert and random baselines, and is preferred by human evaluators in terms of balance and complementarity (Kreutz et al., 2021).

4. Conflict Management, Practical Constraints, and System Integration

Conflict-of-interest (COI) detection is an indispensable part of assignment integrity. Systems combine explicit self-reports (bids, keyword selection) with automatic COI detection via institutional matching, co-authorship detection using local and bibliographic indices (DBLP, Google Scholar), and increasingly, web and network data (Kalmukov, 2011, Mahmud et al., 26 Jun 2025). Load balancing (reviewer capacity/demand) is encoded as hard or soft constraints in assignment optimization, with additional sampling-based or reranking workload balancing for dynamic workloads in code review and collaboration platforms (Rigby et al., 2023).

Production systems in large conferences or enterprises (e.g., ALMA, Meta/Phabricator) integrate modern assignment algorithms with prediction, data ingestion, and matching infrastructure:

• Feature extraction pipelines for suitability computation (embedding databases, precomputed code- or author-centric features),
• Assignment engines based on scalable LP, flow, or greedy mechanisms,
• Evaluation, transparency, and interface layers for manual oversight or hybrid assignments (Carpenter et al., 2024, Rigby et al., 2023, Kalmukov, 2011).

5. Empirical Results, Benchmarks, and Community Assessment

Recent benchmarks and deployment studies have shifted evaluation from purely algorithmic or synthetic data to comprehensive, dataset-driven and end-to-end system settings:

• Benchmarks: exHarmony offers a large-scale open dataset for CAB tasks in computer science, with “weak gold” via author/citation/similarity–induced reviewer pools, supporting dense, lexical, and hybrid retrieval models (Ebrahimi et al., 11 Feb 2025).
• Real-World A/B Testing: Meta’s production A/B tests reveal that learn-to-rank recommenders using precomputed reviewer–author interaction features yield both ∼14× lower latency and up to +14% Top-3 accuracy improvements over blame/ownership models. Workload rebalancing (WhoDo-style reranking) can reduce overload by 18% with modest (∼5pp) accuracy loss. For group assignments, explicit individual selection reduces review completion time by 12% with no observed negative impacts (Rigby et al., 2023).
• Human Evaluation: Human judges consistently prefer multi-aspect optimized reviewer sets (RevASIDE) and context-aware semantic recommenders due to increased diversity/authority balance, explainability, and reduced bias (Kreutz et al., 2021, Anjum et al., 2022).

6. Open Challenges, Limitations, and Future Directions

Despite substantial advances, several challenges remain, as highlighted in the RAP survey (Aksoy et al., 2023) and recent empirical work:

• Profiling and Data Expansion: Dynamic enlargement of the reviewer pool via external bibliometric sources (ArnetMiner, ORCID), and integration of reviewer reliability signals from review histories.
• Similarity Modeling: Continued exploration of domain-tuned transformers, hybrid neural/IR models, and full document/citation/context integration to further close the performance gap between human and algorithmic expertise assessments (Stelmakh et al., 2023, Ebrahimi et al., 11 Feb 2025).
• Diversity and Fairness: Explicit incorporation of diversity regularization (e.g., DPPs, Maximal Marginal Relevance), multi-objective optimization, and strategy-proof mechanisms in large-scale assignment (Kreutz et al., 2022, Xu et al., 2023, Cui et al., 20 Jan 2026).
• Robustness and Uncertainty: Robust assignment under noisy or adversarial suitability estimation, as well as resilience to reviewer collusion, reviewer pool churn, and manipulation (Cousins et al., 2023, Cui et al., 20 Jan 2026).
• Standardization and Evaluation: Development of public datasets with real assignments, COI annotations, and user-centered benchmarks, enabling statistical comparability across methods and objective tuning aligned with actual program chair/editorial needs (Stelmakh et al., 2023, Ebrahimi et al., 11 Feb 2025).

Recent advances demonstrate that hybrid, interpretable, and multi-criteria systems—fusing semantic, social, operational, and fairness signals—are both deployable at scale and empirically beneficial for review quality, efficiency, and equity. Nonetheless, further progress in data integration, active learning (e.g., for cold-start or niche submissions), and standardization of evaluation is needed as RAP remains a central challenge in scholarly publishing and software collaboration.