Molecular Preference-Aligned Instance Ranking

• The paper introduces Mole-PAIR, a framework that reformulates OOD detection as a pairwise preference learning problem to directly optimize AUROC-based ranking.
• The methodology attaches a lightweight scoring head to a frozen molecular encoder and employs a negative log-likelihood loss with β-temperature scaling and ℓ2 regularization to focus on challenging ID–OOD pairs.
• Empirical results demonstrate substantial improvements in AUROC and reduced false positive rates across multiple molecular benchmarks under scaffold, assay, and size shifts.

Searching arXiv for Mole-PAIR and closely related molecular preference-ranking work. Molecular Preference-Aligned Instance Ranking (Mole-PAIR) denotes a post-training framework for molecular foundation models in which out-of-distribution (OOD) detection is formulated as a pairwise preference-learning problem over molecular instances. In its named formulation, Mole-PAIR attaches a small scoring head to a frozen molecular encoder and learns an OOD affinity score whose induced ordering places in-distribution (ID) molecules below OOD molecules, thereby directly targeting ranking-based OOD metrics such as AUROC rather than pointwise confidence calibration (He et al., 29 Sep 2025).

1. Definition and problem setting

Mole-PAIR is introduced to address a specific reliability failure of molecular foundation models: on OOD molecules, models may make incorrect predictions while outputting high confidence. This behavior is described as chemical hallucination, and it is especially salient under scaffold, size, and assay shifts in drug-discovery settings (He et al., 29 Sep 2025).

The framework assumes an ID distribution $D_{\mathrm{in}}$ and an OOD distribution $D_{\mathrm{out}}$, together with finite train and test subsets derived from these distributions. For a molecular input $S$, Mole-PAIR learns an OOD detector

$g(S; \tau, E_\phi) = \begin{cases} 1 \text{ (ID)}, & E_\phi(S) < \tau,\[2pt] 0 \text{ (OOD)}, & E_\phi(S) \ge \tau, \end{cases}$

where $E_\phi(S)$ is an OOD affinity score and larger values indicate that a molecule is more OOD-like (He et al., 29 Sep 2025).

The intended ranking property is pairwise: $$E_\phi(S_{\mathrm{in}}) < E_\phi(S_{\mathrm{out}}), \quad \forall S_{\mathrm{in}} \sim D_{\mathrm{in}},\ S_{\mathrm{out}} \sim D_{\mathrm{out}}.$$ This formulation makes Mole-PAIR an instance-ranking method in a literal sense: the objects being ordered are molecular instances themselves, not bags, trajectories, or class logits. The article introducing the method studies this formulation on DrugOOD and GOOD benchmarks under assay, scaffold, size, and covariate shifts, treating OOD detection as an ordering problem over molecules rather than a calibrated binary classification problem (He et al., 29 Sep 2025).

A common misconception is that Mole-PAIR is primarily a generative alignment method. In its named 2025 formulation, it is instead a reliability module for OOD detection. The broader phrase “preference-aligned instance ranking” has close conceptual links to earlier molecular preference-learning and ranking literature, but the specific method called Mole-PAIR is defined around OOD affinity scoring and AUROC-aligned pairwise learning (He et al., 29 Sep 2025).

2. Pairwise preference formulation and AUROC alignment

Mole-PAIR reframes OOD detection as a preference problem over ID–OOD pairs. Given an encoder $f_{\mathrm{Encoder}}$ and a scoring head $g_{\mathrm{Head}}$, the scalar score is

$$E_\phi(S) = g_{\mathrm{Head}}( f_{\mathrm{Encoder}}(S); \phi ) \in \mathbb{R}.$$

The preference relation is encoded as

$$S_{\mathrm{in}} \succ S_{\mathrm{out}} \quad \Leftrightarrow \quad E_\phi(S_{\mathrm{out}}) - E_\phi(S_{\mathrm{in}}) > 0,$$

and the pairwise probability is modeled with a Bradley–Terry / DPO-style logistic form,

$D_{\mathrm{out}}$0

where $D_{\mathrm{out}}$1 is a temperature (He et al., 29 Sep 2025).

The core training objective is the negative log-likelihood over ID–OOD pairs: $D_{\mathrm{out}}$2 Because this loss depends only on score differences, the method adds an $D_{\mathrm{out}}$3 regularization on scores,

$D_{\mathrm{out}}$4

with small $D_{\mathrm{out}}$5, to fix the global shift and prevent unbounded score drift (He et al., 29 Sep 2025).

The central theoretical claim is that this objective is a smooth surrogate for the pairwise ranking loss underlying AUROC. In the OOD setting,

$D_{\mathrm{out}}$6

so AUROC depends only on the ordering induced by $D_{\mathrm{out}}$7. The analysis in the Mole-PAIR paper establishes a local pairwise optimum

$D_{\mathrm{out}}$8

whose sign matches $D_{\mathrm{out}}$9, and a global population risk whose minimizer induces the same ordering as the true ID posterior almost everywhere, thereby achieving the Bayes-optimal AUROC in a sufficiently rich function class (He et al., 29 Sep 2025).

The gradient structure also emphasizes hard pairs. If

$S$0

then the per-pair gradient weight is

$S$1

which is largest for misranked or borderline pairs and decays for already well-separated pairs. The associated first-order margin improvement,

$S$2

formalizes the claim that training concentrates on the molecular comparisons most detrimental to AUROC (He et al., 29 Sep 2025).

This ranking-centric view is closely aligned, in a broader statistical sense, with the notion of a model-level ordinal observable defined by pairwise reference alignment. A plausible interpretation is that Mole-PAIR instantiates such an observable in the molecular OOD setting, with the reference pair distribution induced by ID–OOD comparisons and the model score given by $S$3 (Li, 29 May 2026).

3. Architecture, integration, and optimization

Mole-PAIR is designed as a plug-and-play module. The molecular foundation encoder is frozen, and only a lightweight detector head is trained. The paper evaluates this setup with two pretrained encoders: MiniMol (2D), which converts SMILES to a 2D molecular graph and produces a 512-dimensional embedding through a GNN, and Uni-Mol (3D), which builds 3D conformations from SMILES and produces a 512-dimensional representation through a transformer-based 3D model (He et al., 29 Sep 2025).

The detector head is a small MLP with architecture 512 → 256 → 128 → 1 and dropout $S$4. Training uses balanced mini-batches containing ID and OOD samples, from which ID–OOD pairs are formed. The encoder remains unchanged throughout post-training, so Mole-PAIR does not alter the pretrained molecular representation space directly; it learns a ranking functional over that space (He et al., 29 Sep 2025).

The reported optimization setup is explicit. The loss is $S$5 above, optimized with AdamW, learning rate $S$6, weight decay $S$7, StepLR with step size $S$8 and $S$9, and gradient clipping with norm $g(S; \tau, E_\phi) = \begin{cases} 1 \text{ (ID)}, & E_\phi(S) < \tau,\[2pt] 0 \text{ (OOD)}, & E_\phi(S) \ge \tau, \end{cases}$0. The default temperature is $g(S; \tau, E_\phi) = \begin{cases} 1 \text{ (ID)}, & E_\phi(S) < \tau,\[2pt] 0 \text{ (OOD)}, & E_\phi(S) \ge \tau, \end{cases}$1, the regularization is approximately $g(S; \tau, E_\phi) = \begin{cases} 1 \text{ (ID)}, & E_\phi(S) < \tau,\[2pt] 0 \text{ (OOD)}, & E_\phi(S) \ge \tau, \end{cases}$2, training runs up to $g(S; \tau, E_\phi) = \begin{cases} 1 \text{ (ID)}, & E_\phi(S) < \tau,\[2pt] 0 \text{ (OOD)}, & E_\phi(S) \ge \tau, \end{cases}$3 epochs, and batch sizes are $g(S; \tau, E_\phi) = \begin{cases} 1 \text{ (ID)}, & E_\phi(S) < \tau,\[2pt] 0 \text{ (OOD)}, & E_\phi(S) \ge \tau, \end{cases}$4 for MiniMol and $g(S; \tau, E_\phi) = \begin{cases} 1 \text{ (ID)}, & E_\phi(S) < \tau,\[2pt] 0 \text{ (OOD)}, & E_\phi(S) \ge \tau, \end{cases}$5 for Uni-Mol (He et al., 29 Sep 2025).

At inference time, the procedure is minimal: compute $g(S; \tau, E_\phi) = \begin{cases} 1 \text{ (ID)}, & E_\phi(S) < \tau,\[2pt] 0 \text{ (OOD)}, & E_\phi(S) \ge \tau, \end{cases}$6 for each molecule and use it as the OOD score. AUROC and AUPR require no threshold tuning, while deployment as a detector uses a threshold $g(S; \tau, E_\phi) = \begin{cases} 1 \text{ (ID)}, & E_\phi(S) < \tau,\[2pt] 0 \text{ (OOD)}, & E_\phi(S) \ge \tau, \end{cases}$7 chosen to satisfy a desired operating point such as FPR95 on validation data. This architecture-level separation between a frozen encoder and a learned ranking head is central to the method’s low post-training cost and broad encoder compatibility (He et al., 29 Sep 2025).

4. Benchmarks, baselines, and empirical performance

The experimental study covers five real-world molecular datasets and multiple realistic shifts. On DrugOOD’s ligand-based affinity prediction tasks, the study uses assay, scaffold, and size splits for both EC50 and IC50 settings. On GOOD, it uses covariate-shift splits for GOOD-HIV, GOOD-PCBA, and GOOD-ZINC. For detector training and evaluation, the paper further down-samples to balanced ID/OOD subsets, such as train_id = 2000 and train_ood = 2000 on DrugOOD, and train_id/ood = 5000 on GOOD (He et al., 29 Sep 2025).

The baseline set is broad: MSP, ODIN, Energy, Mahalanobis, KNN, and LOF, all using the same frozen encoder and a supervised classifier head trained on ID data and, in some cases, OOD data. Evaluation uses AUROC and AUPR with larger being better, and FPR95 with smaller being better (He et al., 29 Sep 2025).

The headline result is that Mole-PAIR improves OOD detection substantially, with up to 45.8%, 43.9%, and 24.3% improvements in AUROC under distribution shifts of size, scaffold, and assay, respectively. Representative AUROC results include MiniMol, EC50-Scaffold: best baseline Energy 0.685 versus Mole-PAIR 0.970; MiniMol, IC50-Scaffold: best baseline KNN 0.655 versus Mole-PAIR 0.983; MiniMol, PCBA-Scaffold: best baseline Energy 0.642 versus Mole-PAIR 0.924; and Uni-Mol, ZINC-Size: best baseline KNN 0.686 versus Mole-PAIR 1.000 (He et al., 29 Sep 2025).

The supplementary metrics show corresponding reductions in false acceptance of OOD molecules. Examples include MiniMol, EC50-Scaffold, where the best baseline FPR95 is 0.716 and Mole-PAIR achieves 0.178, and MiniMol, IC50-Scaffold, where the best baseline FPR95 is 0.563 and Mole-PAIR achieves 0.084. The paper interprets these reductions as a direct mitigation of chemical hallucination, because OOD molecules are much less likely to be misclassified as ID at high true-positive operating points (He et al., 29 Sep 2025).

The method is not uniformly dominant on every split. On MiniMol, ZINC-Scaffold, the best baseline Mahalanobis 0.638 slightly exceeds Mole-PAIR 0.614. This exception matters because it shows that the pairwise ranking objective is not a universal replacement for distance-based OOD methods in every low-signal or noise-dominated regime (He et al., 29 Sep 2025).

Ablations show that $g(S; \tau, E_\phi) = \begin{cases} 1 \text{ (ID)}, & E_\phi(S) < \tau,\[2pt] 0 \text{ (OOD)}, & E_\phi(S) \ge \tau, \end{cases}$8 and $g(S; \tau, E_\phi) = \begin{cases} 1 \text{ (ID)}, & E_\phi(S) < \tau,\[2pt] 0 \text{ (OOD)}, & E_\phi(S) \ge \tau, \end{cases}$9 are consequential, especially on harder assay and scaffold shifts. On EC50–Scaffold, best AUROC occurs around $E_\phi(S)$0; on EC50–Assay, best performance occurs around $E_\phi(S)$1. Very small $E_\phi(S)$2 is preferred on assay shifts, modest $E_\phi(S)$3 is favorable on scaffold shifts, and size shifts remain robust until $E_\phi(S)$4 becomes too large. Training-dynamics analyses show that the misranked fraction and boundary mass decrease quickly while the mean margin grows steadily, supporting the claim that the pairwise objective progressively sharpens ID–OOD separation (He et al., 29 Sep 2025).

5. Conceptual antecedents and related preference-ranking formulations

Although the named framework is introduced for OOD detection in 2025, its formulation sits within a wider body of molecular and bioinformatics work on pairwise preference learning, instance ranking, and latent-instance scoring.

In "pyLEMMINGS: Large Margin Multiple Instance Classification and Ranking for Bioinformatics Applications", multiple instance learning is used to rank bags by the score of their highest-scoring instance,

$E_\phi(S)$5

and ranking is optimized with a pairwise large-margin loss

$E_\phi(S)$6

This framework localizes binding sites, prion-forming domains, and amyloid cores from bag-level labels by inducing instance scores $E_\phi(S)$7 or $E_\phi(S)$8, and it offers a direct antecedent for the idea that coarse molecular preferences can induce fine-grained instance ranking (Asif et al., 2017).

In "Preference Optimization for Molecular LLMs", molecular LLMs over SMILES are aligned to chemist-like preferences through Direct Preference Optimization, using pairwise comparisons $E_\phi(S)$9 derived from medicinal chemist filters or predicted EGFR activity. The DPO loss

$$E_\phi(S_{\mathrm{in}}) < E_\phi(S_{\mathrm{out}}), \quad \forall S_{\mathrm{in}} \sim D_{\mathrm{in}},\ S_{\mathrm{out}} \sim D_{\mathrm{out}}.$$0

makes the model rank preferred molecules above dispreferred ones in its induced probability ordering. This suggests a generative interpretation of preference-aligned instance ranking in which likelihood itself becomes the ranking score (Park et al., 2023).

In "Mol-MoE: Training Preference-Guided Routers for Molecule Generation", the preference signal is multi-objective. A molecule has property vector $$E_\phi(S_{\mathrm{in}}) < E_\phi(S_{\mathrm{out}}), \quad \forall S_{\mathrm{in}} \sim D_{\mathrm{in}},\ S_{\mathrm{out}} \sim D_{\mathrm{out}}.$$1, a preference vector $$E_\phi(S_{\mathrm{in}}) < E_\phi(S_{\mathrm{out}}), \quad \forall S_{\mathrm{in}} \sim D_{\mathrm{in}},\ S_{\mathrm{out}} \sim D_{\mathrm{out}}.$$2 specifies trade-offs, and scalarized utility is

$$E_\phi(S_{\mathrm{in}}) < E_\phi(S_{\mathrm{out}}), \quad \forall S_{\mathrm{in}} \sim D_{\mathrm{in}},\ S_{\mathrm{out}} \sim D_{\mathrm{out}}.$$3

A mixture-of-experts router is then trained by reinforcement learning to generate molecules aligned with the requested trade-off, enabling test-time steering without retraining. This is not a pairwise OOD detector, but it broadens the preference-alignment perspective by treating user-specified molecular trade-offs as control signals that induce an implicit ranking over generated instances (Calanzone et al., 8 Feb 2025).

In "siDPT: siRNA Efficacy Prediction via Debiased Preference-Pair Transformer", efficacy prediction is recast as preference learning within target-specific groups. The model constructs preference pairs from inhibition differences, uses a Bradley–Terry-style probability

$$E_\phi(S_{\mathrm{in}}) < E_\phi(S_{\mathrm{out}}), \quad \forall S_{\mathrm{in}} \sim D_{\mathrm{in}},\ S_{\mathrm{out}} \sim D_{\mathrm{out}}.$$4

and introduces a noise-aware target distribution with gene-specific temperature $$E_\phi(S_{\mathrm{in}}) < E_\phi(S_{\mathrm{out}}), \quad \forall S_{\mathrm{in}} \sim D_{\mathrm{in}},\ S_{\mathrm{out}} \sim D_{\mathrm{out}}.$$5. This makes pair construction, ranking supervision, and debiasing explicit in a molecular-sequence setting, showing that preference-pair learning is useful not only for OOD detection or generation but also for efficacy ranking under heterogeneous experimental noise (Zhang et al., 19 Sep 2025).

Related work on ReAlignFit emphasizes pair-adaptive representational alignment between molecular substructures under chemical induced fit, while Preference-Based Dynamic Ranking Structure Recognition develops dynamic Bradley–Terry grouping and change-point detection for preference data. These works do not define Mole-PAIR, but they underscore two broader themes: molecular ranking benefits from chemically grounded representations, and preference structure may itself be dynamic or grouped rather than purely pointwise (Zhang et al., 7 Feb 2025, Lu et al., 29 Sep 2025).

6. Limitations, misconceptions, and significance

Mole-PAIR has explicit limitations. It requires some OOD supervision: labeled ID and OOD subsets or environments are needed to construct preference pairs. In a fully unsupervised setting with no OOD signals, the method cannot be applied directly. Its effectiveness also depends on the frozen encoder having enough latent separability between ID and OOD; if the underlying representation is poor, the ranking head has limited capacity to recover robust OOD structure (He et al., 29 Sep 2025).

Another misconception is that AUROC optimization eliminates all reliability issues. The paper does not make that claim. It states that Mole-PAIR aligns the training objective with the ranking metric used for OOD detection, but it also notes sensitivity to $$E_\phi(S_{\mathrm{in}}) < E_\phi(S_{\mathrm{out}}), \quad \forall S_{\mathrm{in}} \sim D_{\mathrm{in}},\ S_{\mathrm{out}} \sim D_{\mathrm{out}}.$$6 and $$E_\phi(S_{\mathrm{in}}) < E_\phi(S_{\mathrm{out}}), \quad \forall S_{\mathrm{in}} \sim D_{\mathrm{in}},\ S_{\mathrm{out}} \sim D_{\mathrm{out}}.$$7 on harder shifts, and reports the ZINC scaffold case where a baseline is slightly better. The method therefore reduces, rather than abolishes, failure under distribution shift (He et al., 29 Sep 2025).

Its broader significance lies in changing the framing of molecular reliability. Classical OOD methods such as MSP, ODIN, and Energy depend on absolute logits, energies, or confidence calibration. Mole-PAIR instead treats OOD detection as an ordinal problem over molecular instances. This is well matched to molecular practice, where many downstream decisions are ranking-based and where relative judgments can be more robust than absolute probabilities under scaffold, assay, and size shifts (He et al., 29 Sep 2025).

The article introducing Mole-PAIR also points toward extensions to protein sequence or structure foundation models, material-science models such as crystals and polymers, and multi-modal protein–ligand or drug–target encoders. A plausible implication is that the framework’s core contribution is not restricted to OOD detection per se, but to the use of pairwise preference learning as a modular mechanism for reliability-oriented ranking in scientific foundation models more broadly (He et al., 29 Sep 2025).