SynRXN: Unified Benchmarking for CASP
- SynRXN is a unified benchmarking framework for computer-aided synthesis planning that standardizes datasets and provides reproducible, leakage-aware evaluations.
- It organizes reaction tasks into five families—reaction rebalancing, atom-to-atom mapping, reaction classification, property prediction, and route design—to enable consistent comparisons.
- It implements deterministic standardization, leakage-aware splitting, and transparent evaluation workflows to mitigate contamination and enhance benchmarking integrity.
to=arxiv_search.search 一级a做爰片 {"query":"SynRXN benchmark curated dataset computational reaction modeling arXiv (Phan et al., 5 Jan 2026)", "max_results": 5} to=arxiv_search.search 天天中彩票是 {"query":"\"Autonomous Discovery of Unknown Reaction Pathways from Data by Chemical Reaction Neural Network\" (Ji et al., 2020)", "max_results": 3} to=arxiv_search.search 玩彩神争霸 {"query":"\"SynCoGen: Synthesizable 3D Molecule Generation via Joint Reaction and Coordinate Modeling\" (Rekesh et al., 16 Jul 2025)", "max_results": 3} SynRXN is a unified benchmarking framework and curated dataset suite for computer-aided synthesis planning (CASP), introduced as an open-data resource for computational reaction modeling (Phan et al., 5 Jan 2026). In its primary usage, SynRXN decomposes end-to-end synthesis planning into five task families—reaction rebalancing, atom-to-atom mapping, reaction classification, reaction property prediction, and synthesis route design—and packages harmonized corpora, leakage-aware splitting functions, and standardized evaluation workflows for each (Phan et al., 5 Jan 2026). In adjacent literature, the name also functions as a systems-level shorthand for synthesis-aware reaction modeling pipelines, including reaction-network discovery, reaction-centric molecular generation, reaction embeddings, route-constrained design, and mechanistic exploration (Ji et al., 2020, Rekesh et al., 16 Jul 2025, Liu et al., 7 Jan 2026, Seo et al., 2024, Unsleber et al., 2022). This dual usage makes SynRXN both a specific benchmark release and a broader organizing concept for reaction informatics.
1. Definition and organizing role
SynRXN was created to address three persistent issues in reaction informatics: dataset heterogeneity, fair comparison and contamination control, and reproducibility (Phan et al., 5 Jan 2026). The framework harmonizes representations via deterministic standardization with SynKit Standardize and CanonRSMI, canonicalizes reaction SMILES, removes exact and near-duplicate records, and provides deterministic, leakage-aware splitting functions with fixed random seed $42$ (Phan et al., 5 Jan 2026). For sensitive benchmarking, it combines public training and validation data with held-out gold-standard test sets, while contamination-prone tasks such as reaction rebalancing and atom-to-atom mapping are distributed only as evaluation sets and are explicitly not intended for model training (Phan et al., 5 Jan 2026).
The broader significance of SynRXN lies in how it reorganizes CASP into interoperable subtasks rather than treating synthesis planning as a monolith. This decomposition aligns with neighboring research directions. Reaction-network identification from concentration–time data is addressed by the Chemical Reaction Neural Network, which infers pathways and kinetic parameters directly from time-resolved species concentration data while embedding the Law of Mass Action and Arrhenius Law in the architecture (Ji et al., 2020). Reaction-centric molecular generation is represented by models such as SynCoGen, which samples from the joint distribution of molecular building blocks, chemical reactions, and atomic coordinates, and by RxnFlow, which sequentially assembles molecules using predefined molecular building blocks and chemical reaction templates (Rekesh et al., 16 Jul 2025, Seo et al., 2024). Reaction-level indexing and analysis are represented by RXNEmb, a fixed-length reaction descriptor learned by a pre-trained RXNGraphormer (Liu et al., 7 Jan 2026). This suggests that SynRXN is best understood as a unifying benchmark-centered node within a larger synthesis-aware computational ecosystem.
2. Task families and corpus structure
SynRXN organizes the reaction-informatics pipeline into five task families and provides versioned datasets, manifests, and standardized evaluation scaffolding for each (Phan et al., 5 Jan 2026).
| Task family | Scope | Example datasets |
|---|---|---|
| Reaction rebalancing | Restore element- and charge-conserving stoichiometry | MNC, MOS, MBS, Complex |
| Atom-to-atom mapping | Assign correspondence between reactant and product atoms | Golden, Jaworski/NatComm, USPTO_3K, Recon3D, EColi |
| Reaction classification | Map reactions to predefined classes | Schneider, USPTO_TPL, USPTO_50K, SynTemp, ECREACT |
| Reaction property prediction | Predict continuous reaction attributes | B97XD3, SNAr, E2SN2, Rad6Re, LogRate, Phosphatase, E2, SN2, RDB7, CycloAdd, RGD1 |
| Synthesis route design | Compose one-step predictions into multi-step synthetic routes | USPTO_50K, USPTO_MIT, USPTO_500 |
For reaction rebalancing, SynRXN distributes canonical, unmapped reaction SMILES and evaluates corrected reaction SMILES against curated ground truth under identical standardization (Phan et al., 5 Jan 2026). The released corpora comprise MNC with examples, MOS with , MBS with $491$, and Complex with (Phan et al., 5 Jan 2026). For atom-to-atom mapping, the curated gold standards include small-molecule reactions—Golden , Jaworski/NatComm $491$, USPTO_3K —and $655$ biochemical reactions—Recon3D 0, EColi 1 (Phan et al., 5 Jan 2026).
The reaction classification family spans multiple granularities and label systems. Schneider provides 2 classes with 3 examples each in incomplete and balanced variants and uses a 4 train:validation:test split (Phan et al., 5 Jan 2026). USPTO_TPL provides 5 classes over 6 examples; USPTO_50K provides 7 classes over 8 examples; SynTemp relabels provide 9, 0, and 1 classes at radii R0, R1, and R2 over 2 examples each; and ECREACT provides 3 examples across EC hierarchies with 4, 5, and 6 classes at first, second, and third level (Phan et al., 5 Jan 2026).
The property-prediction family contains both QM-derived and experimentally motivated datasets. Sizes include B97XD3 7, SNAr 8, E2SN2 9, Rad6Re $491$0, LogRate $491$1, Phosphatase $491$2, E2 $491$3, SN2 $491$4, RDB7 $491$5, CycloAdd $491$6, and RGD1 $491$7 (Phan et al., 5 Jan 2026). The synthesis route design family packages USPTO_50K with $491$8 reactions and $491$9 splits, USPTO_MIT with 0 reactions and 1 splits, and USPTO_500 with 2 reactions and 3 splits (Phan et al., 5 Jan 2026).
3. Harmonization, leakage control, and reproducibility
A defining feature of SynRXN is that benchmarking is coupled to deterministic preprocessing and provenance tracking rather than treated as a separate reporting layer (Phan et al., 5 Jan 2026). Each release contains a manifest.json recording relative path, cryptographic checksum, row and column counts, column names, description, and license tag for each file (Phan et al., 5 Jan 2026). Scripted build recipes and a CLI regenerate datasets identically; cross-machine determinism is verified through checksums and CI tests (Phan et al., 5 Jan 2026).
Leakage control is implemented task-specifically. For classification, a deterministic group-wise split procedure first canonicalizes reactions with CanonRSMI, removes duplicates by exact canonical SMILES and structure-level isomorphism, then groups reactions by template when classification is template-based, and finally performs stratified assignment with seed 4 so that each group remains within a single split (Phan et al., 5 Jan 2026). For property prediction, deterministic splits again remove duplicates and may stratify by property bins, while mechanism or template grouping can be used to avoid leakage of near-identical reaction centers (Phan et al., 5 Jan 2026). For route design, deterministic splits, duplicate removal, and template-aware grouping prevent train–test sharing of near-identical templates (Phan et al., 5 Jan 2026).
The contamination policy is strictest for upstream corrective tasks. Reaction rebalancing and atom-to-atom mapping are distributed only as held-out evaluation sets, and training on these labels is disallowed (Phan et al., 5 Jan 2026). This is a deliberate response to the fact that these tasks are especially vulnerable to contamination through memorization of curated corrections or mappings. A common misconception is that SynRXN is simply a collection of public reaction datasets repackaged under a new name. The framework is more specific than that: its distinctive contribution is the combination of deterministic standardization, leakage-aware partitioning, explicit provenance, and evaluation-only handling of contamination-prone tasks (Phan et al., 5 Jan 2026).
4. Evaluation methodology and metric suites
SynRXN standardizes metrics by task type and thereby makes cross-paper comparisons less sensitive to ad hoc evaluation choices (Phan et al., 5 Jan 2026). For reaction rebalancing, the principal criteria are success rate, defined as the fraction of outputs satisfying element-wise and charge balance, and exact-match accuracy, defined as canonical string equality to curated ground truth (Phan et al., 5 Jan 2026). Element conservation is formalized through the stoichiometric constraint 5 and, equivalently, through per-element equality
6
These definitions keep evaluation mapping-free and avoid trivial leakage through formatting artifacts (Phan et al., 5 Jan 2026).
For atom-to-atom mapping, SynRXN reports atom-level mapping accuracy,
7
edge-level precision, recall, and 8 on atom correspondence graphs, and exact match under ITS graph isomorphism (Phan et al., 5 Jan 2026). The mapping validator uses SynKit standardization and ITS graph isomorphism rather than raw mapped SMILES equality, which is important because multiple mapping strings can encode the same correspondence pattern (Phan et al., 5 Jan 2026).
For reaction classification, SynRXN provides accuracy, 9, precision, recall, 0, cross-entropy loss, weighted 1, and multiclass MCC, with repeated cross-validation protocols available for robust variance estimation (Phan et al., 5 Jan 2026). For property prediction, the core regression metrics are MAE, RMSE, and 2 (Phan et al., 5 Jan 2026). For route design, the metric suite is more heterogeneous: route-level exact match, top-3 route accuracy, coverage, recognition rate, 4, max-fragment overlap, and round-trip success are all defined (Phan et al., 5 Jan 2026). Route objectives are also formalized either as cost minimization,
5
or as success-probability maximization,
6
This breadth reflects that route evaluation cannot be reduced to exact string equality alone (Phan et al., 5 Jan 2026).
A practical implication is that SynRXN does not privilege a single modeling paradigm. Its metric design can evaluate graph-based, fingerprint-based, sequence-based, or structured-prediction methods on the same footing, provided they emit outputs in the expected standardized form (Phan et al., 5 Jan 2026).
5. Baselines, empirical patterns, and failure modes
SynRXN includes benchmark baselines and, more importantly, documents characteristic performance asymmetries across domains and task formulations (Phan et al., 5 Jan 2026). On the curated rebalancing test set, SynRBL achieves 7 success and 8 exact-match accuracy, but this is explicitly described as a consequence of manual label verification and test-set construction rather than as evidence of unconstrained automatic robustness (Phan et al., 5 Jan 2026). For atom-to-atom mapping, RXNMapper, GraphormerMapper, LocalMapper, and RDTool perform strongly on USPTO_3K and Golden, while biochemical sets such as Recon3D and EColi are substantially more difficult, with Recon3D reported around 9–0 depending on tool and EColi around 1–2 (Phan et al., 5 Jan 2026).
In reaction classification, DRFP and RXNFP embeddings paired with RandomForest classifiers yield weighted 3 and MCC often greater than 4 on several corpora under stratified repeated cross-validation (Phan et al., 5 Jan 2026). Yet performance declines as label granularity increases: SynTemp R0, R1, and R2 form a graded sequence in which finer reaction-center specificity reduces classification accuracy (Phan et al., 5 Jan 2026). Balanced datasets often outperform incomplete variants, indicating sensitivity to mapping completeness and stoichiometric quality (Phan et al., 5 Jan 2026). In property prediction, RandomForest regressors on DRFP or RXNFP show learnable signal with single-digit to low-tens MAE across datasets, but transition-state properties remain sensitive to representation choice and out-of-distribution transfer (Phan et al., 5 Jan 2026).
SynRXN also makes common failure modes explicit. Upstream imbalance caused by missing inorganic species or byproducts biases downstream mapping and template extraction (Phan et al., 5 Jan 2026). Biochemical mapping is harder because of complex rearrangements, cofactor participation, polymeric species, and ambiguous proton transfers (Phan et al., 5 Jan 2026). Finer-grained template taxonomies impose sharper demands on reaction-center modeling (Phan et al., 5 Jan 2026). A plausible implication is that SynRXN’s main value is not only to rank methods but to expose where reaction-informatics pipelines break under realistic perturbations.
6. SynRXN in the wider synthesis-aware modeling ecosystem
Although SynRXN is formally an open benchmark and curated dataset, its task decomposition mirrors a broader research program in which reaction modeling, route design, dynamics, and mechanistic interpretation are treated as interoperable components (Phan et al., 5 Jan 2026). This wider ecosystem can be organized around several neighboring strands.
A first strand is reaction-network discovery from dynamical data. The Chemical Reaction Neural Network infers the number of elementary reactions, reactant and product stoichiometry, reaction orders, and Arrhenius-form kinetic parameters directly from time-resolved species concentrations and temperature profiles by training a single-hidden-layer neural ODE whose hidden nodes represent candidate elementary reactions (Ji et al., 2020). A related but distinct direct-optimization approach learns reduced and sparse biochemical reaction networks from trajectory data through a dynamically constrained optimization problem with an accelerated proximal gradient algorithm, avoiding derivative estimation and mitigating error accumulation relative to indirect SINDy-style identification (Filo et al., 25 Aug 2025). For programmable synthetic networks, non-competitive CRNs provide rate-independent equilibria determined solely by stoichiometric structure, with a translation from ReLU neural networks to non-competitive CRNs and one bimolecular reaction per neuron in the optimized binary-weight construction (Vasic et al., 2021). RNCRNs extend the design space toward arbitrary dynamics, using executive species and a neural module of chemical perceptrons to approximate target vector fields with error bounded by 5 on compact sets (Dack et al., 2024).
A second strand is reaction prediction and reaction representation. Non-autoregressive electron redistribution modeling predicts reaction outcomes in one shot by decoding bond breaking and bond forming events in parallel through multi-pointer networks, treating reactions as electron redistribution across a molecular graph (Bi et al., 2021). Doubly stochastic graph-based non-autoregressive reaction prediction further constrains these electron-redistribution predictions so that electron-counting and symmetry are satisfied simultaneously by enforcing doubly stochastic self-attention via Sinkhorn normalization (Meng et al., 2023). For reaction-level indexing, RXNEmb provides a fixed-length descriptor 6, where 7, enabling similarity search, data-driven reclustering, reaction-space visualization, and attention-based mechanistic interpretation without reliance on expert rule labels (Liu et al., 7 Jan 2026).
A third strand is synthesis-aware generative design. SynCoGen jointly models building blocks, reaction trees, and 3D coordinates through simultaneous masked graph diffusion and flow matching, using reaction-center compatibility masks, edge-count limits, no-self-edge constraints, and a single-parent tree constraint to keep generation inside a high-yield, high-reliability chemistry regime (Rekesh et al., 16 Jul 2025). RxnFlow instead formulates synthesis-aware molecular design as a GFlowNet over explicit reaction templates and 8 million vendor-available building blocks, with action-space subsampling and importance weighting to scale pocket-conditioned generation while maintaining synthesizability (Seo et al., 2024).
A fourth strand is high-fidelity mechanism exploration and representation interoperability. AutoRXN combines autonomous DFT-based reaction-network discovery, cloud-native CCSD(T) refinement, and automated multi-reference validation and fallback through T1 and orbital-entanglement diagnostics, producing reaction-network graphs with method-aware uncertainty labels (Unsleber et al., 2022). At the representation level, an interoperable PEG-defined syntax for gas scattering reaction definition proposes a unified, human-readable and machine-processable notation for reactions, molecular species, charge states, excitation states, and branched channels, with a reference parser and explicit support for probabilities and quantum-state labels (Ciubotaru et al., 14 Apr 2025).
Within this broader landscape, SynRXN occupies the benchmarking and data-governance layer. It does not replace these modeling frameworks, and it does not ship full planners (Phan et al., 5 Jan 2026). Rather, it provides the harmonized, leakage-aware, provenance-tracked substrate on which fair comparison of such methods becomes possible. This suggests that SynRXN is likely to remain important less as a single model than as an infrastructural standard for longitudinal CASP evaluation.
