---
title: 'PepThink-R1: Interpretable Peptide Optimization'
url: https://www.emergentmind.com/topics/pepthink-r1
type: topic
---

# PepThink-R1: Interpretable Peptide Optimization

PepThink-R1 is a generative framework for interpretable optimization of head-to-tail cyclic peptides, integrating large language models (LLMs) with chain-of-thought (CoT) supervised fine-tuning (SFT) and reinforcement learning (RL). Designed for the generation of peptides with enhanced pharmacological properties while maintaining explicit, stepwise traceability of sequence modifications, PepThink-R1 is the first LLM-based peptide design system to achieve interpretable, monomer-level reasoning jointly with multi-property RL-driven control [2508.14765].

## 1. System Architecture and Workflow

PepThink-R1 is built upon LlaSMol, an LLM derived from Mistral-7B v0.1, enhanced with LoRA adapters to enable domain-specific instructional tuning. The framework’s workflow consists of two major stages:

- **Chain-of-Thought Supervised Fine-Tuning (CoT-SFT):** Given a cyclic peptide input in SMILES notation and specified property-improvement goals, the model generates a detailed monomer-level reasoning trace (CoT) followed by a single, explicit SMILES edit reflecting one monomer substitution.
- **Reinforcement Learning via Group-Relative Policy Optimization (GRPO):** Starting from the CoT-SFT policy, RL is used to optimize for a composite reward function. This function balances (i) improvements in predicted pharmacological properties (LogD, rat mean residence time [MRT], simulated intestinal fluid [SIF] stability), (ii) Tanimoto structural similarity to the parent peptide, and (iii) diversity via duplication penalties. KL-regularization maintains distributional proximity to the SFT policy, counteracting mode collapse and reward hacking.

The practical result is a system that, in deployment, automatically selects a mutational site and substitution, provides a corresponding rationale, and outputs an optimized cyclic-peptide SMILES.

## 2. Monomer-Level Reasoning and Validity Constraints

PepThink-R1’s reasoning process operates at the monomer (single-residue) level. At generation time, input peptides are tokenized so each monomer is an explicit vocabulary element (vocabulary: 10,385 monomers, including 385 natural and 10,000 synthetic non-natural amino acids [NNAAs]). For a property-improvement objective, the model:

- Identifies candidate positions for substitution.
- Articulates a rationale referencing the specific change (e.g., “At position 2, replace the glycine monomer with the NNAA [NH2+][C@@H](Cc1cc(C(C)(C)C)cc(c1Cl)Cl)C(=O)[O-] to (i) raise lipophilicity (LogD), (ii) extend mean residence time (MRT_Rat), and (iii) enhance SIF stability”).
- Yields a mutated peptide with only the specified monomer edit.

Structural validity is enforced by filtering outputs through RDKit parsing and QSAR prediction coverage checks, and by syntactic constraints ensuring only a single-site edit per generation. Invalid or out-of-vocabulary substitutions are filtered out during both SFT and RL.

## 3. Mathematical Formulation and Reward Structure

The supervised SFT loss is the standard token-level cross-entropy:

$$
L_{\textrm{SFT}}(\theta) = -\sum_{(x, y) \in D} \sum_{t=1}^{T} \log \pi_\theta(y_t \mid x, y_1,\ldots, y_{t-1})
$$

The RL reward for output $y$ in state $x$ is defined as:

$$
R(y \mid x) = \textrm{dup\_fac} \cdot [w_{\textrm{prop}} \cdot \textrm{prop\_smooth} + w_{\textrm{sim}} \cdot \textrm{sim\_fac}]
$$

where:

- $w_{\textrm{prop}} = 0.8$, $w_{\textrm{sim}} = 0.2$ (chosen via grid search to maximize high-quality success rate [HQSR]),
- $\textrm{prop\_smooth} = \frac{1}{3} \sum_{i=1}^3 \sigma((x_i - t_i)/k_i)$ for property endpoints (LogD, MRT, SIF),
- $\textrm{sim\_fac} = \sigma(\alpha \cdot (s - s_0))$ (Tanimoto similarity $s$, with $s_0=0.6, \alpha=10$),
- $\textrm{dup\_fac} = [1/\max(1, n+1)]^\gamma$ (duplication penalty with $n$ prior occurrences, $\gamma=0.5$).

Reinforcement learning proceeds under the GRPO objective, which incorporates KL-regularization with coefficient $\beta=1\mathrm{e}{-3}$:

$$
J_{\textrm{GRPO}}(\theta) = \mathbb{E}_{x, \{y_i\}} \left[ \frac{1}{G} \sum_i \min(r_i A_i, \textrm{clip}(r_i, 1-\epsilon, 1+\epsilon)A_i) - \beta D_{\textrm{KL}}[\pi_\theta(\cdot|x) \| \pi_{\textrm{ref}}(\cdot|x)] \right], \quad \epsilon\approx 0.1
$$

Ablation studies confirm that removal of KL-regularization or property weighting degrades output validity and diversity, and that CoT is essential to preserving rational, interpretable optimization trajectories.

## 4. Training Regimes, Data, and Hyperparameters

Training data comprises approximately 380,000 cyclic peptide pairs generated by performing 100 random single-HELM monomer edits on 3,778 seed peptides. Each pair is annotated with QSAR-predicted LogD, rat-MRT, and SIF stability, and stratified by the number of improved endpoints (single, dual, triple).

Fine-tuning uses 26,000 samples (triple-improvement plus up to 4,000 dual/single examples per group), with 10% validation. RL is seeded from 600 sampled SFT-train peptides. The held-out test set contains 1,880 pairs with <3 property improvements.

Hyperparameters include:
- SFT: epoch=3, lr=$5\mathrm{e}{-5}$, batch=4, grad-accum=8, LoRA rank=8, $\alpha=16$, dropout=0.05.
- RL: batch=128 seeds × 8 rollouts, temp=1.0, lr=$1\mathrm{e}{-6}$, $\beta=1\mathrm{e}{-3}$, AdamW optimizer, total RL steps~10,000.

Qualitative validity and property improvements are tracked throughout, and RL policies are checkpointed against quantitative metrics detailed below.

## 5. Quantitative Evaluation and Benchmarks

PepThink-R1’s performance is summarized as follows (see [2508.14765] Table 1):

| Model           | Validity | Novelty | Uniqueness | HQSR  | UHQS/seed | HQSR-S |
|-----------------|---------|--------|-----------|-------|-----------|--------|
| Random mutation | 0.876   | 0.998  | 0.991     | 0.003 | 0.61/100  | 0.208  |
| GPT-5           | 0.821   | 0.996  | 0.959     | 0.107 | 0.85/10   | 0.386  |
| CoT-SFT         | 0.956   | 0.999  | 0.954     | 0.196 | 8.82/50   | 0.879  |
| SFT-RL          | 0.987   | 0.998  | 0.238     | 0.833 | 9.84/50   | 0.956  |
| PepThink-R1     | 0.900   | 1.000

Source: https://www.emergentmind.com/topics/pepthink-r1