KinForm: Enzyme Kinetics Prediction

• KinForm is a machine learning framework for enzyme kinetics parameter prediction that optimizes protein feature representations using residue embeddings from multiple pretrained models.
• It integrates binding-site-aware weighted pooling and PCA-based dimensionality reduction to combat overfitting and improve predictions on low-similarity proteins.
• The framework employs similarity-aware rebalancing and sequence-exclusive cross-validation to ensure robust evaluation and generalization beyond homologous protein sequences.

KinForm is a machine learning framework for predicting enzyme kinetic parameters, specifically turnover number $k_{cat}$ and Michaelis constant $K_M$, by optimising protein feature representations rather than relying on a more complex downstream predictor (Alwer et al., 19 Jul 2025). It combines residue-level embeddings from Evolutionary Scale Modeling Cambrian, Evolutionary Scale Modeling 2, and ProtT5-XL-UniRef50; applies weighted pooling based on per-residue binding-site probability; reduces dimensionality with principal-component analysis on concatenated protein features; rebalances training data with a similarity-based oversampling strategy; and emphasizes sequence-overlap-free evaluation (Alwer et al., 19 Jul 2025). The framework is positioned against a common convention in enzyme-kinetics prediction, exemplified by methods such as UniKP, in which a single protein LLM is represented by mean pooling of the last transformer layer. KinForm argues that this convention is biologically blunt, statistically fragile, and especially vulnerable to evaluation protocols that leak highly similar proteins across folds.

1. Problem formulation and motivation

KinForm addresses supervised regression from an enzyme–substrate pair to a log-transformed kinetic value (Alwer et al., 19 Jul 2025). Each sample consists of a protein sequence, a substrate representation derived from SMILES, and a target value $k_{cat}$ or $K_M$. The paper emphasizes that experimental kinetic data are scarce, noisy, and biased toward a small number of well-studied families, so models trained under standard protocols tend to overfit repeated sequence families and perform poorly on low-similarity proteins.

The framework is motivated by two explicit aims: improve predictive accuracy by making the protein representation more task-relevant, and improve generalization by discouraging memorization and benchmarking under a stricter evaluation regime (Alwer et al., 19 Jul 2025). The paper frames the task primarily as a representation-learning problem. In that formulation, the major gains are expected to come from how the protein is represented, pooled, compressed, and evaluated, rather than from the use of a more elaborate prediction head. Consistent with that stance, the final regressor is an Extra Trees model.

A central premise is that catalytic function is usually controlled by a small subset of residues, that the final layer of a protein LLM is not always the most informative layer for downstream kinetics prediction, and that random train/test splits can overstate performance by placing nearly identical proteins in both training and test folds (Alwer et al., 19 Jul 2025). KinForm is constructed as a direct response to those three issues.

2. Protein LLMs and intermediate-layer selection

KinForm uses residue-level embeddings from three pretrained protein LLMs: ESMC, ESM-2, and ProtT5-XL-UniRef50 (Alwer et al., 19 Jul 2025). For a given model, the hidden-state tensor is written as

$\mathbf{E}\in \mathbb{R}^{N\times L\times H},$

where $N$ is the number of transformer layers, $L$ is sequence length, and $H$ is hidden dimension. The specific layer counts and hidden sizes are ESM-2 with $N=34$, $H=1280$; ProtT5-XL with $K_M$0, $K_M$1; and ESMC with $K_M$2, $K_M$3.

For layer selection, the framework first mean-pools each layer over sequence length: $K_M$4 These layerwise vectors are evaluated across all layers. The paper reports that the best-performing representations are often not from the final layer; intermediate layers tend to work better (Alwer et al., 19 Jul 2025). To choose one layer per model and task, KinForm uses the heuristic score

$K_M$5

where $K_M$6 is median $K_M$7, $K_M$8 is mean $K_M$9, and $k_{cat}$0 is the standard deviation of $k_{cat}$1 across folds.

Model	$k_{cat}$2 layer	$k_{cat}$3 layer
ESM-2	30	26
ESMC	32	34
ProtT5-XL-UniRef50	19	last layer retained for fairness with UniKP

The task-specific layer choices support the claim that there is no universally best layer across tasks (Alwer et al., 19 Jul 2025). A plausible implication is that $k_{cat}$4 and $k_{cat}$5 depend on different abstractions within pretrained sequence models, so fixed use of the last layer can discard usable biochemical structure.

3. Binding-site-aware pooling and representation assembly

A defining design choice in KinForm is to avoid uniform averaging across residues and instead use binding-site-aware pooling guided by per-residue probabilities from Pseq2Sites (Alwer et al., 19 Jul 2025). For each residue $k_{cat}$6, Pseq2Sites provides a binding probability $k_{cat}$7. These are normalized as

$k_{cat}$8

and used to compute the binding-weighted representation

$k_{cat}$9

The conventional global mean-pooled representation is

$K_M$0

and KinForm also uses the concatenated representation

$K_M$1

The biological rationale stated in the paper is that catalytic activity is often determined by a small subset of residues near the active site or binding site, so uniform mean pooling can wash out the relevant signal (Alwer et al., 19 Jul 2025). In KinForm, each of the three LLMs contributes both a global-mean pooled stream and a binding-weighted stream. Because ESMC, ESM-2, and ProtT5 have hidden sizes $K_M$2, $K_M$3, and $K_M$4, the concatenated global stream has dimension $K_M$5, the binding-weighted stream also has dimension $K_M$6, and the full two-stream protein representation has $K_M$7 features.

This multi-model assembly is intended to capture complementary information across encoders (Alwer et al., 19 Jul 2025). The paper also warns that simply increasing dimensionality is not always beneficial for generalization, which motivates the later PCA stage.

4. Dimensionality reduction, molecular encoding, and predictor variants

KinForm-L applies dimensionality reduction to the protein representation with a two-stage scaling-plus-PCA pipeline (Alwer et al., 19 Jul 2025). Each of the six feature blocks—three global and three binding-weighted—is first independently robust-scaled using training-set medians and interquartile ranges. The three global blocks are concatenated into $K_M$8, and the three binding blocks into $K_M$9. Each matrix is then standardized with training-set mean and standard deviation, PCA is fit separately to the standardized global and binding matrices, and the final compressed protein representation is

$\mathbf{E}\in \mathbb{R}^{N\times L\times H},$0

In the main KinForm-L configuration, each side is reduced to $\mathbf{E}\in \mathbb{R}^{N\times L\times H},$1, yielding a $\mathbf{E}\in \mathbb{R}^{N\times L\times H},$2-dimensional protein vector.

For the substrate, the main experiments use the SMILES Transformer representation, selected because it performs most consistently across tasks and validation settings (Alwer et al., 19 Jul 2025). The molecular embedding is formed by taking the last and penultimate encoder layers, pooling each with both mean and max pooling over tokens, and concatenating the four resulting vectors into $\mathbf{E}\in \mathbb{R}^{N\times L\times H},$3.

The final predictor remains deliberately simple. Protein and substrate representations are concatenated and passed to an Extra Trees regressor, which is used because it performed best in the UniKP benchmark of multiple machine-learning models (Alwer et al., 19 Jul 2025). Two variants are distinguished. KinForm-H uses the full $\mathbf{E}\in \mathbb{R}^{N\times L\times H},$4-dimensional protein representation with no PCA and an input vector in $\mathbf{E}\in \mathbb{R}^{N\times L\times H},$5. KinForm-L uses the PCA-reduced protein representation and an input vector in $\mathbf{E}\in \mathbb{R}^{N\times L\times H},$6. The paper characterizes KinForm-H as emphasizing high similarity or in-distribution performance, and KinForm-L as emphasizing low similarity or generalization.

5. Similarity-aware rebalancing and evaluation protocol

KinForm is evaluated on three benchmark datasets (Alwer et al., 19 Jul 2025). For $\mathbf{E}\in \mathbb{R}^{N\times L\times H},$7, the paper uses DLKcat with 16,775 samples after filtering invalid SMILES and negative values, and Shen with 35,001 samples after filtering. For $\mathbf{E}\in \mathbb{R}^{N\times L\times H},$8, it uses a dataset with 11,722 samples. Both $\mathbf{E}\in \mathbb{R}^{N\times L\times H},$9 datasets are compiled from BRENDA and SABIO-RK, and all targets are $N$0-transformed.

A major methodological claim of the paper concerns evaluation. Standard 5-fold cross-validation randomly splits datapoints into five folds, but this can place highly similar or identical protein sequences in both training and test sets (Alwer et al., 19 Jul 2025). KinForm therefore emphasizes Sequence-Exclusive Cross Validation (SE-CV), also in 5 folds, with the constraint that no protein sequence in the training set may appear in the test set for that fold. Beyond overall $N$1 and RMSE, the paper bins test proteins by sequence similarity to the training set using MMseqs2 at identity thresholds $N$2, $N$3, $N$4, and $N$5, assigning each test protein to the highest threshold for which it shares a cluster with a training protein; otherwise it is placed in the $N$6 bin.

The similarity-based oversampling scheme is applied only to the training data (Alwer et al., 19 Jul 2025). Unique protein sequences are clustered with MMseqs2 at identity thresholds $N$7, $N$8, and $N$9. Each sequence is assigned to the highest threshold for which its cluster has more than one member; singleton sequences at all thresholds are assigned to the $L$0 bin. Sequences are then grouped into the four similarity bands $L$1, $L$2, $L$3, and $L$4, and lower-similarity bands are oversampled by random duplication with replacement until they match the size of the $L$5 band.

The reported metrics are

$L$6

and

$L$7

The paper’s broader argument is that similarity-binned evaluation reveals whether a model genuinely generalizes beyond homologous families, rather than merely memorizing local sequence neighborhoods.

6. Results, ablations, and methodological significance

For $L$8, KinForm-L is reported as the best model under SE-CV, whereas KinForm-H is best under standard random 5-fold cross-validation (Alwer et al., 19 Jul 2025). The strongest gains occur in low-similarity bins. On DLKcat, KinForm-L improves UniKP by about 100% in the $L$9 similarity bin, from roughly 0.077 to 0.15 $H$0. The paper also states that this can be described as roughly 44% improvement in another reporting context, noting that the exact percentage depends on the baseline value and bin summary being referenced. Across both DLKcat and Shen, the qualitative pattern is consistent: high-similarity sequences are matched or slightly trailed in some settings, but low-similarity sequences are clearly improved. In a comparable split setting on Shen, KinForm-H achieves an $H$1 around 0.72, described as on par with EITLEM’s final transfer-learning result but with a much simpler training pipeline.

For $H$2, the full-resolution configuration equivalent to KinForm-H is used for the main benchmark, and KinForm outperforms UniKP under both CV and SE-CV by around 5% overall (Alwer et al., 19 Jul 2025). The paper explicitly notes that KinForm-L does not provide a consistent advantage for $H$3, so PCA-compressed features were not retained for the main $H$4 experiments. It also observes that the CV–SE-CV gap is smaller for $H$5 than for $H$6, which suggests that $H$7 may depend more heavily on substrate representation and less on protein-sequence generalization, or that the dataset is less sensitive to sequence-overlap leakage.

The ablation results isolate the roles of the major design components (Alwer et al., 19 Jul 2025). Binding-weighted pooling improves over plain global mean, and the strongest settings usually concatenate global and binding vectors. Intermediate layers often outperform the last layer, with the optimal layer depending on both model and task. PCA has a clear regularization effect: no PCA gives the best standard CV but the worst SE-CV; moderate PCA reduction gives the best SE-CV; too many components restore standard CV but degrade SE-CV. The best SE-CV performance is often obtained with about 200–400 components, capturing roughly 90–95% of variance. Oversampling barely changes overall SE-CV accuracy but gives a small, consistent improvement in the $H$8 similarity bin while leaving high-similarity performance mostly unchanged.

The paper’s principal methodological conclusion is that sequence-overlap-free evaluation should be the standard for benchmarking enzyme kinetic prediction models (Alwer et al., 19 Jul 2025). Random splits are presented as misleading because identical or highly similar proteins can inflate apparent predictive performance. KinForm’s broader significance lies in demonstrating that relatively simple regressors can outperform prior methods when paired with carefully engineered protein representations, binding-site-aware pooling, dimensionality control, and evaluation protocols that measure generalization beyond homology leakage.