Tx-LLM: Unified Therapeutic & Biomedical LLMs

• Tx-LLM is a class of domain-adapted large language models that integrate chemical, biological, and clinical data for unified therapeutic reasoning.
• The model fine-tunes state-of-the-art architectures like PaLM-2 using specialized prompts and diverse datasets across 66 tasks to achieve competitive multi-task performance.
• Tx-LLM demonstrates positive transfer across modalities, enhancing predictive accuracy in target discovery, safety screening, and reaction planning.

Tx-LLM, in the context of therapeutic and biomedical LLMs, denotes a class of domain-adapted LLMs engineered for broad, multi-modal, and multi-task application in drug discovery, clinical informatics, and healthcare prediction scenarios. The defining characteristic is the integration—through instruction and domain fine-tuning—of structured chemical, biological, and clinical representations (e.g., SMILES strings, protein sequences, cell line/disease metadata) with natural language processing, to enable comprehensive end-to-end reasoning and property prediction across the full therapeutic development pipeline. Tx-LLM approaches represent a paradigm shift from narrowly focused models to generalist frameworks capable of simultaneous access and prediction for heterogeneous entity types within a unified architecture (Chaves et al., 10 Jun 2024).

1. Model Architecture: Base Selection and Instruction Tuning

Tx-LLM models are built by domain-specific fine-tuning of state-of-the-art LLMs. The canonical example uses PaLM-2 as the foundational model, subsequently fine-tuned on a therapeutics-focused prompt collection, designated TxT, comprising instruction, context, question, and answer components (Chaves et al., 10 Jun 2024).

This process includes the following schematic:

• Input encoding: All modalities (small molecules, proteins, nucleic acids, etc.) are mapped to strings (e.g., SMILES for molecules, amino acid sequences for proteins).
• Prompt format:

$$\text{Prompt} = \{\text{Instruction},\ \text{Context},\ \text{Question},\ \text{Answer}\}$$

• Instruction tuning: Training leverages 709 datasets spanning 66 distinct tasks (classification, regression, and generative tasks such as chemical reaction synthesis).
• Weighting and mixture: Datasets are sampled for training in proportions corresponding to their respective sizes within the Therapeutics Data Commons (TDC).

The fine-tuning paradigm uses approximately 70% zero-shot and 30% few-shot prompt compositions, with explicit ablation experiments testing shot selection strategies (random vs. KNN-nearest shots).

2. Multitask Performance and Evaluation

Tx-LLM achieves state-of-the-art or competitive results across a broad task portfolio. It was benchmarked against prior models on the TDC suite, showing:

• Upper-bound task performance: Outperformed SOTA on 22/66 tasks (12 binary, 10 regression).
• Near-SOTA performance: For an additional 21/66 tasks, performance was within 10% of SOTA.
• Specialization for SMILES+Text tasks: Median relative difference in favor of Tx-LLM for tasks interleaving molecular and textual descriptors.
• Metrics: Classification performance is measured by AUROC, AUPRC, accuracy; regression by Pearson/Spearman correlations, MAE, MSE; and generation tasks by set accuracy (e.g., exact SMILES matching).

Relative difference formulation is used for quantitative comparison:

$$\text{Relative Difference} = \frac{\text{Performance}_{\text{Tx-LLM}} - \text{Performance}_{\text{SOTA}}}{\text{Performance}_{\text{SOTA}}}$$

For metrics where lower is better (MAE, MSE), the sign is adjusted.

Statistical significance of improvements is established via Wilcoxon signed-rank tests.

3. Generalist Modeling and Positive Transfer

A salient property is the positive transfer observed between heterogeneous tasks. Unlike conventional models trained exclusively on small molecules (SMILES), Tx-LLM is trained jointly with proteins, nucleic acids, and other types. Empirically:

• The model fine-tuned on all entity types consistently outperforms models restricted to single-type datasets.
• For example, training on both small molecule and protein datasets led to improved small molecule task performance.

This phenomenon suggests that contextual knowledge acquired during pre-training on natural language corpora can be effectively reused when entity features are richly represented by textual context.

4. Technical Implementation, Prompt Engineering, and Domain Adaptation

Tx-LLM adapts transformer architectures (PaLM-2) with custom attention mechanisms for long-context inputs. For regression tasks, values are binned:

$y \in [0, Y_{max}] \rightarrow \text{bin } b$

The LLM predicts bin $b$; during evaluation, predicted bins are mapped back to the continuous scale.

Ablation analyses evaluated:

• Model scale (S/M): Larger models outperform smaller ones with statistical significance.
• Context impact: Omission of contextual features degrades performance in 49/66 tasks.
• Few-shot strategy: Minor differences found across zero/one/five/ten shots; contextual richness is the dominant factor.

Instruction tuning and prompt engineering are central as future research directions, especially for enhancing the model’s explicit reasoning capacity.

5. Therapeutics Pipeline Applications

The key functional areas are:

• Target discovery: Predicting affinities and interactions for small molecules and proteins.
• Safety/efficacy screening: Estimating properties such as toxicity, permeability, clinical trial outcomes (including phase progression).
• Reaction/synthesis planning: Generative tasks for retrosynthetic route prediction.
• End-to-end reasoning: Integration across stages from early candidate generation to clinical outcome prediction within a unified model.

Tx-LLM’s applicability includes direct property prediction from raw inputs (e.g., SMILES, sequences) and the potential for prospective experimental screening.

6. Future Research and Limitations

Anticipated advancements include:

• Enhanced explanation capabilities: Developing models able to articulate mechanistic reasoning beyond prediction.
• Robustness to data contamination: Systematic monitoring as fine-tuning datasets expand and potentially overlap.
• Integration with future architectures: Potential adoption of Gemini-like models to better address complex chemical and protein language features.
• Deployment in laboratory pipelines: Using Tx-LLM as an upstream screening tool prior to wet-lab validation.

Current limitations include the focus on prediction accuracy with less attention to natural language explanation and incomplete characterization of generalizability in the lowest data regimes.

7. Comparative Developments and Evolution

Tx-LLM inspired subsequent models such as TxGemma (Wang et al., 8 Apr 2025), which further improved data efficiency, interactive reasoning, and explainability. The Agentic-Tx system, powered by Gemini 2.5, extends Tx-LLM’s principles into agentic workflow orchestration with modular tool integration.

Together, these developments represent a transition in therapeutic AI from specialized, narrowly focused models toward robust, multimodal, agentic frameworks that unify predictive power, interpretability, and end-to-end process management in drug discovery and healthcare.