HYPOGENEAGENT: LLM-Driven Cluster Annotation

• HYPOGENEAGENT is an LLM-driven framework for automating gene-set cluster resolution and functional annotation in single-cell and Perturb-seq studies.
• It transforms cluster interpretation into an optimization task by embedding GO-term hypotheses and quantifying intra-cluster coherence and inter-cluster distinctiveness.
• Preliminary evaluations on K562 CRISPRi data show improved alignment with known pathways, outperforming traditional metrics like silhouette and modularity scores.

Searching arXiv for the named paper and closely related work to ground the article. arXiv search query: "HypoGeneAgent Perturb-seq cluster resolution GO term"

HYPOGENEAGENT is a LLM-driven framework for gene-set cluster resolution selection and functional annotation in single-cell and Perturb-seq analyses. It addresses two linked sources of subjectivity: resolution selection for community-detection algorithms such as Leiden or Louvain, and Gene Ontology annotation of the resulting clusters. Its central move is to recast cluster interpretation as a quantitatively optimizable task: an LLM generates GO-based hypotheses for each cluster, those hypotheses are embedded into a semantic vector space, and a combined score rewards clusters whose annotations are internally coherent yet externally distinct. In the reported preliminary evaluation on a public K562 CRISPRi Perturb-seq dataset, the method selected clustering granularities that aligned with known pathway structure more closely than classical metrics such as silhouette score and modularity score (Yuan et al., 10 Sep 2025).

1. Problem formulation and stated objectives

Single-cell and Perturb-seq studies routinely generate clusters of cells or perturbations whose biological interpretation depends on the resolution chosen by community-detection algorithms. In the formulation used by HYPOGENEAGENT, two difficulties are coupled rather than independent. First, cluster resolution selection is commonly driven by subjective heuristics, including marker-gene heatmaps and manual tuning. Second, functional annotation by assigning GO terms is also manual, time-consuming, and poorly reproducible. HYPOGENEAGENT is designed to address both stages in one loop by replacing manual curation with an optimization criterion derived from language-agent outputs (Yuan et al., 10 Sep 2025).

The framework has two explicit goals. One is to automate resolution tuning by defining a quantitative score that rewards clusters whose LLM-derived annotations are internally coherent yet distinct from those of other clusters. The other is to generate, for each cluster, a ranked list of GO-term hypotheses with calibrated confidences. This construction places biological interpretation, rather than geometric compactness alone, at the center of clustering selection.

A concise summary of the pipeline components is given below.

Component	Operation	Output
LLM-based gene-set analyst	Analyzes each cluster signature	Ranked GO-based hypotheses with confidences
Sentence embedding model	Embeds hypothesis descriptions	Dense vectors $e \in \mathbb{R}^d$
Agent-derived scoring stage	Computes intra-cluster agreement and inter-cluster separation	Resolution score $R(r)$

This suggests that HYPOGENEAGENT is not merely an annotation layer applied after clustering; it is a criterion for choosing clustering granularity itself.

2. LLM-based hypothesis generation and confidence calibration

The core inference unit is an LLM-based gene-set analyst that processes a cluster signature, defined as top over-expressed genes or perturbation modules, and returns up to $H=5$ candidate GO-term explanations

$\{(h_{k1},c_{k1}),\dots,(h_{kH},c_{kH})\}$

for cluster $k$, sorted by decreasing confidence $c_{ki}\in[0,1]$ (Yuan et al., 10 Sep 2025).

The prompting strategy is highly structured. The “hypothesis” prompt forces the model to retrieve concise functional summaries for each gene via integrated GO/KEGG/PubMed retrieval, propose up to five distinct GO-based process names, and self-calibrate its confidences through internal verification steps. Several backbone models were tested, including GPT-4o, “thinking” models such as GPT-o3 with chain-of-thought and self-verification, and Gemini variants. In practice, GPT-o3 with the “hypothesis” prompt performed best.

Confidence calibration is treated as part of the framework rather than as an incidental model output. Temperature scaling was explored in $[0,1]$ in steps of $0.1$. GPT-4o was reported to be stable across temperatures, whereas GPT-o3 used self-verification to align confidence scores $c_{ki}$ with true semantic match likelihood. In a benchmark on curated GOBP sets, the top-1 hypothesis confidence correlated strongly with ground-truth cosine similarity, which the authors use as validation of calibration.

The stated design implies a specific notion of annotation quality. A good hypothesis is not only biologically plausible in a descriptive sense; it should also receive a confidence value that tracks semantic agreement with a reference ontology description. A plausible implication is that HYPOGENEAGENT treats language-model uncertainty as a measurable signal for downstream clustering decisions rather than as auxiliary metadata.

3. Semantic embedding, similarity structure, and the resolution objective

Each hypothesis description $h$ is mapped to a dense vector $R(r)$0 by a sentence embedding model. The default configuration uses OpenAI text-embedding-3-large with $R(r)$1, described as trained on large web/text corpora and fine-tuned for semantic similarity. Alternatives, including SapBERT and Nomic AI, were evaluated but not adopted (Yuan et al., 10 Sep 2025).

Pairwise semantic similarity is computed with cosine similarity:

$R(r)$2

For a clustering partition at resolution $R(r)$3,

$R(r)$4

the framework distinguishes two quantities.

The first is intra-cluster agreement, which measures semantic coherence among the $R(r)$5 hypotheses of cluster $R(r)$6:

$R(r)$7

The second is inter-cluster separation, which measures distinctiveness between clusters. Using the top hypothesis embedding $R(r)$8 for each cluster, the simplified average over cluster pairs is

$R(r)$9

A fuller form weighted by cluster sizes is also given in the specification.

The combined agent-derived resolution score is

$H=5$0

with

$H=5$1

and weighting hyperparameters $H=5$2. In practice, $H=5$3 and $H=5$4 gave stable results. Maximizing $H=5$5 over $H=5$6 yields the optimal clustering granularity.

This construction is notable because it does not evaluate clusters through cell-level geometry alone. Instead, the score is based on whether language-model hypotheses are mutually consistent within a cluster and mutually exclusive across clusters. The agent referee panel score described in the abstract is therefore an embedding-space adjudication over biological descriptions.

4. Pipeline specification and experimental configuration

The reported high-level pipeline initializes resolutions $H=5$7, $H=5$8, $H=5$9, $\{(h_{k1},c_{k1}),\dots,(h_{kH},c_{kH})\}$0, temperature $\{(h_{k1},c_{k1}),\dots,(h_{kH},c_{kH})\}$1, and embedding dimension $\{(h_{k1},c_{k1}),\dots,(h_{kH},c_{kH})\}$2. Data preprocessing consists of normalization, highly variable gene selection, PCA to 40 dimensions, and construction of a 10-NN graph. For each resolution $\{(h_{k1},c_{k1}),\dots,(h_{kH},c_{kH})\}$3, Leiden clustering is run; for each cluster $\{(h_{k1},c_{k1}),\dots,(h_{kH},c_{kH})\}$4, the signature is taken as the top $\{(h_{k1},c_{k1}),\dots,(h_{kH},c_{kH})\}$5 marker genes by logFC; the LLM returns $\{(h_{k1},c_{k1}),\dots,(h_{kH},c_{kH})\}$6 hypotheses; each hypothesis is embedded; intra-cluster agreement is computed; inter-cluster similarity is computed from top hypotheses across cluster pairs; and the final score $\{(h_{k1},c_{k1}),\dots,(h_{kH},c_{kH})\}$7 is evaluated. The algorithm returns $\{(h_{k1},c_{k1}),\dots,(h_{kH},c_{kH})\}$8, the score profile, and the labels at the optimal resolution (Yuan et al., 10 Sep 2025).

The K562 CRISPRi Perturb-seq preprocessing protocol is stated explicitly. Raw counts were subjected to QC using cells with at least 200 genes, at most 10% mitochondrial reads, and genes present in at least 3 cells, yielding $\{(h_{k1},c_{k1}),\dots,(h_{kH},c_{kH})\}$9 cells $k$0 $k$1 HVGs. Counts were normalized to $k$2 per cell, transformed with $k$3, projected by PCA to 40 components, and used to build a 10-NN graph in PC space. Leiden clustering was then evaluated at resolutions $k$4, yielding 3–20 clusters.

The experimental comparison used three baseline families. The first was silhouette score $k$5 on PCA embedding, and separately on UMAP embedding, with

$k$6

where $k$7 is mean intra-cluster distance and $k$8 is nearest-other-cluster distance. The second was modularity $k$9 on the 10-NN weighted graph,

$c_{ki}\in[0,1]$0

with $c_{ki}\in[0,1]$1, $c_{ki}\in[0,1]$2 the total edge weight, and $c_{ki}\in[0,1]$3 node degrees. The third was a functional enrichment baseline using Fisher-exact test on GO Biological Process 2023, BH-adjusted $c_{ki}\in[0,1]$4, retaining the top five terms per cluster.

5. Reported empirical behavior on K562 CRISPRi Perturb-seq

In the reported preliminary test, the Resolution Score $c_{ki}\in[0,1]$5 exhibited a clear single peak at $c_{ki}\in[0,1]$6 for gene-expression clustering and $c_{ki}\in[0,1]$7 for perturbation-level clustering, selecting nine and ten clusters respectively (Yuan et al., 10 Sep 2025).

The comparison to classical metrics is specific. Silhouette score peaked weakly at approximately $c_{ki}\in[0,1]$8–$c_{ki}\in[0,1]$9, while modularity peaked at approximately $[0,1]$0, with marginal gain beyond $[0,1]$1. The framework’s main empirical claim is not merely that it selects a different resolution, but that the selected resolution aligns better with known perturbation biology. At the $[0,1]$2-optimum, clusters recovered known pathway modules such as DNA damage response and cell-cycle control with high recall $[0,1]$3 and precision $[0,1]$4 compared to manual annotations.

The enrichment comparison is similarly quantitative. Enrichment $[0,1]$5-values of top GO terms in $[0,1]$6-selected clusters were lower, with median adjusted $[0,1]$7, than those in silhouette- or modularity-selected clusters, where the median adjusted $[0,1]$8. In addition, LLM self-confidence scores $[0,1]$9 for the top hypothesis correlated at approximately $0.1$0 with semantic similarity to ground-truth GO definitions.

These results support the claim that the framework is sensitive to biological coherence rather than only to geometric or graph-theoretic regularity. A cautious interpretation is still warranted: the paper describes the K562 application as a preliminary test, so the current evidence is strongest for proof-of-concept rather than for universal performance across perturbational contexts.

6. Interpretive significance, limitations, and relation to adjacent agentic systems

The stated advantages of HYPOGENEAGENT are interpretability and unification. The agent’s confidences and embeddings are presented as transparent measures of annotation consistency, and the framework is described as subsuming manual curation because both resolution tuning and cluster annotation emerge from one unified score (Yuan et al., 10 Sep 2025). This suggests an operational shift in single-cell analysis: annotation is no longer treated as an informal downstream narrative attached after clustering, but as a measurable object that feeds back into model selection.

The limitations are equally explicit. The method depends on LLM capabilities and on the currency of model background knowledge, so model updates may shift behavior. It also incurs computational cost and API latency when scaled to millions of cells or thousands of clusters. Prompt sensitivity remains a concern because small instruction changes can affect hypothesis diversity. The future extensions proposed in the source include adaptation to multi-omics summaries such as ATAC peaks and proteomic modules, closed-loop active learning that iteratively refines clustering based on agent feedback and cell-level re-annotation, and open-source embedding-only variants using SapBERT or BioSentVec.

A potential source of confusion is the name. HYPOGENEAGENT should be distinguished from HypoAgent, an agentic framework for interactive abductive hypothesis generation over knowledge graphs. HypoAgent operates with an Intent Recognition Agent, a Hypothesis Generation Agent, and a Root Cause Analysis Agent, and its object of reasoning is a first-order logic hypothesis over a KG rather than GO-based explanations for gene-set clusters (Gao et al., 29 May 2026). The lexical overlap between the names does not indicate identity of task, representation, or evaluation protocol.

Taken together, the framework defines a biology-aware alternative to resolution selection by coupling LLM-generated GO hypotheses, semantic embeddings, and similarity-based optimization. Its central claim is that coherent and mutually exclusive biological descriptions can serve as an objective criterion for choosing clustering granularity in single-cell and Perturb-seq studies.