Nephrobase Cell+: Kidney Single-Cell Model

• Nephrobase Cell+ is a kidney-centric single-cell foundation model that integrates multimodal omics to deliver robust, transferable, and precise representations of kidney biology.
• It employs a transformer-based architecture with gene-token cross-attention and mixture-of-experts modules, achieving superior batch correction and cross-species annotation.
• The model accelerates kidney genomics research by enabling detailed cell-type annotation, in silico perturbations, and predictive regulatory network analysis in a unified framework.

Nephrobase Cell+ is a large-scale, kidney-focused single-cell foundation model designed to deliver robust, transferable, and high-fidelity representations of kidney biology from multimodal single-cell omics. Developed to address the limitations of generalized foundation models in capturing organ-specific cellular architecture, batch effect correction, cross-modality integration, and cross-species generalization, Nephrobase Cell+ is pretrained on approximately 100 billion gene tokens from 39.5 million single-cell and single-nucleus profiles across 4,319 samples. The model sets new benchmarks for embedding quality, batch harmony, and cross-species cell-type annotation in the kidney, outperforming previous state-of-the-art single-cell models and traditional dimension-reduction techniques. Nephrobase Cell+ thus serves as a unified computational framework for kidney genomics, single-cell perturbation, and disease studies (Li et al., 30 Sep 2025).

1. Model Architecture and Technical Design

Nephrobase Cell+ employs a transformer-based encoder–decoder architecture specifically optimized for cell-by-gene matrices. Each cell is represented as a sequence of gene tokens, constructed by embedding the gene's unique identifier and normalized count, with optional additional metadata. The architecture integrates three principal innovations:

• Gene-token cross-attention: Initial tokenization combines gene identity and expression, allowing the transformer layers to model context-dependent gene-gene relationships. The cross-attention modules in the decoder establish direct communication between the cell-level embedding and all tokenized gene identities, permitting flexible weighting of gene contributions to the cell phenotype.
• Mixture-of-Experts (MoE) module: Within each transformer block, an MoE layer dispatches each token to a sparingly selected subset of specialized expert networks, guided by a softmax-based routing mechanism and a top-k selection strategy. The output is aggregated as a weighted sum across the selected experts with an auxiliary load-balancing loss to ensure even utilization and prevent bottlenecking. A shared expert path complements token-specific transformations with global features.
• Loss Functions and Optimization: The reconstruction head is optimized with a zero-inflated negative binomial (ZINB) loss to accommodate count overdispersion and excess zeros endemic to single-cell data. Cell-type annotation utilizes a focal loss to handle class imbalance. Auxiliary objectives include an elastic cell similarity (ECS) loss, which regularizes representation space, and a supervised contrastive loss (when labels are available) for enhanced cluster separation.

The model is trained on multiple GPUs using fully sharded data parallelism (FSDP), with learning rate warmup, gradient clipping, and mixed precision. The variable capacity (e.g., 1B and 500M parameter variants) permits scaling according to resource availability.

2. Training Data and Preprocessing

Nephrobase Cell+ is pretrained on a uniquely large, kidney-centric multimodal dataset:

Composition and Scope

Species	# Samples	Cell Types	Modalities
Human, Mouse, Rat, Pig	4,319	Kidney, immune, peripheral	scRNA-seq, snRNA-seq, snATAC-seq, spatial transcriptomics (COSMx, Xenium)

Approximately 30 million out of 39.5 million total cells are kidney-derived, ensuring dominant organ specificity. Gene sets are mapped to a fixed space of 32,768 orthologous features to facilitate cross-species alignment.

Data Preparation

Single-cell/single-nucleus expression counts are log-normalized, with batch effects arising from donor, assay, or technology incorporated explicitly into the training objectives. Domain-invariant representation is enforced using adversarial discriminators with gradient reversal layers, ensuring that cell type structure, not technical noise, dominates the learned latent space.

3. Benchmarking: Performance and Biological Fidelity

Quantitative Metrics

• Clustering: On held-out human and mouse kidney profiles, the 1B variant achieves KMeans-adjusted Rand index (ARI) of 0.82 and silhouette scores of ~0.68, yielding tightly clustered, hyper-coherent cell type embeddings (vs. ARI 0.22–0.55 for autoencoders, scGPT, and Geneformer).
• Batch Correction: Label and integration local inverse Simpson’s index (cLISI ≈ 1.00, iLISI ≈ 0.17–0.18) confirm near-complete batch mixing with retention of cell type integrity, an advance over prior frameworks that either over-integrate or under-correct.
• Cross-Species Transfer: Homologous cell types are strongly aligned across human and mouse data, attaining >90% zero-shot cell type annotation accuracy in both directions, as validated by Sankey diagrams, confusion matrices, and normalized mutual information (NMI) scores outperforming competing models.
• Zero-Shot Classification and Perturbation: The model not only clusters known cell types but annotates new or rare states, and can predict the outcome of in silico gene perturbations (e.g., CCL2, VCAM1, GDF15, SOX4), supporting dynamic regulatory network analysis.

Comparative Table

Method	ARI (clustering)	cLISI (batch mixing)	Human–Mouse Zero-Shot Accuracy
Nephrobase Cell+ (1B)	0.82	1.00	>90%
scGPT	0.54	0.70	<80%
Geneformer	0.37	0.61	<75%
UCE	0.22	0.43	<60%

These results confirm the performance gains across all tested axes, with even reduced-parameter variants (500M) surpassing alternative models.

4. Applications and Impact in Kidney Genomics Research

Nephrobase Cell+ facilitates several research and translational applications:

• Reference Mapping and Annotation: Enables unified and fine-grained annotation of kidney cellular hierarchies and their dynamic states in both health and disease, including rare or previously unstructured populations.
• Cross-Species Translation: Harmonizes data from diverse mammals, directly supporting comparative genomics, conservation of gene programs, and preclinical model evaluation.
• Multimodal Data Integration: Projects dissociated scRNA-seq, snRNA-seq, and spatial transcriptomics into a common latent space, allowing for spatial niche identification, neighborhood deconvolution, and microenvironment studies.
• Batch Harmonization: Removes technical and donor-driven artifacts without erasing genuine biological signal, enhancing meta-analysis and collaborative data sharing.
• In Silico Experimental Design: Supports predictive modeling of regulatory consequences following genetic perturbation, informing the design of targeted experiments and candidate gene prioritization.

A plausible implication is that Nephrobase Cell+ will accelerate generation of robust biomarkers and mechanistic hypotheses in nephrology, directly supporting both basic and translational research.

5. Limitations

Despite the achieved performance, several limitations are acknowledged:

• Under-Representation of Rare Cell Types or Extreme States: Even with a large dataset, certain populations (e.g., rare endocrine cells, severe injury states) may be insufficiently sampled, limiting annotation granularity.
• Feature Space Constraints: The model operates on a fixed universe of 32,768 genes; novel transcripts or species-exclusive features cannot currently be modeled unless included in the embedded gene set.
• Computational Requirements: Large models (e.g., 1B parameters) necessitate access to high-performance compute (multi-GPU clusters), which may restrict immediate adoption in smaller laboratories.
• Modality and Taxonomic Breadth: While integrating RNA and ATAC data from four mammalian species, extension to proteomics, methylomics, imaging, or non-mammalian vertebrates remains to be implemented.

This suggests that while Nephrobase Cell+ constitutes a significant resource, broader utility and accessibility will depend on ongoing expansion of data types, species, and hardware-efficient training strategies.

6. Future Directions

Key future directions identified in the data include:

• Extension to More Species and Multi-Omic Modalities: Incorporating additional taxonomic diversity and profiling data (spatial multi-omics, proteogenomics) to broaden biological scope.
• Downstream Fine-Tuning and Disease Subtyping: Stratified or patient-specific fine-tuning on organoid, xenograft, or human disease sub-cohorts to increase clinical relevance for diagnostics and therapeutic development.
• Model Compression and Training Efficiency: Ongoing efforts to implement compression, knowledge distillation, and more efficient parallelization to lower resource barriers.
• Interpretability and Biological Insight Mining: Systematic analysis of the model’s attention weights and latent space manifold to reveal candidate co-regulatory circuits and guide empirical validation.

A plausible implication is that integration of these directions will establish Nephrobase Cell+ as a backbone for large-scale, reproducible nephrogenomics studies, personalized medicine, and cross-disciplinary systems biology.

---

Nephrobase Cell+ thus establishes a kidney-centric computational infrastructure for the next generation of single-cell, spatial, and multimodal studies, offering reliability, transferability, and high biological fidelity unmatched by prior models (Li et al., 30 Sep 2025).