SpecCLIP: Stellar Spectroscopy Foundation Model

• SpecCLIP is a cutting-edge foundation model for stellar spectroscopy that employs contrastive alignment and decoder-augmented reconstruction to harmonize data from diverse surveys.
• It leverages modality-specific encoders, including transformer and MLP architectures, to reconcile LAMOST and Gaia XP spectra for robust cross-survey calibration and parameter estimation.
• The framework supports efficient transfer learning with lightweight few-shot adaptations to new surveys like DESI, enhancing spectral translation, anomaly detection, and stellar parameter accuracy.

SpecCLIP is a foundation model framework for stellar spectroscopy that adapts CLIP-style contrastive alignment to heterogeneous spectroscopic surveys. In its original formulation, it pre-trains modality-specific encoders for LAMOST low-resolution spectra and Gaia XP spectra, aligns the two modalities in a shared latent space using paired observations of the same stars, and augments the aligned representation with spectrum-aware decoders for reconstruction and cross-spectrum prediction. The resulting embedding space is intended to support intrinsic cross-survey calibration, parameter estimation, similarity search, anomaly detection, and translation between spectral types; subsequent work treats SpecCLIP as a pre-trained spectral foundation model and adapts it to DESI spectra with Low-Rank Adaptation (LoRA) in a few-shot regime (Zhao et al., 2 Jul 2025, Zhao et al., 28 Jul 2025).

1. Definition and conceptual framing

SpecCLIP, introduced as “Aligning and Translating Spectroscopic Measurements for Stars,” applies large-scale, self-supervised representation learning to stellar spectra by treating different instruments as different spectroscopic “languages” describing the same underlying star (Zhao et al., 2 Jul 2025). The central objective is to learn instrument-agnostic, information-rich embeddings from massive unlabeled corpora and then align these embeddings across surveys with a CLIP-like contrastive loss.

The framework addresses two linked problems. The first is survey heterogeneity: LAMOST and Gaia XP differ in wavelength coverage, resolution, calibration, and systematics. The second is the dependence of classical pipelines on task-specific supervision and survey-specific calibration. SpecCLIP therefore seeks a shared representation in which spectra from different instruments but the same star map to nearby points, while preserving enough modality-specific information to reconstruct and translate spectra.

A recurrent analogy in the literature is that stellar spectra behave like structured language. A spectrum is a one-dimensional sequence whose local structures include lines, blends, and continuum segments, while long-range correlations encode global stellar physics such as effective temperature, gravity, composition, extinction, and evolutionary state. This motivates transformer-style tokenization and masked modeling for some modalities, and more generally a foundation-model treatment of spectroscopy rather than a parameter-specific pipeline.

A common misconception is to identify SpecCLIP with the later DESI adaptation study. In that work, SpecCLIP is explicitly the pre-trained spectral foundation model; the contribution of the later paper is not the original architecture but its LoRA-based transfer to DESI (Zhao et al., 28 Jul 2025).

2. Surveys, preprocessing, and modality-specific encoders

SpecCLIP focuses on two low-resolution spectral domains: LAMOST low-resolution spectra and Gaia XP spectra (Zhao et al., 2 Jul 2025). For LAMOST DR11, the broader archive contains more than 10 million spectra, while the pre-training set comprises 966,082 high-quality spectra selected with SNR in SDSS $g$ band $> 50$ and $g < 15.8$. The wavelength interval used for modeling is 400–560 nm, sampled at 1462 flux points. Spectra are continuum-normalized, and for some losses the inputs are standardized per spectrum by subtracting a mean $\mu$ and dividing by a standard deviation $\sigma$.

For Gaia XP, the pre-training set contains 1,000,000 spectra with 343 flux points spanning 336–1021 nm. These spectra are normalized by the flux at 550 nm, yielding a representation centered on color information rather than absolute flux scale. Roughly 80% of the XP sample has matching LAMOST LRS spectra.

The LAMOST branch uses a masked transformer. A normalized 1462-point spectrum is segmented into overlapping windows of length 20 with step 10, producing 146 tokens per spectrum. The encoder applies a linear projection to model dimension 768, adds learned positional embeddings, and uses 6 Transformer encoder layers with 6-head self-attention. Its parameter count is about 42.7M. During pre-training, the sequence is divided into 6 segments, and in each segment a contiguous chunk of 20 tokens is randomly masked and reconstructed.

For Gaia XP, two encoder families are evaluated. One is a transformer-based masked model analogous to the LAMOST branch. The other, used for most reported XP results, is an MLP-based autoencoder designated OAE. The OAE consumes a 345-dimensional vector consisting of 343 flux values plus two scalar statistics, projects it to a hidden dimension of about 1290, processes it through 3 MLP blocks, and compresses it to a 768-dimensional bottleneck; the decoder mirrors the encoder. This model also has about 42.7M parameters.

The choice of encoder is modality-dependent. The experiments report that the simpler MLP OAE performs better than the transformer pre-trainer for XP at similar parameter count, whereas transformers are more effective for LRS, particularly for $T_{\rm eff}$ and $\log g$. This suggests that long-sequence structure is more important for the 1462-point LRS representation than for the shorter 343-point XP representation.

3. Shared latent space, projection heads, and decoder-augmented training

After modality-specific pre-training, SpecCLIP aligns LAMOST and XP in a joint embedding space by training projection heads on 820,568 paired spectra from the same stars (Zhao et al., 2 Jul 2025). For transformer encoders, the projection head uses a learnable query vector and multi-head cross-attention to pool a sequence of token embeddings into a single 768-dimensional representation, followed by an MLP. For the XP OAE branch, the projection is MLP-based but parameter-matched at roughly 7.1M.

All embeddings used by the contrastive loss are $L_2$-normalized. For a mini-batch of $N$ matched pairs, SpecCLIP defines similarities

$$\ell_{ij} = \tau \cdot \langle z_i^{\rm xp}, z_j^{\rm lrs} \rangle,$$

with learnable temperature scale $> 50$0, and optimizes a symmetric CLIP-style cross-entropy over XP$> 50$1LRS and LRS$> 50$2XP pairings.

A distinguishing property of SpecCLIP is that alignment is not the only objective. The model also includes decoders for in-modality reconstruction and cross-modality prediction. There are four decoders in total: LRS$> 50$3LRS, XP$> 50$4XP, LRS$> 50$5XP, and XP$> 50$6LRS. The total loss is

$> 50$7

with $> 50$8. Reconstruction and prediction are trained with $> 50$9 losses.

The role of reconstruction is not merely auxiliary regularization. The original paper interprets the $g < 15.8$0 reconstruction term as maximizing a variational lower bound on mutual information between input spectra and embeddings, since minimizing $g < 15.8$1 under a Laplacian decoder corresponds to minimizing $g < 15.8$2. This is intended to prevent the aligned latent space from becoming too lossy.

A further architectural variant, CLIP-split, decomposes the embedding into a 512-dimensional shared component and a 256-dimensional non-shared component. The shared branch is optimized for cross-modality alignment and prediction, while the non-shared branch captures modality-specific information useful for within-modality reconstruction and tasks that benefit from instrument-dependent structure.

4. Model variants and operational modes

The framework defines five principal variants, differing only in which objectives and embedding structure are active (Zhao et al., 2 Jul 2025).

Variant	Active objectives	Embedding design
CLIP	$g < 15.8$3 only	Single 768-d vector
CLIP-r	Contrastive + reconstruction	Single 768-d vector
CLIP-p	Contrastive + cross-prediction	Single 768-d vector
CLIP-pr	Contrastive + reconstruction + prediction	Single 768-d vector
CLIP-split	CLIP-pr with explicit decomposition	Shared 512-d + non-shared 256-d

This taxonomy is important because SpecCLIP is not a single encoder checkpoint but a family of representations spanning different trade-offs between alignment, information preservation, and modality specificity. In practice, CLIP-r, CLIP-pr, and CLIP-split frequently outperform pure CLIP on downstream tasks, indicating that decoder-based objectives preserve useful physical content beyond what is retained by alignment alone.

Downstream inference is deliberately lightweight. For point estimates, the framework trains per-parameter MLPs with architecture $g < 15.8$4, where the input can be raw fluxes or a 768-dimensional embedding. For uncertainty-aware inference, SpecCLIP also supports simulation-based inference with conditional normalizing flows, specifically MAF and NSF variants, using embeddings as conditioning inputs. The reported flow models use 2 transformations with 60 hidden units, sometimes 30, and have about 0.1M parameters.

The model family supports several operational modes. It can estimate atmospheric parameters and abundances from a single survey, apply regressors in a cross-survey setting because the embeddings are aligned, retrieve similar spectra in either modality, and translate spectra between LAMOST and XP. The later LoRA paper adds a further mode: adapting the existing embedding space and its downstream heads to a new survey, DESI, with very few labels (Zhao et al., 28 Jul 2025).

5. Empirical behavior across surveys and tasks

On the alignment task itself, SpecCLIP substantially improves agreement between LAMOST and XP embeddings relative to unaligned encoders (Zhao et al., 2 Jul 2025). On a validation set of 50,000 paired spectra, the mean cosine similarity of matched embeddings is 0.7828 for CLIP, 0.7783 for CLIP-r, 0.7820 for CLIP-p, 0.7778 for CLIP-pr, and 0.7854 for CLIP-split, whereas the encoder-only baseline without contrastive alignment yields 0.0533.

For spectrum translation, the paper reports weighted MSE in both directions. For XP$g < 15.8$5LRS, CLIP-split gives the best value at 0.3929, compared with 0.3932 for CLIP-p and 0.3934 for CLIP-pr. For LRS$g < 15.8$6XP, CLIP-pr is best at $g < 15.8$7, compared with $g < 15.8$8 for CLIP-p and $g < 15.8$9 for CLIP-split. Qualitatively, cross-modal prediction residuals are proposed as anomaly indicators for peculiar stars, unresolved binaries, wrong extinction, or calibration failures.

Downstream parameter estimation shows systematic gains over raw spectra and over modality-specific pre-training alone. On LAMOST LRS, $\mu$0 improves from $\mu$1 for raw spectra to $\mu$2 for the pre-trained encoder and to $\mu$3 for CLIP-split. For $\mu$4, the best result is obtained by CLIP-pr with 124 K and $\mu$5, compared with 149 K and $\mu$6 for the pre-trained encoder and 183 K and $\mu$7 for raw spectra. For $\mu$8, CLIP-split reaches $\mu$9. The same pattern extends across elemental abundances such as $\sigma$0, $\sigma$1, $\sigma$2, $\sigma$3, $\sigma$4, $\sigma$5, $\sigma$6, $\sigma$7, $\sigma$8, $\sigma$9, $T_{\rm eff}$0, and $T_{\rm eff}$1.

On Gaia XP, the gains are especially notable because raw XP is weak for several labels. For $T_{\rm eff}$2, raw XP gives $T_{\rm eff}$3, the pre-trained XP OAE gives $T_{\rm eff}$4, CLIP and CLIP-r give $T_{\rm eff}$5, and CLIP-split gives $T_{\rm eff}$6. For $T_{\rm eff}$7, CLIP-r is best at 170 K and $T_{\rm eff}$8. For $T_{\rm eff}$9, CLIP-pr is best at $\log g$0. For extinction $\log g$1, CLIP-split reaches $\log g$2. External validation against GALAH and Gaia RVS indicates that XP-based SpecCLIP metallicities remain accurate down to $\log g$3 or even $\log g$4, with scatter $\log g$5 dex for $\log g$6 and $\log g$7 dex for $\log g$8.

The later DESI adaptation study further probes transferability (Zhao et al., 28 Jul 2025). DESI spectra are normalized with the same pipeline as LAMOST LRS and interpolated to the 400–560 nm LAMOST grid with 1462 points. The main supervised task is $\log g$9 regression against APOGEE DR17, using 89 DESI–APOGEE stars for training, 9 for validation, and 396 for test. In the full test set, zero-shot MLP1 gives $L_2$0, zero-shot MLP2 gives $L_2$1, and the best overall result is LoRA4—fine-tuning only the projected/XP-aligned downstream MLP—which reaches $L_2$2. In the metal-rich regime $L_2$3, LoRA4 gives $L_2$4, whereas all methods struggle in the metal-poor regime $L_2$5, where $L_2$6 becomes negative and zero-shot MLP1 is the least poor option. This indicates that the cross-survey latent space is already informative and that modest adaptation of downstream heads can transfer it to a new instrument, but also that few-shot adaptation can overfit the dominant regime when metal-poor examples are scarce.

6. Scientific significance, limitations, and future directions

SpecCLIP is significant within stellar spectroscopy because it operationalizes a foundation-model workflow rather than a survey-specific calibration pipeline (Zhao et al., 2 Jul 2025). Large unlabeled corpora are used for self-supervised pre-training, cross-survey alignment is done once in a shared embedding space, and downstream tasks are delegated to comparatively small regressors or posterior estimators. This suggests a route toward survey-agnostic calibration, joint retrieval across archives, and scalable label transfer to very large datasets such as Gaia XP.

The framework also has methodological implications. The consistent gains of CLIP-r, CLIP-pr, and CLIP-split over pure contrastive alignment imply that an aligned latent space alone is insufficient if the embedding is allowed to discard physically relevant information. The decoder-augmented formulation instead preserves spectrum-specific content and allows cross-spectrum translation, which in turn creates a basis for anomaly detection through prediction residuals. The success of the shared/non-shared split in several tasks further suggests that explicit factorization of instrument-agnostic and instrument-specific information is useful in multi-survey spectroscopy.

At the same time, the current formulation has clear limitations. Downstream regressors still depend on high-quality labels from surveys such as APOGEE, GALAH, and Kepler, so systematic biases in those labels can propagate. Performance for asteroseismic parameters is less decisive because the labeled sample is small. Low-S/N regimes, strong emission-line objects, heavy reddening, and peculiar spectra are not fully characterized. The DESI transfer study shows an additional limitation: in the metal-poor regime, few-shot LoRA adaptation may degrade performance because the fine-tuning set is dominated by metal-rich stars (Zhao et al., 28 Jul 2025).

Future directions identified in the papers include extension to more instruments—LAMOST MRS, APOGEE, Subaru PFS, and DESI are named explicitly—together with efficient adapters for scalable multi-survey alignment (Zhao et al., 2 Jul 2025). The LoRA study strengthens that direction by showing that only 0.30% of the LRS encoder parameters, about 0.29% of the projection network, or 2.30% of a downstream MLP need to be updated in the tested DESI configurations, and that each experiment finishes in 10–180 seconds on a single NVIDIA V100 GPU (Zhao et al., 28 Jul 2025). A plausible implication is that SpecCLIP is less a fixed two-survey model than a prototype for a broader multi-instrument spectral hub in which new surveys are attached through lightweight adaptation rather than full retraining.