AppleCiDEr: Multimodal Early Transient Classifier

• AppleCiDEr is a unified multimodal ML framework that integrates photometry, image cutouts, metadata, and spectra to classify both transients and variable stars early in synoptic surveys.
• It employs a modular design with branch-specific models such as a [CLS]-Transformer for photometry and ConvNeXt-based AstroMiNN for image-plus-metadata processing, yielding high accuracy.
• Integrated within broker infrastructures, AppleCiDEr facilitates rapid follow-up prioritization for rare, fast-evolving events, outperforming baseline methods.

AppleCiDEr, expanded as Applying multimodal learning to Classify transient Detections Early, is a multimodal machine-learning framework for time-domain astronomy that integrates photometry, image cutouts, metadata, and spectra in order to classify transients and variable stars from real observational data, with explicit emphasis on early-time inference and broker integration (Junell et al., 21 Jul 2025). Within the same program, the companion paper “AppleCiDEr II: SpectraNet -- A Deep Learning Network for Spectroscopic Data” develops the spectroscopy branch as a dedicated model for low-resolution transient spectra and redshift regression (Xu et al., 8 Oct 2025).

1. Scientific setting and intended scope

AppleCiDEr is motivated by the operational scale of contemporary and forthcoming synoptic surveys. The framework is presented against the backdrop that ZTF already produces roughly $10^5$–$10^6$ alerts per night, while LSST is expected to produce about $2\times 10^7$ alerts per night, making manual vetting infeasible and forcing classification systems to be automatic, early-time capable, multimodal, and production-oriented (Junell et al., 21 Jul 2025).

The scientific rationale is tied to the heterogeneity of the alert stream. AppleCiDEr is designed to handle both explosive transients and variable sources in one classification ecosystem, rather than treating those as separate problems. The stated use case is not merely archival labeling: early classification is intended to guide scarce follow-up resources, especially for rare and fast-evolving events such as tidal disruption events, kilonovae, and fast blue optical transients. This positions AppleCiDEr as both a classifier and a prioritization mechanism inside a broker-driven follow-up loop (Junell et al., 21 Jul 2025).

A defining claim of the project is that it provides a unified framework using real observational data to classify both transients and variable stars together. In that sense, “unified” has two meanings: unified across source regimes, and unified across modalities. The framework is not limited to a single sensor stream; it is organized around the premise that temporal evolution, morphology, contextual metadata, and spectroscopy provide complementary evidence that should be combined operationally rather than post hoc (Junell et al., 21 Jul 2025).

2. Data model, labels, and multimodal corpus

The primary alert source is ZTF, queried through Kowalski, while spectra are associated from Fritz, WISeREP, SDSS, DESI, GROWTH Marshal, and the Transient Name Server (Junell et al., 21 Jul 2025). The object-level class distribution reported for the broader dataset covers 18,245 objects, including 6322 SN Ia, 145 SN Ib, 153 SN Ic, 695 SN II, 126 SN IIb, 627 SN IIP, 234 SN IIn, 963 CV, 8905 AGN, and 75 TDE; 9376 of the 18,245 objects are from BTS (Junell et al., 21 Jul 2025).

For the shared multimodal evaluation, AppleCiDEr restricts to a common-data subset with 7410 objects and 31,065 alerts. In this subset, the labels are coarsened into SN I, SN II, CV, AGN, and TDE, with the following object and alert counts: SN I 4830 objects and 22,791 alerts; SN II 1014 objects and 4520 alerts; CV 279 objects and 1262 alerts; AGN 1251 objects and 2375 alerts; and TDE 36 objects and 117 alerts (Junell et al., 21 Jul 2025). The extreme rarity of TDEs is consequently a structural property of the training set rather than an incidental evaluation artifact.

The label-generation process is partly heterogeneous. The paper states that all ZTF sources in the dataset were classified into one of the transient categories with probability $\ge 0.6$ using non-machine-learning methods, while spectral labels are drawn from TNS classifications or internal classification pipelines (Junell et al., 21 Jul 2025). This suggests that AppleCiDEr is trained on a curated, operationally assembled corpus rather than on synthetic alerts or a single homogeneous archive.

The four modalities are represented differently and arrive on different timescales. All alert packets contain images and metadata; photometry accumulates over time; spectra are available only for a minority of objects. AppleCiDEr therefore adopts a modular architecture that permits partial-modality inference in production and uses a common-data subset when jointly evaluating all branches (Junell et al., 21 Jul 2025).

3. Multimodal architecture and fusion strategy

At the system level, AppleCiDEr combines three modality-specific model families: a photometry branch, an image-plus-metadata branch, and a spectra branch. The photometry branch uses transformer encoders; the image-plus-metadata branch is AstroMiNN; and the spectroscopy branch is SpectraNet-1D in the AppleCiDEr I formulation, extended in AppleCiDEr II as SpectraNet (Junell et al., 21 Jul 2025).

Modality	Model family	Role
Photometry	[CLS]-Transformer; Informer baseline	Early-time sequence classification
Image cutouts + metadata	AstroMiNN	Contextual classification from alert packets
Spectra	SpectraNet-1D / SpectraNet	Spectral typing and redshift-related processing
Global combination	Class-wise averaging of predicted probabilities	Late multimodal fusion

The integrated fusion scheme is explicit and comparatively simple. AppleCiDEr trains all networks jointly on the same dataset and performs class-wise averaging over their predicted probabilities, so the present system is a late-fusion framework rather than an end-to-end cross-modal attention model (Junell et al., 21 Jul 2025). This simplicity is operationally useful because it supports missing modalities naturally, but it also defines one of the main future directions of the project.

The framework is embedded in astronomy broker infrastructure. Kowalski is used to query and store alert and photometric data, while Fritz and SkyPortal provide source management and spectroscopic information. Planned or described deployment targets include SkyPortal as a classification pipeline, BOOM as a broker-side classification pipeline, and LSST-facing public-broker deployment through BABAMUL (Junell et al., 21 Jul 2025). This infrastructure orientation is central to AppleCiDEr’s identity: it is presented not only as a benchmark model but as a broker-integrated classification service.

4. Photometry and image–metadata encoders

The photometric input is modeled as tuples $(t_i, f_i, \sigma_{f,i}, b_i)$, with observations within a 12-hour window merged by inverse-variance weighting (Junell et al., 21 Jul 2025). Each merged observation is encoded as a 7-dimensional feature vector consisting of time since first detection, time since previous observation, log-scaled flux, propagated uncertainty on log flux, and one-hot band encoding for $g,r,i$. This representation is designed for irregular sampling, multiband observations, and partial early-time light curves.

The principal photometric encoder is the [CLS]-Transformer. A special [CLS] token is prepended to the projected observation sequence, and the final [CLS] representation is used for classification. The model uses learnable temporal embeddings combining linear and sinusoidal components, rather than fixed positional encodings, and the paper explicitly situates it as an early-epoch classifier for incomplete light curves (Junell et al., 21 Jul 2025). Self-supervised pretraining is based on masked event modeling with losses on flux, band labels, and relative timing, while fine-tuning uses weighted focal loss: $$\mathcal{L}_{\text{focal}} = -\sum_c \alpha_c (1-p_c)^\gamma y_c \log p_c.$$ The encoder itself follows standard self-attention: $$\mathrm{Attention}(Q,K,V)=\mathrm{softmax}\left(\frac{QK^\top}{\sqrt{d_k}}\right)V.$$

The image-plus-metadata branch, AstroMiNN, is organized around domain-structured metadata towers and Mixture-of-Experts fusion. Metadata are partitioned into semantically coherent groups, including PSF, magnitude, light-curve history, spatial context, nearest-source context, coordinates, and a “Mega Tower” covering all metadata jointly (Junell et al., 21 Jul 2025). The image encoder is ConvNeXt-Tiny pretrained on ImageNet, with custom heads that include per-sample sine-based positional embedding derived from RA/Dec. After concatenating tower outputs with image embeddings, a router selects top-2 experts in a Mixture-of-Experts module: $$w = \mathrm{softmax}(R(h)), \qquad z = \sum_{k\in \mathcal{T}(h)} w_k E_k(h).$$ The total loss combines focal loss with an auxiliary specialization term: $$\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{focal}} + \lambda \cdot \mathrm{MSE}(w_{\text{expert}}, w_{\text{ideal}}).$$

This design reflects a domain-specific interpretation of alert metadata. Rather than treating metadata as a flat tabular vector, AppleCiDEr imposes a partial scientific structure on the feature space before multimodal fusion. A plausible implication is that the system is intended to encode alert-broker priors directly into network topology, especially for contextual classes such as CVs and nuclear events.

5. Spectroscopic branch and AppleCiDEr II

In the integrated AppleCiDEr framework, alerts do not themselves contain spectra; the system associates a single archival spectrum per object, specifically the spectrum closest in time to the object’s first photometric observation, while the spectral-only model uses all spectra per object (Junell et al., 21 Jul 2025). Spectral preprocessing in AppleCiDEr includes rest-frame correction,

$10^6$0

followed by interpolation to 4096 evenly spaced points and per-spectrum normalization

$10^6$1

The spectroscopy-focused companion paper develops this branch into SpectraNet, a 1D CNN tailored to low-resolution transient spectra from SEDM and other instruments (Xu et al., 8 Oct 2025). Its stated motivation is that template fitting and parametric models are limited for low-resolution, heterogeneous, time-series spectra, especially when broad continuum structure, blended features, and missing wavelength regions are prominent. In that paper, spectra are standardized by redshift correction to the rest frame, restriction to $10^6$2, interpolation to 4096 wavelength points, and Z-score normalization, with random masking on either the blue or red side to simulate detector gaps and edge truncation (Xu et al., 8 Oct 2025).

SpectraNet’s central architectural idea is multi-scale convolution. Each SpectraBlock contains parallel 1D convolutions with small, medium, and large kernels, concatenated and projected with a $10^6$3 convolution: $10^6$4 The preferred model uses five stacked SpectraBlocks, adaptive max pooling, and two fully connected layers, with LayerNorm and GELU in the selected configuration (Xu et al., 8 Oct 2025). For classification it uses class-balanced focal loss; for redshift regression it outputs a scalar passed through Softplus,

$10^6$5

so that predictions remain nonnegative.

AppleCiDEr II also expands the spectroscopy task itself. After merging and filtering, the main spectral experiments use 10 classes, including 22,258 SN Ia spectra, 3750 AGN, 2380 SN Ic, 1743 SN Ib, 1556 SN IIn, 1385 SN IIb, 1083 CV, 1033 TDE, 1019 SLSN-I, and 876 SN IIP spectra (Xu et al., 8 Oct 2025). Multiple spectra from the same object are retained when available, and the paper explicitly reports that multi-epoch spectra improve performance relative to using only spectra close to photometric peak.

6. Reported performance and operational workflow

AppleCiDEr reports strong results at the branch level and competitive integrated performance on the common multimodal classes (Junell et al., 21 Jul 2025).

Component	Benchmark	Reported result
Photometry	Informer vs [CLS]-Transformer	$10^6$6 vs $10^6$7 accuracy
Photometry	Macro-average AUC	0.89 vs 0.97
Image+metadata	AstroMiNN vs BTSbot, CV AUC	0.99 vs 0.97
Image+metadata	AstroMiNN vs BTSbot, Nuclear AUC	0.99 vs 0.97
Spectra	SpectraNet vs GalSpecNet accuracy	$10^6$8 vs $10^6$9
Spectra	SpectraNet vs GalSpecNet micro-average AUC	0.98 vs 0.96
Integrated AppleCiDEr test set	Correct-class rates	SN I 95%, SN II 93%, CV 90%, AGN 98%, TDE 31%

For photometry, the [CLS]-Transformer substantially outperforms the Informer baseline on light curves cut off at 30 days, with macro-average AUC improving from 0.89 to 0.97 and TDE AUC from 0.82 to 0.95 (Junell et al., 21 Jul 2025). For AstroMiNN, the gains are not uniform across all classes, but the paper emphasizes improvements for CV and nuclear events relative to BTSbot. For the spectral branch in AppleCiDEr I, SpectraNet improves overall accuracy from $2\times 10^7$0 to $2\times 10^7$1 and raises per-class AUCs above 0.96 for all listed classes (Junell et al., 21 Jul 2025).

The dedicated SpectraNet paper reports still stronger spectral classification numbers on its broader 10-class setup. Against prior CNN baselines trained and tested on the same spectral data, SpectraNet reaches accuracy 0.95, Top-3 accuracy 0.98, macro-F1 0.88, and macro-AUC 0.99 (Xu et al., 8 Oct 2025). Per-class AUCs are frequently 0.99–1.00, and the main remaining confusion pair is SN Ib versus SN Ic. The same paper also reports a redshift regression bias of $2\times 10^7$2, $2\times 10^7$3, MAD $2\times 10^7$4, MAE $2\times 10^7$5, RMSE $2\times 10^7$6, max error $2\times 10^7$7, $2\times 10^7$8, and outlier rate $2\times 10^7$9 (Xu et al., 8 Oct 2025).

Operationally, the framework is meant to support a closed-loop follow-up system. The stated broker workflow is: broker receives alerts; AppleCiDEr classifies the source; the classification is sent to SkyPortal; the system can suggest spectroscopic follow-up, especially with SEDM; and the resulting spectra are then fed back into AppleCiDEr for refined inference (Junell et al., 21 Jul 2025). This makes AppleCiDEr not only a multimodal classifier but also a mechanism for allocating follow-up effort.

7. Limitations, open problems, and future directions

The most explicit limitation of the integrated system is class imbalance, especially for TDEs. On the common multimodal dataset, TDE comprises only 36 objects and 117 alerts, and the integrated test-set confusion matrix reports only 31% correct-classification for TDE, with major confusions into SN II, SN I, and AGN (Junell et al., 21 Jul 2025). This is not an isolated weakness but the clearest reported failure mode of the current framework.

A second limitation is the fusion strategy. AppleCiDEr currently combines modalities by averaging predicted probabilities rather than by a richer joint fusion mechanism. The paper is explicit that future work will explore stronger fusion techniques rather than “just averaging the output” (Junell et al., 21 Jul 2025). This suggests that the present system should be understood as a modular multimodal ensemble with shared operational infrastructure, not yet as a fully unified cross-modal representation learner.

The spectroscopy branch introduces its own open problems. AppleCiDEr II states that multiple spectra are retained at the dataset level, but the model processes one spectrum at a time; a promising extension is to account for multiple spectra from different phases within the network (Xu et al., 8 Oct 2025). The same paper also states that softmax scores are used as confidence and that no explicit uncertainty calibration or Bayesian uncertainty model is developed. Thus, despite the project’s operational orientation, predictive uncertainty remains comparatively underdeveloped.

Finally, AppleCiDEr’s LSST-era positioning is architectural rather than already demonstrated on Rubin data. The framework is described as ready for the LSST era in the sense of scale, modality coverage, and broker integration, but the papers also note that domain adaptation will be required, especially because of filter-system differences between ZTF and LSST (Junell et al., 21 Jul 2025). The same combination of production ambition and acknowledged incompleteness defines AppleCiDEr’s current status: a broker-oriented multimodal framework with strong branch-level performance, a clear rare-event follow-up use case, and identifiable bottlenecks in fusion, class imbalance, and uncertainty handling.