---
title: 'AppleCiDEr: Multimodal Early Transient Classifier'
url: https://www.emergentmind.com/topics/applecider
type: topic
---

# AppleCiDEr: Multimodal Early Transient Classifier

AppleCiDEr, expanded as *Applying multimodal learning to Classify transient Detections Early*, is a multimodal machine-learning framework for time-domain astronomy that integrates photometry, image cutouts, metadata, and spectra in order to classify transients and variable stars from real observational data, with explicit emphasis on early-time inference and broker integration [2507.16088]. Within the same program, the companion paper *“AppleCiDEr II: SpectraNet -- A Deep Learning Network for Spectroscopic Data”* develops the spectroscopy branch as a dedicated model for low-resolution transient spectra and redshift regression [2510.07215].

## 1. Scientific setting and intended scope

AppleCiDEr is motivated by the operational scale of contemporary and forthcoming synoptic surveys. The framework is presented against the backdrop that ZTF already produces roughly \(10^5\)–\(10^6\) alerts per night, while LSST is expected to produce about \(2\times 10^7\) alerts per night, making manual vetting infeasible and forcing classification systems to be automatic, early-time capable, multimodal, and production-oriented [2507.16088].

The scientific rationale is tied to the heterogeneity of the alert stream. AppleCiDEr is designed to handle both explosive transients and variable sources in one classification ecosystem, rather than treating those as separate problems. The stated use case is not merely archival labeling: early classification is intended to guide scarce follow-up resources, especially for rare and fast-evolving events such as tidal disruption events, kilonovae, and fast blue optical transients. This positions AppleCiDEr as both a classifier and a prioritization mechanism inside a broker-driven follow-up loop [2507.16088].

A defining claim of the project is that it provides a unified framework using real observational data to classify both transients and variable stars together. In that sense, “unified” has two meanings: unified across source regimes, and unified across modalities. The framework is not limited to a single sensor stream; it is organized around the premise that temporal evolution, morphology, contextual metadata, and spectroscopy provide complementary evidence that should be combined operationally rather than post hoc [2507.16088].

## 2. Data model, labels, and multimodal corpus

The primary alert source is ZTF, queried through Kowalski, while spectra are associated from Fritz, WISeREP, SDSS, DESI, GROWTH Marshal, and the Transient Name Server [2507.16088]. The object-level class distribution reported for the broader dataset covers 18,245 objects, including 6322 SN Ia, 145 SN Ib, 153 SN Ic, 695 SN II, 126 SN IIb, 627 SN IIP, 234 SN IIn, 963 CV, 8905 AGN, and 75 TDE; 9376 of the 18,245 objects are from BTS [2507.16088].

For the shared multimodal evaluation, AppleCiDEr restricts to a common-data subset with 7410 objects and 31,065 alerts. In this subset, the labels are coarsened into SN I, SN II, CV, AGN, and TDE, with the following object and alert counts: SN I 4830 objects and 22,791 alerts; SN II 1014 objects and 4520 alerts; CV 279 objects and 1262 alerts; AGN 1251 objects and 2375 alerts; and TDE 36 objects and 117 alerts [2507.16088]. The extreme rarity of TDEs is consequently a structural property of the training set rather than an incidental evaluation artifact.

The label-generation process is partly heterogeneous. The paper states that all ZTF sources in the dataset were classified into one of the transient categories with probability \(\ge 0.6\) using non-machine-learning methods, while spectral labels are drawn from TNS classifications or internal classification pipelines [2507.16088]. This suggests that AppleCiDEr is trained on a curated, operationally assembled corpus rather than on synthetic alerts or a single homogeneous archive.

The four modalities are represented differently and arrive on different timescales. All alert packets contain images and metadata; photometry accumulates over time; spectra are available only for a minority of objects. AppleCiDEr therefore adopts a modular architecture that permits partial-modality inference in production and uses a common-data subset when jointly evaluating all branches [2507.16088].

## 3. Multimodal architecture and fusion strategy

At the system level, AppleCiDEr combines three modality-specific model families: a photometry branch, an image-plus-metadata branch, and a spectra branch. The photometry branch uses transformer encoders; the image-plus-metadata branch is AstroMiNN; and the spectroscopy branch is SpectraNet-1D in the AppleCiDEr I formulation, extended in AppleCiDEr II as SpectraNet [2507.16088].

| Modality | Model family | Role |
|---|---|---|
| Photometry | [CLS]-Transformer; Informer baseline | Early-time sequence classification |
| Image cutouts + metadata | AstroMiNN | Contextual classification from alert packets |
| Spectra | SpectraNet-1D / SpectraNet | Spectral typing and redshift-related processing |
| Global combination | Class-wise averaging of predicted probabilities | Late multimodal fusion |

The integrated fusion scheme is explicit and comparatively simple. AppleCiDEr trains all networks jointly on the same dataset and performs class-wise averaging over their predicted probabilities, so the present system is a late-fusion framework rather than an end-to-end cross-modal attention model [2507.16088]. This simplicity is operationally useful because it supports missing modalities naturally, but it also defines one of the main future directions of the project.

The framework is embedded in astronomy broker infrastructure. Kowalski is used to query and store alert and photometric data, while Fritz and SkyPortal provide source management and spectroscopic information. Planned or described deployment targets include SkyPortal as a classification pipeline, BOOM as a broker-side classification pipeline, and LSST-facing public-broker deployment through BABAMUL [2507.16088]. This infrastructure orientation is central to AppleCiDEr’s identity: it is presented not only as a benchmark model but as a broker-integrated classification service.

## 4. Photometry and image–metadata encoders

The photometric input is modeled as tuples \((t_i, f_i, \sigma_{f,i}, b_i)\), with observations within a 12-hour window merged by inverse-variance weighting [2507.16088]. Each merged observation is encoded as a 7-dimensional feature vector consisting of time since first detection, time since previous observation, log-scaled flux, propagated uncertainty on log flux, and one-hot band encoding for \(g,r,i\). This representation is designed for irregular sampling, multiband observations, and partial early-time light curves.

The principal photometric encoder is the [CLS]-Transformer. A special [CLS] token is prepended to the projected observation sequence, and the final [CLS] representation is used for classification. The model uses learnable temporal embeddings combining linear and sinusoidal components, rather than fixed positional encodings, and the paper explicitly situates it as an early-epoch classifier for incomplete light curves [2507.16088]. Self-supervised pretraining is based on masked event modeling with losses on flux, band labels, and relative timing, while fine-tuning uses weighted focal loss:
\[
\mathcal{L}_{\text{focal}} = -\sum_c \alpha_c (1-p_c)^\gamma y_c \log p_c.
\]
The encoder itself follows standard self-attention:
\[
\mathrm{Attention}(Q,K,V)=\mathrm{softmax}\left(\frac{QK^\top}{\sqrt{d_k}}\right)V.
\]

The image-plus-metadata branch, AstroMiNN, is organized around domain-structured metadata towers and Mixture-of-Experts fusion. Metadata are partitioned into semantically coherent groups, including PSF, magnitude, light-curve history, spatial context, nearest-source context, coordinates, and a “Mega Tower” covering all metadata jointly [2507.16088]. The image encoder is ConvNeXt-Tiny pretrained on ImageNet, with custom heads that include per-sample sine-based positional embedding derived from RA/Dec. After concatenating tower outputs with image embeddings, a router selects top-2 experts in a Mixture-of-Experts module:
\[
w = \mathrm{softmax}(R(h)), \qquad z = \sum_{k\in \mathcal{T}(h)} w_k E_k(h).
\]
The total loss combines focal loss with an auxiliary specialization term:
\[
\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{focal}} + \lambda \cdot \mathrm{MSE}(w_{\text{expert}}, w_{\text{ideal}}).
\]

This design reflects a domain-specific interpretation of alert metadata. Rather than treating metadata as a flat tabular vector, AppleCiDEr imposes a partial scientific structure on the feature space before multimodal fusion. A plausible implication is that the system is intended to encode alert-broker priors directly into network topology, especially for contextual classes such as CVs and nuclear events.

## 5. Spectroscopic branch and AppleCiDEr II

In the integrated AppleCiDEr framework, alerts do not themselves contain spectra; the system associates a single archival spectrum per object, specifically the spectrum closest in time to the object’s first photometric observation, while the spectral-only model uses all spectra per object [2507.16088]. Spectral preprocessing in AppleCiDEr includes rest-frame correction,
\[
\lambda_{\text{rest}} = \frac{\lambda_{\text{obs}}}{1+z},
\]
followed by interpolation to 4096 evenly spaced points and per-spectrum normalization
\[
f_{\text{norm}} = \frac{f-\mu}{\sigma}.
\]

The spectroscopy-focused companion paper develops this branch into SpectraNet, a 1D CNN tailored to low-resolution transient spectra from SEDM and other instruments [2510.07215]. Its stated motivation is that template fitting and parametric models are limited for low-resolution, heterogeneous, time-series spectra, especially when broad continuum structure, blended features, and missing wavelength regions are prominent. In that paper, spectra are standardized by redshift correction to the rest frame, restriction to \([3000\,\text{\AA}, 10400\,\text{\AA}]\), interpolation to 4096 wavelength points, and Z-score normalization, with random masking on either the blue or red side to simulate detector gaps and edge truncation [2510.07215].

SpectraNet’s central architectural idea is multi-scale convolution. Each SpectraBlock contains parallel 1D convolutions with small, medium, and large kernels, concatenated and projected with a \(1\times 1\) convolution:
\[
h = \mathrm{Concat}\bigl(\mathrm{Conv}_{k_s}(x), \mathrm{Conv}_{k_m}(x), \mathrm{Conv}_{k_l}(x)\bigr), \qquad
\tilde h = \mathrm{Conv}_{1\times 1}(h).
\]
The preferred model uses five stacked SpectraBlocks, adaptive max pooling, and two fully connected layers, with LayerNorm and GELU in the selected configuration [2510.07215]. For classification it uses class-balanced focal loss; for redshift regression it outputs a scalar passed through Softplus,
\[
\mathrm{Softplus}(x)=\ln(1+e^x),
\]
so that predictions remain nonnegative.

AppleCiDEr II also expands the spectroscopy task itself. After merging and filtering, the main spectral experiments use 10 classes, including 22,258 SN Ia spectra, 3750 AGN, 2380 SN Ic, 1743 SN Ib, 1556 SN IIn, 1385 SN IIb, 1083 CV, 1033 TDE, 1019 SLSN-I, and 876 SN IIP spectra [2510.07215]. Multiple spectra from the same object are retained when available, and the paper explicitly reports that multi-epoch spectra improve performance relative to using only spectra close to photometric peak.

## 6. Reported performance and operational workflow

AppleCiDEr reports strong results at the branch level and competitive integrated performance on the common multimodal classes [2507.16088].

| Component | Benchmark | Reported result |
|---|---|---|
| Photometry | Informer vs [CLS]-Transformer | \(59 \pm 0.3\%\) vs \(88 \pm 0.4\%\) accuracy |
| Photometry | Macro-average AUC | 0.89 vs 0.97 |
| Image+metadata | AstroMiNN vs BTSbot, CV AUC | 0.99 vs 0.97 |
| Image+metadata | AstroMiNN vs BTSbot, Nuclear AUC | 0.99 vs 0.97 |
| Spectra | SpectraNet vs GalSpecNet accuracy | \(87.0 \pm 0.5\%\) vs \(82.6 \pm 0.6\%\) |
| Spectra | SpectraNet vs GalSpecNet micro-average AUC | 0.98 vs 0.96 |
| Integrated AppleCiDEr test set | Correct-class rates | SN I 95%, SN II 93%, CV 90%, AGN 98%, TDE 31% |

For photometry, the [CLS]-Transformer substantially outperforms the Informer baseline on light curves cut off at 30 days, with macro-average AUC improving from 0.89 to 0.97 and TDE AUC from 0.82 to 0.95 [2507.16088]. For AstroMiNN, the gains are not uniform across all classes, but the paper emphasizes improvements for CV and nuclear events relative to BTSbot. For the spectral branch in AppleCiDEr I, SpectraNet improves overall accuracy from \(82.6 \pm 0.6\%\) to \(87.0 \pm 0.5\%\) and raises per-class AUCs above 0.96 for all listed classes [2507.16088].

The dedicated SpectraNet paper reports still stronger spectral classification numbers on its broader 10-class setup. Against prior CNN baselines trained and tested on the same spectral data, SpectraNet reaches accuracy 0.95, Top-3 accuracy 0.98, macro-F1 0.88, and macro-AUC 0.99 [2510.07215]. Per-class AUCs are frequently 0.99–1.00, and the main remaining confusion pair is SN Ib versus SN Ic. The same paper also reports a redshift regression bias of \(-0.002\), \(\sigma = 0.016\), MAD \(=0.004\), MAE \(=0.008\), RMSE \(=0.018\), max error \(=0.34\), \(R^2 = 0.77\), and outlier rate \(=1.6\) [2510.07215].

Operationally, the framework is meant to support a closed-loop follow-up system. The stated broker workflow is: broker receives alerts; AppleCiDEr classifies the source; the classification is sent to SkyPortal; the system can suggest spectroscopic follow-up, especially with SEDM; and the resulting spectra are then fed back into AppleCiDEr for refined inference [2507.16088]. This makes AppleCiDEr not only a multimodal classifier but also a mechanism for allocating follow-up effort.

## 7. Limitations, open problems, and future directions

The most explicit limitation of the integrated system is class imbalance, especially for TDEs. On the common multimodal dataset, TDE comprises only 36 objects and 117 alerts, and the integrated test-set confusion matrix reports only 31% correct-classification for TDE, with major confusions into SN II, SN I, and AGN [2507.16088]. This is not an isolated weakness but the clearest reported failure mode of the current framework.

A second limitation is the fusion strategy. AppleCiDEr currently combines modalities by averaging predicted probabilities rather than by a richer joint fusion mechanism. The paper is explicit that future work will explore stronger fusion techniques rather than “just averaging the output” [2507.16088]. This suggests that the present system should be understood as a modular multimodal ensemble with shared operational infrastructure, not yet as a fully unified cross-modal representation learner.

The spectroscopy branch introduces its own open problems. AppleCiDEr II states that multiple spectra are retained at the dataset level, but the model processes one spectrum at a time; a promising extension is to account for multiple spectra from different phases within the network [2510.07215]. The same paper also states that softmax scores are used as confidence and that no explicit uncertainty calibration or Bayesian uncertainty model is developed. Thus, despite the project’s operational orientation, predictive uncertainty remains comparatively underdeveloped.

Finally, AppleCiDEr’s LSST-era positioning is architectural rather than already demonstrated on Rubin data. The framework is described as ready for the LSST era in the sense of scale, modality coverage, and broker integration, but the papers also note that domain adaptation will be required, especially because of filter-system differences between ZTF and LSST [2507.16088]. The same combination of production ambition and acknowledged incompleteness defines AppleCiDEr’s current status: a broker-oriented multimodal framework with strong branch-level performance, a clear rare-event follow-up use case, and identifiable bottlenecks in fusion, class imbalance, and uncertainty handling.

Source: https://www.emergentmind.com/topics/applecider