--- title: 'RAIL: Redshift Assessment Infrastructure Layers' url: https://www.emergentmind.com/topics/redshift-assessment-infrastructure-layers-rail type: topic --- # RAIL: Redshift Assessment Infrastructure Layers Redshift Assessment Infrastructure Layers (RAIL) is an open source Python library for at-scale probabilistic photometric redshift estimation that organizes end-to-end workflows into modular layers for forward modeling, estimator training and inference, and evaluation. Initiated by the LSST Dark Energy Science Collaboration with contributions from the LSST Interdisciplinary Network for Collaboration and Computing Frameworks team, RAIL was designed for Rubin-era photo-$z$ stress-testing and production, but its engines, degraders, estimators, and metrics are survey-agnostic by construction. A defining capability is the coupling of a unified estimator API to forward models that can generate realistic photometry and, in PZFlow-based workflows, expose true conditional posteriors $p(z\mid\mathrm{photometry})$ for galaxy-by-galaxy PDF assessment [2505.02928]. ## 1. Scientific motivation and scope RAIL emerged from the requirements of Rubin Observatory extragalactic science, where most analyses must infer distance information for galaxies that lack spectroscopic redshifts. The v1 overview states that LSST will produce deep ugrizy imaging over $\sim 18{,}000\,\mathrm{deg}^2$ and a catalog of $\sim 20$ billion galaxies down to $i \approx 26.4$, while the vast majority of these galaxies will not have spectroscopic redshifts. In this regime, redshift uncertainties are complex and often non-Gaussian, so per-galaxy photo-$z$ probability density functions are the preferred representation rather than point estimates with Gaussian errors [2505.02928]. The library’s design responds directly to limitations identified in earlier DESC experiments. Those experiments showed disagreement among methods even with identical inputs, the possibility that some commonly used metrics can be gamed or are uninformative, and the need for probabilistic forward models with access to true conditional PDFs. RAIL therefore targets three linked tasks: at-scale probabilistic photo-$z$ estimation with a unified API, stress-testing under realistic LSST-like systematics, and optimization and validation of both per-galaxy PDFs and ensemble redshift distributions $N(z)$ for science cases including cosmic shear, large-scale structure, clusters, and supernova cosmology [2505.02928]. A recurring misconception is that RAIL is only a benchmarking harness. The software and its applications show a broader scope: it is simultaneously a stress-testing framework, a production environment, and a calibration-and-evaluation toolkit. This is explicit in the DP1 analysis, where RAIL is used to ingest and standardize Rubin photometry, train and benchmark multiple algorithms, produce per-galaxy PDFs and ensemble $n(z)$, and interoperate with Rubin Data Management, the Photo-$z$ Server, and LSDB [2510.07370]. ## 2. Layered software architecture RAIL v1 is organized into three major subpackages, each corresponding to a distinct layer in the photo-$z$ workflow [2505.02928]. | Subpackage | Primary role | Representative components | |---|---|---| | `RAIL.creation` | Forward modeling of truth catalogs and realistic photometry | FSPS, DSPS, PZFlow, degraders | | `RAIL.estimation` | Per-galaxy PDF estimation, ensemble summarization, tomographic classification | Informer, Estimator, Summarizer, Classifier | | `RAIL.evaluation` | Metrics and diagnostics for PDFs, point estimates, and tomographic overlap | PIT, KS, CvM, AD, CDELoss, bias, scatter | The core execution model is built atop `ceci`, with stages as atomic operations and pipelines as directed acyclic graphs with provenance. Configuration is split across YAML files: a `ceci` pipeline configuration for stage classes, inputs, outputs, and parallel parameters, and a RAIL YAML for stage-specific parameters. Interactive `RailStage` wrappers expose `inform()`, `estimate()`, and `summarize()` for notebook use, while `run()` supports reproducible batch execution. MPI parallelization and chunked I/O are central to the design for scale [2505.02928]. RAIL’s internal data model is based on `DataHandle` abstractions in `rail.core.data`. The v1 description identifies `TableHandle` for tabular catalogs in `numpy`, `pandas`, `pyarrow`, or `Astropy`; `QPHandle` for `qp.Ensemble` objects containing 1D PDFs; and `ModelHandle` for serialized models. These handles support chunked reading and writing for memory efficiency. The same standardization allows drop-in replacement of algorithms wrapped as subclasses of common stage types such as `CatInformer`, `CatEstimator`, `PZInformer`, `PZSummarizer`, `PZClassifier`, and `SZPZSummarizer` [2505.02928]. The architecture has also been adopted in operational settings. In the DP1 work, `RAIL_projects` generates `ceci` pipeline configurations from named “flavors,” while versioned configurations in `rail_project_config` preserve training cuts, algorithms, parameters, and target fields. This establishes a reproducible bridge between the research software stack and Rubin production-style workflows [2510.07370]. ## 3. Forward modeling, degraders, and stress-testing The `RAIL.creation` layer provides engines for generating photometry and degraders for imposing realistic systematics. The v1 release describes three principal engines. `rail_fsps` generates rest-frame galaxy SEDs including stars, gas, dust, and AGN from supplied physical parameters and computes apparent AB magnitudes through user-specified filters. `rail_dsps` provides a JAX-based differentiable SPS path. `rail_pzflow` trains a normalizing-flow generative model of the joint distribution of colors and redshift and can both sample new galaxies and provide access to true conditional PDFs $p(z\mid\mathrm{photometry})$ [2505.02928]. The degradation layer encodes observational and spectroscopic imperfections. Documented examples include `LSSTErrorModel`, which wraps `PhotErr` to add heteroscedastic Gaussian magnitude noise; `ObservingConditionDegrader`, which assigns per-galaxy observing-condition maps; `QuantityCut` for magnitude, color, or SNR selection; `LineConfusion` for emission-line misidentification; `SpectroscopicSelectors` and `GridSelection` for survey-specific targeting and success rates; `SOMSpecSelector` for self-organizing-map-based matched subsets; and a blending degrader that constructs unrecognized blends via Friends-of-Friends on sky position [2505.02928]. A particularly detailed demonstration appears in the study of spectroscopic training-set imperfection, where RAIL served as the backbone for building, degrading, and running controlled photo-$z$ experiments. The workflow followed the sequence `Creation` $\rightarrow$ `Degradation` $\rightarrow$ `Informer` $\rightarrow$ `Estimator`. In that study, the `Creation` layer used a PZFlow normalizing flow trained on Rubin–Roman OpenUniverse Tile 10307 to emulate the joint distribution $P(z,m)$, and the `FlowPosterior` stage evaluated per-galaxy true posteriors $P(z\mid m)$ by conditioning on individual galaxy magnitudes. Degraders then applied an LSST-like photometric error model at 10-year depth, inverse-redshift incompleteness, or survey-specific spectroscopic selection functions before estimator training and inference [2601.10797]. The spectroscopic stress tests in that work were deliberately heterogeneous. The survey-based degraders emulated pseudo-BOSS, GAMA, the HSC training union, zCOSMOS-bright, pseudo-DEEP2, and VVDSf02, with surviving training-set sizes ranging from $\approx 195$ galaxies for pseudo-BOSS to $\approx 80{,}845$ for VVDSf02. The same paper also defined an inverse redshift incompleteness degrader with survival probability $$ S(z; z_0)=1-\min(1,z_0/z), $$ tested at pivot values $z_0 \in \{0.1,0.3,0.6,1.0,1.4\}$, while the v1 documentation describes `InvRedshiftIncompleteness` as keeping low-$z$ galaxies and randomly removing high-$z$ galaxies with $$ p(z)=\min\left(1,\frac{z_{\mathrm{th}}}{z}\right). $$ Both descriptions underscore the same architectural point: RAIL treats training-set non-representativeness as a configurable degradation problem rather than an external nuisance [2601.10797]. ## 4. Estimators, summarizers, and image-based extensions The `RAIL.estimation` layer exposes a unified API across template-fitting, machine-learning, hybrid, and image-based methods. The v1 release lists `BPZ` and `LePHARE` as template-fitting estimators; `CMNN`, `DNF`, `FlexZBoost`, `GPz`, scikit-learn wrappers for k-nearest neighbors and an `MLPRegressor`, and a `PZFlow` estimator as machine-learning methods; `Delight` as a hybrid Gaussian-process template-learning method; and `DeepDISC` as an image-based estimator. Ensemble-level products are handled by summarizers such as naive stacking, variational inference stacking, `NZDir`, SOM-based summarizers, and `Yet Another Wizz`, while tomographic assignments can be produced by `Uniform_binning`, `EqualCount`, and a `RandomForestClassifier` [2505.02928]. This common interface is consequential because the wrapped algorithms have markedly different inductive biases. In the training-imperfection study, `TrainZ` was used as a control that assigns each test galaxy the same PDF equal to the training-set redshift distribution; `CMNN` selected variable-size neighborhoods in color space and returned unimodal Gaussian PDFs; `GPz` returned single-Gaussian PDFs whose variance combined aleatoric and epistemic components; `FlexZBoost` estimated nonparametric conditional densities via a basis expansion using training neighbors in color space; and `PZFlow` conditioned a learned joint flow $P(z,m)$ on test magnitudes and could refuse to produce an estimate when conditioning was out of support [2601.10797]. RAIL’s extensibility is also visible in `rail_deepdisc`, which integrates the image-based DeepDISC-photoz model as an `Estimator` stage consuming six-band LSST ugrizy coadded images rather than catalogs. In that integration, DeepDISC is a detectron2-based instance-segmentation framework with an added redshift Region of Interest head. The redshift head is a Mixture Density Network with $K=5$ Gaussian components and outputs a per-object redshift PDF on a configured common $z$-grid, along with point estimates such as the mean and mode and summary intervals derived from the CDF [2411.18769]. The DeepDISC integration also illustrates the breadth of the RAIL abstraction. A typical pipeline can couple `rail.data` for reading LSST DC2 coadds or cutouts and loading truth redshifts, `rail.estimation` for training or informing DeepDISC and catalog-based estimators such as `rail_bpz` and `rail_flexzboost`, and `rail.evaluation` for standardized metrics. YAML configuration files define model checkpoint paths, image channels, ROI-head settings, training schedules, and the common redshift grid, allowing image-based and catalog-based methods to be evaluated on equal footing [2411.18769]. ## 5. Evaluation methodology and metric design RAIL’s evaluation layer formalizes three complementary families of diagnostics: PDF-to-PDF, PDF-to-point, and point-to-point metrics. The v1 documentation lists Cramér–von Mises, Kolmogorov–Smirnov, RMSE over densities, Kullback–Leibler divergence, and Anderson–Darling as PDF-to-PDF measures; Conditional Density Estimation loss, Probability Integral Transform, and Brier score as PDF-to-point measures; and residual-based bias, scatter, and outlier statistics as point-to-point measures. For point estimates, the normalized residual is $$ \Delta z_i = \frac{z_{p,i}-z_{t,i}}{1+z_{t,i}}, $$ with scatter often summarized through $\sigma_{\mathrm{MAD}}$ or $\sigma_{\mathrm{IQR}}$, and outliers defined in v1 by $|\Delta z|>\max(0.06,3\sigma_{\mathrm{IQR}})$ [2505.02928]. The PIT occupies a central place in RAIL’s calibration philosophy. In v1, $$ \mathrm{PIT}=\int_{-\infty}^{z_t} p(z)\,dz, $$ and well-calibrated PDFs should yield a uniform PIT distribution. The evaluation module therefore supports PIT histograms, PIT–QQ plots against $U(0,1)$, and application of CvM, KS, and AD directly to the PIT distribution. This emphasis on calibration distinguishes RAIL from workflows that reduce photo-$z$ quality to point-estimate residuals alone [2505.02928]. The 2026 training-set-imperfection analysis sharpened this metric perspective by comparing each estimator’s $\hat{p}_i(z)$ to a per-galaxy true posterior $q_i(z)=P(z\mid m_i)$ obtained from `FlowPosterior`. Metrics were computed per galaxy and then aggregated as distributions, often through quantile curves. In that setting, the Kullback–Leibler divergence, Wasserstein distance, and PIT were found to be particularly informative for diagnosing training-set imperfection and algorithmic behavior, whereas RMSE and KS were less discriminating except in catastrophic cases. The same study notes that these metrics were implemented outside the RAIL version used there with `numpy`, `scipy`, `sklearn`, and `qp`, although some are available in later RAIL versions including v1 [2601.10797]. The paper also gives precise diagnostic interpretations for PIT morphology. “Side spikes” indicate too-narrow PDFs or offset means, while “middle bumps” indicate over-broad PDFs or symmetric bimodality around the truth. Under spectroscopic degradations, CMNN and FlexZBoost showed roughly symmetric side spikes corresponding to high/low catastrophes, whereas GPz and TrainZ showed pronounced right-side spikes in some degraded cases. This metric behavior is not incidental; it is tied to RAIL’s ability to evaluate estimated PDFs against true conditional posteriors rather than only against scalar redshift labels [2601.10797]. ## 6. Empirical applications, deployment, and limitations RAIL has been exercised in both simulation-driven and on-sky settings. In the DeepDISC-photoz study on LSST DESC DC2 simulations, RAIL was used to compare DeepDISC-photoz against BPZ and FlexZBoost on a matched test set of $N=1{,}254{,}379$ objects. Using the PDF mode as the point estimate, the reported global metrics were: DeepDISC-photoz with bias $=0.0007$, $\sigma_{\mathrm{IQR}}=0.0412$, outlier fraction $\eta=0.1191$, CDE loss $=-4.249$, KS $=0.0115$, CvM $=100.6$, and AD $=455.3$; BPZ with bias $=0.0049$, $\sigma_{\mathrm{IQR}}=0.0727$, $\eta=0.1555$, CDE loss $=9.211$, KS $=0.2514$, CvM $=2.716\times 10^4$, and AD $=1.527\times 10^5$; and FlexZBoost with bias $=-0.0024$, $\sigma_{\mathrm{IQR}}=0.0450$, $\eta=0.1371$, CDE loss $=-3.412$, KS $=0.0468$, CvM $=753.6$, and AD $=8184$. The paper further reports that DeepDISC produced the visually most uniform PIT among the three estimators and that its Mixture Density Network exhibited a secondary-peak recovery fraction of $\sim 0.515$ over the full sample [2411.18769]. RAIL has also been used for the first systematic photo-$z$ analysis of Rubin Observatory Data Preview 1. In that application, it ingested DP1 photometry and external reference catalogs, trained and benchmarked eight algorithms, produced per-galaxy PDFs over a common redshift grid $0