RAIL: Redshift Assessment Infrastructure Layers

• RAIL is an open source Python library for probabilistic photometric redshift estimation, organizing work into modular layers for forward modeling, estimation, and evaluation.
• Its layered architecture supports complete workflows, enabling stress-testing of LSST-like systematics and production-scale calibration of redshift distributions.
• RAIL integrates diverse estimators and diagnostics to produce and validate per-galaxy PDFs, addressing challenges from training-set imperfections and heterogeneous survey data.

Redshift Assessment Infrastructure Layers (RAIL) is an open source Python library for at-scale probabilistic photometric redshift estimation that organizes end-to-end workflows into modular layers for forward modeling, estimator training and inference, and evaluation. Initiated by the LSST Dark Energy Science Collaboration with contributions from the LSST Interdisciplinary Network for Collaboration and Computing Frameworks team, RAIL was designed for Rubin-era photo-$z$ stress-testing and production, but its engines, degraders, estimators, and metrics are survey-agnostic by construction. A defining capability is the coupling of a unified estimator API to forward models that can generate realistic photometry and, in PZFlow-based workflows, expose true conditional posteriors $p(z\mid\mathrm{photometry})$ for galaxy-by-galaxy PDF assessment (Team et al., 5 May 2025).

1. Scientific motivation and scope

RAIL emerged from the requirements of Rubin Observatory extragalactic science, where most analyses must infer distance information for galaxies that lack spectroscopic redshifts. The v1 overview states that LSST will produce deep ugrizy imaging over $\sim 18{,}000\,\mathrm{deg}^2$ and a catalog of $\sim 20$ billion galaxies down to $i \approx 26.4$, while the vast majority of these galaxies will not have spectroscopic redshifts. In this regime, redshift uncertainties are complex and often non-Gaussian, so per-galaxy photo-$z$ probability density functions are the preferred representation rather than point estimates with Gaussian errors (Team et al., 5 May 2025).

The library’s design responds directly to limitations identified in earlier DESC experiments. Those experiments showed disagreement among methods even with identical inputs, the possibility that some commonly used metrics can be gamed or are uninformative, and the need for probabilistic forward models with access to true conditional PDFs. RAIL therefore targets three linked tasks: at-scale probabilistic photo-$z$ estimation with a unified API, stress-testing under realistic LSST-like systematics, and optimization and validation of both per-galaxy PDFs and ensemble redshift distributions $N(z)$ for science cases including cosmic shear, large-scale structure, clusters, and supernova cosmology (Team et al., 5 May 2025).

A recurring misconception is that RAIL is only a benchmarking harness. The software and its applications show a broader scope: it is simultaneously a stress-testing framework, a production environment, and a calibration-and-evaluation toolkit. This is explicit in the DP1 analysis, where RAIL is used to ingest and standardize Rubin photometry, train and benchmark multiple algorithms, produce per-galaxy PDFs and ensemble $n(z)$, and interoperate with Rubin Data Management, the Photo-$z$ Server, and LSDB (Zhang et al., 8 Oct 2025).

2. Layered software architecture

RAIL v1 is organized into three major subpackages, each corresponding to a distinct layer in the photo-$p(z\mid\mathrm{photometry})$0 workflow (Team et al., 5 May 2025).

Subpackage	Primary role	Representative components
RAIL.creation	Forward modeling of truth catalogs and realistic photometry	FSPS, DSPS, PZFlow, degraders
RAIL.estimation	Per-galaxy PDF estimation, ensemble summarization, tomographic classification	Informer, Estimator, Summarizer, Classifier
RAIL.evaluation	Metrics and diagnostics for PDFs, point estimates, and tomographic overlap	PIT, KS, CvM, AD, CDELoss, bias, scatter

The core execution model is built atop ceci, with stages as atomic operations and pipelines as directed acyclic graphs with provenance. Configuration is split across YAML files: a ceci pipeline configuration for stage classes, inputs, outputs, and parallel parameters, and a RAIL YAML for stage-specific parameters. Interactive RailStage wrappers expose inform(), estimate(), and summarize() for notebook use, while run() supports reproducible batch execution. MPI parallelization and chunked I/O are central to the design for scale (Team et al., 5 May 2025).

RAIL’s internal data model is based on DataHandle abstractions in rail.core.data. The v1 description identifies TableHandle for tabular catalogs in numpy, pandas, pyarrow, or Astropy; QPHandle for qp.Ensemble objects containing 1D PDFs; and ModelHandle for serialized models. These handles support chunked reading and writing for memory efficiency. The same standardization allows drop-in replacement of algorithms wrapped as subclasses of common stage types such as CatInformer, CatEstimator, PZInformer, PZSummarizer, PZClassifier, and SZPZSummarizer (Team et al., 5 May 2025).

The architecture has also been adopted in operational settings. In the DP1 work, RAIL_projects generates ceci pipeline configurations from named “flavors,” while versioned configurations in rail_project_config preserve training cuts, algorithms, parameters, and target fields. This establishes a reproducible bridge between the research software stack and Rubin production-style workflows (Zhang et al., 8 Oct 2025).

3. Forward modeling, degraders, and stress-testing

The RAIL.creation layer provides engines for generating photometry and degraders for imposing realistic systematics. The v1 release describes three principal engines. rail_fsps generates rest-frame galaxy SEDs including stars, gas, dust, and AGN from supplied physical parameters and computes apparent AB magnitudes through user-specified filters. rail_dsps provides a JAX-based differentiable SPS path. rail_pzflow trains a normalizing-flow generative model of the joint distribution of colors and redshift and can both sample new galaxies and provide access to true conditional PDFs $p(z\mid\mathrm{photometry})$1 (Team et al., 5 May 2025).

The degradation layer encodes observational and spectroscopic imperfections. Documented examples include LSSTErrorModel, which wraps PhotErr to add heteroscedastic Gaussian magnitude noise; ObservingConditionDegrader, which assigns per-galaxy observing-condition maps; QuantityCut for magnitude, color, or SNR selection; LineConfusion for emission-line misidentification; SpectroscopicSelectors and GridSelection for survey-specific targeting and success rates; SOMSpecSelector for self-organizing-map-based matched subsets; and a blending degrader that constructs unrecognized blends via Friends-of-Friends on sky position (Team et al., 5 May 2025).

A particularly detailed demonstration appears in the study of spectroscopic training-set imperfection, where RAIL served as the backbone for building, degrading, and running controlled photo-$p(z\mid\mathrm{photometry})$2 experiments. The workflow followed the sequence Creation $p(z\mid\mathrm{photometry})$3 Degradation $p(z\mid\mathrm{photometry})$4 Informer $p(z\mid\mathrm{photometry})$5 Estimator. In that study, the Creation layer used a PZFlow normalizing flow trained on Rubin–Roman OpenUniverse Tile 10307 to emulate the joint distribution $p(z\mid\mathrm{photometry})$6, and the FlowPosterior stage evaluated per-galaxy true posteriors $p(z\mid\mathrm{photometry})$7 by conditioning on individual galaxy magnitudes. Degraders then applied an LSST-like photometric error model at 10-year depth, inverse-redshift incompleteness, or survey-specific spectroscopic selection functions before estimator training and inference (Crafford et al., 15 Jan 2026).

The spectroscopic stress tests in that work were deliberately heterogeneous. The survey-based degraders emulated pseudo-BOSS, GAMA, the HSC training union, zCOSMOS-bright, pseudo-DEEP2, and VVDSf02, with surviving training-set sizes ranging from $p(z\mid\mathrm{photometry})$8 galaxies for pseudo-BOSS to $p(z\mid\mathrm{photometry})$9 for VVDSf02. The same paper also defined an inverse redshift incompleteness degrader with survival probability

$\sim 18{,}000\,\mathrm{deg}^2$0

tested at pivot values $\sim 18{,}000\,\mathrm{deg}^2$1, while the v1 documentation describes InvRedshiftIncompleteness as keeping low-$\sim 18{,}000\,\mathrm{deg}^2$2 galaxies and randomly removing high-$\sim 18{,}000\,\mathrm{deg}^2$3 galaxies with

$\sim 18{,}000\,\mathrm{deg}^2$4

Both descriptions underscore the same architectural point: RAIL treats training-set non-representativeness as a configurable degradation problem rather than an external nuisance (Crafford et al., 15 Jan 2026).

4. Estimators, summarizers, and image-based extensions

The RAIL.estimation layer exposes a unified API across template-fitting, machine-learning, hybrid, and image-based methods. The v1 release lists BPZ and LePHARE as template-fitting estimators; CMNN, DNF, FlexZBoost, [GPz](https://www.emergentmind.com/topics/gaussian-processes-for-photometric-redshifts-gpz), scikit-learn wrappers for k-nearest neighbors and an MLPRegressor, and a PZFlow estimator as machine-learning methods; Delight as a hybrid Gaussian-process template-learning method; and DeepDISC as an image-based estimator. Ensemble-level products are handled by summarizers such as naive stacking, variational inference stacking, NZDir, SOM-based summarizers, and Yet Another Wizz, while tomographic assignments can be produced by Uniform_binning, EqualCount, and a RandomForestClassifier (Team et al., 5 May 2025).

This common interface is consequential because the wrapped algorithms have markedly different inductive biases. In the training-imperfection study, TrainZ was used as a control that assigns each test galaxy the same PDF equal to the training-set redshift distribution; CMNN selected variable-size neighborhoods in color space and returned unimodal Gaussian PDFs; GPz returned single-Gaussian PDFs whose variance combined aleatoric and epistemic components; FlexZBoost estimated nonparametric conditional densities via a basis expansion using training neighbors in color space; and PZFlow conditioned a learned joint flow $\sim 18{,}000\,\mathrm{deg}^2$5 on test magnitudes and could refuse to produce an estimate when conditioning was out of support (Crafford et al., 15 Jan 2026).

RAIL’s extensibility is also visible in rail_deepdisc, which integrates the image-based DeepDISC-photoz model as an Estimator stage consuming six-band LSST ugrizy coadded images rather than catalogs. In that integration, DeepDISC is a detectron2-based instance-segmentation framework with an added redshift Region of Interest head. The redshift head is a Mixture Density Network with $\sim 18{,}000\,\mathrm{deg}^2$6 Gaussian components and outputs a per-object redshift PDF on a configured common $\sim 18{,}000\,\mathrm{deg}^2$7-grid, along with point estimates such as the mean and mode and summary intervals derived from the CDF (Merz et al., 2024).

The DeepDISC integration also illustrates the breadth of the RAIL abstraction. A typical pipeline can couple rail.data for reading LSST DC2 coadds or cutouts and loading truth redshifts, rail.estimation for training or informing DeepDISC and catalog-based estimators such as rail_bpz and rail_flexzboost, and rail.evaluation for standardized metrics. YAML configuration files define model checkpoint paths, image channels, ROI-head settings, training schedules, and the common redshift grid, allowing image-based and catalog-based methods to be evaluated on equal footing (Merz et al., 2024).

5. Evaluation methodology and metric design

RAIL’s evaluation layer formalizes three complementary families of diagnostics: PDF-to-PDF, PDF-to-point, and point-to-point metrics. The v1 documentation lists Cramér–von Mises, Kolmogorov–Smirnov, RMSE over densities, Kullback–Leibler divergence, and Anderson–Darling as PDF-to-PDF measures; Conditional Density Estimation loss, Probability Integral Transform, and Brier score as PDF-to-point measures; and residual-based bias, scatter, and outlier statistics as point-to-point measures. For point estimates, the normalized residual is

$\sim 18{,}000\,\mathrm{deg}^2$8

with scatter often summarized through $\sim 18{,}000\,\mathrm{deg}^2$9 or $\sim 20$0, and outliers defined in v1 by $\sim 20$1 (Team et al., 5 May 2025).

The PIT occupies a central place in RAIL’s calibration philosophy. In v1,

$\sim 20$2

and well-calibrated PDFs should yield a uniform PIT distribution. The evaluation module therefore supports PIT histograms, PIT–QQ plots against $\sim 20$3, and application of CvM, KS, and AD directly to the PIT distribution. This emphasis on calibration distinguishes RAIL from workflows that reduce photo-$\sim 20$4 quality to point-estimate residuals alone (Team et al., 5 May 2025).

The 2026 training-set-imperfection analysis sharpened this metric perspective by comparing each estimator’s $\sim 20$5 to a per-galaxy true posterior $\sim 20$6 obtained from FlowPosterior. Metrics were computed per galaxy and then aggregated as distributions, often through quantile curves. In that setting, the Kullback–Leibler divergence, Wasserstein distance, and PIT were found to be particularly informative for diagnosing training-set imperfection and algorithmic behavior, whereas RMSE and KS were less discriminating except in catastrophic cases. The same study notes that these metrics were implemented outside the RAIL version used there with numpy, scipy, sklearn, and qp, although some are available in later RAIL versions including v1 (Crafford et al., 15 Jan 2026).

The paper also gives precise diagnostic interpretations for PIT morphology. “Side spikes” indicate too-narrow PDFs or offset means, while “middle bumps” indicate over-broad PDFs or symmetric bimodality around the truth. Under spectroscopic degradations, CMNN and FlexZBoost showed roughly symmetric side spikes corresponding to high/low catastrophes, whereas GPz and TrainZ showed pronounced right-side spikes in some degraded cases. This metric behavior is not incidental; it is tied to RAIL’s ability to evaluate estimated PDFs against true conditional posteriors rather than only against scalar redshift labels (Crafford et al., 15 Jan 2026).

6. Empirical applications, deployment, and limitations

RAIL has been exercised in both simulation-driven and on-sky settings. In the DeepDISC-photoz study on LSST DESC DC2 simulations, RAIL was used to compare DeepDISC-photoz against BPZ and FlexZBoost on a matched test set of $\sim 20$7 objects. Using the PDF mode as the point estimate, the reported global metrics were: DeepDISC-photoz with bias $\sim 20$8, $\sim 20$9, outlier fraction $i \approx 26.4$0, CDE loss $i \approx 26.4$1, KS $i \approx 26.4$2, CvM $i \approx 26.4$3, and AD $i \approx 26.4$4; BPZ with bias $i \approx 26.4$5, $i \approx 26.4$6, $i \approx 26.4$7, CDE loss $i \approx 26.4$8, KS $i \approx 26.4$9, CvM $z$0, and AD $z$1; and FlexZBoost with bias $z$2, $z$3, $z$4, CDE loss $z$5, KS $z$6, CvM $z$7, and AD $z$8. The paper further reports that DeepDISC produced the visually most uniform PIT among the three estimators and that its Mixture Density Network exhibited a secondary-peak recovery fraction of $z$9 over the full sample (Merz et al., 2024).

RAIL has also been used for the first systematic photo-$z$0 analysis of Rubin Observatory Data Preview 1. In that application, it ingested DP1 photometry and external reference catalogs, trained and benchmarked eight algorithms, produced per-galaxy PDFs over a common redshift grid $z$1, and validated the results using spectroscopic and grism redshifts in ECDFS and DESI DR1 in the SV_38_7 field. Across algorithms in the ECDFS six-band test set, the paper reports per-galaxy scatter $z$2 and outlier fractions $z$3 for the high-SNR sample, with machine-learning methods satisfying the LSST Y1 per-galaxy requirements of overall bias $z$4 and scatter $z$5. FlexZBoost, for example, achieved bias $z$6, $z$7, outlier fraction $z$8, and binning accuracy $z$9; BPZ yielded bias $N(z)$0, $N(z)$1, outlier fraction $N(z)$2, and binning accuracy $N(z)$3 (Zhang et al., 8 Oct 2025).

These applications also establish several substantive findings about RAIL-enabled experimentation. The training-imperfection study concluded that inverse redshift incompleteness alone is too simple to represent realistic spectroscopic selection, that training-set representativity in photometry–redshift coverage matters more than size alone, and that multiple complementary metrics are essential because reliance on a single metric can mislead. It also recommended incorporating algorithm-specific failure signaling into pipelines, citing GPz’s extreme variances, FlexZBoost multimodality, and PZFlow refusal to estimate as examples of informative failure indicators under severe covariate shift (Crafford et al., 15 Jan 2026).

Operationally, RAIL has been integrated with Rubin’s Butler, the Photo-$N(z)$4 Server, and LSDB in the DP1 workflow, and the DeepDISC work emphasizes pipeline checkpointing, common $N(z)$5-grid serialization, and chunked or streamed metric computation for LSST-scale runs. At the same time, the documented limitations are substantial. The v1 paper notes that CRPS and some calibration metrics such as explicit ECE are standard but not explicitly listed as built-ins. The training-imperfection study states that algorithm hyperparameters were not tuned, the tested algorithm set was non-exhaustive, blending and image-level systematics were not modeled, and the truth-emulating flow was trained on $N(z)$6 and may suffer sample variance. The DP1 analysis adds limitations from high-$N(z)$7 training scarcity, non-detections and upper limits, possible LSSTComCam $N(z)$8-band red leak, and training-set sample variance when models trained in ECDFS are applied to SV_38_7 (Team et al., 5 May 2025).

Taken together, these studies define RAIL less as a single estimator than as a structured environment for producing, perturbing, comparing, and validating probabilistic redshift inferences. Its encyclopedic significance lies in the way it unifies forward modeling, heterogeneous estimator interfaces, calibration-aware evaluation, and operational deployment within one reproducible software stack, while preserving enough modularity to accommodate new algorithms, new degraders, and new survey regimes (Team et al., 5 May 2025).