Z-RMSE: Context-Dependent Metric Analysis

• Z-RMSE is a context-dependent metric that, in photometric redshift estimation, denotes the RMSE of redshift prediction errors computed from z_phot and z_spec.
• In applications like self-organizing maps, Z-RMSE reflects model sensitivity where multiple local minima can appear, affecting the optimization of redshift estimates.
• The literature contrasts Z-RMSE with standard RMSE, potential RMSE skill scores, and Z-residuals, highlighting the importance of careful interpretation and methodological nuances.

“Z-RMSE” does not denote a single standardized quantity across the surveyed arXiv literature. In photometric redshift work, it is most naturally read as the RMSE of redshift prediction errors, computed from $z_{\rm phot}$ and $z_{\rm spec}$. In several other papers that might appear terminologically adjacent, however, the authors explicitly do not define a literal Z-RMSE and instead use standard RMSE, potential RMSE skill scores, Z-residuals, or Z-scores for different diagnostic or inferential purposes [(Way et al., 2012); (Moretti et al., 2024); (Thombre, 2024); (Wu et al., 2023); (Mayer et al., 2024); (Hayase, 2017); (Pereira et al., 2018)]. The term is therefore best understood as context dependent rather than as the name of a unique metric.

1. Terminological scope and usage patterns

The literature separates into three distinct usages. First, in photometric redshift estimation, RMSE is applied directly to redshift errors and thus functions as a de facto “redshift RMSE.” Second, several papers use ordinary RMSE in unrelated settings and explicitly note that no special Z-RMSE, zero-centered RMSE, normalized RMSE, or z-score-based RMSE is defined. Third, some papers introduce Z-based residual or score constructions that are not RMSE at all, even though the notation may invite confusion [(Way et al., 2012); (Moretti et al., 2024); (Thombre, 2024); (Wu et al., 2023); (Mayer et al., 2024); (Hayase, 2017); (Pereira et al., 2018)].

Context	Quantity actually defined	Status of “Z-RMSE”
Photometric redshift estimation	RMSE of redshift errors	Informal reading is plausible
SVR explanation, forecast verification	Standard RMSE or potential RMSE skill score	No literal Z-RMSE
Survival, compound Wishart, zero-adjusted regression	Z-residuals or Z-scores	Not RMSE

A common misconception is that the prefix “Z” always signals a z-score normalization of RMSE. The surveyed papers do not support that interpretation uniformly. In photo-$z$ work, the “z” is the redshift variable. In survival analysis and random matrix theory, the “Z” refers to a normal-quantile transform or to a standardized score. In zero adjusted regression, the relevant object is a residual tailored to zero inflation, not a root-mean-square criterion.

2. Photometric-redshift RMSE in self-organizing maps

A clear early use of redshift RMSE appears in photometric redshift estimation with Self-Organizing Maps. The unsupervised SOM approach takes 5-dimensional SDSS magnitudes,

$(u,g,r,i,z),$

maps them to a discrete Kohonen layer, and then assigns each test object to a neuron whose associated spectroscopic redshifts are averaged to produce the photometric redshift estimate. The paper evaluates regression accuracy through the residual

$\Delta z = z_{\rm phot} - z_{\rm spec},$

with RMSE computed from that residual in the usual way. The evaluated datasets are the SDSS DR7 Main Galaxy Sample, Luminous Red Galaxy sample, Quasar sample, Galaxy Zoo morphology-based subdivisions, and the PHAT0 synthetic data set. For the training-set method, each dataset is split into 89% training, 10% testing, and 1% validation, although validation is used only for ANNz and not for SOM (Way et al., 2012).

The reported SOM RMSE values are $0.02339$ for MGS, $0.02689$ for LRG, $0.02044$ for MGS–ELL, $0.02426$ for MGS–SP, $0.01848$ for LRG–SP, $z_{\rm spec}$0 for LRG–ELL, $z_{\rm spec}$1 for QSO, and $z_{\rm spec}$2 for PHAT0. Outliers are defined by $z_{\rm spec}$3, yielding outlier percentages of $z_{\rm spec}$4 for MGS, $z_{\rm spec}$5 for LRG, $z_{\rm spec}$6 for MGS–ELL, $z_{\rm spec}$7 for MGS–SP, $z_{\rm spec}$8 for LRG–SP, $z_{\rm spec}$9 for LRG–ELL, $z$0 for QSO, and $z$1 for PHAT0. The comparison set includes GPR, ANNz or ANN, linear regression, and quadratic regression; SOM is competitive but is usually not the best-performing method in these tests (Way et al., 2012).

An important technical point is that SOM RMSE is not unique with respect to model configuration. The paper treats the number of Kohonen neurons as a regularization parameter $z$2 and shows that the RMSE curve can be rough and contain multiple local minima. In the LRG–ELL case, two local minima are reported at $z$3 and $z$4. This motivates the conclusion that optimization is sensitive to map size, that traditional gradient-based methods may yield sub-optimal solutions, and that more global strategies such as genetic programming may be preferable. In this specific photo-$z$5 setting, “Z-RMSE” therefore refers not to a new metric but to the RMSE of redshift residuals, together with a nontrivial optimization landscape (Way et al., 2012).

3. Covariate shift, selection effects, and redshift RMSE in StratLearn-z

A more recent photo-$z$6 treatment studies redshift RMSE under covariate shift caused by selection bias in the spectroscopic training set. The setup distinguishes a source set

$z$7

from a target set

$z$8

with the key assumption

$z$9

The point prediction is defined as

$(u,g,r,i,z),$0

and the main RMSE metric is

$(u,g,r,i,z),$1

The method estimates propensity scores $(u,g,r,i,z),$2, uses logistic regression on magnitudes and colors, splits the pooled sample into $(u,g,r,i,z),$3 propensity-score quintile strata, fits two conditional density estimators in each stratum—ker-NN and Series—and combines them through

$(u,g,r,i,z),$4

The four simulated scenarios are no CS, weak CS with $(u,g,r,i,z),$5, mild CS with $(u,g,r,i,z),$6, and strong CS with $(u,g,r,i,z),$7 (Moretti et al., 2024).

The RMSE results show markedly different degradation profiles for StratLearn-z and GPz. StratLearn-z yields $(u,g,r,i,z),$8 under no CS, $(u,g,r,i,z),$9 under weak CS, $\Delta z = z_{\rm phot} - z_{\rm spec},$0 under mild CS, and $\Delta z = z_{\rm phot} - z_{\rm spec},$1 under strong CS. GPz yields $\Delta z = z_{\rm phot} - z_{\rm spec},$2, $\Delta z = z_{\rm phot} - z_{\rm spec},$3, $\Delta z = z_{\rm phot} - z_{\rm spec},$4, and $\Delta z = z_{\rm phot} - z_{\rm spec},$5, respectively. The paper summarizes this as GPz RMSE being nearly doubled from no CS to strong CS, whereas StratLearn-z is only marginally impacted by covariate shift. In the strongest-shift scenario, the ratio $\Delta z = z_{\rm phot} - z_{\rm spec},$6 is described as roughly a factor of 2 improvement in RMSE (Moretti et al., 2024).

Bias and catastrophic-error complements reinforce the RMSE interpretation. StratLearn-z bias ranges from $\Delta z = z_{\rm phot} - z_{\rm spec},$7 to $\Delta z = z_{\rm phot} - z_{\rm spec},$8, while GPz bias increases from $\Delta z = z_{\rm phot} - z_{\rm spec},$9 to $0.02339$0. StratLearn-z keeps FR15 around $0.02339$1 and FR05 around $0.02339$2, whereas GPz degrades more substantially, especially in FR05. The PIT analysis shows a symmetric bump near $0.02339$3 and very few outliers near $0.02339$4 or $0.02339$5 for StratLearn-z, suggesting that predictions are centered around the true redshift while the predictive PDFs are conservative. In this literature, the most precise meaning of “Z-RMSE” is again redshift RMSE, now situated within a covariate-shift correction framework (Moretti et al., 2024).

4. Z-residual diagnostics in shared frailty models

In survival analysis, the relevant Z-based object is the Z-residual rather than any RMSE. The paper considers the semi-parametric shared frailty Cox model

$0.02339$6

equivalently

$0.02339$7

with survival function

$0.02339$8

It extends randomized survival probabilities to define

$0.02339$9

where $0.02689$0, and then introduces the Z-residual

$0.02689$1

Under the true model, these residuals are approximately $0.02689$2. In practice they are computed from coxph output, the Breslow estimator $0.02689$3, fitted $0.02689$4, and estimated frailties $0.02689$5, treating the random effects as fixed effects for residual calculation (Wu et al., 2023).

The diagnostic system has graphical and numerical components. Graphically, the paper recommends QQ plots against the standard normal and plots of Z-residuals against a covariate or the linear predictor, using LOWESS to identify trends. Numerically, it divides a covariate or linear predictor into $0.02689$6 equally spaced intervals, groups the Z-residuals, and tests equality of group means via an ANOVA F-test. Because the residuals depend on randomization, the p-values are also random; repeated residual generation yields replicated p-values $0.02689$7, and the paper reports the conservative upper bound

$0.02689$8

A threshold around $0.02689$9 is suggested as more appropriate for this conservative summary than the conventional $0.02044$0. In simulations, Z-AOV-LP and especially Z-AOV-$0.02044$1 show high power for detecting functional-form misspecification, whereas overall GOF tests such as Z-SW, Z-SF, and CZ-CSF have lower power for that task. In the acute myeloid leukemia application, the key Z-AOV-$0.02044$2 p-values are about $0.02044$3 for the wbc model and $0.02044$4 for the lwbc model, indicating that the log transformation is inappropriate; AIC also favors the wbc model, $0.02044$5 versus $0.02044$6 (Wu et al., 2023).

This usage is conceptually adjacent to “Z-RMSE” only in the weak sense that both are error summaries. Formally, however, the paper defines neither a literal Z-RMSE nor an RMSE-like loss. Its central quantity is a normal-quantile residual combined with an ANOVA-based non-homogeneity test.

5. Other Z-based constructions that are not RMSE

Random matrix theory provides a different Z-based object: the free deterministic equivalent Z-score for compound Wishart models. For

$0.02044$7

with amplified parameter $0.02044$8, the paper defines the FDE mean and variance,

$0.02044$9

and then the statistic

$0.02426$0

Under the condition

$0.02426$1

the statistic converges in distribution to $0.02426$2. Low-order formulas include $0.02426$3, $0.02426$4, $0.02426$5, and an explicit polynomial for $0.02426$6. In the numerical section, the test rejects when $0.02426$7. This is a variance-normalized trace fluctuation, not an RMSE (Hayase, 2017).

Zero adjusted regression introduces yet another distinct object. The proposed residual class is

$0.02426$8

where $0.02426$9 is a residual for the continuous component and $0.01848$0 estimates $0.01848$1. When $0.01848$2 is the quantile residual, the resulting residual $0.01848$3 is called the zero adjusted quantile residual, ZAQR. Theorem 2 states that if $0.01848$4 is standard normal and $0.01848$5 is known, then for

$0.01848$6

the tails satisfy

$0.01848$7

Monte Carlo experiments use 25,000 replications with sample size $0.01848$8, and the application to ENEM 2014 data highlights observation #242 as an instructive case: when its score is hypothetically perturbed from $0.01848$9 to $z_{\rm spec}$00, the residuals become $z_{\rm spec}$01 and $z_{\rm spec}$02, showing the stronger outlier sensitivity of ZAQR. Here again, the Z-based quantity is a residual, not an RMSE (Pereira et al., 2018).

6. Standard RMSE, potential RMSE skill scores, and randomized RMSE

Some adjacent literature is useful precisely because it excludes a specialized Z-RMSE interpretation. In explaining SVR models with interpretable surrogates, RMSE is the fidelity metric between the SVR predictions and the surrogate predictions: $z_{\rm spec}$03 The paper compares decision trees, LIME, and multi-linear regression over 5 UCI datasets in 15 total runs. Decision trees have lower RMSE than LIME in 87% of runs, with paired Wilcoxon $z_{\rm spec}$04, while multi-linear regression has lower RMSE than LIME in 73% of runs, with $z_{\rm spec}$05. The paper explicitly states that it does not define any special RMSE variant such as Z-RMSE, zero-centered RMSE, normalized RMSE, or a z-score-based RMSE (Thombre, 2024).

Forecast verification introduces the “potential RMSE skill score,” denoted $z_{\rm spec}$06, which is likewise not a Z-RMSE. Relative to the CLIPER reference, the ordinary RMSE skill score is

$z_{\rm spec}$07

After assuming MSE-optimal linear calibration, the central closed form is

$z_{\rm spec}$08

The score depends only on the forecast–observation correlation $z_{\rm spec}$09 and the lag-$z_{\rm spec}$10 autocorrelation $z_{\rm spec}$11 of the observations. It is calibration-independent in the limited sense used in the paper, and it measures potential skill after MSE-optimal calibration rather than the actual RMSE skill of the raw forecast. In the photovoltaic example, the actual RMSE skill scores are $z_{\rm spec}$12 for MSE-optimized forecasts and $z_{\rm spec}$13 for MAE-optimized forecasts, whereas the potential RMSE skill scores are $z_{\rm spec}$14 and $z_{\rm spec}$15, respectively (Mayer et al., 2024).

A still different setting is randomized numerical integration in Gaussian Sobolev spaces, where the object of interest is the worst-case root-mean-squared error of a randomized quadrature: $z_{\rm spec}$16 For integer smoothness $z_{\rm spec}$17, the lower bound

$z_{\rm spec}$18

holds for any randomized quadrature with expected cost at most $z_{\rm spec}$19. A randomized trapezoidal rule with random grid size, random shift, and randomized tail nodes attains the upper bound

$z_{\rm spec}$20

for

$z_{\rm spec}$21

The method is unbiased and admits a sample-variance estimator of the MSE. This use of RMSE concerns randomized algorithmic complexity, not any z-transformation (Goda et al., 2022).

Taken together, these strands establish a precise encyclopedia-level conclusion: “Z-RMSE” is not a canonical cross-domain metric. In photometric redshift estimation it is best interpreted as redshift RMSE; in several neighboring literatures the relevant objects are standard RMSE, potential RMSE skill scores, Z-residuals, or Z-scores, each with a distinct mathematical role and no claim of equivalence.