Type-Aware Imputation Metrics

• Type-aware imputation evaluation metrics are quantitative frameworks that tailor assessment based on variable types such as continuous, categorical, binary, and ordinal.
• They combine pointwise error measures like NRMSE and PFC with distributional tests such as KS, Wasserstein, and KL divergence to capture both accuracy and variability.
• These metrics support reliable benchmarking and downstream analysis by preserving marginal distributions, enabling fairness, and guiding model selection across heterogeneous datasets.

Type-aware imputation evaluation metrics are quantitative frameworks and statistical measures designed to assess the quality of missing value imputation with direct sensitivity to the data type of each variable—continuous, categorical, binary, or ordinal. As missing data imputation increasingly supports complex, heterogeneous, and high-dimensional datasets, it has become crucial for evaluation metrics and benchmarking practices to reflect variable-specific characteristics, distributional fidelity, and the nuanced goals of various scientific disciplines. Type-aware metrics provide this granularity by measuring performance in a manner that distinctively respects each variable’s statistical and informational role in the data-generating process.

1. Foundations of Type-Aware Imputation Evaluation

The central principle of type-aware imputation evaluation is differential treatment of continuous and categorical (or other discrete) variables, as these types require distinct measures of imputation quality and may demand preservation of both pointwise accuracy and distributional structure. For classical, mixed-type tabular data, the missForest algorithm exemplifies this principle by using Normalised Root Mean Squared Error (NRMSE) for continuous variables and the Proportion of Falsely Classified entries (PFC) for categorical variables (Stekhoven et al., 2011). This dual evaluation recognizes that minimizing mean squared loss is not appropriate for categorical data, while accuracy alone is insufficient for continuous variables with nontrivial distributions.

Beyond type-specific pointwise errors, modern evaluation frameworks increasingly focus on how well an imputation method preserves the variable’s marginal and conditional distribution, enabling valid downstream inference and learning. Metrics such as distributional divergences (e.g., Kullback–Leibler divergence, Wasserstein distance, and Cramér–von Mises statistics) and proper scoring rules (energy score, density ratio-based I-Scores) capture these aspects in a type-sensitive way. This multipronged assessment is particularly salient in settings such as official statistics, biomedical records, sensor feeds, and time series, where both reproducibility and distributional integrity are critical.

2. Classical Type-Specific Error Metrics

Historically, evaluation of imputation for mixed-type data involved separate metrics for different variable classes. The canonical choices are:

Variable Type	Metric	Formula / Description
Continuous	NRMSE, MSE, MAE	$$\sqrt{\mathrm{mean}((X^\mathrm{true} - X^\mathrm{imp})^2)/\mathrm{var}(X^\mathrm{true})}$$; absolute and squared errors normalized to unit variance (Stekhoven et al., 2011)
Categorical	PFC, misclassification rate	$(\text{\# misclassified})/(\text{\# imputed})$ (Stekhoven et al., 2011)

The NRMSE provides a scale-invariant measure sensitive to absolute deviations, suitable for high-throughput continuous data (e.g., gene expression measures). The PFC directly quantifies the percentage of incorrectly imputed categories, making it appropriate for clinical ratings or genetic polymorphism data.

This type-sensitivity extends to advanced methods, such as out-of-bag (OOB) error estimates in random forests, which can be partitioned by variable type to gauge local imputation reliability without recourse to a test set (Stekhoven et al., 2011).

3. Distributional and Proper Scoring Metrics

A key insight from recent research is that low pointwise error does not guarantee preservation of the variable’s full distribution, especially relevant for inference-driven disciplines. Thus, distribution-sensitive metrics for type-aware settings have been proposed:

• Distribution Distance Metrics:
  • Kolmogorov–Smirnov (KS) statistic and Cramér–von Mises (CM): Compare empirical cumulative distributions of imputed and true data (Thurow et al., 2021).
  • Mallows’ $L^2$ (Wasserstein) distance and KL divergence: Measure global and local shifts in quantile and probability distributions (Thurow et al., 2021).
  • For categorical variables, Cramér’s V or its normalized variant $\kappa$ can capture marginal class distribution similarity (Thurow et al., 2021).

Findings indicate that imputation methods minimizing NRMSE or PFC may nevertheless induce large distributional deviations, while methods with modest pointwise error (e.g., MICE with normal model) may yield nearly indistinguishable marginals (Thurow et al., 2021).

• Proper Scoring Rules (I-Scores, Energy Scores):
  • Imputation Scores (I-Scores) focus on assessing whether the imputed values reproduce the joint and conditional distributions of the true data. For example, the Density Ratio I-Score penalizes the KL divergence between projected densities of true and imputed data, rewarding methods that sample from the correct conditional distributions (Näf et al., 2021).
  • More recently, energy-based I-Scores have been developed, which use the energy distance between multiply-imputed samples and observed data (under a missingness adherence condition) as a ranking criterion (Näf et al., 15 Jul 2025).

These approaches are type-agnostic at the algorithmic level but naturally become type-aware through their use of appropriate distances (e.g., $\ell_2$ for continuous, Hamming for categorical), and through conditional modeling structures that mirror the variable’s statistical nature.

4. Holistic and Fairness-Oriented Metrics

Type-aware evaluation is further extended in methods that holistically account for the impact of imputation on downstream predictive modeling, fairness, and stability:

• Fairness and Group-Specific Metrics:
  • Shades-of-Null introduces assessment schemas where classical imputation quality (RMSE, F1 for cat.) is disaggregated by social group or feature domain, calculating error or divergence differences, and further tracks fairness in classifier outcomes after imputation (Khan et al., 11 Sep 2024). This recognizes that type and group membership can interact, so type-aware metrics may require stratification across privileged and disadvantaged groups.
• Explainable Global Metrics:
  • xGEWFI weighs distributional error (e.g., KS statistic) for each feature by its RF-derived importance, aggregating a global type-sensitive error measure that can be more meaningful when feature relevance is highly non-uniform. This approach is particularly useful when critical features are disproportionately affected by imputation error (Dessureault et al., 2022).
• Diagnostic Toolchains:
  • ITI-IQA integrates per-type statistical (KS, $\chi^2$) tests and imputer comparisons within an interactive pipeline, ensuring that type-specific error, completeness, and bias-avoidance drive the selection and acceptance of imputed features (Pons-Suñer et al., 16 Jul 2024).

This body of work recognizes that, particularly for heterogeneous datasets, evaluation must go beyond marginal metrics by considering the practical and ethical consequences of imputation choices.

5. Benchmarking, Ranking, and Practical Considerations

Standardizing the benchmarking and ranking of imputation methods in type-aware contexts requires:

• Benchmark Platforms:
  • TSI-Bench exemplifies type-aware benchmarking for time series by using masked MAE/MSE/MRE restricted to imputed entries, ensuring that variable-level errors are properly normalized regardless of the missing pattern (block, point, sequence) (Du et al., 18 Jun 2024). This masking naturally adapts to variable type by selective evaluation.
• Ranking and Model Selection:
  • Aggregated ranking methods, such as those using Friedman–Nemenyi post-hoc testing, allow multi-variable and multi-type averaged performance comparisons across complex scenarios, offering guidance on method selection when performance varies by variable type and missingness pattern (Sangari et al., 2021, Nair et al., 2013).
• Variable-Specific Handling:
  • ITI-IQA and similar tools objectively choose the best imputer per variable based on a combined score of completeness and type-normalized error, and can filter out variables where no imputer yields acceptable performance (Pons-Suñer et al., 16 Jul 2024).
• Type-Aware Evaluation in Absence of Ground Truth:
  • In time series and other incomplete domains, the use of distribution-based metrics (Wasserstein, Jensen–Shannon) to compare imputed and pre-gap data enables type-sensitive evaluation when ground truth is unavailable, provided the comparison metric respects variable type (e.g., appropriate distance function for the type) (Farjallah et al., 26 Feb 2025).

6. Emerging Directions and Implications

Current trends demonstrate an increasing sophistication in type-aware evaluation:

• The adoption of mixed-metric approaches (e.g., combining pointwise error and distributional distances (Llera et al., 2022)).
• Use of model-centric metrics (e.g., impact on downstream classification accuracy/F1 by variable type, as in LLM-based imputation (Srinivasan et al., 4 Jun 2025)).
• Adjustment for context and semantics (e.g., combining feature context via embeddings with empirical missingness structure for tabular imputation (Gorla et al., 2 Jun 2025)).
• Theoretical developments, such as proper scoring rules that are robust under realistic missingness assumptions (CIMAR) and can be meaningfully aggregated across variable types (Näf et al., 15 Jul 2025).

This growing toolkit promotes more reliable and context-appropriate imputation in scientific practice, supports the needs of fields with complex, heterogeneous, and sensitive data, and facilitates meta-analyses and benchmarking efforts that are grounded in distributional and inferential validity as opposed to purely pointwise agreements.

7. Summary Table of Principal Type-Aware Imputation Evaluation Metrics

Metric or Method	Variable Type Sensitivity	Distributional Assessment	Notable Features
NRMSE, PFC	Continuous, categorical	No	Canonical, variable-appropriate, widely used (Stekhoven et al., 2011)
KS, Cramér–von Mises, WD, KL	Both (with type-adapted versions)	Yes	Compare empirical/estimated distributions (Thurow et al., 2021, Farjallah et al., 26 Feb 2025)
I-Score (DR, Energy)	Both (via kernel, sample, norm)	Yes	Proper scoring, rewards conditional distribution fidelity (Näf et al., 2021, Näf et al., 15 Jul 2025)
xGEWFI	Both	Yes	Weights per-feature error by feature importance (Dessureault et al., 2022)
Masked MAE/MSE/MRE	Both (via per-feature aggregation)	No (unless combined)	Type-aware via selective evaluation and normalization (Du et al., 18 Jun 2024)
Downstream Model Metrics	Both	Indirect	Accuracy/F1 by variable type, fairness, stability (Khan et al., 11 Sep 2024, Srinivasan et al., 4 Jun 2025)

A plausible implication is that, as data science moves toward ever more heterogeneous and sensitive settings, the next generation of type-aware imputation evaluation will blend per-variable statistical accuracy, distributional fidelity, application-centric loss, and fairness/stability in a unified, rigorously justified framework.