Ordered Categorical Encoding

• Ordered categorical encoding is a technique that assigns meaningful numeric values to ordered categorical variables by preserving their intrinsic or inferred order for improved statistical modeling and machine learning.
• It employs methods such as ordinal cumulative logit models, nonparametric order inference with bootstrapped confidence intervals and dominance DAGs, and likelihood-based order selection to optimize encoding.
• Practical implementations, like CatBoost's ordered target statistics, prevent target leakage and enable the use of encoded features in regression, clustering, and predictive analytics.

Ordered categorical encoding refers to techniques that assign meaningful numeric representations to categorical variables whose levels have an intrinsic or inferred order, with the aim of preserving ordinal structure and enabling downstream statistical modeling or machine learning. Ordered categorical encoding is relevant both when orders are inherent (e.g., severity levels, educational attainment) and when orders must be inferred from data or model structure. The methodologies differ substantially depending on whether the categories are interpreted as nominal, ordinal, or when no canonical order is available but an order is statistically advantageous. Current research delineates robust approaches for inference, encoding, and bias mitigation in ordered settings.

1. Statistical Formulations for Ordered Encoding

Ordered categorical encoding is initiated either from explicit ordering assumptions or through inferential procedures on category-real value pairs or model fit. In the setting of multinomial responses $y_i \in \{1, \ldots, J\}$ at covariates $x_i \in \mathbb{R}^d$, the multinomial logistic model parameterizes probabilities either nominally (no order) or ordinally (imposing structural constraints reflecting the presumed order):

• Nominal baseline-category model: For $j=1, \ldots, J-1$,

$$\log \frac{P(Y=j\mid x)}{P(Y=J\mid x)} = x^T\beta_j$$

with unconstrained slopes $\beta_j$.

• Ordinal cumulative logit: For imposed $1 \prec 2 \prec \cdots \prec J$,

$$\log \frac{P(Y\le j\mid x)}{P(Y>j\mid x)} = \alpha_j - x^T\beta$$

or its nonproportional odds form with $\beta_j$.

The selection of appropriate ordering for encoding can be treated as a model selection problem, where each permutation/order of categories corresponds to a candidate model (Wang et al., 2022).

2. Nonparametric Order Inference from Category-Real Pairs

Frameworks such as EDOIF ("Estimation statistics-based Dominant-Distribution Network") address the case where categories $c_i \in \mathcal{C}$ are linked to real-valued outcomes $x_i \in \mathbb{R}$, and the statistical objective is to infer both an ordering $\preceq$ and the magnitude of inter-category differences. In EDOIF:

• Categories are associated with distributions $D'_c$ of observed values.
• Dominant-distribution relation $D_1 \preceq D_2$ is defined such that $P(X_1 \geq E[X_2]) \leq P(X_2 \geq E[X_1])$, operationalized empirically via bootstrap confidence intervals for $\mu_2 - \mu_1$.
• A directed acyclic graph (DAG) $G = (V, E)$ is constructed, with nodes for categories and edges encoding dominance relations, supplemented by confidence intervals for node means and mean differences.
• Numeric codes for categories are assigned by point-estimates $\bar X_c$, enabling both rank and distance preservation (Amornbunchornvej et al., 2019).

This approach is strictly nonparametric, employs estimation statistics (bootstrap for effect sizes and intervals), and supports direct downstream use of codes in regression, clustering, or further modeling.

3. Likelihood-Based Order Selection in Multinomial Models

When orders are not predefined, "Identifying the Most Appropriate Order for Categorical Responses" (Wang et al., 2022) treats the order of categorical levels as a free parameter within the model. For $J$ categories, the protocol is:

• Consider all (or reduced, via equivalence rules) permutations $o \in S_J$.
• For each order, permute labels, fit the chosen logit model via MLE, compute log-likelihood, and selection criteria (AIC, BIC).
• Orders that are indistinguishable by likelihood, such as reverse orders for cumulative logit, are grouped by equivalence (Wang–Yang, Table 1).
• Closed-form transformations link MLEs between equivalent orders, enabling efficient search.
• Final encoding is chosen as the order minimizing AIC/BIC and maximizing predictive accuracy, validated by simulation and cross-validation.

This explicit order selection protocol demonstrates empirically that even absent a "true" underlying order, ordinal encodings selected by AIC/BIC can surpass nominal encodings in predictive performance.

4. Ordered Target Statistics: Avoiding Leakage in Machine Learning

CatBoost (Prokhorenkova et al., 2017) introduces ordered categorical encoding, or "ordered target statistics" (TS), for the encoding of categorical variables in machine learning, particularly gradient boosting:

• For categorical feature $x^i$, and fixed permutation $\pi$ of $[1..n]$, ordered TS for sample $k$ is

$s_k^i(c) = \frac{S_k(c) + a \cdot p}{N_k(c) + b}$

where $S_k(c)$ sums targets $y_j$ for prior samples in the permutation with $x_j^i = c$, and $N_k(c)$ counts them. $a, b$ are prior hyperparameters; $p$ is the global target average.

• At test time, TS is computed over all training data.
• Multiple permutations ($k$) are used to reduce variance; TS for unseen categories returns prior $p$.

Critically, CatBoost's ordered encoding is tightly coupled to "ordered boosting," where each model update M for sample $k$ is based only on data occurring before $k$ in the permutation, strictly preventing target leakage and prediction shift. Recursive support models $M_{j-1}$ (per permutation) are efficiently maintained.

This methodology avoids errant conditional distributions (leakage) and supports unbiased encoding, substantiated by theoretical analysis and practical complexity studies.

5. Algorithmic Protocols and Implementation

Algorithmic steps for ordered categorical encoding span statistical frameworks and ML pipelines:

• EDOIF (R package):

1. Split data into category groups $D'_c$.
2. Compute bootstrapped means and CIs per category (Algorithm 1).
3. Compute pairwise bootstrapped mean differences and CIs (Algorithm 2).
4. Test for order using CI bounds or Mann–Whitney U with FDR correction.
5. Build dominance DAG $G$ and extract total order via topological sort.
6. Encode numerically with $\bar X_c$ or re-zeroed codes.

• CatBoost (ordered TS and boosting):

1. Draw multiple random permutations.
2. Compute TS for each permutation by sequentially aggregating preceding targets.
3. For boosting, maintain prefix models $M_{j-1}$ per permutation.
4. Fit trees to ordered gradients; update only supporting models including sample.

• Order selection (likelihood-AIC/BIC):
  • Prune candidate orders by theoretical equivalence.
  • Fit MLEs for each candidate; compute selection metrics.
  • Transform MLEs between equivalent orders if required.
  • Implement cross-validation for predictive verification.

Complexity is generally efficient for moderate category count and substantial data; EDOIF percentile bootstrap, for example, scales linearly in sample size and bootstrap count.

6. Interpretations, Practical Usage, and Case Studies

The coded value for each category in ordered encoding reflects the point estimate (mean, effect size, or TS) consistent with its position in the inferred or imposed order. Key aspects include:

• Numeric codes represent expected values; differences approximate $E[X_q] - E[X_p]$ with attached confidence intervals.
• Encodings are immediately usable as continuous features in regression, clustering, or other pipelines.
• The dominance DAG (EDOIF) can be used for partial order preservation in network-based downstream tasks.

Empirical examples include:

• Income ordering: Application of EDOIF to 350,000 household incomes (14 careers) revealed ordinal gaps exceeding 25,000 THB, with the dominance network illustrating stratification (Amornbunchornvej et al., 2019).
• Sector ordering in finance: EDOIF applied to NASDAQ sectors displayed dynamic changes in dominance network density across intervals.
• Trauma and police-fatality data: Model selection revealed ordinal encodings improve accuracy even without intrinsic order (Wang et al., 2022).
• CatBoost diagnostics: Ordered TS obviated leakage, maintaining test/train distribution alignment.

7. Connections, Limitations, and Theoretical Implications

Ordered categorical encoding intersects nonparametric statistics, likelihood-based model selection, and ML bias correction. Common themes include:

• Avoidance of bias from data re-use (CatBoost) and accurate uncertainty quantification (EDOIF bootstrap CIs).
• Reduction of order search via symmetry/equivalence in logistic models (Wang et al., 2022).
• Encoding selection as a sensor for model interpretability, predictive accuracy, and uncertainty quantification.

A plausible implication is that, for datasets with non-obvious or fluid ordering, empirical model selection (AIC/BIC-guided or nonparametric inference) yields more interpretable and effective numeric encodings than arbitrary nominal approaches. Where partial order must be preserved, dominance networks or TS statistics become preferable to simple integer mappings.

Limitations include computational scaling for large categories (full search is factorial in $J$ without equivalence pruning) and sensitivity of bootstrapped intervals to sample size/skewness. The question of optimal encoding in multiclass ML absent “true” order remains context-dependent, with ordered encoding often—but not always—affording accuracy improvements.