Directed Ordinal Diffusion Regularization (D-ODR)

• The paper’s main contribution is introducing D-ODR, a progression-aware regularizer that enforces an irreversible ordering in diabetic retinopathy grading.
• The method constructs a directed graph over feature embeddings and applies multi-scale diffusion to penalize only clinically valid score inversions.
• Empirical evaluations show that D-ODR significantly improves Quadratic Weighted Kappa and macro F1 scores across multiple DR datasets using state‐of‐the‐art backbones.

Directed Ordinal Diffusion Regularization (D-ODR) is a progression-aware regularization method for diabetic retinopathy (DR) grading that encodes disease severity as a forward-only directed flow in feature space rather than as a static symmetric ranking problem. Introduced for DR, where severity follows a clinically irreversible trajectory from mild to severe stages, the method constructs a progression-constrained directed graph within each mini-batch, performs multi-scale diffusion on that graph, and penalizes score inversions only along clinically valid worsening paths. In the formulation reported in "Directed Ordinal Diffusion Regularization for Progression-Aware Diabetic Retinopathy Grading," D-ODR is combined with a scalar severity regressor and an MSE base loss, improves Quadratic Weighted Kappa (QWK) and macro F1 across APTOS, Messidor, DeepDRiD, and DDR, and adds no inference-time overhead because the regularization module is discarded after training (Chen et al., 25 Feb 2026).

1. Clinical asymmetry and the problem setting

D-ODR is motivated by the claim that DR progresses as a continuous and irreversible deterioration of the retina, with clinically relevant transitions that are asymmetric: a case may remain stable or worsen, but reverse progression is rare under standard screening. The paper argues that conventional ordinal regression methods—including regression-to-integers, cumulative link, rank-consistent classification, and pairwise ranking—typically impose symmetric order constraints, penalizing violations in both directions and treating pairwise relations such as mild-to-severe and severe-to-mild as equally constraining. In this view, symmetric supervision can propagate contradictory gradients, obscure severity semantics, and distort the learned geometry of the representation (Chen et al., 25 Feb 2026).

Two failure modes are explicitly identified. The first is biologically inconsistent reverse transitions, meaning local neighborhoods or paths in feature space or score space that imply movement from a more severe stage toward a milder one. The second is score inversion: for samples $i$ and $j$ with labels $y_i \le y_j$, the model should satisfy $s_i \le s_j$; any case with $s_i > s_j$ violates the progression order. D-ODR addresses these phenomena by introducing a forward-only, progression-aware inductive bias, so that constraints are applied only along clinically plausible worsening directions.

This framing distinguishes D-ODR from generic ordinal learning. The method does not merely require ordered labels; it requires that the geometry of the latent space and the propagation of regularization respect an irreversible clinical trajectory. A plausible implication is that D-ODR is best understood not as a generic ranking loss, but as a structured regularizer for settings in which the label taxonomy itself carries a direction of permissible evolution.

2. Directed graph construction and multi-scale diffusion

Within each training mini-batch, D-ODR constructs a directed weighted graph over feature embeddings. Let $z_i = g_\theta(x_i) \in \mathbb{R}^d$ denote the embedding of image $x_i$ with label $y_i \in \{0,1,2,3,4\}$, and let $s_i = h_\phi(z_i) \in \mathbb{R}$ denote the predicted continuous severity score. Local neighborhoods are defined using mutual $k$-nearest neighbors $j$0, which the paper uses to reduce spurious edges. Edge weights are computed with a locally scaled Gaussian kernel,

$j$1

where $j$2 is a local scale based on distances to neighbors.

Directionality is introduced through a progression mask,

$j$3

which retains edges only from sample $j$4 to sample $j$5 when $j$6 is equally or more severe than $j$7. Self-loops are then added,

$j$8

to avoid zero row sums after masking. If $j$9 is the diagonal out-degree matrix with $y_i \le y_j$0, the directed random-walk operator is

$y_i \le y_j$1

Because reverse edges are removed by the mask, $y_i \le y_j$2 defines a forward-only diffusion process (Chen et al., 25 Feb 2026).

D-ODR uses multi-scale diffusion to move beyond one-hop neighborhoods. For $y_i \le y_j$3,

$y_i \le y_j$4

so that any positive entry $y_i \le y_j$5 indicates that $y_i \le y_j$6 is reachable from $y_i \le y_j$7 along a valid forward path. The powers of $y_i \le y_j$8 therefore encode multi-step progression structure, with one-hop, two-hop, and three-hop connectivity contributing distinct scales of regularization. This design makes the notion of “ordinal consistency” path-dependent rather than merely pairwise.

3. Loss function, optimization, and implementation

The core regularizer penalizes score inversions only on forward-reachable pairs. For diffusion scales $y_i \le y_j$9, the D-ODR loss is

$s_i \le s_j$0

The $s_i \le s_j$1 term activates only when $s_i \le s_j$2, and the factor $s_i \le s_j$3 downweights longer-range constraints. The base supervision is mean squared error against ordinal labels,

$s_i \le s_j$4

and the total objective is

$s_i \le s_j$5

with default $s_i \le s_j$6, reported as robust across $s_i \le s_j$7 to $s_i \le s_j$8 (Chen et al., 25 Feb 2026).

The predictor is $s_i \le s_j$9. The encoder $s_i > s_j$0 is either ViT-B/16 or RETFound, and the head $s_i > s_j$1 maps the embedding to a scalar severity score. D-ODR is attached only during training as a batch-wise regularizer built from $s_i > s_j$2 and $s_i > s_j$3. At inference, only $s_i > s_j$4 is computed, and the D-ODR module is discarded.

The per-mini-batch algorithm consists of six stages: forward pass; neighborhood graph construction via pairwise distances and mutual $s_i > s_j$5-nearest neighbors; transition matrix construction through row normalization; multi-scale diffusion by computing $s_i > s_j$6 for $s_i > s_j$7; accumulation of inversion penalties at each scale; and backpropagation through the combined loss. The computational bottleneck is the $s_i > s_j$8 pairwise distance computation, while the matrix multiplications for $s_i > s_j$9 are reported as negligible relative to the backbone forward pass at batch size $z_i = g_\theta(x_i) \in \mathbb{R}^d$0. Mutual $z_i = g_\theta(x_i) \in \mathbb{R}^d$1-nearest neighbors sparsify $z_i = g_\theta(x_i) \in \mathbb{R}^d$2, self-loops stabilize normalization, and the small diffusion horizon $z_i = g_\theta(x_i) \in \mathbb{R}^d$3 limits overhead.

The default training configuration uses 4× NVIDIA RTX A6000, batch size 128, initial learning rate $z_i = g_\theta(x_i) \in \mathbb{R}^d$4 with linear decay to zero, ImageNet normalization, and random horizontal flip as the only augmentation. Each experiment is repeated three times, and checkpoint selection uses a composite validation criterion based on min–max normalized QWK plus F1. The default graph hyperparameters are mutual $z_i = g_\theta(x_i) \in \mathbb{R}^d$5-nearest neighbors with $z_i = g_\theta(x_i) \in \mathbb{R}^d$6, locally scaled Gaussian weights, row normalization after adding self-loops, and diffusion scales $z_i = g_\theta(x_i) \in \mathbb{R}^d$7 with weights $z_i = g_\theta(x_i) \in \mathbb{R}^d$8.

4. Empirical evaluation and comparative performance

The method is evaluated on APTOS 2019, Messidor, Messidor-2, DeepDRiD (DeepDR), and DDR, with splits and preprocessing following GDRBench protocols for fair cross-dataset comparison. Reported metrics are QWK and macro F1. Against CORN, POEs, GOL, Ord2Seq, CLIP-DR, and AOR-DR, D-ODR attains the highest QWK and F1 on all four datasets with the RETFound backbone, and near-uniform gains with ViT-B/16. The paper attributes these improvements to forward-only diffusion, which suppresses contradictory gradients and enforces monotonicity along clinically plausible transitions (Chen et al., 25 Feb 2026).

With ViT-B/16, D-ODR improves over the baseline across all reported datasets.

Dataset	ViT-B/16 baseline QWK/F1	D-ODR QWK/F1
APTOS	87.6±1.1 / 65.8±1.7	89.8±0.8 / 67.0±2.0
Messidor	57.4±2.3 / 51.2±3.8	68.2±1.4 / 53.0±2.4
DeepDR	74.4±0.5 / 56.6±0.4	80.1±0.7 / 57.4±1.5
DDR	75.5±1.0 / 54.2±0.3	76.8±0.3 / 55.7±0.5

With RETFound, the reported gains are larger on several datasets and are described as state of the art.

Dataset	RETFound baseline QWK/F1	D-ODR QWK/F1
APTOS	92.1±0.2 / 72.3±0.5	93.2±0.3 / 73.3±0.8
Messidor	81.2±0.8 / 70.5±0.4	83.9±0.5 / 70.9±1.8
DeepDR	81.6±1.2 / 62.8±0.5	85.7±0.1 / 64.7±0.1
DDR	85.5±0.5 / 60.2±1.7	88.2±0.1 / 64.6±1.5

Ablation results are organized around three components. Removing directional graph construction (“w/o Dir”) reverts to symmetric smoothing and degrades performance, which the paper interprets as evidence that irreversibility is important. Reversing or dropping the forward constraint (“w/o FC”) destabilizes training and produces large F1 variance, indicating that the sign and direction of the constraint matter critically. Removing multi-scale diffusion (“w/o MSD”) reduces gains by limiting regularization to one-hop neighborhoods, suggesting that multi-step reachability improves transitivity.

Sensitivity analyses report that $z_i = g_\theta(x_i) \in \mathbb{R}^d$9 is robust in $x_i$0, neighborhood size $x_i$1 between 15 and 35 yields stable results, and excessively large batch sizes such as 256 may degrade performance on Messidor and DeepDR, likely because local manifold structure becomes diluted.

5. Representation geometry and interpretability

The qualitative analysis focuses on latent geometry and score distributions. In UMAP visualizations, the baseline ViT-B/16 is described as showing fragmented clusters with entanglement across grades, missing the continuous etiology of DR. Under D-ODR, the embedding is reported to become a contiguous, unidirectional manifold that unfolds from healthy to advanced stages, consistent with monotone progression (Chen et al., 25 Feb 2026).

The score-distribution analysis makes a parallel claim in the one-dimensional prediction space. Baseline predictions exhibit high intra-class variance and inter-class overlap near boundaries, whereas D-ODR produces more orderly severity intervals with medians increasing more consistently across classes and reduced overlap for non-consecutive grades. The paper interprets this as evidence of fewer reverse transitions.

The central interpretive mechanism is explicit: D-ODR penalizes $x_i$2 only when $x_i$3 can reach $x_i$4 through $x_i$5, that is, only along forward-reachable paths. In that sense, the regularizer targets biologically inconsistent reverse transitions directly rather than indirectly through symmetric smoothing or generic ordinal calibration. This suggests that the method reshapes both the embedding manifold and the scalar score field so that transitivity follows clinically permissible disease evolution.

6. Scope, limitations, and extensions

The paper states several limitations. First, D-ODR assumes a strictly forward-only progression pattern in the label taxonomy. Tasks with genuine remission or bidirectional transitions would require relaxing the mask $x_i$6 to encode permitted backward edges or probabilistic directionality. Second, label noise and mis-staging can cause masking and diffusion to propagate incorrect constraints; mutual $x_i$7-nearest neighbors and locally scaled kernels mitigate this but do not eliminate it. Third, diffusion is batch-local, so very large or very small batches can dilute or fragment local manifolds. The paper notes that cross-batch memory or approximate nearest neighbors over larger pools could further stabilize the graph. Fourth, the method is presented for DR staging with discrete grades $x_i$8 through $x_i$9, and adaptation to other taxonomies would require changing the directional mask to reflect the clinically allowed transitions (Chen et al., 25 Feb 2026).

Several extensions are proposed. The method is suggested as applicable to other progressive diseases, including AMD staging, fibrosis, and cancer grading, where forward-biased trajectories exist. It is also proposed that directed diffusion could be paired with longitudinal or time-to-event supervision to refine edge directionality. Finally, D-ODR is described as orthogonal to the base head: instead of MSE on a scalar regressor, it could regularize cumulative link models or rank-consistent classifiers by replacing $y_i \in \{0,1,2,3,4\}$0 with a scalar severity surrogate.

The implementation resources include public code at https://github.com/HovChen/D-ODR. The default reproducibility configuration is explicitly specified: learning rate $y_i \in \{0,1,2,3,4\}$1 with linear decay, batch size 128, $y_i \in \{0,1,2,3,4\}$2, $y_i \in \{0,1,2,3,4\}$3, $y_i \in \{0,1,2,3,4\}$4 with weights $y_i \in \{0,1,2,3,4\}$5, ImageNet normalization, random horizontal flip only, three runs per experiment, and checkpoint selection by normalized QWK plus F1 on validation. Inference remains identical to the baseline model, with no additional compute or memory.