DiFac: Differentiated Consistency in SSL

• DiFac is a framework that distributes consistency constraints across multiple factors, such as sample relations, feature spaces, and architectural branches.
• It employs complementary mechanisms like absolute and relative location consistency, feature equivariance, and cross-branch regularization to overcome the limitations of uniform consistency.
• Empirical evaluations, including improvements in medical image segmentation Dice scores, demonstrate DiFac’s enhanced robustness and adaptability in low-label regimes.

Searching arXiv for the cited papers to ground the article in current records. Differentiated Factor Consistency Semi-supervised Framework (DiFac) is an interpretive formulation of semi-supervised learning in which consistency is not imposed as a single uniform constraint, but is instead distributed across differentiated factors, levels, branches, or spaces. In this view, the central question is not merely whether predictions for perturbed or related samples should agree, but which aspects of the system should be invariant, which should remain variant, and how cross-sample, cross-branch, or cross-hierarchy relations should be regularized. The perspective is particularly relevant to recent semi-supervised work that argues standard consistency regularization is too narrow when it only enforces agreement for a single sample under perturbation, or when it treats all representational levels as if they should behave identically (Qixiang et al., 2024, Fan et al., 2021).

1. Conceptual definition and scope

Within semi-supervised learning, consistency regularization is commonly used to exploit unlabeled data by requiring stable predictions under perturbations or augmentations. A DiFac interpretation departs from the uniform form of that principle. It emphasizes that different components of a learning system may require different regularization behaviors: some factors should be invariant, while others should preserve structured differences. The underlying motivation appears in several distinct lines of work.

One line of work argues that standard consistency regularization typically imposes consensus between the predictions of a single sample from different views, termed as Absolute Location consistency (AL-c), but that only AL-c may be insufficient because it may ignore the relative differences across samples, interpreted as relative locations (Qixiang et al., 2024). Another line argues that prediction consistency should be preserved while internal features should be allowed—or even encouraged—to differ across augmentations, rather than collapsing all levels of representation into invariance (Fan et al., 2021). A further line in semi-supervised factorization couples latent representation, label propagation, and adaptive structure learning by jointly learning an explicit label indicator for unlabeled data and preserving manifold structures in representation space and label space at the same time (Zhang et al., 2019). In semi-supervised medical image segmentation, consistency is extended across multiple differentiated sub-models and hierarchical decoder outputs, with mutual consistency and diagonal hierarchical consistency enforced between branch predictions and soft pseudo labels from other models (Koo, 2023).

Taken together, these formulations suggest that DiFac is best understood not as a single canonical algorithm in the cited material, but as a generalized consistency design principle. This suggests a framework in which differentiated factors may be defined by sample relations, feature versus classifier levels, latent representation versus label space, or architectural branches and scales.

2. From uniform consistency to differentiated consistency

A central premise of DiFac is that standard consistency regularization is too narrow when it enforces only one kind of agreement. In medical image classification, the critique is that relying solely on AL-c may ignore relative differences across samples and only exploit limited information from one perspective (Qixiang et al., 2024). To address this, the cited work proposes Sample Consistency Mean Teacher (SCMT), which not only incorporates AL-c but also additionally enforces consistency between the samples' relative similarities to its related samples, called Relative Location consistency (RL-c). AL-c and RL-c therefore conduct consistency regularization from two different perspectives and jointly extract more diverse semantic information for classification (Qixiang et al., 2024).

A related critique appears in consistency regularization for general semi-supervised learning. There, the limitation identified is that many methods implicitly push intermediate features to become invariant when predictions are forced to be invariant across augmentations. The cited work argues this is unnecessarily restrictive, and contrasts a fully invariant feature-space treatment with a formulation in which classifier outputs remain consistent while feature representations retain augmentation-induced differences (Fan et al., 2021). Its central claim is classifier-level consistency and feature-level equivariance: predictions for weak and strong views should be the same, but representations should not necessarily be identical and may be encouraged to differ (Fan et al., 2021).

Under a DiFac interpretation, these arguments are structurally similar. They reject a single undifferentiated constraint and replace it with multiple coordinated constraints applied to different relational or representational factors. This suggests that the essential object of regularization is not “the sample” alone, but the sample together with its position among related samples, its classifier output, its internal geometry, and its role within a structured model.

3. Principal realizations of the DiFac view

The source material supports several concrete realizations of differentiated consistency. They differ in domain and formalism, but each distributes learning constraints across distinct factors rather than relying on one global consistency objective.

Realization	Differentiated factors	Representative formulation
Sample-relation consistency	Absolute locations and relative locations	SCMT with AL-c and RL-c (Qixiang et al., 2024)
Level-differentiated consistency	Classifier outputs and feature representations	CR-Match with consistency and equivariance (Fan et al., 2021)
Representation-label coupling	Latent representation space and label space	RS2ACF with adaptive manifold preservation (Zhang et al., 2019)
Branch-hierarchy consistency	Distinct sub-models and decoder levels	DiHC-Net with mutual and diagonal hierarchical consistency (Koo, 2023)

In SCMT, the differentiated factors are sample-centric. AL-c regularizes the same sample across views, while RL-c regularizes the sample’s relative similarities to related samples (Qixiang et al., 2024). The same work further proposes Sample Scatter Mean Teacher (SSMT), motivated by the observation that due to the highly similar structures in medical images, the sample distribution could be overly dense in feature space, making their relative locations susceptible to noise. SSMT utilizes contrastive learning to sparsify the sample distribution and obtain robust and effective relative locations (Qixiang et al., 2024). In DiFac terms, this couples relation-aware consistency with geometry control.

In CR-Match, the differentiated factors are representational levels. The model includes a feature extractor $f$, a classifier $g$, a projection head $z$, and a rotation prediction head $h$. The method preserves weak-to-strong prediction consistency through pseudo-labeling while imposing FeatDistLoss at the feature level so that representations from stronger augmentations can be more distinguishable from weak or original features, yet still remain within one semantic cluster (Fan et al., 2021). This is an explicit example of differentiated consistency in which invariance and equivariance coexist.

In RS2ACF, the differentiated factors are latent factors and evolving labels. The method works with partially labeled data $X=[X_L, X_U]$, labeled indicator $A_L$, and a learned unlabeled label-indicator matrix $A_U$. Its objective jointly optimizes robust reconstruction, sparse corruption handling, adaptive manifold preservation, and a linear label predictor $P$, while preserving manifold structures explicitly and adaptively in the representation space and label space at the same time (Zhang et al., 2019). Although the paper does not explicitly formulate “factor consistency” under that name, it is directly relevant to a DiFac reading because it binds together latent representation, pseudo-label prediction, and adaptive structure learning.

In DiHC-Net, the differentiated factors are architectural and hierarchical. The framework is composed of multiple sub-models with identical multi-scale architecture but with distinct sub-layers, such as up-sampling and normalisation layers, and enforces mutual consistency between one model's intermediate and final prediction and soft pseudo labels from other models in a diagonal hierarchical fashion (Koo, 2023). Here, differentiation is induced at the operator level and regularization is propagated across branch and scale.

4. Mathematical structures and learning objectives

The mathematical structure of a DiFac-style framework is multi-objective. The source material does not provide a single unified DiFac objective, but it does provide several concrete objective families from which the formulation can be abstracted.

For level-differentiated consistency, the standard semi-supervised form uses labeled supervision and an unlabeled pseudo-label consistency term. In the CR-Match formulation, the supervised classification loss is

$$\mathcal{L}_S = \frac{1}{B_s} \sum_{i=1}^{B_s} \ell_{CE}(p_i, g(\alpha(x_i))).$$

For unlabeled data, weak and strong augmentations are passed through the feature extractor, a confidence score is computed from the weak view, and a FixMatch-style pseudo-label consistency loss is imposed on the strong view when the confidence exceeds $\tau$ (Fan et al., 2021). The distinctive DiFac element is that classifier-level consistency is combined with feature-level equivariance through FeatDistLoss, so that semantic agreement in label space and non-collapsed, augmentation-aware geometry in feature space are optimized simultaneously (Fan et al., 2021).

For sample-relation consistency, SCMT introduces AL-c and RL-c as complementary consistency mechanisms (Qixiang et al., 2024). The detailed equations are not provided in the data block, but the functional division is explicit: AL-c constrains one sample across views, whereas RL-c constrains consistency of relative similarities to related samples. SSMT adds contrastive learning to sparsify the distribution and thereby stabilize RL-c under dense medical-image feature geometry (Qixiang et al., 2024).

For representation-label coupling, RS2ACF provides the most explicit joint objective. Its full objective includes a robust reconstruction term $g$0, a sparse corruption penalty $g$1, adaptive manifold preservation terms involving $g$2, and label prediction terms involving $g$3, with $g$4 learned jointly for unlabeled data (Zhang et al., 2019). The key point in a DiFac reading is that the representation factor $g$5, the adaptive graph factor $g$6, the corruption factor $g$7, and the label factor $g$8 are co-optimized rather than treated independently.

For branch-hierarchy consistency, DiHC-Net uses a supervised multi-scale Dice loss

$g$9

a mutual consistency loss $z$0 between sharpened final predictions and peer branches’ final predictions, and a diagonal hierarchical consistency loss $z$1 between a branch’s final pseudo label and selected intermediate or final outputs of other branches (Koo, 2023). The combined consistency loss is $z$2, and the total objective is

$z$3

The consistency coefficient is warmed up by

$z$4

to prevent the consistency objective from harming early training (Koo, 2023).

A plausible implication is that a general DiFac framework can be described as an optimization program with at least two properties: differentiated constraints are attached to distinct factors, and these constraints are coupled through shared semantic targets, pseudo-labels, or structural relations.

5. Architectural and representational mechanisms

The architectural design space associated with DiFac is defined by controlled diversity. In DiHC-Net, the three sub-models share the same macro-architecture, a multi-scale V-Net, but differ in normalization and upsampling operators. The first sub-model uses Group Normalization and linear interpolation, the second uses Batch Normalization and transposed convolution, and the third uses Instance Normalization and nearest interpolation (Koo, 2023). The point of this design is not arbitrary heterogeneity. Rather, it is to produce non-identical predictions and internal feature statistics while preserving a common high-level topology so that consistency remains meaningful.

CR-Match reaches a similar objective through representational rather than architectural differentiation. The model decomposes into feature extractor $z$5, classifier $z$6, projection head $z$7, and auxiliary rotation head $z$8, and uses weak and strong augmentation pipelines. Strong augmentations include Autocontrast, Brightness, Color, Contrast, Equalize, Identity, Posterize, Rotate, Sharpness, Shear_x, Shear_y, Solarize, Translate_x, and Translate_y (Fan et al., 2021). The feature head is not forced to erase the effects of these transformations; instead, it is explicitly allowed to encode them while the classifier output remains stable.

In SCMT and SSMT, the operative mechanism is relation-aware feature organization. The sample’s own prediction under perturbation provides one source of consistency, but the sample’s relative similarity to related samples provides another. Because medical images may produce overly dense feature distributions, SSMT uses contrastive learning to sparsify the sample distribution and obtain robust and effective relative locations (Qixiang et al., 2024). This shifts consistency from a purely unary object to one defined over local sample neighborhoods.

In RS2ACF, the operative mechanism is factorization with jointly evolving latent and label variables. The matrix $z$9 adaptively preserves neighborhood reconstruction in data space, representation space, and projected label space, while $h$0 explicitly predicts unlabeled labels during optimization (Zhang et al., 2019). This makes the “factor” notion literal: the method is built from interacting matrix factors with different semantics and constraints.

These mechanisms indicate that DiFac is not tied to a single network motif. It can be instantiated through operator-diverse branches, projection heads with differentiated losses, relation-aware neighborhood constraints, or latent matrix factors. The unifying feature is selective consistency across differentiated structures rather than indiscriminate agreement.

6. Empirical behavior, benefits, and limitations

The empirical evidence in the source material consistently supports the claim that differentiated consistency can outperform more uniform baselines, especially in low-label regimes, although the forms of evidence differ by paper.

In semi-supervised medical image classification, the SCMT and SSMT framework is reported to show superiority through extensive experiments on different datasets (Qixiang et al., 2024). The precise dataset-level metrics are not included in the provided block, so broader quantitative claims should not be added. What is explicit is the claimed advantage of combining AL-c with RL-c and further stabilizing relative locations through contrastive sparsification (Qixiang et al., 2024).

In general semi-supervised learning, the CR-Match results are described as defining a new state of the art for various datasets and settings and outperforming previous work by a significant margin, particularly in low data regimes (Fan et al., 2021). The accompanying interpretation in the provided synthesis is that improved representation geometry leads to better decision boundaries, lower pseudo-label error, and better class separation in t-SNE (Fan et al., 2021). This suggests that differentiated treatment of classifier and feature levels is not merely a conceptual refinement but has measurable consequences for pseudo-label quality.

In semi-supervised medical image segmentation, DiHC-Net reports explicit gains over previous approaches on public benchmark datasets covering organ and tumour segmentation (Koo, 2023). On the LA dataset, under 10% labeled data, DiHC-Net reports Dice 90.42 compared with 89.82 for CC-Net and 88.96 for MC-Net+; under 20% labeled data, it reports Dice 91.94 compared with 91.27 for CC-Net and 91.07 for MC-Net+ (Koo, 2023). On BraTS 2019, under 10% labeled data, DiHC-Net reports 84.96 Dice compared with 82.74 for CC-Net and 82.42 for MC-Net+; under 20% labeled data, it reports 85.47 Dice compared with 83.30 for CC-Net and 83.24 for MC-Net+ (Koo, 2023). The ablations further indicate that Intra-Model Diversity alone does not consistently improve overall performance and can worsen contour metrics, while IMD plus multi-scale supervision helps, and adding DiHC yields the strongest results (Koo, 2023).

These results clarify an important limitation and a common misconception. Differentiation alone is not sufficient. The source material explicitly notes that diversity alone can complicate convergence and that its value emerges when combined with multi-scale supervision and structured consistency (Koo, 2023). Likewise, in the CR-Match interpretation, feature divergence is not unconstrained dispersion; features are encouraged to differ while remaining close enough to remain in the same semantic cluster (Fan et al., 2021). A DiFac framework therefore should not be equated with simple disagreement maximization. Its logic is controlled differentiation under shared semantic alignment.

A further limitation is specification heterogeneity. DiHC-Net is described as somewhat under-detailed, with inconsistent scale indexing and omitted minibatch composition and optimization details in the paper text (Koo, 2023). RS2ACF belongs to a matrix factorization lineage rather than the modern augmentation-based SSL family, so its transfer into neural DiFac designs is interpretive rather than direct (Zhang et al., 2019). SCMT/SSMT is specifically motivated by medical image classification, where high structural similarity makes relative locations susceptible to noise (Qixiang et al., 2024). These domain and formulation differences matter when generalizing the concept.

7. Relation to adjacent paradigms and likely research directions

DiFac intersects with several established semi-supervised paradigms but is not reducible to any one of them. It overlaps with Mean Teacher and weak-to-strong pseudo-labeling because it preserves classifier-level agreement under perturbation (Fan et al., 2021). It overlaps with metric and contrastive learning because SSMT uses contrastive learning to sparsify dense feature distributions and stabilize relative locations (Qixiang et al., 2024). It overlaps with mutual learning and co-training style ideas because DiHC-Net uses peer predictions as soft pseudo labels across branches (Koo, 2023). It also overlaps with semi-supervised matrix factorization because RS2ACF jointly learns latent representation, unlabeled label indicators, and adaptive neighborhood structure (Zhang et al., 2019).

The distinctive contribution of the DiFac perspective is the systematic separation of what should agree and what should differ. In the provided material, this separation appears in several forms: semantic agreement in label space versus non-collapsed augmentation-aware geometry in feature space (Fan et al., 2021); absolute versus relative sample consistency (Qixiang et al., 2024); representation space versus label space manifold preservation (Zhang et al., 2019); and final-output versus intermediate-hierarchy cross-branch consistency (Koo, 2023).

Several transferable principles are stated explicitly in the material. One is that differentiation should be deliberate but bounded: branch diversity can be useful, but diversity alone is unstable (Koo, 2023). Another is that consistency targets need not stay at the same level: a reliable high-level output can regularize lower or intermediate representations of peers (Koo, 2023). A third is that cross-factor consistency should be selective, not exhaustive: the diagonal pattern in DiHC-Net functions as a structured sparse consistency graph rather than all-to-all consistency (Koo, 2023). A fourth is that multi-scale regularization matters, since final-output consistency alone may leave useful information unused (Koo, 2023). A fifth is that weighting can depend on representation maturity, as reflected by $h$1 in DiHC-Net, where earlier or less mature predictions are softly regularized with smaller weights (Koo, 2023).

This suggests that future DiFac-style systems may formalize differentiated factors more explicitly, for example by defining separate semantic, structural, relational, and nuisance-sensitive components with distinct objectives. Such a direction is consistent with the cited evidence, but it remains an inference from the existing formulations rather than a directly stated unified proposal.