ConstStyle: Unified Domain Generalization

• ConstStyle is a unified approach to domain generalization that projects data into a common style space to minimize variability across domains.
• It leverages barycentric averaging of style statistics and an α-weighted partial projection, ensuring robust feature alignment during training and inference.
• Empirical results on benchmarks like PACS, Digits5, and CIFAR10-C show significant accuracy gains even with limited source domains and large domain shifts.

ConstStyle describes a unified approach to domain generalization that addresses the longstanding challenge of performance degradation when deep neural networks are deployed in environments exhibiting significant domain shift. Rather than relying exclusively on extracting domain-invariant features or traditional data augmentation, ConstStyle employs a unified style transformation: all source and target data—regardless of domain of origin—are projected into a common “unified” domain in which style variability is minimized. By centering both training and test samples around this barycentric style space, ConstStyle bridges the gap between seen and unseen domains, leading to robust generalization even when the number of observed source domains is small or the shift to the target domain is large (Tran et al., 7 Sep 2025).

1. Motivation and Conceptual Foundation

ConstStyle is motivated by the observation that empirical risk minimization and standard domain generalization (DG) approaches are vulnerable to domain shift, particularly when source domains are limited or the unseen domain deviates significantly from those seen during training. Existing DG techniques either focus on learning features invariant to domain-specific style (e.g., through adversarial loss or normalization modules), or on increasing diversity using random augmentations and style mixing. However, these methods may fail to sufficiently “span” the unseen domain’s style, leading to substantial accuracy drops in practical scenarios.

ConstStyle hypothesizes that the core challenge is not merely to learn invariances, but to actively “align” the style distributions of both seen and unseen domains by transforming all data into a shared, domain-agnostic style space. This unified style—represented by barycentric mean and covariance statistics—is constructed to minimize style discrepancies, thereby promoting extraction of consistent, domain-robust features.

2. Unified Domain Construction

The central methodological principle of ConstStyle is the explicit definition of a “unified domain.” Each source domain $S_k$ is associated with its own style distribution, characterized by feature means ($\varepsilon_{S_k}$) and covariances ($\Sigma_{S_k}$). The unified domain $T$ is computed as the barycenter of these domain-wise statistics.

Mathematically:

• The barycentric mean:

$$\varepsilon_B = \frac{1}{N} \sum_{k=1}^N \varepsilon_{S_k}$$

• The barycentric covariance (iteratively updated):

$$\Sigma_{B_{i+1}} = \frac{1}{N} \sum_{k=1}^N \left[\Sigma_{B_i}^{1/2} \Sigma_{S_k} \Sigma_{B_i}^{1/2}\right]^{1/2}$$

By barycentrically aggregating the style statistics, ConstStyle defines a unique “average style” that serves as an anchor for style normalization.

3. Style Transformation Mechanism

Aligning a sample’s latent representation to the unified domain is achieved by re-normalizing its internal style features. Given an input $x$ with style representation $z_x$, its channel-wise mean and standard deviation form the style statistic $\varepsilon_x = \mathrm{concat}(\mu_x, \sigma_x)$. At training time, sampled style statistics $(\mu_s, \sigma_s)$ from the unified style distribution $\varepsilon_{S_k}$0 are used to transform features:

$\varepsilon_{S_k}$1

This transformation projects latent features into the unified style domain. During inference, to preserve structure that may be unique to the unseen domain, a partial projection is applied:

$\varepsilon_{S_k}$2

where $\varepsilon_{S_k}$3 interpolates between full target-domain and unified domain statistics, controlling the degree of alignment at test time.

4. Theoretical Guarantees

ConstStyle provides theoretical justification for its projection mechanism, bounding the difference in empirical loss between the original and unified-style domains. For any domain $\varepsilon_{S_k}$4:

$\varepsilon_{S_k}$5

where $\varepsilon_{S_k}$6 and $\varepsilon_{S_k}$7 are distances in mean and std statistics, and $\varepsilon_{S_k}$8 is a function of the Lipschitz constant of the loss function.

This bound supports the claim that as the unified domain is placed barycentrically with respect to seen domains, the “cost” of projection is minimized in terms of task loss, and generalization to unseen domains (with bounded divergence from the unified domain) is theoretically controlled.

5. Training and Inference Pipeline

The training regime consists of two stages:

• Initial Training: Standard Empirical Risk Minimization is performed; style statistics are extracted and stored using a style feature extractor (e.g., a shallow network branch), then clustered to estimate domain-specific distributions.
• Unified-Style Training: Samples are transformed according to the unified domain formula before being fed to the model. The unified domain parameters are periodically updated to reflect evolving feature statistics.

At inference, unseen domain samples undergo partial projection using the $\varepsilon_{S_k}$9-weighted interpolation. Prediction is performed on transformed features with no need for additional fine-tuning or separate style-adaptation modules.

6. Empirical Performance and Analysis

ConstStyle has been evaluated across diverse visual recognition settings—including image classification (PACS, Digits5), image corruptions (CIFAR10-C), and instance retrieval (Market1501, Duke). Results indicate:

• Consistent improvement over MixStyle, DSU, CSU, and other DG baselines.
• Up to 19.82% absolute increase in classification accuracy in low source-domain settings.
• Substantial mAP and ranking P improvements in instance retrieval.
• Robustness to increasing corruption severity on CIFAR10-C.

Ablation studies highlight the importance of careful barycentric averaging and dynamic partial alignment at test time; excessive alignment ($\Sigma_{S_k}$0) may underfit subtle target-domain idiosyncrasies, while insufficient alignment ($\Sigma_{S_k}$1) exposes the model to greater domain shift.

Benchmark	Best Prior	ConstStyle	Improvement
PACS	85.7%	86.2%	+0.5%
Digits5	91.8%	92.6%	+0.8%
Market1501	81.3%	84.1%	+2.8%
CIFAR10-C	69.5%	72.4%	+2.9%

Performance gains are magnified as the number of source domains decreases or the domain gap increases.

7. Broader Implications and Future Work

The unified style transformation paradigm introduced by ConstStyle demonstrates that domain generalization is strongly facilitated by constructing a barycentric “style anchor” for all samples, rather than seeking merely to suppress domain-specific features or rely on implicit invariance. This insight generalizes beyond style shifts, potentially guiding the development of similar barycentric or unified representations for other axes of distributional shift (such as semantic, spatial, or temporal shifts).

The paper anticipates further investigation into extensions of ConstStyle to feature and semantic space alignment, refinement of the barycenter estimation process (e.g., using alternative metrics or priors), and generalization to non-classification tasks. Moreover, careful design of the partial projection parameter $\Sigma_{S_k}$2 at inference may enable dynamic adaptation to specific deployment scenarios without retraining (Tran et al., 7 Sep 2025).