Distribution-Aware Knowledge Unification (DKUA)

• The paper introduces DKUA, a novel framework that preserves prior knowledge without storing exemplars by employing a distribution-aware domain-style encoder.
• It combines adaptive knowledge consolidation, unified knowledge association, and distribution-based transfer to balance anti-forgetting with new domain adaptation.
• Empirical results demonstrate significant improvements, including up to +13.9% Seen-Avg mAP gains over existing methods on standard lifelong person re-ID benchmarks.

Searching arXiv for the specified DKUA paper and closely related lifelong person re-identification work. Distribution-aware Knowledge Unification and Association (DKUA) is a framework for exemplar-free lifelong person re-identification (LReID) that addresses the tension between preserving previously acquired knowledge and adapting to newly arriving domains. It is introduced in "Distribution-aware Knowledge Unification and Association for Non-exemplar Lifelong Person Re-identification" (Liu et al., 5 Aug 2025). The framework is motivated by the observation that existing LReID methods typically rely on knowledge distillation to enforce representation alignment, but neglect specific distribution awareness and cross-domain unified knowledge learning. DKUA therefore combines domain-style modeling, adaptive knowledge consolidation, unified knowledge association, and distribution-based knowledge transfer to propagate domain-specific representations, construct a cross-domain representation center, model inter-domain associations, and constrain distributional drift without storing old samples (Liu et al., 5 Aug 2025).

1. Problem setting and motivation

DKUA is formulated for the exemplar-free lifelong person Re-ID problem, where a sequence of $T$ training domains arrives incrementally as $D=\{D^1,\dots,D^T\}$, with each domain defined as

$$D^t = \{(x_i,y_i)\}_{i=1}^{N_t},\quad x_i\in\mathbb R^{C\times H\times W},\;y_i\in\{1,\dots,|Y^t|\}.$$

At step $t$, the model only has access to $D^t$ and must preserve all knowledge learned on $\{D^1,\dots,D^{t-1}\}$ without revisiting their data, while simultaneously adapting to the new domain $D^t$ (Liu et al., 5 Aug 2025).

The paper denotes the model parameters after domain $t-1$ as $\Theta^{t-1}$ and after domain $t$ as $D=\{D^1,\dots,D^T\}$0, and states the overarching objective as

$D=\{D^1,\dots,D^T\}$1

Within this formulation, the central difficulty is catastrophic forgetting under the non-exemplar constraint. Existing distillation-or-rehearsal methods align instance embeddings between $D=\{D^1,\dots,D^T\}$2 and $D=\{D^1,\dots,D^T\}$3, but the paper argues that they overlook two issues: explicit modeling of each domain’s style and learning a unified cross-domain center to connect domain-specific representations (Liu et al., 5 Aug 2025).

This framing places DKUA within the rehearsal-free branch of lifelong Re-ID. A plausible implication is that the method is designed not merely to stabilize features over time, but to make domain structure itself an explicit object of optimization.

2. Distribution-aware model and domain-style encoding

At each new domain $D=\{D^1,\dots,D^T\}$4, DKUA first extracts a backbone feature

$D=\{D^1,\dots,D^T\}$5

where $D=\{D^1,\dots,D^T\}$6 is a ViT-Small/16 pretrained on LUPerson and $D=\{D^1,\dots,D^T\}$7 (Liu et al., 5 Aug 2025). To preserve past knowledge without storing samples, the framework introduces a Domain-Style Encoder (DSE) composed of $D=\{D^1,\dots,D^T\}$8 lightweight Transfer Modules $D=\{D^1,\dots,D^T\}$9. The first $$D^t = \{(x_i,y_i)\}_{i=1}^{N_t},\quad x_i\in\mathbb R^{C\times H\times W},\;y_i\in\{1,\dots,|Y^t|\}.$$0 modules are frozen and carry the style of domains $$D^t = \{(x_i,y_i)\}_{i=1}^{N_t},\quad x_i\in\mathbb R^{C\times H\times W},\;y_i\in\{1,\dots,|Y^t|\}.$$1, while the $$D^t = \{(x_i,y_i)\}_{i=1}^{N_t},\quad x_i\in\mathbb R^{C\times H\times W},\;y_i\in\{1,\dots,|Y^t|\}.$$2-th is trainable and captures the new domain’s style (Liu et al., 5 Aug 2025).

Applying each module to the backbone feature yields $$D^t = \{(x_i,y_i)\}_{i=1}^{N_t},\quad x_i\in\mathbb R^{C\times H\times W},\;y_i\in\{1,\dots,|Y^t|\}.$$3 domain-specific embeddings,

$$D^t = \{(x_i,y_i)\}_{i=1}^{N_t},\quad x_i\in\mathbb R^{C\times H\times W},\;y_i\in\{1,\dots,|Y^t|\}.$$4

Thus, each incoming instance is re-stylized into the representation spaces of all domains encountered so far. The paper characterizes this as encoding each domain’s distribution in the model parameters rather than in stored images (Liu et al., 5 Aug 2025).

This design is the distribution-aware component of DKUA. Instead of approximating earlier domains solely through distillation targets, the model explicitly maps a current sample into multiple domain-style spaces. This suggests that anti-forgetting is pursued through representational transplantation across domains rather than by direct recovery of prior exemplars.

3. Adaptive knowledge consolidation

After generating the set of domain-specific embeddings $$D^t = \{(x_i,y_i)\}_{i=1}^{N_t},\quad x_i\in\mathbb R^{C\times H\times W},\;y_i\in\{1,\dots,|Y^t|\}.$$5, DKUA unifies them into a single cross-domain center,

$$D^t = \{(x_i,y_i)\}_{i=1}^{N_t},\quad x_i\in\mathbb R^{C\times H\times W},\;y_i\in\{1,\dots,|Y^t|\}.$$6

where the per-instance weights are dynamically computed from the raw feature $$D^t = \{(x_i,y_i)\}_{i=1}^{N_t},\quad x_i\in\mathbb R^{C\times H\times W},\;y_i\in\{1,\dots,|Y^t|\}.$$7:

$$D^t = \{(x_i,y_i)\}_{i=1}^{N_t},\quad x_i\in\mathbb R^{C\times H\times W},\;y_i\in\{1,\dots,|Y^t|\}.$$8

The unified vector $$D^t = \{(x_i,y_i)\}_{i=1}^{N_t},\quad x_i\in\mathbb R^{C\times H\times W},\;y_i\in\{1,\dots,|Y^t|\}.$$9 is described as a cross-domain representation center (Liu et al., 5 Aug 2025).

To learn new identities while reducing drift, DKUA applies the standard Re-ID objective on $t$0:

$t$1

$t$2

Here $t$3 denotes the identity classifier head, $t$4 and $t$5 are positive and negative samples in the batch, and $t$6 is the triplet margin (Liu et al., 5 Aug 2025).

The paper further notes that the unified vector can be biased by the largest domain $t$7. To counter this, it defines, for each $t$8, a cosine-distance-based consistency term between the current domain style $t$9 and each prior style $D^t$0:

$D^t$1

The paper names this component Knowledge Alignment and states that it pulls the new-domain style representation toward each older style so that the unified center remains balanced (Liu et al., 5 Aug 2025).

Adaptive Knowledge Consolidation (AKC) is therefore the mechanism by which DKUA forms a weighted, instance-conditioned cross-domain center. In context, AKC differs from static centroiding because the weights $D^t$2 depend on each instance, allowing domain participation to vary sample by sample.

4. Unified knowledge association and cross-domain coupling

DKUA extends beyond fusion of domain-style embeddings by explicitly modeling inter-domain associations through Unified Knowledge Association (UKA). Let

$D^t$3

be the soft-assignment score between the unified vector $D^t$4 and style $D^t$5 (Liu et al., 5 Aug 2025).

The UKA loss aligns each $D^t$6 for $D^t$7 to the current $D^t$8 using Kullback–Leibler divergence:

$D^t$9

According to the paper, minimizing $\{D^1,\dots,D^{t-1}\}$0 ensures that all domain-style modules agree on their similarity to the global $\{D^1,\dots,D^{t-1}\}$1, thereby pulling older style spaces closer together (Liu et al., 5 Aug 2025).

The stated role of UKA is to use the unified representation as a bridge for explicit inter-domain association. This is significant because it moves beyond pairwise preservation of previous knowledge and toward a shared relational geometry across domains. A plausible implication is that UKA serves as the principal mechanism for reducing inter-domain gaps after AKC has already established a common center.

5. Distribution-based knowledge transfer

DKUA also includes a distribution-level regularization term termed Distribution-based Knowledge Transfer (DKT). Whereas AKC and UKA operate at the level of instance embeddings and inter-style relations, DKT aligns the full distributions of embeddings by maintaining a covariance matrix for each domain:

$\{D^1,\dots,D^{t-1}\}$2

A unified covariance is then updated as

$\{D^1,\dots,D^{t-1}\}$3

where $\{D^1,\dots,D^{t-1}\}$4 is the total number of identities seen so far (Liu et al., 5 Aug 2025).

The corresponding loss penalizes divergence between the new domain covariance and the global covariance:

$\{D^1,\dots,D^{t-1}\}$5

The paper states that this term prevents the current domain distribution from deviating from the cross-domain distribution center and improves adaptation capacity (Liu et al., 5 Aug 2025).

The progression from AKC to UKA to DKT reflects three distinct levels of unification: representation fusion, association alignment, and distribution regularization. This suggests that DKUA is structured as a multi-scale continual learning objective in which local instance geometry and global domain statistics are optimized jointly.

6. Optimization, implementation, and training protocol

At domain step $\{D^1,\dots,D^{t-1}\}$6, DKUA optimizes only the backbone $\{D^1,\dots,D^{t-1}\}$7 and the new module $\{D^1,\dots,D^{t-1}\}$8, while all earlier transfer modules $\{D^1,\dots,D^{t-1}\}$9 are frozen. The total loss is

$D^t$0

In practice, the paper sets $D^t$1 (Liu et al., 5 Aug 2025).

The per-domain training procedure is summarized as follows in the source:

$t-1$0

The reported hyper-parameters are a ViT-Small/16 backbone, feature dimension $D^t$2, batch size $D^t$3, epochs per domain $D^t$4, Adam optimizer, learning rate $D^t$5, decay $D^t$6 every $D^t$7 epochs, temperature $D^t$8, and triplet margin $D^t$9 (Liu et al., 5 Aug 2025).

These choices indicate that DKUA is implemented as a lightweight continual extension around a fixed backbone-plus-module decomposition. The freezing of earlier transfer modules is especially important: it constrains plasticity to the new style module while allowing the shared backbone to remain trainable.

7. Empirical results, interpretation, and relation to misconceptions

The experimental protocol evaluates DKUA on five Seen datasets—Market-1501, CUHK-SYSU, DukeMTMC, MSMT17_V2, and CUHK03—and six Unseen datasets—VIPeR, GRID, CUHK02, Occ_Duke, Occ_REID, and PRID2011. Two domain sequences are tested to simulate varying gaps, and performance is measured with mean Average Precision (mAP) and Rank-1 accuracy (R@1), averaged over Seen and Unseen domains to assess anti-forgetting and generalization (Liu et al., 5 Aug 2025).

For Training-order 1, DKUA achieves 64.1% Seen-Avg mAP and 74.5% Seen-Avg R@1, compared with the prior best DASK at 55.4%/69.3%, corresponding to +8.7% mAP and +5.2% R@1. On Unseen domains, DKUA attains 64.7% mAP and 61.3% R@1, improving over DASK’s 56.3%/52.6% by +8.4% mAP and +8.7% R@1 (Liu et al., 5 Aug 2025). The abstract summarizes the aggregate effect as outperforming existing methods by 7.6%/5.3% average mAP/R@1 improvement on anti-forgetting and generalization capacity, respectively (Liu et al., 5 Aug 2025).

The ablation reported for order 1 isolates the contribution of each component:

Configuration	Seen	Unseen
Baseline (no KA, UKA, DKT)	58.2% mAP / 65.8% R@1	57.5% / 54.6%
+KA	59.5% / 67.3%	58.9% / 55.8%
+UKA	62.6% / 72.1%	62.4% / 59.7%
+DKT	64.1% / 74.5%	64.7% / 61.3%

These ablations indicate that the gains are cumulative across the three added mechanisms (Liu et al., 5 Aug 2025). The paper also reports that anti-forgetting curves on Market-1501 remain essentially flat after training on MSMT17, while most baselines drop sharply, and that unseen generalization curves steadily climb at each step, with DKUA ending +5.3% R@1 above prior art (Liu et al., 5 Aug 2025).

Qualitative evidence is presented in the form of t-SNE visualizations and retrieval examples. The unified representations across five domains are said to tightly cluster same-ID points across all domains, whereas prior methods leave them scattered, and qualitative retrievals under low-light, occlusion, and viewpoint changes retrieve many more correct matches, such as 9–10/10, than DASK (Liu et al., 5 Aug 2025).

A common misconception in continual Re-ID is that anti-forgetting can be handled sufficiently by embedding distillation alone. DKUA explicitly contests this view by arguing that distillation-based alignment omits domain-style modeling and unified cross-domain knowledge learning (Liu et al., 5 Aug 2025). Another plausible misconception is that rehearsal-free methods necessarily sacrifice cross-domain generalization in favor of memory efficiency; the reported Seen and Unseen improvements are presented as evidence against that assumption. The paper concludes that the combination of a distribution-aware domain-style encoder with AKC, UKA, and DKT yields a balance between forgetting and adaptation, including up to +13.9% Seen-Avg mAP gains over rehearsal-based rivals and +12.0% Unseen mAP gains over rehearsal-free methods (Liu et al., 5 Aug 2025).