Decision-Boundary-informed MMD

• Decision-Boundary-informed MMD is a refined metric that reweights samples based on their proximity to classifier decision boundaries.
• It enhances unsupervised domain adaptation by focusing alignment on boundary-critical regions, which minimizes domain shift and improves target error bounds.
• Empirical evaluations on standard DA benchmarks show that DB-MMD consistently outperforms traditional MMD, yielding higher accuracy and robust performance.

Decision Boundary-informed Maximum Mean Discrepancy (DB-MMD) is a strengthened variant of the Maximum Mean Discrepancy (MMD) metric tailored for unsupervised domain adaptation (DA). Unlike traditional MMD, which aligns distributions without considering their classifier-decision geometry, DB-MMD explicitly incorporates decision boundary information, thereby optimizing both distribution alignment and the induced cross-domain classifier. This hybrid framework simultaneously minimizes domain shift and refines the class boundary localization, resulting in improved theoretical target-error bounds and demonstrable empirical gains on standard DA benchmarks (Luo et al., 10 Feb 2025).

1. Standard Maximum Mean Discrepancy in Domain Adaptation

Given a feature map $\phi:\mathcal X\to\mathcal H$ into a universal reproducing kernel Hilbert space (RKHS) $\mathcal H$, the squared Maximum Mean Discrepancy between source distribution $P_S$ and target distribution $P_T$ is defined as

$$\mathrm{MMD}^2(P_S,P_T) = \left\|\mathbb E_{x\sim P_S}\left[\phi(x)\right] - \mathbb E_{x\sim P_T}\left[\phi(x)\right]\right\|_{\mathcal H}^2.$$

With empirical samples $\{x_i^s\}_{i=1}^{n_s}$, $\{x_j^t\}_{j=1}^{n_t}$ and a linear projection $z=A^T x$, this is estimated as

$$\mathrm{MMD}^2(P_S,P_T) \approx \mathrm{tr}\left(A^T X M_0 X^T A\right),$$

where $X=[x^s_1,\ldots,x^s_{n_s},x^t_1,\ldots,x^t_{n_t}]$ and $\mathcal H$0 is the marginal-MMD matrix,

$\mathcal H$1

Minimization of MMD in the projected space drives alignment of feature distributions across domains. In classic DA generalization bounds, MMD controls the domain-shift term, yielding for any hypothesis $\mathcal H$2,

$\mathcal H$3

where $\mathcal H$4 is the $\mathcal H$5-divergence and $\mathcal H$6 the irreducible minimum joint error (Luo et al., 10 Feb 2025).

2. Limitations of Standard MMD in Domain Adaptation

Vanilla MMD, while effective in aligning marginal and conditional distributions, operates indiscriminately across all sample pairs. It applies equal weighting to pairs far from or near the classifier's decision boundary. This property limits its effectiveness for classification-aware DA, as illustrated by persistent misclassification for boundary-lying samples. While the distribution-shift term in the DA error bound may decrease, failure to target boundary-adjacent regions impedes reduction of source error and the disagreement term between source and target labeling functions. Empirical analyses show that MMD can globally align the domains yet still produce mixed states for inter-class and boundary samples (Luo et al., 10 Feb 2025).

3. Formal Definition and Construction of DB-MMD

DB-MMD introduces a boundary-aware reweighting of the kernel mean-embedding by incorporating a classifier-dependent function. For base classifier $\mathcal H$7 (e.g., soft nearest neighbor, SVM), define a weighting function

$\mathcal H$8

such that samples near the decision boundary (small margin) have higher weights.

The DB-MMD metric is thus

$\mathcal H$9

Empirically, this becomes

$P_S$0

or, in projected form,

$P_S$1

with the DB-MMD matrix composed as

$P_S$2

where:

• $P_S$3: class-conditional MMD blocks,
• $P_S$4: repulsive MMD blocks,
• $P_S$5 ("compacting graph"): boundary-aware reweighting for same-class cross-domain pairs,
• $P_S$6 ("separation graph"): boundary-aware weighting for different-class cross-domain pairs,
• $P_S$7: Hadamard (entrywise) product.

This construction yields an MMD measure that places adaptive emphasis on samples in proximity to the decision boundary (Luo et al., 10 Feb 2025).

4. Tightened Theoretical Target-Error Bound

In the adapted generalization bound, utilizing DB-MMD yields

$P_S$8

where $P_S$9 but selectively focuses alignment on regions critical for classification performance. Minimizing $P_T$0 thus more directly reduces domain-shift in a classification-aware manner, effectively tightening the upper bound on the target domain error compared to vanilla MMD (Luo et al., 10 Feb 2025).

5. Integration with Existing Domain Adaptation Frameworks

DB-MMD serves as a drop-in replacement for traditional MMD in a variety of DA algorithms. Two frameworks exemplify its integration:

DGA-DA + DB-MMD

For the DGA-DA method, the original objective with marginal and conditional MMD is modified by:

• Replacing $P_T$1 with $P_T$2,
• Updating repulsive forces with $P_T$3,
• Leaving regularization and graph constraints unchanged.

MEDA + DB-MMD

Within MEDA, adding compacting and separation graph terms yields: $P_T$4 where $P_T$5 is the kernel matrix, $P_T$6 the base MMD matrix, $P_T$7 a graph Laplacian, and $P_T$8 incorporates boundary information. The separation graph can be appended analogously (Luo et al., 10 Feb 2025).

6. Empirical Performance and Benchmark Results

DB-MMD-enhanced frameworks were evaluated on eight standard DA datasets (e.g., USPS$P_T$9MNIST, COIL-20, CMU-PIE, Office+Caltech, Office-Home, VisDA). Key results include:

• CMU-PIE (PIE-9→PIE-27): MMD-only DGA-DA baseline $$\mathrm{MMD}^2(P_S,P_T) = \left\|\mathbb E_{x\sim P_S}\left[\phi(x)\right] - \mathbb E_{x\sim P_T}\left[\phi(x)\right]\right\|_{\mathcal H}^2.$$0 49.8%; DGA-DA+CG: 53.0%; DGA-DA+DB: 56.7% ($$\mathrm{MMD}^2(P_S,P_T) = \left\|\mathbb E_{x\sim P_S}\left[\phi(x)\right] - \mathbb E_{x\sim P_T}\left[\phi(x)\right]\right\|_{\mathcal H}^2.$$1 points vs. vanilla).
• Office-Home (ResNet-50 features): MEDA: 67.3%; MEDA+CG: 71.1% ($$\mathrm{MMD}^2(P_S,P_T) = \left\|\mathbb E_{x\sim P_S}\left[\phi(x)\right] - \mathbb E_{x\sim P_T}\left[\phi(x)\right]\right\|_{\mathcal H}^2.$$2), DGA-DA+DB: 77.1%.
• In all 60 source–target pairs, DB-MMD variants match or exceed their respective MMD-only baselines.

Task	MMD-Only Base	+CG	+DB
PIE-9→PIE-27	49.8	53.0	56.7
Office-Home (avg.)	75.7	77.7	78.1
Office+Caltech	89.1	90.0	90.4

These results indicate robust improvement and no instances where DB-MMD degraded baseline performance (Luo et al., 10 Feb 2025).

7. Practical Considerations and Implementation

• Kernel Selection: Gaussian RBF kernel $$\mathrm{MMD}^2(P_S,P_T) = \left\|\mathbb E_{x\sim P_S}\left[\phi(x)\right] - \mathbb E_{x\sim P_T}\left[\phi(x)\right]\right\|_{\mathcal H}^2.$$3 is used. For shallow DA, a single sigma via the median heuristic suffices; deep modules often adopt kernel mixtures.
• Hyper-parameter Tuning: DB-MMD does not introduce additional “balance” parameters. Existing trades (e.g., $$\mathrm{MMD}^2(P_S,P_T) = \left\|\mathbb E_{x\sim P_S}\left[\phi(x)\right] - \mathbb E_{x\sim P_T}\left[\phi(x)\right]\right\|_{\mathcal H}^2.$$4) are tuned by grid search or reverse-validation. Stability persists over a broad range $$\mathrm{MMD}^2(P_S,P_T) = \left\|\mathbb E_{x\sim P_S}\left[\phi(x)\right] - \mathbb E_{x\sim P_T}\left[\phi(x)\right]\right\|_{\mathcal H}^2.$$5.
• Computational Complexity: Matrix construction steps are $$\mathrm{MMD}^2(P_S,P_T) = \left\|\mathbb E_{x\sim P_S}\left[\phi(x)\right] - \mathbb E_{x\sim P_T}\left[\phi(x)\right]\right\|_{\mathcal H}^2.$$6 in $$\mathrm{MMD}^2(P_S,P_T) = \left\|\mathbb E_{x\sim P_S}\left[\phi(x)\right] - \mathbb E_{x\sim P_T}\left[\phi(x)\right]\right\|_{\mathcal H}^2.$$7; dominant cost arises from eigen-decomposition ($$\mathrm{MMD}^2(P_S,P_T) = \left\|\mathbb E_{x\sim P_S}\left[\phi(x)\right] - \mathbb E_{x\sim P_T}\left[\phi(x)\right]\right\|_{\mathcal H}^2.$$8 full, $$\mathrm{MMD}^2(P_S,P_T) = \left\|\mathbb E_{x\sim P_S}\left[\phi(x)\right] - \mathbb E_{x\sim P_T}\left[\phi(x)\right]\right\|_{\mathcal H}^2.$$9 for partial SVD).
• Convergence: Iterative procedures (project $\{x_i^s\}_{i=1}^{n_s}$0 update pseudo-labels/graphs $\{x_i^s\}_{i=1}^{n_s}$1 solve eigensystem) converge in 5–6 iterations (Luo et al., 10 Feb 2025).

DB-MMD constitutes a classification-aware advance in domain adaptation methodology, enforcing kernel alignment with explicit attention to decision boundaries. Its compatibility with existing MMD-based frameworks, favorable error bounds, and consistent empirical gains establish it as an influential tool in modern domain adaptation (Luo et al., 10 Feb 2025).