---
title: Maximum Subgroup Discrepancy (MSD)
url: https://www.emergentmind.com/topics/maximum-subgroup-discrepancy-msd
type: topic
---

# Maximum Subgroup Discrepancy (MSD)

Searching arXiv for the cited MSD and related discrepancy papers to ground the article in the current literature.
Maximum Subgroup Discrepancy (MSD) is a subgroup-focused, interpretable discrepancy for comparing two distributions with respect to protected attributes. In its concrete 2025 formulation, MSD asks which conjunction-defined protected subgroup has the largest difference in probability mass between two distributions, thereby targeting representation imbalance at the level of semantic or legally salient subpopulations rather than only at the level of global averages [2502.02221]. The notion is motivated by bias detection, data-quality auditing, and intersectional fairness auditing, and it is explicitly designed for settings in which the number of candidate subgroups is exponential but the analyst still wants finite-sample guarantees and an explicit witness subgroup.

## 1. Motivation and problem setting

The primary setting is distribution comparison. One has two distributions
\[
\mu,\nu \quad \text{on } X \subseteq \mathbb{R}^d,
\]
and the objective is to quantify whether some protected subgroup is represented differently across them. The motivating examples are data-quality auditing, where a dataset is compared against a trusted reference distribution, and model-output fairness auditing, where one compares the feature distributions of predicted classes such as \(Y=1\) and \(Y=0\) [2502.02221].

The central critique of standard global distances in this setting is twofold. First, the relevant signal is often subgroup-local rather than global: one wants to know whether there exists a protected subgroup whose mass differs substantially between the two distributions. Second, the paper argues that classical distances such as Total Variation and Wasserstein can be statistically too expensive to estimate in high-dimensional protected-feature settings, especially when the number of protected-feature subgroups is exponential and many intersectional groups are small [2502.02221].

The intended feature structure is explicit. Data points lie in \(X \subseteq \mathbb{R}^d\), there is a distinguished set of protected attributes \(\mathcal P \subseteq [d]\), protected attributes are treated as categorical, and for the main theory section they are assumed binary. The binary assumption is justified by preprocessing through quantization for continuous attributes and one-hot encoding for categorical attributes [2502.02221].

| Distance notion | Sample complexity statement |
|---|---|
| Total Variation | \(\exp(d)\) |
| Wasserstein-2 | \(\exp(d)\) |
| MMD | \(\exp(d)\) |
| Maximum Subgroup Discrepancy | \(d\) w.h.p. |

This comparison is the paper’s summary of worst-case behavior in Table 1. A plausible implication is that MSD is positioned less as a universal probability metric than as a task-specific discrepancy tailored to protected-feature subgroup auditing [2502.02221].

## 2. Formal definition and subgroup language

In the formal development, protected attributes are binary. For \(x \in \mathbb{R}^d\) and \(p \in \mathcal P\), the coordinate \(x_p \in \{0,1\}\), and the negated literal is
\[
\bar{x}_p := 1-x_p.
\]
The set of literals is
\[
L = \{x_p,\bar{x}_p\}_{p\in \mathcal P}.
\]
A term is a conjunction of literals. For a subset \(S \subset L\), the associated indicator is
\[
\chi_S(x) = \prod_{s\in S} s(x).
\]
The paper refers both to the literal set \(S\) and, by abuse of notation, to the corresponding region
\[
\mathcal X_S = \{x \in \mathbb{R}^d \mid \chi_S(x)=1\}
\]
as a subgroup. The set of all such subgroups is denoted \(\mathcal S\) [2502.02221].

This construction captures intersectional groups directly. A subgroup may specify only some protected features and leave others unspecified, so the search space includes single protected features, pairwise intersections, higher-order intersections, and partially specified combinations.

For a distribution \(\mu\) and function \(f\), the paper writes
\[
\mu(f) = \int f(x)\,d\mu(x).
\]
Thus \(\mu(\chi_S)\) is the probability mass of subgroup \(S\) under \(\mu\). The central one-sided definition is
\[
(\mu,\nu;\mathcal P) = \sup_{S \in \mathcal S} \bigl\{\mu(\chi_S) - \nu(\chi_S)\bigr\}.
\]
In practical evaluation, the absolute-difference version is used:
\[
\sup_{S\in\mathcal S}\left|\mu(\chi_S)-\nu(\chi_S)\right|.
\]
The paper obtains this empirical form by solving the one-sided problem twice, once with labels as given and once with labels flipped [2502.02221].

The introduction also presents a more general template,
\[
\mathrm{MSD}[\Delta](\mu,\nu,\mathcal P) = \sup_{S \in \mathcal S} \Delta(\mu(S),\nu(S)),
\]
which motivates a broader family of bias-detection distances. In the main instantiated version, the discrepancy is subgroup mass difference, one-sided in the theory section and absolute in empirical evaluation [2502.02221].

The paper gives an equivalent classification view. If samples from \(\mu\) are labeled \(y=1\) and samples from \(\nu\) are labeled \(y=0\), then minimizing 0–1 classification error over conjunctions \(f=\chi_S\) is equivalent to maximizing
\[
\mu(\chi_S)-\nu(\chi_S).
\]
With flipped labels one obtains the reverse difference, which yields the absolute-discrepancy version. This equivalence is central to the optimization formulation [2502.02221].

## 3. Statistical properties and metric status

The main theorem concerns empirical estimation. Given samples
\[
\{x_i\}_{i\le N_1} \sim \mu,\qquad \{x'_i\}_{i\le N_2} \sim \nu,
\]
with empirical distributions
\[
\hat\mu = \frac1{N_1}\sum_i \delta_{x_i}, \qquad \hat\nu = \frac1{N_2}\sum_i \delta_{x'_i},
\]
and \(N=\min(N_1,N_2)\), the theorem states that with probability at least \(1-2\delta\),
\[
(\mu,\nu;\mathcal P) \le (\hat\mu,\hat\nu;\mathcal P) + 4\sqrt{\frac{2|\mathcal P| + \log \frac{2}{\delta}}{2N}}.
\]
Equivalently, to achieve estimation error at most \(\varepsilon\), one needs
\[
N = O\!\left(\frac{|\mathcal P| + \log(1/\delta)}{\varepsilon^2}\right).
\]
The dependence is therefore linear in the number of protected features despite the exponential number of candidate subgroups [2502.02221].

The reason is combinatorial rather than measure-theoretic. For each protected feature \(p\), a term may include \(x_p\), include \(\bar x_p\), or include neither, so
\[
|\mathcal S| \le 3^{|\mathcal P|}.
\]
The proof then uses a finite-class uniform concentration bound of the form
\[
\sup_{S\in\mathcal S} \bigl(\hat\mu(S)-\mu(S)\bigr) \le 2\sqrt{\frac{\log |\mathcal S|+\log\frac{2}{\delta}}{2N_1}},
\]
together with
\[
\log |\mathcal S| \le 2|\mathcal P|,
\]
since \(\log 3 < 2\). The exponential class size does not produce exponential sample complexity because only its logarithm enters the bound [2502.02221].

The paper also situates MSD between two classical notions on the protected marginal. The fully specified atomic cells are the base terms
\[
B_{\mathcal P} = \left\{ \prod_{p\in\mathcal P} l_p \;\middle|\; l_p = x_p \text{ or } l_p=\bar x_p \right\},
\]
with exactly \(2^{|\mathcal P|}\) members. Define
\[
\|\mu-\nu\|_\infty = \sup_{S\in B} \bigl\{\mu(\chi_S)-\nu(\chi_S)\bigr\},
\]
and
\[
\|\mu-\nu\|_{TV} = \frac12 \sup_{f\in F_{\mathcal P,\infty}} \bigl\{\mu(f)-\nu(f)\bigr\}
= \frac12 \sum_{S\in B} \left|\mu(\chi_S)-\nu(\chi_S)\right|.
\]
Then
\[
\|\mu-\nu\|_\infty \le (\mu,\nu;\mathcal P) \le \|\mu-\nu\|_{TV}.
\]
Accordingly, MSD is stronger than checking only the worst fully specified protected-value atom, because it also considers partially specified subgroups, but weaker than Total Variation, because it does not aggregate all atomic discrepancies through an \(\ell_1\)-type sum [2502.02221].

The paper is careful about metric status. The formally defined object
\[
(\mu,\nu;\mathcal P)=\sup_{S\in\mathcal S}\bigl(\mu(\chi_S)-\nu(\chi_S)\bigr)
\]
is not symmetric, so it is not a metric. The practical absolute version is symmetric and nonnegative, but the paper does not provide a full metric-proof discussion such as identity of indiscernibles or triangle inequality. It is therefore more precise to treat MSD as a distance-like discrepancy over a restricted function class [2502.02221].

## 4. Computation through mixed-integer optimization

Although the sample complexity is favorable, the subgroup family is exponential. The computational solution proposed in the paper is a Mixed-Integer Optimization (MIO) formulation for conjunction search on empirical data [2502.02221].

Let \(\mathcal D_+\) index samples from \(\mu\), \(\mathcal D_-\) index samples from \(\nu\), and \(\mathcal D=\mathcal D_+\cup\mathcal D_-\). The MIO is
\[
\begin{aligned}
\max \quad & o \\
\mathrm{s.t.}\quad 
& o \le \frac{1}{|\mathcal D_+|} \sum_{i \in \mathcal D_+}\hat y_i
      - \frac{1}{|\mathcal D_-|} \sum_{i \in \mathcal D_-}\hat y_i + 2b \\
& o \le \frac{1}{|\mathcal D_-|} \sum_{i \in \mathcal D_-}\hat y_i
      - \frac{1}{|\mathcal D_+|} \sum_{i \in \mathcal D_+}\hat y_i + 2(1-b) \\
& \hat y_i \le 1 - (u_j - x_{i,j}u_j) \qquad i\in\mathcal D,\; j\in\mathcal P \\
& \hat y_i \ge 1 - \sum_{j\in\mathcal P}(u_j - x_{i,j}u_j) \qquad i\in\mathcal D \\
& \sum_{i\in\mathcal D}\hat y_i \ge N_{\min} \\
& 0 \le \hat y_i \le 1 \qquad i\in\mathcal D \\
& u_j,b \in \{0,1\} \qquad j\in\mathcal P.
\end{aligned}
\]

The variables have direct subgroup semantics. \(u_j \in \{0,1\}\) indicates whether feature \(j\) is present in the conjunction, \(\hat y_i\) indicates whether sample \(i\) belongs to the subgroup, \(o\) is the objective variable, and \(b\in\{0,1\}\) linearizes the absolute-value objective. The objective maximizes
\[
\left| \frac{1}{|\mathcal D_+|}\sum_{i\in\mathcal D_+}\hat y_i - \frac{1}{|\mathcal D_-|}\sum_{i\in\mathcal D_-}\hat y_i \right|.
\]
The paper notes that the difference of two means lies in \([-1,1]\), so a tight big-\(M\) value of \(2\) suffices in the absolute-value constraints [2502.02221].

The subgroup-membership constraints enforce conjunction semantics. Constraint
\[
\hat y_i \le 1 - (u_j - x_{i,j}u_j)
\]
forces \(\hat y_i=0\) if sample \(i\) violates any selected literal, and
\[
\hat y_i \ge 1 - \sum_{j\in\mathcal P}(u_j - x_{i,j}u_j)
\]
forces \(\hat y_i=1\) when all selected literals are satisfied. The paper remarks that \(\hat y_i\) is effectively binary because of these constraints together with the binary nature of \(x_{i,j}\) and \(u_j\) [2502.02221].

A minimum subgroup size constraint,
\[
\sum_{i\in\mathcal D}\hat y_i \ge N_{\min},
\]
is included to prevent degenerate solutions on tiny, statistically unreliable subgroups. In experiments the paper sets
\[
N_{\min}=10.
\]
If all \(u_j=0\), the conjunction is empty and always true, so \(\hat y_i=1\) for all samples; this corresponds to the trivial subgroup containing everyone [2502.02221].

The paper emphasizes exactness rather than heuristic search. It compares MIO against Ripper and BRCG-light and reports that MIO “almost always finds the global optimum within 10 minutes,” with only a few exceptions on the largest or highest-dimensional datasets. This suggests that, for the studied sample sizes, exact conjunction search is practically feasible even though exhaustive enumeration of subgroups is not [2502.02221].

## 5. Relation to adjacent discrepancy notions

MSD belongs to a broader family of max-over-subgroups discrepancy formulations. The most direct generalization in the fairness literature is Maximum Weighted Loss Discrepancy (MWLD),
\[
D\left(w, \ell, h\right) \triangleq \sup \limits_{g \in \mathcal G} w(g) \left| \mathbb E[\ell \mid g = 1] - \mathbb E[\ell] \right|,
\]
which treats groups as measurable indicator functions and introduces a weighting function \(w\). In the summary provided for that work, MWLD is described as a strict generalization of MSD-style objectives, with equal-weight max subgroup disparity appearing as a special case and size-weighted versions yielding tractable estimation regimes [1906.03518].

A distinct but related line studies maximum discrepancy over geometric subgroup classes. “Computing Approximate Statistical Discrepancy” formulates
\[
A^* = \arg\max_{A\in\mathcal A} \Phi(A)
\]
for geometric range spaces, with basic discrepancy
\[
\Phi(A)=|\mu_R(A)-\mu_B(A)|
\]
and more general scan-statistic objectives \(\Phi(A)=\phi(\mu_R(A),\mu_B(A))\). In that literature, the subgroup is a geometric range such as a halfspace, disk, or axis-aligned rectangle; the problem is therefore a geometric analogue of maximum subgroup discrepancy rather than protected-feature conjunction search [1804.11287]. “Approximate Maximum Halfspace Discrepancy” specializes this viewpoint to halfspaces \(h\in\mathcal H_d\) and searches for
\[
h^*=\arg\max_{h\in \mathcal H_d}\Phi_X(h),
\]
again with discrepancy defined through subgroup red/blue composition [2106.13851].

By contrast, the kernel-testing literature summarized in “\(k\)-Sample problem based on generalized maximum mean discrepancy” studies a weighted aggregate of pairwise discrepancies,
\[
\mathrm{GMMD}^2(\mathbb P_1,\dots,\mathbb P_k;\pi)
= \sum_{j=1}^k\sum_{\substack{\ell=1\\ \ell\neq j}}^k \pi_\ell\,\|m_j-m_\ell\|_{\mathcal H}^2,
\]
which is explicitly characterized as an aggregate multi-group discrepancy rather than a worst-case subgroup discrepancy. GMMD is therefore structurally different from MSD: it assumes fixed groups and sums pairwise discrepancies instead of maximizing over a subgroup family [1811.09103].

In robustness and fairness evaluation for image classification, “Better May Not Be Fairer: A Study on Subgroup Discrepancy in Image Classification” does not define MSD by name, but it is explicitly described as conceptually aligned with MSD-style reasoning. Its central empirical lesson is that overall accuracy can hide large subgroup harms, and its proposed metric, MacroStd,
\[
\sigma_{Macro}= \sqrt{\frac{1}{C}\sum_{i=1}^{C}{\sigma_{w}^{(i)}^2},
\]
is presented as a complementary class-balanced dispersion measure rather than a max-based worst-subgroup statistic [2212.08649].

Finally, “Set-Codes with Small Intersections and Small Discrepancies” studies worst-case discrepancy over a designed family of subsets under a common \(\pm1\) labeling, with
\[
\max_{1\le j\le s}|D_L(F_j)|\le 1.
\]
That problem is not MSD in the protected-feature or distribution-comparison sense, but it is a combinatorial analogue of controlling maximum discrepancy over a structured family [1901.05559].

## 6. Empirical behavior, applications, and limitations

The empirical evaluation in the MSD paper uses 10 real-world datasets derived from the 2018 American Community Survey via Folktables. Five are standard Folktables tasks in California—ACSIncome, ACSPublicCoverage, ACSMobility, ACSEmployment, and ACSTravelTime—where the feature distributions of \(y=1\) and \(y=0\) are compared. Five are state-versus-state population comparisons—Hawaii vs Maine, California vs Wyoming, Mississippi vs New Hampshire, Maryland vs Mississippi, and Louisiana vs Utah—where protected-feature distributions are compared across state pairs [2502.02221].

After preprocessing, the feature space is binary or categorical: continuous features are binned into 10 bins, some high-cardinality features are semantically grouped, the number of protected features ranges from 3 to 14, and the number of candidate subgroups ranges from about \(1.2\times 10^3\) to over \(5.7\times 10^9\). Exhaustive subgroup enumeration is therefore impossible in the studied regime [2502.02221].

The baselines are Total Variation, Wasserstein-1, Wasserstein-2, MMD, Ripper, and BRCG-light. For MMD on categorical data the paper uses the overlap kernel
\[
k(x_{i_1},x_{i_2}) = \sum_{j=1}^{d}\frac{\mathbf 1\{x_{i_1,j}=x_{i_2,j}\}}{d}.
\]
Each dataset is subsampled at five sizes in a geometric sequence from 1000 to the full sample size, each configuration is repeated with five random seeds, and each distance computation is given a 10-minute time limit [2502.02221].

The reported findings are consistent with the theory. Total Variation and Wasserstein distances “do not converge quickly to the final value,” while MIO-based MSD is described as stable and useful at much smaller sample sizes. The paper concludes that MSD “requires exponentially fewer samples to obtain a good estimate of the true value of the distance” than TV and Wasserstein. MMD is reported as the closest competitor empirically in convergence and variability, but without the same subgroup-level interpretability. Among subgroup learners, MIO consistently finds the highest MSD value because it solves for the global optimum, whereas Ripper and BRCG-light show more variability and often return weaker subgroups [2502.02221].

The main applications follow directly from the formulation: data-quality auditing against trusted reference distributions, model-output fairness auditing through subgroup representation imbalance between predicted classes, and intersectional bias detection over conjunction-defined protected groups. The witness subgroup is itself the diagnostic object, which the paper emphasizes as an interpretability advantage because it indicates where bias lives and therefore where intervention might be directed [2502.02221].

Several limitations are explicit. The theory assumes binary protected attributes, so discretization or one-hot encoding is required and results depend on the chosen representation. The subgroup class is restricted to conjunctions of protected-value indicators; if relevant bias patterns are not well represented by conjunctions, MSD may miss them. The discrepancy probes only the subgroup family \(\mathcal S\), not arbitrary measurable sets, so it is not a full distribution metric in the strongest sense. Optimization can still become hard on very large instances, and the appendix notes seed-to-seed variability that is plausibly related to the supremum structure. The broader MSDD framework is only sketched, with the paper developing mainly the subgroup mass discrepancy instantiation [2502.02221].

Source: https://www.emergentmind.com/topics/maximum-subgroup-discrepancy-msd