Maximum Subgroup Discrepancy (MSD)

• Maximum Subgroup Discrepancy (MSD) is a measure that detects representation imbalances in protected subgroups by comparing differences in probability mass between two distributions.
• It employs interpretable conjunctions of binary protected attributes to identify specific intersectional groups with significant discrepancies.
• MSD is efficiently computed via mixed-integer optimization, offering finite-sample guarantees and explicit witness subgroups for bias detection and fairness auditing.

Searching arXiv for the cited MSD and related discrepancy papers to ground the article in the current literature. Maximum Subgroup Discrepancy (MSD) is a subgroup-focused, interpretable discrepancy for comparing two distributions with respect to protected attributes. In its concrete 2025 formulation, MSD asks which conjunction-defined protected subgroup has the largest difference in probability mass between two distributions, thereby targeting representation imbalance at the level of semantic or legally salient subpopulations rather than only at the level of global averages (Němeček et al., 4 Feb 2025). The notion is motivated by bias detection, data-quality auditing, and intersectional fairness auditing, and it is explicitly designed for settings in which the number of candidate subgroups is exponential but the analyst still wants finite-sample guarantees and an explicit witness subgroup.

1. Motivation and problem setting

The primary setting is distribution comparison. One has two distributions

$\mu,\nu \quad \text{on } X \subseteq \mathbb{R}^d,$

and the objective is to quantify whether some protected subgroup is represented differently across them. The motivating examples are data-quality auditing, where a dataset is compared against a trusted reference distribution, and model-output fairness auditing, where one compares the feature distributions of predicted classes such as $Y=1$ and $Y=0$ (Němeček et al., 4 Feb 2025).

The central critique of standard global distances in this setting is twofold. First, the relevant signal is often subgroup-local rather than global: one wants to know whether there exists a protected subgroup whose mass differs substantially between the two distributions. Second, the paper argues that classical distances such as Total Variation and Wasserstein can be statistically too expensive to estimate in high-dimensional protected-feature settings, especially when the number of protected-feature subgroups is exponential and many intersectional groups are small (Němeček et al., 4 Feb 2025).

The intended feature structure is explicit. Data points lie in $X \subseteq \mathbb{R}^d$, there is a distinguished set of protected attributes $\mathcal P \subseteq [d]$, protected attributes are treated as categorical, and for the main theory section they are assumed binary. The binary assumption is justified by preprocessing through quantization for continuous attributes and one-hot encoding for categorical attributes (Němeček et al., 4 Feb 2025).

Distance notion	Sample complexity statement
Total Variation	$\exp(d)$
Wasserstein-2	$\exp(d)$
MMD	$\exp(d)$
Maximum Subgroup Discrepancy	$d$ w.h.p.

This comparison is the paper’s summary of worst-case behavior in Table 1. A plausible implication is that MSD is positioned less as a universal probability metric than as a task-specific discrepancy tailored to protected-feature subgroup auditing (Němeček et al., 4 Feb 2025).

2. Formal definition and subgroup language

In the formal development, protected attributes are binary. For $x \in \mathbb{R}^d$ and $Y=1$0, the coordinate $Y=1$1, and the negated literal is

$Y=1$2

The set of literals is

$Y=1$3

A term is a conjunction of literals. For a subset $Y=1$4, the associated indicator is

$Y=1$5

The paper refers both to the literal set $Y=1$6 and, by abuse of notation, to the corresponding region

$Y=1$7

as a subgroup. The set of all such subgroups is denoted $Y=1$8 (Němeček et al., 4 Feb 2025).

This construction captures intersectional groups directly. A subgroup may specify only some protected features and leave others unspecified, so the search space includes single protected features, pairwise intersections, higher-order intersections, and partially specified combinations.

For a distribution $Y=1$9 and function $Y=0$0, the paper writes

$Y=0$1

Thus $Y=0$2 is the probability mass of subgroup $Y=0$3 under $Y=0$4. The central one-sided definition is

$Y=0$5

In practical evaluation, the absolute-difference version is used: $Y=0$6 The paper obtains this empirical form by solving the one-sided problem twice, once with labels as given and once with labels flipped (Němeček et al., 4 Feb 2025).

The introduction also presents a more general template,

$Y=0$7

which motivates a broader family of bias-detection distances. In the main instantiated version, the discrepancy is subgroup mass difference, one-sided in the theory section and absolute in empirical evaluation (Němeček et al., 4 Feb 2025).

The paper gives an equivalent classification view. If samples from $Y=0$8 are labeled $Y=0$9 and samples from $X \subseteq \mathbb{R}^d$0 are labeled $X \subseteq \mathbb{R}^d$1, then minimizing 0–1 classification error over conjunctions $X \subseteq \mathbb{R}^d$2 is equivalent to maximizing

$X \subseteq \mathbb{R}^d$3

With flipped labels one obtains the reverse difference, which yields the absolute-discrepancy version. This equivalence is central to the optimization formulation (Němeček et al., 4 Feb 2025).

3. Statistical properties and metric status

The main theorem concerns empirical estimation. Given samples

$X \subseteq \mathbb{R}^d$4

with empirical distributions

$X \subseteq \mathbb{R}^d$5

and $X \subseteq \mathbb{R}^d$6, the theorem states that with probability at least $X \subseteq \mathbb{R}^d$7,

$X \subseteq \mathbb{R}^d$8

Equivalently, to achieve estimation error at most $X \subseteq \mathbb{R}^d$9, one needs

$\mathcal P \subseteq [d]$0

The dependence is therefore linear in the number of protected features despite the exponential number of candidate subgroups (Němeček et al., 4 Feb 2025).

The reason is combinatorial rather than measure-theoretic. For each protected feature $\mathcal P \subseteq [d]$1, a term may include $\mathcal P \subseteq [d]$2, include $\mathcal P \subseteq [d]$3, or include neither, so

$\mathcal P \subseteq [d]$4

The proof then uses a finite-class uniform concentration bound of the form

$\mathcal P \subseteq [d]$5

together with

$\mathcal P \subseteq [d]$6

since $\mathcal P \subseteq [d]$7. The exponential class size does not produce exponential sample complexity because only its logarithm enters the bound (Němeček et al., 4 Feb 2025).

The paper also situates MSD between two classical notions on the protected marginal. The fully specified atomic cells are the base terms

$\mathcal P \subseteq [d]$8

with exactly $\mathcal P \subseteq [d]$9 members. Define

$\exp(d)$0

and

$\exp(d)$1

Then

$\exp(d)$2

Accordingly, MSD is stronger than checking only the worst fully specified protected-value atom, because it also considers partially specified subgroups, but weaker than Total Variation, because it does not aggregate all atomic discrepancies through an $\exp(d)$3-type sum (Němeček et al., 4 Feb 2025).

The paper is careful about metric status. The formally defined object

$\exp(d)$4

is not symmetric, so it is not a metric. The practical absolute version is symmetric and nonnegative, but the paper does not provide a full metric-proof discussion such as identity of indiscernibles or triangle inequality. It is therefore more precise to treat MSD as a distance-like discrepancy over a restricted function class (Němeček et al., 4 Feb 2025).

4. Computation through mixed-integer optimization

Although the sample complexity is favorable, the subgroup family is exponential. The computational solution proposed in the paper is a Mixed-Integer Optimization (MIO) formulation for conjunction search on empirical data (Němeček et al., 4 Feb 2025).

Let $\exp(d)$5 index samples from $\exp(d)$6, $\exp(d)$7 index samples from $\exp(d)$8, and $\exp(d)$9. The MIO is

$\exp(d)$0

The variables have direct subgroup semantics. $\exp(d)$1 indicates whether feature $\exp(d)$2 is present in the conjunction, $\exp(d)$3 indicates whether sample $\exp(d)$4 belongs to the subgroup, $\exp(d)$5 is the objective variable, and $\exp(d)$6 linearizes the absolute-value objective. The objective maximizes

$\exp(d)$7

The paper notes that the difference of two means lies in $\exp(d)$8, so a tight big-$\exp(d)$9 value of $\exp(d)$0 suffices in the absolute-value constraints (Němeček et al., 4 Feb 2025).

The subgroup-membership constraints enforce conjunction semantics. Constraint

$\exp(d)$1

forces $\exp(d)$2 if sample $\exp(d)$3 violates any selected literal, and

$\exp(d)$4

forces $\exp(d)$5 when all selected literals are satisfied. The paper remarks that $\exp(d)$6 is effectively binary because of these constraints together with the binary nature of $\exp(d)$7 and $\exp(d)$8 (Němeček et al., 4 Feb 2025).

A minimum subgroup size constraint,

$\exp(d)$9

is included to prevent degenerate solutions on tiny, statistically unreliable subgroups. In experiments the paper sets

$d$0

If all $d$1, the conjunction is empty and always true, so $d$2 for all samples; this corresponds to the trivial subgroup containing everyone (Němeček et al., 4 Feb 2025).

The paper emphasizes exactness rather than heuristic search. It compares MIO against Ripper and BRCG-light and reports that MIO “almost always finds the global optimum within 10 minutes,” with only a few exceptions on the largest or highest-dimensional datasets. This suggests that, for the studied sample sizes, exact conjunction search is practically feasible even though exhaustive enumeration of subgroups is not (Němeček et al., 4 Feb 2025).

5. Relation to adjacent discrepancy notions

MSD belongs to a broader family of max-over-subgroups discrepancy formulations. The most direct generalization in the fairness literature is Maximum Weighted Loss Discrepancy (MWLD),

$d$3

which treats groups as measurable indicator functions and introduces a weighting function $d$4. In the summary provided for that work, MWLD is described as a strict generalization of MSD-style objectives, with equal-weight max subgroup disparity appearing as a special case and size-weighted versions yielding tractable estimation regimes (Khani et al., 2019).

A distinct but related line studies maximum discrepancy over geometric subgroup classes. “Computing Approximate Statistical Discrepancy” formulates

$d$5

for geometric range spaces, with basic discrepancy

$d$6

and more general scan-statistic objectives $d$7. In that literature, the subgroup is a geometric range such as a halfspace, disk, or axis-aligned rectangle; the problem is therefore a geometric analogue of maximum subgroup discrepancy rather than protected-feature conjunction search (Matheny et al., 2018). “Approximate Maximum Halfspace Discrepancy” specializes this viewpoint to halfspaces $d$8 and searches for

$d$9

again with discrepancy defined through subgroup red/blue composition (Matheny et al., 2021).

By contrast, the kernel-testing literature summarized in “$x \in \mathbb{R}^d$0-Sample problem based on generalized maximum mean discrepancy” studies a weighted aggregate of pairwise discrepancies,

$x \in \mathbb{R}^d$1

which is explicitly characterized as an aggregate multi-group discrepancy rather than a worst-case subgroup discrepancy. GMMD is therefore structurally different from MSD: it assumes fixed groups and sums pairwise discrepancies instead of maximizing over a subgroup family (Balogoun et al., 2018).

In robustness and fairness evaluation for image classification, “Better May Not Be Fairer: A Study on Subgroup Discrepancy in Image Classification” does not define MSD by name, but it is explicitly described as conceptually aligned with MSD-style reasoning. Its central empirical lesson is that overall accuracy can hide large subgroup harms, and its proposed metric, MacroStd,

$x \in \mathbb{R}^d$2

is presented as a complementary class-balanced dispersion measure rather than a max-based worst-subgroup statistic (Chiu et al., 2022).

Finally, “Set-Codes with Small Intersections and Small Discrepancies” studies worst-case discrepancy over a designed family of subsets under a common $x \in \mathbb{R}^d$3 labeling, with

$x \in \mathbb{R}^d$4

That problem is not MSD in the protected-feature or distribution-comparison sense, but it is a combinatorial analogue of controlling maximum discrepancy over a structured family (Gabrys et al., 2019).

6. Empirical behavior, applications, and limitations

The empirical evaluation in the MSD paper uses 10 real-world datasets derived from the 2018 American Community Survey via Folktables. Five are standard Folktables tasks in California—ACSIncome, ACSPublicCoverage, ACSMobility, ACSEmployment, and ACSTravelTime—where the feature distributions of $x \in \mathbb{R}^d$5 and $x \in \mathbb{R}^d$6 are compared. Five are state-versus-state population comparisons—Hawaii vs Maine, California vs Wyoming, Mississippi vs New Hampshire, Maryland vs Mississippi, and Louisiana vs Utah—where protected-feature distributions are compared across state pairs (Němeček et al., 4 Feb 2025).

After preprocessing, the feature space is binary or categorical: continuous features are binned into 10 bins, some high-cardinality features are semantically grouped, the number of protected features ranges from 3 to 14, and the number of candidate subgroups ranges from about $x \in \mathbb{R}^d$7 to over $x \in \mathbb{R}^d$8. Exhaustive subgroup enumeration is therefore impossible in the studied regime (Němeček et al., 4 Feb 2025).

The baselines are Total Variation, Wasserstein-1, Wasserstein-2, MMD, Ripper, and BRCG-light. For MMD on categorical data the paper uses the overlap kernel

$x \in \mathbb{R}^d$9

Each dataset is subsampled at five sizes in a geometric sequence from 1000 to the full sample size, each configuration is repeated with five random seeds, and each distance computation is given a 10-minute time limit (Němeček et al., 4 Feb 2025).

The reported findings are consistent with the theory. Total Variation and Wasserstein distances “do not converge quickly to the final value,” while MIO-based MSD is described as stable and useful at much smaller sample sizes. The paper concludes that MSD “requires exponentially fewer samples to obtain a good estimate of the true value of the distance” than TV and Wasserstein. MMD is reported as the closest competitor empirically in convergence and variability, but without the same subgroup-level interpretability. Among subgroup learners, MIO consistently finds the highest MSD value because it solves for the global optimum, whereas Ripper and BRCG-light show more variability and often return weaker subgroups (Němeček et al., 4 Feb 2025).

The main applications follow directly from the formulation: data-quality auditing against trusted reference distributions, model-output fairness auditing through subgroup representation imbalance between predicted classes, and intersectional bias detection over conjunction-defined protected groups. The witness subgroup is itself the diagnostic object, which the paper emphasizes as an interpretability advantage because it indicates where bias lives and therefore where intervention might be directed (Němeček et al., 4 Feb 2025).

Several limitations are explicit. The theory assumes binary protected attributes, so discretization or one-hot encoding is required and results depend on the chosen representation. The subgroup class is restricted to conjunctions of protected-value indicators; if relevant bias patterns are not well represented by conjunctions, MSD may miss them. The discrepancy probes only the subgroup family $Y=1$00, not arbitrary measurable sets, so it is not a full distribution metric in the strongest sense. Optimization can still become hard on very large instances, and the appendix notes seed-to-seed variability that is plausibly related to the supremum structure. The broader MSDD framework is only sketched, with the paper developing mainly the subgroup mass discrepancy instantiation (Němeček et al., 4 Feb 2025).