---
title: 'Min-Max Median Formulation: Robust VB'
url: https://www.emergentmind.com/topics/min-max-median-formulation
type: topic
---

# Min-Max Median Formulation: Robust VB

The min-max median formulation is an advanced robust aggregation principle introduced for scalable variational Bayes (VB) inference over partitioned data. Designed to handle contamination and outliers, it replaces traditional mean-based Kullback-Leibler (KL) aggregation with a robust, saddle-point objective involving the median of per-block divergences. This approach has theoretical guarantees for robustness and statistical optimality, especially when combined with a two-stage aggregate-and-rescale strategy in the presence of local latent variables [2512.12676].

## 1. Mathematical Definition and Motivation

Consider a dataset $\mathcal{D}$ partitioned into $m$ disjoint subsets $\mathcal{D}_1, ..., \mathcal{D}_m$. For each subset, define the $m$-powered local variational "posterior":
\[
\tilde{P}^m_j(\theta) \propto \pi(\theta) [p(\mathcal{D}_j \mid \theta)]^m,
\]
where $\pi$ is the prior. The classical (non-robust) aggregation in distributed VB computes
\[
\hat{Q} = \arg\min_{Q\in\mathcal{F}} \frac{1}{m} \sum_{j=1}^m KL(Q \| \tilde{P}^m_j).
\]
To robustify against dataset contamination, the min-max median (M-VB) formulation introduces a game-theoretic saddle-point structure:
\[
\hat{F} = \arg\min_{F\in\mathcal{F}} \max_{G\in\mathcal{F}} \mathrm{Med}_{1\leq j \leq m}\left\{ KL(F \| \tilde{P}^m_j) - KL(G \| \tilde{P}^m_j) \right\},
\]
or, by exploiting the relationship $KL(Q \| \tilde{P}^m_j) = \text{const}_j - ELBO_j(Q)$,
\[
\hat{F} = \arg\min_{F\in\mathcal{F}} \max_{G\in\mathcal{F}} \mathrm{Med}_{1\leq j \leq m}\left\{ ELBO_j(G) - ELBO_j(F) \right\},
\]
where $ELBO_j(Q)$ denotes the evidence lower bound for the $j$th block. This shift from mean to median enables resistance to outlier blocks (insensitivity up to $\lfloor m/2 \rfloor$ arbitrary corruptions).

## 2. Equivalence to ELBO Maximization and Aggregation Consistency

For mean-based aggregation, KL minimization is equivalent to maximization of the average ELBO:
\[
\arg\min_Q \frac{1}{m}\sum_j KL(Q \| \tilde{P}^m_j)
= \arg\max_Q \frac{1}{m}\sum_j ELBO_j(Q).
\]
Directly replacing the mean with a median fails to ensure this equivalence, due to block-specific normalization constants. The min-max median fix enforces normalization cancellation by maximizing (in $G$) over the difference, thereby restoring the consistent mapping between divergence aggregation and median ELBO optimization.

## 3. Optimization Formulation and Robustness

The optimization takes the form
\[
\Phi(F,G) = \mathrm{Med}_{1\leq j \leq m} \left\{ ELBO_j(G) - ELBO_j(F) \right\},
\]
with the saddle point given by
\[
\hat{F} = \arg\min_F\max_G \Phi(F,G).
\]
Here, both $F$ and $G$ belong to a tractable variational family $\mathcal{F}$ (e.g., mean-field Gaussians). The median operator ensures the resulting aggregate is robust to up to $\lfloor m/2 \rfloor$ adversarial block corruptions. Outer minimization in $F$ seeks a distribution minimizing the "worst-half" ELBO relative to $G$, while inner maximization in $G$ aligns constants by targeting the block at which $F$ is least favorable.

## 4. Algorithmic Implementation

The min-max median objective can be solved via a coordinate-ascent-style alternating update. At iteration $t$:

- **Median Block Selection:**
  \[
  j_t \leftarrow \arg\max_{1 \leq j \leq m}\left\{ ELBO_j(G^{(t)}) - ELBO_j(F^{(t)}) \right\}
  \]
- **CAVI Update for $F$:** Within block $j_t$,
  \[
  f^{(t+1)}_l(\theta_l) \propto \exp\left\{ \mathbb{E}_{F^{(t)}_{-l}}[\log \pi(\theta) p(\mathcal{D}_{j_t} \mid \theta)^m] \right\},
  \]
  separately for each coordinate $\theta_l$.
- **Update for $G$:** Select $j'_t$ analogously, then update $g^{(t+1)}$.

For models with local latent variables $S$, the procedure first solves the min-max median problem over the unnormalized joint density $p(\mathcal{D}_j, S_j \mid \theta)$ (without the $m$-power), obtaining $\tilde{F}$. The global marginal $\tilde{f}(\theta)$ is then rescaled:
\[
\hat{f}(\theta) = m^{p/2} \tilde{f} \bigl(\mu_{\tilde{F}} + \sqrt{m}(\theta - \mu_{\tilde{F}})\bigr),
\]
ensuring $\mathrm{Cov}(\hat{F}) = \mathrm{Cov}(\tilde{F})/m$, which aligns with the full-data posterior covariance.

## 5. Theoretical Properties and Rates

Under regularity conditions—smoothness, sub-exponential tails, bounded-parameter space—the local VB posteriors satisfy a non-asymptotic Bernstein–von Mises theorem:
\[
KL(\hat{Q}_j \| N(\hat{\theta}_j, \tfrac{1}{mn}H^{-1})) = O_p\left( \frac{m(\log n)^{3/2}}{\sqrt{n}} \right).
\]
If these limits are Gaussian, the M-VB aggregate $\bar{F}$ remains Gaussian with mean given by the solution to a min-max quadratic program across the local means $\hat{\theta}_j$:
\[
\mu_{\bar{F}} = \arg\min_{\theta_f}\max_{\theta_g} \mathrm{med}_{1\leq j\leq m} \left\{ (\theta_f - \hat{\theta}_j)^\top\Omega(\theta_f - \hat{\theta}_j) - (\theta_g - \hat{\theta}_j)^\top\Omega(\theta_g - \hat{\theta}_j) \right\}.
\]
For general $\mathcal{F}$, the aggregate satisfies
\[
KL(\hat{F} \| N(\mu_{\hat{F}}, \tfrac{1}{mn}H^{-1})) = O_p\left( \frac{m(\log n)^{3/2}}{\sqrt{n}} + n\Delta_{\tau} \right),
\]
where $\Delta_{\tau}$ quantifies the empirical gap between upper and lower quantiles of the ELBO loss differences; for $\alpha_n = O(1/\sqrt{m})$, $\Delta_{\tau} = O(1/(mn))$. The posterior mean achieves the near-optimal rate:
\[
\|\mu_{\hat{F}} - \theta^*\|_2 = O \left( \frac{\alpha_n}{\sqrt{n}} + \sqrt{\frac{\log n}{mn}} + \frac{(\log n)^{3/4}}{n^{3/4}} \right).
\]

## 6. Two-Stage Aggregate-and-Rescale Versus Direct Aggregation

Two primary aggregation strategies are analyzed:

| Approach                         | Limiting Behavior                          | Concentration Rate      |
|-----------------------------------|--------------------------------------------|------------------------|
| One-stage $m$-powered M           | Limit is minimizer of $\log \int p(\mathcal{D}, S \mid \theta)^m dS$, generally $\neq \theta^*$ | Extra factor $m$ in error upper bound |
| Two-stage aggregate-and-rescale   | Minimizer matches true mode $\theta^*$     | No extra $m$ factor; improved rate   |

Direct application of the $m$-powered min-max median to the full likelihood does not yield the true posterior mode unless the model lacks local latent variables. The two-stage procedure—first solving the min-max median problem on unpowered joint likelihoods and subsequently rescaling the global marginal—ensures the population-level estimator recovers the correct mode and concentration rate. This yields improved finite-sample KL bounds and mean concentration compared to direct aggregation or median-of-means VB [2512.12676].

## 7. Robustness, Practical Impact, and Statistical Significance

The min-max median formulation robustifies VB aggregation under partitioned data, offering insensitivity to adversarial contamination and finite-sample guarantees. The saddle-point median-based aggregation, together with the aggregate-and-rescale technique in the presence of local latent variables, supports both theoretical optimality and computational scalability. The demonstrated improved rates and robustness positions the min-max median approach as a substantial advancement in distributed inference within contaminated or heterogeneous environments [2512.12676].

Source: https://www.emergentmind.com/topics/min-max-median-formulation