Disagreement-Aware Regularization

Updated 5 July 2026

The paper demonstrates that disagreement-aware regularization augments the task loss with an auxiliary divergence term to promote richer representations and model diversity.
It applies across diverse scenarios including multi-head attention, decentralized training, and explanation learning, yielding measurable performance gains such as improved BLEU scores and flatter minima.
The method emphasizes structured, space-dependent disagreement, balancing trade-offs to improve interpretability, robustness, and accuracy in various machine learning tasks.

Searching arXiv for the cited papers and related disagreement-aware regularization work. Disagreement-aware regularization denotes a family of training strategies in which disagreement between model components, workers, explainers, annotators, reconstructions, or view-specific predictors is treated as a useful signal rather than only as a defect to be removed. In explicit forms, disagreement appears as an auxiliary term added to the task objective; in implicit forms, it is preserved or measured so that optimization is biased toward richer representations, flatter minima, or more reliable uncertainty estimates. Across recent work, the same motif recurs in multi-head attention, decentralized optimization, post-hoc explanation training, soft-label modeling, robust subsampling, sparse-view 3D reconstruction, and uncertainty-aware medical segmentation (Li et al., 2018, Wang et al., 2 Feb 2026, Schwarzschild et al., 2023, Sawkar et al., 11 Aug 2025, Zhang et al., 2024, Mohammadi et al., 22 Mar 2026).

1. General formulation and problem setting

A representative explicit formulation augments a task loss with a disagreement term,

$J(\theta)=L(\mathbf{y}\mid\mathbf{x};\theta)+\lambda D(\theta),$

where $\lambda$ controls the strength of the regularization and $D(\theta)$ measures a chosen form of divergence among model components (Li et al., 2018). This template is direct in attention models and explanation-consensus training, but the same logic also appears in implicit forms: decentralized learning preserves non-vanishing consensus errors, annotator-aware systems up-weight high-entropy items, and reconstruction systems use disagreement to locate and suppress unreliable structure (Wang et al., 2 Feb 2026, Sawkar et al., 11 Aug 2025, Zhang et al., 2024).

The central design choice is not simply whether disagreement is “good” or “bad,” but what object disagrees with what, and in which space. The literature distinguishes disagreement in representation subspaces, attended positions, output states, explanation vectors, worker parameters, soft-label distributions, proxy-model rankings, Gaussian fields, and voxel-wise predictions. In some settings the regularizer maximizes diversity; in others it preserves a controlled amount of disagreement; in still others it suppresses disagreement once it has been identified as a proxy for error.

Setting	Disagreement signal	Role in training
Multi-head attention (Li et al., 2018)	Head subspaces, attended positions, output states	Auxiliary regularizer added to NLL
Decentralized training (Wang et al., 2 Feb 2026)	Consensus error $\delta_i^t=w_i^t-\bar w^t$	Implicit curvature-aware regularizer
Explanation learning (Schwarzschild et al., 2023, Jukić et al., 2022)	Explainer attribution mismatch or saliency disagreement	Consensus term or indirect representation penalty
Disagreement-labeled data (Sawkar et al., 11 Aug 2025, Horváth et al., 8 May 2026)	Item entropy or proxy rank-variance	Reweighting, entropy regularization, robust subsampling
Reconstruction and segmentation (Zhang et al., 2024, Mohammadi et al., 22 Mar 2026)	Field disagreement or multi-view prediction disagreement	Pruning, co-regularization, refinement, uncertainty targeting

This taxonomy shows that disagreement-aware regularization is better understood as a design pattern than as a single algorithm. The shared claim is that disagreement can encode complementary structure, ambiguity, or unreliability that the base objective does not model directly.

2. Canonical explicit form: multi-head attention diversification

In "Multi-Head Attention with Disagreement Regularization" (Li et al., 2018), multi-head attention is treated as an architecture whose standard formulation does not force different heads to learn distinct features. Multiple heads may collapse to similar subspaces or attend to the same positions. The paper therefore introduces three auxiliary regularizers over the $H$ heads. With $V^h\in\mathbb{R}^{M\times d_k}$ the value projection, $A^h\in\mathbb{R}^{N\times M}$ the attention weight matrix, and $O^h\in\mathbb{R}^{N\times d_v}$ the head output, the regularizers are:

Subspace disagreement: negative average cosine similarity between $\mathrm{vec}(V^i)$ and $\mathrm{vec}(V^j)$ .
Positional disagreement: negative average overlap $\lambda$ 0.
Output disagreement: negative average cosine similarity between $\lambda$ 1 and $\lambda$ 2.

Each term is averaged uniformly over $\lambda$ 3 head pairs, and in the reported experiments $\lambda$ 4 is fixed at $\lambda$ 5 without further tuning. The regularizers are applied to encoder self-attention, encoder-decoder attention, and decoder self-attention. Because the method adds no extra parameters, its cost is only marginal additional matrix operations, with a slight slowdown in training steps per second (Li et al., 2018).

The empirical results on WMT’17 Chinese→English and WMT’14 English→German establish the method’s practical motivation. On Zh→En with Transformer-Base, the baseline scores 24.13 BLEU, while subspace, position, and output disagreement individually reach 24.64, 24.42, and 24.78, respectively; combining all three gives 24.60. Applying disagreement regularization to all attention modules yields 24.85 BLEU, a gain of +0.72 over the baseline. In the overall comparison, Transformer-Base improves from 24.13/27.64 BLEU to 24.85/28.51, and Transformer-Big improves from 24.56/28.58 to 25.08/29.28 on Zh→En/En→De, with all improvements statistically significant at $\lambda$ 6 (Li et al., 2018).

The analysis is equally important. Direct disagreement measurements reported as $\lambda$ 7 show that each regularizer succeeds in its intended space: subspace disagreement reaches 0.999 when optimized, positional disagreement reaches 0.219 under the position term, and output disagreement reaches 0.997 under the output term, whereas the vanilla model exhibits almost zero positional disagreement (0.007) (Li et al., 2018). The paper’s interpretation is that positional diversity is not critical for translation, while subspace and output diversity better force heads to extract complementary features. A common misconception is therefore that all head diversity must be spatial; this study instead suggests that representational and output-level diversity are more consequential in this setting.

3. Controlled disagreement as implicit regularization in decentralized optimization

"Controlled disagreement improves generalization in decentralized training" (Wang et al., 2 Feb 2026) reframes disagreement in a different way. In canonical decentralized SGD, worker parameters $\lambda$ 8 are repeatedly averaged so that the consensus error

$\lambda$ 9

vanishes asymptotically. The paper argues against the view that consensus errors are merely harmful noise. Its DSGD-AC variant uses a time-dependent consensus factor $D(\theta)$ 0 in the update so that, for $D(\theta)$ 1, the consensus radius satisfies

$D(\theta)$ 2

rather than collapsing to zero (Wang et al., 2 Feb 2026).

The key theoretical result is that these non-vanishing disagreements are structured. Around a local minimizer with Hessian $D(\theta)$ 3, the disagreement dynamics align with the dominant Hessian subspace, and high-curvature modes remain closer to instability than low-curvature modes. In the quadratic regime, the average local loss expands as

$D(\theta)$ 4

so the method implicitly adds a curvature-tilted penalty that penalizes sharp directions more heavily (Wang et al., 2 Feb 2026). The paper names this effect Disagreement-Aware Regularization.

The empirical evidence aligns with that interpretation. On CIFAR-100 with WRN28-10, 8 workers, and a ring topology, synchronous SGD reaches 80.15 ± 0.29% test accuracy with top Hessian eigenvalue $D(\theta)$ 5, standard DSGD reaches 79.93 ± 0.23% with eigenvalue $D(\theta)$ 6, and DSGD-AC reaches 81.98 ± 0.19% with eigenvalue $\approx 0.426*<em>. On WMT14 En→De with Transformer-big and 8 workers, Adam achieves 28.68 ± 0.07 BLEU, decentralized Adam without AC 28.38 ± 0.22, and **DAdam-AC 28.89 ± 0.17</em>* (<a href="/papers/2602.02899" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Wang et al., 2 Feb 2026</a>). The paper explicitly compares the mechanism to Sharpness-Aware Minimization: flatter minima are favored, but the perturbation arises “for free” from the consensus mechanism rather than from a second gradient evaluation.</p> <p>This work directly contradicts a frequent assumption that agreement among workers should always be maximized. Here, controlled disagreement is beneficial precisely because it is not isotropic: it is Hessian-aligned, persists as the step size decays, and probes sharp directions of the loss landscape (<a href="/papers/2602.02899" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Wang et al., 2 Feb 2026</a>).</p> <h2 class='paper-heading' id='explanation-consensus-and-saliency-disagreement'>4. Explanation consensus and saliency disagreement</h2> <p>A second line of work studies disagreement not between model parameters but between post-hoc explanations. "Reckoning with the Disagreement Problem: Explanation Consensus as a Training Objective" (<a href="/papers/2303.13299" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Schwarzschild et al., 2023</a>) introduces Post hoc Explainer Agreement Regularization (PEAR), which augments the task loss with a convex combination of Spearman and Pearson correlation between two attribution vectors:</p> <p>$D(\theta)$7

Here $D(\theta)$8 trades off task performance against explainer consensus, while $D(\theta)$9 trades off rank-based and magnitude-based agreement. Because Spearman correlation depends on ranks, the implementation replaces the non-differentiable ranking operator with a differentiable soft-ranking layer such as torchsort (Schwarzschild et al., 2023).

The experiments on Bank Marketing, California Housing, and Electricity show that increasing $\delta_i^t=w_i^t-\bar w^t$0 improves all six reported agreement metrics, including for explainer pairs that were never used in the training loss. On Electricity, moving from $\delta_i^t=w_i^t-\bar w^t$1 to $\delta_i^t=w_i^t-\bar w^t$2 shifts accuracy from approximately 81.3% to approximately 80.1%, while LIME–SHAP Pairwise Rank Agreement rises from approximately 0.50 to approximately 0.82 (Schwarzschild et al., 2023). The paper further reports that junk features appear in the top 5 less frequently under PEAR, for example for LIME on Electricity from 11.8% down to 9.1%, and that increasing $\delta_i^t=w_i^t-\bar w^t$3 lowers the MAE of linear fits to local logit surfaces. At the same time, an ablation shows that simply increasing weight decay also lowers the linear-fit MAE but does not improve explainer consensus, indicating that mere linearity is not sufficient (Schwarzschild et al., 2023).

Jukić et al. address the same problem from a different angle in "Easy to Decide, Hard to Agree: Reducing Disagreements Between Saliency Methods" (Jukić et al., 2022). They argue that Kendall-$\delta_i^t=w_i^t-\bar w^t$4 is a poor agreement diagnostic because small noise in low-importance tokens can cause large rank changes, and they instead compute instance-wise Pearson-$\delta_i^t=w_i^t-\bar w^t$5 between saliency vectors. Rather than optimizing a Pearson term directly, they regularize the internal representation with either conicity, which measures alignment of hidden states to their mean, or tying, which penalizes deviation of contextualized hidden states from their corresponding input embeddings. The total loss is cross-entropy plus $\delta_i^t=w_i^t-\bar w^t$6 times one of these penalties (Jukić et al., 2022).

Across SUBJ, SST-2, TREC, and IMDB, and across both a Bi-LSTM+self-attention model and DistilBERT, the base variants show low average agreement, for example $\delta_i^t=w_i^t-\bar w^t$7 for dbert/SST, whereas conicity or tying often pushes agreement above 0.5–0.6. Classification performance remains essentially unchanged; on SST the jwa base model reaches 0.78 F₁, while tying and conicity produce 0.78 and 0.76 F₁, respectively (Jukić et al., 2022). The dataset-cartography analysis further shows that unregularized models exhibit the lowest explanation agreement on easy-to-learn examples ($\delta_i^t=w_i^t-\bar w^t$8), whereas ambiguous examples show higher agreement ($\delta_i^t=w_i^t-\bar w^t$9); tying raises all three instance categories by roughly 0.2 points. The paper links this pattern to local geometry: easy instances occupy smooth, dense regions of representation space, whereas ambiguous instances are higher-curvature and more isolated (Jukić et al., 2022).

Taken together, these papers establish both a methodological and a diagnostic point. Explanation consensus can be optimized directly, but the choice of agreement metric is itself contested, and improved agreement does not reduce to a generic preference for smoother or more linear models.

5. Disagreement under ambiguous or corrupted supervision

In tasks where the label itself is distributional or contested, disagreement-aware regularization often appears as reweighting rather than as pairwise divergence between model components. "LPI-RIT at LeWiDi-2025: Improving Distributional Predictions via Metadata and Loss Reweighting with DisCo" (Sawkar et al., 11 Aug 2025) extends the DisCo architecture for joint item-level and annotator-level prediction. Each item is encoded as $H$0, each annotator as $H$1, and the joint code predicts an annotator-conditioned distribution $H$2 together with aggregated item- and annotator-level distributions. The paper replaces one-hot annotator identifiers with a 768-dimensional transformer-encoded metadata vector derived from structured fields such as age, gender, nationality, and education (Sawkar et al., 11 Aug 2025).

Its disagreement-aware objective combines three ingredients. First, the original composite loss contains a per-annotator negative log-likelihood and KL terms for item- and annotator-level distributions. Second, examples are reweighted by normalized item entropy,

$H$3

so that higher-disagreement items contribute more strongly to the per-annotator NLL. Third, an entropy regularizer

$H$4

discourages spuriously sharp or flat predictions (Sawkar et al., 11 Aug 2025). The reported results are substantial: CSC Wasserstein drops from 1.45 to 0.87 and PE (Absolute Dist.) from 0.33 to 0.22; MultiPico Manhattan drops from 0.54 to 0.45 and PE (Error Rate) from 0.32 to 0.31; Paraphrase Wasserstein drops from 3.71 to 2.21 and PE from 0.43 to 0.28. The gains are largest on high-entropy cases; CSC examples with modal probability below 0.4 see average error drop by 15%, and annotator-level calibration improves with fewer high-confidence, large-error outliers (Sawkar et al., 11 Aug 2025).

A different notion of disagreement is used in "Disagreement-Regularized Importance Sampling for Adversarial Label Corruption" (Horváth et al., 8 May 2026). Here the goal is robust subsampling or reweighting under $H$5-contamination. Instead of prioritizing examples with large loss or gradient norm, which adversarial outliers can exploit, DR-IS trains $H$6 cheap proxy models, converts their per-sample losses into normalized ranks $H$7, and scores each example by empirical rank-disagreement,

$H$8

Static pruning keeps the top $H$9 examples by $V^h\in\mathbb{R}^{M\times d_k}$0, while online importance sampling uses $V^h\in\mathbb{R}^{M\times d_k}$1 and unbiased importance weights (Horváth et al., 8 May 2026).

The theoretical guarantees are unusually explicit. Uniform concentration holds at rate

$V^h\in\mathbb{R}^{M\times d_k}$2

a positive structural expectation gap $V^h\in\mathbb{R}^{M\times d_k}$3 yields strict separation between the bulk of corrupted examples and boundary-clean examples, and the contamination rate of the selected subset is bounded (Horváth et al., 8 May 2026). Empirically, the method is robust under targeted high-norm attacks that catastrophically break magnitude-based baselines. On CIFAR-10 with VGG-19 and 25% targeted noise, random pruning reaches 74.66%, EL2N 9.39%, AUM 79.69%, and DR-IS 81.86%. On CIFAR-100 with ResNet-18 and the same corruption level, random reaches 51.53%, AUM 53.32%, and DR-IS 60.96%. In online importance sampling on CIFAR-10, uniform SGD reaches 74.52%, standard gradient-norm IS 10.00%, and DR-IS-online 84.04% (Horváth et al., 8 May 2026).

These two papers share a common premise: when supervision is ambiguous or corrupted, disagreement can identify where model capacity or sampling budget should be concentrated. Yet they operationalize that premise differently—one via entropy-weighted fitting of soft labels, the other via disagreement-based selection of informative clean boundary points.

6. Reconstruction and segmentation: disagreement as an uncertainty proxy

In geometric reconstruction, disagreement can be used to detect error without ground truth. "CoR-GS: Sparse-View 3D Gaussian Splatting via Co-Regularization" (Zhang et al., 2024) trains two independent 3D Gaussian radiance fields on the same sparse views and measures two signals: point disagreement between Gaussian centers and rendering disagreement between synthesized colors and depths. The paper reports that both disagreements increase as incorrect geometry accumulates and correlate negatively with reconstruction accuracy. CoR-GS therefore uses disagreement in two ways: co-pruning removes Gaussians whose nearest-neighbor correspondence exceeds a distance threshold $V^h\in\mathbb{R}^{M\times d_k}$ 4, and pseudo-view co-regularization suppresses rendering disagreement on sampled intermediate views using an $V^h\in\mathbb{R}^{M\times d_k}$ 5 plus D-SSIM color term with $V^h\in\mathbb{R}^{M\times d_k}$ 6 and $V^h\in\mathbb{R}^{M\times d_k}$ 7 (Zhang et al., 2024).

The algorithm co-prunes every 5 density/optimization cycles and trains for 10k iterations on LLFF, DTU, and Blender, or 30k on Mip-NeRF360. On LLFF with 3/6/9 views, CoR-GS reports PSNR values of 20.45, 24.49, and 26.06. On Mip-NeRF360 with 12/24 views, it improves PSNR by approximately 1 dB over vanilla 3DGS and FSGS while also reaching the best SSIM and AVGE. An ablation on LLFF 3-view shows +0.40 PSNR for co-pruning alone, +1.04 for pseudo-view co-regularization alone, and +1.23 for the full method. The deployed model renders at 349 FPS with approximately $V^h\in\mathbb{R}^{M\times d_k}$ 8 Gaussians, 33% fewer than 3DGS (Zhang et al., 2024).

In clinical segmentation, disagreement serves as a voxel-wise uncertainty estimate. DGRNet attaches four lightweight view-specific adapters to a shared encoder-decoder, generating diverse predictions in a single forward pass. It then builds disagreement maps from prediction variance, pairwise disagreement, and predictive entropy, and uses those maps to drive selective refinement conditioned on clinical reports. The training strategy is explicitly diversity-preserving: it combines pairwise similarity penalties with gradient isolation to prevent view collapse (Mohammadi et al., 22 Mar 2026). The total loss is written as

$V^h\in\mathbb{R}^{M\times d_k}$ 9

On TextBraTS, the full pipeline raises average Dice to 87.6% from 85.3% for the best prior text-guided method and lowers average HD95 to 4.57 mm from 5.13 mm, an approximately 11% reduction. The uncertainty estimates are also quantified: Uncertainty-Ratio is approximately 239.4×, Error-Detection AUC approximately 0.910, and AUSE approximately 0.0006. The ablations are monotonic, with Dice improving from 84.9 to 87.6 and HD95 from 6.88 to 4.56 as multi-view heads, diversity regularization, disagreement-guided attention, refinement, and text conditioning are added (Mohammadi et al., 22 Mar 2026).

These examples show a reversal of emphasis relative to head-diversification or decentralized learning. The goal is not simply to maximize disagreement, but to use disagreement to localize unreliable geometry or ambiguous voxels and then regularize those regions selectively.

7. Recurring principles, limitations, and open directions

Several principles recur across the literature. First, useful disagreement is typically structured rather than arbitrary. In decentralized training it aligns with dominant Hessian directions; in DR-IS it is the variance of normalized loss ranks across independent proxies; in CoR-GS it is disagreement that correlates with reconstruction error; in multi-head attention it is disagreement in output or subspace rather than merely in attended positions (Wang et al., 2 Feb 2026, Horváth et al., 8 May 2026, Zhang et al., 2024, Li et al., 2018).

Second, disagreement is space-dependent. The attention paper shows that output disagreement gives the largest single-term BLEU gain, while positional disagreement underperforms and the vanilla Transformer already exhibits almost zero positional disagreement. The saliency literature likewise shows that the diagnostic metric matters: Jukić et al. prefer Pearson- $A^h\in\mathbb{R}^{N\times M}$ 0 over rank-based measures, while PEAR optimizes a tunable combination of Pearson and Spearman (Li et al., 2018, Jukić et al., 2022, Schwarzschild et al., 2023). A common simplification—that disagreement is a single scalar property of a model—is therefore not supported by the empirical record.

Third, disagreement-aware methods exhibit clear trade-offs and non-additivities. In multi-head attention, combining all three regularizers does not improve over the best single term. In PEAR, larger $A^h\in\mathbb{R}^{N\times M}$ 1 improves consensus but reduces accuracy. In DR-IS, the method complements rather than dominates AUM: under low-to-moderate corruption and small $A^h\in\mathbb{R}^{N\times M}$ 2 it often wins, whereas under extreme corruption or large $A^h\in\mathbb{R}^{N\times M}$ 3 AUM can be more stable. DR-IS also fails on the synthetic linear/R20 benchmark when Assumption 2 is violated (Li et al., 2018, Schwarzschild et al., 2023, Horváth et al., 8 May 2026).

Finally, the open directions proposed by the papers suggest that disagreement-aware regularization remains an active methodological frontier. DSGD-AC motivates adaptive communication compression, curvature-aware graph design, and federated extensions with non-i.i.d. data. The LeWiDi system proposes per-annotator reweighting schedules, richer demographic priors, and meta-learning schemes that adapt the regularizer online as inter-annotator variance shifts. CoR-GS reports that extending from two to three or four fields yields an additional improvement of approximately 0.16 dB, indicating that multi-model disagreement itself can be scaled (Wang et al., 2 Feb 2026, Sawkar et al., 11 Aug 2025, Zhang et al., 2024).

The broad implication is not that disagreement should always be amplified. The literature instead supports a sharper claim: disagreement is useful when its source, geometry, and target are matched to the learning problem. It can diversify heads, flatten minima, stabilize explanations, preserve minority perspectives, filter corrupted labels, diagnose reconstruction failure, or guide uncertainty-aware refinement—but only when the disagreement signal is made commensurate with the task structure it is intended to regularize.