Negative Mask Proposal

• Negative Mask Proposal is a framework of methods that design masks to de-emphasize or suppress regions unlikely to match the target signal.
• It is applied in diverse tasks such as video moment retrieval, object removal, and language modeling to enforce invariance and contrastive learning.
• Key design patterns include complement masks, perturbed masks, and reliability masks, which have demonstrably improved metrics like FID, PSNR, and mAP.

Searching arXiv for the supplied paper and closely related work on mask-based proposal mechanisms. Negative Mask Proposal denotes a family of masking mechanisms in which a model constructs, derives, or perturbs masks so as to emphasize regions that should not align with a target signal, or to suppress unreliable, uninformative, or adversarial content. Across recent work, the concept appears in several technically distinct forms: as complement masks and global hard negatives in weakly supervised video moment retrieval, as perturbed masks enforcing invariance in object removal, as proposal distributions over masked positions in language pretraining, as reliability masks that suppress invalid depth features in RGB–D fusion, as destructive masks for adversarial defense, as robustness mechanisms for noisy proposal masks in open-vocabulary segmentation, as complementary frame masks in music representation learning, and as background-oriented complements of dense temporal masks in semi-supervised temporal action detection (Zhang et al., 14 May 2026, Yuan et al., 12 Sep 2025, Chen, 2020, Feng et al., 2023, Xu et al., 2022, Li et al., 2024, Yao et al., 2022, Nag et al., 2022). The unifying principle is not a single architecture but a recurring operational role: a negative mask proposal specifies where matching, trust, or reconstruction should be reduced, discouraged, or made invariant.

1. Conceptual scope and problem formulation

In the most direct formulation, negative mask proposal arises in weakly supervised Video Moment Retrieval (VMR), where the task is to localize a contiguous temporal segment in an untrimmed video that semantically matches a natural-language query, using only video-level correspondences and no temporal boundary annotations during training (Zhang et al., 14 May 2026). In that setting, a positive mask proposal highlights clips most relevant to the query, while negative mask proposals identify clips or regions that should not match the query. MCMT formalizes this with a positive sample mask $\hat m^p$, an easy negative mask $\hat m^e = \mathbf{1} - \hat m^p$, and a hard negative mask $\hat m^h = [1,1,\dots,1]$, together with the assumed semantic ordering

$R(\hat m^p, Q) > R(\hat m^h, Q) > R(\hat m^e, Q).$

This makes negative masks explicit complements or global alternatives to the positive hypothesis (Zhang et al., 14 May 2026).

A broader interpretation appears in object removal. There, the mask $M \in \{0,1\}^{H \times W}$ denotes a region to be removed and inpainted, and “negative” masks are not semantic negatives in the contrastive sense but perturbed masks—dilated or reshaped—that are not the ground-truth segmentation and must not materially change the model’s prediction (Yuan et al., 12 Sep 2025). Under this view, the negative mask proposal becomes a mechanism for enforcing invariance to nuisance variations in mask geometry rather than directly encoding background.

In masked language modeling, the negative aspect is again different. “Variance-reduced Language Pretraining via a Mask Proposal Network” treats mask proposal as importance sampling over masked positions, replacing uniform masking with a learned proposal distribution over mask patterns (Chen, 2020). Here the non-uniform proposal is “negative” in the sense that it intentionally biases against easy or uninformative masking choices, favoring high-loss or high-gradient masks instead.

These variants suggest that negative mask proposal is best understood as an editorial umbrella term for mask constructions that suppress, complement, perturb, or de-emphasize parts of the input so that a model learns stronger discrimination, robustness, or invariance. The exact semantics depend on the task: non-matching regions in VMR, perturbed geometric alternatives in inpainting, hard masked positions in MLM, invalid sensor regions in RGB–D fusion, adversarially suspect patches in classification, or background complements in temporal detection.

2. Negative masks in weakly supervised video moment retrieval

MCMT provides the clearest explicit instance of negative mask proposal in current retrieval literature (Zhang et al., 14 May 2026). The model first builds cross-modal clip–query features with a Transformer encoder–decoder,

$$H = [h_i]_{i=0}^{n_v-1} = D(V, E(Q)) \in \mathbb{R}^{d_h \times n_v},$$

and predicts $k$ proposal candidates parameterized by normalized centers and widths. Each proposal $(c_i,w_i)$ is converted to a Gaussian temporal mask

$$m_{ij} = \exp \left(- \alpha \frac{\big((j+1)/n_v - c_i\big)^2}{w_i^2} \right),$$

yielding $M \in \mathbb{R}^{n_v \times k}$ (Zhang et al., 14 May 2026). Rather than relying on a single proposal, MCMT aggregates the $\hat m^e = \mathbf{1} - \hat m^p$0 Gaussian masks through attention to obtain the positive sample mask

$\hat m^e = \mathbf{1} - \hat m^p$1

Negative masks are then derived deterministically. The easy negative mask is the complement

$\hat m^e = \mathbf{1} - \hat m^p$2

which marks clips suppressed by the positive mask as intra-video negatives. The hard negative mask is the uniform all-ones vector

$\hat m^e = \mathbf{1} - \hat m^p$3

representing the entire video as a noisy negative region (Zhang et al., 14 May 2026). These are negative mask proposals in a literal sense: masks over video clips that propose where the query should not align.

MCMT couples these masks to forward and inverse masked query reconstruction. For a mask $\hat m^e = \mathbf{1} - \hat m^p$4, a mask-conditioned Transformer reconstructs masked forward and inverse queries, producing cross-entropy losses $\hat m^e = \mathbf{1} - \hat m^p$5, $\hat m^e = \mathbf{1} - \hat m^p$6, $\hat m^e = \mathbf{1} - \hat m^p$7, $\hat m^e = \mathbf{1} - \hat m^p$8, and optionally $\hat m^e = \mathbf{1} - \hat m^p$9, $\hat m^h = [1,1,\dots,1]$0 (Zhang et al., 14 May 2026). The total reconstruction loss is

$\hat m^h = [1,1,\dots,1]$1

Easy negatives are not used directly in the reconstruction loss, because they are expected to reconstruct poorly and instead serve primarily in contrastive constraints.

The Intra-Video Contrastive losses enforce margins between positive and negative masks:

$\hat m^h = [1,1,\dots,1]$2

with an analogous inverse-direction term $\hat m^h = [1,1,\dots,1]$3 (Zhang et al., 14 May 2026). Training alternates between updating the reconstructor with $\hat m^h = [1,1,\dots,1]$4 while freezing the mask generator, and updating the mask generator with $\hat m^h = [1,1,\dots,1]$5 while freezing the reconstructor. This separation makes negative masks operationally significant even though $\hat m^h = [1,1,\dots,1]$6 and $\hat m^h = [1,1,\dots,1]$7 are not parameterized independently.

The paper further notes an extension to explicitly learned negative mask proposals: negative Gaussian masks with parameters $\hat m^h = [1,1,\dots,1]$8, aggregated into $\hat m^h = [1,1,\dots,1]$9, or a signed mask $R(\hat m^p, Q) > R(\hat m^h, Q) > R(\hat m^e, Q).$0 with

$R(\hat m^p, Q) > R(\hat m^h, Q) > R(\hat m^e, Q).$1

That proposal is presented as an extension rather than part of the reported MCMT model, so it should be read as a plausible generalization rather than an implemented component (Zhang et al., 14 May 2026).

3. Mask perturbation as negative proposal and invariance constraint

In object removal, Mask Consistency Regularization reinterprets mask variation as a source of hallucination and mask-shape bias (Yuan et al., 12 Sep 2025). The base setup conditions a ControlNet on an image-mask pair $R(\hat m^p, Q) > R(\hat m^h, Q) > R(\hat m^e, Q).$2 and trains an SDXL denoiser with the standard diffusion reconstruction loss

$R(\hat m^p, Q) > R(\hat m^h, Q) > R(\hat m^e, Q).$3

The mask itself is not injected directly into the denoising iterations; rather, it conditions the ControlNet latent $R(\hat m^p, Q) > R(\hat m^h, Q) > R(\hat m^e, Q).$4 (Yuan et al., 12 Sep 2025).

MCR introduces two mask perturbations. The first is dilation,

$R(\hat m^p, Q) > R(\hat m^h, Q) > R(\hat m^e, Q).$5

where $R(\hat m^p, Q) > R(\hat m^h, Q) > R(\hat m^e, Q).$6 is a square structuring element of ones. The second is reshape, either the minimum enclosing rectangle $R(\hat m^p, Q) > R(\hat m^h, Q) > R(\hat m^e, Q).$7 or a random union mask $R(\hat m^p, Q) > R(\hat m^h, Q) > R(\hat m^e, Q).$8 (Yuan et al., 12 Sep 2025). These perturbed masks are not intended to indicate a true negative class. Instead, they are deliberately “worse” or geometrically altered masks that remove exact boundary or shape cues.

The central regularizer is a consistency loss in noise-prediction space:

$R(\hat m^p, Q) > R(\hat m^h, Q) > R(\hat m^e, Q).$9

with total objective

$M \in \{0,1\}^{H \times W}$0

and $M \in \{0,1\}^{H \times W}$1 in the reported experiments (Yuan et al., 12 Sep 2025). Only ControlNet is trained; SDXL remains frozen.

The negative-proposal interpretation is explicit in the paper’s discussion: $M \in \{0,1\}^{H \times W}$2 and $M \in \{0,1\}^{H \times W}$3 are alternative masks that are not the ground-truth segmentation, and the model is told that predictions should be invariant to them (Yuan et al., 12 Sep 2025). Dilation suppresses dependence on exact mask boundaries, while reshape suppresses dependence on mask silhouette. This makes MCR a “negative mask proposal” strategy in the sense of perturbing the conditioning mask so that the network cannot use mask geometry as a shortcut.

The quantitative evidence supports the complementarity of the two perturbations. On BrushBench, the full model reports FID 60.89, PSNR 23.54, SSIM 0.8969, LPIPS 0.1218, and CMMD 0.0741, whereas the variants using only dilation or only reshape are uniformly worse on these metrics (Yuan et al., 12 Sep 2025). On the 300-sample removal test, the full model reports FID 30.35, PSNR 29.69, LPIPS 0.0835, and CMMD 0.0428, again outperforming the single-perturbation variants (Yuan et al., 12 Sep 2025). This suggests that negative mask proposal via perturbation can regularize both context alignment and shape debiasing.

4. Proposal distributions, reliability masks, and suppressive masks across modalities

Negative mask proposal also appears in more indirect but structurally related forms.

In masked language modeling, the Mask Proposal Network reframes masking as importance sampling over masks (Chen, 2020). The key theoretical result is a variance decomposition:

$M \in \{0,1\}^{H \times W}$4

separating mask variance from sentence variance (Chen, 2020). The optimal proposal is proportional to gradient norm,

$M \in \{0,1\}^{H \times W}$5

which is approximated by a learned proposal distribution over positions. This is not a spatial negative mask, but it is a mask-proposal mechanism that deliberately shifts sampling toward hard or high-loss masked positions. A plausible implication is that negative mask proposal can be generalized beyond geometric masking to the distributional selection of masks themselves.

In RGB–D fusion for drivable road and negative obstacle segmentation, AMFNet uses a binary trust mask derived from depth validity:

$M \in \{0,1\}^{H \times W}$6

At stage $M \in \{0,1\}^{H \times W}$7, the depth reliability mask is

$M \in \{0,1\}^{H \times W}$8

and the complementary RGB mask is

$M \in \{0,1\}^{H \times W}$9

In untrusted regions, depth is completely suppressed and RGB receives full weight (Feng et al., 2023). This is a negative mask proposal in the reliability-gating sense: the mask explicitly identifies where a modality should not contribute. The empirical motivation is that, for most compared fusion models, RGB-only outperforms RGB–D fusion on the DRNO dataset, whereas AMFNet’s masked fusion yields mIoU 68.39 and mF1 71.99, improving over its own RGB-only version at mIoU 67.52 and mF1 70.69 (Feng et al., 2023).

A more literal suppressive interpretation appears in adversarial defense. MAD applies random patch-wise input masks during training and inference, masking a grid cell with probability $$H = [h_i]_{i=0}^{n_v-1} = D(V, E(Q)) \in \mathbb{R}^{d_h \times n_v},$$0 and replacing the pixels with a default constant value (Xu et al., 2022). Conceptually,

$$H = [h_i]_{i=0}^{n_v-1} = D(V, E(Q)) \in \mathbb{R}^{d_h \times n_v},$$1

The defense is “negative” because masked regions are destructively removed, partially eliminating adversarial perturbations. The reported gains are large under the evaluated attacks; for example, on CIFAR-10 with VGG16 and PGD $$H = [h_i]_{i=0}^{n_v-1} = D(V, E(Q)) \in \mathbb{R}^{d_h \times n_v},$$2, $$H = [h_i]_{i=0}^{n_v-1} = D(V, E(Q)) \in \mathbb{R}^{d_h \times n_v},$$3, attack accuracy is 2.30% and MAD defense accuracy is 81.45%, an improvement of 79.15 percentage points (Xu et al., 2022). The paper also notes that the attacks are not fully EOT-adaptive, so the robustness should be interpreted accordingly (Xu et al., 2022).

5. Proposal quality, consistency, and complementary masking in segmentation, music, and temporal detection

In open-vocabulary segmentation, Mask-Adapter addresses the instability of mask pooling rather than defining explicit negative masks (Li et al., 2024). Standard pooling forms a mask embedding as

$$H = [h_i]_{i=0}^{n_v-1} = D(V, E(Q)) \in \mathbb{R}^{d_h \times n_v},$$4

which the paper argues is poorly aligned with CLIP semantics (Li et al., 2024). Mask-Adapter instead predicts semantic activation maps from proposal masks and CLIP features, then pools using these learned activation maps:

$$H = [h_i]_{i=0}^{n_v-1} = D(V, E(Q)) \in \mathbb{R}^{d_h \times n_v},$$5

Its mask consistency loss encourages proposals with similar IoUs to yield similar embeddings, using an IoU-based matcher and cosine similarity loss (Li et al., 2024). While there is no explicit negative mask proposal set, low-quality or low-IoU masks are implicitly down-weighted by the training design. This suggests a related conception of negative masks as unstable or misleading proposals whose influence should be reduced.

In music representation learning, PEMR constructs explicit positive and negative frame masks from Transformer-derived importance scores (Yao et al., 2022). The positive mask keeps important frames and zeroes low-score frames, while the negative mask is its complement,

$$H = [h_i]_{i=0}^{n_v-1} = D(V, E(Q)) \in \mathbb{R}^{d_h \times n_v},$$6

Applied to a Log-Mel spectrogram branch, this yields

$$H = [h_i]_{i=0}^{n_v-1} = D(V, E(Q)) \in \mathbb{R}^{d_h \times n_v},$$7

The negative contrastive loss

$$H = [h_i]_{i=0}^{n_v-1} = D(V, E(Q)) \in \mathbb{R}^{d_h \times n_v},$$8

penalizes correlation between the original view and the negative view, making the latter a counterfactual example consisting of inessential or noisy frames (Yao et al., 2022). On MTAT linear evaluation, the baseline without masks reports ROC-AUC 89.1 and PR-AUC 36.2, positive mask only reports 89.4 and 36.6, and full PEMR reports 89.6 and 36.9 (Yao et al., 2022). This makes negative mask proposal an explicit self-augmented negative construction.

In semi-supervised temporal action detection, SPOT predicts dense foreground masks over time rather than sparse proposals (Nag et al., 2022). The localization head outputs

$$H = [h_i]_{i=0}^{n_v-1} = D(V, E(Q)) \in \mathbb{R}^{d_h \times n_v},$$9

where each column is a foreground probability mask over the timeline. The complement of this foreground mask naturally defines background or negative regions, and the mask loss includes both positive and negative terms through weighted BCE and Dice overlap (Nag et al., 2022). SPOT further extracts hard foreground and hard background snippets via thresholding and erosion, then aligns them with easy foreground and background from the classification stream using a refinement loss. The paper does not name this a “negative mask,” but the complement of the foreground mask is operationally a negative mask proposal over time. The practical payoff is reduced proposal error propagation relative to proposal-based SS-TAD; under ActivityNet with 10% labels, BMN drops from 45.3 avg mAP with ground-truth proposals to 23.6 with pseudo proposals, while SPOT drops from 47.0 with ground-truth masks to 32.1 with pseudo masks (Nag et al., 2022).

6. Design patterns, misconceptions, and research directions

Across these systems, several recurring design patterns define the present landscape of negative mask proposal.

Pattern	Mechanism	Representative work
Complement masks	Derive negatives as $k$0 or as background complement	MCMT (Zhang et al., 14 May 2026), PEMR (Yao et al., 2022), SPOT (Nag et al., 2022)
Global hard masks	Treat the full input or full sequence as a noisy negative	MCMT (Zhang et al., 14 May 2026)
Perturbed masks	Use dilated or reshaped masks as alternative non-ground-truth proposals	MCR (Yuan et al., 12 Sep 2025)
Reliability masks	Zero out untrustworthy modality regions and reweight the complement	AMFNet (Feng et al., 2023)
Suppressive random masks	Stochastically destroy regions to reduce adversarial effect	MAD (Xu et al., 2022)
Proposal-distribution masks	Learn non-uniform sampling over mask patterns or positions	MAP-Net (Chen, 2020)

A common misconception is that negative mask proposal necessarily means learning a separate set of negative regions with dedicated parameters. Several systems derive negatives directly from a positive mask, as in $k$1 in MCMT (Zhang et al., 14 May 2026) or $k$2 in PEMR (Yao et al., 2022). Another misconception is that negative masks must encode semantic background only. MCR shows that “negative” masks can instead be perturbed conditionings used to enforce invariance to shape or boundary changes (Yuan et al., 12 Sep 2025).

A second recurring theme is that negative masks often function best when coupled to an auxiliary criterion stronger than plain classification. In MCMT, they act through forward and inverse reconstruction plus intra-video contrastive margins (Zhang et al., 14 May 2026). In MCR, they act through consistency regularization in denoising space (Yuan et al., 12 Sep 2025). In PEMR, they form counterfactual negatives inside a Barlow Twins-style objective (Yao et al., 2022). In SPOT, background complements are integrated into weighted BCE, Dice, and foreground/background refinement (Nag et al., 2022). This suggests that negative mask proposals are most effective when they induce structured constraints on representation or localization, not merely when they are provided as additional labels.

The research directions described in the source materials point toward more explicit and learnable formulations. MCMT sketches learnable negative Gaussian masks and signed masks as natural extensions (Zhang et al., 14 May 2026). MCR suggests broader mask perturbation families, including affine transforms, erosion, adversarial masks, and curriculum over perturbation strength (Yuan et al., 12 Sep 2025). Mask-Adapter suggests explicit handling of low-IoU masks through richer training on overlapping proposals, even though it stops short of explicit negative sampling (Li et al., 2024). SPOT suggests using background-oriented complements of dense temporal masks as explicit negative proposals for filtering and hard negative mining, although that extension is not part of the reported model (Nag et al., 2022).

Taken together, these works indicate that negative mask proposal is not a single established canonical module but a convergent methodological motif. It appears whenever a model must learn not only where evidence is, but also where evidence should be discounted, ignored, perturbed, or contrasted. In that sense, the concept spans retrieval, reconstruction, self-supervision, multimodal fusion, robustness, and dense prediction, with the most explicit current instantiation being complement and hard negative masks in weakly supervised video moment retrieval (Zhang et al., 14 May 2026).