Masked Boundary Modeling

• Masked boundary modeling is a design principle that leverages explicit and implicit boundary cues to enhance reconstruction and segmentation tasks.
• It employs structured-noise masking, explicit contour supervision, and mask-conditioned distance learning to improve spatial and temporal coherence.
• Empirical results across video, audio, medical imaging, and CFD applications demonstrate sharper boundary localization and improved performance metrics.

Searching arXiv for the cited papers and closely related work on masked boundary modeling. Masked boundary modeling denotes a family of methods in which masks, masked reconstruction, or mask prediction are made sensitive to boundary structure rather than treated as boundary-agnostic token dropping or pixel-wise classification. In recent work, this idea appears in several technically distinct forms: structured-noise masking for video and audio masked modeling, mutual learning of masks and contours in instance segmentation, sparse anatomical boundary prompts for CBCT tooth segmentation, mask-conditioned signed-distance learning for few-shot industrial segmentation, discrete unmasking of segmentation fields in discrete-state models, and physical-domain mask recovery for CNN-ready CFD fields (Bhowmik et al., 20 Mar 2025, Cheng et al., 2020, Dai et al., 2024, Yoshinaga et al., 19 Jun 2026, Hu et al., 2024, Sharifi et al., 17 Feb 2026). Taken together, these formulations suggest that masked boundary modeling is better understood as a design principle than as a single architecture.

1. Scope and technical variants

A common thread across these works is the claim that boundaries encode structure that is lost when masking or segmentation is performed without explicit geometric, spatial, temporal, or spectral constraints. In video and audio self-supervision, the issue is that random masking ignores spatial coherence, temporal continuity, and spectral structure (Bhowmik et al., 20 Mar 2025). In instance segmentation, the issue is that a mask head based on pixel-wise classification produces coarse and indistinct mask prediction results because object contours are not explicitly modeled (Cheng et al., 2020). In CBCT tooth segmentation, boundary blurring and category confusion are aggravated by limited labels and closely positioned teeth (Dai et al., 2024). In industrial few-shot segmentation, weak textures and ambiguous instance definitions make interior appearance less reliable than contours (Yoshinaga et al., 19 Jun 2026). In discrete-state unmasking, boundaries are not separately supervised but emerge from reconstruction of masked label fields (Hu et al., 2024). In CFD preprocessing, the central problem is not semantic segmentation but recovery of a physically consistent mask that suppresses unsupported grid cells after interpolation (Sharifi et al., 17 Feb 2026).

Setting	Boundary representation	Function of masking
Video and audio masked modeling	Structured red, blue, or green noise masks	Modality-aware occlusion
Instance segmentation	Explicit boundary branch	Contour-guided mask localization
CBCT tooth segmentation	Sparse anatomical boundary prompts	Boundary-guided masked pretraining
Few-shot industrial segmentation	Signed Distance Function	Mask-conditioned contour learning
Discrete-state segmentation	$[MASK]$ token fields	Progressive unmasking of label fields
CFD preprocessing	Physical-domain binary mask	Suppression of unsupported regions

This variety matters because the phrase “boundary modeling” does not always imply the same mechanism. In some settings, boundary signals are explicit supervisory targets; in others, they are encoded in the mask generator, in a distance field, or in the topology of masked and unmasked states. A plausible implication is that the main technical distinction is between explicit contour variables and implicit contour recovery.

2. Structured-noise masking in masked modeling

In "Structured-Noise Masked Modeling for Video, Audio and Beyond" (Bhowmik et al., 20 Mar 2025), masked boundary modeling is implemented as a data-independent masking prior derived from filtered white noise. Given an input $X$, the pipeline first patchifies and embeds it,

$X_p = \phi(X),$

then produces a binary mask with masking function $\eta$ at ratio $\gamma$,

$M = \eta(X_p, n, \gamma),$

with visible and masked tokens obtained by element-wise application of $M$. The method replaces white-noise-driven random masks with structured noise obtained by Gaussian filtering: $$n_r = G_{\sigma} * n_w,\qquad n_b = n_w - (G_{\sigma} * n_w),$$

$$n_g = G_{\sigma_1} * n_w - G_{\sigma_2} * n_w,\qquad \sigma_1 < \sigma_2.$$

These generate red, blue, and green masks through the same masking generator $\eta$. Red noise is low-frequency and smooth, blue noise is high-frequency and fine-grained, and green noise occupies a mid-frequency regime.

For video, the paper extends green noise to 3D so that masks evolve smoothly across frames rather than being static over time. The resulting $X$0 preserves temporal continuity and is used as a drop-in replacement in VideoMAE and SIGMA. For audio, the paper uses 2D optimized blue noise over spectrogram patches. Because simple blue noise was not sufficient for uniformly distributed visible spectrogram patches, the method adds an optimization over multiple masks using a local clustering score

$X$1

then selects $X$2, producing an optimized blue-noise mask $X$3 with less clustering. In the audio-visual setting, Green3D is used for video and Optim Blue for audio during CAV-MAE pretraining (Bhowmik et al., 20 Mar 2025).

The empirical pattern is consistent across modalities. On Something-Something V2, VideoMAE improves from $X$4 to $X$5 and SIGMA from $X$6 to $X$7. On Kinetics-400, VideoMAE improves from $X$8 to $X$9 and SIGMA from $X_p = \phi(X),$0 to $X_p = \phi(X),$1. On DAVIS clustering, VideoMAE improves from $X_p = \phi(X),$2 to $X_p = \phi(X),$3, and on YTVOS clustering from $X_p = \phi(X),$4 to $X_p = \phi(X),$5. For audio classification, AudioSet-20K improves from $X_p = \phi(X),$6 to $X_p = \phi(X),$7, AudioSet-2M from $X_p = \phi(X),$8 to $X_p = \phi(X),$9, and ESC-50 from $\eta$0 to $\eta$1. On VGGSound, audio-only improves from $\eta$2 to $\eta$3, video-only from $\eta$4 to $\eta$5, and audio-video from $\eta$6 to $\eta$7 (Bhowmik et al., 20 Mar 2025).

The ablations clarify the boundary-aware interpretation. For video, blue noise is reported as too easy, with low reconstruction loss and weaker accuracy; red noise is too hard, with high reconstruction loss and weaker accuracy; green noise gives the best balance. On mini-Kinetics and mini-SSv2, the accuracies are Random $\eta$8, Blue $\eta$9, Red $\gamma$0, and Green $\gamma$1. For audio, blue noise performs best, with AS-20K $\gamma$2 and ESC-50 $\gamma$3. The best mask ratios remain the standard ones, $\gamma$4 for video and $\gamma$5 for audio. This supports the paper’s claim that structured masks can improve representation learning without requiring new mask-ratio tuning or computational overhead (Bhowmik et al., 20 Mar 2025).

3. Explicit contour supervision in instance segmentation

"Boundary-preserving Mask R-CNN" (Cheng et al., 2020) treats the object boundary as a dual representation of the object mask and incorporates it directly into the mask head. The architecture replaces the standard Mask R-CNN mask branch with a boundary-preserving mask head containing a boundary branch and two feature fusion blocks. Let $\gamma$6 denote RoI features for mask prediction and $\gamma$7 boundary RoI features. The mask features are passed through four $\gamma$8 convolutions to produce $\gamma$9. Boundary features are extracted from the finest-resolution level $M = \eta(X_p, n, \gamma),$0 at larger resolution and then downsampled. The mask-to-boundary fusion is defined as

$M = \eta(X_p, n, \gamma),$1

where $M = \eta(X_p, n, \gamma),$2 is a $M = \eta(X_p, n, \gamma),$3 convolution followed by ReLU. A boundary-to-mask fusion of the same form injects contour information back into the mask stream. Both mask and boundary predictions are class-specific and use the conventional $M = \eta(X_p, n, \gamma),$4 deconvolution followed by a $M = \eta(X_p, n, \gamma),$5 output layer.

Boundary supervision is generated from binary mask ground truth using the Laplacian operator, then binarized using threshold $M = \eta(X_p, n, \gamma),$6. The boundary loss combines Dice loss and BCE,

$M = \eta(X_p, n, \gamma),$7

and the full objective is

$M = \eta(X_p, n, \gamma),$8

The paper’s stated rationale is that BCE gives precise localization, Dice handles strong foreground/background imbalance, and the combination yields crisp boundaries without excessive thickness (Cheng et al., 2020).

The reported gains are concentrated on localization-sensitive evaluation. On COCO val2017, ResNet-50-FPN improves from $M = \eta(X_p, n, \gamma),$9 AP to $M$0 AP, ResNet-101-FPN from $M$1 to $M$2, and HRNetV2-W32-FPN from $M$3 to $M$4. The gains are larger at $M$5, where improvements are about $M$6 AP in several settings. On COCO test-dev2017 with ResNet-101-FPN, Mask R-CNN improves from $M$7 AP to $M$8 AP, and with Mask Scoring the boundary-preserving variant reaches $M$9 AP. On Cityscapes, where contour annotations are more accurate, the gains are larger: test set $$n_r = G_{\sigma} * n_w,\qquad n_b = n_w - (G_{\sigma} * n_w),$$0 AP and validation set $$n_r = G_{\sigma} * n_w,\qquad n_b = n_w - (G_{\sigma} * n_w),$$1 AP (Cheng et al., 2020).

The ablations show that the improvement is tied to boundary modeling rather than to capacity alone. Replacing boundary supervision with mask supervision reduces AP from $$n_r = G_{\sigma} * n_w,\qquad n_b = n_w - (G_{\sigma} * n_w),$$2 to $$n_r = G_{\sigma} * n_w,\qquad n_b = n_w - (G_{\sigma} * n_w),$$3; removing both fusion blocks reduces AP to $$n_r = G_{\sigma} * n_w,\qquad n_b = n_w - (G_{\sigma} * n_w),$$4; using only M2B yields $$n_r = G_{\sigma} * n_w,\qquad n_b = n_w - (G_{\sigma} * n_w),$$5, only B2M $$n_r = G_{\sigma} * n_w,\qquad n_b = n_w - (G_{\sigma} * n_w),$$6, and both together $$n_r = G_{\sigma} * n_w,\qquad n_b = n_w - (G_{\sigma} * n_w),$$7. Dice+BCE is better than BCE, weighted BCE, or Dice alone. A larger mask head with similar computation cost reaches $$n_r = G_{\sigma} * n_w,\qquad n_b = n_w - (G_{\sigma} * n_w),$$8 AP, while BMask R-CNN reaches $$n_r = G_{\sigma} * n_w,\qquad n_b = n_w - (G_{\sigma} * n_w),$$9. A Sobel-derived boundary variant reaches $$n_g = G_{\sigma_1} * n_w - G_{\sigma_2} * n_w,\qquad \sigma_1 < \sigma_2.$$0 AP, improving over Mask R-CNN but remaining $$n_g = G_{\sigma_1} * n_w - G_{\sigma_2} * n_w,\qquad \sigma_1 < \sigma_2.$$1 AP below full BMask R-CNN (Cheng et al., 2020). This directly supports the view that explicit contour reasoning sharpens high-IoU mask localization.

4. Sparse anatomical prompts in CBCT masked pretraining

"Sparse Anatomical Prompt Semi-Supervised Learning with Masked Image Modeling for CBCT Tooth Segmentation" (Dai et al., 2024) combines masked autoencoder pretraining with a boundary-aware prompt branch designed for tooth anatomy. The target problem is multi-class tooth segmentation in CBCT with very limited labeled data and abundant unlabeled scans. The paper emphasizes that tooth boundaries are very close together, inter-class appearance differences are subtle, CBCT intensity is non-uniform and unevenly distributed, and vanilla masked image modeling tends to yield blurred boundaries under these conditions.

The method uses a three-stage pipeline. First, a graph attention-based sparse prompt branch is trained on sparse boundary annotations and then frozen. Second, a U-NetR/MAE-style encoder-decoder is pretrained on unlabeled CBCT images, with the frozen prompt branch supplying boundary prompts or masked tokens. Third, the pretrained weights initialize a downstream segmentation network fine-tuned with supervised losses. Tooth relationships are encoded as a graph $$n_g = G_{\sigma_1} * n_w - G_{\sigma_2} * n_w,\qquad \sigma_1 < \sigma_2.$$2 defined by the ISO tooth numbering criterion, with $$n_g = G_{\sigma_1} * n_w - G_{\sigma_2} * n_w,\qquad \sigma_1 < \sigma_2.$$3. The graph attention update for node $$n_g = G_{\sigma_1} * n_w - G_{\sigma_2} * n_w,\qquad \sigma_1 < \sigma_2.$$4 is

$$n_g = G_{\sigma_1} * n_w - G_{\sigma_2} * n_w,\qquad \sigma_1 < \sigma_2.$$5

with attention coefficients normalized over neighbors and the self-node. Sparse boundary labels are used instead of dense masks, and Tversky loss is chosen for this branch because positives are sparse and imbalanced (Dai et al., 2024).

The masked pretraining stage reconstructs masked content under boundary guidance. With input patches $$n_g = G_{\sigma_1} * n_w - G_{\sigma_2} * n_w,\qquad \sigma_1 < \sigma_2.$$6, the masking formula is given as

$$n_g = G_{\sigma_1} * n_w - G_{\sigma_2} * n_w,\qquad \sigma_1 < \sigma_2.$$7

and the reconstruction objective is MSE. The paper’s stated interpretation is that boundary-aware masked tokens force the encoder-decoder to learn anatomical structure rather than generic texture. Fine-tuning then uses

$$n_g = G_{\sigma_1} * n_w - G_{\sigma_2} * n_w,\qquad \sigma_1 < \sigma_2.$$8

which addresses class imbalance, foreground-background separation, and overlap quality (Dai et al., 2024).

The semi-supervised setup is explicit: the sparse prompt branch is trained on $$n_g = G_{\sigma_1} * n_w - G_{\sigma_2} * n_w,\qquad \sigma_1 < \sigma_2.$$9 cases with sparse boundary annotations; masked pretraining uses native CBCT images as unlabeled data; supervised fine-tuning uses either $\eta$0 full-label cases or $\eta$1 half-label cases. In the full labeled setting, UNetR reaches DSC $\eta$2, MAE $\eta$3, SimCLR $\eta$4, SimMIM $\eta$5, and the proposed method $\eta$6. In the half labeled setting, the corresponding values are $\eta$7, $\eta$8, $\eta$9, $X$00, and $X$01. The method also achieves the best or near-best boundary-sensitive metrics, especially Jaccard, Precision, Recall, and HD (Dai et al., 2024).

This formulation is notable because it uses sparse boundary information not merely as an auxiliary output but as a prompt that shapes masked reconstruction itself. A plausible implication is that it occupies a middle ground between explicit boundary supervision and self-supervised masked pretraining.

5. Mask-conditioned signed-distance learning

"Boundary-by-Mask: Few-Shot Instance Segmentation with Mask-Conditioned Boundary Learning for Texture-Poor Industrial Parts" (Yoshinaga et al., 19 Jun 2026) recasts few-shot instance segmentation as boundary prediction conditioned by the reference mask rather than appearance matching. The paper’s motivation is domain-specific: industrial parts and food items often have weak textures, color-uniform surfaces, limited contextual cues, and application-dependent instance definitions. Under those conditions, the paper argues that “the boundary rather than the interior appearance contains the most discriminative information.”

The method takes a few reference RGB images $X$02, reference instance masks $X$03, and a query RGB image $X$04. DINOv3 features are used for object-feature collection and query localization, while a frozen SAM2 encoder provides dense features for a shallow shared pixel-wise MLP SDF head. The reference binary masks are converted to signed distance maps. The SDF is defined as zero on the object boundary, positive inside the object, negative outside the object, and truncated and normalized to $X$05. The training target is optimized with Smooth-$X$06,

$X$07

During inference, the predicted SDF is thresholded into seed points,

$X$08

and combined with image gradients in an energy map

$X$09

followed by watershed reconstruction (Yoshinaga et al., 19 Jun 2026).

A defining property is that the instance definition is conditioned by the reference mask. If the support mask covers the whole object, the learned SDF describes the whole-object boundary; if it covers a sub-part, the same mechanism learns the sub-part boundary. The paper formalizes offline threshold selection by maximizing

$X$10

where $X$11 measures instance-count consistency, $X$12 is boundary F1, $X$13 is union IoU, and $X$14 penalizes fragmentation. This makes the seed threshold data-driven and aligned with the reference masks (Yoshinaga et al., 19 Jun 2026).

The reported experiments are on a custom industrial dataset with five categories—nuts, screws, tubes, fried chicken, and plastic bottles—with both object-level and part-level annotations. The method outperforms SAM2, PerSAM, PerSAM-F, No time to train!, and YOLOv11 in most low-shot cases. At $X$15, the paper reports AP $X$16, AP50 $X$17, AP75 $X$18, and mIoU $X$19. At $X$20, it reports AP $X$21, AP50 $X$22, AP75 $X$23, and mIoU $X$24. The paper notes that YOLOv11 fine-tuned with more reference data slightly overtakes BbM in AP at $X$25, suggesting that full-network scene-specific training can become advantageous when enough examples are available. A key ablation finds that direct mask supervision is weak, edge supervision is slightly better, and SDF supervision is much stronger (Yoshinaga et al., 19 Jun 2026).

This approach places boundary modeling at the center of few-shot adaptation. It also makes clear that the supervisory role of the mask can be stronger than standard label provision: the mask determines the target geometry itself.

6. Implicit unmasking and geometric mask reconstruction

"[MASK] is All You Need" (Hu et al., 2024) offers a different formulation in which mask transitions are explicit but boundaries are implicit. The framework, called Discrete Interpolants, introduces $X$26 as a legitimate discrete state and defines generation as interpolation from a fully masked sequence $X$27 to a fully observed sequence $X$28: $X$29 with schedules such as linear $X$30, cosine, root, and quadratic. The model learns the reverse conditional $X$31 under a cross-entropy objective applied only at masked positions, with timestep weighting $X$32. The paper reports that masking the cross-entropy loss is crucial to avoid overfitting, and that $X$33 works better than the ELBO-derived weighting for image generation. Segmentation is recast as masked-state completion by concatenating image and segmentation tokens,

$X$34

The central claim is therefore not explicit contour learning, but progressive recovery of spatial label structure from partially masked discrete states (Hu et al., 2024).

The empirical evidence spans generative and discriminative settings. On MS-COCO, the Implicit Timestep Model achieves FID $X$35 and the Explicit Timestep Model FID $X$36. On ImageNet 256, the reported FIDs are $X$37 and $X$38. On FaceForensics, Latte reports Frame-FID $X$39 and FVD $X$40, while the Implicit Timestep Model reports Frame-FID $X$41 and FVD $X$42. On Cityscapes, the joint image-segmask model reaches FID $X$43, mIoU $X$44 with explicit timestep and FID $X$45, mIoU $X$46 with implicit timestep (Hu et al., 2024). In this setting, contour formation is an emergent property of unmasking discrete label fields.

A different but related use of mask recovery appears in "Novel distance-based masking and adaptive alpha-shape methods for CNN-ready reconstruction of arbitrary 2D CFD flow domains" (Sharifi et al., 17 Feb 2026). Here the task is to reconstruct a physically consistent binary mask after interpolating irregular CFD data onto a uniform Cartesian grid. The distance-based method classifies each grid node $X$47 by nearest-sample distance,

$X$48

then applies the rule

$X$49

with the recommended default $X$50, followed by morphological closing. The adaptive alpha-shape defines a normalized scale

$X$51

and retains simplices with $X$52. The paper reports that the distance-based method is robust across the tested geometries under the same threshold rule, achieves $X$53–$X$54 times speedups over classical alpha-shapes, and that the adaptive alpha-shape is stable at $X$55 while being $X$56–$X$57 times faster than the classical variant. A minimal boundary inflation step improves retention by up to $X$58 with unsupported activation less than $X$59 (Sharifi et al., 17 Feb 2026).

These two lines of work are technically distant, but they share a useful abstraction: a mask can be the object of reconstruction rather than merely a supervisory annotation. In one case, the mask defines the path of discrete-state completion; in the other, it defines the physically admissible computational domain.

7. Misconceptions, limitations, and open questions

One recurring misconception is that masked modeling is automatically boundary-aware. The structured-noise results show the opposite: random masking is effective but modality-agnostic, and respecting spatial, temporal, and spectral organization improves both standard and advanced masked modeling methods (Bhowmik et al., 20 Mar 2025). A second misconception is that better boundary localization in instance segmentation can be obtained simply by enlarging the mask head. The BMask R-CNN ablations reject this: a larger mask head with similar computation cost remains below the explicitly boundary-preserving design (Cheng et al., 2020). A third misconception is that segmentation by unmasking necessarily entails explicit contour variables. In Discrete Interpolants, mask transitions are explicit but boundaries remain implicit in the recovered token field (Hu et al., 2024).

The limitations are correspondingly heterogeneous. In structured-noise masking, the best noise color is modality-specific—green for video and blue for audio—so there is no universal structured-noise choice; performance depends on the spectral parameters $X$60; the paper provides empirical evidence rather than a formal optimality guarantee; and experiments substantially cover video, audio, and audio-visual learning but not text, point clouds, or medical signals (Bhowmik et al., 20 Mar 2025). In Boundary-by-Mask, evaluation is mostly on simplified backgrounds, and the paper explicitly notes possible issues with clutter, severe occlusion, and illumination changes, while suggesting uncertainty-aware reference selection, temporal cues, or multi-view cues for future work (Yoshinaga et al., 19 Jun 2026). In the CFD setting, the adaptive alpha-shape is presented as a strong alternative when grid-spacing information for threshold selection is unavailable, which implies that the distance-based default depends on access to source-resolution information (Sharifi et al., 17 Feb 2026).

Taken together, these results suggest that masked boundary modeling is not defined by a particular loss or architecture, but by where boundary information enters the pipeline. It may enter at mask generation time, through explicit contour supervision, through sparse prompts, through distance-field regression, through discrete-state unmasking, or through geometric reconstruction of the valid domain. The major open question across these formulations is not whether boundaries matter—the reported results consistently indicate that they do—but which representation of boundary information is most appropriate for a given modality, annotation regime, and inference constraint.