MAUDS: Mask-Aware Up/Down-Sampler

• The paper introduces MAUDS, a module that maintains strong mask sensitivity across resolution scales via explicit channel-wise feature selection and mask-guided fusion.
• It integrates into the CMAMRNet architecture to preserve fine structural details and clean boundaries by keeping mask signals aligned with feature hierarchies.
• Empirical results demonstrate that MAUDS improves restoration quality—boosting metrics like PSNR and SSIM—by overcoming the degradation of mask information in standard up/down-sampling.

The Mask-Aware Up/Down-Sampler (MAUDS) is a module introduced in CMAMRNet, a Contextual Mask-Aware Mural Restoration Network for digital mural restoration. In that framework, MAUDS is one of two key components, alongside the Co-Feature Aggregator (CFA), and is designed to ensure consistent mask sensitivity across resolution scales through dedicated channel-wise feature selection and mask-guided feature fusion. MAUDS addresses a specific limitation of learning-based restoration systems: although many methods provide a degradation mask at the input, mask information can deteriorate as features pass through deeper layers, especially across down-/up-sampling, which weakens focus on damaged regions and compromises restoration quality (Lei et al., 10 Aug 2025).

1. Concept and motivation

MAUDS was proposed for digital mural restoration, where the location and extent of degradation must remain explicitly available throughout the network. The stated motivation is that most prior works provide the degradation mask at the input, but mask information deteriorates as features pass through deeper layers, especially across resolution transitions, causing the network to lose focus on damaged regions. The reported consequence is less faithful or blurry restorations (Lei et al., 10 Aug 2025).

Within this formulation, MAUDS is designed to maintain strong mask awareness at all spatial scales, so that the model consistently attends to damaged regions during both encoding (downsampling) and decoding (upsampling). Its defining distinction from standard resolution-changing operators is that it performs explicit channel-wise feature-masking and fusion during resolution changes, whereas conventional up/down-sampling simply rescales features and ignores mask structure (Lei et al., 10 Aug 2025).

A common misconception is that mask-awareness is fully achieved once the mask is concatenated with the input. The MAUDS formulation rejects that assumption: the core claim is that mask guidance must survive repeated resolution changes, and that this persistence requires explicit module-level propagation rather than input-level injection alone (Lei et al., 10 Aug 2025).

2. Internal organization

MAUDS comprises two symmetric modules:

Module	Role	Core mechanism
MAUS	Upsampling	Channel-wise feature selection and mask-guided feature fusion
MADS	Downsampling	Channel-wise feature interleaving and mask-guided feature fusion

Both modules are used at each resolution transition in the U-Net-like CMAMRNet backbone. Their shared base is a Sampling Block consisting of a resolution-changing operation (upsample or downsample) and channel shuffle, the latter described as promoting inter-channel information flow. This block is applied separately to feature maps $F_{img}$ and the mask $M$, so that masks are explicitly propagated and aligned at every scale (Lei et al., 10 Aug 2025).

This design places MAUDS at the specific point where ordinary encoder–decoder architectures are most likely to dilute mask semantics. In CMAMRNet, the module is therefore not an auxiliary refinement layer, but the mechanism that governs how mask information survives the network hierarchy (Lei et al., 10 Aug 2025).

3. Mask-Aware Up-Sampler and Mask-Aware Down-Sampler

For the Mask-Aware Up-Sampler (MAUS), the inputs are image features $F_{img} \in \mathbb{R}^{H \times W \times C}$ and a mask $M \in \mathbb{R}^{H \times W \times 1}$. The process is specified in four stages:

1. Feature upsampling

$$F^{up}_{img} = \phi_{up}(F_{img}) \in \mathbb{R}^{2H \times 2W \times C}$$

2. Mask upsampling

$$M^{up} = \xi_{up}(\omega(M)) \in \mathbb{R}^{2H \times 2W \times C/2}$$

The description states that the mask is transformed with a convolution, then spatially upsampled via pixel shuffle, and promoted to a multi-channel tensor.

3. Channel-wise feature selection

$$F^{select}_{img} = \{F^{up}_{img,i} \mid i=1,3,\ldots,C\} \in \mathbb{R}^{2H \times 2W \times C/2}$$

4. Mask-guided feature fusion

$F^{fusion} = F^{select}_{img} + M^{up}$

$$F^{out}_{img} = \gamma(F^{fusion}) \in \mathbb{R}^{2H \times 2W \times C/2}$$

The final operator $\gamma$ is a depth-wise convolution used for local mixing and refinement (Lei et al., 10 Aug 2025).

For the Mask-Aware Down-Sampler (MADS), the same variables are used at higher resolution. Its process is likewise stated explicitly:

5. Feature downsampling

$M$0

6. Mask downsampling

$M$1

7. Channel-wise feature interleaving

$M$2

$M$3

where $M$4, yielding

$M$5

8. Mask-guided feature fusion

$M$6

The asymmetry between selection in MAUS and interleaving in MADS is part of the stated design. In MAUS, every other channel, specifically the odd indices, is retained to align with the upsampled mask’s channel count. In MADS, the mask is interleaved with downsampled image features before refinement. In both directions, the decisive operation is the explicit fusion of feature channels with mask channels prior to depth-wise convolution (Lei et al., 10 Aug 2025).

4. Channel-wise masking and multi-scale behavior

Two technical ideas are emphasized in the description of MAUDS. The first is channel-wise feature selection. In MAUS, retaining every other channel is said to promote feature sparsity and efficient mask coupling, ensuring that learned features relevant to damaged regions are preserved and guided by the mask. The second is mask-guided feature fusion, in which mask channels are integrated directly into image features after mask transformation, preserving explicit, strong mask signals (Lei et al., 10 Aug 2025).

The depth-wise convolution following fusion is described as helping blend local mask and feature context without smearing non-damaged context. This is a local refinement step rather than a generic smoothing stage, and its function is explicitly tied to preserving region specificity around degraded areas (Lei et al., 10 Aug 2025).

MAUDS modules are placed at every scale transition in the U-Net encoder/decoder. In the downsampling path, the module guides how context should be summarized at coarser scales with the mask. In the upsampling path, it restores mask focus as features return to higher resolution, which is described as crucial for fine structure recovery in murals. The mural image and degradation mask are processed in parallel through the hierarchy, and at each scale the mask’s influence remains strong, preventing the network from forgetting the damaged region even after many layers (Lei et al., 10 Aug 2025).

5. Integration within CMAMRNet and empirical effects

In CMAMRNet, MAUDS is used at all encoder–decoder transition points and works in tandem with Transformer-based Restormer blocks and with the Co-Feature Aggregator (CFA) at the highest and lowest resolutions. The stated purpose of this integration is to preserve focus on the mask area throughout the entire pipeline, even as features become coarser or finer spatially (Lei et al., 10 Aug 2025).

The paper contrasts MAUDS with standard up/down-sampling in typical U-Nets, such as stride-2 conv/pool for downsampling and bilinear/nearest upsample or transposed conv for upsampling. These standard operations are said to ignore mask guidance after initial input, so mask significance fades layer by layer. By contrast, MAUDS explicitly preserves mask information across transitions, performs feature–mask channel fusion and interleaving, and ensures that channel allocation always reflects current mask structure and learned mask–feature correlations (Lei et al., 10 Aug 2025).

The reported restoration effects are specific. Conventional methods are described as sometimes overly blur[ring] repairs or leak[ing] artifact colors/structures into undamaged regions. With MAUDS, CMAMRNet is reported to maintain strong, scale-consistent focus on the mask area, yielding:

• More structurally and visually faithful mural repairs
• Better preservation of fine lines, cracks, and painterly textures
• Cleaner boundaries between inpainted regions and intact background

The ablation evidence summarized from Table 2 states that without MAUDS, PSNR drops (from 36.26 → 35.45), SSIM drops, and MAE and LPIPS worsen, with the accompanying interpretation that finer details [are] lost. The qualitative comparison summarized from Figure 1 states that CMAMRNet with MAUDS yields more plausible reconstructions with clean boundaries, authentic color/texture continuity, and avoids the “washed out” artifacts of standard upsampling (Lei et al., 10 Aug 2025).

6. Relation to broader mask-aware and learned-sampling research

MAUDS belongs to a broader class of methods that treat the mask as an active computational signal rather than a passive input annotation. In “$M$7-Transformer for Mask-Aware Hyperspectral Image Reconstruction”, the mask-aware strategy uses the mask-encoded prediction as an uncertainty estimator and adaptively prioritizes the loss penalty for harder masked pixels, with a theoretical discussion distinguishing convergence tendencies in masked and unmasked regions (Wang et al., 2022). In “PMA-Diffusion: A Physics-guided Mask-Aware Diffusion Framework for TSE from Sparse Observations”, mask-awareness is present in both training and inference through Single-Mask and Double-Mask strategies and a posterior sampler that alternates reverse-diffusion updates, observation projection, and physics-guided projection while preserving observed entries (Liu et al., 5 Dec 2025). These examples indicate that mask-aware design can operate at the level of loss construction, prior learning, posterior sampling, or architectural transitions.

A separate but relevant line of work is “Learning Affinity-Aware Upsampling for Deep Image Matting”, which formulates upsampling through learned affinity and shows that the operator can also be extended to downsampling (Dai et al., 2020). That work is not mask-aware in the MAUDS sense, but it demonstrates a related principle: resolution changes can be made content-adaptive rather than treated as fixed rescaling. A plausible implication is that MAUDS can be understood as a specialized content-adaptive sampler in which the conditioning signal is not generic affinity alone, but the explicit degradation mask.

Within this broader context, the distinguishing property of MAUDS is narrow and specific: it is a resolution-transition module for mural restoration that keeps the degradation mask explicitly aligned with feature hierarchies through separate sampling of features and mask, channel-wise selection or interleaving, and depth-wise mask-guided fusion. Its reported benefit is not merely improved numerical performance, but preservation of mask sensitivity across scales, especially where fine damaged structures must be reconstructed without contaminating intact regions (Lei et al., 10 Aug 2025).