Weakly Supervised Camouflaged Object Detection Based on the SAM Model and Mask Guidance

Abstract: Camouflaged object detection (COD) from a single image is a challenging task due to the high similarity between objects and their surroundings. Existing fully supervised methods require labor-intensive pixel-level annotations, making weakly supervised methods a viable compromise that balances accuracy and annotation efficiency. However, weakly supervised methods often experience performance degradation due to the use of coarse annotations. In this paper, we introduce a new weakly supervised approach for camouflaged object detection to overcome these limitations. Specifically, we propose a novel network, MGNet, which tackles edge ambiguity and missed detections by utilizing initial masks generated by our custom-designed Cascaded Mask Decoder (CMD) to guide the segmentation process and enhance edge predictions. We introduce a Context Enhancement Module(CEM) to reduce the missing detection, and a Mask-guided Feature Aggregation Module (MFAM) for effective feature aggregation. For the weak supervision challenge, we propose BoxSAM, which leverages the Segment Anything Model (SAM) with bounding-box prompts to generate pseudo-labels. By employing a redundant processing strategy, high quality pixel-level pseudo-labels are provided for training MGNet. Extensive experiments demonstrate that our method delivers competitive performance against current state-of-the-art methods.

• The paper presents a weak supervision framework for camouflaged object detection leveraging SAM with bounding-box prompts to generate refined pseudo-labels.
• It introduces a Mask-guided Network (MGNet) with a Cascaded Mask Decoder, Context Enhancement Module, and Mask-guided Feature Aggregation for precise segmentation.
• Extensive experiments on COD, SOD, polyp segmentation, and defect detection benchmarks demonstrate improved Sα and Fβ metrics, validating the approach.

Weakly Supervised Camouflaged Object Detection Based on SAM and Mask Guidance

Overview of Problem Setting

Camouflaged Object Detection (COD) encounters intrinsic challenges due to the high similarity between objects and their surrounding environments, causing detection failures primarily in edge ambiguity and missed detections. Annotation burden in fully supervised methods motivates weakly supervised approaches, but these tend to suffer from coarse label quality, resulting in performance degradation. The paper introduces a novel weakly supervised COD pipeline leveraging the Segment Anything Model (SAM) with bounding-box prompts (BoxSAM), along with a dedicated Mask-guided Network (MGNet) designed to handle typical COD challenges by producing refined segmentation masks.

Methodological Contributions

BoxSAM: Pseudo-label Generation with Bounding-Box Prompts

BoxSAM addresses the noisy pseudo-label challenge by leveraging bounding-box annotations to prompt SAM for initial mask generation. Bounding-boxes are less affected by annotation subjectivity than points or scribbles, resulting in more consistent localization. However, direct application of SAM masks can yield erroneous labels due to background-object similarity in COD imagery. This is mitigated via a Redundancy Processing Strategy (RPS) that refines pseudo-labels by cross-validating initial SAM outputs with network predictions, retaining only regions substantiated by MGNet outputs.

Figure 1: Overview of WSCOD with BoxSAM, where bounding-box annotations prompt SAM to generate pseudo-label masks, further refined by MGNet and redundancy processing.

Figure 2: Illustration showing bounding-box annotation scenarios on CAMO images, covering single and multiple object cases.

MGNet: Mask-Guided Network for Robust Segmentation

MGNet is a modular architecture composed of:

• Cascaded Mask Decoder (CMD): This decoder progressively fuses multiscale features extracted from a Pyramid Vision Transformer (PVTv2) backbone. CMD produces initial object masks, guiding subsequent stages.
• Context Enhancement Module (CEM): Employs dilated convolutions with varied dilation rates (multiscale receptive field aggregation) and a high-frequency attention branch (BA) to mitigate information loss from downsampling and enhance fine detail discrimination.
• Mask-guided Feature Aggregation Module (MFAM): Aggregates features across different levels guided by masks from CMD, using foreground and background attention maps to spatially refine edge and object region segmentation.

  Figure 3: MGNet overview, illustrating CMD, CEM, and MFAM integration for mask prediction.

  

  Figure 4: Architectural details of the Cascaded Mask Decoder, with multi-level feature fusion for initial mask output.

  

  Figure 5: CEM structure, highlighting multiscale dilation and high-frequency branches for context enhancement.

  

  Figure 6: MFAM composition, showing mask-based feature aggregation for edge and region refinement.

The model is trained using a hybrid loss combining Weighted Binary Cross-Entropy and Weighted Intersection-over-Union, supervising multiple intermediate outputs to encourage both pixel-level and global structural accuracy.

Empirical Validation

Performance on COD and SOD Benchmarks

BoxSAM and MGNet are extensively evaluated on CAMO, COD10K, NC4K, and four SOD datasets (ECSSD, HKU-IS, DUT-OMRON, DUTS-TE). MGNet consistently achieves superior $S_\alpha$, $F_\beta$, and lower $\mathcal{M}$ compared to competitive CNN-based (SINet, UGTR, BSANet) and transformer-based (MSCAF-Net, HitNet, CamoFormer-P) baselines, both in fully and weakly supervised configurations.

BoxSAM further outperforms prior WSCOD methods (WS-SAM, SAM-COD) in bounding-box and scribble settings, with a 2.6% improvement in $S_\alpha$ over SAM-COD on CAMO. In bounding-box supervised SOD, BoxSAM surpasses recent approaches, improving $F^m_\beta$ by up to 4.0% on the challenging DUT-OMRON dataset.

Figure 7: Visual comparison of MGNet results with FSPNet, highlighting superior edge completeness and missed detection mitigation.

Figure 8: Qualitative comparison across WSCOD methods and supervision types, showcasing improved object completeness and edge accuracy.

Figure 9: Visual comparison with transformer-based COD methods, demonstrating MGNet's superior segmentation of tiny and complex objects.

Ablation Study

Detailed ablations confirm the individual contributions of CMD, CEM, and MFAM. Removal of CMD and MFAM in MGNet yields over 2% drop in $F_\beta$ and $S_\alpha$; BA and dilated convolutions in CEM also provide substantial performance gains. Redundancy Processing Strategy (RPS) significantly improves pseudolabel quality—removal results in average performance drop of 0.7% in $F_\beta$.

Figure 10: Component effectiveness visualization—MGNet variants lacking CEM or CMD/MFAM exhibit degraded mask quality and missed object regions.

Figure 11: Examples of redundancy processing refining SAM outputs, with cleaned masks enabling improved training signal.

Application to Polyp Segmentation and Defect Detection

MGNet, when retrained for polyp segmentation (CVC-ClinicDB, CVC-ColonDB), exceeds PraNet, SSFormer, PVT-CASCADE, and even COD-specialized architectures. $mDice$ and $mIoU$ improve up to 0.8% and 1.0% over best previous models. In defect detection (CDS2K), MGNet outperforms SINet-V2 and MSCAF-Net, leading by 7.3% in $F_\beta$0, demonstrating generalization capacity to broad concealed object tasks.

Figure 12: Qualitative comparison on CVC-ClinicDB, illustrating MGNet's superior segmentation on large and tiny polyps compared to competitors.

Figure 13: Visual results on CDS2K defect detection, highlighting robustness across diverse materials and defect typologies.

Limitations

SAM, when used with bounding-box prompts in highly camouflaged scenes, exhibits segmentation failures, including object omission and background confusion. The redundancy processing strategy partially mitigates these issues but is not universally effective. Cross-method prompt fusion and improved pseudo-label synthesis are potential avenues for enhancing initial mask quality in future work.

Figure 14: Failure cases of SAM under bounding-box prompting—segmentation errors and missed objects remain open challenges.

Implications and Future Directions

This work demonstrates that well-designed weakly supervised pipelines, leveraging large vision models (SAM) and mask guidance, can approach and even match the performance of fully supervised COD/SOD models at a fraction of annotation cost. The modular architecture of MGNet enables both robustness to noisy inputs and effective multi-scale feature fusion. The approach is extensible to multiple vision domains, as evidenced by polyp segmentation and defect detection results.

Practically, the proposed pipeline reduces annotation effort and enables rapid deployment of class-agnostic segmentation models in domains where pixel-level labels are infeasible. Theoretically, BoxSAM+MGNet validates the efficacy of attention-guided feature aggregation and redundancy-aware label refinement in weakly supervised settings.

Future work should focus on improving prompt strategies by synthesizing multiple annotation types or automatically adapting prompt selection, potentially integrating SAM with other foundation models. Additionally, more sophisticated pseudo-label correction mechanisms, possibly driven by uncertainty estimation or active learning, could further enhance performance in challenging camouflaged or concealed scenarios.

Conclusion

The paper presents a comprehensive solution for weakly supervised camouflaged object detection, uniting SAM-based pseudo-label generation via bounding-box prompts with a tailored Mask-guided Neural Network. Through rigorous empirical validation and systematic ablations, the work establishes new performance standards for WSCOD, SOD, and related applications, offering theoretical and practical insights into efficient, high-accuracy object segmentation under limited supervision. Continued research in prompt refinement and pseudo-label correction promises further advances in annotation-efficient vision systems.