Boundary-Aware Segmentation Network for Mobile and Web Applications (2101.04704v2)

Abstract: Although deep models have greatly improved the accuracy and robustness of image segmentation, obtaining segmentation results with highly accurate boundaries and fine structures is still a challenging problem. In this paper, we propose a simple yet powerful Boundary-Aware Segmentation Network (BASNet), which comprises a predict-refine architecture and a hybrid loss, for highly accurate image segmentation. The predict-refine architecture consists of a densely supervised encoder-decoder network and a residual refinement module, which are respectively used to predict and refine a segmentation probability map. The hybrid loss is a combination of the binary cross entropy, structural similarity and intersection-over-union losses, which guide the network to learn three-level (ie, pixel-, patch- and map- level) hierarchy representations. We evaluate our BASNet on two reverse tasks including salient object segmentation, camouflaged object segmentation, showing that it achieves very competitive performance with sharp segmentation boundaries. Importantly, BASNet runs at over 70 fps on a single GPU which benefits many potential real applications. Based on BASNet, we further developed two (close to) commercial applications: AR COPY & PASTE, in which BASNet is integrated with augmented reality for "COPYING" and "PASTING" real-world objects, and OBJECT CUT, which is a web-based tool for automatic object background removal. Both applications have already drawn huge amount of attention and have important real-world impacts. The code and two applications will be publicly available at: https://github.com/NathanUA/BASNet.

• The paper introduces a boundary-aware segmentation network that leverages a predict-refine architecture and hybrid loss to enhance segmentation accuracy in challenging settings.
• It refines coarse segmentation maps with a residual module, balancing computational efficiency with precise boundary delineation.
• Empirical evaluations show that the model outperforms state-of-the-art methods on benchmarks and supports real-time applications above 70 FPS.

Boundary-Aware Segmentation Network for Mobile and Web Applications

The paper introduces a Boundary-Aware Segmentation Network (BASNet), designed to address the challenges associated with achieving high boundary accuracy in image segmentation tasks. This research focuses particularly on applications serving mobile and web environments, where efficiency and precision are paramount.

Overview of BASNet

BASNet is comprised of a predict-refine architecture and leverages a hybrid loss function to enhance segmentation accuracy. The model features:

• Predict-Refine Architecture:
  • An encoder-decoder network offers dense supervision, enabling the model to predict a coarse segmentation output.
  • A residual refinement module enhances the segmentation map by focusing on the residuals between the predicted and actual segmentation maps.
• Hybrid Loss Function:
  • The hybrid loss integrates binary cross-entropy (BCE), structural similarity (SSIM), and intersection-over-union (IoU) losses.
  • It enables the learning of hierarchical feature representations at pixel, patch, and map levels, thus improving boundary and structural accuracy.

Mathematical and Methodological Insights

The predict-refine structure in BASNet differentiates from traditional cascaded architectures, which often become computationally expensive. By refining at a singular step, it balances efficiency and accuracy, maintaining a sharp focus on boundaries, a recurring challenge in image segmentation.

The hybrid loss is instrumental to BASNet's performance. While BCE drives general pixel accuracy, SSIM emphasizes local structural integrity, and IoU promotes robust model performance on significant regions. Together, these facilitate superior boundary delineation and fine structure representation, aligning learning targets with performance metrics.

Empirical Evaluation

BASNet's empirical evaluation covers salient object segmentation and camouflaged object detection tasks, where maintaining boundary sharpness is crucial. The model outperformed numerous state-of-the-art methods across several benchmark datasets, including DUT-OMRON and ECSSD, in terms of boundary-aware and regional metrics. It showed particular strength in boundary measures, as evidenced by high $F^b_\beta$ scores.

Furthermore, BASNet exhibits significant real-time capability, operating at over 70 frames per second on a single GPU, which is crucial for real-world applications.

Practical Applications

The integration of BASNet into practical applications demonstrates its utility:

• AR COPY {content} PASTE: A mobile application that allows users to "copy" real-world objects and "paste" them into digital spaces using augmented reality. This system showcases BASNet's ability to perform high-fidelity segmentation in a user-interactive setting.
• OBJECT CUT: A web-based tool for automatic background removal, leveraging BASNet's capabilities. Its architecture ensures efficient processing, thereby scaling effectively in online environments.

Both applications underscore BASNet's adaptability and potent impact in real-time segmentation tasks.

Implications and Future Work

The architectural simplicity and efficiency of BASNet suggest broad applicability. It could serve as a foundation for future developments in other segmentation-intensive tasks. Additionally, the predict-refine approach and hybrid loss composition offer potential adaptations for multi-class segmentation challenges.

Considering the dynamic field of AI and computer vision, further research could explore integrating advanced attention mechanisms or incorporating transformer architectures with BASNet's foundational components. This could enhance contextual understanding and extend BASNet's utility across diverse segmentation tasks.

Overall, BASNet represents a substantial advancement in boundary-aware segmentation, particularly well-suited for mobile and web applications, where precision and efficiency must coexist.