Boundary Attention: Learning curves, corners, junctions and grouping (2401.00935v3)

Abstract: We present a lightweight network that infers grouping and boundaries, including curves, corners and junctions. It operates in a bottom-up fashion, analogous to classical methods for sub-pixel edge localization and edge-linking, but with a higher-dimensional representation of local boundary structure, and notions of local scale and spatial consistency that are learned instead of designed. Our network uses a mechanism that we call boundary attention: a geometry-aware local attention operation that, when applied densely and repeatedly, progressively refines a pixel-resolution field of variables that specify the boundary structure in every overlapping patch within an image. Unlike many edge detectors that produce rasterized binary edge maps, our model provides a rich, unrasterized representation of the geometric structure in every local region. We find that its intentional geometric bias allows it to be trained on simple synthetic shapes and then generalize to extracting boundaries from noisy low-light photographs.

• The paper introduces a novel boundary attention network that iteratively refines local geometric representations for accurate sub-pixel detection.
• It employs a local attention mechanism that adapts to varying noise levels, outperforming state-of-the-art methods in speed and accuracy.
• The model demonstrates strong generalization by effectively processing real images of any size or aspect ratio using low-level cues.

Introduction

In computer vision, one of the significant tasks is to detect and interpret boundaries in images, such as edges, corners, and junctions. These boundaries are crucial for understanding the geometric details in a scene or object. Existing techniques often struggle with faint boundary signals or high noise levels, which can obscure critical details. Classical edge-detection methods have limitations in accuracy, particularly near corners and junctions. Recent deep learning models show promise but come with their own challenges, including a dependency on training datasets and difficulty in achieving sub-pixel precision.

Representing Boundaries with Attention Mechanisms

A novel network design is proposed to model boundaries in images more robustly and accurately. This design introduces "boundary attention," a concept that entails iteratively refining the local geometric representation around every pixel in an image. The network essentially builds a field of boundary descriptors that evolve to capture the image's local geometry precisely.

Adaptable Accuracy and Noise Resilience

The ability to adapt to various noise levels and geometric detail is a standout feature of the model. It achieves this through a local attention mechanism that adjusts its processing based on the particular image region, enabling it to handle faint boundaries amid noise efficiently. Unlike some earlier methods, this model does not rely on global features or human annotation during training, focusing instead on low-level cues and geometric consistency. This focus endows the model with the ability to accurately find sub-pixel level boundaries while being resilient to high amounts of noise.

Evaluating the Model

The performance of the network is noteworthy. It's been evaluated on images with severe noise conditions, demonstrating better or comparable results to other state-of-the-art methods while running significantly faster. Additionally, despite being trained on simple synthetic data, the model shows strong generalization capabilities to real images. Importantly, it can handle images at any size and aspect ratio, making it highly flexible and applicable in various practical scenarios.

Conclusion

This research presents a significant step forward in boundary detection, especially in challenging conditions, such as noisy environments or when dealing with fine details. By combining deep learning with a focus on low-level cues and adaptability, the model sets itself apart from traditional methods and stands out as an efficient and robust solution for a wide range of applications in computer vision.