Crack-Segmenter: Self-Supervised Crack Detection

• Crack-Segmenter is a self-supervised, multi-scale transformer framework for segmentation of pavement cracks without manual annotations.
• It employs a scale-adaptive embedder, directional attention transformer, and attention-guided fusion to extract and refine features across multiple scales.
• Empirical results on public datasets show superior mIoU and Dice scores compared to supervised methods, enabling efficient large-scale infrastructure monitoring.

Crack-Segmenter refers to a fully self-supervised, multi-scale transformer-based framework designed for efficient and annotation-free pavement crack detection and segmentation (Kyem et al., 12 Oct 2025). This system integrates robust multi-scale feature extraction, directional attention mechanisms tailored to linear crack morphology, and adaptive fusion modules. The framework is evaluated across diverse public datasets and consistently surpasses supervised baselines, demonstrating that annotation-free crack detection is not only feasible but also performant for large-scale infrastructure monitoring.

1. Architecture and Multi-Scale Design

Crack-Segmenter’s architecture is structured around three cascaded modules:

1. Scale-Adaptive Embedder (SAE): Extracts features at three complementary spatial scales—fine, small, and large—using parallel tailored convolutional operations. For the fine-scale representation, the operation is formalized as:

$$F_{(f)} = \sigma(W_{(f)} \ast I + b_{(f)}),\quad F_{(f)} \in \mathbb{R}^{B \times D \times H \times W}$$

where $I$ is the input image, $W_{(f)}$ is a $1 \times 1$ convolution kernel, $b_{(f)}$ is the bias, $\sigma$ is a nonlinearity, and $B$, $D$, $H$, $W$ index batch, channel, height, and width.

1. Directional Attention Transformer (DAT): Refines the multi-scale features by imposing direction-aware attention. After layer normalization and reshaping, DAT applies directional convolutions (e.g., horizontal, vertical):

$$Q_k = g(\hat{F}; W_{Q_k}, b_{Q_k}),\quad K_k = g(\hat{F}; W_{K_k}, b_{K_k})$$

where $g(X; W, b) = W \ast X + b$. The directional attention map $A_k$ for each direction $k$ is then:

$A_k = \mathrm{softmax}(Q_k K_k^T / \sqrt{D})$

ensuring continuity and alignment with the elongated topology of cracks.

1. Attention-Guided Fusion (AGF): Fuses multi-scale outputs adaptively. The AGF module first upsamples large-scale features and projects them to the correct spatial resolution. All scales are concatenated along the channel dimension, then scale-specific attention weights $A_l$, $A_s$, $A_f$ are learned and applied:

$$F_{\mathrm{fused}} = (F_l \odot A_l) + (F_s \odot A_s) + (F_f \odot A_f)$$

where $\odot$ denotes element-wise multiplication. This fusion is immediately followed by a decoding layer to yield the final segmentation map.

2. Self-Supervision Strategy

Crack-Segmenter is trained without any human-provided mask annotations, relying entirely on self-supervised losses that leverage the intrinsic structure of pavement images.

• Inter-scale Consistency Loss: Encourages feature representations across different scales to be mutually consistent:

$$\mathcal{L}_{\text{inter}}(G^x, G^y) = 1 - \frac{G^x \cdot G^y}{\lVert G^x \rVert \lVert G^y \rVert}$$

where $G^x$, $G^y$ denote context vectors extracted from different scales.

• Intra-scale Consistency Loss: Regularizes attention maps within a given scale to approach the identity, defined using an $L_1$ loss to promote spatial stability.
• Additional Self-Supervision: Pseudo-labels are generated from model high-confidence predictions, and a cross-entropy loss is computed accordingly, refining the model iteratively.

By optimizing the sum of these consistency objectives, the network converges on robust crack segmentations without ground truth supervision.

3. Empirical Performance and Metrics

Crack-Segmenter is benchmarked against 13 supervised methods on 10 public datasets, utilizing segmentation metrics:

• Mean Intersection over Union (mIoU): $$\frac{1}{C} \sum_{c=1}^C \frac{TP_c}{TP_c + FP_c + FN_c}$$
• Dice Score: $\mathrm{Dice} = \frac{2TP}{2TP + FP + FN}$
• XOR: Measures mismatches between binary masks.
• Hausdorff Distance (HD): Quantifies maximal boundary misalignment.

On datasets including CFD and CRACK500, Crack-Segmenter achieves higher mIoU and Dice than all competitors, and demonstrates superior spatial alignment (lower HD) and reduced false detections (lower XOR).

Dataset	mIoU (Crack-Segmenter)	Dice (Crack-Segmenter)	Supervised SOTA
CFD	Superior	Superior	Lower
CRACK500	Superior	Superior	Lower
Remaining 8	Consistently better	Consistently better	Lower

These results support that learning from image-intrinsic cues (such as inter-scale and intra-scale consistency) yields segmentation quality on par with, or surpassing, that of annotation-dependent supervised schemes.

4. Key Module Analysis

Crack-Segmenter’s modules are individually critical:

• SAE ensures multi-scale robustness, capturing thin cracks and wide, irregular patterns.
• DAT provides targeted, direction-sensitive attention, explicitly preserving crack continuity—a crucial aspect for binary crack topology.
• AGF adaptively weights and unifies contributions from all scales, allowing the network to dynamically shift focus between local texture and global context, depending on the crack’s morphological characteristics.

Ablation experiments indicate that omitting any module results in significantly degraded segmentation, confirming the necessity of the combined multi-scale, directional, and attention-guided design.

5. Applications and Infrastructure Impact

The annotation-free paradigm of Crack-Segmenter has direct implications for infrastructure monitoring:

• Scalability: Enables processing of national-scale road and bridge networks with no manual data labeling, reducing cost and time by at least an order of magnitude.
• Automation: Suits real-time surveillance using vehicle-mounted or drone-based imaging, allowing continuous highway or urban infrastructure assessment.
• Maintenance and Prevention: Accurate early detection facilitates preventive repairs and can lower future costs by up to 50–70% due to timely maintenance interventions.

The methodological advance signals a shift from reliance on annotated datasets towards scalable, data-efficient solutions in structural health monitoring, transportation safety, and asset management.

6. Limitations and Future Directions

Despite marked improvements, Crack-Segmenter’s complexity (due to transformer-based components and multi-scale processing) entails higher computational demand compared to minimal CNN architectures. Potential avenues for further research include:

• Extension to Multiple Pavement Anomalies: Adapting the self-supervised framework to segment other defects such as potholes, ruts, or spalling.
• Model Compression: Investigation of architectural refinements to lower computational and memory footprints, enabling deployment on edge or low-resource devices.
• Domain Adaptation: Enhancing generalization across diverse environmental conditions and image acquisition modalities.
• Temporal Consistency: Exploiting video streams for temporally consistent segmentation across multiple inspection intervals.

7. Mathematical and Diagrammatic Summary

Key formalizations and architectural flow can be summarized as:

• SAE module (fine-scale):

$F_{f} = \sigma(W_{f} \ast I + b_{f})$

• Direction-aware attention (per direction $k$):

$A_k = \mathrm{softmax}(Q_k K_k^T / \sqrt{D})$

• Attention-guided fusion:

$$F_{\text{fused}} = F_l \odot A_l + F_s \odot A_s + F_f \odot A_f$$

• Inter-scale consistency loss:

$$\mathcal{L}_{\text{inter}} = 1 - \frac{G^x \cdot G^y}{\lVert G^x \rVert\lVert G^y \rVert}$$

The modular flow can be visualized as:

1. Input image → SAE (multi-scale embeddings)
2. ↓
3. DAT (direction-refined embeddings)
4. ↓
5. AGF (fusion and decoding)
6. ↓
7. Output: segmentation map

This design achieves annotation-free, high-accuracy crack detection, fully exploiting multi-scale and directional cues while obviating the need for manual training data. Its demonstrated effectiveness and extensibility suggest it will serve as a cornerstone for future research and practice in large-scale, efficient infrastructure monitoring.