SURDS: Self-Supervised Attention-guided Reconstruction and Dual Triplet Loss for Writer Independent Offline Signature Verification (2201.10138v2)

Abstract: Offline Signature Verification (OSV) is a fundamental biometric task across various forensic, commercial and legal applications. The underlying task at hand is to carefully model fine-grained features of the signatures to distinguish between genuine and forged ones, which differ only in minute deformities. This makes OSV more challenging compared to other verification problems. In this work, we propose a two-stage deep learning framework that leverages self-supervised representation learning as well as metric learning for writer-independent OSV. First, we train an image reconstruction network using an encoder-decoder architecture that is augmented by a 2D spatial attention mechanism using signature image patches. Next, the trained encoder backbone is fine-tuned with a projector head using a supervised metric learning framework, whose objective is to optimize a novel dual triplet loss by sampling negative samples from both within the same writer class as well as from other writers. The intuition behind this is to ensure that a signature sample lies closer to its positive counterpart compared to negative samples from both intra-writer and cross-writer sets. This results in robust discriminative learning of the embedding space. To the best of our knowledge, this is the first work of using self-supervised learning frameworks for OSV. The proposed two-stage framework has been evaluated on two publicly available offline signature datasets and compared with various state-of-the-art methods. It is noted that the proposed method provided promising results outperforming several existing pieces of work. The code is publicly available at: https://github.com/soumitri2001/SURDS-SSL-OSV

• The paper introduces a framework that combines self-supervised attention-based reconstruction with dual triplet loss to enhance writer-independent signature verification.
• It utilizes a ResNet-18 encoder-decoder with spatial attention to capture key signature features for precise metric learning.
• The SURDS approach outperforms state-of-the-art methods on Bengali and Hindi datasets, achieving lower false acceptance and rejection rates.

An Overview of SURDS: Self-Supervised Attention-guided Reconstruction and Dual Triplet Loss for Writer Independent Offline Signature Verification

The paper "SURDS: Self-Supervised Attention-guided Reconstruction and Dual Triplet Loss for Writer Independent Offline Signature Verification" presents a novel framework focusing on offline signature verification (OSV), a critical biometric task across numerous applications. The research introduces a two-phase deep learning pipeline uniquely combining self-supervised learning and metric learning to address the writer-independent OSV challenge effectively.

Methodological Approach

The approach is premised on the difficulty of capturing subtle distinctions between genuine and forged signatures, which require more refined modeling compared to other biometric verification tasks. The proposed framework, termed SURDS, integrates self-supervised representation learning by employing a reconstruction-based pre-training method using an encoder-decoder model enhanced with a 2D spatial attention mechanism. The pipeline consists of two distinct phases:

1. Self-Supervised Pre-training: This phase involves training a ResNet-18-based convolutional encoder-decoder architecture with an attention component that emphasizes crucial local signature patches. It reconstructs signature images to learn detailed, influential signature characteristics without requiring label supervision.
2. Metric Learning with Dual Triplet Loss: The pre-trained encoder is subsequently fine-tuned using a dual triplet loss within a supervised metric learning framework. This loss function optimizes by sampling negatives both intra-writer and cross-writer, thereby enhancing the discriminative power of the signature embeddings. The objective ensures the tighter clustering of genuine pairs while maintaining adequate separation from forgeries, even when they belong to similar writer classes.

Experimental Results

The efficacy of SURDS was validated across two substantial offline signature datasets, namely BHSig260 Bengali and Hindi. The method outperformed existing state-of-the-art techniques in terms of accuracy, with lower rates of false acceptance and rejection, emphasizing its superior capabilities in discerning genuine signatures from forgeries.

Implications and Future Directions

The introduction of self-supervised learning to OSV is particularly novel, marking a deviation from traditional approaches predominantly reliant on fully supervised models. This approach is especially beneficial in contexts where label scarcity is common, as it allows for effective representation learning without extensive labeled data.

From a theoretical standpoint, this research extends self-supervised learning beyond its conventional domains, such as natural image processing, to the nuanced field of signature verification. Practically, it suggests potential for scalable, robust OSV systems less dependent on pre-defined writer-specific data.

However, while promising, the model's cross-dataset and cross-script evaluations indicate the need for further examination into enhancing the universality of the learned representations. Future research may delve into expanding this framework to encompass additional scripts and exploring augmentation strategies or leveraging synthetic data to further bolster robustness across varied signature styles.

In summary, the SURDS framework sets a solid foundation for integrating self-supervised learning into OSV tasks, providing a potential blueprint for advancing signature verification methodologies towards heightened reliability and adaptability in real-world applications.