Retrieval-Augmented Approach for Unsupervised Anomalous Sound Detection and Captioning without Model Training (2410.22056v1)

Abstract: This paper proposes a method for unsupervised anomalous sound detection (UASD) and captioning the reason for detection. While there is a method that captions the difference between given normal and anomalous sound pairs, it is assumed to be trained and used separately from the UASD model. Therefore, the obtained caption can be irrelevant to the differences that the UASD model captured. In addition, it requires many caption labels representing differences between anomalous and normal sounds for model training. The proposed method employs a retrieval-augmented approach for captioning of anomalous sounds. Difference captioning in the embedding space output by the pre-trained CLAP (contrastive language-audio pre-training) model makes the anomalous sound detection results consistent with the captions and does not require training. Experiments based on subjective evaluation and a sample-wise analysis of the output captions demonstrate the effectiveness of the proposed method.

• The paper introduces a retrieval-augmented approach that unifies unsupervised sound anomaly detection with difference captioning using pre-trained CLAP embeddings.
• It employs a k-nearest neighbors strategy and text decoding to bypass extensive task-specific training while ensuring coherent anomaly descriptions.
• Experimental results demonstrate competitive performance, offering practical benefits for predictive maintenance and real-time anomaly reporting.

Assessment of a Retrieval-Augmented Approach for Unsupervised Anomalous Sound Detection and Captioning

In this paper, the authors introduce a novel methodology for unsupervised anomalous sound detection (UASD) paired with the generation of textual descriptions explaining the anomalies. Their approach significantly deviates from traditional methods by eliminating the need for model training specific to the task of difference captioning. This is achieved through leveraging a pre-trained architecture, namely the contrastive language-audio pre-training (CLAP) model.

Methodology

The authors critique existing models, which separate the UASD framework from the process of captioning the differences between normal and abnormal sound events. These methods often require extensive annotated data and may lead to disconnected or irrelevant captions relative to the detected anomalies. The proposed approach overcomes these limitations by utilizing a retrieval-augmented generation (RAG) method. Specifically, it simultaneously addresses detection and captioning within the same framework using CLAP, thereby ensuring coherence between the UASD outcomes and the generated captions.

The CLAP model serves as the backbone for their methodology. This model outputs embeddings for audio inputs that are foundational to both classification in UASD, using k-nearest neighbors (k-NN) for determining anomaly scores, and captioning. The innovation lies in the use of pre-generated embeddings from a large, pre-trained dataset, negating the necessity for additional training specific to industrial audio data. The method capitalizes on a text decoder to generate initial captions for both normal and anomalous sounds. Then, a more refined caption describing the differences is produced by comparing the outputs using GPT-4.

Experimental Results

The paper applies the proposed methodology to benchmark datasets, demonstrating competitive performance in UASD tasks when compared with state-of-the-art models. Importantly, the method shows efficiency in generating meaningful captions that reflect real-world anomaly causes, as validated through subjective evaluations. The authors introduce two captioning approaches: one based on text decoding and another leveraging predefined descriptors with zero-shot classification. The latter shows utility in scenarios where the former falls short, particularly in cases where the audio embeddings' differences aren't adequately captured through textual description alone.

Implications and Future Work

This research advances the field by integrating anomaly detection and descriptive intelligence for audio anomaly tasks, effectively broadening the application scope for unsupervised models in industrial settings. The potential implications are substantial for predictive maintenance and real-time anomaly reporting systems, where understanding the nature of an anomaly can significantly impact decision-making processes.

Future work could explore extended use cases across diverse audio domains, further refine the integration of audio and textual embeddings, and enhance the retrieval-augmented approach by incorporating more comprehensive multimodal datasets. Additionally, investigations into optimizing computational efficiency could be valuable for real-time implementation.

Conclusion

This paper presents a comprehensive strategy to merge anomaly detection with descriptive explanation generation in unsupervised settings without additional training. It positions the retrieval-augmented approach using CLAP embeddings as a robust tool compatible with varied audio tasks, fostering advancements in the intersection of audio surveillance and natural language generation.