Medical Image Retrieval using Deep Convolutional Neural Network (1703.08472v1)

Abstract: With a widespread use of digital imaging data in hospitals, the size of medical image repositories is increasing rapidly. This causes difficulty in managing and querying these large databases leading to the need of content based medical image retrieval (CBMIR) systems. A major challenge in CBMIR systems is the semantic gap that exists between the low level visual information captured by imaging devices and high level semantic information perceived by human. The efficacy of such systems is more crucial in terms of feature representations that can characterize the high-level information completely. In this paper, we propose a framework of deep learning for CBMIR system by using deep Convolutional Neural Network (CNN) that is trained for classification of medical images. An intermodal dataset that contains twenty four classes and five modalities is used to train the network. The learned features and the classification results are used to retrieve medical images. For retrieval, best results are achieved when class based predictions are used. An average classification accuracy of 99.77% and a mean average precision of 0.69 is achieved for retrieval task. The proposed method is best suited to retrieve multimodal medical images for different body organs.

• The paper introduces a two-phase deep CNN framework that integrates image classification and feature extraction for effective medical image retrieval.
• It achieves a remarkable 99.77% classification accuracy and a mean average precision of 0.69 by leveraging multimodal imaging data.
• The approach narrows the semantic gap in CBMIR by reducing reliance on handcrafted features, thereby simplifying clinical image annotation.

Medical Image Retrieval using Deep Convolutional Neural Networks

The paper "Medical Image Retrieval using Deep Convolutional Neural Network" proposes an innovative framework for Content-Based Medical Image Retrieval (CBMIR) that utilizes a deep learning strategy, specifically a Convolutional Neural Network (CNN), to bridge the semantic gap inherent in traditional feature extraction methodologies. As clinical environments produce increasingly large datasets of digital imaging data, efficient systems for data management and retrieval become crucial. The proposed CNN framework serves not only to classify medical images effectively but also to leverage those classifications for improved image retrieval from multimodal datasets.

Methodology and Experimental Framework

The proposed method first trains a deep CNN model for the classification of medical images across 24 classes using a dataset comprised of five imaging modalities. Critical to this method is its two-phase process: classification and feature extraction. The CNN architecture detailed consists of five convolutional layers and three fully connected layers, following standard procedures for deep learning with distinctive configurations like the use of ReLU nonlinearities, dropout regularization, and the stochastic gradient descent algorithm for optimization.

1. Classification Phase: The CNN is optimized to achieve superior classification accuracy. The architecture, with convolutional and pooling layers, extracts hierarchical feature representations directly from input medical images, reducing the need for handcrafted feature engineering. The training involved careful initialization of weights and biases and was fine-tuned using an error minimization strategy.
2. Retrieval Phase: Post classification training, feature representations are extracted from the last three fully connected layers and stored in a feature database. Retrieval of medical images leverages these features using a Euclidean distance metric, with the option to limit the search within predicted classes to enhance performance.

Strong Numerical Results

The framework demonstrates robust performance, achieving an average classification accuracy of 99.77% across diverse medical image classes. This accuracy underscores the system's capability to use learned features for efficient retrieval with a mean average precision of 0.69 when class prediction is included, outperforming several contemporary CBMIR systems, though comparisons are challenging due to the lack of a standardized dataset.

Theoretical and Practical Implications

The deep learning-based approach to CBMIR addresses a significant challenge in medical imaging: the semantic gap between pixel-level data processed by machines and the high-level semantic understanding required for clinical interpretation. By automating feature extraction through machine learning, this framework reduces annotation efforts and mitigates the reliance on domain-specific knowledge. This has both practical implications for medical diagnostic support systems and theoretical implications for advancing CBMIR methodologies using AI.

The system's application for multimodal medical images marks a notable advancement, facilitating the retrieval of 2D slices across diverse imaging modalities with varying anatomical and pathological characteristics, thus expanding its utility in practical, clinical settings.

Future Prospects

Future work can explore scalability to 3D volumetric data, which is pivotal for applications such as MRI and CT scans, by adapting network architectures to handle volumetric data representations. This would require comprehensive datasets representing various geometric views and diseases. Furthermore, integrating such systems with fully automated PACS could transform how image data is utilized in therapeutic and diagnostic protocols.

The paper underlines the potential of deep learning to evolve CBMIR, suggesting promising avenues for future research in improving retrieval efficiencies and broadening the application scope to encompass a wider variety of medical imaging tasks.