Content-Based Mammographic Image Retrieval

• Content-Based Mammographic Image Retrieval is a method that retrieves mammograms based on visual and learned feature similarity with exact BIRADS matching instead of relying on textual metadata.
• It employs pretrained deep convolutional neural networks with advanced fine-tuning and ensemble techniques to extract and combine features for improved accuracy and low latency retrieval.
• The approach integrates rigorous statistical validation and efficient FAISS indexing to ensure precision in clinical applications such as diagnostic decision support and quality assurance.

Searching arXiv for papers on content-based mammographic image retrieval and the specified works. Content-based mammographic image retrieval denotes the retrieval of mammograms by visual and learned feature similarity rather than by textual metadata, with relevance defined by BIRADS category matching. In the setting reported for the CDD-CESM collection on The Cancer Imaging Archive, the task is formulated as exact BIRADS categorical matching across five distinct classes, a problem described as significantly more complex than binary classification tasks commonly addressed in the literature (Rahman et al., 6 Aug 2025). Recent work has framed this problem as a combination of deep feature extraction, similarity search with FAISS, and statistically rigorous evaluation on a 2,006-image mammographic corpus (Rahman et al., 6 Aug 2025, Rahman et al., 2024).

1. Problem definition and retrieval objective

In the reported mammographic retrieval setting, a query image is mapped to an embedding, nearest neighbors are identified in a database, and relevance is determined by whether retrieved images share the same BIRADS label as the query. Precision-oriented evaluation is central because the intended use is case comparison rather than only category prediction. One study explicitly defines the task as exact BIRADS matching over five classes, with BIRADS 1 (“normal”), BIRADS 2 (“benign”), BIRADS 3 (“probably benign”), BIRADS 4 (“suspicious”), and BIRADS 5 (“highly suspicious”) used in retrieval experiments, while BIRADS 6 (“known malignancy”) comprises 8 images and is not used in retrieval experiments (Rahman et al., 6 Aug 2025).

This framing distinguishes mammographic retrieval from binary benign–malignant classification. The five-class setting requires the retrieval system to preserve clinically meaningful intra-domain structure, not merely lesion presence or malignancy status. The same study states that literature suggests 20–25% Precision@10 represents achievable performance for exact BIRADS matching, and presents this as the realistic baseline against which improved retrieval systems should be interpreted (Rahman et al., 6 Aug 2025). A plausible implication is that retrieval quality in this domain depends as much on label granularity and relevance definition as on encoder capacity.

2. Dataset composition and label regimes

Both reported systems use the CDD-CESM dataset from TCIA and agree on a total of 2,006 mammograms (Rahman et al., 6 Aug 2025, Rahman et al., 2024). One study describes the corpus as all manually annotated by expert radiologists and gives a naturally imbalanced BIRADS distribution: 801 images for BIRADS 1, 333 for BIRADS 2, 187 for BIRADS 3, 319 for BIRADS 4, 358 for BIRADS 5, and 8 for BIRADS 6 (Rahman et al., 6 Aug 2025). The other describes the same 2,006 images as 1,003 low-energy and 1,003 subtracted contrast-enhanced JPEG images, each originally 2,355×1,315 px, with annotations in CSV detailing lesion shape, margins, density, and BIRADS scores, alongside medical reports in DOCX (Rahman et al., 2024).

The evaluation-oriented study reports stratified splitting intended to preserve class proportions and avoid data leakage, with Train = 50% (1,123 images), Validation = 20% (281), and Test = 30% (602 queries) (Rahman et al., 6 Aug 2025). It states that this split ensures each BIRADS class is well represented in train, validation, and test. The DenseNet–FAISS study also emphasizes the retrieval context but does not report the same statistical validation apparatus (Rahman et al., 2024).

These reports illustrate an important methodological issue. One paper characterizes the dataset by natural imbalance, while the other states that images are evenly distributed across categories (Rahman et al., 6 Aug 2025, Rahman et al., 2024). This suggests that cohort accounting, view handling, and protocol specification materially affect how retrieval difficulty is represented.

3. Representation learning and embedding construction

Recent systems use ImageNet-pretrained CNNs as feature extractors, with the final classification layers removed or replaced so that penultimate representations become image embeddings. The following architectures are reported for systematic comparison (Rahman et al., 6 Aug 2025):

Architecture	Parameters	Embedding dimension
DenseNet121	≈7.98 million	1,024
ResNet50	≈25.6 million	2,048
VGG16	≈138 million	4,096

The preprocessing pipeline reported for the multi-architecture framework resizes mammograms to 224×224 pixels with bilinear interpolation, converts them to RGB with 3 channels, and normalizes them with ImageNet statistics: mean = [0.485, 0.456, 0.406] and std = [0.229, 0.224, 0.225]. Training-time augmentation includes random horizontal and vertical flips, rotations of ±15°, color jitter, Gaussian blur, and random erasing (Rahman et al., 6 Aug 2025). The DenseNet–FAISS system similarly converts each grayscale mammogram to 3 identical channels, resizes to 224×224, and applies ImageNet normalization before feature extraction (Rahman et al., 2024).

In the DenseNet-121 formulation, the truncated encoder is written as a function $$\Phi:\mathbb{R}^{224\times224\times3}\to\mathbb{R}^{1024}$$, with the embedding for image $x_i$ given by

$$f_i = \Phi(x_i)\,,\quad \Phi:\mathbb{R}^{224\times224\times3}\to\mathbb{R}^{1024}.$$

No additional embedding layers were added in that study; the raw 1,024-dimensional vectors were indexed directly (Rahman et al., 2024).

The more elaborate framework adds advanced fine-tuning with differential learning rates: pretrained layers use learning rate $1\times10^{-5}$, newly initialized FC or embedding layers use $1\times10^{-4}$, and cosine-annealing is used for learning-rate decay. Additional regularization includes label smoothing with $\epsilon=0.1$, early stopping with patience = 7 epochs, and gradient clipping with $\text{max\_norm}=1.0$ (Rahman et al., 6 Aug 2025).

Metric learning is introduced through triplet loss, center loss, and a combined objective. The reported losses are

$$L_{triplet} = \sum_{(a,p,n)} \max\bigl(||f(x_a)\!-\!f(x_p)||_2^2 \;-\; ||f(x_a)\!-\!f(x_n)||_2^2 \;+\; \alpha,\;0\bigr),$$

$L_{center} = \sum_{i} ||f(x_i) - c_{y_i}||_2^2,$

where $c_{y_i}$ is the centroid of class $x_i$0 in feature space, and

$x_i$1

with example experimental weights $x_i$2, $x_i$3, and $x_i$4 (Rahman et al., 6 Aug 2025). This design explicitly treats retrieval as an embedding-geometry problem rather than only a classification problem.

4. Similarity search, indexing, and ensemble retrieval

The DenseNet–FAISS system uses FAISS flat indexes for exact nearest-neighbor retrieval: IndexFlatL2 for Euclidean distance and IndexFlatIP for inner-product search after $x_i$5 normalization to approximate cosine similarity (Rahman et al., 2024). The reported distances are

$x_i$6

and

$x_i$7

With only $x_i$8 images, the study argues that a flat index yields exact nearest-neighbor results with negligible latency, while more complex structures such as IVF-PQ and HNSW are reserved for $x_i$9 (Rahman et al., 2024).

The retrieval pipeline is decomposed into offline index building and online querying. Offline, each database image is preprocessed, embedded, and inserted into a FAISS index; this has cost $$f_i = \Phi(x_i)\,,\quad \Phi:\mathbb{R}^{224\times224\times3}\to\mathbb{R}^{1024}.$$0 to store $$f_i = \Phi(x_i)\,,\quad \Phi:\mathbb{R}^{224\times224\times3}\to\mathbb{R}^{1024}.$$1 vectors of dimensionality $$f_i = \Phi(x_i)\,,\quad \Phi:\mathbb{R}^{224\times224\times3}\to\mathbb{R}^{1024}.$$2. Online, the query mammogram is preprocessed, passed through DenseNet to obtain $$f_i = \Phi(x_i)\,,\quad \Phi:\mathbb{R}^{224\times224\times3}\to\mathbb{R}^{1024}.$$3, and used in a $$f_i = \Phi(x_i)\,,\quad \Phi:\mathbb{R}^{224\times224\times3}\to\mathbb{R}^{1024}.$$4-NN search against all indexed vectors. The reported overall per-query complexity is $$f_i = \Phi(x_i)\,,\quad \Phi:\mathbb{R}^{224\times224\times3}\to\mathbb{R}^{1024}.$$5, where $$f_i = \Phi(x_i)\,,\quad \Phi:\mathbb{R}^{224\times224\times3}\to\mathbb{R}^{1024}.$$6 denotes DenseNet forward-pass cost and is described as approximately $$f_i = \Phi(x_i)\,,\quad \Phi:\mathbb{R}^{224\times224\times3}\to\mathbb{R}^{1024}.$$7 FLOPs for DenseNet-121 (Rahman et al., 2024).

The more advanced framework extends single-model retrieval through super-ensemble optimization. It concatenates the $$f_i = \Phi(x_i)\,,\quad \Phi:\mathbb{R}^{224\times224\times3}\to\mathbb{R}^{1024}.$$8-normalized feature vectors from the two best fine-tuned models, DenseNet121_AdvancedFT and ResNet50_AdvancedFT, producing a final embedding dimension of $$f_i = \Phi(x_i)\,,\quad \Phi:\mathbb{R}^{224\times224\times3}\to\mathbb{R}^{1024}.$$9 (Rahman et al., 6 Aug 2025). The reported optimization objective is

$1\times10^{-5}$0

subject to $1\times10^{-5}$1 and $1\times10^{-5}$2, where $1\times10^{-5}$3 and $1\times10^{-5}$4 are the normalized embeddings of the two component models. In practice, equal concatenation without learned scalar weights is reported to provide the best trade-off between simplicity, speed, and performance (Rahman et al., 6 Aug 2025).

5. Evaluation methodology and reported retrieval performance

Evaluation protocols differ sharply across studies, and the most statistically explicit study treats this as a substantive methodological issue. It states that current medical image retrieval studies suffer from inadequate sample sizes, improper data splitting, and insufficient statistical validation, all of which hinder clinical translation (Rahman et al., 6 Aug 2025). Its own protocol uses a stratified 50%/20%/30% split, 602 test queries, and bootstrap confidence intervals with 1,000 samples (Rahman et al., 6 Aug 2025).

For that framework, Precision@10 is defined as

$1\times10^{-5}$5

Bootstrap confidence intervals are computed by resampling the 602 queries with replacement 1,000 times and taking the 2.5th and 97.5th percentiles of the resulting Precision@10 estimates (Rahman et al., 6 Aug 2025). Statistical comparison uses a paired $1\times10^{-5}$6-test and Cohen’s $1\times10^{-5}$7, with the reported formulas

$1\times10^{-5}$8

and

$1\times10^{-5}$9

The reported Precision@10 results for the multi-architecture framework are as follows (Rahman et al., 6 Aug 2025):

Method	Precision@10	Reported note
DenseNet121 baseline	29.08%	95% CI [27.99%, 30.22%]
ResNet50 baseline	30.02%	95% CI [28.78%, 31.19%]
VGG16 baseline	26.89%	95% CI [25.89%, 27.89%]
DenseNet121_AdvancedFT	34.79%	95% CI [33.24%, 36.34%]
ResNet50_AdvancedFT	34.54%	95% CI [32.99%, 36.09%]
Super-ensemble	36.33%	95% CI [34.78%, 37.88%]

The same study reports that advanced fine-tuning yields a 19.6% relative gain for DenseNet121_AdvancedFT and a 15.1% gain for ResNet50_AdvancedFT, while the super-ensemble yields a 24.9% improvement over baseline and approximately 3.6 relevant images per query (Rahman et al., 6 Aug 2025). It further reports statistically significant improvements over all baselines with $1\times10^{-4}$0 and Cohen’s $1\times10^{-4}$1, alongside a mean search time of $1\times10^{-4}$2 per query using a FAISS FlatL2 index on standard hardware (Rahman et al., 6 Aug 2025).

A different protocol, using DenseNet-121 with FAISS FlatL2, reports precision and recall across several values of $1\times10^{-4}$3. At $1\times10^{-4}$4, it reports Precision@5 = 0.80, Recall@5 = 0.012, mean NDCG@5 = 0.98, and search time = 1.04 ms. At $1\times10^{-4}$5, it reports Precision@10 = 0.90, Recall@10 = 0.020, mean NDCG@10 = 0.96, and search time = 1.24 ms. At $1\times10^{-4}$6, $1\times10^{-4}$7, and $1\times10^{-4}$8, it reports Precision@k values of 0.85, 0.80, and 0.79 respectively, with mean NDCG remaining between 0.94 and 0.95 (Rahman et al., 2024). The study also defines Recall@k and mAP:

$1\times10^{-4}$9

and

$\epsilon=0.1$0

Taken together, these reports show that headline retrieval numbers are protocol-dependent. One study explicitly frames 20–25% Precision@10 as the literature expectation for exact five-class BIRADS retrieval, while another reports Precision@10 = 0.90 on the same 2,006-image resource under a DenseNet–FAISS pipeline (Rahman et al., 6 Aug 2025, Rahman et al., 2024). This suggests that split design, relevance definition, and statistical validation strongly condition the comparability of reported CBMIR performance.

6. Applications, limitations, and methodological issues

The reported application space includes diagnostic decision support, medical education, quality assurance, peer review, and screening triage. The multi-architecture framework recommends ResNet50_AdvancedFT or DenseNet121_AdvancedFT for maximum standalone performance with minimal tuning, the 3,072-dimensional super-ensemble for peak retrieval accuracy with modest additional cost, and a single DenseNet121_AdvancedFT embedding for constrained hardware or storage (Rahman et al., 6 Aug 2025). It states that the system can retrieve approximately 3–4 confirmed BIRADS-matched cases in less than 3 ms to aid radiologist interpretation, and it proposes nearest-neighbor voting as a means to flag high-suspicion mammograms in BIRADS 4 or 5 (Rahman et al., 6 Aug 2025).

The DenseNet–FAISS study emphasizes a different set of strengths and limitations. It reports high precision at small $\epsilon=0.1$1, very low query latency, and strong ranking quality, attributing part of this behavior to DenseNet’s dense connectivity and feature reuse across layers (Rahman et al., 2024). At the same time, it reports a recall trade-off: even at $\epsilon=0.1$2, only about 9% of all relevant images are found. It also notes that flat indexing scales linearly in $\epsilon=0.1$3, making approximate structures necessary for databases much larger than $\epsilon=0.1$4, and identifies failure modes involving subtle texture changes between BIRADS-2 and BIRADS-3 as well as out-of-distribution cases such as implants and post-surgical scars (Rahman et al., 2024).

A recurrent methodological issue is whether retrieval should be treated as an extension of classification or as a separate ranking problem. The stronger evidence in the reported literature supports the second view: retrieval systems are evaluated by Precision@k, Recall@k, mAP, NDCG, latency, and robustness of statistical validation rather than only by classification accuracy (Rahman et al., 6 Aug 2025, Rahman et al., 2024). Another persistent issue is comparability. One study foregrounds rigorous stratified splitting, bootstrap confidence intervals, and formal significance testing, whereas the other foregrounds efficient DenseNet feature extraction and FAISS-based similarity search with high small-$\epsilon=0.1$5 precision (Rahman et al., 6 Aug 2025, Rahman et al., 2024). This suggests that the field’s central debate is not only architectural, but also protocol-centric: performance claims depend materially on how relevance is defined, how leakage is prevented, and how uncertainty is quantified.