Packing and Padding: Coupled Multi-index for Accurate Image Retrieval (1402.2681v2)

Abstract: In Bag-of-Words (BoW) based image retrieval, the SIFT visual word has a low discriminative power, so false positive matches occur prevalently. Apart from the information loss during quantization, another cause is that the SIFT feature only describes the local gradient distribution. To address this problem, this paper proposes a coupled Multi-Index (c-MI) framework to perform feature fusion at indexing level. Basically, complementary features are coupled into a multi-dimensional inverted index. Each dimension of c-MI corresponds to one kind of feature, and the retrieval process votes for images similar in both SIFT and other feature spaces. Specifically, we exploit the fusion of local color feature into c-MI. While the precision of visual match is greatly enhanced, we adopt Multiple Assignment to improve recall. The joint cooperation of SIFT and color features significantly reduces the impact of false positive matches. Extensive experiments on several benchmark datasets demonstrate that c-MI improves the retrieval accuracy significantly, while consuming only half of the query time compared to the baseline. Importantly, we show that c-MI is well complementary to many prior techniques. Assembling these methods, we have obtained an mAP of 85.8% and N-S score of 3.85 on Holidays and Ukbench datasets, respectively, which compare favorably with the state-of-the-arts.

• The paper presents the Coupled Multi-index (c-MI) framework, which integrates SIFT and color features to significantly reduce false matches in image retrieval.
• It employs a novel two-dimensional indexing and binary signature computation to minimize quantization errors and enhance retrieval efficiency.
• Experimental results demonstrate a mean average precision of 85.8% and halved query time compared to traditional Bag-of-Words methods.

Packing and Padding: Coupled Multi-index for Accurate Image Retrieval

The paper "Packing and Padding: Coupled Multi-index for Accurate Image Retrieval" proposes a novel Coupled Multi-Index (c-MI) framework to enhance the precision and efficiency of image retrieval systems. Image retrieval applications often employ the Bag-of-Words (BoW) model, where features like SIFT are used to describe images. However, the SIFT descriptor alone frequently results in false positive matches due to its limited discriminative power and the inherent quantization loss during feature encoding. To address this, the authors present an innovative method that fuses multiple distinct features – specifically SIFT and local color features – at the indexing level to improve retrieval accuracy.

Methodology

The core contribution of this work is the c-MI framework, in which complementary features are coupled into a multi-dimensional inverted index. For implementation, the authors chose to integrate SIFT descriptors with local color features, forming a two-dimensional index. This approach differs from traditional one-dimensional indices that depend solely on a single descriptor type. Key aspects of the c-MI method involve:

1. Feature Extraction and Quantization: SIFT features are extracted from scale-invariant keypoints, while color descriptors are derived from Color Names (CN) vectors. Both sets of descriptors are quantized using independently trained codebooks, allowing for improved recall through the application of Multiple Assignment (MA), particularly enhancing robustness to illumination changes for color features.
2. Binary Signature Calculation: Besides quantizing features to visual words, the authors generate binary signatures to further reduce quantization errors. The integration of these signatures in conjunction with traditional features allows for more intricate and discriminative matching.
3. Coupled Multi-Index Design: The c-MI allows pairs of visual words from SIFT and color features to be indexed. This organization facilitates multi-dimensional voting during retrieval, where the system searches for candidate images that are similarly represented in both feature spaces.
4. Inverted Index Structure: The c-MI framework is constructed to store not only image IDs but also additional metadata, supporting more efficient similarity comparisons and rankings of candidate images.

Experimentation and Results

The authors conducted extensive experiments across several benchmark datasets, including Ukbench, Holidays, and others, to demonstrate the effectiveness of the proposed approach. The results show that the c-MI framework outperforms traditional BoW methods by achieving notably higher retrieval accuracies while maintaining half the query time compared to baseline systems. Specifically, they reported a mean Average Precision (mAP) of 85.8% and an N-S score of 3.85 on the Holidays and Ukbench datasets, respectively.

Furthermore, the paper discusses the compatibility of c-MI with existing enhancement techniques such as Hamming Embedding, burstiness weighting, and graph fusion. The c-MI framework is not only shown to be complementary to these methods but also facilitates further accuracy improvements when combined.

Implications and Future Directions

The implications of this research are significant for both theoretical understanding and practical applications of image retrieval systems. By illustrating an efficient method for feature fusion at the index level, c-MI paves the way for more accurate and scalable image search applications. This is particularly advantageous in scenarios involving large-scale databases, where query efficiency and accuracy are critical.

Moving forward, there is scope to expand this framework by incorporating other types of local descriptors or employing higher-order indices to capture more complex feature dependencies. Additionally, exploring further feature selection strategies might enhance the system's ability to generalize across diverse datasets and image types.

In conclusion, this paper presents a substantial advancement in indexed image retrieval methodologies, offering a robust solution capable of achieving superior results compared to traditional models and methods.