Sketch-based Manga Retrieval using Manga109 Dataset (1510.04389v1)

Abstract: Manga (Japanese comics) are popular worldwide. However, current e-manga archives offer very limited search support, including keyword-based search by title or author, or tag-based categorization. To make the manga search experience more intuitive, efficient, and enjoyable, we propose a content-based manga retrieval system. First, we propose a manga-specific image-describing framework. It consists of efficient margin labeling, edge orientation histogram feature description, and approximate nearest-neighbor search using product quantization. Second, we propose a sketch-based interface as a natural way to interact with manga content. The interface provides sketch-based querying, relevance feedback, and query retouch. For evaluation, we built a novel dataset of manga images, Manga109, which consists of 109 comic books of 21,142 pages drawn by professional manga artists. To the best of our knowledge, Manga109 is currently the biggest dataset of manga images available for research. We conducted a comparative study, a localization evaluation, and a large-scale qualitative study. From the experiments, we verified that: (1) the retrieval accuracy of the proposed method is higher than those of previous methods; (2) the proposed method can localize an object instance with reasonable runtime and accuracy; and (3) sketch querying is useful for manga search.

• The paper presents a novel manga-specific image descriptor using an EOH+PQ framework that outperforms traditional SIFT-based methods.
• It integrates a user-friendly sketch-based interface with relevance feedback to enhance search accuracy and object localization.
• The Manga109 dataset, comprising 21,142 pages from 109 titles, offers a robust resource for advancing manga retrieval research.

Sketch-based Manga Retrieval using Manga109 Dataset

Overview and Contributions

The paper "Sketch-based Manga Retrieval using Manga109 Dataset" introduces an innovative content-based manga retrieval system tailored for the unique characteristics of manga images. The proposed system leverages a novel combination of a manga-specific image-describing framework and a sketch-based interface to enhance the manga search experience. Central to this system are three primary contributions: a robust image descriptor tailored for manga, a sketch-based querying interface, and the creation of a substantial manga image dataset, Manga109.

Key Components

1. Image Describing Framework:
   • The paper introduces an objectness-based edge orientation histogram (EOH) descriptor combined with product quantization (PQ).
   • The system involves three stages: margin area labeling to exclude non-informative regions, EOH feature extraction to describe the visual content, and PQ for efficient nearest-neighbor search.
2. Sketch-based Interface:
   • A user-friendly interface that allows for sketch-based querying, relevance feedback, and query retouch capabilities.
   • This interface makes it intuitive for users to interact with manga content by drawing sketches to find specific manga scenes or objects.
3. Manga109 Dataset:
   • Manga109 is a comprehensive dataset comprising 109 manga titles, amounting to 21,142 pages drawn by professional artists.
   • This dataset, which addresses the significant gap of readily accessible manga images for research, is publicly available for academic use.

Experimental Evaluation

1. Comparative Study:
   • Evaluations indicate that the proposed EOH+PQ framework outperforms existing methods like BoF, FV, and Compact OCM in recall@k and mean average precision (mAP) metrics.
   • The experiments reveal that traditional methods tailored for natural images, like SIFT-based descriptors, are not well-suited for the distinctive visual style of manga, underscoring the necessity of a specialized descriptor.
2. Localization Evaluation:
   • The system demonstrates an ability to accurately localize objects within manga pages.
   • On the large Manga109 dataset, the retrieval system managed to achieve localization with reasonable accuracy and runtime, highlighting its efficiency in managing extensive data.
3. Large-Scale Qualitative Study:
   • Extensive qualitative evaluations using a public sketch dataset and relevance feedback interactions showcase the practical applicability of the system.
   • The retrieval system effectively handled a variety of sketch-based queries, demonstrating robustness in identifying and ranking relevant manga content.

Implications and Future Directions

The proposed retrieval system, with its unique approach to manga image description and efficient search capabilities, positions itself as a valuable tool for manga enthusiasts and researchers. By combining content-based retrieval techniques with an intuitive sketch-based interface, it provides a more engaging and effective way to search through vast manga archives.

Practical Implications:

• The system can be integrated into digital manga libraries, enhancing the user experience by enabling more intuitive searches beyond the traditional keyword and tag-based methods.
• Potential applications include mobile apps or web interfaces for e-manga services, making large collections of manga more accessible and navigable.

Theoretical Implications:

• The paper bridges a gap in content-based multimedia retrieval by focusing on the unique characteristics of manga, thus laying the groundwork for further exploration in specialized content retrieval from other media forms.
• The publicly available Manga109 dataset represents a significant resource for future research, potentially catalyzing advances in related image processing and retrieval fields.

Future Developments:

• The integration of textual metadata with the sketch-based search might yield a more comprehensive retrieval system, allowing for combined keyword and content-based searches.
• Enhancements to the query interaction capabilities, such as more refined relevance feedback mechanisms and advanced image editing functionalities, may further improve user satisfaction and retrieval accuracy.
• Investigating the application of deep learning approaches, which have shown promise in various image processing tasks, to the domain of manga retrieval might offer insights and performance boosts.

In conclusion, this paper presents a robust and efficient system for sketch-based manga retrieval, backed by thorough evaluations and a substantial dataset contribution. The methods and findings outlined here set a solid foundation for future advancements in content-based manga and broader image retrieval research.