MajinBook: Digital Literature Catalogue

• MajinBook is an open digital literature catalogue that integrates metadata from shadow libraries and Goodreads to provide a corpus of over 539,000 works.
• It employs a multi-step record linkage strategy using MinHash, LSH, and fuzzy title matching to ensure high accuracy in clustering and metadata mapping.
• The dataset enables research in computational social science, cultural analytics, and AI with high-quality, machine-readable data and clear legal frameworks.

MajinBook is an open, rigorously linked catalogue of digital world literature that links metadata from shadow libraries—specifically Library Genesis and Z-Library—with structured bibliographic and popularity data from Goodreads. Designed for facilitating text mining and data-driven research in computational social science and cultural analytics, MajinBook comprises a high-precision corpus of over 539,000 English-language book references, spanning more than three centuries. It prioritizes natively digital EPUB-format files for optimal machine readability and metadata quality, and incorporates secondary collections for French, German, and Spanish works. The dataset’s construction methodology emphasizes accuracy, legal clarity for research contexts, and reproducibility, offering a bridge between crowd-sourced digital archives and curated bibliographic platforms (Mazières et al., 14 Nov 2025).

1. Data Sources and Integration

MajinBook integrates data from both shadow libraries and social reading platforms:

• Shadow Libraries: Library Genesis (LibGen, March 2025 dump) and Z-Library (January 2025 snapshot) provide metadata for over 77.6 million items, distributed as 15.2 million EPUBs, 50.5 million PDFs, and 3.5 million of other formats. Only natively digital EPUBs are retained for the core corpus, following conversion and filtering for machine readability.
• Structured Metadata: Goodreads is crawled for 4.78 million works, 28.1 million editions, and 2.15 million authors, contributing bibliographic, genre, and popularity fields.
• Integration Strategy: Metadata from LibGen and Z-Library are merged and filtered to retain EPUB files in the 10 KB–10 MB size range after conversion. These EPUB clusters are linked to Goodreads works using identifier (ISBN/ASIN) overlaps and fuzzy-matched title comparisons, ensuring high-confidence mappings across platforms.

2. Corpus Construction and Metadata Fields

MajinBook’s construction pipeline prioritizes the extraction of high-quality, natively digital EPUB files to maximize text quality and structural consistency.

• Filtering: EPub files are filtered for conversion success and required file size constraints, yielding 11.13 million files.
• Metadata Fields: Each record from the shadow libraries incorporates file size, language, and embedded identifiers (ISBN/ASIN). At the work level from Goodreads, records include Work ID, author information, first publication year, title, genres (with 74.5% coverage in English), and popularity metrics: average rating $\bar r$, number of ratings $N_{\rm rate}$, and number of reviews $N_{\rm rev}$.
• Language Coverage: The EPUB subset demonstrates increased linguistic diversity (Herfindahl Index, HI = 0.24) relative to other large digital libraries, with 47.9% English, 8.3% French, 5.9% German, and 9.5% Spanish.

3. Record Linkage Methodology

MajinBook employs a multi-step, human-in-the-loop linkage strategy:

• Content Clustering: Each EPUB file is summarized by a 128-element MinHash signature. Locality-Sensitive Hashing (LSH) splits these signatures into 9 bands of 13 rows, blocking files with approximate Jaccard similarity $\ge 0.8$. This produces 1.95 million clusters (grouping near-duplicates and different editions), of which 1.52 million contain at least one book identifier.
• Identifier-Based Linking: For each cluster of EPUBs and every Goodreads work, candidate links are established when their identifier sets intersect.
• Title-Based Verification: Further validation uses a partial‐ratio fuzzy match on titles, with the score $s_{\rm title}$ defined as the mean pairwise similarity between all cluster and work titles. Median $s_{\rm title}$ is 99.1, IQR [86.6, 100.0].
• Threshold Selection: Human validation on a stratified sample (n=200) determines that a score threshold $\tau=80$ achieves Precision $\approx 1.0$ (bootstrap 95% CI) and Recall $\approx 0.92$.

4. Corpus Properties and Bias Assessment

• Scale: The primary English-language corpus contains 539,530 works, with secondary French (47,960), German (35,559), and Spanish (30,169) datasets.
• Temporal Span: Works cover first publication years from circa 1700–2025; growth exhibits a super-exponential pattern (convex on a log scale), indicating the prevalence of natively digital and digitally re-issued works.
• Genre and Popularity: In the English subset, 74.5% of works possess at least one genre tag. Median ratings per work is 17 (IQR 4–99), and review presence is 98.8%. Mean rating is computed as

$\bar r = \frac{1}{N}\sum_{i=1}^{N} r_i.$

• Language Diversity: Compared with PDF shadow library and HathiTrust, the EPUB subset shows greater linguistic balance.

Corpus	HI (normalized)	English (%)	French (%)	German (%)	Spanish (%)
EPUB subset	0.24	47.93	8.26	5.94	9.45
PDF subset	0.74	85.97	1.09	3.96	0.94
HathiTrust	0.32	54.95	7.04	8.61	5.44
Goodreads	0.57	75.44	4.63	3.66	3.84

• Biases: There is an intentional skew towards digitally re-issued post-1950s texts (“productive sieve”), trading comprehensive historical coverage for machine-readability and text quality. Linguistic diversity is more balanced than in most comparable resources.

5. Legal and Ethical Considerations

The release of MajinBook (metadata only) is consistent with key US and EU legal frameworks:

• US Law: The Feist v. Rural Telephone (1991) precedent states that factual data (such as bibliographic metadata) is non-copyrightable; text and data mining (TDM) activities may fall under fair use (Copyright Act §107, DMCA exemption 37 C.F.R. 201.40).
• EU Law: Metadata-only releases are exempt under the EU Database Directive (96/9/EC) for sui generis database rights. TDM is further supported by Article 3 of the EU Copyright in the Digital Single Market Directive (2019/790).
• Research Use: Public-interest, institutional, and non-commercial research contexts are explicitly prioritized in the curation and deployment of the catalogue.

6. Applications in Research and Modeling

MajinBook opens several research avenues:

• Computational Social Science: Enables diachronic studies of genre shifts, correlations between popularity and publication era, and network analyses of author–work communities (using Goodreads recommendations).
• Cultural Analytics/Distant Reading: Supports n-gram and phrase analyses across languages and genres (with integration into tools like Gallicagram), facilitates cross-linguistic canon formation studies, and permits examination of “social curation” effects compared to legacy library-driven corpora such as HathiTrust.
• AI and Language Modeling: Provides a high-quality, machine-readable, and popularity-weighted training corpus for LLMs and related NLP tasks, allowing resource optimization and fine-tuning.

MajinBook’s open, metadata-enriched structure addresses longstanding data quality and diversity limitations in computational humanities, establishing a reproducible foundation for large-scale literary and cultural data analysis (Mazières et al., 14 Nov 2025).