---
title: 'DocMine: Multi-Artifact Documentation Dataset'
url: https://www.emergentmind.com/topics/docmine-dataset
type: topic
---

# DocMine: Multi-Artifact Documentation Dataset

Searching arXiv for the cited paper and any directly related DocMine/DocFetch entries.
DocMine is a large-scale, multi-artifact corpus for automated and semi-automated software documentation research. It was introduced to consolidate, label, and expose the natural-language fragments scattered across source code repositories so that downstream tools, including DocFetch, can exploit this “hidden” documentation [2508.17719]. The dataset spans five software artifact types—commit messages, issue comments, pull-request descriptions, source-code comments, and textual documentation files—drawn from 950 popular GitHub repositories, and comprises more than 1,380,000 artifact records. Each record is annotated with one or more of six documentation-type labels: File-related, Error/Bug-related, Project-related, API-related, License-related, and Architecture-related.

## 1. Scope and corpus design

DocMine was constructed to address a limitation in automated documentation generation: existing approaches largely focus on source code, even though information useful for documentation is scattered across multiple artifacts that co-evolve with the codebase [2508.17719]. Its design therefore treats documentation not as a single-file or single-source phenomenon, but as a repository-wide, multi-artifact signal.

The corpus covers 950 open-source GitHub projects. Repository selection was constrained by popularity and activity: stars at least 500, forks at least 100, at least one commit per week over the 12 months prior to dataset creation, and exclusion of archived or read-only projects. The repositories span Java, Python, JavaScript, C++, and Ruby, and cover frameworks, libraries, developer tools, machine-learning projects, and system utilities. Examples named in the dataset description include Django, React, NumPy, TensorFlow, and VSCode extensions.

This composition makes DocMine suitable for studies that require both breadth and heterogeneity. A plausible implication is that the dataset supports analyses of documentation behavior across repository types rather than within a single ecosystem alone.

## 2. Repository artifacts and schema

DocMine spans five artifact categories: `Commit`, `IssueComment`, `PullRequest`, `CodeComment`, and `TextualFile`. Every artifact is stored as a JSON object with the core fields `repo_name`, `artifact_id`, `artifact_type`, `text`, `metadata`, and `doc_types`.

The field `repo_name` stores the GitHub `owner/repo` identifier. The field `artifact_id` is a unique GUID whose form depends on artifact type: commit hash; issue or pull-request number with a type prefix; or file path with line range for comments. The `artifact_type` field is an enum over the five artifact classes. The `text` field contains extracted natural-language content, with code snippets stripped and markup cleaned. The `metadata` object holds type-specific attributes. For commits, metadata includes `author`, `timestamp`, and `parent_hashes`; for issues and pull requests, `author`, `timestamp`, `issue_number`, and `title`; for code comments, `author`, `timestamp`, `file_path`, `language`, and `comment_style`; and for textual files, `file_path` and file extension among `.md`, `.rst`, `.txt`, and `.adoc`. The `doc_types` field is an array containing one or more labels from the six-type documentation taxonomy.

This schema is designed for cross-artifact retrieval and generation workflows. Because `doc_types` is multi-label rather than single-label, an individual artifact can simultaneously contribute to multiple documentation views.

## 3. Data collection and preprocessing pipeline

Artifact harvesting used the GitHub REST API v3 to clone and iterate commits, issues, pull requests, and repository file trees. Several preprocessing steps were then applied to normalize the corpus.

For commit messages, merge commits were excluded and messages longer than 500 characters were truncated. For issues and pull requests, both titles and comment bodies were extracted, and bot-generated content was removed. Source-code comments were parsed with tree-sitter for each language; only comment tokens were retained, while commented-out code was discarded. Textual documentation files were selected only if their extensions were in `{.md, .rst, .txt, .adoc}`; YAML front matter was removed, and boilerplate license headers were filtered out.

Deduplication collapsed artifacts that shared identical `text` and `artifact_type` within a repository into a single record. Language filtering retained only English content, detected via `langdetect` with confidence at least 0.90.

These choices constrain the corpus to cleaned natural-language units rather than arbitrary repository text. This suggests that DocMine is intended to support both generation tasks and controlled empirical analysis, where noisy boilerplate and duplicate fragments would otherwise distort results.

## 4. Documentation taxonomy and annotation methodology

DocMine uses six documentation types: File-related, Error/Bug-related, Project-related, API-related, License-related, and Architecture-related. These categories were defined through a preliminary qualitative study on 1.38 million artifacts across 950 repositories by Venigalla and Chimalakonda in *IST 2024*.

The annotation workflow combined manual and automated stages. First, a stratified random sample of 1,205 artifacts, chosen at 99% confidence and a 5% margin of error, was labeled by two expert raters. Inter-rater agreement was reported as Cohen’s $\kappa = 1.00$, with 100% agreement after two adjudication rounds. Those manual annotations then seeded automated labeling through a rule-and-statistical-classifier hybrid using TF-IDF and Naïve Bayes, which was applied to the full 1.38 million artifact set. A secondary 10% random hold-out was manually checked, yielding $F_1 \geq 0.82$ for each `doc_type` label.

The annotation procedure is notable for combining a high-agreement expert sample with full-corpus automatic propagation. In practical terms, this positions DocMine as a labeled corpus rather than merely a harvested archive.

## 5. Statistical profile

The dataset reports more than 1,380,000 artifacts overall, with a detailed type-level count totaling 1,379,400 records across the five artifact classes.

| Artifact type | Count | % of total |
|---|---:|---:|
| Commit Messages | 460,200 | 33.3% |
| Issue Comments | 315,400 | 22.8% |
| Pull-Request Descriptions | 274,600 | 19.9% |
| Source-Code Comments | 203,800 | 14.8% |
| Textual Files | 125,400 | 9.1% |

Across all artifact sources, the aggregate proportions of documentation types are reported as follows.

| Documentation type | Aggregate proportion |
|---|---:|
| File-related | 15.1% |
| Error/Bug-related | 27.5% |
| Project-related | 28.1% |
| API-related | 5.2% |
| License-related | 11.2% |
| Architecture-related | 12.9% |

The source-wise distribution excerpt shows that documentation content is unevenly distributed across artifact categories. Commits contain 31.5% Error/Bug-related and 39.0% Project-related content. Issues contain 35.5% Error/Bug-related content. Pull requests contain 36.7% Error/Bug-related content. Textual files contain 15.3% License-related content. Code comments contain 11.6% API-related and 29.2% License-related content. A common simplification in documentation mining is to privilege only source-code comments or textual documentation files; DocMine’s reported distributions indicate that substantial documentation-relevant content also resides in commits, issues, and pull requests.

## 6. Validation, distribution, and role in DocFetch

DocMine includes several quality and integrity checks. Manual-to-automated label consistency on hold-out samples reached $F_1$-scores of at least 0.82, precision of at least 0.80, and recall of at least 0.84 for all six documentation types. All records conform to a tight JSON Schema, with no missing required fields and correct enum values. SHA-256 checksums were verified for all `text` fields to detect inadvertent corruption. After deduplication, fewer than 0.1% of artifacts shared identical SHA-256 hashes across different `artifact_id` values, indicating negligible duplicate leakage. Coverage validation via spot checks against live GitHub APIs confirmed at least 99.5% recall of commit and issue records for a sample of 50 repositories.

DocMine is distributed as compressed JSONL files, one per artifact type, together with index files mapping `repo_name` to `artifact_id`. It can be loaded through standard JSON parsers. Because artifacts may carry multiple `doc_type` labels, filtering by a single documentation category may require explicit label selection.

Within the DocFetch framework, DocMine functions as the consolidated substrate from which structured documentation is generated [2508.17719]. DocFetch employs a multi-layer prompt-based LLM and generates documentation for different documentation types from multiple software artifacts. Its evaluation, conducted using a manually curated ground-truth dataset derived by analyzing artifacts in DocMine, reported a highest BLEU-4 score of 43.24% and a ROUGE-L score of 0.39 for generation of API-related and File-related information from five documentation sources; other documentation-type outputs reported BLEU-4 scores close to 30%. In that setting, DocMine is not only a corpus for descriptive analysis, but also an operational benchmark for semi-automatic documentation generation, documentation co-evolution studies, and supervised or semi-supervised learning in software engineering research.

Source: https://www.emergentmind.com/topics/docmine-dataset