---
title: 'Code2Doc: Automatic Code Documentation'
url: https://www.emergentmind.com/topics/code2doc
type: topic
---

# Code2Doc: Automatic Code Documentation

to=arxiv_search.query _人人碰ing  муһим 彩娱乐彩票 买天天中彩票  天天中彩票会json
{"query":"all:(Code2Doc OR code documentation generation OR code-to-documentation)","max_results":10,"sort_by":"submittedDate","sort_order":"descending"}
to=arxiv_search.search  大发快三开奖结果 天天好彩票  天天彩票与你同行  曾道人 to=arxiv_search.search code 彩票开号  天天中彩票中奖json
{"query":"Code2Doc code documentation generation", "max_results": 10}
to=arxiv_search  рҟынӡа  彩神争霸平台json
{"query":"Code2Doc automatic code documentation generation", "max_results": 10}
Code2Doc denotes the family of software-engineering tasks in which source code, or code together with repository context, is transformed into documentation in natural language or other documentation-oriented representations. In the literature, the term covers short summaries of methods, explanatory docstrings, template-based Javadoc, maintenance-oriented documentation updates, repository-level documentation, and structured outputs synthesized from multiple software artifacts. Formalizations range from sequence-to-sequence generation of a summary $Y$ conditioned on a method token sequence $S$ to maintenance settings in which a model receives $(\text{old\_code}, \text{old\_docstring}, \text{new\_code})$ and must generate an aligned $\text{new\_docstring}$ [2004.00998] [2502.00519]. Recent systems extend beyond isolated function-level prompting by using dependency-aware traversal, multi-agent workflows, hierarchical synthesis, and multi-artifact fusion [2504.08725] [2510.24428] [2508.17719].

## 1. Scope of the task

At its narrowest, Code2Doc is the generation of a short natural-language description of code behavior. DeepSumm formulates the problem as learning a conditional model $P(Y \mid S)$ from Java method tokens to summaries, with training under token-level cross-entropy [2004.00998]. CodeExp broadens the target from short summaries to explanatory docstrings that describe both high-level intent and low-level implementation details, with explicit attention to arguments, returns, exceptions, branching behavior, coherence, and fluency [2211.15395].

A broader view treats Code2Doc as a maintenance problem rather than a one-shot summarization problem. CoDocBench defines Code2Doc as “Generate Updated Docstring,” where the input is $(\text{old\_code}, \text{old\_docstring}, \text{new\_code})$ and the output is $\text{new\_docstring}$ aligned with $\text{new\_code}$; it also defines the inverse Doc2Code task and an alignment-verification setting based on temporal order [2502.00519]. This framing shifts emphasis from generic summarization toward incremental edits, traceability, and developer effort.

The output space is also heterogeneous. DocAgent generates function, method, and class docstrings with explicit sections such as Summary, Description, Args, Returns, Raises, Examples, and Attributes [2504.08725]. The Java-focused context-aware Javadoc work adopts a canonical schema with conditional tags including `@param`, `@return`, `@throws`, `@deprecated`, and `@see` [2509.14273]. At higher abstraction levels, HGEN generates hierarchies of artifacts such as user stories, requirements, and design specifications, while CodeWiki synthesizes repository-level overviews, public-API usage guides, architecture diagrams, and data-flow diagrams [2408.05829] [2510.24428].

A common misconception is that Code2Doc is equivalent to first-line comment generation for single functions. The literature instead describes a continuum from local summarization to explanatory documentation, change-aware repair, template-constrained documentation, multi-level document hierarchies, and repository-scale synthesis [2211.15395] [2502.00519] [2408.05829].

## 2. Datasets, corpora, and benchmark construction

Progress in Code2Doc has been closely tied to dataset design. Early large-scale work used a Java method–comment corpus derived from LeClair et al. and filtered to methods of at most 100 tokens and comments of length 3–13 words, producing a final tokenized set of approximately 2.1 million pairs [2004.00998]. Later work increasingly emphasized quality control over raw scale.

| Resource | Focus | Scale |
|---|---|---|
| DeepSumm corpus [2004.00998] | Java method ↔ comment pairs | ~2.1 million |
| CodeExp(refined) [2211.15395] | High-quality Python explanatory docstrings | 158,024 |
| CoDocBench [2502.00519] | Paired Python code–docstring changes | 4,573 |
| Context-aware Javadoc dataset [2509.14273] | Java code-Javadoc pairs | 3,614 |
| Code2Doc dataset [2512.18748] | Curated multilingual function documentation | 13,358 |

CodeExp distinguishes between a large raw corpus and a refined subset. It reports 2,285,387 raw Python code–docstring pairs extracted from approximately 55,000 GitHub repositories, then constructs a 158,024-example refined set by filtering with a BERT-base regressor trained on 13,186 human-annotated examples [2211.15395]. This design encodes a strong claim already explicit in the paper: refined supervision can outperform much larger but noisier training data.

CoDocBench addresses a different data regime: temporally coupled changes in code and documentation. It begins from the top 200 Python projects on GitHub, excludes forks and archived repositories, mines commits with PyDriller, identifies function/docstring pairs with regular expressions, validates pairing with Tree-Sitter and CodeSearchNet’s `function_parser`, and retains only commits where both function code and its docstring changed in the same commit [2502.00519]. The final benchmark contains 4,573 code–docstring paired-change samples, split 50/50 at the project level.

The 2025 Code2Doc dataset is explicitly “quality-first curated.” Starting from 52,069 extracted candidates, it applies four stages—basic filtering, automated quality scoring, deduplication, and AI-generated documentation detection—and retains 13,358 samples spanning Python, Java, TypeScript, JavaScript, and C++ [2512.18748]. The dataset reports a mean documentation quality score of 6.93 out of 10, 86.9% explicit type annotations, and 2.9% potentially AI-generated samples.

Dataset construction also reflects domain-specific output constraints. The Java Javadoc study scans approximately 5,128 `.java` files containing Javadoc blocks, filters to 3,614 high-quality pairs, and augments each entry with package, enclosing class, imports, method modifiers, and related method signatures [2509.14273]. This suggests that, for template-based documentation, context fields are part of the benchmark definition rather than merely auxiliary metadata.

## 3. Methodological families

One major methodological line is end-to-end neural sequence generation. DeepSumm uses a Transformer encoder–decoder for Java method summarization and argues that recurrence is unnecessary for the task; its best large-scale configuration has three encoder layers, three decoder layers, model dimension 256, eight attention heads, and approximately 76.6 million parameters [2004.00998]. CodeExp treats explanation generation as sequence-to-sequence learning and studies multi-stage fine-tuning strategies over GPT-2, GPT-Neo, and CodeT5 backbones [2211.15395].

A second line combines discriminative alignment modeling with generation. DocChecker builds on UniXcoder, attaches contrastive, binary-classification, and text-generation heads, pre-trains jointly with Code–Text Contrastive, classification, and generation losses, and then uses a detect-then-repair inference pattern: inconsistent code–comment pairs are flagged and repaired by generating a fresh comment [2306.06347]. This architecture is oriented toward inconsistency detection and resolution rather than unconditional summarization.

A third line uses prompt engineering, templates, and structured outputs. COMCAT parses C/C++ code with Clang/LLVM, identifies comment locations, predicts one of four comment types—Function, Variable, SnippetFunctionality, or Branch—with a CodeBERT-based classifier, and then fills a type-specific template before calling ChatGPT-3.5 [2407.13648]. The context-aware Javadoc work similarly encodes structural and semantic context with markers such as `[PACKAGE]`, `[CLASS]`, `[IMPORTS]`, `[SIGNATURE]`, and `[CALL_GRAPH]`, then evaluates zero-shot, few-shot, and LoRA-fine-tuned open-source LLMs [2509.14273]. DocFetch generalizes this template-and-prompt logic to multi-source documentation, using five parallel level-1 prompts over PullRequest, Issue, Commit, CodeComment, and TextFile sources, followed by a level-2 fusion prompt that consolidates intermediate JSON outputs [2508.17719].

A fourth line introduces dependency-aware and agentic processing. DocAgent constructs a dependency graph $G=(V,E)$ over functions, methods, and classes, condenses cycles via Tarjan’s algorithm, topologically sorts the resulting DAG, and then documents components in “Dependencies First” order with specialized Reader, Searcher, Writer, Verifier, and Orchestrator agents [2504.08725]. CodeWiki extends this logic to repository scale through hierarchical decomposition, recursive agentic processing with dynamic delegation, cross-module reference tracking, and synthesis of textual and visual artifacts [2510.24428].

A fifth line treats Code2Doc as an intermediate representation problem. In the specification-based Code→Text→Code framework, the Code2Text stage generates a neutral textual specification capturing intent, inputs, outputs, control flow, branches, side effects, dependencies, and excluded behavior, then validates and refines that specification against AST- and graph-derived metadata before handoff to Text2Code [2605.25232]. The paper reports that Code→Natural-Language IR delivered the best end-to-end performance among direct text, pseudocode IR, natural-language IR, graph IR, and compiler IR.

## 4. Maintenance, alignment, and verification

Maintenance-oriented Code2Doc emphasizes temporal consistency between evolving code and evolving documentation. CoDocBench formalizes this with both update generation and alignment verification. For RQ1, a generated docstring or code artifact for version $V$ is correctly aligned if its edit distance to the matching reference is smaller than its edit distance to the mismatched reference; for RQ2, a generated new artifact is correct if it is closer to the new reference than to the old one [2502.00519]. The benchmark therefore measures whether a model captures the delta rather than whether it can restate the entire function.

This maintenance framing exposes a key difficulty: documentation edits are not simply scaled-down versions of code edits. CoDocBench reports a weak correlation between code-diff size and docstring-diff size, and notes that documentation updates vary in granularity relative to code changes [2502.00519]. A plausible implication is that models must learn not only semantics but also editorial selectivity.

DocChecker studies a related but distinct problem: detecting and resolving code-comment inconsistencies. Its fine-tuning data for ICCD comes from the Just-In-Time dataset in the post-hoc setting, and inference first predicts whether a comment is inconsistent before regenerating it if necessary [2306.06347]. This detect-then-repair strategy contrasts with unconditional generation and is closer to deployment scenarios where existing documentation is present but stale.

Verification mechanisms recur across the literature. DocAgent’s Verifier checks coverage of required sections, factual consistency, and level of detail, and can trigger additional context retrieval when needed [2504.08725]. The specification-based Code2Text framework validates generated specifications against source ASTs and factual context, patching only missing segments rather than regenerating the whole specification [2605.25232]. These designs reflect an explicit shift from one-pass generation toward controlled documentation synthesis.

## 5. Evaluation methodology

Evaluation in Code2Doc is notably pluralistic. Traditional n-gram overlap metrics remain common: DeepSumm reports BLEU on method summaries [2004.00998]; CodeExp reports BLEU, ROUGE-1, ROUGE-L, METEOR, BERTScore, CodeBERTScore, and Common Entity Recall, and studies their Kendall’s $\tau$ correlation with human judgments [2211.15395]; the Java Javadoc study uses BLEU-N and ROUGE variants and recommends CodeBLEU for future systems, though it does not evaluate CodeBLEU in the paper [2509.14273].

Maintenance benchmarks often require different metrics. CoDocBench uses Levenshtein edit distance as its primary metric because it applies to both code and text and sensitively reflects small incremental changes; the paper explicitly defines no BLEU, CodeBLEU, or ChrF [2502.00519]. This choice aligns the metric with the underlying maintenance objective: minimizing developer effort for small updates.

Several papers argue that overlap metrics alone are inadequate. DocAgent introduces Completeness, Helpfulness, and Truthfulness. Completeness is computed from required sections derived from the code signature; Helpfulness is an LLM-as-judge score over summary, description, and parameter descriptions using a 5-point Likert rubric; Truthfulness is an Existence Ratio obtained by extracting entity mentions from the generated docstring and checking whether they exist in the dependency graph [2504.08725]. CodeWikiBench similarly uses repository-specific hierarchical rubrics with LLM-as-Judge agents and propagates scores and uncertainties from leaves to the repository root [2510.24428].

Human-centered evaluation remains prominent. COMCAT evaluates whether generated comments improve code comprehension across short-answer, code-writing, and debugging tasks, and whether developers prefer them over human or vanilla ChatGPT comments [2407.13648]. HGEN assesses artifact quality on six human-judged metrics—Readability, Appropriateness, Conciseness, Importance, Usefulness, and Helpfulness—alongside concept coverage and traceability [2408.05829]. The field therefore treats evaluation as a combination of textual similarity, factuality, usefulness, traceability, and maintenance utility.

## 6. Empirical results, limitations, and future directions

Empirical results show clear progress but also persistent difficulty. DeepSumm reports a BLEU score of 17.99 on the large Java test set and more than 50% reduction in training time relative to an attention-RNN baseline [2004.00998]. DocChecker achieves 72.3% accuracy and 74.3 F1 on inconsistency detection and 33.64 BLEU-4 on code summarization, surpassing GPT-3.5 and CodeLlama baselines reported in the paper [2306.06347]. CodeExp finds that two-stage or refined-data fine-tuning can produce long docstrings comparable to human-written ones, with CodeT5(refined) nearly matching human reference overall and exceeding it on Coverage in the reported human evaluation [2211.15395].

Context and structure repeatedly help. In DocAgent, DA-GPT reaches overall Completeness 0.934 and Truthfulness 95.74%, and ablations show that randomizing the processing order degrades helpfulness and truthfulness [2504.08725]. In the Java Javadoc study, fine-tuned LLaMA-3.1-8B reaches BLEU 0.661, outperforming the other evaluated open-source models after fine-tuning [2509.14273]. The curated Code2Doc dataset reports that PEFT fine-tuning of Llama 3.1 8B improves BLEU by 29.47% and ROUGE-L by 24.04% over zero-shot performance despite the dataset’s modest size [2512.18748].

Maintenance-oriented tasks remain challenging. On CoDocBench, the best overall result is Mixtral docstring update in 3-shot plus context with 1,311 correct cases out of 2,273 test samples, approximately 58%, and the paper notes that models are notably better at docstring updates than code updates [2502.00519]. This is consistent with the benchmark’s conclusion that current models find maintenance-related tasks challenging.

Higher-level documentation shows both promise and remaining gaps. HGEN reports concept coverage of 87.5%, 84.4%, and 100.0% on three projects, substantially above the baseline figures of 6.3%, 37.8%, and 50.0%, and achieves strong traceability metrics with very few orphans [2408.05829]. CodeWiki reports a 68.79% quality score with proprietary models and 64.80% with open-source alternatives, with systems languages remaining difficult at approximately 50–56% [2510.24428]. DocFetch attains a highest BLEU-4 of 43.24% and ROUGE-L of 0.39 for generation of API-related and file-related information from five documentation sources, while also acknowledging occasional hallucination and lower performance for textual files and project-level documentation [2508.17719].

The limitations reported across the literature are consistent. Several systems are Python-only or Java-specific; static AST analysis misses dynamic behavior; repository context can exceed context windows; multi-agent coordination is resource-intensive; maintenance datasets cannot always track moved or renamed functions; and automatic metrics remain imperfect proxies for factuality or developer usefulness [2504.08725] [2502.00519] [2512.18748]. Future directions named explicitly in the papers include richer semantic metrics beyond edit distance, dynamic analysis and runtime traces, multi-language and multi-file extensions, hybrid retrieval over vector and graph stores, project-level context integration, hierarchical agent planning, human-in-the-loop validation, and stronger safeguards against hallucination and AI-generated-data contamination [2502.00519] [2605.25232] [2512.18748].

Taken together, these results suggest that Code2Doc has evolved from code summarization into a broader research area centered on alignment, factual grounding, and multi-level software documentation. The dominant trend is away from isolated one-shot generation and toward systems that combine curated data, explicit structural context, verification, and task-specific evaluation.

Source: https://www.emergentmind.com/topics/code2doc