Code2Doc: Automatic Code Documentation
- Code2Doc is a suite of tasks that converts source code and repository context into natural language documentation, ranging from short summaries to repository-wide overviews.
- It employs diverse methods including sequence-to-sequence learning, contrastive alignment, prompt engineering, and structured multi-artifact synthesis.
- The approach emphasizes maintenance and alignment, ensuring that documentation evolves in sync with code changes to reduce developer effort.
to=arxiv_search.query _人人碰ing муһим 彩娱乐彩票 买天天中彩票 天天中彩票会json {"query":"all:(Code2Doc OR code documentation generation OR code-to-documentation)","max_results":10,"sort_by":"submittedDate","sort_order":"descending"} to=arxiv_search.search 大发快三开奖结果 天天好彩票 天天彩票与你同行 曾道人 to=arxiv_search.search code 彩票开号 天天中彩票中奖json {"query":"Code2Doc code documentation generation", "max_results": 10} to=arxiv_search рҟынӡа 彩神争霸平台json {"query":"Code2Doc automatic code documentation generation", "max_results": 10} Code2Doc denotes the family of software-engineering tasks in which source code, or code together with repository context, is transformed into documentation in natural language or other documentation-oriented representations. In the literature, the term covers short summaries of methods, explanatory docstrings, template-based Javadoc, maintenance-oriented documentation updates, repository-level documentation, and structured outputs synthesized from multiple software artifacts. Formalizations range from sequence-to-sequence generation of a summary conditioned on a method token sequence to maintenance settings in which a model receives and must generate an aligned (Gupta, 2020, Pai et al., 1 Feb 2025). Recent systems extend beyond isolated function-level prompting by using dependency-aware traversal, multi-agent workflows, hierarchical synthesis, and multi-artifact fusion (Yang et al., 11 Apr 2025, Anh et al., 28 Oct 2025, Venigalla et al., 25 Aug 2025).
1. Scope of the task
At its narrowest, Code2Doc is the generation of a short natural-language description of code behavior. DeepSumm formulates the problem as learning a conditional model from Java method tokens to summaries, with training under token-level cross-entropy (Gupta, 2020). CodeExp broadens the target from short summaries to explanatory docstrings that describe both high-level intent and low-level implementation details, with explicit attention to arguments, returns, exceptions, branching behavior, coherence, and fluency (Cui et al., 2022).
A broader view treats Code2Doc as a maintenance problem rather than a one-shot summarization problem. CoDocBench defines Code2Doc as “Generate Updated Docstring,” where the input is and the output is aligned with ; it also defines the inverse Doc2Code task and an alignment-verification setting based on temporal order (Pai et al., 1 Feb 2025). This framing shifts emphasis from generic summarization toward incremental edits, traceability, and developer effort.
The output space is also heterogeneous. DocAgent generates function, method, and class docstrings with explicit sections such as Summary, Description, Args, Returns, Raises, Examples, and Attributes (Yang et al., 11 Apr 2025). The Java-focused context-aware Javadoc work adopts a canonical schema with conditional tags including @param, @return, @throws, @deprecated, and @see (Sarker et al., 16 Sep 2025). At higher abstraction levels, HGEN generates hierarchies of artifacts such as user stories, requirements, and design specifications, while CodeWiki synthesizes repository-level overviews, public-API usage guides, architecture diagrams, and data-flow diagrams (Dearstyne et al., 2024, Anh et al., 28 Oct 2025).
A common misconception is that Code2Doc is equivalent to first-line comment generation for single functions. The literature instead describes a continuum from local summarization to explanatory documentation, change-aware repair, template-constrained documentation, multi-level document hierarchies, and repository-scale synthesis (Cui et al., 2022, Pai et al., 1 Feb 2025, Dearstyne et al., 2024).
2. Datasets, corpora, and benchmark construction
Progress in Code2Doc has been closely tied to dataset design. Early large-scale work used a Java method–comment corpus derived from LeClair et al. and filtered to methods of at most 100 tokens and comments of length 3–13 words, producing a final tokenized set of approximately 2.1 million pairs (Gupta, 2020). Later work increasingly emphasized quality control over raw scale.
| Resource | Focus | Scale |
|---|---|---|
| DeepSumm corpus (Gupta, 2020) | Java method ↔ comment pairs | ~2.1 million |
| CodeExp(refined) (Cui et al., 2022) | High-quality Python explanatory docstrings | 158,024 |
| CoDocBench (Pai et al., 1 Feb 2025) | Paired Python code–docstring changes | 4,573 |
| Context-aware Javadoc dataset (Sarker et al., 16 Sep 2025) | Java code-Javadoc pairs | 3,614 |
| Code2Doc dataset (Karaman et al., 21 Dec 2025) | Curated multilingual function documentation | 13,358 |
CodeExp distinguishes between a large raw corpus and a refined subset. It reports 2,285,387 raw Python code–docstring pairs extracted from approximately 55,000 GitHub repositories, then constructs a 158,024-example refined set by filtering with a BERT-base regressor trained on 13,186 human-annotated examples (Cui et al., 2022). This design encodes a strong claim already explicit in the paper: refined supervision can outperform much larger but noisier training data.
CoDocBench addresses a different data regime: temporally coupled changes in code and documentation. It begins from the top 200 Python projects on GitHub, excludes forks and archived repositories, mines commits with PyDriller, identifies function/docstring pairs with regular expressions, validates pairing with Tree-Sitter and CodeSearchNet’s function_parser, and retains only commits where both function code and its docstring changed in the same commit (Pai et al., 1 Feb 2025). The final benchmark contains 4,573 code–docstring paired-change samples, split 50/50 at the project level.
The 2025 Code2Doc dataset is explicitly “quality-first curated.” Starting from 52,069 extracted candidates, it applies four stages—basic filtering, automated quality scoring, deduplication, and AI-generated documentation detection—and retains 13,358 samples spanning Python, Java, TypeScript, JavaScript, and C++ (Karaman et al., 21 Dec 2025). The dataset reports a mean documentation quality score of 6.93 out of 10, 86.9% explicit type annotations, and 2.9% potentially AI-generated samples.
Dataset construction also reflects domain-specific output constraints. The Java Javadoc study scans approximately 5,128 .java files containing Javadoc blocks, filters to 3,614 high-quality pairs, and augments each entry with package, enclosing class, imports, method modifiers, and related method signatures (Sarker et al., 16 Sep 2025). This suggests that, for template-based documentation, context fields are part of the benchmark definition rather than merely auxiliary metadata.
3. Methodological families
One major methodological line is end-to-end neural sequence generation. DeepSumm uses a Transformer encoder–decoder for Java method summarization and argues that recurrence is unnecessary for the task; its best large-scale configuration has three encoder layers, three decoder layers, model dimension 256, eight attention heads, and approximately 76.6 million parameters (Gupta, 2020). CodeExp treats explanation generation as sequence-to-sequence learning and studies multi-stage fine-tuning strategies over GPT-2, GPT-Neo, and CodeT5 backbones (Cui et al., 2022).
A second line combines discriminative alignment modeling with generation. DocChecker builds on UniXcoder, attaches contrastive, binary-classification, and text-generation heads, pre-trains jointly with Code–Text Contrastive, classification, and generation losses, and then uses a detect-then-repair inference pattern: inconsistent code–comment pairs are flagged and repaired by generating a fresh comment (Dau et al., 2023). This architecture is oriented toward inconsistency detection and resolution rather than unconditional summarization.
A third line uses prompt engineering, templates, and structured outputs. COMCAT parses C/C++ code with Clang/LLVM, identifies comment locations, predicts one of four comment types—Function, Variable, SnippetFunctionality, or Branch—with a CodeBERT-based classifier, and then fills a type-specific template before calling ChatGPT-3.5 (Grandel et al., 2024). The context-aware Javadoc work similarly encodes structural and semantic context with markers such as [PACKAGE], [[CLASS](https://www.emergentmind.com/topics/colorado-learning-attitudes-about-science-survey-class)], [IMPORTS], [SIGNATURE], and [CALL_GRAPH], then evaluates zero-shot, few-shot, and LoRA-fine-tuned open-source LLMs (Sarker et al., 16 Sep 2025). DocFetch generalizes this template-and-prompt logic to multi-source documentation, using five parallel level-1 prompts over PullRequest, Issue, Commit, CodeComment, and TextFile sources, followed by a level-2 fusion prompt that consolidates intermediate JSON outputs (Venigalla et al., 25 Aug 2025).
A fourth line introduces dependency-aware and agentic processing. DocAgent constructs a dependency graph over functions, methods, and classes, condenses cycles via Tarjan’s algorithm, topologically sorts the resulting DAG, and then documents components in “Dependencies First” order with specialized Reader, Searcher, Writer, Verifier, and Orchestrator agents (Yang et al., 11 Apr 2025). CodeWiki extends this logic to repository scale through hierarchical decomposition, recursive agentic processing with dynamic delegation, cross-module reference tracking, and synthesis of textual and visual artifacts (Anh et al., 28 Oct 2025).
A fifth line treats Code2Doc as an intermediate representation problem. In the specification-based Code→Text→Code framework, the Code2Text stage generates a neutral textual specification capturing intent, inputs, outputs, control flow, branches, side effects, dependencies, and excluded behavior, then validates and refines that specification against AST- and graph-derived metadata before handoff to Text2Code (Grynets et al., 24 May 2026). The paper reports that Code→Natural-Language IR delivered the best end-to-end performance among direct text, pseudocode IR, natural-language IR, graph IR, and compiler IR.
4. Maintenance, alignment, and verification
Maintenance-oriented Code2Doc emphasizes temporal consistency between evolving code and evolving documentation. CoDocBench formalizes this with both update generation and alignment verification. For RQ1, a generated docstring or code artifact for version is correctly aligned if its edit distance to the matching reference is smaller than its edit distance to the mismatched reference; for RQ2, a generated new artifact is correct if it is closer to the new reference than to the old one (Pai et al., 1 Feb 2025). The benchmark therefore measures whether a model captures the delta rather than whether it can restate the entire function.
This maintenance framing exposes a key difficulty: documentation edits are not simply scaled-down versions of code edits. CoDocBench reports a weak correlation between code-diff size and docstring-diff size, and notes that documentation updates vary in granularity relative to code changes (Pai et al., 1 Feb 2025). A plausible implication is that models must learn not only semantics but also editorial selectivity.
DocChecker studies a related but distinct problem: detecting and resolving code-comment inconsistencies. Its fine-tuning data for ICCD comes from the Just-In-Time dataset in the post-hoc setting, and inference first predicts whether a comment is inconsistent before regenerating it if necessary (Dau et al., 2023). This detect-then-repair strategy contrasts with unconditional generation and is closer to deployment scenarios where existing documentation is present but stale.
Verification mechanisms recur across the literature. DocAgent’s Verifier checks coverage of required sections, factual consistency, and level of detail, and can trigger additional context retrieval when needed (Yang et al., 11 Apr 2025). The specification-based Code2Text framework validates generated specifications against source ASTs and factual context, patching only missing segments rather than regenerating the whole specification (Grynets et al., 24 May 2026). These designs reflect an explicit shift from one-pass generation toward controlled documentation synthesis.
5. Evaluation methodology
Evaluation in Code2Doc is notably pluralistic. Traditional n-gram overlap metrics remain common: DeepSumm reports BLEU on method summaries (Gupta, 2020); CodeExp reports BLEU, ROUGE-1, ROUGE-L, METEOR, BERTScore, CodeBERTScore, and Common Entity Recall, and studies their Kendall’s 0 correlation with human judgments (Cui et al., 2022); the Java Javadoc study uses BLEU-N and ROUGE variants and recommends CodeBLEU for future systems, though it does not evaluate CodeBLEU in the paper (Sarker et al., 16 Sep 2025).
Maintenance benchmarks often require different metrics. CoDocBench uses Levenshtein edit distance as its primary metric because it applies to both code and text and sensitively reflects small incremental changes; the paper explicitly defines no BLEU, CodeBLEU, or ChrF (Pai et al., 1 Feb 2025). This choice aligns the metric with the underlying maintenance objective: minimizing developer effort for small updates.
Several papers argue that overlap metrics alone are inadequate. DocAgent introduces Completeness, Helpfulness, and Truthfulness. Completeness is computed from required sections derived from the code signature; Helpfulness is an LLM-as-judge score over summary, description, and parameter descriptions using a 5-point Likert rubric; Truthfulness is an Existence Ratio obtained by extracting entity mentions from the generated docstring and checking whether they exist in the dependency graph (Yang et al., 11 Apr 2025). CodeWikiBench similarly uses repository-specific hierarchical rubrics with LLM-as-Judge agents and propagates scores and uncertainties from leaves to the repository root (Anh et al., 28 Oct 2025).
Human-centered evaluation remains prominent. COMCAT evaluates whether generated comments improve code comprehension across short-answer, code-writing, and debugging tasks, and whether developers prefer them over human or vanilla ChatGPT comments (Grandel et al., 2024). HGEN assesses artifact quality on six human-judged metrics—Readability, Appropriateness, Conciseness, Importance, Usefulness, and Helpfulness—alongside concept coverage and traceability (Dearstyne et al., 2024). The field therefore treats evaluation as a combination of textual similarity, factuality, usefulness, traceability, and maintenance utility.
6. Empirical results, limitations, and future directions
Empirical results show clear progress but also persistent difficulty. DeepSumm reports a BLEU score of 17.99 on the large Java test set and more than 50% reduction in training time relative to an attention-RNN baseline (Gupta, 2020). DocChecker achieves 72.3% accuracy and 74.3 F1 on inconsistency detection and 33.64 BLEU-4 on code summarization, surpassing GPT-3.5 and CodeLlama baselines reported in the paper (Dau et al., 2023). CodeExp finds that two-stage or refined-data fine-tuning can produce long docstrings comparable to human-written ones, with CodeT5(refined) nearly matching human reference overall and exceeding it on Coverage in the reported human evaluation (Cui et al., 2022).
Context and structure repeatedly help. In DocAgent, DA-GPT reaches overall Completeness 0.934 and Truthfulness 95.74%, and ablations show that randomizing the processing order degrades helpfulness and truthfulness (Yang et al., 11 Apr 2025). In the Java Javadoc study, fine-tuned LLaMA-3.1-8B reaches BLEU 0.661, outperforming the other evaluated open-source models after fine-tuning (Sarker et al., 16 Sep 2025). The curated Code2Doc dataset reports that PEFT fine-tuning of Llama 3.1 8B improves BLEU by 29.47% and ROUGE-L by 24.04% over zero-shot performance despite the dataset’s modest size (Karaman et al., 21 Dec 2025).
Maintenance-oriented tasks remain challenging. On CoDocBench, the best overall result is Mixtral docstring update in 3-shot plus context with 1,311 correct cases out of 2,273 test samples, approximately 58%, and the paper notes that models are notably better at docstring updates than code updates (Pai et al., 1 Feb 2025). This is consistent with the benchmark’s conclusion that current models find maintenance-related tasks challenging.
Higher-level documentation shows both promise and remaining gaps. HGEN reports concept coverage of 87.5%, 84.4%, and 100.0% on three projects, substantially above the baseline figures of 6.3%, 37.8%, and 50.0%, and achieves strong traceability metrics with very few orphans (Dearstyne et al., 2024). CodeWiki reports a 68.79% quality score with proprietary models and 64.80% with open-source alternatives, with systems languages remaining difficult at approximately 50–56% (Anh et al., 28 Oct 2025). DocFetch attains a highest BLEU-4 of 43.24% and ROUGE-L of 0.39 for generation of API-related and file-related information from five documentation sources, while also acknowledging occasional hallucination and lower performance for textual files and project-level documentation (Venigalla et al., 25 Aug 2025).
The limitations reported across the literature are consistent. Several systems are Python-only or Java-specific; static AST analysis misses dynamic behavior; repository context can exceed context windows; multi-agent coordination is resource-intensive; maintenance datasets cannot always track moved or renamed functions; and automatic metrics remain imperfect proxies for factuality or developer usefulness (Yang et al., 11 Apr 2025, Pai et al., 1 Feb 2025, Karaman et al., 21 Dec 2025). Future directions named explicitly in the papers include richer semantic metrics beyond edit distance, dynamic analysis and runtime traces, multi-language and multi-file extensions, hybrid retrieval over vector and graph stores, project-level context integration, hierarchical agent planning, human-in-the-loop validation, and stronger safeguards against hallucination and AI-generated-data contamination (Pai et al., 1 Feb 2025, Grynets et al., 24 May 2026, Karaman et al., 21 Dec 2025).
Taken together, these results suggest that Code2Doc has evolved from code summarization into a broader research area centered on alignment, factual grounding, and multi-level software documentation. The dominant trend is away from isolated one-shot generation and toward systems that combine curated data, explicit structural context, verification, and task-specific evaluation.
