Context-as-AI-Service: Surfacing Cross-File Dependency Chains for LLM-Generated Developer Documentation

Abstract: LLM agents increasingly write and maintain developer documentation, but usefulness and accuracy often rely on dependency chains that are not obvious to follow. Even with more files in context, the agent must still decide which cross-file dependencies to trace. We present Context-as-AI-Service (CAIS), a retrieval layer that LLM agents query to find evidence across the codebase as they review or generate documentation. CAIS indexes source code, API references, and upstream documentation, then enables agents to query the index through tool calls that combine keyword and semantic search. We evaluate CAIS in two case studies using Claude Sonnet 4.6 on a production SDK: improving API reference comments in a core source file and validating an LLM-generated tutorial. In both studies, the baseline already had ordinary repository tools such as file reads, keyword search, and symbol navigation. CAIS adds a retrieval layer on top, so the comparison isolates added retrieval rather than basic repository access. In the API-reference review, the CAIS-augmented agent produced the same 5 missing-documentation fixes as the baseline and surfaced 4 findings the baseline missed: 2 cross-file factual errors and 2 underspecified API comments. In the tutorial validation, it surfaced 1 executable bug, 1 API-usage improvement, and 2 missing prerequisites that the baseline pipeline did not catch. These findings required tracing non-obvious dependency chains across utility files, framework internals, usage examples, tests, and component-creation logic. Over five runs per condition, adding CAIS reduced wall-clock time by 22% to 34% across the two tasks and lowered input-token usage.

• The paper introduces CAIS, a retrieval module that enhances documentation accuracy by surfacing cross-file dependencies often overlooked by conventional LLM agents.
• It combines BM25 and DRAMA embedding techniques to fuse lexical and semantic evidence for resolving distributed documentation errors.
• Empirical results show CAIS reduces processing time by 22–34% and uncovers 8 additional cross-file findings, improving code review workflows.

Context-as-AI-Service: Surfacing Cross-File Dependency Chains for LLM-Generated Developer Documentation

Introduction

The proliferation of LLM-based agents in software engineering has enabled automated workflows in code documentation, yet a persistent limitation remains: the validation and accuracy of documentation is fundamentally constrained by cross-file dependencies that are not trivial to trace or aggregate. The paper "Context-as-AI-Service: Surfacing Cross-File Dependency Chains for LLM-Generated Developer Documentation" (2606.04397) addresses the inadequacy of local context and heuristics-driven tools by introducing Context-as-AI-Service (CAIS), a retrieval layer explicitly designed to operationalize evidence gathering across semantically dispersed code artifacts.

Architecture and Workflow

CAIS integrates as an agent-callable retrieval module, mediating between LLM agents and a heterogeneously indexed corpus spanning source code, API references, tests, usage examples, and upstream documentation.

Figure 1: CAIS system architecture, wherein the LLM agent queries the retrieval service mid-documentation workflow, surfacing distributed evidence from the codebase for review and correction.

The architecture comprises four distinct stages:

1. Source Ingestion: Aggregates sources across code, internal APIs, and external documentation, normalizing inputs for downstream search.
2. Storage and Indexing: Utilizes token-level storage (both raw and semantically enriched artifacts), employing both BM25 for lexical and DRAMA embeddings for dense retrieval, thus supporting both keyword and behavioral queries.
3. Retrieval Interface: Provides reciprocal rank fusion between BM25 and DRAMA search. The agent’s queries can be symbolic, semantic, or behavioral, and responses return fine-grained snippet-level evidence, not merely file-level context.
4. Review Layer: Supports LLM-driven review prompts that label documentation claims with structured verdicts (e.g., consistent, contradicted, incomplete), leveraging snippets as candidate—but not definitive—grounding.

Case Study Protocol

The evaluation design contrasts a baseline LLM agent equipped with common repository tools (file reads, lexical search, symbol navigation) with the identical agent augmented by CAIS retrieval across a proprietary SDK.

Figure 2: Case-study protocol design contrasting baseline (repository tools only) and CAIS-augmented workflows on documentation tasks, focusing on cross-file evidence discovery.

The protocol is qualitative, focusing on the surfacing and auditing of additional findings attributable to CAIS queries, in two developer workflow contexts: API reference improvement and tutorial generation/validation.

Empirical Results

API Reference Improvement

Without retrieval augmentation, the agent addresses missing-comments on public members but fails to detect errors whose ground-truth resides in distributed, semantically-linked files. With CAIS, the agent:

• Identifies stale references and semantic errors, such as misdocumented parameter types only resolvable by inspecting registry and query-layer usages.
• Detects temporal semantics inconsistencies (e.g., immediate vs. deferred removal semantics).
• Surfaces underspecified documentation, for example where equality checks depend on backing objects evident only in test and implementation code.
• Augments vague documentation with precise inter-client semantics extracted from distributed filter logic.

Tutorial Generation and Validation

The addition of CAIS enables identification of:

• Executable bugs: E.g., syntactically plausible but functionally incorrect resource URIs, detectable only against canonical examples external to the local file.
• API idiom suboptimality: Recommending more idiomatic constructors from utility modules residing outside the focal codepath.
• Critical missing prerequisites: For instance, inheritance requirements and auto-generation logic for interactive behavior missed by local validation.

Efficiency and Numerical Impact

The inclusion of CAIS reduces wall-clock time by 22–34% across both documentation review and tutorial validation tasks, and significantly decreases input token usage. CAIS-driven workflows yield 8 additional findings not detected by repository tools, where each finding is systematically linked to a cross-file or cross-system dependency chain.

Theoretical and Practical Implications

Failure Modes in LLM-Generated Documentation

The central insight is that LLM agents—even with expanded context windows—suffer from false positives in documentation validation when the relevant dependencies are non-local and lexically non-obvious. The study formalizes a taxonomy:

• Single-file: Local, directly resolvable issues.
• Cross-file: Semantic errors, missing prerequisites, and implicit dependencies.
• Cross-system: Errors contingent on upstream/external specifications.

The demonstrable uplift resides in cross-file/cross-system regimes, which are intractable for standard LLM workflows relying on static or heuristics-based context selection.

Retrieval-Augmentation as an Inductive Bias

By embedding composite retrieval (BM25 + DRAMA) in the agent loop, CAIS effectively acts as an inductive bias, steering the exploration strategy of the agent during code understanding and documentation tasks. This approach decouples repository traversal from context-window limitations and manual symbol selection, shifting the workflow toward evidence-based validation.

Future Directions

• Symbol-level and diff-level retrieval: Precision could be increased by integrating code symbol graphs and temporal provenance, minimizing snippet-level overfetch and context wastage.
• Comparative evaluation with large-context and explicit dependency-chasing strategies: Systematic benchmarks remain to be run against LLMs prompted or fine-tuned for global dependency tracing, as opposed to bag-of-tools approaches.
• Generalization: CAIS-style retrieval could be repurposed for agent-driven code review, test synthesis, and bug detection, given the decoupling of retrieval and reasoning components.

Conclusion

CAIS constitutes a significant advancement in retrieval-augmented agent workflows for developer documentation by operationalizing context discovery over codebases where non-trivial dependency chains are requisite for documentation accuracy. Numeric improvements in both finding recall and agent efficiency are supported by auditing of qualitative evidence trails across semantically separated files. Symbolic, behavioral, and upstream documentation chains are exposed as first-class entities in agent reasoning loops, suggesting broader utility for agent-driven software engineering automation.