Readme_AI Metadata Protocol

• Readme_AI is a dynamic metadata protocol that attaches owner-curated context to LLM queries, addressing model hallucinations and grounding errors.
• It employs a Model Context Protocol server to retrieve and expand a specially formatted Readme_AI.json file at query time.
• The protocol supports versatile content handling with 'fetch', 'crawl', and 'download' modes to enhance context accuracy for niche libraries and fast-moving datasets.

Searching arXiv for the specified Readme_AI paper and closely related work to ground the encyclopedia entry. Readme_AI is a proposed metadata specification and implementation prototype for dynamically constructing LLM context from an owner’s data source. Its central mechanism is an owner-curated Readme_AI.json file, placed at the root of a repository or other source, which is retrieved by a Model Context Protocol (MCP) server and expanded into query-time context for the model. The design targets two recurrent failure modes of LLMs: hallucination and poor query-specific grounding, especially for niche libraries, specialized datasets, and fast-moving codebases whose correct interpretation depends on local, current, and owner-approved information (Vyas et al., 12 Sep 2025).

1. Conceptual definition and problem addressed

Readme_AI is framed as a response to the fact that LLMs often answer from an average of their training distribution rather than from the specific truth of a target data source. The problem becomes acute when a query concerns a niche library, a specialized dataset, a fast-moving codebase, or any source whose correct answer depends on project-specific documentation, code, or publications. In such cases, a model may generate a plausible but false account because it is insufficiently anchored to validated source material (Vyas et al., 12 Sep 2025).

The paper’s motivating example is the NIST-developed Hedgehog library. Without Readme_AI, the model summarized Hedgehog as “a C++ software framework ... for implementing and testing various numerical methods for solving partial differential equations,” which the paper identifies as a hallucination. Hedgehog is instead described as a C++ library developed by NIST for creating parallel computations on heterogeneous nodes. Readme_AI therefore addresses not merely document retrieval in the generic sense, but the more constrained task of supplying the model with owner-provided grounding so that its reasoning is conditioned on the data source’s validated description rather than on diffuse pretraining priors (Vyas et al., 12 Sep 2025).

A common misconception is to treat Readme_AI as a static README variant. The specification is instead a protocol for dynamic context assembly: metadata declares what should be fetched, crawled, or downloaded at query time, and the MCP server operationalizes that declaration into structured context for inference (Vyas et al., 12 Sep 2025).

2. Specification and formal structure

At the specification level, Readme_AI defines a single JSON file, Readme_AI.json, located at the root of the data source. The file is a dictionary of top-level keys to content, and those keys become tags in the final model context. The specification permits two value forms: a simple string, or a structured object of the form {"data": ..., "type": ...}. In the structured case, data holds content locators such as file paths or URLs, while type selects the handling mode (Vyas et al., 12 Sep 2025).

The paper gives the following abstract grammar:

<readmeai_json> ::= <dict_kv_pair> <kv_pair> ::= <string>:<data_value> <data_value> ::= <string> | <structured_object> <structured_object> ::= { "data": <list_string> | <dict_string>, "type": <type_string> } <type_string> ::= "fetch" | "crawl" | "download"

The built-in handling modes are fetch, crawl, and download. fetch retrieves files from within a repository or local source; crawl crawls a website and gathers linked content; download directly downloads and parses files such as PDFs or datasets. The paper emphasizes that the specification is extensible in three senses: top-level keys are arbitrary user-defined tags, the type field can be extended with new handlers, and the structured object can be expanded with additional fields if required (Vyas et al., 12 Sep 2025).

This structure makes Readme_AI less a fixed schema than a general protocol for source-linked grounding. A plausible implication is that its expressive power depends less on a rigid ontology than on disciplined owner curation and stable handler semantics.

3. MCP server and end-to-end workflow

The implementation prototype is a Model Context Protocol server built in Python using FastMCP. The server functions as the bridge between a user query and the owner-curated metadata. A user asks a question in a client and provides either a known library or source name already registered, or a direct URL to the data source. The client sends the question and tool schemas to the model; the model decides whether to call the Readme_AI tool; the tool resolves the source; retrieves Readme_AI.json; parses the JSON; expands structured objects according to their type; and renders the result as model-friendly context. The original query plus the constructed context are then sent back to the model for final answer generation (Vyas et al., 12 Sep 2025).

The workflow described in the paper has several concrete operational details. If a URL is given, it can be stored in a lookup.json file with the name for future reuse. If the source is a git repository, retrieval uses git-based access. The output is formatted as XML, although the paper notes that Markdown would also be possible. The server dispatches on type using handlers described as get_relevant_papers for download, fetch_data for fetch, and web_crawler for crawl (Vyas et al., 12 Sep 2025).

The architecture is therefore not a free-form retrieval system. It is a controlled tool-use pipeline in which the model invokes an MCP tool, the tool resolves an owner-defined metadata file, and only then is context assembled. This controlledness is central to the paper’s account of how hallucinations are reduced.

4. Content types, tag semantics, and output organization

Readme_AI’s content model is organized around user-specified top-level tags. These tags group related content into sections such as description, api_files, documentation, papers, or websites, and they also provide semantic cues to the model about what kind of evidence it is reading. In the rendered output, the paper shows XML-like tags such as <DESCRIPTION>, <API>, <PAPERS>, and <WEBSITES>, so that the resulting context is partitioned into named sections rather than presented as undifferentiated text (Vyas et al., 12 Sep 2025).

Type	Function	Implementation detail
fetch	Retrieve files from a repository or local source	Uses repository access such as GitPython
crawl	Crawl a website and gather linked content	Uses BeautifulSoup4 / CRAWL package
download	Download and parse files such as PDFs or datasets	Uses PyPDF for PDF parsing

For codebases, fetch can be used to pull specific files, for example source files designated by path. For documentation sites, crawl starts from a base URL and iteratively follows links to gather content. For publications or PDF assets, download retrieves the file and parses it into text. General text can also be inserted directly as simple strings. The paper’s emphasis is not on a single data modality, but on a dynamic assembly mechanism that can mix repository files, web content, publications, and direct textual summaries within one context package (Vyas et al., 12 Sep 2025).

The role of tags is especially important. They do not merely label retrieved fragments; they let the owner shape the model’s reasoning space by determining which categories of information will be grouped together. The paper notes that meaningful tags could later be used to include only a subset of the context, which directly links the tagging scheme to token-budget management (Vyas et al., 12 Sep 2025).

5. Hedgehog case study and demonstrated effects on grounding

The paper’s evaluation is qualitative and centered on Hedgehog, a NIST-developed C++ library. In the unguided setting, the model produced an incorrect summary, associating Hedgehog with PDE-related numerical methods. With Readme_AI enabled, the client invoked the MCP tool using the repository URL, the tool fetched the curated metadata file, and the resulting context included a strong description plus structured sections covering documentation URLs, example code snippets, API references from the repository, and parsed publications. Under this grounded setting, the model answered correctly that Hedgehog is a C++ library developed by NIST and used for parallel computations on heterogeneous nodes (Vyas et al., 12 Sep 2025).

The paper also uses a code-generation example: a request for a parallel particle simulator using Hedgehog. Without Readme_AI, the model hallucinated a non-existent API and produced Python-like code, including constructs such as from hedgehog import Hedgehog and self.hedgehog.parallel_update(...). With Readme_AI, after grounding on the documentation and API code, the model produced a much more realistic C++ implementation using constructs such as #include <hedgehog/hedgehog.h>, hh::AbstractTask, and hh::Graph<...> (Vyas et al., 12 Sep 2025).

The significance assigned by the paper is precise. The claim is not that the model generated a complete production-ready simulator, but that grounding enabled it to reason about the library correctly, use the appropriate programming language, and interpolate from examples provided in the Readme_AI file to a new task. This suggests that the protocol’s value lies as much in constraining code synthesis as in improving descriptive factuality (Vyas et al., 12 Sep 2025).

6. Limitations, security, governance, and relation to adjacent work

The paper is explicit that the current evidence is case-study based rather than benchmark-driven. Its evaluation consists of before-and-after comparisons on the Hedgehog summary and coding example, not a large-scale quantitative benchmark. A major practical limitation is context size: the Hedgehog Readme_AI output is reported as about 97,240 tokens, which the paper characterizes as expensive and unwieldy for cloud inference. As a mitigation, the authors suggest an agentic approach in which the model identifies the most relevant tags and includes only those sections, though they also note the corresponding risk of omitting important details (Vyas et al., 12 Sep 2025).

Generalization is proposed but not fully realized. The paper argues that the protocol should generalize beyond git repositories, yet the prototype currently supports only Git-based sources. It also identifies security concerns: dynamic fetching from open sources can ingest malicious or poisoned content, and the paper suggests approved or blocked domain lists in the MCP server so that only validated sources may be used. Governance is likewise central, because the quality of the constructed context depends on the owner keeping Readme_AI.json accurate and current, and because recursive linkage or multiple sources would increase curation complexity (Vyas et al., 12 Sep 2025).

Readme_AI should also be distinguished from automatic README generation systems. “LARCH: LLM-based Automatic Readme Creation with Heuristics” (Koreeda et al., 2023) addresses the generation of a repository README in Markdown from source code, using a representative-file retrieval step before prompting the model. Readme_AI, by contrast, is not a repository summarization system; it is a protocol for owner-authored metadata and dynamic context retrieval at query time. The distinction is substantive: LARCH compresses repository understanding into a generated README, whereas Readme_AI externalizes owner-approved grounding so that a model can answer future queries using fetched, crawled, or downloaded evidence (Koreeda et al., 2023).

Taken together, these limitations and contrasts clarify Readme_AI’s position. It is neither a generic retrieval layer nor an autonomous documentation writer. It is an extensible, owner-governed mechanism for injecting source-linked context into the inference loop of a LLM, with the explicit objective of reducing hallucinations and improving query-specific grounding on specialized data sources (Vyas et al., 12 Sep 2025).