KG2data: Graph-Grounded API for Meteorology
- KG2data is a domain-specific system that integrates a meteorological knowledge graph, an LLM, a ReAct-expert agent, and APIs to support intelligent data acquisition.
- It improves API-calling accuracy by leveraging graph-based persistent memory, achieving 88.57% ACAR and near-zero hallucination errors.
- KG2data offers a maintainable alternative to fine-tuning heavy approaches, enabling dynamic updates and robust handling of implicit, technical queries.
Searching arXiv for KG2data and closely related knowledge-graph-to-data workflow papers. KG2data is a domain-specific LLM API-calling system for meteorology that integrates a knowledge graph, an LLM, a ReAct-expert agent, and API/tool execution to support intelligent data acquisition and query handling from natural-language inputs. It is designed for knowledge-intensive settings in which specialized terminology, implicit intent, and rapidly changing domain knowledge make prompt-only API calling and vector-retrieval-only RAG unreliable. In the reported formulation, the knowledge graph functions as persistent memory and as a domain reasoning layer used to select tools, construct JSON parameters, and combine returned values with meteorological knowledge in the final answer (Yang et al., 14 Jul 2025).
1. Problem setting and motivation
KG2data addresses API calling in a specialized domain in which queries are often implicit and technical, LLMs lack domain-specific knowledge, domain knowledge and APIs evolve quickly, and RAG alone is not enough for deep semantic relations, complex queries, or heterogeneous structured and unstructured data. The target application is intelligent meteorological data acquisition and analysis through API calls rather than generic conversational question answering (Yang et al., 14 Jul 2025).
The system is framed against two alternative patterns. In chat2data, the knowledge graph is removed and only API functional descriptions remain in the prompt. In RAG2data, the overall pipeline is retained but the knowledge graph is replaced by a vector database. The conceptual distinction is that KG2data uses graph reasoning and semantic links as domain memory, RAG2data uses similarity search, and chat2data operates without a comparable domain memory layer. The paper argues that this difference matters most when the query does not explicitly name the needed API or parameter set.
A central motivation is maintainability. The paper explicitly contrasts KG2data with fine-tuning-heavy approaches, arguing that fine-tuning is expensive and slow when APIs or domain knowledge change. KG2data instead keeps knowledge external, structured, and maintainable through the graph and tool layer.
2. System architecture
KG2data consists of four major pieces: a meteorological knowledge graph, a ReAct-expert agent, an LLM, and data acquisition tools / APIs, plus a virtual API environment for evaluation (Yang et al., 14 Jul 2025).
The high-level execution flow is sequential. A natural-language query is submitted; the knowledge graph provides domain knowledge and memory; the ReAct-expert agent reasons over the query and the graph; the agent selects a suitable tool or API, constructs JSON input parameters, and calls it; the API returns structured data; the agent combines the returned values with domain knowledge; and the LLM produces a final natural-language or JSON answer.
The knowledge graph is described as a multi-dimensional, multi-layered integrated dataset focused on meteorology. It combines domain terminology / unstructured text knowledge, structured meteorological data, semantic relations among entities, and expert knowledge and data-driven knowledge. Its construction uses LLM-based entity recognition, relationship extraction, community summarization, and the Leiden algorithm for multi-level community detection and pruning redundant relations.
The ReAct-expert agent is the reasoning and action controller. The paper’s reasoning chain begins from dialogue cache memory and background knowledge provided by the knowledge graph, asks what tools are needed, selects the tool name, provides JSON parameters, observes the API output, combines observation with professional background knowledge, reflects on correctness, and then returns a user-friendly answer in JSON format. This makes the API-calling process explicitly stepwise rather than a single-shot generation.
The tool layer is defined as a data tool with a unique name, parameters, prompt wording or description, and API metadata. The paper summarizes the tool-use mechanism as:
The tool descriptions are designed to be easy for the LLM agent to recognize, with the stated goal of improving intent recognition and reducing wrong tool usage.
3. Knowledge graph as persistent memory and reasoning substrate
The paper explicitly positions the knowledge graph as the system’s long-term memory. In this role it stores domain facts beyond the immediate dialogue, helps recover relevant meteorological concepts, supports multi-step reasoning over long-range correlations, and keeps the system aligned with domain definitions rather than lexical similarity alone (Yang et al., 14 Jul 2025).
The paper attributes several functional advantages to the graph layer: content retrieval coverage, complex query handling, industry-specific reasoning, deep semantic relationship resolution, heterogeneous data integration, knowledge reasoning and expansion, and adaptation to dynamic domain knowledge. The underlying claim is that explicit relations among meteorological entities support API selection in cases where vector similarity is insufficient.
The rainstorm example illustrates this mechanism. A user asks whether rain in Jinan today counts as a rainstorm event. The system uses domain knowledge about rainstorm thresholds and determines that a precipitation API such as get_total_precipitation must be called. In that example, the graph supplies the definition-level knowledge needed to map an implicit question to a specific measurement API.
The comparison to vector retrieval is direct. The paper argues that knowledge graphs offer explicit entity relationships, deep semantic association mining, complex graph queries, heterogeneous data integration, reasoning and knowledge extension, more compact and interpretable retrieval, and less redundant prompt injection. By contrast, the listed RAG limitations are limited retrieval coverage, weak deep semantic relation discovery, poor complex-query handling, weak heterogeneous data fusion, limited reasoning and extension, and delayed updates. This suggests that KG2data is intended not merely as retrieval augmentation, but as a graph-grounded control mechanism for tool use.
4. Evaluation design and reported performance
The evaluation uses virtual APIs because real meteorological APIs may be restricted due to data security. The paper describes 30 virtual meteorological APIs covering common atmospheric variables such as temperature, humidity, precipitation, wind speed, wind direction, atmospheric pressure, and radiation. Both inputs and outputs use JSON. For instruction generation, the authors use the self-instruct paradigm, generating one explicit question and one implicit or specialized question for each API, yielding 70 instruction-answer pairs in the reported benchmark (Yang et al., 14 Jul 2025).
The evaluation metrics are taken from OpsEval / Liu et al. (2023). The reported metrics are ACAR for API-calls Accuracy Rate, FRIR for Failure Rate of Intent Recognition, FRNR for Failure Rate of Name Recognition, FRPR for Failure Rate of Parameter Recognition, and FRHR for Failure Rate of Hallucination Recognition. The paper also states that the three API-call quality indicators emphasized in the study correspond to FRNR for name recognition failure, FRHR for hallucination failure, and ACAR for call correctness.
The exact reported values are as follows. For KG2data, the paper reports FRIR 0.00%, FRNR 1.43%, FRPR 2.86%, FRHR 0.00%, and ACAR 88.57%. For RAG2data, it reports FRIR 8.57%, FRNR 16%, FRPR 10.00%, FRHR 10.00%, and ACAR 72.14%. For chat2data, it reports FRIR 1.43%, FRNR 7.14%, FRPR 7.14%, FRHR 8.57%, and ACAR 71.43%.
The paper states that KG2data improves ACAR by 16.43% relative to RAG2data and reduces error rates across all failure metrics. Against chat2data, the reported advantages are again concentrated in FRNR, FRHR, and ACAR. The strongest summary figure in the paper is therefore 88.57% ACAR with 0% hallucination failure and 1.43% name recognition failure.
5. Error categories, interpretation, and methodological caveats
KG2data’s ReAct structure is also used to analyze failure sources. The paper identifies errors associated with bad intent understanding, wrong API name selection, hallucinated API choice, and wrong parameters (Yang et al., 14 Jul 2025).
The treatment of hallucination is more nuanced than a simple correct-versus-incorrect API label. The paper notes that hallucination is not exactly the same as an incorrect API call: it can mean invoking a fictitious API, calling a real API that does not fit the task, or failing to call any API appropriately. By contrast, ACAR concerns whether the API call in the observation step is valid or correct.
The benchmark itself contains one explicit textual inconsistency. The methodology repeatedly describes 30 virtual APIs and 70 instruction-answer pairs, while the discussion mentions 35 virtual APIs in one place. The safest reading, stated in the source description, is that the main reported benchmark uses 30 virtual APIs and 70 pairs, while the discussion references a broader practical setup or a draft inconsistency.
The paper also notes that some failures remain due to LLM hallucinations in final generation and prompt design limitations. A plausible implication is that the reported gains primarily establish the value of graph-grounded tool selection within the virtual API setting, while leaving open how performance will transfer to operational meteorological infrastructures with changing endpoints, access controls, and production constraints.
6. Position within broader knowledge-graph-to-data research
KG2data belongs to a broader family of work that treats knowledge graphs as operational intermediates rather than static query targets. This broader research area includes toolkits for graph manipulation, frameworks for provenance-aware analytical integration, and methods for KG quality control.
KGTK, presented as a data science-centric toolkit for large knowledge graph manipulation and analysis, represents knowledge graphs in tables using a tab-separated column-based text format and organizes functionality into importing, graph manipulation or curation, graph querying and analytics, and exporting. Its central motivation is that many real workflows extract subsets from multiple KGs, clean and harmonize them, join them with external data, enrich them with new facts or statistics, and export them to downstream systems. That orientation makes KGTK a concrete example of a knowledge-graph-to-data workflow in which the graph is a pipeline substrate for filtering, joining, cleaning, enriching, analyzing, and exporting large KGs such as Wikidata, DBpedia, and ConceptNet (Ilievski et al., 2020).
MetaboKG provides an analysis-centric knowledge graph framework for untargeted metabolomics. It preserves links between repository exports, analytical files, spectra, features, annotation results, confidence evidence, and contextual metadata, and it is grounded in PROV-O and SIO while aligning with MS, ChEBI, NCBITaxon, ENVO, and NCIT. The framework demonstrates how a knowledge graph can remain queryable, provenance-aware, and reusable across studies and workflows rather than collapsing analytical provenance into flat tables (Féraud et al., 23 May 2026).
ADKGD addresses a different but related layer of the KG2data problem: data quality. It formulates anomaly detection in knowledge graphs with dual-channel learning, combining entity-view and triplet-view representations, cross-layer internal and context aggregation, and KL-based consistency loss. Its stated purpose is not only anomaly detection but also KG data quality improvement / KG cleaning by identifying suspicious triples that should be reviewed, corrected, or removed before the graph is used in downstream systems such as question answering, recommendation, retrieval-augmented LLMs, and KG completion or refinement (Wu et al., 13 Jan 2025).
Taken together, these works indicate that KG2data can denote more than a single meteorological API-calling system. It also names a broader engineering orientation in which knowledge graphs are used to preserve semantics, mediate between heterogeneous sources, maintain provenance, and improve downstream analytical reliability.
7. Significance and open directions
Within the reported study, KG2data’s main empirical conclusion is that knowledge graphs substantially improve API-calling accuracy in specialized domains, especially for implicit, long, and semantically complex queries. The claimed mechanism is not simply better retrieval, but graph-based memory that improves semantic filtering, reduces irrelevant retrieval, and supports structured domain reasoning (Yang et al., 14 Jul 2025).
Its broader significance lies in modularity. The paper argues that the system avoids the high computational cost of fine-tuning, does not require retraining when APIs evolve, adapts more easily to changing domain knowledge, and can be updated incrementally through the graph and tools. In this sense, KG2data presents a maintainable architecture for knowledge-based question answering and data analysis in knowledge-intensive industries.
At the same time, the paper emphasizes unresolved issues. Remaining errors are linked to hallucinations in final generation and prompt design limitations; the benchmark is based on a virtual API environment; and the system is evaluated in a meteorological setting rather than across multiple specialized domains. This suggests that future work will need to clarify how graph-grounded API calling scales to other high-precision domains, how it behaves under more heterogeneous tool ecosystems, and how its graph memory should be updated as both domain concepts and API structures change.