AgentSemSteCom: LLM-Driven Semantic Table Annotation

• AgentSemSteCom is an agentic LLM-driven framework for semantic table annotation that integrates modular tool invocations to resolve ambiguous, error-prone tables.
• The system employs a closed-loop ReAct paradigm and five specialized tools to handle missing headers, homonymy, and misspellings, achieving significant gains in CTA F1 and computational efficiency.
• Its dynamic workflow adapts to varying table qualities using context-supported disambiguation and redundancy reduction techniques, thus enhancing accuracy in both Column Type and Cell Entity Annotation.

AgentSemSteCom is an agentic LLM-driven framework for Semantic Table Annotation (STA), excelling at both Column Type Annotation (CTA) and Cell Entity Annotation (CEA) in the presence of complex, error-prone, or ambiguous tabular data. By leveraging the Reasoning+Acting (ReAct) agent paradigm orchestrated by an LLM and five dynamically-invoked external tools, AgentSemSteCom achieves state-of-the-art annotation accuracy and substantial computational efficiency improvements, operating robustly under missing headers, homonymy, misspellings, abbreviations, and strict ontology hierarchies (Geng et al., 18 Aug 2025).

1. Problem Context and Challenges

The Semantic Table Annotation (STA) task requires the mapping of table columns and cell values to their respective semantic classes and entity URIs in a reference knowledge base (e.g., DBpedia). In real-world scenarios, tables are often incomplete or “tough,” exhibiting missing or numerically labeled headers, semantic ambiguity, cell homonymy, typographical and orthographic errors, domain abbreviations, and rigid ontological constraints. These conditions severely degrade the performance of classical annotation methods, especially those not capable of dynamic, context-sensitive reasoning or iterative correction. AgentSemSteCom directly targets these challenges by decomposing the problem into modular, tool-invocation steps under a closed-loop ReAct agent regime (Geng et al., 18 Aug 2025).

2. System Architecture and External Tools

AgentSemSteCom consists of a central LLM “agent” and five encapsulated, prompt-driven external tools. The agent alternately reasons about annotation challenges, issues functionally templated calls to specialized tools, and ingests their structured outputs as new observational context. This orchestrated loop continues until all annotation targets (columns/cells) are resolved or a tool-internal confidence criterion halts further action.

The five external tools and their roles are:

Tool	Function	Key Techniques and Prompts
Data Preprocessing	Spelling correction, abbreviation expansion, dedup.	spaCy NER pre-pass, LLM-based correction, row-context prompt
Column Topic Detection	Header inference for missing/void headers	Cell value aggregation, domain-detection, concise label prompt
Knowledge Graph Lookup	Top-K schema/entity candidates from DBpedia	API calls, multi-entity/homonymy recall, superclass extraction
CTA Candidate Scoring	Ranking ontology classes per column	Score aggregation over collected matches; top-K selection
Context-Supported Selection	Entity/class disambiguation based on table context	LLM disambiguation prompt, justification chains, context injection

Each external tool is accessed via a natural-language prompt template and returns structured JSON to the agent for further deliberation.

3. Dynamic Workflow Selection and Process Logic

Workflow selection is conditional on each column’s header quality and cell completeness. The logic is as follows:

• Workflow A (header missing/vague, valid cells): Run Column Topic Detection → Knowledge Graph Lookup → Context-Supported CEA → CTA Candidate Generation → CTA Ranking → Context CTA.
• Workflow B (header present, no valid cells): Skip CEA; Context-Supported CTA only, with other headers as support.
• Workflow C (header present, valid cells): Standard sequence (KGLookup → CEA → CTA).

The agent cycles through reasoning-acting observations by recursively calling tools as needed, terminating when per-task confidence thresholds are met (Geng et al., 18 Aug 2025).

4. Redundancy Reduction and Computational Efficiency

To address annotation redundancy due to string similarity (e.g., minor typos or case variants), AgentSemSteCom employs a Levenshtein distance-based equivalence criterion: two cell strings $a, b$ are equivalent if $\mathrm{lev}(a, b) < 0.2 \times \min\{|a|, |b|\}$. This pruning reduces distinct cell queries from 177,355 to 60,341, yielding approximately 70% reduction in compute time and 60% fewer LLM tokens per annotation round (Geng et al., 18 Aug 2025).

5. Experimental Evaluation and Quantitative Results

Extensive evaluation was carried out on the Tough Tables (180 tables, 16,464 entities; severe header/cell ambiguity) and BiodivTab (50 biodiversity-related tables) datasets. Performance metrics are computed as precision, recall, and $F_1$ for both CTA and CEA:

• Tough Tables:
  • CTA: $P=0.629$, $F_1=0.596$
  • CEA: $P=0.845$, $F_1=0.843$
• BiodivTab:
  • CTA: $P=0.89$, $F_1=0.89$
  • CEA: $P=0.93$, $\mathrm{lev}(a, b) < 0.2 \times \min\{|a|, |b|\}$0

Relative to the best non-LLM system (KGCODE-Tab: CTA $\mathrm{lev}(a, b) < 0.2 \times \min\{|a|, |b|\}$1, CEA $\mathrm{lev}(a, b) < 0.2 \times \min\{|a|, |b|\}$2), AgentSemSteCom achieves a $\mathrm{lev}(a, b) < 0.2 \times \min\{|a|, |b|\}$3 percentage point improvement in CTA $\mathrm{lev}(a, b) < 0.2 \times \min\{|a|, |b|\}$4 and $\mathrm{lev}(a, b) < 0.2 \times \min\{|a|, |b|\}$5 point gain in CEA $\mathrm{lev}(a, b) < 0.2 \times \min\{|a|, |b|\}$6 on Tough Tables. The string-equivalence redundancy technique corroborates a 70% reduction in annotation time and 60% LLM token savings (Geng et al., 18 Aug 2025).

6. Case Studies and Error Handling

• Handling Missing Headers: When a column is labeled “col0” with cells such as ["Ronaldo", "Portugal", "AI-Nassr FC"], TopicDetection infers “Athlete” as the header; subsequent entity disambiguation resolves “Ronaldo” specifically as “Cristiano_Ronaldo.”
• Disambiguating Homonyms: For repetitious names (e.g., multiple “Robert Baker” entries), context-supported disambiguation using adjunct columns (e.g., work titles) allows the agent to resolve each to distinct DBpedia entities (actor vs. footballer).

These strategies leverage external context and agentic iterative reasoning to overcome limitations inherent in single-pass or classifier-based STA approaches (Geng et al., 18 Aug 2025).

7. Significance, Limitations, and Prospects

AgentSemSteCom establishes a new state-of-the-art in complex STA, combining high annotation performance and efficiency. Its modular, agentic design enables dynamic tool orchestration, context-aware disambiguation, and efficient resource use. However, the method’s reliance on LLM inference and external API latency can present bottlenecks, and steady improvements in prompt and workflow design for richer ontologies remain necessary. Broader generalization to more extreme table irregularity and tighter integration with knowledge graph expansion are recognized as plausible next steps (Geng et al., 18 Aug 2025).