Semi-Structured Annotation Methods

• Semi-structured annotation is a flexible framework that bridges raw, unstructured data and rigid schemas by allowing hierarchical, nested, or graph-based representations.
• It integrates manual, automatic, and interactive workflows, employing models like TAG and DocumentLabeler to optimize labeling accuracy and efficiency.
• Applications in biomedical, legal, and multimodal domains demonstrate its practical impact in enhancing data extraction, search, and reasoning.

Semi-structured annotation refers to workflows, data representations, and algorithmic strategies that operate in the space between unstructured content (e.g., raw text or images) and fully structured, rigidly schema-conforming labels (such as parse trees or relational tables). In semi-structured annotation, the representation often admits hierarchical, graph-based, or tuple-based structures, supports partial or nested labeling, and generally enables richer expressivity and flexibility relative to flat or fully ordered annotation schemes. Such methodologies predominate in domains where linguistic, biomedical, legal, or multimodal engineering data resist full formalization, yet regular patterns and type systems can be imposed to facilitate information extraction, search, and reasoning.

1. Formal Models and Representational Frameworks

Several formal models underpin semi-structured annotation. The Text Annotation Graphs (TAG) system (Forbes et al., 2017) defines every annotated document as a semantic hypergraph $H = (V, E, \lambda)$, with $V$ as annotation entities (tokens, spans, events, or even edges-as-nodes), $E$ as hyperedges denoting relations (potentially relations-on-relations), and $\lambda$ as a labeling function assigning types, roles, or attributes. This general framework accommodates n-ary, nested, and non-tree relationships and is typically materialized in formats such as JSON, with explicit role-argument mapping.

Other frameworks, such as CatalogBank’s DocumentLabeler (Bank et al., 2024), represent each document as a flat list of annotate “segments” (bounding boxes, text regions, or graphical elements), each tagged from a shallow hierarchy (structural, content, categorical, or media entity types). Graph-style relations—explicit or implicit—connect product attributes, categories, and metadata, albeit with a lower degree of nesting compared to true hypergraphs.

LegalSemi (Kang et al., 2024) and the Dhanyavarga annotation pipeline (Terdalkar et al., 2022) employ labeled property graphs (Neo4j/KG) to encode entities, events, relations, and fine-grained properties, often parameterized by a pre-specified ontology (25–30+ entity and relation types) developed for the domain. These ontologies formalize types and signatures, e.g., isPropertyOf: Property → Substance, within a description logic hierarchy.

2. Annotation Workflows and Tooling

Semi-structured annotation regimes blend manual, automatic, and interactive steps. The TAG pipeline (Forbes et al., 2017) involves three main phases:

• Data ingestion: Supports import from standoff, CoNLL-X, or bioC; token-to-node mapping and relation-to-hyperedge construction yield $H = (V, E, \lambda)$.
• Graph construction and visualization: Interactive SVG layouts permit users to reposition tokens/nodes, dynamically create or retype hyperedges, and visualize complex, nested relations (relations-on-relations).
• Querying and semantic summaries: Users select any node or edge $r$; the tool computes the subgraph $S(r)$ via reachability (arguments/relations), enabling structure-driven search and corpus-wide pattern identification.

Semi-automatic tools for document engineering, such as DocumentLabeler (Bank et al., 2024), instantiate four modular stages:

• Preprocessing: PDF-to-image rendering with minimal OCR correction;
• Import/cleanup: Merge/split/delete bounding boxes with batch scripting;
• Labeling: Keyboard-accelerated manual annotation or invocation of integrated extractors (e.g., PICK, BiLSTM/transformer-based KIE), followed by user-correction of suggestions;
• Export: Schema-agnostic JSON/XML to downstream systems.

In biomedical domains, semi-automated QA-based workflows (Vijayaraghavan et al., 5 Apr 2026) leverage small LLMs for extraction, with entity guidelines and few-shot examples embedded in prompts. Disagreement modeling (model-vs-model-vs-gold) flags ambiguous cases for human review, yielding robust pipelines that exploit both computational and expert resources with minimal human effort.

3. Allocation of Annotation Effort: Human-AI Collaboration and Selective Strategies

Budget-aware semi-annotation must allocate limited high-quality human effort efficiently. The SANT framework (Huang et al., 2024) introduces error-aware triage (EAT) and bi-weighted triage, assigning hard/informative examples to humans and easy ones to models. EAT estimates, for each input $x_t$, the likelihood of model mislabeling and incorporates local uncertainty as a decision boundary. The final triage rule combines EAT with standard active-learning scores in a bi-weighting formula:

$$d^{bi}_t(x_t) = [d^{AL}_t(x_t)]^{\eta(t)} \cdot d^{EAT}_t(x_t)$$

This enables streaming, adaptive allocation; empirical results show SANT outperforms single-signal or indiscriminate approaches.

For content annotation at scale, MCHR (Yuan et al., 22 Mar 2025) formalizes a multi-level classification pipeline employing consensus among heterogeneous LLMs (e.g., GPT-4o, Claude 3.5 Sonnet, GPT-o1), with a staged confidence threshold and targeted human review triggering only when consensus or model confidence is weak. This yields human workload reductions of 32–100% with minimal loss in annotation accuracy.

Partial annotation protocols (Ning et al., 2019) formally demonstrate that, for many structured tasks with concave information gain curves, partly annotating many structures can outperform complete annotation of a few, given fixed budget constraints. The mutual-information framework,

$I(Y;A_k) = \log|C(L^d)| - \mathbb{E}[\log f(A_k)]$

where $V$0 is the size of the full-structure equivalence class consistent with observed partials, provides a principled basis for such decisions.

4. Practical Applications and Domain Case Studies

Semi-structured annotation techniques are widely adopted across domains:

• Biomedical and clinical text: TAG (Forbes et al., 2017) supports event extraction and parsing of complex phenomena (e.g., multi-step activation/inhibition in biological pathways); SLM-driven QA pipelines (Vijayaraghavan et al., 5 Apr 2026) extract structured entities from histopathology notes at high accuracy (SLMs at 84.3% vs. spaCy NER at 74.3%).
• Engineering catalogs: DocumentLabeler (Bank et al., 2024) enables rapid multimodal labeling (titles, categories, tables, images) of product datasets, accelerating workflow by 4–5× over manual-only annotation. Models trained on these annotated corpora achieve near-perfect mean entity scores (mEP/mER/mEA ≈ 0.99).
• Legal reasoning: LegalSemi (Kang et al., 2024) couples an IRAC-oriented schema with an explicit, queryable SKG; precision@5 and recall@5 improve 6–10× for rule retrieval when leveraging the semi-structured graph context.
• Radiology reports: Sentence-to-semi-structured mapping using rule-based, synonym, and neural (SCB) matchers (Katic et al., 2021) yields near-perfect matching (0.99 accuracy), and downstream sequence-to-sequence models (SAG-Seq2Seq) outperform pure end-to-end baselines on BLEU, ROUGE, and human ratings.

In domains where full structure (e.g., parse trees; rigid KGs) is impractical due to ambiguous, noisy, or partially regular data, these practices allow rapid, scalable, and high-fidelity labeling and downstream knowledge base construction.

5. Limitations, Challenges, and Best Practices

Despite clear advantages, semi-structured annotation faces challenges:

• Ambiguity and subjectivity: Disagreement modeling (e.g., cross-SLM disagreement, LM-as-judge for string outputs (Vijayaraghavan et al., 5 Apr 2026)) and curator adjudication are essential to maintain quality and consistency.
• Partialness and error propagation: Algorithms must support constraint-based inference with missing data; negative sampling, mention selection heuristics (as in CERES (Lockard et al., 2018)), or structured self-learning (SSPAN (Ning et al., 2019)) are critical.
• Scalability and interoperability: Annotation schemes must permit schema extension, modular pipeline integration, and standard format support (e.g., DocumentLabeler exports PICK, DocBank, FUNSD, XFUND; TAG reads/writes BRAT, CoNLL-X, bioC (Forbes et al., 2017, Bank et al., 2024)).
• Domain-specific knowledge incorporation: Successful applications (Ayurvedic KG (Terdalkar et al., 2022), LegalSemi (Kang et al., 2024)) rely on painstaking ontology and guideline development, often with domain-expert tool integration (e.g., glossary-based entity linking, canonical synonym optimization, templated query design).

Best practices identified include adopting graph- or hypergraph-based data models for nested/n-ary relations, fusing interactive and machine-driven annotation steps, modularizing toolchains for extensibility, and exploiting structure-aware active or selective annotation algorithms to maximize annotation utility under budget or time constraints.

6. Outlook and Future Directions

Research in semi-structured annotation is increasingly focused on improving the scalability, adaptivity, and semantic fidelity of annotation pipelines. Emerging directions evident in the surveyed works include:

• Plug-and-play annotator modules: Open communities (e.g., CatalogBank (Bank et al., 2024)) encourage development of domain-specific extractors—LM-driven, rule-based, or hybrid—integrated with collaborative labeling servers and online/offline modes.
• Integration with advanced IR and retrieval systems: Templatized queries (Cypher/SPARQL via Sangrahaka (Terdalkar et al., 2022), SKG-based filtering for LLM input (Kang et al., 2024)) bridge the gap between annotation and usable semantic retrieval.
• Error-aware, consensus-driven data allocation: Data triage frameworks combine active learning and model error prediction, fusing human and machine efforts at scale (Huang et al., 2024, Yuan et al., 22 Mar 2025).
• Cross-domain transfer and generalization: Empirical assessments (e.g., out-of-institution transfer in (Katic et al., 2021)) and partial annotation protocols (Ning et al., 2019) suggest that principled, structure-aware annotation can support robust learning even with limited supervision.

Semi-structured annotation thus remains central to advancing data-centric research strategies in NLP, IR, and knowledge-based systems, enabling flexible, high-coverage structuring of complex and fluid real-world data.