ScheMatiQ: Interactive Schema Discovery

• ScheMatiQ is an interactive framework that processes natural-language research questions and document collections to generate observation units, dynamic schemas, and structured evidence tables.
• It employs iterative schema discovery using LLMs to propose, refine, and validate schema fields with explicit grounding from source texts.
• Its architecture integrates a React frontend, FastAPI backend, and Python extraction pipeline, enabling expert-guided, evidence‑backed structured data extraction.

Searching arXiv for "ScheMatiQ" and closely related papers to ground the article in the cited literature. ScheMatiQ denotes an interactive framework that takes a natural-language research question together with a document collection and produces an observation unit, a query-guided schema, and a grounded structured database, with a web interface that allows experts to steer and revise extraction (Levy et al., 10 Apr 2026). In the available literature, the name also appears in an earlier and distinct “MathScheme / ScheMatiQ vision,” where it denotes a mechanized mathematics agenda centered on theory presentations, theory morphisms, and generative library engineering (Carette et al., 2011). The contemporary system bearing the name is therefore best understood as a research workflow for interactive schema discovery and corpus-scale structured evidence construction, while the earlier usage associates “ScheMatiQ” with mathematically structured formal libraries. Both uses share a common emphasis on explicit structure, modularity, and human-steerable formalization.

1. Terminological scope and conceptual orientation

In the 2026 usage, ScheMatiQ is designed for a research workflow in which a domain expert begins with a natural-language research question over a large document collection, but answering that question requires structured evidence rather than retrieval alone (Levy et al., 10 Apr 2026). The system’s core output is a table whose rows are observation-unit instances, whose columns are schema fields, and whose cells are extracted values grounded in source text. The paper’s motivating examples include a legal question about judges appointed by different U.S. presidents, a computational-biology question about nuclear export signals in proteins, and an NLP/science-analysis question about when chain-of-thought is helpful (Levy et al., 10 Apr 2026).

A distinct earlier strand uses “ScheMatiQ” in the phrase “MathScheme / ScheMatiQ vision” to describe a mechanized mathematics system integrating computer theorem proving and computer algebra “at the lowest level,” with a library built from modular theories and theory morphisms (Carette et al., 2011). This suggests a broader family resemblance rather than direct continuity: in both cases, the name is associated with explicit structural representation, compositional assembly, and reduction of manual formalization effort.

A plausible implication is that the name “ScheMatiQ” has been used for systems whose central concern is not merely data storage or retrieval, but the discovery, representation, and reuse of structure. In the 2026 system, that structure is a query-conditioned extraction schema; in the 2011 mathematical-library context, it is a graph of theories and morphisms (Levy et al., 10 Apr 2026, Carette et al., 2011).

2. Problem formulation: from question answering to dataset construction

ScheMatiQ is introduced to address a workflow that standard retrieval-style systems support poorly: many research questions over corpora require exhaustive, structured, and source-grounded evidence rather than a small set of retrieved passages (Levy et al., 10 Apr 2026). The paper frames the traditional alternative as a two-stage manual pipeline consisting of manual schema design followed by exhaustive corpus annotation, and characterizes that workflow as slow, expensive, error-prone, constrained by human capacity, and limited by expert preconceptions (Levy et al., 10 Apr 2026).

The key thesis is that a system can go “from research question to structured data through interactive schema discovery” by jointly inferring three things: the observation unit, the schema relevant to the question, and a grounded database populated from the corpus (Levy et al., 10 Apr 2026). This differs from systems that derive schema from only a question or only documents. An input ablation reported in the paper shows that question-only schemas are generic and high-level, document-only schemas are broad but not necessarily question-directed, and question-plus-documents yields richer question-specific fields, with no three-way overlap (Levy et al., 10 Apr 2026).

This framing situates ScheMatiQ in a space adjacent to, but distinct from, generic literature-table generation, retrieval-oriented “deep research,” and schema-on-read introspection. In related work, SkiQL emphasizes the need for schema extraction, schema visualization, and schema query in heterogeneous data systems (Candel et al., 2022), while Shape Expressions provides a graph-schema language for validating whether an RDF node satisfies a structural constraint and produces a proof-like witness for that validation (Boneva et al., 2015). ScheMatiQ’s contribution lies elsewhere: it does not assume a schema already exists, but instead discovers one interactively from the combination of query and corpus (Levy et al., 10 Apr 2026).

3. End-to-end workflow and system architecture

The pipeline has three main stages: observation unit discovery, schema discovery, and structured data extraction (Levy et al., 10 Apr 2026). In the observation-unit stage, the system asks the LLM to identify what kind of entity the question is asking about, using the question together with a batch of documents. Reported examples include judge for the legal corpus, protein for the nuclear-export-signal corpus, and single model evaluation under an experimental configuration for the chain-of-thought corpus (Levy et al., 10 Apr 2026).

Schema discovery is iterative rather than single-shot. The system processes document batches and repeatedly asks whether the current documents suggest adding or refining schema fields. For each proposed field, the LLM produces a field name, a free-form definition, a rationale for why the field helps answer the question, and optional allowed values or type guidance such as numerical versus free-form (Levy et al., 10 Apr 2026). Extraction then proceeds per document by identifying all observation-unit instances, filling all schema fields in a single pass, and performing targeted follow-up extraction for any still-missing fields (Levy et al., 10 Apr 2026).

The system architecture is three-layered: a React frontend built with TypeScript and Tailwind CSS, a FastAPI backend with REST endpoints and a WebSocket channel for streaming updates, and a standalone Python package implementing observation-unit discovery, schema discovery, and value extraction (Levy et al., 10 Apr 2026). Deployment uses Supabase, Railway, and Docker containers (Levy et al., 10 Apr 2026).

The role of the backbone LLM is distributed across all three stages. In the reported evaluation, ScheMatiQ uses the Gemini-2.5 family, specifically Gemini-2.5-flash for observation-unit discovery and schema discovery and Gemini-2.5-flash-lite for structured extraction; the system also supports Together AI, OpenAI GPT-4, Google Gemini, and locally hosted HuggingFace-based open-weight models (Levy et al., 10 Apr 2026). The paper gives no formal optimization objective or mathematical loss for ScheMatiQ, and presents the method as prompt-driven and systems-oriented rather than as a separately trained schema-induction model (Levy et al., 10 Apr 2026).

4. Grounding, interactivity, and human steering

A central design principle is that every output should be grounded and traceable. The extraction rule is explicit: a value can be extracted only if it is clearly supported by text in the document (Levy et al., 10 Apr 2026). The resulting grounded database stores, for each cell, both an extracted value and supporting evidence from the source documents, enabling verification, traceability, and expert review (Levy et al., 10 Apr 2026).

Interactivity is present at every stage. Users can revise or manually specify the observation-unit type, edit field definitions, add fields, remove fields, merge fields, inspect evidence, correct extracted cells, and add more documents so that schema discovery and extraction continue incrementally (Levy et al., 10 Apr 2026). The interface is therefore closer to an interactive data-curation environment than to a conventional question-answering chatbot.

This design aligns with several neighboring architectures in the literature. “Compound Schema Registry” uses an LLM to infer schema-to-schema mappings into an explicit intermediate representation, the Schema Transformation Language, and then compiles that representation into deterministic dataflow operators; its central lesson is to use LLMs for semantic interpretation while insisting on an explicit, inspectable IR and deterministic execution layer (Fu et al., 2024). “AI-assisted JSON Schema Creation and Mapping” follows a similar hybrid pattern in MetaConfigurator, where natural-language intent is interpreted by an LLM but schemas and mappings are validated, visualized, edited, and executed by deterministic software components (Neubauer et al., 7 Aug 2025). ScheMatiQ’s interface and grounding constraints instantiate the same broader pattern: expert-steerable structure induction with inspectable evidence rather than opaque end-to-end generation (Levy et al., 10 Apr 2026).

A plausible implication is that ScheMatiQ belongs to a class of compound or hybrid systems in which model outputs are operational only when they become editable structural artifacts. In ScheMatiQ, those artifacts are the observation unit, the schema, and the grounded table (Levy et al., 10 Apr 2026).

5. Empirical evaluation and domain case studies

The evaluation is conducted in two expert-collaborative domains where prior manual curation exists: law and computational biology (Levy et al., 10 Apr 2026). In law, the corpus consists of 89 U.S. court decisions on immigration cases, with the question asking whether judges appointed by different presidents differ in voting tendencies on immigration injunction cases. The observation unit is Judge, and the appendix schema includes fields such as Judges On Panel, Appointing Presidents On Panel, Court Decision Legal Basis, Policy Instrument Purpose, Court Level, Court Name, and Judge Decision Outcome (Levy et al., 10 Apr 2026).

In computational biology, the corpus is drawn from NESdb references; the appendix specifies 110 scientific papers, while the main text mentions 96 scientific articles. The observation unit is Protein, and listed fields include NES Presence Status, NES Strength Characterization, NES Determination Evidence, NES Critical Residues, NES Binding Affinity, NES Residue Coordinates, Identified NES Sequence, Source Organism, Export Receptor, and Observed Subcellular Localization (Levy et al., 10 Apr 2026).

The paper reports that ScheMatiQ recovers all but two broad miscellaneous fields from the manual schemas across the two domains, and that fields unique to ScheMatiQ were judged relevant, with mean expert relevance of 4.2/5 in computational biology and 3.6/5 in law (Levy et al., 10 Apr 2026). For observation-unit extraction, ScheMatiQ identifies 87% of proteins in computational biology and 74% of judges in the legal domain, with perfect precision on the tested cases; most misses occur in documents containing many observation units, while recall is near-perfect when documents mention a single entity (Levy et al., 10 Apr 2026).

The paper also reports approximate cost at about 1 USD per 100 documents for the two evaluated use cases using Gemini 2.5 models (Levy et al., 10 Apr 2026). The main practical limitation highlighted by the results is degradation on high-density documents with many candidate rows (Levy et al., 10 Apr 2026).

A broader comparison point comes from “Schemex,” which also addresses schema induction but from collections of examples rather than question-conditioned corpora. Schemex operationalizes schema induction in three stages—clustering, abstraction, and refinement via contrasting examples—and reports 95% alignment on 20 CHI’24 Best Paper abstracts and 85% alignment on 20 TikToks against external or manual categorizations (Wang et al., 20 Feb 2025). This suggests that interactive schema discovery can be organized around different inputs—research questions plus documents in ScheMatiQ, example collections in Schemex—but in both cases the system’s utility depends on making latent structure explicit and revisable.

6. Broader lineage, analogies, and open problems

ScheMatiQ sits at the intersection of schema discovery, structured extraction, and human–AI collaboration. Its closest antecedents in the provided literature do not define the same task, but they illuminate neighboring design choices. SkiQL treats schema management as requiring schema extraction, visualization, and query over a unified metamodel for relational and NoSQL systems (Candel et al., 2022). Shape Expressions formalizes graph-schema validation together with proof-producing witnesses and shows that identifying repairs is inherently difficult, with repair checking coNP-complete (Boneva et al., 2015). Compound Schema Registry and MetaConfigurator both advocate an LLM-plus-deterministic-safeguards architecture for schema understanding and transformation (Fu et al., 2024, Neubauer et al., 7 Aug 2025). Schemex contributes a cluster-first, abstraction-and-refinement workflow for human-AI schema induction from examples (Wang et al., 20 Feb 2025).

The earlier “MathScheme / ScheMatiQ vision” is conceptually remote in application domain but notable as a structural analogue. There, the library is a network of modular theory presentations and theory morphisms, organized so that “each separate concept should occur once and only once in the library source code,” with combination interpreted as pushout and generic constructions such as homomorphisms and substructures generated automatically from presentations (Carette et al., 2011). This suggests a recurring theme across uses of the name: explicit structural representation is treated as infrastructure rather than as a by-product.

The 2026 ScheMatiQ paper is explicit about limitations. Its experiments rely on closed-source LLM APIs, so full reproducibility is difficult; run-to-run variation remains even with fixed parameters; extraction performance drops on high-density documents; and the main protection against hallucination is the strict evidence rule plus human inspection rather than an automatic verifier (Levy et al., 10 Apr 2026). The paper does not describe explicit methods for confidence calibration, deduplication, normalization, conflict resolution across documents, or formal verification models (Levy et al., 10 Apr 2026).

A plausible implication is that future development would require stronger formalization of schemas and extraction invariants, especially if ScheMatiQ were extended toward IR-centered schema transformation systems such as those proposed in generalized schema evolution (Fu et al., 2024), or toward richer machine-executable schema representations of the sort needed by model-driven engineering tools (Neubauer et al., 7 Aug 2025). The authors themselves identify ScheMatiQ as a testbed for long-context processing, efficiency, and effective user interfaces (Levy et al., 10 Apr 2026).

7. Significance

ScheMatiQ’s principal significance is that it reframes LLM-based research assistance away from prose answers and toward dataset construction: research question + corpus → observation unit → query-guided schema → grounded database → expert analysis (Levy et al., 10 Apr 2026). In the evaluated domains, the system is intended not to replace expert judgment but to operationalize a collaborative process in which the model proposes structure and populates it while experts inspect, revise, and validate the result (Levy et al., 10 Apr 2026).

This positioning distinguishes it from schema validators, schema registries, and generic schema-query languages. It does not merely check whether data fit a known schema, as in Shape Expressions (Boneva et al., 2015); nor does it merely query an extracted schema, as in SkiQL (Candel et al., 2022); nor does it only infer schema-to-schema mappings for evolution, as in generalized schema evolution systems (Fu et al., 2024). Instead, ScheMatiQ discovers the schema itself from the research problem and the documents, then uses that schema as the basis for grounded extraction (Levy et al., 10 Apr 2026).

The broader literature suggests that its enduring contribution may be methodological as much as task-specific: structured intermediate artifacts, explicit grounding, iterative expert steering, and deterministic or inspectable downstream use are recurring design patterns across contemporary schema systems (Fu et al., 2024, Neubauer et al., 7 Aug 2025, Wang et al., 20 Feb 2025). In that sense, ScheMatiQ is less a single-shot extractor than a schema workbench for turning document collections into structured evidence suitable for real research workflows (Levy et al., 10 Apr 2026).