SchemaCoder: LLM-Driven Log Schema Extraction

• SchemaCoder is an LLM-driven framework for automatic log schema extraction that casts log parsing as an optimization problem to synthesize executable Python code.
• It employs a hierarchical Question-Tree, residual boosting, and evolutionary search to generate and iteratively refine pattern code for diverse log file formats.
• SchemaCoder achieves an average improvement of 21.3% over state-of-the-art methods on benchmarks like LogHub-2.0, supporting robust analysis of both standard and complex EDA logs.

SchemaCoder is an LLM-driven framework for automatic log schema extraction that takes a raw log file and synthesizes parser code which, when executed, produces the log’s schema: human-readable templates plus variable parameters, and, in more complex settings, hierarchical stage structures for EDA logs. Its central design choice is to cast log parsing as an optimization problem and to replace predefined regular expressions and hand-written parsing flows with a hierarchical “Question-Tree” (Q-Tree), residual boosting, and evolutionary search. In the formulation given by the paper, SchemaCoder is the operator $S(b,m)$, where $b$ is background knowledge and $m$ is the input log file; the output is Python parser code $F$, optimized to minimize a task-specific loss $L(y,F(m))$ against ground-truth schema where available. Experimental validation on LogHub-2.0 reports an average improvement of $21.3\%$ over state-of-the-arts (Wan et al., 25 Aug 2025).

1. Problem domain and conceptual scope

Log schema extraction is defined as the process of deriving human-readable templates from large volumes of log data. In the LogHub-2.0 setting, each log line is decomposed into an invariant template and variable runtime parameters. These schemas support monitoring and anomaly detection, clustering and pattern mining, root cause analysis, and debugging. In EDA logs, the target schema is broader: logs contain commands, progress messages, tables, and verification reports partitioned into design stages such as “Opt #1” or “Placement Optimization Stage,” and the desired output includes stage timelines, hierarchy, and stage-level metrics such as WNS, TNS, density, power, and DRVs (Wan et al., 25 Aug 2025).

The paper situates SchemaCoder against two prior families of methods. The first comprises traditional parsers such as Drain, Spell, IPLoM, SLCT, and LogMine, which rely on hand-crafted regular expressions, token trees, thresholds, or frequent subsequences and tend to operate at line level. The second comprises recent LLM-based or deep-learning-based parsers such as LogPPT, UniParser, Lilac, LLMParser, Lemur, LogParser-LLM, and AdaParser, which often still depend on predefined regular expressions or tokenization anchors and typically perform single-pass template-versus-parameter decisions. SchemaCoder is presented as the first fully automated schema extraction framework applicable to a wide range of log file formats without requiring human customization within the flow (Wan et al., 25 Aug 2025).

A common misconception is that “schema” here refers to JSON Schema, XML Schema, or ontology alignment. In SchemaCoder, “schema” denotes the latent structure of logs: templates, parameter slots, grouped messages, and, for EDA logs, higher-order stage structure. This suggests that SchemaCoder belongs to log analysis and program synthesis rather than to conventional schema-conversion tooling.

2. Formalization and output model

The system is formalized as: $S(b,m)$ where $b$ is background knowledge, such as a description of the system or EDA flow, and $m$ is the input log file. The synthesized parser $F$ is evaluated through: $b$0 where $b$1 is the ground-truth schema and $b$2 is a task-specific loss (Wan et al., 25 Aug 2025).

This formulation is notable because the primary artifact is not merely an extracted schema but executable parser code. When run on $b$3, that code yields grouped messages and templates in the LogHub-2.0 style, or stage-tree schemas and metrics in the EDA setting. The paper therefore treats log parsing as synthesis of a transformation program rather than direct structured prediction. The generated code may contain regexes or other pattern logic, but these are synthesized by the LLM-driven pipeline rather than supplied a priori by humans.

The output model has two levels. At the message level, SchemaCoder extracts templates and associated variable segments. At the structural level, especially for EDA logs, it captures hierarchical stage organization and stage-specific analytic quantities. This dual output distinguishes the framework from parsers that only cluster similar lines or label tokens locally. A plausible implication is that the synthesized parser functions as a reusable schema extractor for subsequent runs on logs of the same family, rather than as a one-off annotation of a single file.

3. End-to-end architecture

The high-level pipeline contains five stages: context-bounded segmentation with cluster-based chunk selection; Q-Tree-Driven Pattern Code Generation (Q-Tree-PR); a Textual-Residual-Guided Evolutionary Optimizer; Residual Q-Tree Boosting; and execution of the final schema code $b$4 on the full log (Wan et al., 25 Aug 2025).

Context-bounded segmentation divides a large log file into chunks that fit within the LLM context window. The paper emphasizes that these are contiguous slices preserving local ordering and context, without hardcoded assumptions about timestamps, log levels, or other canonical prefixes. Each chunk is embedded, clustered, and sampled so that only three to four representative chunks per cluster are passed into the subsequent synthesis stage. The rationale is semantic coverage under bounded inference cost.

Q-Tree-PR then processes each representative chunk. For each chunk, SchemaCoder builds a three-layer Q-Tree of LLM queries and obtains pattern code, which is merged into an initial parser $b$5. This initial parser is subsequently improved by a Textual-Residual-Guided Evolutionary Optimizer, described as OpenEvolve-based in the details, which proposes improved variants $b$6 and scores them by $b$7. Finally, Residual Q-Tree Boosting identifies residual errors, reruns Q-Tree-PR on misparsed chunks or lines, synthesizes an additive parser component $b$8, and integrates it with the current parser through an LLM operator $b$9 to obtain $m$0.

The architecture separates semantic discovery from deterministic execution. The LLM is used to propose code and corrections, but the final schema extraction is carried out by executing parser code over the full log. This suggests a hybrid synthesis-execution design in which stochastic proposal generation is followed by reproducible runtime behavior.

4. Residual Q-Tree Boosting mechanism

The Q-Tree is a hierarchical, fixed-depth tree of LLM queries with three layers along each branch: the Exploration Question Layer, the Segment Selection Layer, and the Pattern Code Generation Layer. Each node in the tree is a prompt; responses determine subsequent prompts and ultimately yield parser fragments (Wan et al., 25 Aug 2025).

In the Exploration Question Layer, the LLM generates open-ended questions from the root node to probe salient patterns and anomalies using the chunk content and the supplied background knowledge $m$1. In the Segment Selection Layer, the LLM identifies the most relevant log segments for those questions, effectively zooming into the lines, tables, or regions that matter for pattern induction. In the Pattern Code Generation Layer, the system asks the model to synthesize pattern code, including regexes or grouping logic, for those selected segments.

Residual boosting is the mechanism that turns this fixed-depth questioning process into an iterative optimizer. After executing the current parser, SchemaCoder computes pseudo-residuals $m$2 from remaining errors, identifies residual chunks or lines that were misparsed, and reruns Q-Tree-PR on those residuals. The resulting additive parser component $m$3 is integrated into the current parser $m$4, producing $m$5. This differs from single-pass LLM prompting in two respects: error localization is explicit, and correction is realized as program augmentation rather than prompt-only reconsideration.

An important clarification follows from the paper’s terminology. “Residual” does not denote numerical gradient residuals in the sense of differentiable learning over parser parameters. The procedure is discrete and code-centric: errors are observed after execution, then used to target further synthesis. This suggests that the method is closer to boosting and evolutionary program repair than to end-to-end neural sequence labeling.

5. Segmentation, sampling, and scalability strategy

Scalability is addressed first through context-bounded segmentation and second through embedding-based sampling. The paper states that log files are divided into chunks suitable for the LLM context window, embeddings are computed for each chunk, the chunk embeddings are clustered, and three to four representative chunks per cluster are sampled for pattern extraction (Wan et al., 25 Aug 2025).

This chunk-level strategy is designed to preserve local semantic context while avoiding exhaustive prompting over the entire log. The details note that the paper does not explicitly specify the clustering method, but it does show a PCA projection of BGL chunk embeddings with cluster assignments and representative samples. The purpose of clustering is to ensure diversity and coverage across semantically distinct log regions, such as Apache access lines, Spark job progress, or HealthApp error blocks.

The sampling policy has methodological significance. By selecting representative chunks per cluster rather than only globally frequent patterns, SchemaCoder is positioned to recover both dominant and minority pattern families. This is especially relevant for rare but operationally important events, multi-line entities, or heterogeneous EDA sections that mix prose and tables. A plausible implication is that the residual boosting stage compensates for residual blind spots left by initial representative sampling, yielding a two-stage approximation: coarse semantic coverage first, targeted recovery later.

The same section also explains why earlier line-centric methods are brittle: they often treat each line independently, depend on token positions or fixed-depth trees, and struggle with mixed-content logs, evolving vendor-specific formats, and multi-line structures. SchemaCoder’s chunk-based representation is therefore both a context-management device and a representational commitment.

6. Evaluation, interpretation, and relation to adjacent schema systems

The paper reports experimental validation on the widely-used LogHub-2.0 benchmark, with an average improvement of $m$6 over state-of-the-arts, and frames SchemaCoder as applicable to a wide range of log file formats without human customization within the flow (Wan et al., 25 Aug 2025). The evaluation target includes both conventional benchmark logs and more complex EDA logs, where output quality includes hierarchical stage structures and metric extraction.

The reported improvement should be interpreted together with the method’s scope. SchemaCoder does not merely relabel tokens; it synthesizes executable parser code, and its target outputs range from message templates to stage-tree schemas. This suggests that the empirical gain partly reflects the ability to model cross-line and higher-order structure that line-level baselines ignore. At the same time, the formal objective still depends on a task-specific loss $m$7, and the paper distinguishes settings where ground-truth schema exists from those where evaluation may rely on an LLM judge.

The name “SchemaCoder” can be misleading because adjacent research uses closely related terminology for different tasks. “Orchestrating Black-Box Schema Converters” studies automated, quality-ranked conversion across heterogeneous schema languages such as JSON Schema, XSD, and SHACL (Neubauer et al., 24 Jun 2026). TSCG addresses deterministic compilation of JSON tool schemas into token-efficient structured text for agentic LLM deployments (Sakizli, 4 May 2026). “AI-assisted JSON Schema Creation and Mapping” integrates LLMs with MetaConfigurator for JSON Schema creation, modification, and mapping (Neubauer et al., 7 Aug 2025). These systems all operate on explicit data-model schemas rather than on log-derived templates and stage structures. By contrast, SchemaCoder is a log schema extraction and parser synthesis framework.

Another misconception is that full automation excludes any domain context. The formal input $m$8 includes background knowledge $m$9, such as a system description or an EDA flow description. This suggests that the framework is fully automated with respect to parser construction inside the flow, but not necessarily context-free. Its automation claim is therefore best understood as the removal of predefined regular expressions, static clustering heuristics, and human-authored parser customization from the extraction pipeline itself.