DeepParse: Hybrid Log Parsing with LLM-Synthesized Regex Masks

Abstract: Modern distributed systems produce massive, heterogeneous logs essential for reliability, security, and anomaly detection. Converting these free-form messages into structured templates (log parsing) is challenging due to evolving formats and limited labeled data. Machine-learning-based parsers like Drain are fast but accuracy often degrades on complex variables, while LLMs offer better generalization but incur prohibitive inference costs. This paper presents DeepParse, a hybrid framework that automatically mines reusable variable patterns from small log samples using an LLM, then applies them deterministically through the Drain algorithm. By separating the reasoning phase from execution, DeepParse enables accurate, scalable, and cost-efficient log structuring without relying on brittle handcrafted rules or per-line neural inference. Across 16 benchmark datasets, DeepParse achieves higher accuracy in variable extraction (97.6% average Parsing Accuracy) and better consistency than both heuristic and LLM-only baselines. Integrating DeepParse into an anomaly detection pipeline reduced false alarms by over 30% and reduced inference latency by 36% compared to heuristic baselines.

• The paper demonstrates that combining offline LLM-synthesized regex masks with deterministic Drain parsing achieves state-of-the-art accuracy, with PA of 97.6% and GA of 94.1% across 16 datasets.
• The methodology employs entropy-greedy sampling and regex mask injection to pre-apply semantic constraints, ensuring reproducibility, high throughput, and efficient handling of heterogeneous logs.
• The approach significantly reduces computational cost and false alarms in anomaly detection, while maintaining auditability and robust performance against evolving log formats.

DeepParse: Hybrid Log Parsing with LLM-Synthesized Regex Masks

Motivation and Problem Setting

Modern large-scale systems generate heterogeneous and evolving logs vital for observability, security, and anomaly detection. While these logs retain latent structural templates in their code provenance, analysts and downstream analytics typically access only unstructured text. Stateless, deterministic parsers (such as Drain) can cluster repetitive logs efficiently but struggle to accurately extract fine-grained variables, especially across diverse formats or noisy logs. Recent results suggest that LLMs generalize well to such semi-structured text, achieving high template and variable recovery. However, invoking LLMs per log line is computationally infeasible, introduces stochasticity that violates reproducibility and auditability, and often conflicts with privacy or deployment constraints.

The DeepParse framework addresses this spectrum of requirements by restricting the stochastic and semantic strengths of LLMs to a one-time, offline synthesis phase over a small log subsample. The resulting regex bundle is then integrated as deterministic masks into an extended Drain pipeline, which executes at linear time and enables stable, high-throughput parsing with full local execution. This unification delivers high parsing accuracy, computational efficiency, and transparency, while reducing manual configuration effort.

DeepParse Architecture

The DeepParse architecture is a modular pipeline comprising three primary layers:

1. Ingestion Layer: Loads, preprocesses, and samples the log corpus. Sampling employs entropy-based selection to maximize structural coverage from small log subsets, ensuring rare but complex template coverage.
2. Synthesis Layer: Fine-tunes or leverages an LLM (DeepSeek-R1:8B, with LoRA adapters) to generate regex masks. This model is trained to emit Python-formatted regex lists marking dynamic log variables by semantic class.
3. Execution Layer: Integrates the regex masks into a deterministically extended Drain parser ("Drain3"). Here, a mask-first strategy pre-applies regexes to incoming logs, producing stable placeholders for each variable class. The Drain tree then clusters and groups templates solely based on invariant fields.

   Figure 1: The DeepParse workflow: offline LLM-driven mask synthesis from log samples, deterministic online parsing via regex-masked Drain.

By design, the LLM is invoked only once for mask synthesis, amortizing inference cost and eliminating runtime stochasticity or privacy exposure. Identical logs always yield identical templates, supporting reproducibility and auditability in operational settings.

Sampling, LLM Synthesis, and Integration

Entropy-Greedy Sampling

To ensure the mask synthesis phase is maximally informative with minimal labeling, DeepParse introduces entropy-greedy sampling. It normalizes logs to mask out numerics and hex tokens, then selects samples with the highest token-entropy and low Jaccard similarity between samples, guaranteeing both diversity and coverage. This procedure typically saturates parsing accuracy with as few as 50 sampled logs per system.

LLM Configuration

The synthesis LLM (DeepSeek-R1:8B) is fine-tuned with cross-entropy on annotated samples to output valid regex lists targeting all variable fields. Prompts are formulated such that the LLM must generate canonical Python regexes, directly consumable by downstream processes. Syntax validation and self-consistency replay further enforce robustness, with fallback heuristics for LLM failures. Greedy decoding (temperature zero) is used during inference to enforce determinism.

Mask Injection and Deterministic Parsing

At execution, the finalized mask bundle is loaded by Drain3, whose parser tree is modified: before Drain’s token-level parsing, regex masks are applied for each class, variables are replaced by standardized placeholders (e.g., <VAR:IP>), and grouping proceeds on pre-masked logs. This prevents Drift in mask discovery and enforces stability. Unlike direct LLM parsing, identical lines after mask preprocessing are guaranteed the same template assignment (temperature-zero LLMs are not fully deterministic [nondeterminismdeterministicllmsettings]).

Empirical Results

DeepParse is evaluated on 16 standard LogHub datasets using ground-truth template and variable annotations. Its performance is compared to leading parsers (Drain, Logram, LogPPT, LLMParser). Two metrics are reported:

• Parsing Accuracy (PA): Proportion of logs with perfectly recovered templates and variable boundaries.
• Grouping Accuracy (GA): Proportion of logs grouped into the correct template cluster, regardless of variable identification granularity.

Key outcomes:

• Across all datasets, DeepParse achieves average PA of 97.6% and GA of 94.1%, consistently outperforming Drain and LLMParser by 1.8 percentage points in both.
• Significant PA gains occur in high-variable, inconsistent logs (e.g., Thunderbird PA: Drain 0.046 → DeepParse 0.975), directly attributable to LLM-driven semantic masking correcting heuristic under-masking/over-masking.
• Parsing accuracy saturates with 50 sampled logs per system, due to the structural coverage of entropy-greedy sampling.
• Runtime per 100 logs is 0.3 seconds for DeepParse, nearly 100× faster than any per-line LLM inference baseline.

Robustness, Efficiency, and Operational Impact

Ablation studies confirm that the hybrid approach (offline LLM synthesis, deterministic parsing) is crucial: LLM-only inference introduces high variance and cost; heuristics alone cannot match PA. DeepParse is robust to log schema drift—re-trained masks recover from novel log formats with only a 1.5% PA reduction under temporal log splits.

DeepParse is deployed as a parser frontend for LogBERT anomaly detection in a real-world setting. Improved parsing accuracy translates to 30–42% reduction in false alarms and 36% reduction in downstream inference latency by compressing the effective vocabulary (unique templates) and masking high-cardinality identifiers. The result is more efficient and actionable anomaly detectors and SRE workflows.

Practical and Theoretical Implications

DeepParse fundamentally shifts the operational paradigm for log parsing pipelines:

• Practical: It is production-ready, requires no per-line LLM inference, guarantees reproducibility (determinism), adheres to privacy constraints, and is highly data-efficient (few-shot offline sample).
• Theoretical: It provides a compositional approach where LLM-driven reasoning is distilled into declarative artifacts (regexes), which are then consumed by classical, auditable systems.

Since the only product of the neural model is an explicit regex bundle, the pipeline can be adapted for new LLM backbones, other regex-producing synthesis approaches, and new domains (e.g., multilingual, schema-less logs), without architectural overhaul.

Future Directions

Anticipated extensions include active learning for mask refinement, adaptation to logs with internationalization and obfuscation, and tight integration with multi-modal operational signals (e.g., tracing, metrics). The separation of mask synthesis and deterministic parsing supports compliance audits and privacy certification.

Conclusion

DeepParse advances operational log parsing by decoupling semantic reasoning and deterministic template recovery. By confining LLM inference to a brief, one-time synthesis step and leveraging deterministic, composable execution, DeepParse achieves state-of-the-art parsing accuracy, robust error correction, throughput suitable for production analytics, and large downstream gains (reduced false alarms, latency, and resource consumption).

The framework, code, and pre-trained models are openly available, facilitating reproducible benchmarking and future research.

---

References

For full references and details on datasets, LLM configurations, and evaluation protocols, see "DeepParse: Hybrid Log Parsing with LLM-Synthesized Regex Masks" (2604.20553).