SynRAG: Unified SIEM Query Generation

• SynRAG is a unified framework that automates converting human-readable YAML threat specifications into platform-specific SIEM queries.
• It integrates retrieval-augmented generation with syntax constraints and vendor documentation to achieve 85% query executability on platforms like IBM QRadar and Google SecOps.
• The framework streamlines threat detection and incident investigation in complex enterprise environments while outperforming traditional LLM-based query methods.

SynRAG is a unified framework that automatically generates threat detection or incident investigation queries for multiple SIEM platforms from a platform-agnostic specification. It was introduced to reduce the operational burden created by heterogeneous SIEM environments in which analysts must work across systems whose attributes, architecture, schemas, and query languages differ substantially. The framework takes a structured YAML threat specification written in natural human language and produces platform-specific executable queries, currently AQL for IBM QRadar and YARA-L 2.0 for Google SecOps. Architecturally, it is a retrieval-augmented generation system with an additional syntax-constraining layer rather than a new neural architecture or a formally specified compiler (Saju et al., 31 Dec 2025).

1. Operational setting and intended function

SynRAG is motivated by the practical conditions of large enterprises, managed security providers, and migration-heavy environments in which multiple SIEM platforms coexist. The paper identifies Palo Alto Networks QRadar, Google SecOps, Splunk, Microsoft Sentinel, and the Elastic Stack as representative examples of this heterogeneity, emphasizing that their query languages, event schemas, operators, clause orderings, and platform-specific execution conventions differ enough that analysts otherwise need either extensive cross-platform training or repeated manual query rewrites (Saju et al., 31 Dec 2025).

Within that setting, SynRAG is positioned as a translation-and-grounding layer between analyst intent and native execution. The analyst writes one high-level YAML specification containing elements such as description, fields to select, source, and logic, and the system emits target-specific executable queries. The two principal use cases named in the paper are threat detection and incident investigation. In both cases, the point is not merely to paraphrase analyst intent, but to convert it into native SIEM syntax while preserving platform-specific executability constraints.

The current implementation supports two concrete targets.

Platform	Query language	Status in the paper
IBM QRadar	AQL	Current implementation
Google SecOps	YARA-L 2.0	Current implementation

Support for Splunk, Elastic Stack, and Microsoft Sentinel is described as future work rather than part of the evaluated system (Saju et al., 31 Dec 2025).

2. Input representation and platform heterogeneity

The framework’s central abstraction is the “platform-agnostic specification.” In practice, this is a structured YAML threat specification written in natural human language. The paper says this YAML includes elements such as description, fields to select, source, and logic. This matters because SynRAG does not define a deeper formal intermediate representation, type system, or schema-mapping engine. Instead, the YAML itself acts as the analyst-facing specification layer, and downstream translation is performed by retrieval-conditioned prompting plus syntax constraints (Saju et al., 31 Dec 2025).

The paper treats platform heterogeneity as more than a surface-level syntax problem. QRadar uses AQL, Google SecOps uses YARA-L 2.0, Splunk uses SPL, and Elastic uses ES|QL. Even when tasks are conceptually similar, such as filtering failed logins, grouping by user, or constraining time windows, the implementation details differ across platforms. Field names and source constructs are likewise platform-specific. SynRAG addresses this by grounding generation in vendor documentation and curated inventories of fields, keywords, functions, and data sources rather than assuming that an LLM’s parametric knowledge is sufficient.

This design also reveals one of the framework’s practical constraints: schema handling is largely prompt-driven. The AQL prompt instructs the model to use only the provided field list and, if a required field is absent, to “find the closest logical/semantic match or add a -- TODO comment.” This suggests that exact schema alignment remains partly heuristic rather than formally solved (Saju et al., 31 Dec 2025).

3. Architecture: knowledge extraction, vector indexing, and syntax service

The paper divides SynRAG into three major stages: Knowledge Extraction, Vector Database Creation, and Syntax Service. The first stage constructs the external knowledge base used at runtime. For QRadar, AQL documentation is collected from offline PDF files using a Python pipeline built with PyMuPDF (fitz) that recursively traverses PDFs, extracts page text, and saves each document as UTF-8 Markdown. For Google SecOps, whose documentation is hosted online, the system uses an asynchronous Playwright-based crawler that discovers valid English URLs, launches a headless Chromium browser, extracts the content within the <article> tag, and stores it as Markdown with sanitized filenames and source metadata (Saju et al., 31 Dec 2025).

The second stage converts these documents into a dense retrievable index. Markdown files are loaded using UnstructuredMarkdownLoader from langchain_community.document_loaders, enriched with metadata such as source path, origin folder, and a UUID, segmented with RecursiveCharacterTextSplitter using a chunk size of 500 characters and overlap of 100 characters, embedded with HuggingFaceEmbeddings using sentence-transformers/all-MiniLM-L6-v2, and stored in a persistent Chroma vector database. At inference time, SynRAG retrieves the top-5 semantically relevant documentation chunks for the target platform and the given threat description (Saju et al., 31 Dec 2025).

The third stage, Syntax Service, is intended to reduce syntax hallucination. For AQL, the language is segmented into four curated component sets: keywords, field names, functions, and database/source. Examples given in the paper include reserved words such as SELECT, FROM, and WHERE; field names such as sourceIP and destinationPort; functions such as UTF8, LOWER, and CONTAINS; and databases such as events and offenses. For YARA-L 2.0, the curated components are meta, events, match, condition, functions such as re.capture(), and outcome. The claim is that valid platform queries can be composed from these authorized categories, and that constraining the model with them materially improves executability (Saju et al., 31 Dec 2025).

4. Prompting, runtime execution, and query generation

At runtime, the analyst-provided YAML specification is embedded and used to retrieve the top-5 relevant documentation chunks from the Chroma vector database for the target platform. In parallel, the syntax service supplies the platform-specific inventory of permitted fields, functions, keywords, and sources. The generation model is GPT-4o inside the SynRAG framework; the baselines are the same or other state-of-the-art models queried directly, without SynRAG’s retrieval and syntax infrastructure (Saju et al., 31 Dec 2025).

The paper reproduces the AQL prompt. It begins, “You are an expert in SIEM correlation and threat detection. Your task is to convert a detection description into a QRadar AQL query.” It then states that the model will receive a detection specification in YAML format, a list of allowed AQL fields, keywords, functions, and database names, and additional context from another API call. The prompt imposes strict instructions, including:

SELECT → FROM → WHERE → LAST

It also instructs the model to use only the provided fields, keywords, functions, and databases, and to select the closest semantic field or emit a -- TODO comment if an exact field is unavailable. This is retrieval-conditioned constrained prompting rather than constrained decoding at the parser or tokenizer level. The paper does not describe chain-of-thought prompting, beam search, self-consistency, verifier models, or iterative repair loops (Saju et al., 31 Dec 2025).

The generation process is repeated independently for each target platform. The same YAML specification can therefore yield, for example, both a QRadar AQL query and a Google SecOps YARA-L query. Correctness is then checked either by execution in the actual SIEM or by comparison with an expert-written standard query. The paper reports that 85% of SynRAG-generated queries executed successfully without syntax issues, while the remaining 15% required only small fixes such as missing quotation marks or minor spelling corrections (Saju et al., 31 Dec 2025).

5. Evaluation methodology and reported results

The experimental setup uses 40 YAML-based threat specifications developed in collaboration with a human expert. The two SIEM platforms used experimentally are IBM QRadar and Google SecOps. Reference queries were written by a subject matter expert with strong knowledge of each SIEM query language. The evaluation uses BLEU and ROUGE-L as approximate proxies for similarity to expert queries, and it complements them with an executability check in actual SIEM environments (Saju et al., 31 Dec 2025).

The reported average results across three runs are as follows.

System	BLEU	ROUGE-L
GPT	0.0467	0.4283
Claude	0.0304	0.3063
LLaMA	0.0702	0.4839
DeepSeek	0.0461	0.3793
Gemma	0.0560	0.3591
SynRAG	0.1287	0.6039

Across individual runs, SynRAG scored 0.1243 / 0.6076, 0.1008 / 0.5736, and 0.1610 / 0.6307 for BLEU / ROUGE-L. The strongest baseline is LLaMA with average BLEU 0.0702 and ROUGE-L 0.4839, but SynRAG remains substantially higher on both metrics. The paper interprets this as evidence that retrieval plus syntax constraints materially improve SIEM query generation beyond raw instruction following (Saju et al., 31 Dec 2025).

The paper also reports that every query produced by the other models required some modifications before execution, including field-name corrections, structural changes, or function fixes. The qualitative examples support this pattern. In one brute-force login scenario, SynRAG’s query is described as functionally identical to the expert’s, differing only stylistically in alias formatting or column naming. In another example, it missed quotation marks around “Bytes Received,” which were needed for a multi-word field name in QRadar, but the generated query otherwise closely matched the expert construction. The reported evaluation, however, does not directly measure logical equivalence, result-set equivalence, downstream SOC utility, or end-to-end analyst productivity (Saju et al., 31 Dec 2025).

6. Position within RAG research, scope, and limitations

SynRAG is best understood as a practical architecture that combines document retrieval, syntax curation, and LLM prompting for cross-SIEM executable query generation, not as a mathematically novel retrieval model. The paper does not present a formal optimization objective, a retrieval loss, a symbolic translation semantics, or a theorem-like correctness guarantee. It does not train a custom retriever, does not jointly optimize retriever and generator, and does not introduce a separate formal schema-mapping engine (Saju et al., 31 Dec 2025).

This places it in a distinct part of the RAG design space from several contemporaneous strands of work. Synthetic-data-driven RAG frameworks such as RAGSynth optimize retriever robustness and generator fidelity through synthetic supervision rather than through documentation retrieval and syntax-constrained prompting (Shen et al., 16 May 2025). Agentic data-synthesis systems such as RAGShaper focus on constructing noisy retrieval environments and robust teacher trajectories for training rather than on executable query translation across vendor platforms (Tao et al., 13 Jan 2026). Generator-aware reranking methods such as Rank4Gen optimize ordered document subsets for downstream response quality and generator-specific evidence preferences, whereas SynRAG relies on standard top-5 documentation retrieval and does not model set-level ranking preferences (Fan et al., 16 Jan 2026). Process-supervised orchestration layers such as SIRAG learn when to continue retrieval or stop and which documents to keep, while SynRAG uses a simpler retrieve-and-generate pipeline with syntax curation and execution checking rather than learned retrieval control (Wang et al., 17 Sep 2025).

The paper’s stated and implied limitations are concrete. Coverage is restricted to QRadar and Google SecOps. The benchmark contains only 40 manually curated YAML rules. The system depends on retrieval quality, documentation quality, and completeness of the syntax inventories. Schema mismatch remains possible when a required field is not present in the target platform list. The evaluation does not include a traditional non-LLM compiler baseline, an ablation isolating retrieval versus syntax service, a user study with analysts, or end-to-end detection outcomes on real security datasets. The most precise characterization is therefore the one supplied by the paper itself: SynRAG is a practical, unified framework that turns one YAML threat specification into native, executable SIEM queries across heterogeneous platforms, with demonstrable gains over plain LLM prompting, but with novelty concentrated in system integration rather than in new retrieval or generation theory (Saju et al., 31 Dec 2025).