LogPrompt: Interpretable Online Log Analysis

• LogPrompt is a novel interpretable log analysis approach designed for online environments with scarce in-domain training data.
• It employs task-specific prompt strategies—self-prompt, chain-of-thought, and in-context prompting—to standardize outputs for parsing and anomaly detection.
• Empirical results demonstrate up to 380.7% performance gains over simple prompts, coupling prediction with explana­tions for enhanced program comprehension.

Searching arXiv for LogPrompt and closely related prompt-based log analysis/log generation papers. LogPrompt is a novel interpretable log analysis approach for online scenarios that uses LLMs together with task-specific prompt strategies to perform log parsing and log anomaly detection without substantial in-domain training. In the supplied literature, its defining setting is an online environment in which systems are continuously updated, new log events appear over time, and historical in-domain labeled data is scarce or unavailable; its defining contribution is to couple prediction with explanation, so that outputs are not only parsed templates or anomaly labels but also reasons intended to support program comprehension and operational decision-making (Liu et al., 2023).

1. Problem setting and conceptual scope

In the LogPrompt formulation, online log analysis is not merely streaming inference. It is primarily a data-scarce, evolving, unseen-log setting in which test distributions contain many logs not represented in training data, sometimes because a brand-new service goes online with no in-domain logs available at all. The motivating claim is that existing methods often require thousands of in-domain logs and that their performance can decline by up to 62.5% in these online scenarios involving unseen logs from new domains (Liu et al., 2023).

The paper also makes interpretability a first-class objective. Prior automated log analysis methods are described as typically returning only a parsed structure, a class label, or a binary anomaly decision, without interpretation. LogPrompt instead targets program comprehension: helping engineers understand system status, assess whether an anomaly is credible, trace possible causes, and decide on mitigation actions. In this framing, interpretability is not an auxiliary presentation layer but part of the analytic output itself (Liu et al., 2023).

The two benchmarked tasks are log parsing and log anomaly detection. For parsing, the goal is to recover a parsed log template and associated reasons. For anomaly detection, the system predicts normal or abnormal and also produces a justification. In the anomaly setting, LogPrompt predicts a label for each log template and considers a session abnormal if any abnormal tag exists (Liu et al., 2023).

2. Prompt architecture and task-specific strategies

LogPrompt standardizes both the input and the output of the LLM. The input control function enumerates logs in a numbered list:

$$f_x([X])= \text{ There are N logs, the logs begin: (1)}[X]_1\verb|\n| \text{ (2)}[X]_2\verb|\n| \ldots \verb|\n| \text{ (N)}[X]_N$$

The answer control function enforces one formatted answer per input log and requires a reason:

$$f_z([Z],S)= \text{ Organize your answer to be the following format: (1) } x-y\verb|\n| \text{ (2) } x-y\verb|\n| \ldots \text{ (N) } x-y,$$

$$\text{ where } x \text{ is } S \text{ and } y \text{ is the reason. } [Z]$$

This format control is a central part of the method. The paper states that prompts without such control often caused omitted logs, inconsistent formats, and unrelated content (Liu et al., 2023).

On top of this controlled interface, LogPrompt studies three prompt strategies. The first is self-prompt, used in the main parsing experiment. The system asks the LLM to generate a pool of candidate prompts, evaluates those candidates on a small task-specific dataset, and selects the best one. For log parsing, the paper formalizes this as:

$$Self\text{-}prompt: \arg\max_{p \in C} \{ s(p) \}\ f_x([X])\ f_z([Z],S_p)$$

where $C$ is a pool of candidate prompts, $s(p)$ is the evaluation score, and $S_p$ specifies the parsing answer range. In the reported setup, five candidate prompts were generated and evaluated on the first 100 logs of the Android dataset; the best prompt outperformed the worst by 18.4% (Liu et al., 2023).

The second is chain-of-thought (CoT) prompt, used in the main anomaly detection experiment. Its purpose is to regulate reasoning and reduce hallucinated anomaly explanations. The paper gives the full prompt:

$$CoT\ prompt: \text{ Classify the given log entries into normal and abnormal categories. Do it with these steps:}$$

$$\text{(a) Mark it normal when values (such as memory address, floating number and register value) in a log are invalid.}$$

$$\text{(b) Mark it normal when lack of information.}$$

$$f_z([Z],S)= \text{ Organize your answer to be the following format: (1) } x-y\verb|\n| \text{ (2) } x-y\verb|\n| \ldots \text{ (N) } x-y,$$0

$$f_z([Z],S)= \text{ Organize your answer to be the following format: (1) } x-y\verb|\n| \text{ (2) } x-y\verb|\n| \ldots \text{ (N) } x-y,$$1

$$f_z([Z],S)= \text{ Organize your answer to be the following format: (1) } x-y\verb|\n| \text{ (2) } x-y\verb|\n| \ldots \text{ (N) } x-y,$$2

where $$f_z([Z],S)= \text{ Organize your answer to be the following format: (1) } x-y\verb|\n| \text{ (2) } x-y\verb|\n| \ldots \text{ (N) } x-y,$$3 is “a binary choice between abnormal and normal.” The paper distinguishes implicit CoT, obtained simply by forcing the model to produce a reason, from explicit CoT, obtained by adding these intermediate reasoning rules (Liu et al., 2023).

The third is in-context prompt, in which a small number $$f_z([Z],S)= \text{ Organize your answer to be the following format: (1) } x-y\verb|\n| \text{ (2) } x-y\verb|\n| \ldots \text{ (N) } x-y,$$4 of labeled examples are prepended. For anomaly detection, the template is:

$$f_z([Z],S)= \text{ Organize your answer to be the following format: (1) } x-y\verb|\n| \text{ (2) } x-y\verb|\n| \ldots \text{ (N) } x-y,$$5

$$f_z([Z],S)= \text{ Organize your answer to be the following format: (1) } x-y\verb|\n| \text{ (2) } x-y\verb|\n| \ldots \text{ (N) } x-y,$$6

$$f_z([Z],S)= \text{ Organize your answer to be the following format: (1) } x-y\verb|\n| \text{ (2) } x-y\verb|\n| \ldots \text{ (N) } x-y,$$7

The supplied details state that best performance on BGL occurred at 20 labeled logs, after which performance declined as prompts became longer (Liu et al., 2023).

3. Empirical results and interpretability profile

The paper reports experiments on nine publicly available evaluation datasets across two tasks. For log parsing, LogPrompt achieves average F1-score 0.797, with dataset-level F1 values of 0.863 on HDFS, 0.763 on Hadoop, 0.889 on Zookeeper, 0.865 on BGL, 0.759 on HPC, 0.766 on Linux, 0.653 on Proxifier, and 0.819 on Android. It is reported to outperform LogPPT by 32.8% and LogStamp by 37.4% on average while requiring no in-domain training (Liu et al., 2023).

For anomaly detection, the main reported session-level F1 values are 0.384 on BGL and 0.450 on Spirit, with average improvement over trained baselines reported as 55.9% in F1-score. The paper simultaneously stresses that anomaly detection remains difficult in the online scenario and that the absolute F1 remains modest (Liu et al., 2023).

A central empirical claim is that prompt construction matters substantially. The paper states that LogPrompt’s prompt strategies enhance LLM performance by up to 380.7% compared with simple prompts. In the CoT ablation on BGL anomaly detection, session-F1 rises from 0.189 with a simple prompt to 0.307 with implicit CoT and to 0.384 with explicit CoT; template-F1 rises from 0.115 to 0.183 and then to 0.321 (Liu et al., 2023).

Interpretability is evaluated separately through a human study with six practitioners from a top-tier ICT software company, each with over 10 years of experience. Across 200 logs, the reported average scores are 4.23 usefulness and 4.60 readability for log parsing, and 4.30 usefulness and 4.54 readability for anomaly detection; the abstract summarizes the overall interpretability rating as 4.42/5 (Liu et al., 2023).

Aspect	Reported result	Context
Parsing average F1	0.797	8 datasets
Anomaly session-F1	0.384 / 0.450	BGL / Spirit
Gain over simple prompts	up to 380.7%	Prompt strategy effect
Human interpretability rating	4.42/5	Average in abstract

4. Position within prompt-based log analysis research

Within the supplied literature, LogPrompt sits alongside several other LM-based approaches, but its task profile is distinct. LogPPT reformulates log parsing as parameter recognition using prompt-based few-shot learning with a virtual label token PARAM; on 16 public log datasets it reports average GA 0.923, PA 0.916, and ED 1.130 (Le et al., 2023). ProLog—the paper also notes one figure typo calling it “DivLog”—uses in-context learning for automatic log template extraction, with 200 sampled labeled candidates, five retrieved examples per target log, explicit "<START>" and "<END>" delimiters, and average results of 98.1% Parsing Accuracy, 92.1% Precision Template Accuracy, and 92.9% Recall Template Accuracy on 16 LogPAI datasets (Xu et al., 2023).

These systems show that prompt-based log parsing can be organized in different ways. LogPPT is based on prompt tuning and token-level parameter recognition; ProLog is retrieval-augmented in-context learning for template extraction; LogPrompt uses prompt strategies to support both parsing and anomaly detection, and it makes interpretability a stated design goal rather than a by-product (Le et al., 2023, Xu et al., 2023, Liu et al., 2023).

A different but related line is LogPrécis, which studies malicious Unix shell attack logs. It uses a domain-adapted transformer sequence-labeling pipeline to recover attacker tactics from raw shell sessions and reduces about 400,000 unique Unix shell attacks into about 3,000 fingerprints. The paper explicitly states that it is not a prompting paper in the modern instruction-engineering sense, but it remains relevant as LM-based malicious log analysis in a low-label regime (Boffa et al., 2023). This suggests that the broader LogPrompt landscape includes both prompt-engineering systems and non-prompt LM adaptations for operational log understanding.

5. Extension from log analysis to automated logging and synthetic log generation

The supplied literature also shows a movement from prompt-based analysis toward prompt- or LLM-based log authoring. PDLogger is presented as the first end-to-end log generation technique expressly designed for practical, multi-log scenarios. It uses a three-phase pipeline—log position prediction, log generation, and log refinement—with block-type-based structured prompts, backward slicing, and an expanded variable extractor. On 3,113 log statements from two Java projects, the abstract reports improvements over the strongest prior systems of 139.0 percent in log-position precision, 69.2 percent in F1, 82.3 percent in level accuracy, 131.8 percent in variable precision, and 65.7 percent in message quality (BERTScore) (Duan et al., 26 Jul 2025).

LoGFiLM shifts from prompting to supervised fine-tuning. It builds a dataset of 3,208 high-quality examples from 470 log-related issues across 102 Apache open-source projects, fine-tunes Meta-Llama-3-8B-Instruct, and reports that the fine-tuned model consistently outperforms the original Llama-3-8B and commercial GPT baselines. Its task decomposition is explicitly framed as:

$$f_z([Z],S)= \text{ Organize your answer to be the following format: (1) } x-y\verb|\n| \text{ (2) } x-y\verb|\n| \ldots \text{ (N) } x-y,$$8

$$f_z([Z],S)= \text{ Organize your answer to be the following format: (1) } x-y\verb|\n| \text{ (2) } x-y\verb|\n| \ldots \text{ (N) } x-y,$$9

$$\text{ where } x \text{ is } S \text{ and } y \text{ is the reason. } [Z]$$0

where $$\text{ where } x \text{ is } S \text{ and } y \text{ is the reason. } [Z]$$1 is code context, $$\text{ where } x \text{ is } S \text{ and } y \text{ is the reason. } [Z]$$2 is the instruction, $$\text{ where } x \text{ is } S \text{ and } y \text{ is the reason. } [Z]$$3 is insertion position, and $$\text{ where } x \text{ is } S \text{ and } y \text{ is the reason. } [Z]$$4 is verbosity level (Zhang et al., 2024).

A complementary direction is AutoLog, which is not an LLM paper but a static-analysis framework for synthesizing labeled log sequences for anomaly detection directly from source code. It generates sequences without executing the target system and reports, for example, 58x more distinct log events than an existing HDFS dataset from the same system, with anomaly rate controlled by

$$\text{ where } x \text{ is } S \text{ and } y \text{ is the reason. } [Z]$$5

(Huo et al., 2023).

Taken together, these papers suggest that “LogPrompt” is no longer only a question of how to prompt an LLM on existing logs. It is increasingly connected to the full lifecycle of logging: generating better log statements, synthesizing datasets, and structuring training or prompting corpora for downstream analysis (Duan et al., 26 Jul 2025, Zhang et al., 2024, Huo et al., 2023).

6. Limitations, security implications, and unresolved questions

The supplied evidence also delimits LogPrompt’s current scope. In the main LogPrompt paper, anomaly detection remains challenging; the reported best session-F1 is 0.450 on Spirit, and the paper explicitly states that absolute performance remains modest. Runtime, throughput, token costs, and end-to-end latency are not reported. The human evaluation identifies two major causes of low-rated explanations: domain knowledge limitations (45.95%) and semantic insufficiency (40.54%) (Liu et al., 2023).

A further limitation emerges from benchmark coverage. An arXiv entry titled “LogEval: A Comprehensive Benchmark Suite for LLMs In Log Analysis” (Cui et al., 2024) is described in the supplied details as effectively an empty LaTeX shell, with no substantive abstract, benchmark design, prompts, tasks, or results available in the text. Consequently, no concrete benchmark-level comparison between LogPrompt and that claimed suite can be established from the available material (Cui et al., 2024).

Security concerns have also become sharper in later work. LogJack studies indirect prompt injection through cloud logs against LLM debugging agents that can execute remediation commands. Its benchmark contains 42 payloads across 5 cloud log categories; under the Active condition, reported Hijack Rate ranges from 0.0% for Claude Sonnet 4.6 to 86.2% for Llama 3.3 70B, and remote code execution via curl | bash succeeds on 6 of 8 models. The paper’s broader point is that once logs are fed into action-capable LLM agents, the logging plane becomes part of the command surface (Shah, 15 Apr 2026). This is directly relevant to any future operational deployment of LogPrompt-style systems beyond read-only analysis.

The survey literature identifies longer-term open problems that remain pertinent. Logging research still lacks widely accepted golden-quality benchmarks for good logging statements; there is a need to move beyond patterns and templates toward the hidden semantics of log messages; and adaptive logging, cost-aware logging, and stronger structured representations remain open challenges (Gholamian et al., 2021). This suggests that LogPrompt is best understood not as a finished endpoint but as a prominent step in a broader transition from prediction-only log analytics toward semantically richer, prompt-mediated, and increasingly end-to-end log intelligence.