TeleLogs: Log Analytics & RCA Benchmark

• TeleLogs is a dual-paradigm framework uniting structured log ingestion with annotated RCA benchmarks for both telescope operations and 5G network performance.
• It employs a modular three-stage pipeline (scrape, reduce, analyze) that adapts to evolving log schemas and ensures efficient diagnostic analytics.
• The system underpins robust causal inference and expert-level RCA, driving performance optimization in high-frequency, data-intensive operational settings.

TeleLogs refers to two closely related but independently developed paradigms at the intersection of log data analysis, root cause analysis (RCA), and domain-adaptive machine learning. The term denotes (i) a lightweight, configurable log-processing and diagnostic analytics pipeline exemplified by large-scale telescope operations (notably the VLT/VST pipeline) (Savarese et al., 2020), and (ii) the TeleLogs dataset—a comprehensive, annotated simulation suite designed to benchmark reasoning LLMs for RCA in 5G wireless networks (Sana et al., 29 Jul 2025). Both contexts share an emphasis on structured log ingestion, modular data reduction, trend/statistical analysis, and explicit mapping to actionable causes or performance conditions. TeleLogs systems and benchmarks provide critical infrastructure for automating operational analytics, driving causal inference from heterogeneous free-text logs, and evaluating the capacity of LLMs to perform expert-level RCA tasks.

1. TeleLogs Pipeline Architecture and Data Ingestion

TeleLogs-style systems are architected around a three-stage analytic flow: scrape, reduce, analyze. For VST operations, log ingestion occurs in monthly batches retrieved from the ESO Science Archive Facility via standard download tools. A dedicated "harvester" module catalogues source files, capturing metadata (filenames, checksums, timestamps) to ensure traceability and reproducibility. Parsing employs a sequence of MATLAB scripts that execute rapid substring searches and lightweight sscanf/regexp-based tokenization, eschewing computationally expensive regular expressions in favor of performance and maintainability. Each substring search is specified in an external, user-editable configuration file that encodes literal patterns, mapped variable names, data types, and expected units. This design enables swift adaptation to evolving log schema and facilitates collaboration with non-programmer domain experts (Savarese et al., 2020).

In the context of the TeleLogs 5G RCA benchmark, "log ingestion" is simulated but follows a parallel structure: each scenario is instantiated from synthetic but physically realistic drive-test parameter distributions, simulating sequential base station handovers and KPI logs along mobile user trajectories. The dataset incorporates cell-level network configurations, timestamped KPI logs, and symptom labels (e.g., downlink throughput < 600 Mbps) (Sana et al., 29 Jul 2025).

2. Data Reduction, Annotation, and Label Taxonomy

Following initial parsing ("Level 0" database creation), TeleLogs pipelines implement modular, domain-specific aggregation to generate higher-level event records ("Level 1" database). Examples from the telescope pipeline include reconstructing active optics cycles and exposure events by grouping simple log lines according to temporal logic, such as associating open-loop and closed-loop corrections within pre-exposure intervals. Processing leverages vectorized timestamp operations and is typically staged in MATLAB tables, later saved as compact .mat files for efficient storage and downstream querying.

The TeleLogs RCA dataset introduces an explicit annotation schema with eight root cause categories, balancing representation across a diverse set of 5G troubleshooting scenarios:

Label	ID	Description
Speed	c₁	Test vehicle speed > 40 km/h (excessive mobility)
Downtilt	c₂	Serving cell downtilt too large → weak far-end coverage
Coverage	c₃	Serving cell coverage > 1 km → poor RSRP
Interference	c₄	Non-colocated co-frequency neighbors → interference
PCI_conflict	c₅	Neighbor PCI ≡ serving PCI (mod 30) → RS overlap
Handover	c₆	Frequent handovers degrade performance
Threshold	c₇	Misconfigured handover thresholds
PRB	c₈	Insufficient average PRBs to hit target throughput

Annotation is produced by a multi-agent system, combining elimination- and contradiction-based reasoning prompts, followed by aggregation, redundancy pruning, and restructuring into standardized four-stage explanations: (1) data analysis, (2) root-cause analysis, (3) root-cause identification, and (4) summary. Only trajectories whose boxed answer matches ground truth are retained, and the process is designed to enforce label balance and robustness against surface-pattern memorization via randomized test variants (Sana et al., 29 Jul 2025).

3. Storage Schema, Statistical Methods, and Visualization

Parsed log entries and aggregated events are persistently stored using MATLAB tables and .mat bundles organized by time slice (e.g., monthly). For integration or scale-out, data are optionally exported to SQL backends (SQLite or MySQL), with Event and Telemetry tables indexed for subsecond-precision queries. The five-year VST archive occupies a few hundred MB of Level 0 data and ≈50 MB of Level 1 summaries (Savarese et al., 2020).

The TeleLogs analytic suite incorporates a standard set of time series and statistical tools:

• Trend estimation: moving average, $\mu_N(t) = \frac{1}{N} \sum_{i=0}^{N-1} x(t-i)$
• Variance, anomaly detection: $\sigma_N^2(t)$ and $z(t) = \frac{x(t) - \mu_N(t)}{\sigma_N(t)}$ with anomaly flagged if $|z(t)| > 3$
• Linear regression: e.g., $N_\text{day} = a \times \text{day} + b$, solved via least squares
• Seasonal decomposition: via Fourier analysis or MATLAB’s detrend() and periodogram()
• Correlation: Pearson $r = \frac{\text{cov}(x,y)}{\sigma_x \sigma_y}$

Visualization employs MATLAB functions (e.g., plot, bar, histogram, imagesc, heatmap), producing figures such as time series of dome temperature, error-frequency heatmaps, histograms of event frequency, and scatter plots of control residuals. Figures are exported for reporting via export_fig or built-in print utilities.

4. Performance, Scalability, and Operational Best Practices

A TeleLogs pipeline processes ≈10 GB of raw logs (five years) in monthly batches, with a typical throughput of ~5 MB/s, allowing each data slice to be parsed in about one hour. Level 1 reduction adds ∼30 minutes per month. After validation, intermediate tables are pruned to optimize storage. Key optimizations involve avoiding global regex scans, vectorizing time window queries, and supporting incremental re-indexing—so that new pattern definitions require only forward re-ingestion, not full re-processing (Savarese et al., 2020).

Operational best practices honed in the VST deployment include:

• Relying on simple, consistent log formatting to maximize parser speed and robustness.
• Keeping extraction patterns and aggregation logic modular and version-controlled.
• Marking timestamp or token consistency violations explicitly (NaN or "corrupt") to engineer graceful downstream error handling.
• Fully automating monthly pipeline runs and logging process metrics, including parse rates and error frequencies.
• Providing user-facing dashboards for summary exploration and granular log inspection.

A plausible implication is that this structured modularity and the emphasis on low-overhead, transparent configuration interfaces serve as enablers both for extending the system to new domains and for integrating with domain-adapted machine learning models.

5. TeleLogs as a Benchmark for RCA Reasoning with LLMs

The TeleLogs 5G RCA suite constitutes a primary benchmark for evaluating LLM-based root cause analysis. Each instance presents network configuration $U$, time series log $Y_t$, a symptom $s_t$, and ground-truth cause $c$, with the learning task defined via the mapping $\sigma_N^2(t)$0, or equivalently, $\sigma_N^2(t)$1 (Sana et al., 29 Jul 2025).

Training adopts a two-stage regimen:

• Supervised Fine-Tuning (SFT): Minimize

$\sigma_N^2(t)$2

• Reinforcement Learning (GRPO): Maximize

$\sigma_N^2(t)$3

with reward model $\sigma_N^2(t)$4.

Evaluation metrics include pass@1 (single-shot exact match), and maj@4 (majority-vote accuracy across four samples). Empirical results show that generic reasoning LLMs (Qwen2.5-32B-Instruct, DeepSeek-R1, QwQ-32B) achieve pass@1 below 35% on canonical and randomized test splits. By contrast, models domain-adapted using SFT + GRPO (e.g., Qwen2.5-RCA-32B) reach pass@1 of 95.86% (Test) and 93.23% (Randomized), demonstrating robust causal reasoning not reliant on surface pattern memorization.

Model	pass@1 (%) Test	pass@1 (%) Randomized
Qwen2.5-32B-Instruct	18.85	18.05
DeepSeek-R1	29.42	29.0
Qwen2.5-RCA-32B (adapted)	95.86	93.23

These results underline the necessity of domain-adaptation—via supervised CoT and reward-driven RL—for effective, explainable RCA in high-dimensional network settings.

6. Broader Applicability and Future Directions

The generic structure of TeleLogs—modular, configuration-driven pipelines for extracting structured events from heterogeneous free-text logs—enables its adaptation not only to telescopes and mobile networks but also to any high-frequency instrumentation or operational system generating semi-structured textual output. The emphasis on separating raw extraction from event-type aggregation, maintenance of a compact and query-efficient data representation, and compatibility with both human-driven dashboards and ML-driven inference frameworks, affords broad deployment flexibility.

A plausible implication is that as LLMs improve at domain-specialized reasoning, the integrated TeleLogs approach—combining multi-stage parsing, explainable aggregation, and training datasets tailored for stepwise causal inference—will become foundational across automated RCA applications, scientific instrumentation log analytics, and cyber-physical system monitoring.