AdaptiveLog: Adaptive Logging Techniques

• AdaptiveLog is a suite of adaptive logging systems that optimize storage, parsing, and security across varied applications.
• It employs innovative methods such as hypergraph matching, LLM-driven parsing, and reinforcement learning for efficient log management and anomaly detection.
• Empirical results demonstrate significant improvements in log size reduction, inference speed, and secure logging performance in diverse environments.

AdaptiveLog refers to a diverse set of specialized frameworks, algorithms, and system optimizations unified by the principle of adaptivity in handling logging data. Across literature, the term encapsulates storage-efficient behavior log management for ML mobile apps, dynamic log parsing and analysis with LLMs, secure log recording under adversarial threat models, reinforcement-learned thresholding in anomaly detection, and adaptive filtering in online learning. This article provides a detailed exposition of the central AdaptiveLog paradigms as instantiated in recent research, highlighting design principles, technical mechanisms, and empirical findings in domains spanning mobile systems, distributed databases, anomaly detection, secure logging, and adaptive PAC-Bayes learning.

1. AdaptiveLog for Efficient User Behavior Logging in ML Mobile Apps

The AdaptiveLog system for ML-embedded mobile apps (also called AdaLog) addresses in-device storage redundancy and sparsity in user-behavior logs, central for personalized feature computation in modern mobile services (Gong et al., 15 Oct 2025). The design targets three pathologies of conventional log storage:

1. Redundancy: ML feature extractors for different models often overlap in the raw events and attributes they record, causing multiple storage and inflating the log footprint.
2. Sparsity: Heterogeneous behavioral attributes (hundreds of event types) force traditional logging to use wide tables with many nulls per row.
3. Dynamism: User behavior and model set shift over time, requiring ongoing adjustment or reconstruction of the optimal log structure.

Redundancy Elimination is formalized as a maximum weight matching in a hypergraph, where nodes are features and hyperedges represent candidate groups that share events and attributes. Due to NP-hardness, a hierarchical pairwise matching algorithm is deployed, greedily merging groups via maximum weight (e.g., Blossom algorithm, $O(|F|^3)$), effectively halving groups per round and completing on-device (100 features) in $<$1s.

Virtually Hashed Attribute Design (VHAN) mitigates sparsity by mapping each event type's attribute set to virtual column indices, grouping logs into $\sim$20 dense tables matched to attribute count rather than type, dropping the null fraction near zero and reducing table count drastically ($\sim$250→20).

Incremental Update Mechanism orchestrates minimal I/O log structure evolution in response to new user behaviors or features. It builds a bipartite match between old and new groups, reusing the majority of unchanged data and limiting re-write cost to only $\sim$10% of log volume, with full nightly updates completing in $<$2s and $<$15MB peak memory.

Quantitative evaluation on deployed mobile apps shows log size reductions per user from 19% up to 44% (unified app), typically $30$–$40\%$, without adverse impact on inference latency or feature fetch. The system supports $1.8\times$ more on-device models for the same storage and fits stringent mobile resource budgets (Gong et al., 15 Oct 2025).

2. AdaptiveLog in Machine Log Parsing and Analysis with LLMs

AdaptiveLog frameworks are widely adopted in automated log parsing, particularly for evolving or ambiguous logs. Notable are AdaParser and hybrid LLM+SLM frameworks.

AdaParser: Self-Adaptive LLM-based Parsing

AdaParser couples LLMs with self-generated in-context learning (SG-ICL) and iterative self-correction to parse evolving log streams accurately, particularly robust under limited history and syntax drift (Wu et al., 2024). Its core modules:

• Trie-based Parsing: Fast path for known templates.
• SG-ICL Demonstration Selection: Dynamic candidate set $𝒞$ accumulates (message,template) pairs, from which $k$-NN selects prompts for LLM inference. Demonstration similarity is computed via normalized LCS on tokenized messages.
• Template Self-Correction: Automatic correction of LLM output ensures regex-compatibility and semantic fidelity (especially after key tokens like “Exception”). Corrections are iteratively applied with gradually increasing LLM temperature.

Experimental results on Loghub-2.0 demonstrate state-of-the-art grouping and template F1, with AdaParser delivering FTA=87.9% (with 20% history) and robust zero-shot FTA=78.7%, outperforming advanced baselines and exhibiting graceful accuracy decline under historical-log scarcity or drift (Wu et al., 2024).

LLM+SLM Collaborative AdaptiveLog

A hybrid AdaptiveLog framework combines a fine-tuned SLM for cost-effective, fast inference and a prompting-based LLM invoked only for “hard” examples, as determined by a Bayesian uncertainty estimator (via MC-Dropout and Beta prior over the SLM’s validation error rate). Hard samples are routed to the LLM, which is prompted using error-case retrieval (ECR)—retrieving similar failure cases and corresponding LLM analyses from a database to inform the current prompt. Only $\approx$27% of samples trigger LLM usage, effecting a $73\%$ LLM cost reduction while also increasing F1 by 1–2pp versus LLM-only solutions, validated on anomaly detection, module classification, and semantic log analysis across BGL, Thunderbird, OpenStack, and industrial datasets (Ma et al., 19 Jan 2025).

3. AdaptiveLog for Log-Based Anomaly Detection

Multiple AdaptiveLog instantiations address the challenge of detecting anomalies in log datasets that are heterogeneous and dynamically evolving.

ADALog: Adaptive Transformer-Based Detection

ADALog leverages a DistilBERT backbone, fine-tuned under masked language modeling (MLM) on normal logs (15% random token masking) (Pospieszny et al., 15 May 2025). Inference aggregates negative log-likelihood of masked tokens, yielding a sequence anomaly score $s(l)$. Adaptive thresholding is set at a percentile ($\alpha$) of the validation-set normal anomaly score CDF and readily recalibrated as data drift. ADALog achieves F1 between $0.92$ and $0.95$ across BGL, Thunderbird, and Spirit, matching or slightly exceeding strong baselines, and is robust to drifting log distributions.

Adaptive RL-Driven Filter Threshold Learning

A reinforcement learning AdaptiveLog approach learns per-sequence threshold $k$ for anomaly flagging in sequence-based detectors (DeepLog, LogAnomaly) (Xiong et al., 3 Apr 2025). The decision problem is modeled as an MDP with state features derived from log window embeddings and action set $A$ over possible $k$ values. PPO is used for policy optimization, with reward emphasizing correct anomaly classification. Empirical results show relative F1 improvements of $+16\%$ (DeepLog) and $+29\%$ (LogAnomaly) on HDFS logs; largest gains appear for datasets and sequence types with high inter-sequence variability (e.g., BGL).

4. AdaptiveLog in Secure Logging under Adversarial Threats

AdaptiveLog also refers to a cryptographically secure log protocol that provides forward integrity, truncation, and rewind resistance even under adaptive crash attacks—where the adversary can view all prior log state and force system resets, potentially rolling the log back (Avizheh et al., 2019).

• Architecture: Log entries are appended with per-entry (fast) and epochal (slow) HMAC keys stored in a small, protected KStore. The slow key is updated infrequently (configurable) by a pseudo-random function conditioned on a cryptographic hash, minimizing risk of key loss during crash for legitimate failures.
• Security: Rewind resistance derives from the unpredictability of slow-key updates; truncation or modification is detected unless the adversary can guess the slow-key or hits the small “expendable” in-memory region ($O(cs)$ entries).
• Performance: Desktop throughput is $26$K entries/sec, $>2\times$ faster than SLiC, with negligible risk of undetected attack. Recovery is linear in log size.

AdaptiveLog thus achieves provable security guarantees for logging in high-assurance environments with minimal O(λ)-sized protected memory (Avizheh et al., 2019).

5. AdaptiveLog in Distributed OLTP Database Logging

In distributed in-memory OLTP, AdaptiveLog denotes a dynamic algorithm that determines, per transaction, whether to write a lightweight command log or a heavier ARIES-style data log (Yao et al., 2015). The objective is to deliver the steady-state throughput of command logging and the recovery performance of data logging.

• Mechanism: An online heuristic estimates the anticipated recovery benefit of writing a data log for each transaction, subject to an I/O budget and failure rate. Data logs are selectively placed to break dependency chains and maximize recovery parallelism.
• Analysis: Formulated as an online knapsack problem optimizing recovery savings vs steady-state cost.
• Concurrency: Parallel recovery leverages transaction footprints and dependency graphs to schedule maximal parallel replays.
• Results: Recovery time is reduced by up to $10\times$ relative to command logging, with only minor loss in throughput.

Applicability is highest in highly concurrent workloads with infrequent failures and a mix of distributed transaction types (Yao et al., 2015).

6. AdaptiveLog in Log Level Tuning via Degree-of-Interest Modeling

A further AdaptiveLog usage is as a tool (often called REFELL) for rejuvenating logging statement levels in large codebases, mining developer “degree of interest” (DoI) from Git histories with Mylyn’s exponential decay model (Tang et al., 2021, Tang et al., 2021). The system:

• Maps methods’ edit frequency to normalized DoI intervals, partitioned over the codebase’s available log levels.
• Suggests automated upgrades/downgrades of logging levels for statements in methods whose DoI shifts.
• Implements heuristic rules to avoid semantically unsafe transformations (e.g., never lowering severity in catch blocks, conditional logs, or critical keyword cases).

Empirical evaluation on 18 Java repositories (3M LOC, 4000 log statements) found 99.2% analyzability and an 83% focus improvement on buggy contexts. Most transformations lower verbosity (shift to DEBUG/TRACE) in non-bug contexts, reducing information overload and heightening log utility for debugging without manual developer curation.

7. AdaptiveLog in Logarithmic Smoothing for Adaptive PAC-Bayesian Learning

AdaptiveLog also refers to a family of off-policy evaluation and learning algorithms based on logarithmic smoothing (LS) estimators, extended to the adaptive setting (Haddouche et al., 12 Jun 2025). The method iteratively refines the policy $\pi_k$ after each data batch by minimizing a regularized LS risk estimate. Using online PAC-Bayesian tools and supermartingale arguments, AdaptiveLog algorithms achieve time-uniform generalization bounds and, under margin and coverage conditions, near-optimal convergence:

$$R(\pi_{k+1}) - R(\pi_\star) \leq \frac{C_\alpha}{\sqrt{m}(k+1)^\alpha}$$

AdaptiveLog (adaAdjLS) is empirically superior to naive extensions and SCRM across image classification benchmarks, particularly in high-action or small-batch regimes where adaptivity and variance control are critical (Haddouche et al., 12 Jun 2025).

---

In summary, AdaptiveLog encompasses a spectrum of adaptive, efficient, and principled logging and log-driven analysis schemes across systems, learning, and security settings. Each instantiation demonstrates significant empirical advantages over static or non-adaptive baselines, with rigorous algorithmic, theoretical, or systems engineering foundations.