A unified framework for detecting point and collective anomalies in operating system logs via collaborative transformers

Abstract: Log anomaly detection is crucial for preserving the security of operating systems. Depending on the source of log data collection, various information is recorded in logs that can be considered log modalities. In light of this intuition, unimodal methods often struggle by ignoring the different modalities of log data. Meanwhile, multimodal methods fail to handle the interactions between these modalities. Applying multimodal sentiment analysis to log anomaly detection, we propose CoLog, a framework that collaboratively encodes logs utilizing various modalities. CoLog utilizes collaborative transformers and multi-head impressed attention to learn interactions among several modalities, ensuring comprehensive anomaly detection. To handle the heterogeneity caused by these interactions, CoLog incorporates a modality adaptation layer, which adapts the representations from different log modalities. This methodology enables CoLog to learn nuanced patterns and dependencies within the data, enhancing its anomaly detection capabilities. Extensive experiments demonstrate CoLog's superiority over existing state-of-the-art methods. Furthermore, in detecting both point and collective anomalies, CoLog achieves a mean precision of 99.63%, a mean recall of 99.59%, and a mean F1 score of 99.61% across seven benchmark datasets for log-based anomaly detection. The comprehensive detection capabilities of CoLog make it highly suitable for cybersecurity, system monitoring, and operational efficiency. CoLog represents a significant advancement in log anomaly detection, providing a sophisticated and effective solution to point and collective anomaly detection through a unified framework and a solution to the complex challenges automatic log data analysis poses. We also provide the implementation of CoLog at https://github.com/NasirzadehMoh/CoLog.

• The paper introduces CoLog, a unified framework leveraging collaborative transformers to detect point and collective anomalies in OS logs.
• It employs multimodal encoding with cross-modal fusion via impressed attention and a modality adaptation layer, achieving near-perfect precision, recall, and F1-scores.
• Empirical results on benchmark datasets demonstrate that CoLog outperforms state-of-the-art models, setting a new standard for unified log anomaly detection.

A Unified Framework for Point and Collective Log Anomaly Detection via Collaborative Transformers

---

Introduction and Problem Context

Log anomaly detection constitutes a core capability in system security and reliability monitoring, particularly for identifying deviations from normative behavior recorded in operating system (OS) logs. The multiplicity of information captured in system logs yields several modalities, primarily semantic (content of each record) and sequence (temporal or contextual background), which present significant challenges for anomaly identification. Traditional rule-based and machine learning approaches generally offer only partial coverage due to their reliance on limited feature engineering, inability to handle log evolution, and lack of systematic cross-modal interaction. Unimodal deep learning models (e.g., CNNs, RNNs, transformers) typically operate on either semantic or sequence features, while existing multimodal approaches often struggle with modality fusion and interaction, resulting in high-dimensionality, data complexity, and noise sensitivity.

The paper introduces CoLog, a collaborative transformer-based framework leveraging multimodal sentiment analysis (MSA) as an analytical abstraction to unify the detection of point and collective anomalies in operating system logs (2512.23380). By re-casting log anomaly detection as a supervised MSA task—interpreting anomalies as negative sentiments—CoLog makes explicit use of both semantic and sequence modalities and formulates a robust modality interaction scheme via collaborative transformers and impressed attention.

Methodological Advances

Multimodal Encoding and Collaboration

CoLog initiates log preprocessing via structured parsing (Drain, nerlogparser) to extract semantic message vectors (using SBERT embedding) and sequence context windows. The core architectural component consists of collaborative transformer (CT) blocks, where each modality is encoded in parallel with multi-head impressed attention (MHIA). MHIA enables cross-modal guidance: query vectors are derived from the modality-under-analysis, while key and context vectors are sourced from the complementary modality. This interaction is critical for uncovering latent dependencies and enhancing representation robustness.

To address the intrinsic heterogeneity between modalities, CoLog implements a Modality Adaptation Layer (MAL) comprising stacked soft attention modules and high-dimensional projections, ensuring mutual purity and cross-modality compatibility in the learned representations.

Balancing and Class Imbalance

A balancing layer further projects fused modality features into a shared latent space and applies adaptive weighting to accommodate differential modality informativeness. Class imbalance—a major issue since anomaly samples tend to be rare— is mitigated via the Tomek link under-sampling algorithm, particularly effective in the high-redundancy regimes of log data.

Supervised Sentiment-based Labeling

CoLog formulates anomaly detection as binary classification with supervised sentiment labels: normal events are mapped to positive sentiment, anomalies to negative. Detection of collective anomalies is implemented as a four-class supervised problem, encapsulating combinations of anomalous foreground (event) and background (context).

Experimental Results and Empirical Claims

CoLog is comprehensively evaluated on seven benchmark OS log datasets and three unseen datasets for generalization and robustness. The framework achieves near-perfect mean precision (99.63%), recall (99.59%), and F1-score (99.61%), routinely hitting 100% accuracy on multiple datasets. Ablation studies reveal that sequence modality, MAL, impressed attention, and balancing layers all contribute crucially to performance; the removal of any diminishes metrics significantly.

Comparative analysis with state-of-the-art supervised (Attentional BiLSTM, CNN, pylogsentiment) and unsupervised (Isolation Forest, PCA, LSTM, Transformer) baselines shows CoLog universally surpasses them in F1-score, reducing the gap to the theoretical maximum by over 98%. Furthermore, CoLog demonstrates strong generalization to Spark, Honey5, and Windows datasets with unseen anomalies (~99% F1), as well as robustness to injected log mutations, achieving >98.5% recall even at high mutation rates.

Efficiency analysis indicates that CoLog, while more computationally intensive than lighter sequence-based models due to semantic encoding, remains practical for batch inference and, with optimizations, could be extended to real-time environments.

Theoretical and Practical Implications

1. Unified anomaly detection: CoLog is the first to provide a unified, end-to-end approach for both point and collective anomaly detection in logs, resolving a long-standing separation in the literature.
2. Advanced modality interaction: The collaborative transformer and impressed attention mechanism represent the state-of-the-art for cross-modal fusion in log analysis, enabling nuanced learning of semantic–contextual dependencies.
3. Robustness to log evolution: By leveraging semantic vectors and collaborative encoding, CoLog exhibits resilience to log mutations, template drift, and noisy data, outperforming both template-driven and single-modality models.
4. Benchmark for anomaly detection: Given its supervised paradigm and near-theoretical accuracy, CoLog effectively defines an upper bound benchmark for future unsupervised and semi-supervised models on similar data distributions.
5. Deployability: The availability of public code and support for multimodal log inputs ensure that CoLog is suitable for integration into cybersecurity, monitoring, and operational diagnostics platforms.

Future Directions in AI for Log Analysis

• Real-time adaptation: The next advancement will integrate CoLog into streaming analytics frameworks, supporting online retraining for adapting to log evolution and concept drift.
• Extension to novel modalities: Adding quantitative log features, metrics, or external signals could further boost anomaly detection coverage.
• Cross-domain generalization: Applications in financial fraud, industrial IoT, and health system logs will test the architecture’s domain independence and motivate development of few-shot or continual learning extensions.
• Noise-resilient fusion: Further research on adaptive attention mechanisms and uncertainty modeling may enhance robustness under adversarial or extreme noise conditions.

Conclusion

CoLog sets a new standard in log anomaly detection by harnessing collaborative transformers with impressed attention, achieving state-of-the-art accuracy in unified point and collective anomaly identification across diverse operating system log datasets (2512.23380). Through meticulous modality interaction, adaptive fusion, and effective class balancing, CoLog demonstrates both theoretical and practical superiority. Its methodological advancements and empirical results will inform future directions in multimodal anomaly detection, benchmark development, and AI-powered log analytics both within and beyond cybersecurity applications.