Zoom In, Reason Out: Efficient Far-field Anomaly Detection in Expressway Surveillance Videos via Focused VLM Reasoning Guided by Bayesian Inference

Abstract: Expressway video anomaly detection is essential for safety management. However, identifying anomalies across diverse scenes remains challenging, particularly for far-field targets exhibiting subtle abnormal vehicle motions. While Vision-LLMs (VLMs) demonstrate strong semantic reasoning capabilities, processing global frames causes attention dilution for these far-field objects and incurs prohibitive computational costs. To address these issues, we propose VIBES, an asynchronous collaborative framework utilizing VLMs guided by Bayesian inference. Specifically, to overcome poor generalization across varying expressway environments, we introduce an online Bayesian inference module. This module continuously evaluates vehicle trajectories to dynamically update the probabilistic boundaries of normal driving behaviors, serving as an asynchronous trigger to precisely localize anomalies in space and time. Instead of processing the continuous video stream, the VLM processes only the localized visual regions indicated by the trigger. This targeted visual input prevents attention dilution and enables accurate semantic reasoning. Extensive evaluations demonstrate that VIBES improves detection accuracy for far-field anomalies and reduces computational overhead, achieving high real-time efficiency and explainability while demonstrating generalization across diverse expressway conditions.

• The paper presents VIBES, a novel framework that integrates kinematics-guided Bayesian inference with focused vision-language reasoning for efficient far-field anomaly detection.
• It achieves superior performance by attaining 100% Recall and high eFPS values across multiple benchmarks, significantly surpassing traditional VLM methods.
• The framework reduces computational load by processing only 2.5–4.6% of frames, enabling real-time, accurate semantic explanation in diverse surveillance settings.

Efficient Far-Field Anomaly Detection in Expressway Surveillance Videos via VIBES

Introduction

This paper introduces VIBES, an asynchronous anomaly detection framework designed for expressway surveillance videos, focused on achieving accurate and efficient detection and semantic explanation of far-field anomalous events (2604.23724). Unlike existing global frame-based Vision-LLM (VLM) paradigms that suffer from attention dilution and high computational cost, VIBES integrates kinematics-guided Bayesian inference with selective VLM reasoning. This modular design addresses key challenges: sparse pixel area for distant anomalies, scene and topology variation across camera deployments, and severe efficiency constraints in real-world deployments.

Limitations of Existing Paradigms

Conventional anomaly detection methodologies—object-level tracking, frame-level reconstruction, and even recent VLM-based systems—struggle in expressway settings due to dynamic environments and the minuscule footprint of far-field anomalies. As highlighted in Figure 1, holistic VLM perception over the global frame results in sub-patch signal smoothing and token-level attention dilution, diminishing the model's ability to localize and reason about rare, distant outlier events.

Figure 1: Comparison of expressway anomaly detection paradigms—global perception versus Bayesian-guided focused reasoning. Small far-field anomalies are suppressed in global input but recovered via localization.

The VIBES Framework

VIBES decomposes the anomaly detection and semantic reasoning task into three tightly coupled modules (see Figure 2):

• Trajectory Tracking & Kinematic State Extraction: Employs a slicing-aided detection approach to improve small object recall, followed by lightweight tracking to extract positional and velocity features from sparse far-field targets.
• Kinematics-Guided Bayesian Inference: Maintains an online posterior over the local traffic kinematic distribution, decoupled into longitudinal and lateral components via the Frenet coordinate transformation, thus generalizing effectively across varying camera and road geometries. Anomalous deviations are assigned Bayesian surprise scores, which act as asynchronous high-confidence triggers.
• Focused Vision-Language Reasoning: Spatiotemporal crops, corresponding to the predicted anomalous region and interval, are packaged as concise visual prompts to the VLM, which is tasked with semantic explanation, including event type and detailed entity descriptors.

Figure 2: VIBES framework architecture: trajectory tracking (left), Bayesian inference for boundary update and trigger generation (center), and localized semantic reasoning via VLMs (right).

Theoretical and Methodological Considerations

The paper formalizes the distributional modeling of normal kinematic states as a temporally adaptive, multivariate Gaussian, but subsequently reduces it to a diagonal covariance matrix by leveraging physical independence between longitudinal acceleration and lateral steering. The update mechanism for the posterior mean is an online MAP estimate, efficiently integrating prior and real-time neighborhood statistics via precision-weighted combination. The anomaly score reduces to a sum of squared (Z-score) deviations in each dimension, providing statistical robustness against sampling and tracking noise.

Trigger-based localization narrows the region of interest for downstream semantic querying, minimizing both temporal and spatial visual redundancy. The efficacy of this approach is rooted in the critical observation that VLMs, even those with advanced token focusing or coarse-to-fine patch scanning, remain fundamentally limited when targets are extremely small or anomalous in kinematics rather than appearance.

Empirical Evaluation

Detection Performance

The framework is evaluated across three datasets (TUMTraffic, TADS, CPED), each emphasizing far-field anomaly scenarios. As reported, VIBES achieves Recall up to 100.00% and AUC-ROC up to 1.00 on the TUMTraffic benchmark, significantly outperforming both standard VLMs and specialized approaches like DeepSCAN or APVR.

Semantic Granularity

VIBES demonstrates high event type and detail accuracy for semantic reasoning, outperforming all baselines. The selective, kinematics-driven crop proposal mechanism exposes the VLM to precisely the evidence required for fine-grained differentiation, such as distinguishing harmless stops from collisions or complex multivehicle interactions.

Efficiency Metrics

Computation is quantified by VLM Query Rate (LQR) and effective FPS (eFPS). VIBES achieves effective throughput of 10.65 and 27.82 FPS on TUMTraffic and CPED, respectively, well above the 10 FPS real-time baseline, while invoking the VLM on only 2.5–4.6% of frames. This sharply contrasts with patch-scanning and tool-agent paradigms that incur orders-of-magnitude higher compute costs.

Figure 3: VIBES provides superior computational efficiency versus patch scanning and memory-augmented baselines, as quantified by query rate (LQR) and eFPS.

Ablation Insights

Component ablations remove Bayesian inference, the Frenet decoupling, or the online posterior updating mechanism. Each ablation precipitates marked drops in Recall and semantic accuracy, emphasizing the indispensability of spatiotemporally targeted inference and adaptive local kinematic modeling.

Figure 4: Ablation study quantifying the impact of VIBES core design choices across three datasets.

Qualitative Comparison

A case analysis directly contrasts the standard Qwen3-VL-8B global frame pipeline versus VIBES under two challenging scenarios. Qwen3-VL-8B fails to flag or explain distant maneuvers, while VIBES accurately localizes and describes the critical event, confirming the mitigation of visual attention dilution and semantic loss.

Figure 5: Focused frame selection based on Bayesian surprise enables VIBES to successfully extract and process fine-grained evidence invisible in global input, facilitating correct semantic reasoning.

Implications and Future Developments

VIBES signals a paradigm shift in integrating physically grounded, adaptive priors with high-capacity but resource-constrained VLMs for large-scale surveillance. The modular separation of fast, low-level physical anomaly detection from complex semantic reasoning is not only computationally optimal but also affords explainability, since Bayesian surprise scores alone offer an interpretable anomaly trajectory.

Practically, the VIBES approach is extensible to other class-imbalanced, real-time monitoring domains where anomalies are both spatially sparse and semantically complex—such as avionics, industrial process control, or autonomous navigation in heterogeneous traffic.

Theoretically, future research may expand upon the joint modeling of entity relations, non-Gaussian traffic state distributions, or incorporate active learning to refine both the Bayesian prior and VLM prompts. Additionally, progress in training vision encoders for extremely low-pixel targets (beyond SAHI) could further improve robustness.

Conclusion

VIBES establishes a comprehensive framework for distant expressway anomaly detection, combining kinematics-guided Bayesian trigger generation with targeted VLM-based semantic reasoning. Empirical results demonstrate that this strategy yields both superior accuracy and computational efficiency, particularly for far-field events previously inaccessible to global VLM processing. The approach robustly generalizes across diverse surveillance scenes, paving the way for deployment in real-time, safety-critical environments.