ViDDAR: AR Task-Detrimental Content Detection

• ViDDAR is a vision-language model-based system that compares raw and augmented images to detect obstructions and information manipulation in AR.
• It uses a hybrid deployment combining edge computing for fast geometric tests with cloud VLM semantic reasoning.
• Empirical results demonstrate high accuracy and low latency, underscoring its practicality for real-world AR safety applications.

Searching arXiv for ViDDAR and closely related naming variants to ground the article in the cited literature. arXiv_search.query({"search_query":"all:ViDDAR OR ti:ViDDAR OR abs:ViDDAR", "max_results": 10, "sort_by": "submittedDate", "sort_order": "descending"}) ViDDAR denotes a vision-language-model-based system for detecting task-detrimental content in augmented reality. In the January 2025 formulation, ViDDAR—expanded as “Vision LLM-based task-Detrimental content Detector for Augmented Reality”—is a comprehensive full-reference system that compares a raw image $I_r$ with an augmented image $I_a$ in order to detect both obstruction attacks and information manipulation attacks. A July 2025 follow-on paper uses the same name primarily for obstruction detection within a broader AR safety stack paired with VIM-Sense for semantic manipulation. Across these formulations, ViDDAR combines VLM-based semantic reasoning with specialist vision modules and a user–edge–cloud deployment model to determine when virtual overlays interfere with safety-relevant or task-critical real-world information (Xiu et al., 22 Jan 2025, Xiu, 31 Jul 2025).

1. Conceptual scope and threat model

ViDDAR addresses “task-detrimental” AR content: virtual elements whose placement or design degrades users’ ability to interpret the real world correctly. The original system formalizes two categories. Obstruction attacks occur when virtual content physically blocks key real-world objects, such as stop signs, exit signs, electrical devices, safety equipment, or other objects that require attention or safety inspection. Information manipulation attacks occur when virtual content changes how users interpret the function or meaning of a real object, even when the object is not fully occluded; the paper’s examples include a realistic virtual plant placed on a smart speaker or other overlays that cause semantic misinterpretation (Xiu et al., 22 Jan 2025).

The later AR safety paper narrows ViDDAR’s operational scope to obstruction attacks, describing it as “Vision LLM-based task-detrimental content detection for augmented reality” and emphasizing semantically critical objects such as traffic signs, warning signs or labels, navigation cues, and information boards. In that formulation, ViDDAR is the obstruction-focused module, while VIM-Sense becomes the companion system for visual information manipulation. This shift does not negate the broader January 2025 formulation; rather, it indicates a decomposition of the original task-detrimental-content agenda into specialized subsystems (Xiu, 31 Jul 2025).

A central premise in both formulations is that harm cannot be reduced to geometry alone. Whether an object is “key” is scene-dependent. A stop sign at an intersection is critical; a stop-sign product on a shelf is not. ViDDAR therefore uses a VLM to infer what matters semantically in the current scene before applying geometric overlap or semantic-reasoning tests (Xiu et al., 22 Jan 2025).

2. Formal definitions and decision criteria

The original ViDDAR paper models obstruction detection as a full-reference decision problem over the pair $(I_r, I_a)$. In the raw image $I_r$, the system infers a set of key objects

$K = \{k_1, k_2, \cdots, k_n\},$

with corresponding binary masks

$M_k = \{m_k^1, m_k^2, \cdots, m_k^n\}.$

If $c$ denotes the virtual content in the augmented image and $m_c$ its mask, then the paper defines the set of object–content combinations

$C = \{(k_1, c), (k_2, c), \cdots, (k_n, c)\}.$

An obstruction attack is then

$$O(I_r, I_a) = \begin{cases} 1 & \text{if } \exists (k_i, c) \in C \longrightarrow |m_k^i \cap m_c| \geq \alpha \cdot |m_k^i|,\ 0 & \text{otherwise.} \end{cases}$$

In the obstruction dataset experiments, the threshold is set to $I_a$0 (Xiu et al., 22 Jan 2025).

A later description restates the same geometric idea through an object-specific obstruction ratio. For each key object $I_a$1, with mask $I_a$2 and AR mask $I_a$3,

$I_a$4

and

$I_a$5

A scene is flagged if any $I_a$6 exceeds the application-dependent threshold $I_a$7 (Xiu, 31 Jul 2025).

For information manipulation, the January 2025 paper introduces three Boolean factors for each real-object–virtual-content combination $I_a$8: alignment precision $I_a$9, style similarity $(I_r, I_a)$0, and information misrepresentation $(I_r, I_a)$1. The final decision is

$(I_r, I_a)$2

This AND logic encodes a strict notion of manipulation: virtual content must be well aligned, visually plausible, and semantically misleading in order to count as an attack (Xiu et al., 22 Jan 2025).

3. Architecture and detection workflow

ViDDAR is organized as a user–edge–cloud system. The AR device captures camera frames, renders virtual content, and communicates with the edge server. The edge server runs fast vision modules and geometric post-processing. The cloud runs a VLM such as GPT-4o or LLaVA, which supplies scene-level semantic reasoning. This partition is intended to balance performance with low latency while keeping the AR device lightweight (Xiu et al., 22 Jan 2025, Xiu, 31 Jul 2025).

For obstruction detection, the pipeline begins with a raw frame $(I_r, I_a)$3 or $(I_r, I_a)$4 and an augmented frame $(I_r, I_a)$5 or $(I_r, I_a)$6. The cloud VLM receives the raw scene and a prompt asking for the single most important object to monitor, with the output constrained to a brief object phrase. The edge server maintains the resulting key-object list, then runs Grounding DINO to localize those objects and Segment Anything Model (SAM) to obtain precise binary masks. The mask for virtual content is derived either from direct access to the AR rendering system or by differencing $(I_r, I_a)$7 and $(I_r, I_a)$8. The edge then computes overlap ratios and applies the threshold test. When an obstruction is detected, the AR system can make the virtual object transparent, move or resize it, or trigger warnings (Xiu et al., 22 Jan 2025, Xiu, 31 Jul 2025).

For information manipulation in the original ViDDAR formulation, the AR device sends both $(I_r, I_a)$9 and $I_r$0 to the cloud VLM with a six-question prompt. The prompt asks the model to identify the virtual content, identify the interacting “key object,” assess alignment precision, assess texture realism and blending, assess whether the interaction induces false functionality or false information, and finally answer “yes” only if the preceding three semantic conditions are all satisfied. The edge server parses the VLM output and maps the final answer to a Boolean manipulation decision (Xiu et al., 22 Jan 2025).

A notable property of the obstruction pipeline is that the final decision is not learned end-to-end by ViDDAR itself. Once key objects and masks are available, classification is rule-based, governed by overlap ratios and thresholds. The constituent models—VLMs, Grounding DINO, and SAM—are pre-trained; ViDDAR’s contribution lies in system integration, task formulation, prompting, and deployment (Xiu, 31 Jul 2025).

4. Datasets, empirical results, and deployments

The original ViDDAR paper introduces two custom datasets. The obstruction dataset contains 306 image pairs $I_r$1, one key object per pair, and 23 fixed key-object classes. Each pair includes a manual key-object class label, a binary key-object mask, and an obstruction label. The information manipulation dataset contains 114 image pairs with 10 distinct virtual–real interaction combinations and manual labels for alignment precision, style similarity, information misrepresentation, and overall manipulation (Xiu et al., 22 Jan 2025).

On the obstruction dataset, ViDDAR with GPT-4o achieved 91.83% key object recognition accuracy, 72.15% segmentation mIoU, and 92.15% obstruction detection accuracy. The “Prior Knowledge” upper bound, in which key object labels are provided manually, achieved 93.14% obstruction accuracy. An end-to-end GPT-4o baseline without the Grounding DINO–SAM decomposition reached 81.04%, while saliency-map and Canny-edge baselines were approximately 52%, indicating that low-level visual cues alone were insufficient for the task. With LLaVA-Next-8b, ViDDAR remained competitive at 89.21% obstruction accuracy, though weaker than GPT-4o (Xiu et al., 22 Jan 2025).

For information manipulation, the original ViDDAR system with GPT-4o achieved 82.46% accuracy, 74.00% precision, and 84.09% recall. LLaVA-Next-8b achieved 42.98% accuracy, 39.80% precision, and 95.35% recall, which the paper interprets as a high-recall but impractical tendency to over-flag scenes as manipulated (Xiu et al., 22 Jan 2025).

Human validation is an important part of the empirical record. In the January 2025 paper, an IRB-approved user study with 20 participants found mean agreement scores above 4 on a 5-point Likert scale for key-object recognition, obstruction, alignment, style, information misrepresentation, and overall manipulation. In the July 2025 obstruction-focused study, the 306-pair dataset was again validated by an IRB-approved user study, and the average agreement level for attack annotations was 4.55/5, supporting the claim that the “key objects” used by ViDDAR align with human intuition about what is critical in a scene (Xiu et al., 22 Jan 2025, Xiu, 31 Jul 2025).

Deployment results emphasize practical feasibility. On an Android ARCore + Unity application running on a Google Pixel 7 Pro, obstruction detection had a mean latency of 533 ms in the one-hop network setting and 960 ms in the six-hop setting; information manipulation detection required 9.62 s and 12.30 s respectively. The later obstruction-only paper reports an average latency of 533 ms per detection and describes a Meta Quest 3 deployment in which obstructing virtual content is automatically made transparent (Xiu et al., 22 Jan 2025, Xiu, 31 Jul 2025).

5. Relation to VIM-Sense, perceptual alignment, and open problems

The July 2025 paper reorganizes the original ViDDAR agenda into two systems. ViDDAR handles obstruction attacks through semantic key-object identification plus geometric overlap analysis. VIM-Sense handles visual information manipulation through OCR, natural-language prompts describing raw-versus-AR differences, and VLM-based multimodal reasoning. On the AR-VIM dataset of 452 raw–AR video pairs, VIM-Sense achieved 88.94% detection accuracy with an average latency of 7.17 s, and thereby functions as the semantic counterpart to ViDDAR’s obstruction detector (Xiu, 31 Jul 2025).

This split clarifies a practical distinction. Obstruction detection is comparatively amenable to deterministic geometry once key objects are identified, which explains ViDDAR’s sub-second response. Information manipulation is semantically richer and more difficult to reduce to overlap tests, which explains both the original ViDDAR paper’s lower accuracy on manipulation and the later decision to treat semantic manipulation as a separate reasoning problem (Xiu et al., 22 Jan 2025, Xiu, 31 Jul 2025).

Several limitations recur across the two papers. ViDDAR depends on VLM correctness in identifying which objects are “key,” so missed semantic priorities can lead directly to missed attacks. The current system lacks explicit AR-specific perceptual modeling of legibility, user gaze, device field of view, or obstruction severity beyond thresholded overlap. The threat model is primarily visual rather than multimodal. The final classification remains modular and rule-based rather than jointly learned from large human-rated severity corpora. The obstruction dataset’s 306 image pairs are useful for evaluation but limited for broader generalization, and cloud-based VLM calls raise latency, privacy, and bandwidth concerns (Xiu, 31 Jul 2025).

The papers frame these limitations as openings for further work. Proposed directions include automated perceptually aligned quality assessment of virtual content along AR-specific dimensions such as spatial alignment, lighting plausibility, and texture realism; detection of multimodal attacks using speech understanding, ASR, or audio scene analysis; hierarchical inference in which routine checks are handled on the edge and cloud reasoning is invoked only when needed; and fine-tuning or adapting VLMs using AR-specific failure cases and human perceptual heuristics (Xiu, 31 Jul 2025).

6. Nomenclature and disambiguation

A persistent source of confusion is the similarity between ViDDAR and several unrelated ViDAR names in the 2025 arXiv literature. The January and July AR-safety papers are about ViDDAR, a vision-language-model-based detector for task-detrimental AR content (Xiu et al., 22 Jan 2025, Xiu, 31 Jul 2025).

By contrast, “ViDAR” in “A Novel ViDAR Device With Visual Inertial Encoder Odometry and Reinforcement Learning-Based Active SLAM Method” refers to Video Detection and Ranging: a spinning-camera, fixed-IMU, encoder-augmented sensor head with visual–inertial–encoder odometry and DRL-based active SLAM. That paper explicitly states that it does not use the spelling “ViDDAR” and consistently uses “ViDAR” for a hardware–algorithm co-design in SLAM (Xin et al., 16 Jun 2025).

A second unrelated use appears in “ViDAR: Video Diffusion-Aware 4D Reconstruction From Monocular Inputs,” where ViDAR expands to Video Diffusion-Aware Reconstruction and denotes a 4D reconstruction framework that uses a personalized diffusion model to synthesize pseudo multi-view supervision for monocular dynamic novel view synthesis (Nazarczuk et al., 23 Jun 2025).

A third related-looking name, FastViDAR, denotes a real-time omnidirectional depth-estimation framework based on Alternative Hierarchical Attention and ERP fusion for four fisheye cameras. It is a multi-camera depth system rather than an AR safety detector (Zhao et al., 28 Sep 2025).

The orthographic resemblance among these terms is therefore accidental rather than conceptual. In current 2025 arXiv usage, ViDDAR refers to AR task-detrimental-content detection, whereas ViDAR and FastViDAR refer to distinct sensing, reconstruction, or depth-estimation systems in robotics and computer vision (Xiu et al., 22 Jan 2025, Xin et al., 16 Jun 2025, Nazarczuk et al., 23 Jun 2025, Zhao et al., 28 Sep 2025).