DISCOVR: VR Dataset & Echocardiography SSL Framework

• DISCOVR is an acronym representing two distinct contributions: a VR dataset with 17,691 annotated frames for social accessibility and an SSL framework for echocardiographic video representation.
• In the VR domain, the dataset supports fine-tuning of YOLOv8n with [email protected] of 67.3%, enabling real-time detection and enhanced accessibility in Meta Quest applications.
• For echocardiography, the dual-branch self-supervised framework employs ViT models and Semantic Cluster Distillation to boost zero-shot classification and segmentation across diverse datasets.

DISCOVR is an acronym used for two unrelated 2025 research artifacts in arXiv literature. In social virtual reality, DISCOVR expands to DIgital Social Context Objects in VR and denotes a dataset of annotated frames from Meta Quest social VR applications, introduced within the VRSight accessibility system for post-hoc object detection in off-the-shelf VR apps (Killough et al., 4 Aug 2025). In echocardiography, DISCOVR expands to Distilled Image Supervision for Cross Modal Video Representation and denotes a self-supervised dual-branch framework for cardiac ultrasound video representation learning, coupling temporal video modeling with an online image encoder through Semantic Cluster Distillation (Mishra et al., 13 Jun 2025). The shared acronym therefore refers not to a single method, but to two domain-specific contributions addressing different technical problems.

1. DISCOVR in social virtual reality

The VR DISCOVR dataset was created because existing real-world object detection datasets, including COCO, perform poorly on purely virtual scenes due to differences in texture, lighting, and rendering style. Its stated purpose is accessibility research in social VR, specifically enabling reliable post-hoc object detection in off-the-shelf VR applications without requiring developer intervention (Killough et al., 4 Aug 2025). The dataset was used to fine-tune a YOLOv8n model for real-time detection in VRSight, an end-to-end system that recognizes VR scenes through a set of AI models and generates tone-based, spatial audio feedback. In the VRSight study, nine participants used the system to explore Rec Room, with reported effectiveness for social tasks such as avatar awareness and available seat identification.

DISCOVR contains 17,691 annotated still frames at 640×640 px, captured from 17 free, “top selling” social VR applications+platform demos on Meta Quest. The source titles are Oculus First Steps, Oculus First Contact, VRChat, Rec Room, ROBLOX VR, Remio, Half + Half, Flipside, Alcove, Engage, Spatial, Meta Horizon Worlds, Zoe, vTime XR, MeetinVR, Multiverse, and Arthur (Killough et al., 4 Aug 2025).

Its 30 object classes are organized into six high-level groups. Avatars include avatar, avatar-nonhuman, chat bubble, and chat box. Informational elements include sign-text, ui-text, sign-graphic, menu, ui-graphic, progress bar, hud, and indicator-mute. Interactables include interactable, button, target, portal, writing utensil, watch, writing surface, and spawner. Safety includes guardian and out of bounds. Seating Areas includes seat-single, table, seat-multiple, and campfire. VR System includes hand, controller, dashboard, and locomotion-target. The classes were developed via open-coding by HCI researchers and refined through model-in-the-loop iterations.

2. Curation, format, and benchmark behavior of the VR dataset

Annotation was performed in the Roboflow online annotation workspace on 640×640 resized images. The initial open-coding stage involved six researchers, each annotating 300 images drawn as 60 images from five seed apps. Their free-form labels were consolidated into 40 classes, and after model testing and redundancy pruning the list was reduced to the final 30 classes (Killough et al., 4 Aug 2025). The main annotation workload was then carried out by four researchers, each annotating ~2,000 images, while a fifth researcher performed cross-review to correct errors and enforce consistency. Periodic team meetings were used to merge seldom-seen or ambiguous classes and to split overgeneral ones. No $\kappa$ statistic was reported, but the paper states that the cross-review step ensured high consistency, with estimated >95% agreement on class boundaries.

For data augmentation, Roboflow generated 3 variants for each training image using random flips, rotations (0–360°), scale (±20%), and shear (±15°). The released dataset is in standard YOLOv8 format under a CC BY 4.0 license, with images/train, images/val, images/test, matching labels directories, and metadata fields including image_id, filename, app_name, width, height, n_instances, average_box_area, and capture_datetime. Labels follow the normalized YOLO schema <class> <x_center> <y_center> <width> <height>.

After augmentation, DISCOVR contains 92,040 total object instances. The split-level counts are as follows (Killough et al., 4 Aug 2025):

Split	Images	Instances	Instances / Image $(\mu \pm \sigma)$
Train (70%)	15,207	79,548	5.23 ± 3.87
Validation (20%)	1,645	8,235	5.01 ± 3.80
Test (10%)	839	4,257	5.07 ± 3.94

Class imbalance is summarized by a Gini index of approximately 0.78. The average bounding-box area over all instances is reported as approximately 12,400 px², with $\sigma_{\text{area}} \approx 8,700\text{ px}^2$. The primary benchmark task is single-class and multi-class object detection using YOLOv8n fine-tuned for 250 epochs. On the test split, the reported metrics are [email protected] = 67.3%, [email protected] = 49.5%, and mAP@[.50–.95] = 46.3%. For comparison, a base YOLOv8n trained on real-world COCO yields mAP@[.50–.95] ≈ 37.3%. Per-class performance ranges from “out of bounds” at 69.1% down to “locomotion-target” at 14.5%. Inference speed on an RTX 4090 is reported as ~22 FPS or 45 ms/frame.

The recommended integration practices are likewise explicit: use image augmentations to close domain gaps; set the detection confidence threshold at $\ge 0.25$; apply non-maximum suppression with IOU = 0.45; fine-tune for at least 200 epochs when extending classes or adding new VR apps; preserve the 70–20–10 split by whole-app exclusion to evaluate generalization to unseen apps; and, when both real and virtual objects are present, combine DISCOVR with real-world detectors through ensembling or multi-dataset training.

3. DISCOVR in echocardiography

The echocardiographic DISCOVR framework addresses self-supervised learning in cardiac ultrasound, where subtle anatomical structures, complex temporal dynamics, low PSNR, and the lack of domain-specific pretrained models limit the effectiveness of standard contrastive, masked-modeling, and clustering-based SSL methods (Mishra et al., 13 Jun 2025). Its stated design principle is to combine temporal video modeling with fine-grained spatial semantics while avoiding external supervision and heavy augmentations.

The architecture is dual-branch. The video branch is a student–teacher ViT in which both student $E_{\theta_s}$ and teacher $E_{\theta_t}$ operate on 3D tube-tokens of patch size $2\times16\times16$, extracted from 64-frame clips with a 90% masking ratio. The teacher sees the full unmasked clip, whereas the student sees $M$ differently masked variants. A learnable CLS token is prepended, and the CLS outputs $z_t$ and $z_s^{(m)}$ serve as global video representations.

The image branch is an online spatial encoder, a 2D ViT denoted $(\mu \pm \sigma)$0, trained in parallel on single frames. Its teacher $(\mu \pm \sigma)$1 sees full frames, while the student $(\mu \pm \sigma)$2 sees $(\mu \pm \sigma)$3 random masks per frame. This branch produces dense spatial feature maps intended to capture fine-grained anatomical semantics. The branches interact through Semantic Cluster Distillation (SCD): token-level video features reconstructed by a small decoder $(\mu \pm \sigma)$4 are aligned with spatial features from the image encoder through online clustering and a cross-entropy alignment loss. The paper characterizes this as transfer of fine-grained anatomical knowledge into the video representation without requiring any external supervision or heavy augmentations.

4. Objective functions, clustering, and optimization

The video self-distillation component uses teacher and student logits derived from linear heads and temperature-scaled softmax: $(\mu \pm \sigma)$5 The corresponding loss averages cross-entropy over the masked student views: $(\mu \pm \sigma)$6 The teacher parameters are updated by EMA,

$(\mu \pm \sigma)$7

The image branch uses an analogous self-distillation objective: $(\mu \pm \sigma)$8 with

$(\mu \pm \sigma)$9

SCD introduces a shared prototype matrix $\sigma_{\text{area}} \approx 8,700\text{ px}^2$0 and temperature $\sigma_{\text{area}} \approx 8,700\text{ px}^2$1 for both video and image token features: $\sigma_{\text{area}} \approx 8,700\text{ px}^2$2 Soft cluster assignments are obtained via the Sinkhorn–Knopp algorithm: $\sigma_{\text{area}} \approx 8,700\text{ px}^2$3 The bidirectional cluster-alignment loss is

$\sigma_{\text{area}} \approx 8,700\text{ px}^2$4

The full training objective is

$\sigma_{\text{area}} \approx 8,700\text{ px}^2$5

The clustering mechanism uses learnable prototypes updated by backpropagation through the two cross-entropy terms in $\sigma_{\text{area}} \approx 8,700\text{ px}^2$6. Sinkhorn–Knopp balancing, typically 3–5 iterations, enforces near-uniform cluster occupancy across batches. The same prototype matrix is used to project both video and image features into a shared semantic-cluster space. Training is performed on normal videos only, with 64-frame clips, stride = 3, ViT-Base backbone settings of patch size 16, depth 12, width 768, AdamW, weight decay, learning rate approximately $\sigma_{\text{area}} \approx 8,700\text{ px}^2$7, cosine decay, batch size typically 256 across GPUs, and 400–800 pretraining epochs. Augmentations are intentionally minimal—random crop and flip only—because aggressive color or geometric perturbations would distort clinically relevant anatomy.

5. Evaluation across echocardiography datasets

DISCOVR is evaluated on six echocardiography datasets spanning fetal, pediatric, and adult populations (Mishra et al., 13 Jun 2025). The reported train/validation/test splits are FetalEcho 1: 8,273 / 414 / 317 videos, FetalEcho 2: 4,154 / 320 / 305, EchoNet-Dynamic: 7,378 / 1,326 / 1,326, EchoPediatric-LVH: 7,837 / 1,592 / 1,592, RVENet: 2,516 / 487 / 573, and CAMUS, which is used as a segmentation benchmark with separate train/validation/test splits for left-ventricular endocardium, epicardium, and left-atrium masks.

Three evaluation protocols are used. Zero-shot classification employs a frozen backbone with weighted kNN on CLS or global pooled features, with $\sigma_{\text{area}} \approx 8,700\text{ px}^2$8 selected on validation. Linear probing freezes the backbone and trains a single linear layer for 30 epochs on labeled validation data. Segmentation transfer freezes the backbone and adds one linear layer plus Conv2D upsampling blocks, trained on CAMUS.

On video anomaly detection with zero-shot kNN, DISCOVR reports Balanced Acc. 63.20%, F1 61.45%, and AUC 67.06% on EchoNet-Dynamic, compared with 57.36%, 57.35%, and 59.00% for C2FPL. On RVENet, it reports 56.23% versus 47.88%, with F1 53.88% versus 47.86%. On EchoPediatric-LVH, it reports 55.63% versus 51.39%, with F1 54.63% versus 51.31%. In linear probing, the framework reports FetalEcho 1 accuracy 65.70% versus SIGMA 63.11%, with Balanced Acc. 65.52% and F1 65.39% versus 62.78%; on EchoNet-Dynamic it reports accuracy 77.68% versus SIGMA 75.57%, with F1 77.63% versus 75.50%. The paper summarizes these as consistent ≥ 2–3% gains across all five classification corpora. In zero-shot classification, DISCOVR reports EchoNet-Dynamic Balanced Acc. 63.20% versus MVD 60.94%, with F1 61.45% versus 57.56%, and FetalEcho 1 Balanced Acc. 62.79% versus MGMAE 61.03%, with F1 61.79% versus 60.64%. For segmentation transfer on CAMUS, the reported Dice score is 0.844, compared with 0.816 for UNet + BYOL, 0.819 for DeepLabV3 + BYOL, 0.747 for VideoMAE, 0.767 for MGMAE, and 0.759 for SIGMA.

An ablation noted in the training summary is that ViT-Small degrades performance, which supports the choice of ViT-Base in the main experiments. The reported strengths are elimination of the need for abnormal labels during pretraining, robustness to ultrasound-specific challenges through anatomy-preserving augmentations, and production of spatio-temporal features that generalize across populations and tasks.

6. Ambiguity, limitations, and significance

A recurrent source of confusion is nomenclature: DISCOVR currently designates two separate artifacts rather than a single research lineage. One is a VR object-detection dataset released with model weights and demo scripts for accessibility-oriented scene understanding (Killough et al., 4 Aug 2025); the other is a self-supervised echocardiographic video-learning framework intended for zero-shot classification, anomaly detection, and segmentation transfer (Mishra et al., 13 Jun 2025). Disambiguation therefore requires the domain, expansion of the acronym, or the associated paper title.

The VR dataset’s stated limitations include the absence of a reported $\sigma_{\text{area}} \approx 8,700\text{ px}^2$9 statistic for inter-annotator agreement, dependence on the selected class inventory, and domain specificity to 17 social VR titles on Meta Quest. The paper’s own guidance suggests that extension to new applications should retain whole-app exclusion in evaluation and should use further fine-tuning, which implies that cross-app generalization is an explicit design concern rather than an assumed property. The benchmark itself shows non-uniform difficulty across categories, with strong performance on some safety elements and weaker performance on classes such as locomotion-target.

The echocardiographic framework’s limitations are likewise explicit. Prototype size $\ge 0.25$0 and temperature $\ge 0.25$1 require careful tuning; cluster quality may fluctuate early in training; and pretraining is compute-intensive because of the dual-branch architecture and high masking ratio. The proposed future extensions include multi-scale prototypes, integration of Doppler flow or Doppler-derived motion fields, and extension of SCD to semi-supervised or multi-modal settings, including video+report.

Taken together, the two uses of DISCOVR exemplify domain-specific infrastructure in contemporary machine learning research: in one case, a curated detection corpus for virtual environments and accessibility workflows; in the other, a self-supervised representation-learning framework specialized for echocardiographic video. The shared acronym masks a substantive distinction between dataset engineering for VR scene semantics and end-to-end SSL design for medical video understanding.