- The paper introduces DF3DV-1K, a large-scale dataset with 1,048 indoor and outdoor scenes that capture diverse distractor conditions for novel view synthesis.
- It benchmarks nine state-of-the-art radiance field methods using metrics like PSNR and LPIPS to systematically evaluate robustness and failure modes.
- The study demonstrates DI2FIX, a diffusion-based image enhancer that improves radiance field rendering by +0.96 dB PSNR and -0.057 LPIPS.
DF3DV-1K: Dataset and Benchmark for Distractor-Free Novel View Synthesis
Motivation and Contributions
Distractor-free radiance field methods aim to synthesize photorealistic novel views from casually captured images containing diverse visual distractors. Progress in this area has been hampered by the limited scale and challenge level of existing benchmarks, which are predominantly scene-specific, lack generalized evaluation, and do not support systematic robustness assessment across distractor scenarios. The DF3DV-1K dataset addresses these limitations by providing a large-scale, diverse, real-world collection of 1,048 indoor and outdoor scenes, each with clean and cluttered image sets spanning 128 distractor types and 161 themes. This dataset supports both qualitative and quantitative benchmarking and is explicitly designed to enable generalized research in distractor-free vision.
The paper's primary contributions are:
- The introduction of DF3DV-1K, a manually curated, large-scale dataset captured over nine months, ensuring diversity in acquisition settings, scene semantics, and distractor types.
- Benchmarking nine state-of-the-art distractor-free radiance field methods and 3D Gaussian Splatting (3DGS) across all scenarios, systematically identifying robustness trends and failure cases.
- Demonstrating the practical applicability of the dataset by fine-tuning a diffusion-based 2D image enhancer, DI2FIX, which improves radiance field rendering by a mean of 0.96 dB PSNR and a 0.057 reduction in LPIPS.
Dataset Design and Acquisition
DF3DV-1K distinguishes itself through its scale, diversity, and real-world capture conditions. Each scene is meticulously designed to include both clean and cluttered states, with distractors manually introduced or naturally present according to indoor/outdoor protocols. Capture was conducted using consumer-grade devices under casual acquisition settings (automatic exposure/focus, anti-shake disabled), yielding 89,924 4K-resolution images distributed across 12 devices. Manual curation and verification involved removal of low-quality images, joint pose estimation with COLMAP for clean and cluttered sets, rigorous geometry checks with instant-ngp, and metadata annotation.
The dataset spans numerous distractor and scene types, including semantically similar distractors, fluid phenomena, appearance-altering shadows, frontal occlusions, and challenging nighttime scenes. A curated subset, DF3DV-41, systematically targets the most problematic scenarios for distractor-free radiance field methods, enabling scenario-wise robustness evaluation.
Benchmark Analysis
DF3DV-1K and DF3DV-41 are benchmarked with nine recent radiance field methods and 3DGS, including AsymGS, RobustSplat, OCSplats, DeGauss, SLS, DeSplat, WildGaussians, T-3DGS and T-3DGS-TMR. The experiments yield several key findings:
- Benchmark Difficulty: DF3DV-1K exhibits greater difficulty and diversity compared to prior benchmarks such as RobustNeRF and On-the-go, with wider LPIPS and SSIM distributions and systematically lower PSNR. Notably, performance differences are more pronounced and directly observable, enabling clear robustness ranking without reliance on subtle background artifacts.
- Method Ranking: AsymGS and RobustSplat emerge as the most robust across both DF3DV-1K and DF3DV-41, followed by OCSplats and DeGauss. The ranking generally aligns with publication timeline, indicating steady methodological advancement. Qualitative evaluation further reveals the comparative resilience of these methods to color-similar, fluid, and semantically similar distractors.
- Failure Modes: Semantically similar distractors and fluid phenomena, as well as nighttime conditions, pose the greatest challenges. Methods relying on semantic features or fixed preprocessing thresholds (e.g., mask learning, threshold-based filtering) often fail to fully remove distractors, generating blending artifacts or unnecessary edits to static scene content.
Generalizable Enhancement: DI2FIX
The paper extends evaluation beyond scene-specific methods by fine-tuning DIFIX—a diffusion-based image enhancement model—using DF3DV-1K images. DI2FIX, the resulting plug-and-play 2D enhancer, is trained on radiance field renderings of cluttered and clean views, filtered with a moderate LPIPS threshold to balance data quality and diversity.
- Performance Gains: DI2FIX achieves consistent improvements for all tested radiance field methods (+0.96 dB PSNR, -0.057 LPIPS), outperforming vanilla DIFIX and DIFIX fine-tuned on smaller datasets.
- Data Scale and Diversity: Larger training sets yield superior performance and robustness, while excessively strict or loose filtering thresholds degrade qualitative results by either excluding challenging distractor scenarios or introducing noisy training samples.
- Generalization: Leave-one-method-out evaluations demonstrate negligible degradation, proving DI2FIX's reliable out-of-distribution generalization.
Implications and Future Directions
The DF3DV-1K dataset supports robust and scalable benchmarking for distractor-free novel view synthesis, enabling the transition from scene-specific optimization to generalized radiance field models. The experimental findings highlight the necessity of dataset scale and semantic diversity for developing resilient methods and data-driven enhancements. Practically, DI2FIX exemplifies the utility of large-scale curation in enabling plug-and-play post-processing tools for radiance field rendering, paving the way for real-time quality improvement across diverse capture environments.
Theoretically, this work emphasizes the importance of challenging benchmarks for evaluating semantic segmentation, mask learning, and feature-based distractor removal strategies in neural rendering. Remaining limitations include dataset capture cost, residual minor distractors in clean images, and DI2FIX's reduced efficacy on severely corrupted inputs or confirmation bias scenarios. The integration of multi-reference frameworks and further scaling of data diversity are probable evolution directions.
Conclusion
DF3DV-1K establishes a comprehensive foundation for distractor-free vision research, supporting large-scale performance evaluation, facilitating method development beyond scene-specific settings, and enabling generalizable image enhancement via diffusion modeling. The dataset's scale and diversity directly contribute to improved robustness, practical rendering quality, and reliable benchmarking, with implications for generalizable radiance field models and adaptive post-processing in future AI-powered visual synthesis.
