Poison-splat: Computation Cost Attack on 3D Gaussian Splatting (2410.08190v2)

Abstract: 3D Gaussian splatting (3DGS), known for its groundbreaking performance and efficiency, has become a dominant 3D representation and brought progress to many 3D vision tasks. However, in this work, we reveal a significant security vulnerability that has been largely overlooked in 3DGS: the computation cost of training 3DGS could be maliciously tampered by poisoning the input data. By developing an attack named Poison-splat, we reveal a novel attack surface where the adversary can poison the input images to drastically increase the computation memory and time needed for 3DGS training, pushing the algorithm towards its worst computation complexity. In extreme cases, the attack can even consume all allocable memory, leading to a Denial-of-Service (DoS) that disrupts servers, resulting in practical damages to real-world 3DGS service vendors. Such a computation cost attack is achieved by addressing a bi-level optimization problem through three tailored strategies: attack objective approximation, proxy model rendering, and optional constrained optimization. These strategies not only ensure the effectiveness of our attack but also make it difficult to defend with simple defensive measures. We hope the revelation of this novel attack surface can spark attention to this crucial yet overlooked vulnerability of 3DGS systems. Our code is available at https://github.com/jiahaolu97/poison-splat .

• The paper introduces the Poison-splat attack, revealing how adaptive complexity in 3D Gaussian Splatting can be exploited to spike computation costs.
• It employs a bi-level optimization framework with surrogate metrics to induce extreme GPU memory usage, reaching up to 80GB under unconstrained conditions.
• Experimental results expose critical security flaws in 3DGS, underscoring the need for advanced defenses against computation cost attacks.

Analysis of "Poison-splat: Computation Cost Attack on 3D Gaussian Splatting"

The paper "Poison-splat: Computation Cost Attack on 3D Gaussian Splatting" delineates a significant vulnerability in 3D Gaussian Splatting (3DGS), introducing an attack strategy that can exponentially increase computation costs. This paper provides a comprehensive examination of a previously overlooked aspect of 3DGS security, thereby emphasizing potential weaknesses in adaptive model designs.

Overview

3D Gaussian Splatting serves as a dominant 3D representation, renowned for its efficiency and performance, particularly in rendering and reconstructive tasks from multi-view 2D images. Unlike neural network-based systems, 3DGS adapts its model complexity, which although beneficial, is highlighted as a vulnerability in this work. The research introduces the "Poison-splat" attack, revealing how maliciously crafted input data can drastically escalate the computational demands involved in 3DGS training.

Methodology

The proposed attack leverages a bi-level optimization framework. Here, the inner layer pertains to the learning process of 3DGS, focused on minimizing reconstruction loss, while the outer layer is designed to maximize computation costs. The innovative attack is achieved through three tailored strategies:

1. Attack Objective Approximation: Utilizing the number of Gaussians as a surrogate metric for computation cost, the research showcases a strong correlation with GPU memory usage and latency.
2. Proxy Model Rendering: A proxy 3DGS model is employed to maintain consistency across multi-view images, enabling more effective and seamless perturbations.
3. Constrained Optimization: The inclusion of a constraint via the $L_\infty$ norm limits perturbations, balancing the attack's impact with its detectability, thus ensuring stealthy execution.

These strategies underscore the attack’s effectiveness and pose significant challenges against simple defensive measures.

Results

The experimental evaluations demonstrate profound impacts on computational resources across varied datasets, with peak GPU memory usage often exceeding common GPU capacities in constrained scenarios. In unconstrained conditions, the attack's intensity increases drastically, consuming up to 80 GB of GPU memory, which could potentially lead to Denial-of-Service (DoS) incidents.

Additionally, the attack's robustness is illustrated by its effect on a black-box victim (Scaffold-GS), indicating broad applicability across different Gaussian Splatting implementations.

Implications

The introduction of this attack has several implications:

• Theoretical: It points to vulnerabilities inherent in adaptive model architectures, urging a reevaluation of such designs in terms of security.
• Practical: It threatens the operational stability of service providers, highlighting the necessity for robust defensive measures beyond the naïve constraining of Gaussian counts.

Future Directions

Future research should focus on creating resilient 3DGS algorithms and comprehensively understanding adversarial impact in real-time 3D rendering systems. Moreover, exploring alternative metrics for cost approximation could refine attack strategies or inform more targeted defenses.

Conclusion

This paper significantly contributes to the understanding of computation cost vulnerabilities within 3DGS, potentially influencing the development of more secure adaptive architectures. By shedding light on these vulnerabilities, it encourages the community towards innovation in safeguarding 3D technologies.