TeleSparse: Practical Privacy-Preserving Verification of Deep Neural Networks (2504.19274v2)

Abstract: Verification of the integrity of deep learning inference is crucial for understanding whether a model is being applied correctly. However, such verification typically requires access to model weights and (potentially sensitive or private) training data. So-called Zero-knowledge Succinct Non-Interactive Arguments of Knowledge (ZK-SNARKs) would appear to provide the capability to verify model inference without access to such sensitive data. However, applying ZK-SNARKs to modern neural networks, such as transformers and large vision models, introduces significant computational overhead. We present TeleSparse, a ZK-friendly post-processing mechanisms to produce practical solutions to this problem. TeleSparse tackles two fundamental challenges inherent in applying ZK-SNARKs to modern neural networks: (1) Reducing circuit constraints: Over-parameterized models result in numerous constraints for ZK-SNARK verification, driving up memory and proof generation costs. We address this by applying sparsification to neural network models, enhancing proof efficiency without compromising accuracy or security. (2) Minimizing the size of lookup tables required for non-linear functions, by optimizing activation ranges through neural teleportation, a novel adaptation for narrowing activation functions' range. TeleSparse reduces prover memory usage by 67% and proof generation time by 46% on the same model, with an accuracy trade-off of approximately 1%. We implement our framework using the Halo2 proving system and demonstrate its effectiveness across multiple architectures (Vision-transformer, ResNet, MobileNet) and datasets (ImageNet,CIFAR-10,CIFAR-100). This work opens new directions for ZK-friendly model design, moving toward scalable, resource-efficient verifiable deep learning.

Overview of TeleSparse: Practical Privacy-Preserving Verification of Deep Neural Networks

In this paper, the authors present TeleSparse, an approach aimed at efficiently verifying deep neural network inferences using Zero-Knowledge Succinct Non-Interactive Arguments of Knowledge (ZK-SNARKs) without accessing sensitive data, addressing the unique challenges posed by modern neural networks. This work addresses the need for robust verification of machine learning models in scenarios such as Machine Learning as a Service (MLaaS), where service consumers may not trust the service provider or might need assurance of model integrity without exposing proprietary or sensitive model parameters.

Core Contributions

The paper identifies two major challenges when applying ZK-SNARKs to neural networks: the computational overhead due to over-parameterization and the inefficiency in handling non-linear activation functions. Here are the primary contributions and solutions proposed in this paper:

1. Reduction of Circuit Constraints via Sparsification: The authors apply model sparsification to reduce the number of constraints needed during the proving process while maintaining the model’s accuracy, thereby enhancing proof efficiency. Using this approach, the computational and memory requirements for generating proofs are significantly decreased. The authors demonstrate that such pruning does not compromise the security or integrity of the ZK-SNARK verification process.
2. Optimization of Activation Function Range via Neural Teleportation: This novel technique reduces the required resources for lookup tables essential for non-linear activation functions. By employing neural teleportation, originally intended for accelerating neural network training, the authors constrain the range of activation inputs, thus minimizing the size of lookup tables and further improving verification efficiency.
3. Implementation and Performance: TeleSparse is implemented using the Halo2 proving system, which is well-suited to this optimization strategy. The paper reports substantial improvements in resource utilization: reducing prover memory usage by 67% and proof generation time by 46% across evaluated architectures such as Vision Transformer, ResNet, and MobileNet using datasets like ImageNet, CIFAR-10, and CIFAR-100. The accuracy trade-off was minimal, approximately 1%.

Theoretical and Practical Implications

The theoretical advances presented in this paper revolve around adapting sparsification techniques in a way that is compatible with ZK-SNARKs without sacrificing model performance or verification soundness. Additionally, the introduction of neural teleportation to address activation function challenges in ZK environments highlights the potential for cross-disciplinary innovation in this space.

Practically, TeleSparse opens new potential avenues for deploying large neural models in privacy-centric applications. The reduced overhead could make privacy-preserving ML solutions more attractive, particularly in edge AI applications where resources are constrained.

Speculation on Future Developments

Future research could extend these methodologies to other ZK-SNARK systems beyond Halo2, expanding their applicability. Furthermore, the adaptability of these techniques to different machine learning architectures or more complex models such as LLMs remains a promising area. Another possible avenue could be the integration of adaptive sparsification techniques during model training to further enhance efficiency.

In summary, TeleSparse provides a foundational approach to efficiently and privately verify neural network inferences, which could be crucial as more industries seek to leverage machine learning while maintaining data privacy and model confidentiality. This work stands as a significant contribution to the field of privacy-preserving machine learning, with strong potential for practical application and further academic exploration.