Privatar: Scalable Privacy-preserving Multi-user VR via Secure Offloading

Abstract: Multi-user virtual reality enables immersive interaction. However, rendering avatars for numerous participants on each headset incurs prohibitive computational overhead, limiting scalability. We introduce a framework, Privatar, to offload avatar reconstruction from headset to untrusted devices within the same local network while safeguarding attacks against adversaries capable of intercepting offloaded data. Privatar's key insight is that domain-specific knowledge of avatar reconstruction enables provably private offloading at minimal cost. (1) System level. We observe avatar reconstruction is frequency-domain decomposable via BDCT with negligible quality drop, and propose Horizontal Partitioning (HP) to keep high-energy frequency components on-device and offloads only low-energy components. HP offloads local computation while reducing information leakage to low-energy subsets only. (2) Privacy level. For individually offloaded, multi-dimensional signals without aggregation, worst-case local Differential Privacy requires prohibitive noise, ruining utility. We observe users' expression statistical distribution are slowly changing over time and trackable online, and hence propose Distribution-Aware Minimal Perturbation. DAMP minimizes noise based on each user's expression distribution to significantly reduce its effects on utility, retaining formal privacy guarantee. Combined, HP provides empirical privacy against expression identification attacks. DAMP further augments it to offer a formal guarantee against arbitrary adversaries. On a Meta Quest Pro, Privatar supports 2.37x more concurrent users at 6.5% higher reconstruction loss and 9% energy overhead, providing a better throughout-loss Pareto frontier over quantization, sparsity and local construction baselines. Privatar provides both provable privacy guarantee and stays robust against both empirical and NN-based attacks.

• The paper introduces PRIVATAR, a framework that secures multi-user VR by partitioning facial texture data using block-wise DCT to protect sensitive features.
• It employs distribution-aware minimal perturbation (DAMP) to inject calibrated noise, reducing noise by up to 17.6× compared to uniform differential privacy.
• Experimental results show PRIVATAR supports 2.37× more concurrent users with only a 5.7–6.5% reconstruction loss increase and around 9% additional energy consumption.

Scalable Privacy-Preserving Multi-User VR via Secure Offloading: An Analysis of PRIVATAR

Introduction

Scalable, high-fidelity multi-user VR requires fast, efficient, and privacy-preserving avatar reconstruction. The computationally intensive nature of real-time avatar rendering fundamentally limits user scalability on mobile VR headsets. Offloading computation to local, untrusted devices offers a potential solution but introduces substantial privacy risks related to the exposure of sensitive facial features. The PRIVATAR framework, as introduced in "Privatar: Scalable Privacy-preserving Multi-user VR via Secure Offloading" (2604.17476), presents a solution architecture that jointly optimizes for privacy, utility, energy, and scalability in multi-user VR settings.

System and Threat Model

In PRIVATAR’s architecture, each user trusts only their own VR headset. Offloading involves transmission of latent code representations of facial data to untrusted devices connected through potentially adversarial local networks. The threat model assumes a computationally unbounded adversary with complete knowledge of the protocol, data distributions, and encoding/decoding mechanisms, with the exception of individualized user randomness and noise realizations.

Conventional Approaches and Limitations

Previous privacy-preservation paradigms for offloading—cryptographic encryption (e.g., HE, MPC), trusted hardware (e.g., TEEs), and information-theoretic mechanisms (e.g., Differential Privacy via local perturbation)—are individually ineffective for real-time, high-fidelity, scalable VR avatar reconstruction. Homomorphic encryption incurs three orders of magnitude higher latency and impractical computational communication complexity; MP protocols similarly suffer prohibitive bandwidth requirements (2604.17476). Trusted hardware mechanisms (e.g., Intel SGX, AMD SEV-SNP) increase compute isolation but are bottlenecked by CPU throughput and cannot match the compute demands of neural avatar decoders. Information-theoretic DP approaches, when directly applied, require such a large magnitude of isotropic noise that avatar utility collapses even at moderate privacy levels due to high data dimensionality and lack of aggregation.

PRIVATAR Design: Horizontal Partitioning and Distribution-Aware Minimal Perturbation

Horizontal Partitioning (HP)

PRIVATAR exploits frequency-domain decomposability of high-resolution facial texture data. It applies block-wise Discrete Cosine Transform (DCT) to each region of the unwrapped facial texture. Statistical analysis reveals that the lowest base frequency component captures more than 94% of the total signal energy, which is highly sensitive from a privacy perspective. HP retains these high-energy, privacy-rich components and the mesh fully on-device, while only offloading the low-energy, high-frequency components to external devices. By never transmitting a complete or high-salience facial representation, HP provides empirical resistance against expression identification and facial reconstruction attacks.

Distribution-Aware Minimal Perturbation (DAMP)

Even after partitioning, the direct application of local DP to the offloaded components would result in significant utility loss. DAMP leverages two critical insights: (1) the distribution of users’ facial expression embeddings is highly structured and drifts slowly over time, and (2) not all frequency components have equivalent entropy. DAMP employs PAC Privacy to calibrate the amount and allocation of noise injected into each offloaded dimension, targeting the minimal mutual information between raw data and the noisy release compatible with a specified adversarial success bound. Calibration occurs per-user, per-dimension, and adapts continuously as the expression distribution evolves, yielding up to 17.6x noise reduction in norm compared to isotropic local DP for equivalent formal guarantees.

Performance Evaluation

Experimental evaluation on the Meta Quest Pro VR headset with an untrusted RTX 5090-based PC demonstrates that PRIVATAR can support 2.37× more concurrent users than baseline (local only) approaches while incurring only a 5.7–6.5% increase in reconstruction loss and approximately 9% additional energy consumption. These results place PRIVATAR on a superior throughput-loss Pareto frontier compared to standard quantization, sparsity, and local-only workflows.

PRIVATAR achieves a maximal posterior success rate (PSR) for expression classification below that of random guessing (e.g., 1.54% for 65-way classification) for both empirical and trained neural network attackers—the same holds under formal PAC-privacy-based analysis. In ablations, increases in offloaded component count correspond to greater throughput improvements, plateauing when communication rather than local computation becomes the bottleneck.

Comparative Perspective

Relative to quantization (8-bit) and 10% sparsity methods, PRIVATAR yields 2.27× and 2.06× higher throughput, respectively, with visual quality degradation well below human perceptibility thresholds (LPIPS increases <1%). In contrast to fully offloaded pipelines with uniform DP noise—which completely obliterate utility (105× loss increase)—PRIVATAR maintains high rendering fidelity. Deployments leveraging CPU-based TEE (e.g., AMD SEV-SME) provide theoretical privacy, but at the expense of 30% lower throughput relative to PRIVATAR due to inferior CPU inference performance on decoding paths.

Security and Practical Implications

PRIVATAR’s approach fundamentally shifts the privacy-utility balance in multi-user VR systems. By using HP and DAMP, it demonstrates the achievability of strong empirical and formally certified privacy guarantees with practical utility for high-dynamic-range visual data. For real deployments, this means supporting larger collaborative or social VR experiences (e.g., live concerts, team sports, co-design sessions) without degrading user experience or exposing sensitive facial telemetry to local network adversaries.

Theoretical Implications and Future Directions

The PRIVATAR framework integrates representation-theoretic priors (i.e., spectrum energy concentration, user-level statistical stability) deeper into privacy mechanism design. Combining frequency-domain partitioning with PAC-privacy based noise calibration provides a template for future privacy-preserving perceptual pipelines—where sensitivity is often highly anisotropic and modeling local distribution is tractable online.

Future lines of inquiry include extending frequency-domain partitioning to other graphics pipelines, incorporating adaptive re-partitioning based on scene dynamics or user context, and integrating hardware accelerators for secure, low-latency, multi-modal processing. Moreover, seamless hardware-software co-design for offloading and noise calculation could further close the gap to confidential computing approaches.

Conclusion

PRIVATAR establishes a robust, empirically validated framework for privacy-preserving scalable VR. Its combination of frequency-domain partitioning and distribution-aware noise minimization delivers strong privacy-utility trade-offs, outperforming prior cryptographic, trusted execution, or local DP strategies. As VR and AR systems increase in user count and interaction complexity, such system-level innovations will be critical for balancing scale, performance, and user privacy.