Instant Skinned Gaussian Avatars for Web, Mobile and VR Applications (2510.13978v1)

Abstract: We present Instant Skinned Gaussian Avatars, a real-time and cross-platform 3D avatar system. Many approaches have been proposed to animate Gaussian Splatting, but they often require camera arrays, long preprocessing times, or high-end GPUs. Some methods attempt to convert Gaussian Splatting into mesh-based representations, achieving lightweight performance but sacrificing visual fidelity. In contrast, our system efficiently animates Gaussian Splatting by leveraging parallel splat-wise processing to dynamically follow the underlying skinned mesh in real time while preserving high visual fidelity. From smartphone-based 3D scanning to on-device preprocessing, the entire process takes just around five minutes, with the avatar generation step itself completed in only about 30 seconds. Our system enables users to instantly transform their real-world appearance into a 3D avatar, making it ideal for seamless integration with social media and metaverse applications. Website: https://sites.google.com/view/gaussian-vrm

• The paper presents a system that binds Gaussian splats to a skinned mesh for real-time avatar animation while preserving high visual fidelity.
• It employs a single-camera scan with minimal preprocessing and achieves 40–50 fps on mobile devices through parallel splat updates.
• The browser-based implementation leverages Three.js and the VRM mesh format, enabling seamless deployment on web, mobile, and VR platforms.

Instant Skinned Gaussian Avatars: A Technical Analysis

Introduction

The paper "Instant Skinned Gaussian Avatars for Web, Mobile and VR Applications" (2510.13978) introduces a real-time, cross-platform system for generating and animating photorealistic 3D avatars using Gaussian Splatting. The approach is designed for accessibility, enabling avatar creation and animation on commodity hardware, including smartphones and web browsers, with minimal preprocessing time. The system addresses key limitations of prior Gaussian Splatting-based avatar methods, such as the need for multi-camera setups, extensive preprocessing, or high-end GPUs, while maintaining high visual fidelity and real-time performance.

Technical Contributions

Efficient Splat-Mesh Binding and Animation

The core technical contribution is a pipeline that binds Gaussian splats to a skinned background mesh, enabling the splats to follow mesh deformations in real time. The process consists of:

1. Acquisition: A single-camera video is captured using a mobile 3D scanning app (Scaniverse), with the subject in an A-pose to facilitate downstream processing.
2. Filtering and Normalization: Non-subject splats are removed via rule-based spatial filtering, and the remaining splats are normalized in position and scale.
3. Pose Estimation and Mesh Alignment: The subject's pose and orientation are estimated, and a neutral VRM-format mesh is aligned accordingly.
4. Splat-to-Vertex Assignment: Each splat is assigned to its nearest mesh vertex using a nearest-neighbor search, and the relative transformation between the splat and the vertex is stored.
5. Data Export: The system outputs the pose, scale, vertex indices, and relative transformations for use at runtime.

At runtime, the mesh is animated (e.g., via motion capture or procedural animation), and each splat's position is updated in parallel according to the current transformation of its associated mesh vertex. This per-splat parallelism is critical for achieving real-time performance on both mobile and desktop hardware.

Web and Mobile Implementation

The entire system, excluding the initial scan, is implemented as a browser application using JavaScript and Three.js. The use of the VRM mesh format ensures compatibility with existing avatar and VRM-based applications. The pipeline is fully automated, requiring no manual intervention after the initial scan.

Performance Metrics

• Preprocessing Time: Full avatar creation, including scanning, is completed in approximately five minutes on an iPhone 13 Pro; the avatar generation step itself takes about 30 seconds.
• Runtime Performance: The system achieves 40–50 fps on an iPhone 13 Pro and up to 240 fps (display-limited) on a laptop with an NVIDIA GeForce RTX 3060 GPU.
• Visual Fidelity: The method preserves the photorealism of Gaussian Splatting, avoiding the fidelity loss associated with mesh conversion approaches.

Comparison with Prior Work

Previous methods for animating Gaussian Splatting avatars, such as "Animatable Gaussians" [li2024animatablegaussians], "3DGS-Avatar" [qian20233dgsavatar], and "GauHuman" [GauHuman], often require multi-view capture, lengthy optimization, or high-end hardware. ExAvatar [moon2024exavatar] can generate avatars from a single video but requires hours of preprocessing and relies on SMPL meshes, which reduce geometric detail. Mesh conversion approaches improve efficiency but at the cost of visual quality.

The presented system distinguishes itself by:

• Single-View, Fast Pipeline: Only a single video is required, and the entire process is completed in minutes on consumer hardware.
• No Mesh Conversion: The method retains the expressive power of Gaussian Splatting, avoiding the artifacts and loss of detail inherent in mesh-based simplifications.
• Real-Time, Cross-Platform: The browser-based implementation enables deployment on a wide range of devices, including smartphones and VR headsets, without specialized hardware.

Implementation Details

Splat-to-Vertex Assignment

The assignment of splats to mesh vertices is performed via a nearest-neighbor search in 3D space. For each splat $s_i$ with position $\mathbf{p}_i$, the nearest mesh vertex $v_j$ is found by minimizing $\|\mathbf{p}_i - \mathbf{v}_j\|_2$. The relative transformation (offset and orientation) between $s_i$ and $v_j$ is stored for use during animation.

Parallel Splat Updates

At runtime, the mesh is deformed according to animation parameters (e.g., skeletal pose). For each frame, the position and orientation of each splat are updated in parallel:

// Pseudocode for per-frame splat update for (let i = 0; i < numSplats; ++i) { let vertexIdx = splatToVertex[i]; let meshTransform = mesh.getVertexTransform(vertexIdx); splat[i].position = meshTransform.apply(splatRelativeOffset[i]); splat[i].orientation = meshTransform.apply(splatRelativeOrientation[i]); }

This parallelism is well-suited to GPU or multi-core CPU execution, and the use of JavaScript with Three.js allows for efficient WebGL-based rendering.

Integration and Deployment

The use of the VRM mesh format and Three.js enables seamless integration with existing VR, AR, and web-based avatar systems. The system is compatible with WebXR, facilitating deployment in metaverse and social media applications.

Limitations and Trade-offs

• Pose Generalization: The method relies on the accuracy of the initial pose estimation and mesh alignment. Large deviations from the A-pose or significant non-rigid deformations may reduce fidelity.
• Splat Assignment Granularity: Assigning each splat to the nearest mesh vertex may introduce artifacts in regions with sparse mesh resolution or high curvature.
• Background Mesh Expressiveness: The use of a neutral VRM mesh may limit the expressiveness of body shapes and fine-grained deformations compared to subject-specific meshes.

Implications and Future Directions

The system demonstrates that high-fidelity, animatable avatars can be generated and deployed in real time on commodity hardware, significantly lowering the barrier to entry for photorealistic avatar creation. This has direct implications for virtual social interaction, telepresence, and digital twin applications, where rapid and accessible avatar generation is critical.

Future research directions include:

• Improved Splat-Mesh Correspondence: Leveraging learned correspondences or denser mesh templates to improve deformation fidelity.
• Dynamic Splat Reassignment: Allowing splats to dynamically rebind to mesh vertices to better handle extreme deformations.
• Personalized Meshes: Integrating subject-specific mesh estimation to improve anatomical accuracy and expressiveness.
• End-to-End Mobile Pipelines: Further optimizing the pipeline for on-device execution, including the initial scan and pose estimation stages.

Conclusion

"Instant Skinned Gaussian Avatars for Web, Mobile and VR Applications" presents a practical and efficient system for real-time, photorealistic avatar creation and animation using Gaussian Splatting. By binding splats to a skinned mesh and leveraging parallel per-splat updates, the method achieves high visual fidelity and real-time performance on commodity devices. The approach addresses key limitations of prior work and opens new avenues for accessible, high-quality avatar generation in consumer applications.