DSFFNet: Dual-Side Feature Fusion Network for 3D Pose Transfer (2305.14951v2)

Abstract: To solve the problem of pose distortion in the forward propagation of pose features in existing methods, this pa-per proposes a Dual-Side Feature Fusion Network for pose transfer (DSFFNet). Firstly, a fixed-length pose code is extracted from the source mesh by a pose encoder and combined with the target vertices to form a mixed feature; Then, a Feature Fusion Adaptive Instance Normalization module (FFAdaIN) is designed, which can process both pose and identity features simultaneously, so that the pose features can be compensated in layer-by-layer for-ward propagation, thus solving the pose distortion problem; Finally, using the mesh decoder composed of this module, the pose are gradually transferred to the target mesh. Experimental results on SMPL, SMAL, FAUST and MultiGarment datasets show that DSFFNet successfully solves the pose distortion problem while maintaining a smaller network structure with stronger pose transfer capability and faster convergence speed, and can adapt to meshes with different numbers of vertices. Code is available at https://github.com/YikiDragon/DSFFNet