MoReMouse: Dense 3D Mouse Reconstruction

• MoReMouse is a monocular dense 3D reconstruction system for laboratory mice that maps a single RGB image to a detailed 3D surface with RGB appearance, normals, and semantic embedding.
• It leverages a high-fidelity synthetic dataset from a realistic Gaussian mouse avatar and a two-stage transformer-based pipeline to overcome challenges of non-rigid deformations and textureless fur.
• MoReMouse demonstrates superior performance over sparse keypoint methods by accurately recovering fine anatomical details and achieving high quantitative metrics (PSNR, SSIM, LPIPS).

Searching arXiv for the MoReMouse paper and closely related entries to ground the article and citations. Search query: "MoReMouse Monocular Reconstruction of Laboratory Mouse" MoReMouse is a monocular dense 3D reconstruction system for laboratory mice that maps a single RGB image to a detailed 3D surface representation together with RGB appearance, normals, and a continuous semantic embedding over the surface. It is presented as the first monocular dense 3D reconstruction network tailored to laboratory mice, and it addresses a domain in which accurate surface recovery is impeded by complex non-rigid deformation, textureless appearance, and the absence of structured dense 3D datasets. Its design combines three elements: a high-fidelity dense-view synthetic dataset rendered from a realistic Gaussian mouse avatar, a transformer-based feed-forward architecture with triplane representation, and geodesic-based continuous correspondence embeddings used as semantic priors for stable surface reconstruction (Zhong et al., 6 Jul 2025).

1. Problem setting and scientific scope

MoReMouse targets dense 3D surface reconstruction of a laboratory mouse from a single RGB image. In this context, “dense” denotes full-surface recovery rather than sparse keypoint estimation, with outputs intended to preserve fine structures such as tail and paws. The problem is substantially harder than standard articulated pose estimation because laboratory mice exhibit highly non-rigid, complex deformations, often have black or white textureless fur with few stable local landmarks, and lack the dense multi-view 3D corpora that support general object- or human-centric reconstruction pipelines (Zhong et al., 6 Jul 2025).

The method is positioned against three limitations of prior practice. First, parametric animal models driven by Linear Blend Skinning (LBS) can produce self-penetration and unrealistic deformation under rapid motion. Second, feature-based correspondence estimation is unreliable when appearance is uniform and local texture is weak. Third, computational ethology pipelines such as DeepLabCut, SLEAP, and DANNCE primarily operate on sparse keypoints rather than dense surfaces. This makes it difficult to recover curvature of the back, abdomen, and tail, or to infer detailed surface contact in social behavior.

The scientific motivation is correspondingly broad. Dense monocular mouse reconstruction is relevant to neuroscience and motor control, disease modeling and movement disorders, and social behavior analysis, especially where subtle posture, gait, tremor, or body-contact patterns are the quantity of interest. A feed-forward monocular method lowers the deployment barrier relative to optimization-heavy multi-camera systems, including in settings using standard video capture such as handheld phones.

2. Synthetic data foundation and Gaussian mouse avatar

A central component of MoReMouse is its synthetic supervision pipeline. Because no large dense 3D dataset of real mice is available, the system constructs its own training corpus from a realistic animatable avatar. The starting point is the articulated mouse mesh used in An et al. (MAMMAL), with 14,522 vertices and 140 articulated joints. Its parameters are defined as

$\Psi = \{\theta, t, r, s, B\},$

where $\theta \in \mathbb{R}^{3K}$ are per-joint rotations with $K=140$, $t \in \mathbb{R}^3$ is global translation, $r \in \mathbb{R}^3$ is global rotation, $s \in \mathbb{R}$ is global scale, and $B \in \mathbb{R}^{21}$ are bone-length deformation parameters. The authors separate local parameters $\Psi_l = \{\theta, B\}$ from global parameters $\Psi_g = \{r, t, s\}$, and animate the mesh with LBS (Zhong et al., 6 Jul 2025).

The avatar augments this articulated model with Gaussian splatting. Each Gaussian is parameterized by mean position $\mu \in \mathbb{R}^3$, covariance $\theta \in \mathbb{R}^{3K}$0, color $\theta \in \mathbb{R}^{3K}$1, opacity $\theta \in \mathbb{R}^{3K}$2, and a covariance factorization $\theta \in \mathbb{R}^{3K}$3, where $\theta \in \mathbb{R}^{3K}$4 and $\theta \in \mathbb{R}^{3K}$5 is obtained from a normalized quaternion $\theta \in \mathbb{R}^{3K}$6. The Gaussian scalar field is

$\theta \in \mathbb{R}^{3K}$7

Using a UV-based deformation scheme inspired by Animatable Gaussians, each valid texel in the mesh UV map corresponds to a Gaussian. A position map stores the canonical 3D location $\theta \in \mathbb{R}^{3K}$8, LBS deforms it to $\theta \in \mathbb{R}^{3K}$9, and a StyleUNet predicts pose-dependent position offsets and attributes. The final Gaussian means are

$K=140$0

The avatar is trained on the “markerless_mouse_1” sequence from Dunn et al. (2019, 2021), which contains 6 synchronized cameras, 18,000 frames, resolution $K=140$1, and 100 FPS. The training subset uses 800 frames uniformly sampled from the first 8,000 frames. Optimization runs for 400k steps with a loss combining $K=140$2, SSIM, LPIPS, and a Total Variation regularizer. Once trained, the avatar can render novel views in arbitrary poses driven by the fitted MAMMAL motion.

This avatar is then used to generate the MoReMouse training set. The first 6,000 frames of “markerless_mouse_1” are used for training and the last 6,000 for testing. For each frame, two sets of 64 viewpoints are rendered on a sphere with optical axes converging at the scene origin, producing 12,000 multi-view scenes. Each rendered view provides RGB, depth, mask/opacity, and a geodesic embedding texture. Images are translated and scaled so that the mouse centroid is centered and the animal appears at a consistent scale.

3. Architecture and reconstruction pipeline

MoReMouse is a mouse-specialized large reconstruction model with a feed-forward transformer backbone. At inference, it consumes a single RGB image at resolution $K=140$3, encodes it with DINOv2-base, and maps the resulting 768-dimensional token sequence into a triplane representation via a transformer decoder. The decoder uses 12 attention layers, 16 heads per layer, head dimension 64, and cross-attention dimension 768. Learnable triplane tokens are modulated by the image tokens through cross-attention and then reshaped and upsampled into three feature planes of shape $K=140$4 (Zhong et al., 6 Jul 2025).

The triplane representation stores features on the $K=140$5, $K=140$6, and $K=140$7 planes. For a 3D query point $K=140$8, features are bilinearly interpolated from the three planes and combined by the standard triplane query

$K=140$9

A multi-head MLP then maps this feature to density, color, and geodesic embedding; in the DMTet stage it also predicts an implicit surface quantity used for mesh extraction. The MLP has hidden width 64, 10 shared hidden layers, and one additional hidden layer per head for density, feature, and deformation.

Training proceeds in two stages. Stage 1 is a triplane-NeRF phase of 60 epochs. Rays are sampled with 128 points per ray inside a sphere of radius 0.87, density uses trunc_exp, and the density bias is $t \in \mathbb{R}^3$0. This stage emphasizes volumetric consistency and coarse geometry. Stage 2 is a triplane-DMTet phase of 100 epochs, with SDF bias $t \in \mathbb{R}^3$1 and isosurface resolution $t \in \mathbb{R}^3$2. This stage extracts and refines an explicit mesh via marching tetrahedra. The final inference output is a canonical-space mouse mesh together with rendered RGB, normals, and semantic embedding visualizations.

The architecture is explicitly feed-forward: no per-instance bundle adjustment or test-time optimization is performed. This distinguishes MoReMouse from optimization-heavy animal avatar pipelines and is central to its intended practical use.

4. Geodesic-based continuous correspondence embeddings

A defining element of MoReMouse is its geodesic-based continuous correspondence embedding, which functions as an intrinsic semantic prior over the mouse surface. The canonical mesh contains approximately 13k vertices; the method computes pairwise geodesic distances along that surface and learns a three-dimensional embedding whose Euclidean distances approximate those geodesic distances (Zhong et al., 6 Jul 2025).

Let $t \in \mathbb{R}^3$3 denote canonical mesh vertices. The geodesic distance matrix is

$t \in \mathbb{R}^3$4

and the embedding matrix is $t \in \mathbb{R}^3$5, with induced Euclidean distance

$t \in \mathbb{R}^3$6

The embedding is optimized by

$t \in \mathbb{R}^3$7

This creates a metric-preserving low-dimensional coordinate system tied to intrinsic surface geometry rather than to appearance.

For visualization, the learned embedding is transformed by PCA: the first two principal components are used as hue and saturation in HSV space, with value fixed at 1. The resulting color map provides high-contrast anatomical differentiation across the mouse body. During synthetic rendering, this embedding is rasterized as a surface texture, so the training data contains both conventional RGB images and embedding images.

The reconstruction network is then supervised to predict not only geometry and color but also the embedding. This encourages semantic consistency in visually ambiguous regions such as paws, snout, and tail. The ablation reported in the paper shows that removing the embedding degrades synthetic performance from PSNR 22.027, SSIM 0.9660, LPIPS 0.05279 to PSNR 21.805, SSIM 0.9655, LPIPS 0.05467, and real performance from PSNR 18.422, SSIM 0.9478, LPIPS 0.08674 to PSNR 18.250, SSIM 0.9469, LPIPS 0.08767. Qualitatively, the version without embedding yields blurrier paws, weaker tail shape, and less stable snout contours.

5. Training objective, evaluation protocol, and empirical performance

The MoReMouse training loss combines image fidelity, geometric supervision, foreground segmentation, perceptual similarity, and embedding supervision. The total objective is

$t \in \mathbb{R}^3$8

Here $t \in \mathbb{R}^3$9 supervises RGB images and geodesic embedding images, $r \in \mathbb{R}^3$0 enforces foreground depth consistency, $r \in \mathbb{R}^3$1 regularizes RGB differences, $r \in \mathbb{R}^3$2 is a binary cross-entropy loss on opacity or mask, and $r \in \mathbb{R}^3$3 is a perceptual loss. The reported weights are $r \in \mathbb{R}^3$4, $r \in \mathbb{R}^3$5, $r \in \mathbb{R}^3$6, $r \in \mathbb{R}^3$7, and $r \in \mathbb{R}^3$8 (Zhong et al., 6 Jul 2025).

Evaluation is performed on two datasets. The synthetic benchmark uses the last 6,000 frames of “markerless_mouse_1,” with four orthogonal viewpoints per frame. The real benchmark consists of 5,400 frames captured by four calibrated industrial cameras, with camera rotations relative to the reference view of $r \in \mathbb{R}^3$9, $s \in \mathbb{R}$0, $s \in \mathbb{R}$1, and $s \in \mathbb{R}$2. The task is novel-view synthesis from one input view, and the metrics are PSNR, SSIM, and LPIPS.

Method	Synthetic (PSNR / SSIM / LPIPS)	Real (PSNR / SSIM / LPIPS)
MoReMouse	22.027 / 0.9660 / 0.05279	18.422 / 0.9478 / 0.08674
Triplane-GS	18.049 / 0.9268 / 0.10151	16.789 / 0.9298 / 0.11002
InstantMesh	15.821 / 0.8987 / 0.11312	15.631 / 0.9175 / 0.11339
LGM	14.460 / 0.8805 / 0.13274	15.215 / 0.9197 / 0.12838
TripoSR	13.673 / 0.8032 / 0.18255	11.518 / 0.8114 / 0.19672

These results show a substantial margin over the strongest single-view baseline, Triplane-GS, on both synthetic and real data. The paper also compares MoReMouse to multi-view diffusion methods under a stronger input setting: on synthetic data, LGM with 4 views reaches PSNR 23.998, SSIM 0.9698, LPIPS 0.06458, and InstantMesh with 6 views reaches PSNR 28.295, SSIM 0.9831, LPIPS 0.02716, while MoReMouse uses only 1 view and attains PSNR 22.027, SSIM 0.9660, LPIPS 0.05279.

A further control fine-tunes TripoSR on the same synthetic mouse dataset for 120 epochs with learning rate $s \in \mathbb{R}$3. The tuned model reaches synthetic PSNR 21.996, SSIM 0.9676, LPIPS 0.06333 and real PSNR 18.162, SSIM 0.9461, LPIPS 0.09354. This suggests that MoReMouse’s gains are not reducible to dataset specialization alone; the paper attributes the remaining advantage to architecture choices, higher triplane resolution, geodesic embedding supervision, and the two-stage training regime.

6. Limitations, relation to adjacent research, and significance

MoReMouse has several explicit limitations. Its training corpus derives from a single mouse subject, one environment, and one lighting setup, so generalization across fur colors, body shapes, and illumination is not guaranteed. The model operates in a canonical centered coordinate frame and therefore does not reconstruct global 3D trajectory in a world coordinate system. Self-occlusion and object occlusion are not explicitly modeled, and fine-scale details such as whiskers, nose details, and foot tips remain sensitive to resolution. Failure cases include unusual upright poses and scenarios in which the tail crosses the head region (Zhong et al., 6 Jul 2025).

The method sits at an intersection of several prior literatures. Relative to SMAL-derived animal models, AWOL, and related LBS-based quadruped reconstruction, it replaces direct dependence on a fixed parametric body model at inference with an implicit triplane field and explicit mesh extraction. Relative to human-centric monocular avatar systems, it adapts transformer-based large reconstruction model ideas to a morphology and appearance regime for which general human priors do not transfer. Relative to general single-image 3D models such as TripoSR, Triplane-GS, LGM, and InstantMesh, it adds a mouse-specific synthetic dataset, higher-resolution triplanes, and intrinsic geodesic priors.

The name should also be distinguished from “MoRe: Motion-aware Feed-forward 4D Reconstruction Transformer,” which addresses monocular video-based dynamic 4D scene reconstruction with depth, pose, point maps, and motion masks rather than single-image dense mouse surface reconstruction (Fang et al., 5 Mar 2026). In MoReMouse, the focus is not dynamic scene decomposition but high-quality canonical-surface generation of a laboratory mouse from one view.

A common misconception is that MoReMouse is merely a fine-tuned general-purpose large reconstruction model. The TripoSR fine-tuning experiment argues against that reading: after adaptation on the same dataset, the baseline improves substantially, but MoReMouse still achieves lower LPIPS and better qualitative anatomical coherence. A second misconception is that MoReMouse solves full 3D behavioral tracking in world coordinates. The paper does not claim that; it reconstructs relative pose and shape in canonical space and would require additional tracking or camera geometry for global motion analysis.

Its broader significance lies in making dense mouse reconstruction tractable with monocular input and feed-forward inference. Within computational ethology, this provides a route from ordinary video to detailed 3D surfaces, potentially enabling richer quantification of posture, deformation, and inter-animal contact than sparse keypoint pipelines alone.