LoomNet: Multi-View Latent Diffusion

• LoomNet is a multi-view latent diffusion model that generates mutually consistent novel views from a single image using latent space weaving and a shared triplane representation.
• The framework bridges high-quality single-view generation with cross-view consistency, outperforming methods like Zero-1-to-3 and EpiDiff in geometric and texture fidelity.
• It employs parallel diffusion branches with frozen UNet weights and communication modules, enabling robust 3D reconstruction for applications in AR/VR and content creation.

LoomNet is a multi-view latent diffusion framework for synthesizing mutually consistent novel-view images from a single input image and a set of target camera poses. Introduced in "LoomNet: Enhancing Multi-View Image Generation via Latent Space Weaving" (Federico et al., 7 Jul 2025), it addresses the central failure mode of single-image novel-view synthesis: independently generated views often disagree in geometry and appearance, and those inconsistencies substantially degrade downstream 3D reconstruction. LoomNet retains the single-view generative prior of a pretrained Zero-1-to-3 latent diffusion model, but augments it with an explicit shared triplane representation built through a process termed latent space weaving, so that multiple viewpoint-specific diffusion branches collaboratively construct and query a unified 3D-aware latent scene.

1. Terminology and scope

In the arXiv record, LoomNet most precisely denotes the architecture introduced for multi-view image generation and 3D-aware latent fusion in (Federico et al., 7 Jul 2025). Its core task is single-image-conditioned generation of $N$ novel views $V^{1:N}$ from an input image $V^0$ and relative camera poses $[R,T]^{1:N}$, with the requirement that the generated set be both high-quality and multi-view consistent.

The term has also appeared in looser, interpretive usage around other systems, but those are distinct from the 2025 multi-view diffusion model. In the LoomVideo paper, the name used throughout is LoomVideo rather than LoomNet; there, “LoomNet” is treated only as shorthand for the underlying unified video generation and editing architecture built from a 5B Diffusion Transformer, Qwen3-VL, Deepstack injection, Scale-and-Add conditioning, and Negative Temporal RoPE (Wu et al., 4 Jun 2026). In Shadow-Loom, “LoomNet” is likewise presented only conceptually, as a possible abstraction over a typed, versioned graphical world model for narrative causal reasoning (Wilmot, 4 May 2026). In the older graph-partitioning literature, the 2017 system is simply called Loom, and “LoomNet” is not the paper’s term (Firth et al., 2017).

For technical work on image-based novel-view synthesis, reconstruction-oriented multi-view generation, and latent triplane communication, the primary referent is therefore the architecture of (Federico et al., 7 Jul 2025).

2. Problem formulation and prior approaches

LoomNet considers the setting

$V^{1:N} = f\left(V^{0}, [R,T]^{1:N}\right),$

where $V^0 \in \mathbb{R}^{H\times W\times 3}$ is a single observed image and each target pose is specified by rotation $R\in\mathbb{R}^{3\times 3}$ and translation $T\in\mathbb{R}^3$. The objective is not merely to synthesize plausible images independently, but to generate a set of views that agree on the same underlying object geometry and appearance (Federico et al., 7 Jul 2025).

The difficulty is fundamentally ill-posed. A single 2D image is compatible with many 3D explanations, and diffusion-based novel-view models such as Zero-1-to-3 typically generate each target view independently. The resulting inconsistencies may be small at the image level, but they are severe for downstream NeRF- or NeuS-style reconstruction: geometry becomes incomplete or warped, and textures become smeared or inconsistent across the surface (Federico et al., 7 Jul 2025).

The paper positions LoomNet between two earlier design families. Zero-1-to-3 preserves strong single-view generative quality, but lacks any explicit mechanism forcing agreement across generated views. SyncDreamer introduces a global 3D feature volume shared by all views, improving consistency at the cost of substantial computation and lower image quality. EpiDiff uses epipolar constraints to improve speed and quality, but it does not maintain a persistent 3D latent representation, so error propagation may remain incoherent and there is no explicit unified scene representation (Federico et al., 7 Jul 2025).

LoomNet’s stated motivation is threefold: preserve the pretrained single-view generative capability of Zero-1-to-3, add inter-view communication so viewpoint-specific branches can collaborate, and make the shared 3D latent explicit through three orthogonal planes. This yields a middle ground between a fully explicit 3D volume and purely local cross-view constraints (Federico et al., 7 Jul 2025).

3. Parallel multi-view diffusion architecture

The model instantiates $N$ parallel copies of the same latent diffusion model $f_\theta$, initialized from Zero-1-to-3. Each branch handles one target camera pose and receives a noisy latent $V^{1:N}$0, the encoded input view $V^{1:N}$1, the relative pose $V^{1:N}$2, and the diffusion timestep $V^{1:N}$3. The underlying reverse process is written as

$V^{1:N}$4

with the standard Zero-1-to-3 diffusion objective

$V^{1:N}$5

A defining training choice is that all $V^{1:N}$6 share the same pretrained Zero-1-to-3 UNet weights and those weights remain frozen; only the newly introduced communication modules are trained (Federico et al., 7 Jul 2025).

Architecturally, each UNet has 4 encoder blocks, a bottleneck, and 4 decoder blocks. LoomNet inserts a communication module at the bottleneck and at every decoder block. At decoder level $V^{1:N}$7, each view $V^{1:N}$8 produces a feature map

$V^{1:N}$9

interpreted as that view’s current hypothesis of the scene at that resolution. The communication pathway then performs four operations: per-view splatting, fusion, weaving, and latent rendering at the final decoder layer (Federico et al., 7 Jul 2025).

Data flow is synchronized across views at each diffusion step. Each branch encodes its own latent independently, but decoder features are repeatedly projected into a shared triplane workspace, fused across viewpoints, refined, and then fed back into the per-view decoding path. After $V^0$0 diffusion steps, the final latents $V^0$1 are decoded by the shared VAE decoder into RGB novel views (Federico et al., 7 Jul 2025).

This structure preserves viewpoint-specific inference while introducing a persistent cross-view channel. Each branch proposes a view-specific image consistent with its target pose, but those proposals are no longer independent; they are constrained by a jointly woven latent scene.

4. Latent space weaving and triplane communication

The defining mechanism of LoomNet is latent space weaving. At each decoder level, the feature map of a given view is first treated as a camera-centric 2D feature field. For each pixel, a ray is cast through a fixed cube of side length $V^0$2 centered at the origin, and $V^0$3 points are uniformly sampled along the portion of that ray inside the cube. For each sampled 3D point $V^0$4, the model forms an enriched feature vector by concatenating the pixel feature from $V^0$5 with a Harmonic Embedding of a Plücker representation of the ray direction and the depth of the point along the ray (Federico et al., 7 Jul 2025).

These 3D samples are then projected onto the three orthogonal planes

$V^0$6

For each viewpoint, LoomNet maintains separate plane features $V^0$7, $V^0$8, and $V^0$9. Projection uses inverse bilinear splatting rather than nearest-neighbor assignment: $[R,T]^{1:N}$0 The decision to keep planes separate per view is explicit: direct joint splatting would cause feature smearing, loss of fine detail, and noise accumulation as the number of views grows (Federico et al., 7 Jul 2025).

Fusion occurs independently for each orientation. A learnable fusion plane $[R,T]^{1:N}$1 aggregates the corresponding per-view planes via pixel-wise cross-attention. At location $[R,T]^{1:N}$2, the query comes from the learnable plane and keys and values come from all viewpoint planes at that location: $[R,T]^{1:N}$3 followed by

$[R,T]^{1:N}$4

This avoids naive averaging and allows the model to emphasize informative views, resolve conflicts, and handle occlusion (Federico et al., 7 Jul 2025).

The weaving stage converts that discrete multi-view consensus into a continuous latent scene. Each fused plane is first normalized with AdaLayerNorm conditioned on the diffusion timestep $[R,T]^{1:N}$5, then all plane tokens are concatenated and processed with transformer-like self-attention and MLP blocks: $[R,T]^{1:N}$6

$[R,T]^{1:N}$7

Because attention spans all planes simultaneously, information propagates both spatially within each plane and across plane orientations. The fusion-plus-weaving sequence is repeated multiple times per decoder level, specifically 3 times in the bottleneck and 4, 6, and 8 times in deeper levels, so that missing regions can be interpolated and underconstrained areas can be inferred from neighboring or orthogonal evidence (Federico et al., 7 Jul 2025).

At the final decoder layer, the refined planes constitute a triplane scene representation. For each pixel in each target view, the model samples the shared triplane along the viewing ray, combines the resulting global features with the per-pixel decoder feature through a small MLP, and aggregates samples using learned importance weights. The paper reports that applying latent rendering only at the final decoder layer gives the best tradeoff; applying it at all decoder layers degrades quality (Federico et al., 7 Jul 2025).

5. Training regime and empirical performance

Training uses the LVIS subset of Objaverse, approximately 40K 3D models. Each object is rendered from 96 views at $[R,T]^{1:N}$8 resolution, with cameras arranged on 6 rings, elevations in $[R,T]^{1:N}$9, and azimuth uniformly sampled in $V^{1:N} = f\left(V^{0}, [R,T]^{1:N}\right),$0. For each object, $V^{1:N} = f\left(V^{0}, [R,T]^{1:N}\right),$1 views are randomly chosen during training. In addition to the frozen Zero-1-to-3 diffusion loss, LoomNet applies Total Variation regularization on the fused triplane maps,

$V^{1:N} = f\left(V^{0}, [R,T]^{1:N}\right),$2

with $V^{1:N} = f\left(V^{0}, [R,T]^{1:N}\right),$3. Training is performed on 4× H100 GPUs with learning rate $V^{1:N} = f\left(V^{0}, [R,T]^{1:N}\right),$4 and batch size 2 per GPU (Federico et al., 7 Jul 2025).

On Google Scanned Objects, following the EpiDiff evaluation protocol, LoomNet is reported in two settings. Under fixed elevation, it achieves PSNR $V^{1:N} = f\left(V^{0}, [R,T]^{1:N}\right),$5, SSIM $V^{1:N} = f\left(V^{0}, [R,T]^{1:N}\right),$6, LPIPS $V^{1:N} = f\left(V^{0}, [R,T]^{1:N}\right),$7, and runtime $V^{1:N} = f\left(V^{0}, [R,T]^{1:N}\right),$8 s for 16 views, compared with EpiDiff at $V^{1:N} = f\left(V^{0}, [R,T]^{1:N}\right),$9, $V^0 \in \mathbb{R}^{H\times W\times 3}$0, $V^0 \in \mathbb{R}^{H\times W\times 3}$1, and $V^0 \in \mathbb{R}^{H\times W\times 3}$2 s; SyncDreamer at $V^0 \in \mathbb{R}^{H\times W\times 3}$3, $V^0 \in \mathbb{R}^{H\times W\times 3}$4, $V^0 \in \mathbb{R}^{H\times W\times 3}$5, and $V^0 \in \mathbb{R}^{H\times W\times 3}$6 s; and Zero123 at $V^0 \in \mathbb{R}^{H\times W\times 3}$7, $V^0 \in \mathbb{R}^{H\times W\times 3}$8, $V^0 \in \mathbb{R}^{H\times W\times 3}$9, and $R\in\mathbb{R}^{3\times 3}$0 s. Under variable elevation, LoomNet reaches PSNR $R\in\mathbb{R}^{3\times 3}$1, SSIM $R\in\mathbb{R}^{3\times 3}$2, LPIPS $R\in\mathbb{R}^{3\times 3}$3, and $R\in\mathbb{R}^{3\times 3}$4 s, whereas EpiDiff obtains $R\in\mathbb{R}^{3\times 3}$5, $R\in\mathbb{R}^{3\times 3}$6, $R\in\mathbb{R}^{3\times 3}$7, and $R\in\mathbb{R}^{3\times 3}$8 s; SyncDreamer $R\in\mathbb{R}^{3\times 3}$9, $T\in\mathbb{R}^3$0, $T\in\mathbb{R}^3$1, and $T\in\mathbb{R}^3$2 s; and Zero123 $T\in\mathbb{R}^3$3, $T\in\mathbb{R}^3$4, $T\in\mathbb{R}^3$5, and $T\in\mathbb{R}^3$6 s. The paper notes that LoomNet’s performance is almost unchanged between fixed and variable elevation, which suggests robustness to more irregular camera layouts (Federico et al., 7 Jul 2025).

For 3D reconstruction, generated multi-view images are passed through foreground masking with CarveKit and then used to optimize an Instant-NGP-based SDF for 2000 steps, approximately 1.5 minutes per object. LoomNet reports Chamfer Distance $T\in\mathbb{R}^3$7 and Volume IoU $T\in\mathbb{R}^3$8, improving on EpiDiff’s $T\in\mathbb{R}^3$9 and $N$0, SyncDreamer’s $N$1 and $N$2, and Zero123’s $N$3 and $N$4. The paper states this is approximately a 40% reduction in Chamfer distance relative to EpiDiff (Federico et al., 7 Jul 2025).

Ablations isolate the contribution of the communication design. The reference configuration—final-layer latent rendering, $N$5, cross-attention fusion, and positional encoding—achieves PSNR $N$6, SSIM $N$7, LPIPS $N$8, and $N$9 s. Latent rendering at all decoder layers drops performance to $f_\theta$0, $f_\theta$1, $f_\theta$2, and $f_\theta$3 s. Replacing attention fusion with mean fusion yields $f_\theta$4, $f_\theta$5, $f_\theta$6, and $f_\theta$7 s. Reducing ray samples to $f_\theta$8 gives $f_\theta$9, $V^{1:N}$00, $V^{1:N}$01, and $V^{1:N}$02 s. Removing positional encoding produces $V^{1:N}$03, $V^{1:N}$04, $V^{1:N}$05, and $V^{1:N}$06 s, which the paper identifies as the largest drop among those ablations (Federico et al., 7 Jul 2025).

In efficiency terms, LoomNet generates 16 views in about 15 seconds. The paper attributes this to latent-space operation, use of triplanes rather than full 3D volumes, shared UNet backbones across views, and a fixed fused triplane of size $V^{1:N}$07. Memory use scales with the number of views $V^{1:N}$08 and the number of ray samples $V^{1:N}$09, while the fused triplane remains fixed-size (Federico et al., 7 Jul 2025).

6. Applications, limitations, and related uses of the name

The principal application is single-image to 3D reconstruction: a single image is expanded into 16 consistent views, which are then passed to SDF optimization to obtain a mesh. The paper also presents LoomNet as a method for novel view synthesis, a tool for content creation in AR/VR and games, and a possible source of shared latent triplane priors for downstream tasks such as pose estimation or shape completion. It further notes that the model can exhibit some “creativity” by generating diverse, plausible novel views from the same input, reflecting the generative character of the underlying diffusion model (Federico et al., 7 Jul 2025).

The reported limitations are equally specific. First, there is a consistency-versus-accuracy tradeoff: SyncDreamer’s global latent volume can propagate errors more consistently, so outputs may be coherently wrong, whereas LoomNet’s weaving produces high-quality and mostly consistent views but may not propagate errors perfectly across all viewpoints. Second, the overall workflow remains a two-stage pipeline in which view generation and 3D reconstruction are separate stages; the paper suggests that joint optimization could improve both consistency and efficiency. Third, LoomNet inherits the biases and limitations of Zero-1-to-3, including category and texture biases present in the base diffusion model (Federico et al., 7 Jul 2025).

The broader naming landscape is heterogeneous. In video generation, “LoomNet” has been used informally to denote the architecture underlying LoomVideo, but the paper’s formal system name remains LoomVideo; its defining mechanisms are Deepstack MLLM–DiT coupling, zero-overhead Scale-and-Add conditioning, and Negative Temporal RoPE (Wu et al., 4 Jun 2026). In narrative AI, Shadow-Loom presents a typed causal world graph and reader-state scoring framework, and “LoomNet” there is only a conceptual abstraction over modules such as WorldStateV1, AMWN-style branching, and causal versus narrative physics (Wilmot, 4 May 2026). In distributed graph databases, the related but historically separate system is Loom, a streaming, workload-aware graph partitioner rather than an image or video generative architecture (Firth et al., 2017).

Within the technical literature, then, LoomNet is best understood first and foremost as the explicit triplane-based multi-view diffusion architecture of (Federico et al., 7 Jul 2025): a system in which per-view hypotheses are splatted into orthogonal latent planes, fused by cross-attention, woven into a coherent shared 3D-aware latent space, and rendered back into individual viewpoint decoders to enforce mutual consistency.