MACRO: Training-free Multi-plane Attention for Closeup Render Optimization

Abstract: Close-up rendering, zooming into a scene well beyond any training camera, is important for virtual production and interactive 3D content, yet remains an open challenge. 3D Gaussian splatting (3DGS) enables high-fidelity, real-time novel view synthesis, but its rendering quality degrades at close range. Recent diffusion-based methods that enhance the rendering by conditioning on reference images from the training set produce significant artifacts in this setting. We analyze this failure and identify its root cause: the scale gap between the close-up and reference views. We show that the features in reference-conditioned enhancement models are not scale-invariant, causing cross-view attention to retrieve incorrect correspondences when the same content appears at different scales, and that this mismatch cannot be corrected in latent space because the VAE encoder is not scale-equivariant. Building on this analysis we introduce MACRO, Multi-plane Attention for Closeup Render Optimization, a training-free method for high-quality close-up novel view synthesis from 3DGS. MACRO resolves the scale gap by leveraging the scene's known 3D structure: it decomposes the close-up into depth planes, crops and resizes references in image space to match the scale of each plane before encoding, and applies a depth-aware attention mask so each token attends only to scale-matched references. The method requires no architectural changes or additional training. We further contribute two new close-up novel view synthesis benchmarks, the first standardized evaluation protocol for this setting, and demonstrate state-of-the-art results on both, outperforming existing 3DGS and diffusion-based methods on both reconstruction and perceptual metrics. Project page: https://nitzanhod.github.io/MACRO

• The paper introduces a training-free MACRO approach that employs multi-plane attention to overcome scale mismatch in close-up novel view synthesis.
• It combines image-space scale matching and depth-masked attention, yielding significant improvements in perceptual metrics over previous diffusion methods.
• Empirical results on DL3DV-Closeup and MobileClose-10 benchmarks demonstrate enhanced texture fidelity and computational efficiency.

Training-Free Multi-Plane Attention for Robust Close-Up Novel View Synthesis: Analysis of MACRO

Overview and Motivation

Close-up novel view synthesis from sparse, distant training images is unsolved in neural rendering, presenting critical obstacles for applications in virtual production, interactive 3D visualization, and content re-editing. While 3D Gaussian Splatting (3DGS) delivers high-fidelity synthesis near training views, its reconstructions deteriorate under severe zoom extrapolation. Recent reference-conditioned diffusion approaches attempt to compensate with texture transfer from authentic images, but under wide-to-tight zoom conditions, they introduce severe texture mapping errors due to fundamental scale misalignment between references and close-up views. The MACRO approach directly targets this limitation, combining geometric reasoning with latent cross-view attention to enable training-free, plug-and-play enhancement of close-up renderings (2607.03875).

Figure 1: MACRO restores close-up detail matching the ground truth, correcting scale-mismatch texture errors present in prior methods.

Analysis of Scale Mismatch in Reference-Guided Enhancement

MACRO’s development is predicated on a thorough analysis of why reference-guided diffusion enhancement methods fail at large scale deviations. Empirical evidence demonstrates that VAE and U-Net token features are not scale-invariant. When reference and target views depict the same content at disparate spatial scales, cross-view attention retrieves erroneous correspondences, corrupting the transferred textures. The authors further establish that latent-space warping or rescaling is futile for correcting this mismatch: the VAE encoder is not scale-equivariant, so encoding a rescaled image is not equivalent to rescaling its encoded latent. Quantitative measurements confirm the rapid decay of token correspondence and VAE latent similarity as the scale gap increases.

Figure 2: Cross-view token matching accuracy degrades and VAE latent similarity drops as scale divergence increases, invalidating naive latent-space alignment.

Consequently, effective scale alignment must occur in image space prior to encoding.

The MACRO Architecture: Multi-Plane, Scale-Matched Attention

The MACRO pipeline introduces a training-free strategy that injects 3D scene awareness into 2D diffusion models via three core stages:

1. Multi-Plane Depth Decomposition:

A rendered depth map from the 3DGS reconstruction is discretized into $P$ planar bins using $k$-means clustering in inverse depth space. Each depth plane corresponds to a distinct scale factor between target and reference views.

2. Scale-Matched Reference Cropping:

For each plane, MACRO computes the transformation between the close-up and each selected reference’s camera projection. It crops and resizes, in image space, the region of every reference view that spatially aligns (in 3D) and matches the real-world scale for that plane. As a result, a dense grid of $K \times P$ reference patches is produced (for $K$ reference images), each patch containing content mapped to target scale.

3. Depth-Masked Attention:

Encoded tokens from the close-up and all reference crops are supplied to the diffusion U-Net backbone. Critically, MACRO applies a blockwise attention mask so that each token in depth plane $p$ of the synthesized view attends only to tokens in the corresponding plane's patches of the reference images. This ensures that cross-view attention occurs exclusively between scale-matched tokens, ensuring semantic and texture correspondence.

Figure 3: Overview of the MACRO pipeline, including depth decomposition, image-space scale matching, and depth-masked attention during reference mixing.

This design is fully training-free and both computationally and memory efficient, as it requires a single pass through the backbone and leverages spatial masking instead of expensive iterative retraining.

Empirical Results and Ablations

MACRO’s evaluation leverages two rigorous close-up benchmarks: DL3DV-Closeup (283 pairs across 40 in-the-wild scenes) and MobileClose-10 (10 new scenes with 39 pairs). Both test conditions involve significant zoom factors (mean range $2.5\times$–$3.2\times$), with the ground-truth close-up views depicting objects at spatial scales never present in the training images.

MACRO achieves strict improvements over all baselines, including pure 3DGS, Mip-Splatting, generative models like SEVA, and reference-guided diffusion methods such as Difix and GSFixer. In particular, perceptual metrics (LPIPS, DreamSim, DINOv2) increase by $+5$–$29\%$ over best diffusion enhancement baselines and by $+29$–$k$0 over the 3DGS reference across both benchmarks, while maintaining parity on pixel-aligned measures such as PSNR.

Figure 4: MACRO is uniquely able to reconstruct sharp, semantically accurate details absent in prior approaches—see the restored text and figurine details.

Figure 5: On MobileClose-10, MACRO’s close-up renders are perceptually faithful, whereas others exhibit blur or incorrect transferred textures.

Additionally, ablation studies confirm that accuracy gains are predominantly due to multi-reference, multi-plane cropping and masking: reducing $k$1 (number of references), removing depth-mask, or discarding learned upsampling all result in degraded perceptual performance. The number of depth planes and references is optimized at $k$2, balancing accuracy and runtime.

Figure 6: Perceptual metrics and runtime for different values of $k$3; $k$4 delivers optimal balance before memory overhead becomes prohibitive.

Implications and Limitations

MACRO’s approach demonstrates that 3D geometric priors combined with 2D diffusion models can yield state-of-the-art close-up rendering under wide-to-tight zoom disparities without auxiliary retraining or changes to the generative backbone. The plug-and-play nature—applicable to any reference-conditioned single-step diffusion pipeline—makes MACRO suitable for real-world, data-constrained, or dynamic scenarios where specific close-up ground truth is unavailable.

Theoretically, this work highlights crucial limitations in latent variable models for cross-scale matching and suggests that explicit geometric reasoning remains essential for certain extrapolative vision tasks, especially where scale is nonuniform across the field of view.

Nonetheless, MACRO is not without constraints. Its performance is bounded by the upsampling capability of the super-resolution module, as high-frequency texture in scale-matched reference crops can be intrinsically unrecoverable from distant views. Misestimated 3DGS depth can propagate plane misalignments. Further, the method currently uses a fixed number of planes; adaptive local segmentation by depth complexity may provide even finer improvements.

Application and Prospects for Future Research

Practically, MACRO enables photorealistic zoom-in exploration for virtual production pipelines, digital twins, and immersive 3D visualization, as well as robust correction tools for generative asset re-editing. Its framework can be trivially combined with stronger backbone diffusion architectures and 3D priors, promising further performance increases. Additionally, synchronizing this multi-plane attention strategy with scale-equivariant backbone models and more sophisticated reference selection policies remains an important avenue. Exploiting monocular depth estimates in settings lacking explicit 3DGS could broaden its applicability.

Figure 7: Visualization: Masked attention avoids scale-mismatch artifacts, as each region attends only to aligned, scale-correct areas in reference images.

Conclusion

MACRO proposes a decisive solution to the persistent scale-mismatch challenge in close-up novel view synthesis from sparse training images, establishing that training-free, 3D geometry-aware attention mechanisms can robustly supervise reference-guided diffusion models for high-fidelity rendering across severe scale disparities (2607.03875). The method’s empirical superiority and modularity signal a critical step toward practical, training-free, task-agnostic, close-up rendering in neural graphics pipelines.