MuRF: Unlocking the Multi-Scale Potential of Vision Foundation Models

Abstract: Vision Foundation Models (VFMs) have become the cornerstone of modern computer vision, offering robust representations across a wide array of tasks. While recent advances allow these models to handle varying input sizes during training, inference typically remains restricted to a single, fixed scale. This prevalent single-scale paradigm overlooks a fundamental property of visual perception: varying resolutions offer complementary inductive biases, where low-resolution views excel at global semantic recognition and high-resolution views are essential for fine-grained refinement. In this work, we propose Multi-Resolution Fusion (MuRF), a simple yet universally effective strategy to harness this synergy at inference time. Instead of relying on a single view, MuRF constructs a unified representation by processing an image at multiple resolutions through a frozen VFM and fusing the resulting features. The universality of MuRF is its most compelling attribute. It is not tied to a specific architecture, serving instead as a fundamental, training-free enhancement to visual representation. We empirically validate this by applying MuRF to a broad spectrum of critical computer vision tasks across multiple distinct VFM families - primarily DINOv2, but also demonstrating successful generalization to contrastive models like SigLIP2.

• The paper presents MuRF, a training-free multi-resolution fusion method that constructs input pyramids and concatenates features to blend global recognition with fine detail refinement.
• Its approach yields improved performance on benchmarks, achieving 47.4% mIoU on ADE20K and a 0.368 RMSE on NYU Depth V2 compared to single-scale baselines.
• The universal fusion pipeline supports diverse tasks, including segmentation, depth estimation, and anomaly detection, without requiring backbone retraining.

Multi-Resolution Feature Fusion for Vision Foundation Models: An Expert Analysis of "MuRF: Unlocking the Multi-Scale Potential of Vision Foundation Models" (2603.25744)

Introduction

Multi-resolution analysis is a foundational theme in computer vision, bridging the gap between global semantic recognition and fine-grained localization. Modern Vision Foundation Models (VFMs), particularly architectures such as DINOv2, are trained for increased adaptability with respect to input image resolution. Yet, conventional inference paradigms rigidly enforce single-scale input, which fails to exploit the full representational spectrum learnable from various spatial scales. The "MuRF" paper introduces a strategy to unlock this latent capacity by constructing and fusing feature pyramids at inference time, thereby synthesizing representations that are robust across tasks and architectures.

Recognition vs. Refinement: The Synergistic Multi-Scale Phenomenon

The central premise of MuRF derives from the scale-dependent inductive biases of VFMs: lower resolutions facilitate recognition via globally coherent feature spaces, whereas higher resolutions accentuate boundary sharpness required for refinement. Figure 1 from the paper characterizes this dynamic—low-resolution feature maps furnish holistic object segmentation but lack detail granularity, while high-resolution maps produce refined edges at the cost of increased interior noise and incomplete coverage.

Figure 1: Illustration of the recognition-refinement dichotomy as a function of image scaling in VFM feature maps.

The MuRF hypothesis contends that optimal representation for downstream dense tasks is achievable only by aggregating these scale-specific strengths.

The MuRF Architecture

MuRF operationalizes its central thesis through a training-free, universal pipeline as depicted in Figure 2. The method comprises the following sequence:

1. Creation of an input pyramid by resizing the source image to multiple, strategically selected resolutions.
2. Each resolution is independently encoded by a frozen VFM (e.g., DINOv2), generating distinct spatial feature maps.
3. These multi-scale features are upsampled/interpolated to a unified spatial grid and concatenated along the channel dimension.
4. The resulting multi-resolution tensor feeds lightweight, task-specific heads (e.g., for segmentation, depth estimation, VQA), wherein only the heads are trained while the backbone remains frozen.

   Figure 2: Complete MuRF pipeline—multi-resolution inputs processed by a frozen encoder, fused, and routed to diverse downstream heads.

Rigorously, concatenation is preferred over additive or pooling fusions to preserve the full complement of scale-specific activations, avoiding destructive interference, and enabling the task head to adaptively select scale-appropriate features.

Empirical Results Across Vision Benchmarks

Semantic Segmentation

The MuRF paradigm yields robust improvements on ADE20K and PASCAL VOC benchmarks. Notably, with DINOv2-ViT-B/14 as the backbone, MuRF achieves 47.4% (ADE20K) and 83.1% (VOC) mIoU, surpassing all single-scale configurations. Qualitative segmentation outputs reveal more accurate boundaries and fewer interior artifacts when using the multi-scale approach.

Figure 3: Qualitative improvement in ADE20K segmentation through multi-scale feature fusion.

Depth Estimation

For monocular depth prediction (NYU Depth V2, SUN RGB-D), MuRF delivers lower RMSE scores than mono-scale or multi-layer alternatives (e.g., 0.368 RMSE on NYU Depth V2 with MuRF versus 0.394 using the best single-scale baseline). The performance scalability with increasing resolutions is consistent, demonstrating multi-scale fusion's superiority across both in-domain and out-of-domain generalization.

Figure 4: MuRF produces sharper and globally consistent depth maps on NYU Depth V2 compared to single-scale VFM inference.

Multimodal LLM Integration (VQA)

MuRF's multi-scale context enriches the visual grounding of large multimodal LLMs without increasing inference sequence length (by concatenating patches channel-wise at constant spatial granularity). Substantial improvements are observed on VQA suites (e.g., MME, POPE, MMB), and performance gains transfer to multiple encoder families (e.g., DINOv2, SigLIP2).

Unsupervised Anomaly Detection

On the MVTec AD 2 benchmark, MuRF attains state-of-the-art pixel-level AU-PRO scores (62.3 on TEST_priv,mix), outperforming training-free and supervised baselines. The fusion of coarse and fine scale features is directly evidenced in qualitative anomaly masks—low-resolution views yield robust, coarse localization, while high-resolution views delineate sharp boundaries.

Figure 5: Anomaly detection maps and feature PCA. Fusing scales produces robust detection and precise boundaries; single scales trade off between coverage and sharpness.

Ablation Studies: Resolution, Fusion Depth, and Model Choice

Empirical ablations elucidate that:

• Increasing fusion depth (more scales) monotonically enhances accuracy across dense tasks, albeit with diminishing returns.
• Low-resolution views inject essential global context, which cannot be compensated by merely stacking higher resolutions.
• Multi-resolution fusion is complementary to multi-layer feature extraction; combined, these yield the best zero-shot transfer performance.
• Application to alternate backbones (SigLIP2) and head architectures (e.g., LLaVA for MLLMs) produces analogous gains, underscoring universality.

Theoretical and Practical Implications

By reviving explicit input-level pyramids—now realized via frozen, giant transformers—MuRF upends the notion that modern FPNs or high-resolution upsamplings are sufficient for scale-robust representations. It validates that VFM scale bias is not mitigated by flexible training alone but must be systematically exploited at inference. This principle has several implications:

• Multi-resolution fusion—deployed as a wrapper around any frozen VFM—provides a simple axis of improvement for dense, open-vocabulary, and multimodal systems without costly backbone retraining.
• Practices such as patch tiling and feature upsampling in MLLMs, which disrupt semantic continuity or fail to add cross-scale richness, are empirically inferior to MuRF-style channel fusion.
• The diminishing gains with increasing scales suggest that careful selection of scale triplets can provide most of the benefit with limited computational overhead.
• Unified, scale-robust visual feature engineering can boost the data efficiency and accuracy of models used in high-stakes environments (e.g., autonomous robots, industrial inspection, high-resolution creative tools).

Conclusion

The MuRF method substantiates multi-resolution feature fusion as a foundational tool for modern visual representation, applicable across vision and vision-language tasks. Its simplicity (no backbone training, universal interface) and consistent, strong empirical results recommend it as a default inference protocol for future frozen VFM deployments. Continued research may explore dynamic, content-adaptive scale selection or synergistic multi-resolution/multi-layer architectures to further push generalization boundaries and performance ceilings.