FAVS: Feature-Aware Adaptive View Stitching

• The paper introduces FAVS, a deterministic image stitching algorithm that leverages adaptive feature matching and geometric projection to create geometrically plausible pseudo-views.
• FAVS employs a four-stage hierarchical process—geometric transformation, feature matching, object alignment, and adaptive fusion—to optimize view synthesis without using depth or LiDAR data.
• The method significantly improves image quality, achieving enhancements in PSNR and SSIM while reducing artifacts compared to purely geometric projections.

Feature-Aware Adaptive View Stitching (FAVS) is a deterministic, image-level stitching procedure introduced as a core component of ArbiViewGen for controllable arbitrary-viewpoint camera image generation in autonomous driving. Its function is to synthesize a pseudo-view $I_{\text{target}}$ at an arbitrary target pose from multi-camera RGB images and calibrated camera poses, without requiring depth maps, LiDAR, or supervision from extrapolated viewpoints. In the ArbiViewGen pipeline, FAVS is upstream of the diffusion model: it constructs geometrically plausible stitched pseudo-images, and Cross-View Consistency Self-Supervised Learning (CVC-SSL) then uses those pseudo-views as conditions for reconstructing real camera images during training (Lan et al., 7 Aug 2025).

1. Definition, scope, and problem formulation

FAVS addresses a specific supervision gap in arbitrary-viewpoint generation: no ground-truth images exist for extrapolated viewpoints. Within ArbiViewGen, the response is to first generate pseudo-images at novel viewpoints with FAVS and then use them in a self-supervised loop. The method is designed to stitch six camera images in nuScenes into a single pseudo-view at an arbitrary target pose using only RGB inputs and camera poses (Lan et al., 7 Aug 2025).

The operational problem is not merely image mosaicing. FAVS is explicitly structured to handle pose-based geometric projection, residual misalignment caused by imperfect far-field assumptions and non-planar scenes, pose inaccuracies and calibration noise, object misalignment and occlusion issues, and fusion of overlapping regions so as to reduce seams, tearing, and black borders (Lan et al., 7 Aug 2025). In this sense, the term “feature-aware” refers to the use of feature correspondences, clustering of matched keypoints, and confidence-driven selection of reliable regions; the term “adaptive” refers to hierarchical refinement and adaptive weighting during fusion rather than to a learned end-to-end neural seam predictor.

A common misconception is that FAVS is itself a generative model. It is not. The paper states that FAVS is fundamentally an image-level stitching algorithm and not a learned network. It produces a stitched pseudo-view, whereas the downstream diffusion model in CVC-SSL performs reconstruction and completion (Lan et al., 7 Aug 2025).

2. Four-stage hierarchical procedure

The paper defines FAVS as a four-stage hierarchical algorithm comprising geometric transformation foundation, feature matching optimization, object alignment fine-tuning, and adaptive fusion generation (Lan et al., 7 Aug 2025).

Stage	Main operation	Purpose
1	Geometric Transformation Foundation	Coarse correspondences from camera intrinsics and rotations
2	Feature Matching Optimization	Refine coarse alignment with SIFT, FLANN, Lowe’s ratio test, and RANSAC
3	Object Alignment Fine-tuning	Cluster matched keypoints with DBSCAN and estimate translation correction
4	Adaptive Fusion Generation	Warp all selected views and blend them with confidence-based weights

In Stage 1, each source image is mapped toward the target viewpoint by a homography derived from camera intrinsics and rotations under a far-field approximation. This provides a coarse warp matrix for each source-target pair (Lan et al., 7 Aug 2025).

In Stage 2, the method extracts SIFT keypoints and descriptors, matches them with FLANN, filters them by Lowe’s ratio test, and estimates a feature-based homography with RANSAC. Reliability is then assessed using the number of good matches, a homography validity check, and consistency relative to the geometry-derived homography. The outcome is a refined transformation $H_{\text{base}}$ (Lan et al., 7 Aug 2025).

In Stage 3, FAVS performs what the paper calls “object detection and alignment,” but it also states that this is not conventional semantic detection. Instead, DBSCAN clusters matched keypoints into spatial groups that often correspond to physical objects such as vehicles, pedestrians, or buildings. Cluster centers in source and target images are used to estimate a weighted translation correction $\Delta T$, which is applied as a global adjustment on top of $H_{\text{base}}$ to obtain $H_{\text{aligned}}$ (Lan et al., 7 Aug 2025).

In Stage 4, all selected source images are warped into the target view and fused adaptively with per-pixel weights that combine distance, gradient, global quality, and primary-camera preference. The result is the final pseudo-view $I_{\text{target}}$ (Lan et al., 7 Aug 2025).

3. Geometric and feature-level formulation

The geometric foundation of FAVS uses a rotation-based homography. Given source-camera intrinsic matrix $\mathbf{K}_1$ and rotation $\mathbf{R}_1$, and target-camera intrinsic matrix $\mathbf{K}_2$ and rotation $\mathbf{R}_2$, the geometric homography is

$H_{\text{base}}$0

The intrinsic matrix is written as

$H_{\text{base}}$1

and the rotation decomposition is

$H_{\text{base}}$2

The far-field approximation is invoked when

$H_{\text{base}}$3

so that translation can be neglected and the homography simplifies to

$H_{\text{base}}$4

The paper contrasts this with the planar-scene homography

$H_{\text{base}}$5

which would require plane normal $H_{\text{base}}$6 and distance $H_{\text{base}}$7, and is therefore avoided (Lan et al., 7 Aug 2025).

Feature refinement begins with SIFT extraction:

$H_{\text{base}}$8

Matches are filtered by Lowe’s ratio test,

$H_{\text{base}}$9

and a robust feature homography is estimated as

$\Delta T$0

The stage then combines geometry-based and feature-based transforms according to a consistency test:

$\Delta T$1

This consistency-driven combination is central to FAVS. It prevents the system from relying exclusively on feature matches in sparse-feature or low-texture settings, while also avoiding blind trust in pose-only projection when calibration error or far-field mismatch is non-negligible (Lan et al., 7 Aug 2025).

4. High-confidence regions, clustering, and adaptive fusion

FAVS identifies high-confidence matching regions through clustering analysis. The matched keypoints are grouped by DBSCAN,

$\Delta T$2

and each cluster is represented as

$\Delta T$3

For cluster $\Delta T$4 with $\Delta T$5 correspondences, the source and target centers are

$\Delta T$6

The translation correction is estimated by minimizing

$\Delta T$7

with cluster weights

$\Delta T$8

The final alignment homography is then

$\Delta T$9

The use of $H_{\text{base}}$0 and $H_{\text{base}}$1 means that not all matched regions influence the final adjustment equally. High-confidence clusters dominate the optimization, which is precisely how the method operationalizes “feature-aware” behavior (Lan et al., 7 Aug 2025).

Adaptive fusion proceeds after warping each source image:

$H_{\text{base}}$2

The per-pixel fusion weight for warped image $H_{\text{base}}$3 is

$H_{\text{base}}$4

The distance and gradient terms are

$H_{\text{base}}$5

The final pseudo-view is the weighted average

$H_{\text{base}}$6

This fusion rule gives preference to pixels farther from unreliable boundaries, deemphasizes high-gradient locations where seam artifacts are most visible, incorporates global matching quality, and can privilege a designated primary camera (Lan et al., 7 Aug 2025).

5. Empirical behavior, assumptions, and common misconceptions

The quantitative ablation reported for ArbiViewGen isolates the contribution of FAVS relative to a purely geometric baseline and the full diffusion-based system (Lan et al., 7 Aug 2025).

Method	Sparse-PSNR↑	Sparse-SSIM↑
Geometric	9.3167	0.8339
FAVS	11.8707	0.8813
Ours	14.2335	0.9691

The same ablation reports Sparse-MAE and Sparse-RMSE values of $H_{\text{base}}$7 and $H_{\text{base}}$8 for Geometric, $H_{\text{base}}$9 and $H_{\text{aligned}}$0 for FAVS, and $H_{\text{aligned}}$1 and $H_{\text{aligned}}$2 for the full method. The paper further states that FAVS alone significantly improves over pure geometric projection, with PSNR $H_{\text{aligned}}$3 dB, SSIM $H_{\text{aligned}}$4, MAE $H_{\text{aligned}}$5, and RMSE $H_{\text{aligned}}$6 (Lan et al., 7 Aug 2025).

Qualitatively, the geometric baseline preserves rigid structure but shows tearing, seams, and large black borders in extrapolated directions. FAVS improves alignment by combining pose-based and feature-based correspondences and yields better alignment of cars, lane markers, and buildings across overlapping views, but it still exhibits discontinuities and missing regions where no source camera covers the target view. The full ArbiViewGen model then uses FAVS as a coarse geometric prior that the diffusion model refines and completes (Lan et al., 7 Aug 2025).

Several assumptions define the regime in which FAVS is intended to work. The method assumes far-field behavior, accurate camera intrinsics and rotations, and a scene that is quasi-static or rigid at frame time. It does not estimate depth or planar scene parameters explicitly and therefore cannot directly resolve strong parallax induced by large depth variation. The paper also notes that the overall framework still faces limitations in capturing fine-grained structural details in highly dynamic environments, particularly under sparse geometric constraints (Lan et al., 7 Aug 2025).

Two misconceptions recur in discussions of FAVS. First, the “object alignment” stage is not a semantic detector in the conventional sense; the paper explicitly states that DBSCAN groups matched keypoints into clusters that often correspond to physical objects, but this is not conventional semantic detection. Second, FAVS is not intended to hallucinate unseen regions. Missing areas remain missing when no camera contributes source content; the generative completion is delegated to the downstream diffusion model (Lan et al., 7 Aug 2025).

6. Relation to adjacent stitching paradigms

Although the term FAVS is explicitly introduced in ArbiViewGen, several earlier and later stitching systems instantiate related principles. A hardware-oriented panoramic video stitcher based on Harris corners used adaptive feature point extraction, overlap-restricted matching, and orientation-normalized descriptors on a Xilinx Spartan-6 FPGA. That system was designed for real-time video from a single CCD camera under small horizontal motion, not for arbitrary multi-camera pseudo-view synthesis, but it already exemplified adaptive feature selection and feature-driven alignment under resource constraints (Sindhu, 2016). This suggests an earlier feature-aware and adaptive stitching logic at the level of feature extraction and matching, even though the FAVS name was not used.

In a different application domain, esophageal video stitching combined LoFTR, SIFT, and ORB in a feature filtering pool and introduced Density-Weighted Homography Optimization (DWHO) to prevent dense local feature clusters from dominating global alignment. That framework paired domain-specific geometric unfolding with multi-feature pooling and density-aware weighting, which closely parallels FAVS’s reliance on heterogeneous evidence and confidence-driven refinement, albeit in cylindrical endoscopic video rather than surround-view driving imagery (Li et al., 2024).

SemanticStitch moved the adaptive component from feature matching to seam selection by using semantic priors of salient foreground objects to learn soft blending masks $H_{\text{aligned}}$7 and $H_{\text{aligned}}$8, with object completeness, exclusivity, and smoothness losses. Its emphasis is different from FAVS: semantics shape seam placement rather than coarse geometric alignment, and the method is framed for pairwise image stitching after alignment rather than pseudo-view synthesis from calibrated multi-camera rigs (Jin et al., 15 Nov 2025).

Unsupervised and supervised deep stitching systems provide a second axis of contrast. Unsupervised Deep Image Stitching learned coarse alignment and feature-to-pixel reconstruction without stitched ground truth, using overlap-aware losses in a stitching-domain representation (Nie et al., 2021). “Learning Edge-Preserved Image Stitching from Large-Baseline Deep Homography” combined a large-baseline deep homography module with an edge-preserved deformation module to remove ghosting while preserving structural continuity (Nie et al., 2020). Relative to those models, FAVS remains a deterministic, non-neural image-level pipeline whose outputs serve as conditions for a diffusion model rather than as a final stitched result (Lan et al., 7 Aug 2025).

A later multimodal direction is represented by UniStitch, which explicitly unifies semantic and geometric features through a Neural Point Transformer and an Adaptive Mixture of Experts. UniStitch predicts global homography and local TPS-based residual warps from fused multimodal features, and its AMoE module dynamically shifts emphasis toward more reliable semantic or geometric evidence (Mei et al., 11 Mar 2026). A plausible implication is that such multimodal fusion mechanisms could extend the feature stage of FAVS beyond its current pose-plus-SIFT-plus-DBSCAN formulation, but that extension is not part of the FAVS definition in ArbiViewGen.

Within this broader landscape, FAVS occupies a specific position: it is a hierarchical, pose-aware, feature-refined, cluster-guided, adaptively fused view-stitching procedure that produces pseudo supervisory signals for arbitrary-viewpoint generation. Its distinct contribution is not generic panorama creation, semantic seam carving, or end-to-end learned warping in isolation, but the construction of geometrically plausible stitched pseudo-views from RGB images and poses alone, so that cross-view consistency can be learned without ground truth from unseen viewpoints (Lan et al., 7 Aug 2025).