Solv3R: Iterative Solver for 3D Reconstructions

• Solv3R is a learned iterative solver for multi-view 3D reconstruction that uses a transformer update operator to refine reconstructions iteratively and progressively.
• Its iterative nature allows for the updating of existing views by switching progressively inaccurate state towards a more accurate solution, thus reinforcing reconstruction accuracy.
• Solv3R's architecture includes optimized transformations capable of creating transformational states and global attention for preserving context during compositional refinement.

Solv3R is a learned iterative solver for multi-view 3D reconstruction that represents the current geometric estimate as a latent multi-view state and repeatedly applies a stationary transformer update operator. Unlike feed-forward geometric foundation models, which generally predict geometry after a fixed computation, Solv3R is trained to preserve accurate reconstructions, correct inaccurate states, incorporate newly introduced views, and continue refining an existing image collection. Its iteration-count-independent dynamics support offline, incremental, relocalization, pipelined, and streaming reconstruction within a single model (Kwak et al., 2 Nov 2026).

1. Problem formulation and solver paradigm

Solv3R formulates multi-view reconstruction as an iterative solving problem. Given images $\mathcal{I}=\{I_i\}_{i=1}^{N}$, an image encoder produces an initial latent state,

$\mathbf{z}^{0}=E(\mathcal{I}).$

The solver then repeatedly applies the same learned operator,

$$\mathbf{z}^{k+1}=\mathcal{T}_{\theta}(\mathbf{z}^{k};\mathbf{z}^{0}),$$

where $\mathbf{z}^{k}$ is the reconstruction state after $k$ solver calls and $\mathbf{z}^{0}$ provides persistent image evidence and geometric constraints. The operator $\mathcal{T}_{\theta}$ is stationary: it does not receive the iteration index, the planned computation length, or the number of updates previously applied to any individual view.

The state can contain views with heterogeneous computational histories,

$$\mathbf{z}^{k}= \left( \mathbf{z}_1^{[h_1]},\ldots,\mathbf{z}_N^{[h_N]} \right),$$

where $h_i$ is the number of solver calls received by view $i$. Solv3R does not need to know these values. Consequently, an already refined view and a newly initialized view can be updated jointly by the same operator.

This design is analogous to a stationary iterative method for a linear system $\mathbf{z}^{0}=E(\mathcal{I}).$0,

$\mathbf{z}^{0}=E(\mathcal{I}).$1

in which the constraints remain fixed while the current solution estimate is progressively improved. In Solv3R, the images provide the constraints, latent view tokens represent the current state, and prediction heads decode cameras, depths, ray maps, and scene geometry.

The intended solver properties are:

1. Stability: an accurate reconstruction should remain approximately unchanged under further calls.
2. Correction: perturbed or inaccurate states should move toward a more accurate reconstruction.
3. Continuation: new views and constraints should be incorporated without recomputing existing views from scratch.

The principal distinction from ordinary weight-shared transformers is therefore not parameter sharing alone, but iteration-count independence combined with training for stable and corrective dynamics.

2. Architecture and state update

Solv3R is initialized from the pretrained looped transformer D. It retains an image encoder, per-view tokens, frame-wise attention, global multi-view attention, and camera, ray-map, and depth prediction heads. The principal changes are the replacement of step-conditioned gates with constant gates, repeated injection of the initial image encoding, RMS state normalization, and explicit training on heterogeneous solver states.

A single transformer block $\mathbf{z}^{0}=E(\mathcal{I}).$2 is reused at every call. It performs frame-wise attention within individual views and global attention across views, allowing image correspondences and shared geometry to constrain the latent state.

The implementation-level update is

$\mathbf{z}^{0}=E(\mathcal{I}).$3

followed by

$\mathbf{z}^{0}=E(\mathcal{I}).$4

Here, $\mathbf{z}^{0}=E(\mathcal{I}).$5 is the layer-normalized initial state without affine parameters, $\mathbf{z}^{0}=E(\mathcal{I}).$6 is a zero-initialized linear projection, and $\mathbf{z}^{0}=E(\mathcal{I}).$7 is a fixed target RMS measured from the final D state before training. The reported value is approximately $\mathbf{z}^{0}=E(\mathcal{I}).$8. A single scaling factor is applied to the entire state, preserving relative magnitudes between views and tokens.

D uses gates whose values depend on the refinement step. Solv3R replaces them with learned constants, so each transformer sub-block receives the same channel-wise scaling at every call. The gates are initialized by averaging D’s gate values over its 16 refinement steps.

Repeated initial-state injection serves as persistent problem data. The current state acts as the current solution estimate, while the initial encoding carries the image evidence that should remain available throughout refinement. The paper reports that removing this injection causes long-horizon collapse.

After each call, prediction heads decode camera poses, dense depth maps, ray maps, and derived scene geometry. Reconstruction can therefore be read after a fixed number of iterations, after adaptive convergence, or while the state is still being refined.

3. Training for stability, correction, and continuation

Stationarity alone does not guarantee useful solver dynamics. Solv3R is trained using states generated from different computational histories and is exposed to perturbations and partial resets.

Each training example begins with $\mathbf{z}^{0}=E(\mathcal{I}).$9 and receives a random number of non-differentiable pre-calls followed by differentiable calls. The main procedure uses 16–40 gradient-free pre-calls and 8 differentiable calls, with geometric supervision applied after the final differentiable call. Random-depth pre-calls prevent the model from associating a state with one fixed refinement stage.

With probability $$\mathbf{z}^{k+1}=\mathcal{T}_{\theta}(\mathbf{z}^{k};\mathbf{z}^{0}),$$0, the state is perturbed before differentiable updates. Two perturbation mechanisms are used. Gaussian state noise is applied according to

$$\mathbf{z}^{k+1}=\mathcal{T}_{\theta}(\mathbf{z}^{k};\mathbf{z}^{0}),$$1

with noise scale proportional to the state RMS and

$$\mathbf{z}^{k+1}=\mathcal{T}_{\theta}(\mathbf{z}^{k};\mathbf{z}^{0}),$$2

Alternatively, a random subset of up to half of the views is reset to $$\mathbf{z}^{k+1}=\mathcal{T}_{\theta}(\mathbf{z}^{k};\mathbf{z}^{0}),$$3. The resulting state combines refined views with newly initialized or inaccurate views, directly simulating incremental view insertion.

The geometric objective includes L1 losses for ray directions, scale-aligned ray-origin losses, geodesic relative-rotation losses, translation-direction losses, scale-aligned log-L1 depth loss, multi-scale depth-gradient loss, and consistency terms. The depth loss and multi-scale depth-gradient loss have weight $$\mathbf{z}^{k+1}=\mathcal{T}_{\theta}(\mathbf{z}^{k};\mathbf{z}^{0}),$$4, while the consistency terms have weight $$\mathbf{z}^{k+1}=\mathcal{T}_{\theta}(\mathbf{z}^{k};\mathbf{z}^{0}),$$5 each. The total objective is

$$\mathbf{z}^{k+1}=\mathcal{T}_{\theta}(\mathbf{z}^{k};\mathbf{z}^{0}),$$6

The consistency terms address the fact that separately scale-aligned ray, camera, and depth losses do not automatically enforce a common scale.

Solv3R is also distilled from D’s predictions after 16 calls using ray, log-depth, and camera losses. For datasets with sparse or noisy ground truth, the distillation loss is used alone.

Training comprises approximately 70,000 iterations with gradually increasing window size and call count, followed by approximately 20,000 iterations with the full procedure. Window sizes are sampled from 4–16 views with probability $$\mathbf{z}^{k+1}=\mathcal{T}_{\theta}(\mathbf{z}^{k};\mathbf{z}^{0}),$$7, 24–32 views with probability $$\mathbf{z}^{k+1}=\mathcal{T}_{\theta}(\mathbf{z}^{k};\mathbf{z}^{0}),$$8, and 48–64 views with probability $$\mathbf{z}^{k+1}=\mathcal{T}_{\theta}(\mathbf{z}^{k};\mathbf{z}^{0}),$$9. Only the shared transformer block, gates, and projection $\mathbf{z}^{k}$0 are fine-tuned; the image encoder and prediction heads remain frozen.

Optimization uses AdamW with learning rate $\mathbf{z}^{k}$1 for the transformer and $\mathbf{z}^{k}$2, learning rate $\mathbf{z}^{k}$3 for the gates, weight decay $\mathbf{z}^{k}$4, and EMA decay $\mathbf{z}^{k}$5. The model has 117M parameters and processes images whose longer side is 504 pixels. Training uses a mixture of indoor, object-centric, outdoor, driving, and synthetic datasets, including ScanNet++, ScanNet, CA-1M, Taskonomy, CO3D, NAVI, Skoltech3D, OmniObject3D, Objaverse renders, DL3DV, BlendedMVS, MegaDepth, Oxford Spires, Waymo, Argoverse 2, ParallelDomain-4D, Virtual KITTI 2, Infinigen, TartanAir, Hypersim, MVS-Synth, Spring, MatrixCity, and UnrealStereo4K.

4. Convergence, correction, and caching

Solv3R measures state convergence through the relative update norm

$\mathbf{z}^{k}$6

Inference can stop when $\mathbf{z}^{k}$7, with default threshold $\mathbf{z}^{k}$8. The standard adaptive configuration uses at least 4 and at most 64 solver calls.

On 148 eight-view windows from DTU, ETH3D, and ScanNet++, 144 windows reached $\mathbf{z}^{k}$9 within 64 calls. The stopping call occurred between 13 and 32 calls, with mean stopping call 17.5. Of the 144 stopped windows, 143 had camera rotation error below $k$0. Median rotation error was $k$1 at stopping and $k$2 at call 64. Tighter thresholds increased computation without consistent accuracy gains, whereas looser thresholds reduced computation but lowered pose AUC.

Solv3R also exhibits corrective behavior. When states were perturbed after eight calls and refined for eight further calls, the model recovered accurate camera poses from Gaussian noise and partial resets. Alternative initializations, including noisy states and states initialized from other views, produced median differences from the standard initialization of at most $k$3 after 128 calls.

The stationary representation enables solved views to be cached as transformer key-value context. A new query view can attend to cached map representations without recomputing the map views. In contrast, stacked transformers use layer-specific representations, and step-conditioned models require stage-specific caches. Solv3R can reuse one cache across solver calls, reducing cache storage by approximately $k$4 relative to stage-specific caching in D.

In map-based relocalization, training frames are reconstructed once, their key-value representations are cached, and a query frame attends to selected map views. With the same attended map views across methods, Solv3R used 2.86 MB per view and 51 ms per query, achieving recall of 72.8 on 7-Scenes and 24.9 on Cambridge under the reported thresholds. These results exceeded the listed VGGT, VGGT-$k$5, $k$6, and D configurations.

5. Reconstruction workflows

Offline reconstruction

For offline reconstruction, all images are encoded together, followed by 16 fixed solver calls or adaptive continuation. On DTU, ETH3D, 7-Scenes, ScanNet++, and nuScenes, Solv3R is evaluated against geometric foundation models including D, VGGT, $k$7, and Depth Anything 3-L.

At 24 views, the reported compute is 76.5 TFLOPs for fixed Solv3R and 91.3 TFLOPs for adaptive Solv3R. D requires 75.9 TFLOPs, VGGT 190 TFLOPs, $k$8 153.8 TFLOPs, and Depth Anything 3-L 71.4 TFLOPs. Solv3R has 117M parameters, compared with 1.26B for VGGT, 959M for $k$9, and 411M for Depth Anything 3-L.

Representative camera AUC results are:

Dataset	Fixed AUC@3	Fixed AUC@30	Adaptive AUC@3	Adaptive AUC@30
DTU	84.5	99.0	85.5	99.1
ETH3D	65.4	94.8	70.4	96.3
7-Scenes	13.8	81.8	14.2	81.9
ScanNet++	80.7	98.2	81.2	98.2
nuScenes	41.5	85.3	41.5	85.3

On Mill19 aerial scenes, Solv3R achieved AUC@3/AUC@30 values of 48.9/94.1 on Rubble and 22.5/89.6 on Building. Adaptive stopping generally improved metrics with approximately 3–4 additional calls on average.

Incremental reconstruction

In incremental reconstruction, an existing state is retained while new observations are added. A typical procedure solves four views for eight calls, introduces twelve new views initialized from their image encodings, and continues applying the same stationary operator to all views.

Old views are not permanently frozen. New observations can revise their cameras and geometry. On DL3DV and ScanNet++ evaluation windows, state reuse achieved comparable accuracy to restarting from scratch with fewer calls and lower runtime. At 64 views, continuing with eight calls per insertion achieved AUC@10 of 92.2, compared with 84.8 for restarting with eight calls. Restarting required 16 calls per insertion to reach 91.8.

Performance remained within approximately two AUC points of re-solving across capture, reverse, and random view orders, indicating that the solver is not tied to one particular arrival sequence.

Pipelined and streaming reconstruction

A sliding-window pipeline can advance after every solver call rather than waiting for convergence. Views within a window can therefore have different refinement levels. In the described 16-view pipeline, the delay between successive frame estimates is one call after the initial delay. Relative to repeatedly resolving the full window, the paper reports up to $\mathbf{z}^{0}$0 more frames per second in the core solver and approximately $\mathbf{z}^{0}$1 end-to-end throughput.

The streaming configuration maintains anchor frames, recent active frames, selected memory frames, and the incoming frame. The default L8 configuration uses

$\mathbf{z}^{0}$2

where $\mathbf{z}^{0}$3 is the number of anchors, $\mathbf{z}^{0}$4 the number of recent frames, $\mathbf{z}^{0}$5 the memory stride, $\mathbf{z}^{0}$6 the maximum memory size, and $\mathbf{z}^{0}$7 the number of calls per incoming frame.

The first eight frames are jointly solved for 16 calls. Incoming frames are warm-started from their own encodings and the solved offset of the previous frame. Recent frames are jointly refined, while anchors and memory frames are normally frozen and used as key-value context. Every four incoming frames, frozen views are tested for possible updates using the relative update criterion. If the update exceeds $\mathbf{z}^{0}$8, frozen views are reactivated and jointly refined; otherwise their cache is retained.

This mechanism combines stability, continuation, and correction: stable views avoid repeated computation, new views enter from existing states, and old views can be reactivated when later evidence reveals inconsistency.

6. Evaluation, limitations, and significance

Streaming evaluation covers Sintel, TUM dynamics, ScanNet, Bonn, KITTI, 7-Scenes, and NRGBD. The default L8 configuration reports 95 ms per frame and 5.0 GB GPU memory. Its reported camera ATE values are 0.157 on Sintel, 0.015 on TUM dynamics, and 0.077 on ScanNet. Reported depth AbsRel values are 0.322 on Sintel, 0.064 on Bonn, and 0.078 on KITTI. Chamfer distance is 0.018 m on both 7-Scenes and NRGBD.

The smaller S4 configuration uses 51 ms per frame and 2.1 GB memory, with lower accuracy but competitive results against CUT3R and TTT3R. The paper reports approximately one-third the memory of those cited comparisons. Once the memory queue reaches $\mathbf{z}^{0}$9, GPU memory remains bounded for long sequences. In the reported tests, offline VGGT ran out of memory at approximately 320 frames on an 80 GB H100, while StreamVGGT did so at approximately 365 frames.

Ablations show that each solver-specific design component has a distinct role. Step-conditioned gates work at their planned horizon but are sensitive to schedule length and fail under long continuation. Removing initial-state injection causes long-horizon collapse. Removing RMS normalization causes state magnitudes to drift. Removing random-depth pre-calls makes the model brittle to differing numbers of preceding calls. Removing perturbation training substantially degrades joining new views, correction from inaccurate states, and inference with frozen cached context.

The final model reports AUC@5 of 77.2 at 16 calls, 79.1 at 128 calls, and 78.8 at 1024 calls, together with stability error $\mathcal{T}_{\theta}$0, join error $\mathcal{T}_{\theta}$1, and cache error $\mathcal{T}_{\theta}$2. The results indicate that fixed-budget accuracy and reusable solver dynamics are related but distinct objectives.

Attention analysis found that three global-attention heads identify corresponding patches in adjacent views with 51–74% accuracy at call 16. Solv3R preserved these correspondences at later calls, with attention-map similarity above $\mathcal{T}_{\theta}$3 between calls 16 and 64. D’s correspondence quality fell below 3% by call 64 after its step schedule ended.

Solv3R does not guarantee globally correct reconstruction. A reported failure involved two groups of views with little overlap; the second group was placed at an incorrect position and orientation and remained in that configuration. Among the 144 windows meeting the stopping criterion, one stopped at a suboptimal solution. Other limitations include computational growth with the number of solver calls, the cost of global attention for large active windows, dependence on anchor and memory policies in streaming, and possible incorrect convergence when the latent state stabilizes around a wrong configuration.

The method is therefore not a classical numerical solver with a formal convergence guarantee. Its solver behavior is learned and empirical: accurate states tend to remain stable, perturbed states can be corrected, and mixed-progress states can be jointly refined. Its central contribution is the conversion of a geometric foundation model into a reusable iterative operator whose latent state can persist across computation, view insertion, caching, and streaming.