Lite Any Stereo: Efficient Zero-Shot Matching

• Lite Any Stereo is a lightweight feed-forward stereo depth estimation framework that uses a hybrid 3D-2D cost aggregation to predict dense disparity maps.
• It employs a three-stage training strategy combining synthetic supervision, self-distillation, and real-world fine-tuning to boost zero-shot generalization.
• The method achieves competitive accuracy and rapid inference (17–23 ms on GPUs) while operating at less than 1% of the computational cost of traditional models.

Lite Any Stereo is a feed-forward stereo depth estimation framework for predicting a dense disparity map from a rectified left-right stereo image pair under a dual constraint: the model is intended to be lightweight and fast enough for practical deployment, while also exhibiting strong zero-shot generalization on unseen real-world domains without target-domain fine-tuning. Introduced in "Lite Any Stereo: Efficient Zero-Shot Stereo Matching" (Jing et al., 20 Nov 2025), it was proposed against the prevailing view that efficient stereo models lack sufficient capacity for zero-shot applications. Its central claim is that an ultra-light stereo model can generalize strongly when a compact but expressive backbone, a hybrid cost aggregation module, and a three-stage million-scale training strategy are designed jointly.

1. Problem setting and design objective

Lite Any Stereo addresses stereo matching in the standard rectified setting: given a left and right RGB image, the model estimates dense disparity and, by geometry, depth. The paper focuses on a particularly difficult regime in stereo literature, namely the combination of deployment-oriented efficiency and strong zero-shot transfer. In the paper’s formulation, zero-shot ability means that a fixed trained model is evaluated on real-world benchmark datasets that were not used in training, with weights and parameters fixed and without target-domain fine-tuning (Jing et al., 20 Nov 2025).

The method is motivated by a tension that had become central in stereo research. High-accuracy stereo methods, especially large iterative models and methods augmented with monocular depth foundation priors, provide strong benchmark performance and increasingly good cross-domain robustness, but are computationally expensive. Efficient stereo methods reduce inference time and cost, often by replacing expensive 3D aggregation with lighter processing, but generally sacrifice accuracy and, more critically, tend to generalize poorly outside their training domain. Lite Any Stereo explicitly targets this gap by pursuing strong zero-shot stereo matching without using foundation depth priors in the inference model.

The paper frames its contribution around three simultaneous gaps: the gap between accurate but heavy stereo methods and efficient but weakly generalizing ones; the dependence of zero-shot stereo on large monocular priors; and the lack of a training recipe that enables efficient stereo models to transfer well across domains. This focus distinguishes Lite Any Stereo from work whose primary target is in-domain KITTI efficiency or absolute leaderboard accuracy.

2. End-to-end architecture

The Lite Any Stereo pipeline consists of four stages: feature extraction from the left and right images using a shared-weight lightweight backbone; correlation-based cost volume construction at quarter resolution; hybrid cost aggregation using a small amount of 3D processing followed by more extensive 2D processing; and disparity estimation at low resolution followed by convex upsampling to full resolution (Jing et al., 20 Nov 2025).

Feature extraction uses a conventional CNN backbone pretrained on ImageNet rather than DepthAnything or similar foundation priors. The backbone is built around a lightweight MobileNetV2-style design. Two weight-sharing feature extraction networks process the left and right images and produce multi-scale features at $1/4$, $1/8$, $1/16$, and $1/32$ resolution. All scales are then upsampled to $1/4$ using residual upsampling blocks, following LightStereo, so that matching is performed once at quarter resolution.

At quarter resolution, Lite Any Stereo constructs a correlation cost volume from the left and right feature maps $\mathbf{F}_L^{1/4}$ and $\mathbf{F}_R^{1/4}$. The paper gives the cost definition as

$$\mathbf{C}(d, h, w) = \frac{1}{N_c} \left\langle \mathbf{F}_{L}^{\frac{1}{4}}(h, w), \mathbf{F}_{R}^{\frac{1}{4}}(h, w-d) \right\rangle ,$$

where $d$ ranges over $[0, D_{\max}/4]$, $1/8$0 is the number of channels, and $1/8$1 denotes inner product. In implementation, the maximum disparity is

$1/8$2

so the quarter-resolution disparity search range is effectively up to $1/8$3.

After cost aggregation, disparity is estimated by softmax regression over the disparity axis: $1/8$4 where $1/8$5 is softmax over disparity. This produces a low-resolution disparity map, and convex upsampling then recovers the full-resolution output $1/8$6. The paper does not provide the explicit convex upsampling equation in the main text.

3. Hybrid 3D-2D cost aggregation

The central architectural contribution of Lite Any Stereo is its hybrid cost aggregation module. The paper argues that purely 2D cost aggregation lowers compute but weakens explicit reasoning across the disparity dimension, and that this harms zero-shot generalization because such aggregation has limited receptive field in spatial dimensions and insufficient disparity-dimension perception. To address this, Lite Any Stereo combines a 3D block $1/8$7, which provides disparity-dimension perception, with a 2D block $1/8$8, which enhances spatial-dimension perception (Jing et al., 20 Nov 2025).

Among several candidate arrangements, the paper reports that the best design is serial 3D followed by 2D: $1/8$9 Operationally, the 3D block uses multi-scale 3D convolutions, while the 2D block uses ConvNeXt layers. The design principle is not to split computation evenly between 3D and 2D processing, but to retain only a small proportion of 3D computation and assign most aggregation capacity to the cheaper 2D branch.

The ablations are unusually explicit about this design choice. Under a roughly fixed compute budget, the tested layouts produce the following zero-shot results on KITTI12 / KITTI15 / ETH3D / Middlebury with MACs in parentheses: 2D only, $1/16$0 $1/16$1; bilateral, $1/16$2 $1/16$3; 2D-3D, $1/16$4 $1/16$5; 3D-2D, $1/16$6 $1/16$7; and interleaved, $1/16$8 $1/16$9. The sharp ETH3D degradation of 2D-3D is one of the strongest empirical arguments for placing 3D processing first.

Further ablations define the final aggregation module more narrowly. For the 2D block, ConvNeXt gives $1/32$0 at $1/32$1, compared with MobileNetV2 at $1/32$2 and ConvNeXt v2 at $1/32$3. For the 3D kernel, $1/32$4 gives $1/32$5, while $1/32$6, $1/32$7, $1/32$8, and $1/32$9 all perform worse. For the 3D proportion, $1/4$0 gives $1/4$1 at $1/4$2, while $1/4$3 and $1/4$4 do not improve performance systematically under the same MAC budget. The paper’s conclusion is therefore specific: the useful regime is not “more 3D,” but a small and carefully placed 3D component followed by stronger 2D spatial aggregation.

4. Three-stage training strategy

A major part of Lite Any Stereo’s zero-shot behavior is attributed to its three-stage training strategy, which is designed to bridge the sim-to-real gap. The paper treats zero-shot transfer not only as an architectural problem but also as a training problem, and organizes supervision into synthetic annotated stereo data and realistic unannotated stereo data (Jing et al., 20 Nov 2025).

In Stage 1, the model is trained end-to-end on $1/4$5 annotated synthetic stereo pairs from SceneFlow $1/4$6, FallingThings $1/4$7, FSD $1/4$8, CREStereo $1/4$9, VKITTI2 $\mathbf{F}_L^{1/4}$0, TartanAir $\mathbf{F}_L^{1/4}$1, and Dynamic Replica $\mathbf{F}_L^{1/4}$2. IRS is excluded due to annotation quality issues, and Sintel, Spring, and InfinigenSV are excluded due to domain mismatch. This stage is done without data augmentation using the supervised disparity loss

$\mathbf{F}_L^{1/4}$3

In Stage 2, the model is further trained using self-distillation on synthetic data. Teacher and student have the same architecture and are both initialized from Stage 1. The teacher receives clean inputs; the student receives strongly perturbed inputs. In addition to the disparity loss, the paper introduces the feature alignment loss

$\mathbf{F}_L^{1/4}$4

where $\mathbf{F}_L^{1/4}$5 and $\mathbf{F}_L^{1/4}$6 are teacher and student feature vectors at pixel $\mathbf{F}_L^{1/4}$7. The simplest update strategy, fixed teacher weights, works best. On a $\mathbf{F}_L^{1/4}$8 synthetic subset, the reported reduced experiment gives no extra training strategy at KITTI12 $\mathbf{F}_L^{1/4}$9, KITTI15 $\mathbf{F}_R^{1/4}$0, ETH3D $\mathbf{F}_R^{1/4}$1, Middlebury $\mathbf{F}_R^{1/4}$2; direct data augmentation at $\mathbf{F}_R^{1/4}$3, $\mathbf{F}_R^{1/4}$4, $\mathbf{F}_R^{1/4}$5, $\mathbf{F}_R^{1/4}$6; and knowledge/self-distillation at $\mathbf{F}_R^{1/4}$7, $\mathbf{F}_R^{1/4}$8, $\mathbf{F}_R^{1/4}$9, $$\mathbf{C}(d, h, w) = \frac{1}{N_c} \left\langle \mathbf{F}_{L}^{\frac{1}{4}}(h, w), \mathbf{F}_{R}^{\frac{1}{4}}(h, w-d) \right\rangle ,$$0. The gains are therefore not uniform across all datasets in that reduced experiment.

In Stage 3, Lite Any Stereo is fine-tuned on roughly $$\mathbf{C}(d, h, w) = \frac{1}{N_c} \left\langle \mathbf{F}_{L}^{\frac{1}{4}}(h, w), \mathbf{F}_{R}^{\frac{1}{4}}(h, w-d) \right\rangle ,$$1 unlabeled real-world stereo samples using pseudo labels generated by a frozen accurate model. The real-world corpus includes Flickr1024 $$\mathbf{C}(d, h, w) = \frac{1}{N_c} \left\langle \mathbf{F}_{L}^{\frac{1}{4}}(h, w), \mathbf{F}_{R}^{\frac{1}{4}}(h, w-d) \right\rangle ,$$2, InStereo2K $$\mathbf{C}(d, h, w) = \frac{1}{N_c} \left\langle \mathbf{F}_{L}^{\frac{1}{4}}(h, w), \mathbf{F}_{R}^{\frac{1}{4}}(h, w-d) \right\rangle ,$$3, Holopix50K $$\mathbf{C}(d, h, w) = \frac{1}{N_c} \left\langle \mathbf{F}_{L}^{\frac{1}{4}}(h, w), \mathbf{F}_{R}^{\frac{1}{4}}(h, w-d) \right\rangle ,$$4, DrivingStereo $$\mathbf{C}(d, h, w) = \frac{1}{N_c} \left\langle \mathbf{F}_{L}^{\frac{1}{4}}(h, w), \mathbf{F}_{R}^{\frac{1}{4}}(h, w-d) \right\rangle ,$$5, SouthKenSV $$\mathbf{C}(d, h, w) = \frac{1}{N_c} \left\langle \mathbf{F}_{L}^{\frac{1}{4}}(h, w), \mathbf{F}_{R}^{\frac{1}{4}}(h, w-d) \right\rangle ,$$6, and UASOL $$\mathbf{C}(d, h, w) = \frac{1}{N_c} \left\langle \mathbf{F}_{L}^{\frac{1}{4}}(h, w), \mathbf{F}_{R}^{\frac{1}{4}}(h, w-d) \right\rangle ,$$7. The paper emphasizes that data quality matters more than sheer scale, noting Stereo4D, HRWSI, and SCOD as examples where limited resolution, poor rectification quality, or domain limitation are problematic. Self-distillation is not applied in Stage 3 because it gives no observable gains when training with pseudo labels.

Implementation details are explicit. The framework is PyTorch; the optimizer is AdamW; the learning-rate schedule is one-cycle; the peak learning rate is $$\mathbf{C}(d, h, w) = \frac{1}{N_c} \left\langle \mathbf{F}_{L}^{\frac{1}{4}}(h, w), \mathbf{F}_{R}^{\frac{1}{4}}(h, w-d) \right\rangle ,$$8; the training durations are $$\mathbf{C}(d, h, w) = \frac{1}{N_c} \left\langle \mathbf{F}_{L}^{\frac{1}{4}}(h, w), \mathbf{F}_{R}^{\frac{1}{4}}(h, w-d) \right\rangle ,$$9, $d$0, and $d$1 steps for Stages 1, 2, and 3; the total batch size is $d$2; hardware is NVIDIA A100 GPUs; and training uses random crop to $d$3, then fine-tuning on $d$4, with $d$5.

5. Quantitative performance, runtime, and ablation profile

Lite Any Stereo’s main zero-shot evaluation uses KITTI 2012, KITTI 2015, ETH3D, and Middlebury, with EPE for all datasets, Bad-$d$6 for Middlebury and ETH3D, and D1 for KITTI. In the paper’s “Efficient methods: Million-scale” setting, Lite Any Stereo reports KITTI 2012 D1 $d$7, EPE $d$8; KITTI 2015 D1 $d$9, EPE $[0, D_{\max}/4]$0; ETH3D Bad 1.0 $[0, D_{\max}/4]$1, EPE $[0, D_{\max}/4]$2; Middlebury Bad 2.0 $[0, D_{\max}/4]$3, EPE $[0, D_{\max}/4]$4; at 33 G MACs (Jing et al., 20 Nov 2025).

Benchmark	Metrics	Lite Any Stereo
KITTI 2012	D1 / EPE	3.04 / 0.79
KITTI 2015	D1 / EPE	3.87 / 0.99
ETH3D	Bad 1.0 / EPE	3.53 / 0.32
Middlebury	Bad 2.0 / EPE	7.51 / 0.94
Compute	MACs	33 G

In the same setting, LightStereo-M$[0, D_{\max}/4]$5 reports $[0, D_{\max}/4]$6, $[0, D_{\max}/4]$7, $[0, D_{\max}/4]$8, $[0, D_{\max}/4]$9 at $1/8$00; BANet-2D$1/8$01 reports $1/8$02, $1/8$03, $1/8$04, $1/8$05 at $1/8$06; and StereoAnything-L$1/8$07 reports $1/8$08, $1/8$09, $1/8$10, $1/8$11 at $1/8$12. The paper therefore characterizes Lite Any Stereo as the best result in every listed metric except ETH3D EPE, where StereoAnything-L is slightly better at $1/8$13 versus $1/8$14.

A particularly emphasized comparison is against Selective-IGEV, reported at $1/8$15 MACs with KITTI12 D1 $1/8$16, KITTI15 D1 $1/8$17, ETH3D Bad1.0 $1/8$18, and Middlebury Bad2.0 $1/8$19. Lite Any Stereo’s $1/8$20 versus $1/8$21 supports the paper’s statement that it requires less than 1% computational cost relative to Selective-IGEV. Against FoundationStereo at $1/8$22, the ratio is even smaller.

The method’s SceneFlow-only efficient zero-shot comparison is stricter because training uses only SceneFlow. In that setting, Lite Any Stereo reports KITTI12 D1 $1/8$23, EPE $1/8$24; KITTI15 D1 $1/8$25, EPE $1/8$26; ETH3D Bad1.0 $1/8$27, EPE $1/8$28; and Middlebury Bad2.0 $1/8$29, EPE $1/8$30, again at $1/8$31. The paper reports it as best on KITTI12, KITTI15 D1, and Middlebury metrics in that comparison.

DrivingStereo weather results are especially notable because Lite Any Stereo surpasses FoundationStereo on that benchmark. FoundationStereo reports rainy $1/8$32, sunny $1/8$33, foggy $1/8$34, cloudy $1/8$35, and overall $1/8$36. Lite Any Stereo reports rainy $1/8$37, sunny $1/8$38, foggy $1/8$39, cloudy $1/8$40, and overall $1/8$41, with 33 G MACs.

For deployment, the paper reports Lite Any Stereo as the fastest among compared methods on consistent local hardware: 21 ms on GTX 1080, 19 ms on RTX 4090, 23 ms on RTX A5000, and 17 ms on A100. It also explicitly states that for 2K inputs, the model requires only 2.5 GB GPU memory. On KITTI online leaderboard test sets, after fine-tuning on KITTI depth without using original KITTI annotations, the paper reports rank 1st among efficient methods at submission time, with KITTI 2012 $1/8$42-noc 1.09, $1/8$43-all 1.49, $1/8$44-noc 0.76, $1/8$45-all 1.04, EPE noc/all 0.4 / 0.5; and KITTI 2015 D1-bg 1.36, D1-fg $1/8$46, D1-all 1.71.

The stage-wise training ablation shows Stage 1 at KITTI12 $1/8$47, KITTI15 $1/8$48, ETH3D $1/8$49, Middlebury $1/8$50; Stage 2 at $1/8$51, $1/8$52, $1/8$53, $1/8$54; and Stage 3 at $1/8$55, $1/8$56, $1/8$57, $1/8$58. Stage 3 therefore provides major gains on KITTI and ETH3D, while Middlebury slightly worsens relative to Stage 2.

6. Relation to adjacent work, successors, and limitations

Lite Any Stereo sits within a broader line of lightweight stereo systems, but its position is specific. Relative to "LightStereo: Channel Boost Is All You Need for Efficient 2D Cost Aggregation" (Guo et al., 2024), Lite Any Stereo shares quarter-resolution correlation-volume matching and an emphasis on efficient aggregation, yet differs in using a hybrid 3D-2D cost aggregation module rather than a fully 2D aggregation path. LightStereo’s thesis is that efficient stereo can avoid full 4D cost-volume aggregation by treating disparity as channels in a 3D cost volume and strengthening disparity-channel interactions; Lite Any Stereo instead retains a small amount of 3D processing before 2D aggregation, and combines this with a training strategy explicitly designed for zero-shot transfer.

The direct successor, "Lite Any Stereo V2: Faster and Stronger Efficient Zero-Shot Stereo Matching" (Jing et al., 23 Jun 2026), revises both architecture and training. LAS2 replaces the previous LAS hybrid 3D-2D aggregation with a deployment-oriented 2D-only cost aggregation framework, uses FasterNet instead of MobileNetV2 on practical-latency grounds, and adds pseudo-label filtering and error clamping in the real-world distillation stage. The paper explicitly states that LAS2-M reduces error by 13.7% across the four real-world benchmarks relative to LAS while running 1.9× faster, and organizes the model family into LAS2-S, LAS2-M, LAS2-L, and the iterative LAS2-H.

"Litematch: Lightweight Zero-Shot Stereo Matching via Cost Volume Stabilization" (Khan et al., 30 Jun 2026) addresses a similar problem but through a different mechanism. Rather than hybrid 3D-2D aggregation and million-scale staged distillation, LiteMatch attributes much of stereo cost to repairing an unstable cost volume and proposes a dual feature encoder, a lightweight cost-volume refinement block, and the Cost Volume Consistency Loss (CVC-Loss) to encourage sharp and unimodal disparity probabilities. The shared theme is strong zero-shot generalization without expensive 3D convolutions, but the technical center of gravity differs: Lite Any Stereo emphasizes hybrid aggregation plus training curriculum, whereas LiteMatch emphasizes cost-volume stabilization.

"LeanStereo: A Leaner Backbone based Stereo Network" (Rahim et al., 24 Mar 2025) is relevant as a lighter 3D stereo baseline rather than a zero-shot stereo system. LeanStereo uses a lean two-branch backbone, an attention-based cost volume, and LogL1 loss, while remaining a true 3D cost-volume network with four initial 3D convolution layers followed by two stacked hourglass networks. A plausible implication is that LeanStereo and Lite Any Stereo occupy adjacent points on the efficient-stereo design spectrum: the former preserves a more classical 3D regularization regime, whereas the latter makes zero-shot generalization the primary target.

The limitations stated for Lite Any Stereo are concrete. The paper is explicit that it still trails depth-prior-based approaches overall, and that FoundationStereo remains much more accurate on some real-world benchmarks despite far higher compute. It identifies the limited availability of high-quality real-world stereo data as a major bottleneck, notes that performance on Middlebury can drop after Stage 3, and states that robustness on transparency and reflection remains improvable. The method also assumes rectified stereo images, a predefined maximum disparity of $1/8$59, and quarter-resolution matching during inference. These constraints make its reported results best understood as a specific accuracy-efficiency-generalization trade-off rather than a universal replacement for heavier prior-based stereo systems.