StereoAdapter: Adaptation in Stereo Vision
- StereoAdapter is a research label that unifies techniques for adapting stereo systems using optical, algorithmic, and PEFT-based methods.
- These approaches range from smartphone catadioptric rigs and online/federated learning adaptations to geometry-aware cross-view fusion in super-resolution.
- The diverse strategies enable robust stereo matching, depth estimation, and 3D reconstruction in dynamic, domain-shifted, and underwater environments.
Searching arXiv for papers related to "StereoAdapter" and closely named variants to ground the article in current literature. StereoAdapter is a name that has been used for multiple, technically distinct research artifacts in stereo vision and related imaging systems. In the literature, it denotes at least four major lines of work: a planar-mirror smartphone catadioptric stereo rig for 3D reconstruction (Bartol et al., 2021); a meta-learning and online-adaptation framework for deep stereo, later extended to federated adaptation and geometry-aware domain adaptation (Tonioni et al., 2019, Poggi et al., 2024, Sakuma et al., 2021); a parameter-efficient stereo image super-resolution method based on adapter modules inserted into a frozen single-image transformer (Zhou et al., 2024); and an underwater stereo depth estimation framework that combines LoRA-adapted monocular foundation encoders with recurrent stereo refinement, followed by a second generation using a selective state-space updater (Wu et al., 19 Sep 2025, Ren et al., 18 Feb 2026). The shared label therefore refers not to a single canonical architecture, but to a family of adapter-oriented or adaptation-oriented approaches that address stereo acquisition, stereo matching, stereo super-resolution, and underwater stereo depth estimation.
1. Terminological scope and disambiguation
The term “StereoAdapter” first appears, in the supplied corpus, in “Learning to Adapt for Stereo,” where it refers to a framework that enables deep stereo methods to continuously adapt to new target domains in an unsupervised manner (Tonioni et al., 2019). In that work, the emphasis is on online parameter adaptation of a stereo network through a learned unsupervised objective weighting mechanism.
A different use appears in “Catadioptric Stereo on a Smartphone,” where StereoAdapter is a compact, low-cost catadioptric rig that converts the front- and back-facing cameras of an off-the-shelf smartphone into a calibrated stereo pair (Bartol et al., 2021). Here the term denotes a physical optical-mechanical device rather than a learning algorithm.
The name is later used again in distinct algorithmic contexts. In “ASTeISR,” “stereo adapters” are small cross-view fusion modules and “spatial adapters” are local refinement modules inserted into a frozen single-image super-resolution transformer for stereo image super-resolution (Zhou et al., 2024). In “StereoAdapter: Adapting Stereo Depth Estimation to Underwater Scenes,” the term refers to a self-supervised underwater stereo depth estimation system with LoRA-adapted monocular priors and a recurrent stereo refinement module (Wu et al., 19 Sep 2025). “StereoAdapter-2” then replaces the ConvGRU updater with a ConvSS2D operator based on selective state-space models and augments the training data with UW-StereoDepth-80K (Ren et al., 18 Feb 2026).
This multiplicity of meanings suggests that “StereoAdapter” functions as a recurrent research label for stereo-specific adaptation mechanisms. A plausible implication is that any technical discussion must distinguish between physical stereo conversion, online domain adaptation, parameter-efficient transfer, and underwater stereo estimation, because their assumptions, objectives, and evaluation protocols differ substantially.
2. Smartphone catadioptric StereoAdapter
In “Catadioptric Stereo on a Smartphone,” StereoAdapter is implemented as a 3D-printed mount (“mask”) and two planar mirrors, each , which steer light from the scene into the front and rear smartphone cameras (Bartol et al., 2021). The two mirrors are held at an identical tilt angle with respect to the phone’s body, one in front of the rear camera and one in front of the selfie camera. By reflection, each real camera produces an image of a virtual camera whose optical center lies behind the mirror. The effective baseline is , where is the distance from the phone’s camera lens to the mirror plane.
The optical design is expressed in terms of mirror side length , mirror tilt , camera-to-mirror distance , and mirror height . For a single virtual camera, the field of view is
For , 0, and 1, the paper reports 2, which is approximately 3 of the stock single-camera field of view 4 (Bartol et al., 2021).
Because the virtual cameras tilt toward each other, the inner convergence angle is
5
For a full 6-tall person, the minimal camera-object distance is reported as
7
with 8 and 9. Only about 0 of the original vertical field of view is shared by both virtual cameras (Bartol et al., 2021). Mechanically, the mask is printed in PLA, includes adjustable slots for mirror angle and tuning for different phone models, and uses a screw-driven slider to vary 1. The mirrors sit in 2-precision grooves.
Calibration uses Zhang’s chessboard method in MATLAB. Each raw mirror image is flipped and rotated so that both virtual images share the same upright convention and optical axis direction. The calibration procedure uses 14 stereo pairs of a 3 chessboard with 4 squares, with autofocus and auto-exposure locked (Bartol et al., 2021). The virtual stereo transform 5 yields an estimated baseline 6, matching the design. Reprojection error is minimized with nonlinear Levenberg-Marquardt, and the final mean reprojection error is approximately 7 pixels. Distances between reconstructed 3D points on the calibration planes and their true plane positions are on average 8–9 along 0 (Bartol et al., 2021).
The reconstruction pipeline runs each virtual image through OpenPose, producing up to 25 human-joint detections. Since the keypoints are already semantically matched across views, no additional stereo matching is required. Triangulation uses either the linear system
1
or the horizontal-stereo depth-from-disparity form
2
On a standing subject at approximately 3, six anthropometric segments were evaluated against manual tape measurements. The reconstructed lengths were 4 cm, manual lengths were 5 cm, and the corresponding errors were 6 cm, with mean absolute error approximately 7 (Bartol et al., 2021).
The paper attributes the largest deviations to keypoint mis-detections at elbows and knees and to the relatively small baseline of approximately 8, which causes depth sensitivity issues. Proposed improvements include larger mirrors to increase baseline, asymmetric mirror tilts or variable-focus auxiliary optics to enlarge stereo overlap, more varied chessboard views and sub-pixel corner detection for calibration, and fusion with model-based pose priors or temporal filtering for more stable 3D skeletons (Bartol et al., 2021).
3. StereoAdapter as online and federated adaptation for deep stereo
In “Learning to Adapt for Stereo,” StereoAdapter denotes a “learning-to-adapt” framework for robust stereo matching under environmental variation (Tonioni et al., 2019). The base network is DispNet-Corr1D, written as
9
with 0 the network parameters. The central objective is to learn an initialization 1 that is especially amenable to rapid unsupervised online adaptation.
The meta-learning setup uses short stereo sequences 2. For each sequence, 3 consecutive inner-loop steps minimize an unsupervised loss 4, while the outer loop evaluates a supervised loss 5 on the next frame: 6 The outer-loop objective minimizes
7
The unsupervised loss combines photometric reprojection, edge-aware smoothness, and left-right consistency: 8 The photometric term uses SSIM and 9, smoothness is edge-aware, and left-right consistency uses predictions from both view orderings (Tonioni et al., 2019).
A learned confidence measure is then appended. A small 3-layer fully convolutional network 0 takes the pixel-wise error map 1 derived from 2 and outputs a confidence mask 3, normalized so that 4. The weighted unsupervised loss becomes
5
At test time, no ground truth is used. The model updates online from each incoming stereo pair by computing 6 and taking a gradient step with respect to the weighted unsupervised objective (Tonioni et al., 2019).
The reported training recipe includes supervised pretraining on FlyingThings3D, meta-training on synthetic sequences such as Carla or Synthia with 7, 8, 9, and online test-time adaptation with momentum 0 and learning rate 1 (Tonioni et al., 2019). On synthetic-to-real transfer, the best reported variant, L2A + weighted adaptation, achieves D1-all approximately 2 and EPE approximately 3, compared with 4 and 5 for supervised learning plus unsupervised online adaptation and 6 and 7 for static supervised learning on Carla 8 KITTI. The paper states that this matches the “ideal” fine-tuned model at 9 D1-all and 0 EPE (Tonioni et al., 2019).
This adaptation-oriented interpretation of StereoAdapter is extended in “Federated Online Adaptation for Deep Stereo,” which introduces a distributed framework based on federated learning (Poggi et al., 2024). There, a pre-trained stereo model with parameters 1 is adapted collaboratively across clients observing non-stationary stereo streams 2. The active set 3 performs local adaptation, while the listening set 4 receives aggregated weights without adapting locally. The server uses FedAvg: 5
The underlying network is MADNet 2, an encoder-decoder stack of five scales that replaces original radius-6 correlation volumes with all-pairs correlation from RAFT-Stereo, samples a fixed number 7 of channels at each scale, and removes the heavy context network to keep runtime low (Poggi et al., 2024). This supports both full adaptation and modular adaptation, the latter updating only one decoder block per step.
The paper reports, on KITTI single-device adaptation, that MADNet 2 without adaptation reaches D1-all 8 and EPE 9, while FULL (photo) adaptation reaches D1-all 0, EPE 1, and MAD (photo) reaches D1-all 2, EPE 3. FULL++ (proxy) and MAD++ (proxy) obtain D1-all 4, EPE 5, and D1-all 6, EPE 7, respectively (Poggi et al., 2024). In the federated setting with three active clients and update interval 8, FedFULL yields on KITTI D1-all values 9 across City, Residential, Campus0, and Road, whereas FedMAD gives 1. Communication is 2 to server and 3 to client for FedFULL, versus 4 and 5 for FedMAD (Poggi et al., 2024).
These results frame StereoAdapter not simply as a model family, but as a deployment paradigm in which stereo systems are designed for unsupervised adaptation during operation, including collaborative adaptation across devices.
4. Geometry-aware adaptation and cross-view attention
A further development appears in “Geometry-Aware Unsupervised Domain Adaptation for Stereo Matching,” summarized in the provided data as another StereoAdapter line (Sakuma et al., 2021). The problem addressed is that image-to-image translation applied independently to the left and right views of a stereo pair can destroy epipolar geometry, leading to mismatched translated pairs and degraded downstream disparity learning.
The proposed mechanism is Stereoscopic Cross Attention (SCA), inserted into a translation network 6 based on TSIT. At a given scale, each view has a feature map 7 and a reprojected 3D point cloud 8. Query and key projections are computed from the concatenated representation 9, while the value is 00. Cross-attention is constrained to the horizontal epipolar line over disparity candidates 01: 02
03
The same construction is applied in the reverse direction (Sakuma et al., 2021).
The overall pipeline has three stages: source-supervised pretraining of a stereo matcher 04, pixel-level adaptation through 05 and a discriminator 06, and target training or fine-tuning of 07 on translated pairs plus optional unsupervised reprojection on target stereo pairs (Sakuma et al., 2021). The losses include adversarial, perceptual, feature matching, stereo-consistency, supervised disparity, and target reprojection terms. The stereo-consistency loss is computed on translated multi-scale features using known source disparities and an occlusion mask.
Quantitatively, on Driving 08 KITTI 2015 with a DispNetC backbone, the paper reports D1-all 09, EPE 10 px without SCA, and D1-all 11, EPE 12 px with SCA (Sakuma et al., 2021). In comparison to other unsupervised methods, the proposed approach reaches D1-all 13, EPE 14 px, compared with StereoGAN at 15, 16 px and L2A+Wad at 17, 18 px. With a GwcNet backbone on SceneFlow 19 KITTI, the proposed method reaches D1-all 20, EPE 21 px (Sakuma et al., 2021).
This line of work is methodologically related to later “adapter” approaches in that it introduces a stereo-specific cross-view mechanism into a pre-existing architecture. The difference is that the inserted mechanism is geometry-preserving attention for domain translation, not parameter-efficient fine-tuning in the modern PEFT sense.
5. Adapter modules for stereo image super-resolution
In “ASTeISR: Adapting Single Image Super-resolution Pre-trained Model for Efficient Stereo Image Super-resolution,” the phrase “stereo adapter” has a more specific PEFT meaning (Zhou et al., 2024). The method starts from a frozen, pre-trained single-image super-resolution transformer, HAT-L, and injects two kinds of lightweight modules: stereo adapters for cross-view fusion and spatial adapters for local feature refinement.
Stereo adapters are placed immediately after each Residual Hybrid Attention Group (RHAG), while spatial adapters are inserted inside each Hybrid Attention Block (HAB), after the hybrid-attention layer and before the feed-forward MLP (Zhou et al., 2024). For a pair of feature tensors 22, the stereo adapter applies LayerNorm, linear projections, and temperature-scaled cross-attention: 23
24
The residual updates are
25
The spatial adapter is a bottleneck MLP within each HAB: 26 The reduction ratio is 27 in practice (Zhou et al., 2024).
Fine-tuning keeps the base HAT-L model, approximately 28M parameters, fully frozen. Only the stereo adapters and spatial adapters are trained, totaling approximately 29M tunable parameters, or approximately 30 of the base (Zhou et al., 2024). The loss is 31 between super-resolved left and right outputs and their ground-truth high-resolution targets. Training uses AdamW with initial learning rate 32, no extra regularizer beyond weight decay, Flickr1024 plus Middlebury with random 33 crops, 34 A40 GPUs, batch size 35, and 36K iterations (Zhou et al., 2024).
The reported quantitative gains center on Flickr1024. For 37 super-resolution, frozen HAT-L yields 38 dB, full fine-tuning yields 39 dB, PEFT with spatial adapters only gives 40 dB, stereo adapters only give 41 dB, and stereo plus spatial adapters give 42 dB (Zhou et al., 2024). For 43 super-resolution on Flickr1024, HAT-L gives 44 dB and ASteISR gives 45 dB, a 46 dB gain. Compared with full fine-tuning, the method reduces fine-tuning time from 47 h to 48 h and GPU memory from 49 GB to 50 GB, corresponding to reductions of 51 and 52, respectively (Zhou et al., 2024).
Ablations indicate that the spatial adapter alone provides negligible gain, approximately 53 dB, whereas the stereo adapter alone provides 54 dB, and both together provide 55 dB in the 56 setting. Performance increases as more stereo-adapter layers are added and saturates after approximately 57–58 adapters, one per RHAG. Alternative spatial-adapter placements in parallel to the MLP or after the MLP increase tunable parameters and GPU time by at least 59 but slightly hurt PSNR by approximately 60 to 61 dB (Zhou et al., 2024). The paper also states that full fine-tuning on the small stereo dataset degrades texture restoration relative to the frozen SISR prior, whereas stereo adapters recover and improve on that prior.
This work is representative of a newer use of “adapter” in the PEFT literature: instead of adapting a stereo model online, the method adapts a monocular pre-trained model into a stereo-capable one by adding small trainable modules.
6. Underwater StereoAdapter and StereoAdapter-2
In “StereoAdapter: Adapting Stereo Depth Estimation to Underwater Scenes,” the term denotes a self-supervised underwater stereo depth estimation framework that addresses severe underwater domain shift, including wavelength-dependent attenuation, backscattering and forward scattering, refraction at water-glass interfaces, specular highlights, and textureless regions (Wu et al., 19 Sep 2025). The paper identifies two key challenges: adapting large pre-trained vision encoders to underwater appearance in a parameter-efficient manner without dense depth labels, and fusing globally coherent but scale-ambiguous monocular priors with locally metric yet photometrically fragile stereo correspondences.
The architecture is two-stage. Stage 1 uses a LoRA-adapted Depth Anything V2 encoder, frozen except for inserted LoRA modules, to generate multi-scale features
62
aggregate them with SDFA blocks into a discrete disparity volume 63, and produce a coarse disparity map 64 and metric depth 65 (Wu et al., 19 Sep 2025). The LoRA update for each transformer weight 66 is
67
Stage 2 performs recurrent stereo refinement. Stereo feature maps 68 are used to build a 4D correlation volume
69
followed by hybrid scale alignment to verify or correct the global scale of 70 through sparse stereo correspondences 71, multi-scale context aggregation, and iterative ConvGRU refinement: 72 The final disparity is upsampled RAFT-style to full resolution (Wu et al., 19 Sep 2025).
The method also introduces dynamic LoRA adaptation, in which the update is written as
73
with 74 sparsity on the importance weights 75. The joint training objective is
76
and the importance weights are updated using a proximal-gradient soft-threshold rule with a threshold 77 ramped from 78 to 79 (Wu et al., 19 Sep 2025).
Self-supervised learning is divided into monocular and stereo losses. The monocular stage uses
80
while the stereo stage uses
81
The guidance term constrains agreement between the coarse monocular disparity 82 and the refined stereo disparity 83 in both gradients and values, including an outlier mask 84 (Wu et al., 19 Sep 2025).
Pre-training uses the synthetic UW-StereoDepth-40K dataset rendered in Unreal Engine 5 with caustics, scattering, color attenuation, and particles. The four scene types are coral reefs, shipwrecks, industrial structures, and natural seabed. Baselines are 85, and the dataset contains 86 stereo pairs at 87 with dense ground-truth depth and segmentation (Wu et al., 19 Sep 2025).
The reported results cover zero-shot, fine-tuned, and real-world deployment. On TartanAir Underwater, zero-shot StereoAdapter trained on UW-StereoDepth-40K yields REL 88, SQ REL 89, RMSE 90, and 91 92, compared with Stereo Anywhere at REL 93, RMSE 94, and FoundationStereo at REL 95, RMSE 96. Fine-tuning on TartanAir gives REL 97, RMSE 98, 99 00 (Wu et al., 19 Sep 2025). On SQUID, zero-shot StereoAdapter yields REL 01, RMSE 02, 03 04, and the fine-tuned version yields REL 05, RMSE 06, 07 08. On BlueROV2 indoor tank deployment, StereoAdapter yields REL 09, SQ REL 10, RMSE 11, and 12 13. Test-time latency on Jetson Orin NX at 14, batch size 15, is 16 ms, compared with 17 ms for FoundationStereo and 18 ms for Stereo Anywhere (Wu et al., 19 Sep 2025).
“StereoAdapter-2” revises this design by retaining the LoRA-adapted monocular foundation encoder, now Depth Anything 3 ViT-B, for feature extraction and disparity initialization, while replacing the ConvGRU-based updater with a ConvSS2D operator based on selective state-space models (Ren et al., 18 Feb 2026). The continuous-time SSM
19
is discretized to a structured convolution kernel and 1D recurrence
20
In ConvSS2D, the recurrence is input-dependent: 21 with
22
The scan is applied in four directions: left-to-right, right-to-left, top-to-bottom, and bottom-to-top (Ren et al., 18 Feb 2026). The horizontal directions align with epipolar geometry, while the vertical directions enforce structural consistency.
The complexity discussion contrasts ConvGRU and ConvSS2D. ConvGRU requires approximately four convolutions of kernel size 23 over 24, giving 25, whereas ConvSS2D performs four linear recurrences of length 26 with 27 per location, giving 28. Since 29 is small, such as 30, the method is effectively linear in the number of pixels with a smaller constant (Ren et al., 18 Feb 2026).
StereoAdapter-2 also expands the synthetic corpus to UW-StereoDepth-80K by generating 31K new diffusion-based samples and merging them with UW-StereoDepth-40K. The pipeline combines Atlantis style transfer conditioned on source depth with NVS-Solver novel-view synthesis. Baselines are 32 cm, and the resulting stereo pairs are 33 with ground-truth disparity (Ren et al., 18 Feb 2026).
The zero-shot TartanAir-UW results reported for StereoAdapter-2 trained on 34K are AbsRel 35, SqRel 36, RMSE 37, LogRMSE 38, 39, 40, and 41. Compared with StereoAdapter-1 trained on 42K, which gives AbsRel 43 and RMSE 44, this corresponds to a 45 REL reduction and 46 RMSE reduction (Ren et al., 18 Feb 2026). On SQUID, StereoAdapter-2 gives AbsRel 47, RMSE 48, and 49, improving over StereoAdapter-1’s AbsRel 50 and RMSE 51. On BlueROV2, it reports REL 52, SqRel 53, RMSE 54, LogRMSE 55, and 56, outperforming the cited baseline FoundationStereo at REL 57, RMSE 58 (Ren et al., 18 Feb 2026). On-board inference with Jetson Orin NX and TensorRT at 59, batch 60, is 61 (Ren et al., 18 Feb 2026).
7. Common themes, distinctions, and interpretive context
Across these works, “StereoAdapter” consistently refers to a mechanism for making stereo systems usable outside the narrow regime assumed by a base setup or model. In the smartphone rig, the adaptation is optical: planar mirrors adapt monocularly separated front and rear cameras into a stereo pair (Bartol et al., 2021). In online and federated stereo matching, the adaptation is parametric: model weights are updated to accommodate domain shift during deployment (Tonioni et al., 2019, Poggi et al., 2024). In geometry-aware domain adaptation, the adaptation is representational: cross-view attention preserves epipolar structure during image translation (Sakuma et al., 2021). In ASteISR, adaptation is parameter-efficient transfer from single-image super-resolution to stereo super-resolution using inserted modules (Zhou et al., 2024). In underwater StereoAdapter and StereoAdapter-2, adaptation combines PEFT, self-supervision, and recurrent stereo refinement to bridge the severe physics-induced gap between terrestrial and underwater imagery (Wu et al., 19 Sep 2025, Ren et al., 18 Feb 2026).
A common misconception would be to treat these as successive versions of a single architecture. The corpus does not support that interpretation. Only the underwater pair, StereoAdapter and StereoAdapter-2, forms an explicit generational sequence (Wu et al., 19 Sep 2025, Ren et al., 18 Feb 2026). The smartphone device, learning-to-adapt framework, federated adaptation system, geometry-aware SCA method, and ASteISR are independent uses of the same or closely related naming convention.
Another distinction concerns what is being “adapted.” In the 2019 and 2024 online-adaptation works, a stereo matcher is adapted to the environment (Tonioni et al., 2019, Poggi et al., 2024). In the 2021 geometry-aware method, the source images are adapted to the target domain while preserving stereo geometry (Sakuma et al., 2021). In ASteISR, a single-image pre-trained transformer is adapted to stereo super-resolution (Zhou et al., 2024). In the 2025 and 2026 underwater works, a monocular foundation encoder is adapted to underwater stereo depth estimation, and then the stereo updater itself is redesigned for more global propagation (Wu et al., 19 Sep 2025, Ren et al., 18 Feb 2026). In the smartphone rig, the cameras are physically adapted into a stereo apparatus (Bartol et al., 2021).
This suggests that “StereoAdapter” has become a productive research label precisely because it is semantically broad: it can denote a device, an online learning rule, a federated adaptation framework, an attention mechanism, a PEFT module, or a domain-specific stereo depth system. The unifying idea is stereo functionality obtained through an added structure—optical, architectural, algorithmic, or optimization-based—rather than through a stereo system designed from scratch.