Self-Stereo: Self-Supervised Stereo Methods

• Self-Stereo is a family of methods that learn stereo correspondences, depth, or synthesized views using self-generated supervision from intrinsic data rather than manual labels.
• These techniques leverage geometric self-consistency, multi-view redundancy, and pseudo-labeling to enhance stereo matching without dense ground-truth supervision.
• Applications include calibrated multi-view stereo, monocular-to-stereo synthesis, and even stereo audio, with strategies addressing adaptation, robustness, and perceptual plausibility.

Across these works, Self-Stereo denotes a family of methods that learn stereo-aware correspondences, depth, synthesized stereo views, or stereo-channel structure from supervision intrinsic to the data itself rather than from dense manually labeled disparity or depth. In the strict binocular-vision sense, this means learning disparity from rectified left-right pairs through warping, reconstruction, consistency, or pseudo-labeling. In broader usage, it extends to calibrated multi-view stereo, monocular-to-stereo synthesis, stereo-aware representation learning, cross-modal stereo between intensity and event sensors, and even stereo audio, where the two channels are treated as two observations of the same scene or sound field (Zhong et al., 2017, Yang et al., 2021, Lv et al., 2024, Zhou, 29 Mar 2026, Chen et al., 2022).

1. Definition, scope, and terminology

Self-Stereo is not a single task or a single training recipe. In classical computer vision, the core formulation is dense stereo matching from rectified left-right images, where a model predicts a disparity map and is trained without ground-truth disparity labels. Early formulations directly minimized image warping error and regularized the solution with smoothness, loop consistency, and priors such as a maximum-depth heuristic (Zhong et al., 2017). Later systems expanded this into stronger self-supervised pipelines using pseudo-labels, teacher–student transfer, multiscopic capture, or stereo-video geometry (Wang et al., 2021, Liu et al., 2020, Yuan et al., 2021).

The term also covers self-supervised multi-view stereo (MVS), where the output is a reference-view depth map rather than a binocular disparity map. In that setting, the inputs are a reference image, several source images, and known camera intrinsics and extrinsics, and supervision may begin from image reconstruction before moving to self-training with refined pseudo-labels (Yang et al., 2021). A related but distinct line uses meta-learning so that self-supervised adaptation on a new target domain becomes more effective; this is self-supervised at adaptation time, but not fully annotation-free end to end because the outer loop remains supervised on source data (Mallick et al., 2020).

A second broad usage of Self-Stereo appears in stereo synthesis and stereo generation. SpatialDreamer treats Self-Stereo as generating a stereo counterpart video from only a monocular input video while learning largely from self-generated supervisory signals rather than large collections of paired stereo videos (Lv et al., 2024). “Single-View View Synthesis with Self-Rectified Pseudo-Stereo” similarly inserts a synthesized stereo partner between a single image and downstream 3D reconstruction, thereby converting a single-view problem into a pseudo multi-view one (Zhou et al., 2023). In representation learning, BINO uses “self-stereo” to mean learning stereo-aware features from unlabeled rectified left-right pairs so that binocular geometry is internalized in the encoder itself rather than delegated to a separate linkage module (Zhou, 29 Mar 2026).

The label is therefore best read as an umbrella term. A recurrent misconception is that all Self-Stereo methods are purely unsupervised from beginning to end. The literature here shows otherwise: some methods are fully self-supervised at training time, some are self-supervised only during target-domain adaptation, some depend on classical stereo seeds or synthetic paired data, and some retain supervised stereo branches while reducing dependence on stereo labels through auxiliary tasks (Mallick et al., 2020, Aleotti et al., 2020, Zhou et al., 2020).

2. Core supervisory principles

The foundational mechanism in Self-Stereo is view synthesis from predicted geometry. In the rectified binocular setting, a predicted disparity should reconstruct one image from the other. In SsSMnet, the right image is synthesized as

$I_R^{'}(u,v) = I_L(u+d_R(u,v),v),$

and training combines SSIM, pixelwise $L_1$, gradient matching, smoothness, loop consistency, and a maximum-depth heuristic instead of disparity residuals to ground truth (Zhong et al., 2017). PVStereo keeps this general reconstruction logic but adds a pseudo-label term and an edge-aware smoothness term,

$$\mathcal{L}=\mathcal{L}_{P}+\lambda_{1} \cdot \mathcal{L}_{R}+\lambda_{2} \cdot \mathcal{L}_{S},$$

so that photometric supervision becomes only one part of the objective rather than the sole signal (Wang et al., 2021).

A second principle is that stereo geometry can be strengthened by additional views or additional constraints. Flow2Stereo treats stereo disparity as a special case of optical flow and trains one model over stereoscopic video, computing all 12 directed pairwise correspondence maps among four images. It imposes triangle and quadrilateral constraints linking temporal motion and stereo disparity, so stereo is no longer supervised only by left-right reconstruction but also by cross-view temporal consistency (Liu et al., 2020). “Stereo Matching by Self-supervision of Multiscopic Vision” further assumes multiple aligned views with equal baselines and adds cross photometric loss plus uncertainty-aware mutual supervision between center-left and center-right disparity estimates, explicitly exploiting the fact that occluded regions in one view may be visible in another (Yuan et al., 2021).

A third principle is that raw photometric consistency is often too brittle, especially outside same-modality RGB stereo. “Self-Supervised Intensity-Event Stereo Matching” reconstructs an image from events and replaces ordinary photometric matching with a structure loss on image gradients, then adds disparity cross-consistency and internal disparity losses: $$\mathcal{L}=\lambda_{gd}\mathcal{L}_{gd}+\lambda_{sm}\mathcal{L}_{sm}+\lambda_{cc}\mathcal{L}_{cc}+\lambda_{itn}\mathcal{L}_{itn}.$$ This is necessary because the reconstructed event image and the intensity image differ too much in color and detail for standard photometric self-supervision to be reliable (Gu et al., 2022).

These works collectively suggest a common pattern: Self-Stereo begins with geometric self-consistency, but high-performing systems rarely stop at naive brightness constancy. They add multiview redundancy, learned confidence, cross-view agreement, or modality-aware structural losses to make the self-supervision usable in practice (Zhong et al., 2017, Liu et al., 2020, Yuan et al., 2021, Gu et al., 2022).

3. Self-generated supervision, pseudo-labels, and teacher–student training

A major branch of Self-Stereo shifts from direct photometric learning toward self-generated supervisory signals. PVStereo’s Pyramid Voting Module is exemplary: it runs a traditional stereo matcher across multiple image scales, computes variance statistics over disparity and matching confidence, and retains only pixels whose disparities and costs are consistent across scales and pass left-right consistency checking. The resulting semi-dense disparity map supervises the dense network with a discounted Huber loss over iterative predictions (Wang et al., 2021). This changes the training problem from “trust the photometric loss everywhere” to “learn densely from sparse reliable pseudo-labels plus dense regularization.”

“Reversing the cycle” adopts an even more explicit pseudo-label pipeline. It first computes sparse reliable disparities with classical stereo, trains a monocular completion network from sparse seeds, then runs the completion model repeatedly under random seed subsets and augmentations and keeps only low-variance predictions: $$D \xleftarrow{\sigma^2(\{D_i\}_{i=1}^{N}) < \gamma} \mu(\{D_i\}_{i=1}^{N}).$$ These distilled dense proxies then supervise a stereo network offline (Aleotti et al., 2020). The method is self-supervised in the sense that the supervision is generated within the stereo dataset itself, but it clearly depends on traditional stereo seeds and an auxiliary monocular completion stage.

Self-supervised MVS extends the same idea into calibrated multi-view geometry. Yang, Alvarez, and Liu first train CVP-MVSNet with image synthesis losses, use it to infer initial pseudo depths, refine those depths using higher-resolution inference and neighboring-view consistency, fuse them into a pseudo point cloud, reconstruct a mesh with screened Poisson surface reconstruction, and finally render pseudo depth labels back into each image. The network is then retrained on those rendered pseudo labels, iteratively (Yang et al., 2021). This is one of the clearest examples of Self-Stereo as self-training rather than merely unsupervised reconstruction.

Teacher–student mechanisms provide another route. BaCon-Stereo observes that a region occluded in a student’s target view is often visible in a teacher’s target view if the baselines differ. With

$z = \frac{B \cdot f}{d}, \qquad r=\frac{B^s}{B^t},$

the teacher disparity is rescaled to the student baseline and used in the consistency term

$\mathcal{L}_{c} = \| d^s - r \cdot d^t \|_1.$

Its “contrastive” language is therefore not InfoNCE-style metric learning but an asymmetric, negative-free teacher–student consistency formulation closer to BYOL, specialized to complementary visibility across baselines (Xu et al., 14 Aug 2025).

SpatialDreamer pushes self-generated supervision into monocular-to-stereo video synthesis. Its Depth based Video Generation module estimates monocular depth, forward-renders to a target stereo pose, fills disocclusions by inpainting, then backward-renders to obtain masks and partial reconstructions. These synthetic target views, occlusion masks, and backward reconstructions supervise a Stable Video Diffusion backbone with RefinerNet, stereo deviation strength

$s(z)=|z_0-z_{ref}|,$

and a stereo-aware regularizer

$l = l_{\epsilon} + \lambda \cdot l_d.$

The method explicitly does not claim physically accurate metric stereo reconstruction; its target is perceptually plausible, temporally coherent stereo generation (Lv et al., 2024).

4. Architectures and representation design

Self-Stereo has also become an architectural question: what inductive biases make self-supervision effective for stereo? OptStereo answers this with a lightweight recurrent stereo architecture. It computes an all-pairs correlation volume

$$\mathbf{M}^0(i,j,k) = \sum_{h=0}^{C} \mathbf{F}_l(i,j,h) \cdot \mathbf{F}_r(i,k,h),$$

builds a correlation pyramid, and iteratively updates disparity at $L_1$0 resolution using a convolutional GRU-like recurrent unit. The point is to avoid both heavy 3D cost-volume regularization and coarse-to-fine error accumulation, while remaining compatible with self-supervision from pseudo-labels and reconstruction losses (Wang et al., 2021).

BINO represents a sharper architectural claim: stereo correspondence can be learned largely inside a compact encoder. It fuses a rectified stereo pair at the input stage by horizontal pixel interleaving,

$L_1$1

so that every token contains binocular content from the first layer onward. It then uses a row-aware patch-phase positional parameterization and one-view masked token-only distillation. The exported descriptor readout

$L_1$2

shows that the learned encoder can serve as a frozen no-linkage probe for dense stereo and retrieval (Zhou, 29 Mar 2026). A plausible implication is that a substantial amount of binocular reasoning need not be deferred to a cost volume or binocular decoder if the stereo inductive bias is present at tokenization time.

SpatialDreamer reorients the design space toward generative video backbones. Rather than repeating per-frame novel-view synthesis, it uses Stable Video Diffusion as a temporal generative prior, adds RefinerNet for paired-view feature extraction, and introduces Temporal Interaction Learning,

$L_1$3

to aggregate neighboring-frame reference features (Lv et al., 2024). This suggests that in synthesis-oriented Self-Stereo, the architectural bottleneck is often temporal coherence rather than only geometric matching.

These architectural choices are diverse, but they share a theme: self-supervision alone is insufficient without the right representational bias. Correlation pyramids, binocular tokens, geometry-conditioned latent controls, and temporal attention all serve to make the self-generated supervisory signal easier for the network to exploit (Wang et al., 2021, Zhou, 29 Mar 2026, Lv et al., 2024).

5. Adaptation, robustness, and confidence estimation

Self-Stereo is also a deployment strategy. “Real-time self-adaptive deep stereo” moves adaptation from an offline stage to a continuous online process. MADNet is a pyramidal stereo network decomposed into scale modules, and MAD updates only one module per frame using a reward/punishment scheduler: $L_1$4 On the concatenated KITTI Raw stream, MADNet without adaptation reports D1-all $L_1$5, full online adaptation $L_1$6, and MAD $L_1$7, while preserving real-time behavior at $L_1$8 FPS for MAD versus $L_1$9 FPS for full online adaptation (Tonioni et al., 2018). Here Self-Stereo means a stereo system that continues to improve itself during deployment without ground-truth disparity.

A more specialized form appears in “Learning to Adapt Multi-View Stereo by Self-Supervision,” where MAML is used so that a few self-supervised photometric updates on a new domain become effective. The inner loop is self-supervised, but the outer loop is supervised on held-out validation data from source domains. This is best understood as meta-learned self-supervised domain adaptation for MVS rather than fully unsupervised stereo (Mallick et al., 2020).

Robustness under degraded visibility has become a distinct theme. RoSe identifies two main weaknesses of standard self-supervised stereo under night, rain, and fog: feature extractors degrade in reflective and textureless regions, and photometric consistency becomes unreliable. It addresses both by injecting robust priors from Depth Anything V2 into an FPN-based feature extractor, adding an Anti-Adverse Feature Enhancement Module, and training with scene correspondence priors between clear and translated adverse images. Step 1 optimizes

$$\mathcal{L}=\mathcal{L}_{P}+\lambda_{1} \cdot \mathcal{L}_{R}+\lambda_{2} \cdot \mathcal{L}_{S},$$0

and Step 2 replaces photometric supervision with masked distillation from a clear-weather teacher (Wang et al., 23 Sep 2025). This line makes explicit that robustness in Self-Stereo requires changing both the feature extractor and the supervisory signal.

A closely related but adjacent problem is confidence estimation. “Self-adapting confidence estimation for stereo” learns a confidence map for any stereo engine using only the stereo pair and the predicted disparity map. It combines reprojection error, disparity agreement, and a uniqueness constraint through a Multi-modal Binary Cross Entropy objective,

$$\mathcal{L}=\mathcal{L}_{P}+\lambda_{1} \cdot \mathcal{L}_{R}+\lambda_{2} \cdot \mathcal{L}_{S},$$1

This is not a disparity-estimation method, but it is highly relevant to Self-Stereo pipelines because self-supervised training, pseudo-label filtering, and online deployment all benefit from knowing which disparities are trustworthy (Poggi et al., 2020).

6. Expanded domains and applications

In synthesis-oriented computer vision, Self-Stereo frequently means manufacturing a second view to stabilize 3D reasoning. “Single-View View Synthesis with Self-Rectified Pseudo-Stereo” first warps a single left image into a pseudo right image, then identifies unreliable regions by pruning-based discrepancy and bidirectional pixel-space matching, fuses them into a mask,

$$\mathcal{L}=\mathcal{L}_{P}+\lambda_{1} \cdot \mathcal{L}_{R}+\lambda_{2} \cdot \mathcal{L}_{S},$$2

and inpaints only the unreliable regions before stereo-based 3D reconstruction and rendering (Zhou et al., 2023). This use of self-generated stereo is upstream of downstream tasks such as MPI- or LDI-based novel-view synthesis.

In audio, the same conceptual structure reappears with different observables. “Sound Localization by Self-Supervised Time Delay Estimation” treats left and right microphone channels as two views and learns interaural correspondences by contrastive random walks. With cross-channel transition matrix

$$\mathcal{L}=\mathcal{L}_{P}+\lambda_{1} \cdot \mathcal{L}_{R}+\lambda_{2} \cdot \mathcal{L}_{S},$$3

the cycle-consistency loss is

$$\mathcal{L}=\mathcal{L}_{P}+\lambda_{1} \cdot \mathcal{L}_{R}+\lambda_{2} \cdot \mathcal{L}_{S},$$4

This is Self-Stereo in audio: correspondence is learned from unlabeled stereo channel structure rather than delay labels (Chen et al., 2022).

“Sep-Stereo” extends the idea to mono-to-stereo audio generation by coupling stereo generation with visually guided source separation. Its main claim is that source separation can be treated as a particular kind of audio spatialization, so mono-only audiovisual data can reduce dependence on paired stereo audio. The result is not fully self-supervised in the strict sense—the stereo branch still uses stereo supervision—but it is a clear example of Self-Stereo as cross-task transfer from cheaper surrogate supervision (Zhou et al., 2020).

A broader, system-level usage appears in speaker-based personal sound zones. The Binaural Spatially Adaptive Neural Network predicts geometry-conditioned complex loudspeaker filters,

$$\mathcal{L}=\mathcal{L}_{P}+\lambda_{1} \cdot \mathcal{L}_{R}+\lambda_{2} \cdot \mathcal{L}_{S},$$5

to reproduce independent stereo programs at the left and right ears of multiple listeners. Its active crosstalk-cancellation stage optimizes an XTC objective after PSZ pretraining and reports log-frequency-weighted values of $$\mathcal{L}=\mathcal{L}_{P}+\lambda_{1} \cdot \mathcal{L}_{R}+\lambda_{2} \cdot \mathcal{L}_{S},$$6 dB for IZI, $$\mathcal{L}=\mathcal{L}_{P}+\lambda_{1} \cdot \mathcal{L}_{R}+\lambda_{2} \cdot \mathcal{L}_{S},$$7 dB for IPI, and $$\mathcal{L}=\mathcal{L}_{P}+\lambda_{1} \cdot \mathcal{L}_{R}+\lambda_{2} \cdot \mathcal{L}_{S},$$8 dB for XTC over 100–20,000 Hz (Jiang et al., 10 Jan 2026). This is not a self-supervised learning paper, but it shows that “self-stereo” can also denote ear-wise personalized stereo rendering rather than monophonic zone control.

7. Limitations, recurring controversies, and open questions

The most persistent technical limitation is the failure of photometric consistency in the regions stereo cares about most: occlusions, reflections, textureless areas, adverse weather, and cross-modal appearance mismatch. This problem is stated directly in BaCon-Stereo, RoSe, event–intensity stereo, and “Reversing the cycle,” and it motivates nearly every stronger supervisory mechanism in the literature: multi-baseline teachers, scene-correspondence priors, gradient-domain losses, consensus filtering, and monocular distillation (Xu et al., 14 Aug 2025, Wang et al., 23 Sep 2025, Gu et al., 2022, Aleotti et al., 2020).

A second limitation is that many Self-Stereo methods are not fully annotation-free or dependency-free. Meta-learned MVS uses supervision in the outer loop (Mallick et al., 2020). RoSe relies on supervised pretraining on SceneFlow and synthetic clear/adverse translations (Wang et al., 23 Sep 2025). “Reversing the cycle” depends on classical stereo seeds and an offline monocular completion stage (Aleotti et al., 2020). SpatialDreamer depends on a monocular depth estimator, optical flow, and inpainting, and at inference still assumes a target stereo pose or baseline (Lv et al., 2024). This does not invalidate the term, but it does narrow what “self” means in each case.

A third recurring issue is calibration and geometric assumptions. Many methods require rectified stereo, known intrinsics and extrinsics, equal-baseline multiscopic capture, or calibrated multi-view input (Yuan et al., 2021, Yang et al., 2021, Mallick et al., 2020). Even approaches that generalize the notion of stereo, such as event–intensity alignment or speaker-based ear-wise rendering, remain deeply geometry-dependent (Gu et al., 2022, Jiang et al., 10 Jan 2026). A plausible implication is that Self-Stereo has advanced fastest in settings where geometry is known and controllable.

Finally, the field remains split between metric reconstruction and perceptual plausibility. Some methods target dense, calibrated depth or disparity; others target perceptually convincing stereo effects. SpatialDreamer explicitly prioritizes geometrically plausible, visually realistic, and temporally stable stereo video rather than physically accurate metric reconstruction (Lv et al., 2024). This suggests that Self-Stereo is now better understood as a family of geometry-informed self-supervision strategies than as a single benchmark problem. The open question is not whether self-generated supervision works—it clearly does—but which forms of self-generated structure are adequate for which stereo objectives, modalities, and deployment constraints.