AquaStereo: Dual Underwater Stereo Methods

• AquaStereo is defined by two distinct methods addressing underwater stereo failure: one uses light field image synthesis to correct refraction, while the other leverages data-driven self-distillation.
• The 2019 approach reforms underwater stereo by synthesizing central-projection images from light field data, enabling standard stereo matching even under refractive distortions.
• The 2026 framework improves zero-shot stereo matching by generating geometry-faithful underwater data and enhancing features through depth-conditioned diffusion and perception-based modules.

AquaStereo denotes two distinct underwater stereo methodologies in the arXiv literature, both centered on recovering reliable binocular geometry in conditions where conventional stereo assumptions fail. The earlier method, introduced in “Underwater Stereo using Refraction-free Image Synthesized from Light Field Camera,” reformulates underwater reconstruction as a standard stereo problem by synthesizing refraction-free, central-projection images from light field observations and then applying ordinary stereo matching and triangulation (Ichimaru et al., 2019). The later method, introduced in “AquaStereo: Enabling Underwater Stereo Matching via Depth-Conditioned Diffusion and Geometry Self-Distillation,” addresses underwater stereo as a zero-shot generalization problem for learning-based matchers, combining depth-conditioned diffusion data synthesis, geometry self-distillation, and perception-enhanced feature learning to improve robustness under attenuation, scattering, backscatter, turbidity, low illumination, and severe texture loss (Wei et al., 5 Jul 2026). Taken together, these works define AquaStereo as a label associated with two different technical responses to underwater stereo failure: one primarily geometric and optics-driven, the other data-driven and representation-driven.

1. Terminological scope and research context

In the available literature, the name AquaStereo refers to two separate systems rather than a single unified framework. The 2019 work is an underwater stereo reconstruction technique based on a pair of Lytro Illum light field cameras and explicit handling of refraction through ray selection and image synthesis (Ichimaru et al., 2019). The 2026 work is a zero-shot underwater stereo matching framework intended for modern learning-based stereo networks, with a data simulation pipeline and a self-distillation strategy that jointly address data scarcity and feature degradation (Wei et al., 5 Jul 2026).

AquaStereo variant	Core problem	Main strategy
2019 AquaStereo	Refraction invalidates central projection underwater	Synthesize refraction-free images from light field rays, then run standard stereo
2026 AquaStereo	Domain gap and degraded correspondences in underwater imagery	Generate geometry-faithful underwater data and train with self-distillation and perception-enhanced features

The distinction matters because the two systems target different failure modes. In the 2019 formulation, the dominant issue is the breakdown of perspective geometry when a camera observes a scene through a water–air interface or a housing with a refractive boundary. In the 2026 formulation, the dominant issue is that underwater imagery is not merely noisier than terrestrial imagery but differs through attenuation, scattering, backscatter, wavelength-dependent color shifts, low illumination, turbidity, and severe texture loss, all of which weaken left-right consistency and degrade cost-volume construction (Wei et al., 5 Jul 2026).

A common misconception is that underwater stereo is a single, homogeneous problem. The two AquaStereo works indicate a more differentiated view. One branch treats underwater stereo chiefly as an optics-and-calibration problem; the other treats it as a domain-shift and feature-robustness problem. This suggests that “underwater stereo” spans both physical image formation and learned correspondence stability, and that the appropriate solution depends strongly on the sensing setup.

2. Geometric AquaStereo: refraction-free stereo from light field imaging

The 2019 AquaStereo method begins from the observation that standard stereo vision assumes a perspective / central projection camera model, which supports epipolar geometry, linear calibration models, and triangulation (Ichimaru et al., 2019). Underwater, that assumption fails because rays bend according to Snell’s law at the refractive interface. As a result, a pixel does not correspond to a single straight ray in the scene, the camera is no longer central projection, and stereo pipelines produce incorrect disparities and poor 3D reconstruction.

The method’s central idea is to turn this refraction problem into a standard stereo problem by first synthesizing refraction-free, central-projection images from a light field camera. A light field camera records a bundle of rays from different directions, making it possible to choose rays that behave as if there were no refraction. Instead of fitting an approximation that maps refracted images into a perspective model, the method seeks to select geometrically correct rays from the captured light field data so that the synthesized image is equivalent to a central-projection image of the underwater scene (Ichimaru et al., 2019).

The camera model is explicitly physical rather than purely two-plane. The paper models a main lens, a micro lens array (MLA), and an image sensor behind the MLA, with the main lens, MLA, and sensor parallel and modeled as thin lenses. The notation includes $f$ for focal length of the main lens, $f_{mla}$ for focal length of the micro lens array, $f'$ for the distance between the main lens and the MLA, and a sensor–MLA distance equal to $f_{mla}$ (Ichimaru et al., 2019).

For each output pixel $p$, the method conceptually traces the ray backward through the refractive interface to determine where that ray would land on the sensor if refraction were absent. Let $r_a$ denote the observed refracted ray in air, $P$ the intersection of that ray with the refractive surface, and $\hat{p}$ the sensor location reached by the “correct” ray if no refraction occurred. Because the light field camera captures many rays, the method searches for a measured ray close to this ideal ray and uses its color in the synthesized output. The selection criterion is

$$D = |P' - P| + \lambda \cos^{-1}(\hat{r}_a' \cdot \hat{r}_a),$$

where $P'$ is the intersection of the selected ray $f_{mla}$0 with the refractive surface and $f_{mla}$1 is a weighting coefficient (Ichimaru et al., 2019).

In practice, the authors assume

$f_{mla}$2

because the main lens focal length $f_{mla}$3 is hard to calibrate precisely. Under this assumption, the angular term becomes zero by their derivation, reducing the criterion to

$f_{mla}$4

This simplification makes the procedure efficient while retaining the geometric objective of selecting the most appropriate ray. The synthesized image is obtained by pixel warping: for each output pixel, compute the ideal refraction-free ray, find a captured ray that best matches it, and copy the corresponding source color. Because the selected rays match straight-through geometry, the resulting image behaves like a central-projection image and can be processed using ordinary stereo vision (Ichimaru et al., 2019).

3. Implementation and stereo reconstruction pipeline in the 2019 method

The 2019 system separates refraction-free image synthesis into a precomputation stage and a warping stage. In light path computation, the system analytically computes the light path for each micro lens and determines pixel-to-pixel correspondences between the input light field image and the refraction-free output image. Micro lens locations are manually provided, and rays whose ideal paths go outside the aperture are discarded or approximated by nearby rays (Ichimaru et al., 2019).

In pixel warping, the algorithm applies the correspondence map and outputs the refraction-free image by bilinear interpolation. Two quality improvements are used. First, several candidate rays in ascending order of $f_{mla}$5 are extracted and their colors are combined by weighted averaging with weights proportional to $f_{mla}$6. Second, super-sampling computes subpixel correspondences to increase the number of valid rays, improving image quality and reducing aliasing (Ichimaru et al., 2019). The paper reports an overview time of about 53 ms on an Intel Xeon E5640 CPU, described as near real-time.

Once a pair of refraction-free images has been synthesized from two light field cameras, the remainder of the pipeline is standard stereo reconstruction. Calibration data are captured using chessboard targets in air and in water. The method estimates the real flange back $f_{mla}$7 and lens distortion coefficients in air, removes ordinary camera lens distortion from all images, and then estimates refractive surface parameters—interface depth $f_{mla}$8 and surface normal $f_{mla}$9—using bundle adjustment following Agrawal et al. (CVPR 2012) (Ichimaru et al., 2019). Using $f'$0, $f'$1, and $f'$2, the system synthesizes a distortion-free image from each light field camera.

Stereo calibration between the two synthesized views is then performed as for ordinary central-projection cameras. A noteworthy procedural point is that the relative camera transformation does not need to be calibrated in water beforehand; air images can be captured separately. Stereo matching is performed using image rectification, an NCC-based matching cost, and OpenCV stereo tools, and 3D shape is recovered by standard triangulation (Ichimaru et al., 2019).

The experimental setup employed two Lytro Illum light field cameras outside a water tank of size 90 × 45 × 45 cm, with clear water and an intentionally slanted camera configuration to create strong distortion. Tested scenes included a planar calibration board at far, medium, and near depths, and two arbitrary-shaped objects: a crocodile figurine and a ceramic bowl (Ichimaru et al., 2019). The comparison against Ferreira’s approximation-based refraction correction showed that approximation methods work reasonably only near the calibration depth, degrade as target depth moves away, and become especially unstable for slanted refractive interfaces, whereas the proposed method is more stable across depths.

The paper also identifies limitations. Because only a finite set of rays is available, perfect refraction-free synthesis is possible only over a limited field of view: the possible field of view for perfect refraction-free imaging is about 30 degrees, whereas the original Lytro Illum field of view is about 60 degrees (Ichimaru et al., 2019). Averaging neighboring rays compensates for missing rays but introduces blur and defocus-like smoothing. The paper further notes that when the refractive interface is nearly orthogonal to the camera axis, approximation-based methods may sometimes match or even exceed the visual quality of the proposed method.

4. Learning-based AquaStereo: zero-shot underwater stereo matching

The 2026 AquaStereo framework addresses a different problem. Rather than focusing on refractive geometry at the camera interface, it considers the failure of learning-based stereo matchers under underwater image degradation and limited supervision. The setting is explicitly framed as zero-shot generalization: a stereo matcher trained largely on terrestrial data should work robustly on underwater images without target-domain fine-tuning (Wei et al., 5 Jul 2026).

The framework identifies two coupled difficulties. First, underwater degradations make correspondences ambiguous and violate the photometric assumptions on which stereo matchers rely. Second, high-quality real underwater stereo with accurate dense metric depth is expensive and sparse, while CG-simulated underwater datasets often lack realism and do not close the domain gap. AquaStereo therefore attacks both the data problem and the feature robustness problem together: it generates geometry-faithful synthetic underwater stereo pairs from clean terrestrial stereo, then trains a stereo model with geometry self-distillation and a perception-enhanced matching encoder (Wei et al., 5 Jul 2026).

The data synthesis pipeline begins from a large corpus of rectified terrestrial stereo pairs. Since dense ground-truth disparities are often unavailable or noisy, pseudo disparities are first obtained from a strong foundation stereo model:

$f'$3

Underwater appearance is then generated by conditioning Stable Diffusion through ControlNet on both depth and textual prompts. Using left and right depth maps, $f'$4 and $f'$5, allows degradation to vary with distance in a physically plausible way. The prompt pool $f'$6 is built from CLIP-retrieved underwater image descriptions and LLM-generated concise prompts summarizing underwater image formation cues such as wavelength-dependent attenuation, backscatter, and veiling light. With a sampled prompt $f'$7, the process is written as

$f'$8

$f'$9

The resulting $f_{mla}$0 are underwater-like stereo images intended to remain geometrically aligned with the original clean pair (Wei et al., 5 Jul 2026).

To preserve binocular geometry during generation, AquaStereo adds a lightweight left-right consistency module, also described as a coherence-enhanced consistency module. It samples patch pairs along epipolar lines and scores candidate correspondences using appearance similarity and coarse disparity priors:

$f_{mla}$1

These scores are normalized into soft confidence masks and used to modulate the ControlNet features. Ablations show that enabling this module reduces “break pixels” from 15 to 5 and improves UWStereo performance (Wei et al., 5 Jul 2026). The paper reports a synthetic UW-Dataset of 40K image pairs generated with Stable Diffusion v1.5, ControlNet depth conditioning, and DDIM sampling with 50 steps.

5. Geometry self-distillation and perception-enhanced features

After synthetic underwater data are generated, the 2026 framework uses cross-domain self-distillation rather than direct supervised training alone. A frozen binocular teacher trained on clean terrestrial pairs supervises a student exposed to rendered underwater pairs and their perturbations. In the experiments, teacher and student share the same IGEV++ architecture, but the teacher parameters are fixed (Wei et al., 5 Jul 2026).

The student receives underwater inputs, including a perturbed version,

$f_{mla}$2

where perturbations include turbidity/backscatter amplification, color shifts, compression noise, and mixed or mismatched patch augmentations. The feature alignment term is

$f_{mla}$3

with $f_{mla}$4 and $f_{mla}$5 denoting teacher and student features, respectively. The text describes this as a stage-weighted sequence loss, applied across the multi-stage or multi-sample sequence of perturbations so that disparities and features remain consistent as the student processes multiple underwater variants tied to the same geometry (Wei et al., 5 Jul 2026).

To prevent scale drift, the framework adds clean-branch supervision using the shared pseudo target:

$f_{mla}$6

and the final objective is

$f_{mla}$7

with $f_{mla}$8 in training (Wei et al., 5 Jul 2026). Ablations reported in the paper indicate that the best performance comes from self-distillation plus supervised training, whereas L2 regularization hurts performance. This is presented as evidence that preserving geometry requires the appropriate supervision signal rather than overly tight parameter constraint.

The feature formulation is likewise modified to address underwater degradation. AquaStereo introduces two learnable perception frames, $f_{mla}$9 and $p$0, appended to the stereo pair to form a pseudo temporal sequence,

$p$1

These are trainable parameters rather than real images and are intended to absorb underwater priors during training. A video encoder in the style of Change3D processes the four-frame sequence and produces view-aligned features,

$p$2

while semantic features are extracted by DINOv2:

$p$3

Perception and semantic features are fused by concatenation and a $p$4 projection,

$p$5

The fused descriptors $p$6 are used to build the cost volume and are refined iteratively following IGEV++ with Soft-Argmin regression (Wei et al., 5 Jul 2026).

Ablation results support the encoder design. The paper reports Total EPE/D1 of 2.483/10.824 for AquaStereo’s encoder, compared with 2.953/12.144 for the plain IGEV++ backbone (Wei et al., 5 Jul 2026). This indicates that perception frames plus DINOv2 semantics outperform the plain baseline as well as alternatives such as VGG19+DINOv2, ResNet+DINOv2, VGGT-style attention, and FoundationStereo-like features.

6. Empirical performance, interpretation, and comparative significance

The 2026 AquaStereo paper evaluates on UWStereo, FLSea-Stereo, Squid, and the TartanAir underwater split, using EPE and D1 as main metrics, with D1 defined as the fraction of pixels with error larger than $p$7 (Wei et al., 5 Jul 2026). On UWStereo, AquaStereo reports the best overall performance among the compared methods, including IGEV, StereoAnything, MonSter, and FoundationStereo, with EPE = 0.590 and D1 = 2.616, and it wins on every scene split: Coral, Default, Industry, and Ship. Training IGEV++ on AquaStereo’s rendered UW-Dataset also outperforms training on SceneFlow, KITTI, SceneFlow+KITTI, UWStereo, or StereoAdapter, particularly on real underwater benchmarks such as FLSea and Squid.

Qualitatively, the paper reports sharper object boundaries, smoother and more complete disparity surfaces, fewer “flying pixels” and halos, fewer depth artifacts in turbid or low-texture regions, and better preservation of thin structures such as rigging and vegetation. On point-cloud reconstruction, the framework yields denser, cleaner 3D geometry with fewer holes and fewer floating outliers (Wei et al., 5 Jul 2026).

The 2019 AquaStereo paper, by contrast, evaluates geometry through explicit reconstruction experiments under refraction. Reconstructed planar boards are assessed by plane fitting after outlier removal, and the proposed method produces lower error than the approximation-based approaches it compares against (Ichimaru et al., 2019). For the crocodile and bowl scenes, the approximation-based method becomes unstable when calibration depth mismatches object depth, while AquaStereo yields more stable and visually plausible 3D shapes.

These results point to a useful conceptual contrast. The 2019 work shows that refraction can be handled by transforming underwater observations into images that satisfy standard stereo assumptions. The 2026 work shows that even when binocular geometry is nominally available, underwater degradation can still break correspondence estimation unless data generation, supervision, and feature learning are adapted accordingly. A plausible implication is that the two works address complementary layers of the same broader problem: the physical validity of the image formation model and the statistical robustness of the matching model.

7. Limitations, misconceptions, and broader implications

Both AquaStereo variants are explicit about the limits of their respective assumptions. In the 2019 system, the light field solution is bounded by finite angular sampling and aperture constraints. Perfect refraction-free synthesis is only possible over a restricted field of view, and compensating for missing rays by averaging introduces blur and defocus-like smoothing (Ichimaru et al., 2019). The paper also notes that when the refractive interface is nearly orthogonal to the camera axis, approximation-based methods may sometimes match or exceed its visual quality. This counters any blanket claim that explicit light-field ray selection universally dominates simpler approximation.

In the 2026 system, the core challenge is not a refractive boundary model but the gap between clean terrestrial supervision and degraded underwater appearance. The paper argues that CG rendering and naïve style transfer do not adequately preserve both realism and geometry, motivating the diffusion-based synthetic corpus and self-distillation pipeline (Wei et al., 5 Jul 2026). A common misconception addressed by the work is that underwater stereo can be solved by treating underwater images as ordinary stereo inputs with added noise. The framework’s emphasis on view-dependent scattering, haze, color attenuation, and weakened left-right consistency suggests that the underwater domain violates stronger assumptions than conventional robustness augmentations are designed to handle.

The broader significance of the two methods lies in their complementary practical implications. The 2019 approach provides a route to underwater reconstruction that avoids custom stereo formulations for refraction, preserves compatibility with mature stereo tools, and improves reliability in difficult underwater scenes; the paper names applications including underwater inspection, marine biology, robotic perception, and shape measurement (Ichimaru et al., 2019). The 2026 approach targets modern learned stereo pipelines and is designed to improve robustness and zero-shot generalization without target-domain fine-tuning, a property that is directly relevant to deployments where collecting underwater dense depth labels is infeasible (Wei et al., 5 Jul 2026).

Viewed together, the AquaStereo label marks two distinct but convergent research directions in underwater stereo. One reconstructs central-projection structure from refracted light-field observations; the other reconstructs reliable disparities from degraded underwater imagery through synthetic data, geometry-preserving distillation, and perception-enhanced descriptors. Both are motivated by the same underlying fact: underwater stereo failure is not reducible to ordinary stereo noise, and effective methods must explicitly model the mechanisms by which underwater imaging departs from terrestrial assumptions.