Synthetic Schlieren Imaging

• Synthetic Schlieren Imaging is a refraction-based method that infers flow and density gradients by analyzing the apparent displacement of engineered backgrounds.
• It uses techniques such as Fourier demodulation and optical flow to transform measured displacement fields into quantitative physical properties like free-surface slopes and density gradients.
• The approach is versatile, capable of assessing gas flows, stratified liquids, and acoustic fields, while supporting advanced tomographic reconstructions and real-time visualization.

Synthetic Schlieren Imaging (SSI), often termed Background Oriented Schlieren (BOS), is a refraction-based imaging methodology in which a camera observes a textured background through a transparent medium or interface, and the refractive disturbance is inferred from the apparent displacement of that background. In quantitative SSI, the measured displacement field is related to refractive-index gradients, density gradients, or free-surface slopes, depending on the optical configuration and constitutive model. The technique spans gas flows, density-stratified liquids, free liquid surfaces, acoustic fields, and tomographic reconstruction problems, and it replaces the knife-edge optics of classical schlieren with digital image registration, Fourier demodulation, or related inverse procedures (Wildeman, 2017, Verso et al., 2015, Li et al., 2021).

1. Physical basis and forward models

At its most general, SSI treats the disturbed image as a warped version of a reference background. One formulation writes

$I(\mathbf r)=I_0(\mathbf r-\mathbf u(\mathbf r)),$

where $\mathbf r=(x,y)$ and $\mathbf u(\mathbf r)$ is the apparent displacement field. In BOS geometry, this displacement arises because refractive-index gradients bend rays; under small-angle conditions, the apparent displacement is proportional to the line-of-sight integral of the transverse refractive-index gradient. A standard relation used in BOS is

$$\delta \mathbf r(x,y)\approx L_b\,\boldsymbol{\theta}(x,y),\qquad \boldsymbol{\theta}\approx \int \frac{\nabla_\perp n}{n}\,dz,$$

so the measurement is intrinsically a line-of-sight projection unless additional assumptions or tomographic views are introduced (Wildeman, 2017, Verso et al., 2015).

For gases and many liquid mixtures, the refractive index is connected to density through the Gladstone–Dale relation,

$n=1+K_{GD}\rho,$

which yields $\nabla n=K_{GD}\nabla \rho$. In this setting, SSI measures projected density gradients rather than density directly. In stratified-liquid BOS, this projected quantity is subsequently inverted through a Poisson formulation; in compressible-flow tomography, it becomes the forward operator for multi-view reconstruction (Verso et al., 2015, Li et al., 2024).

Free-surface SSI is a special case in which the refracting object is an interface rather than a volumetric density field. For small waves on water, one first-order mapping is

$$\mathbf u(\mathbf r)\approx -H\left(1-\frac{n_a}{n_w}\right)\nabla h(\mathbf r),$$

with $h(\mathbf r)$ the surface elevation and $H$ the distance between the surface and the patterned background. A distinct single-camera formulation for flat-bottom containers expresses the apparent marker displacement as a nonlinear function of both $h$ and $\mathbf r=(x,y)$0 through Snell’s law, and its paraxial linearization is

$\mathbf r=(x,y)$1

where $\mathbf r=(x,y)$2 is a calibrated view-angle field. Telecentric free-surface SSI eliminates paraxial distortions and permits exact local Snell-based forward models that extend beyond small-slope and small-amplitude regimes (Wildeman, 2017, Li et al., 2021, Zhang et al., 12 May 2026).

2. Backgrounds, optics, and acquisition geometries

The canonical SSI arrangement uses a camera, a refracting object between camera and background, and at least one reference image acquired without the disturbance. The background may be random-dot, checkerboard, grid-like, or naturally textured. Random-dot BOS emphasizes broadband texture for correlation or optical flow, whereas periodic backgrounds convert the problem into carrier demodulation. In the fast Fourier-demodulation variant, a checkerboard or other 2D periodic pattern is modeled as a superposition of carrier wavevectors, and the Schlieren-induced displacement appears as a phase modulation of those carriers (Wildeman, 2017).

Pattern choice is not neutral. For quantitative checkerboard SSI, the background period must be large enough to avoid sampling aliasing; a background sine period of at least 3.4 pixels is required ideally, while periodic patterns with wavelength at least 6 pixels work well in practice. Slight defocus acts as an optical low-pass filter that suppresses higher harmonics. In random-dot BOS, image quality and feature statistics remain critical: one assessment found that dot coverage of at least about 20–25% yielded artifact-free displacement fields, and that the interrogation-window size dominated the effective step-response width more strongly than the background feature size (Wildeman, 2017, Gojani et al., 2013).

Liquid experiments introduce additional geometric complications because the optical path may traverse air, glass, liquid, glass, and air. In density-stratified liquid BOS, the use of two references—an “air” image $\mathbf r=(x,y)$3 and a “uniform liquid” image $\mathbf r=(x,y)$4—supports a two-step calibration and image remapping transform that removes deterministic multi-media aberrations before the final displacement measurement. The same work reports calibration distortions reaching tens of pixels, with maxima up to about 70 pixels near the image corners over a roughly $\mathbf r=(x,y)$5 field, making remapping essential rather than cosmetic (Verso et al., 2015).

Several recent geometries alter the background itself. A directional-ray BOS configuration replaces the diffusely reflecting background with a spherically concave mirror etched with random dots and illuminated coaxially by an LED. In this arrangement the camera remains focused on the mirror, while the reflected rays are nominally directional rather than cone-like. The reported sensitivity remains

$\mathbf r=(x,y)$6

but the cone-ray averaging that ordinarily couples aperture size to blur is strongly reduced, so spatial resolution improves without the usual penalty in sensitivity (Li et al., 5 Sep 2025).

High-speed and unconventional platforms adopt further optical modifications. BOS has been used to image airborne ultrasonic fields using a random speckle background, a Canon EOS 1200D, and a phase-locked 40 kHz pulsed LED with 5 $\mathbf r=(x,y)$7s pulses and 13 s exposure, with cross-correlation on $\mathbf r=(x,y)$8 pixel windows and 12 pixel overlap (Iodice et al., 2018). At the opposite end of the hardware spectrum, smartphone BOS has operated on both engineered and natural backgrounds; for that implementation, random binary squares produced the best reported reconstruction contrast with $\mathbf r=(x,y)$9, compared with $\mathbf u(\mathbf r)$0 for random dots and $\mathbf u(\mathbf r)$1 for random grayscale patterns (Rabha et al., 2024).

3. Displacement estimation algorithms

The original computational core of SSI is local image registration between a reference and a disturbed image. Standard BOS pipelines therefore use FFT-based cross-correlation, often in PIV-style interrogation windows, or dense optical-flow methods. These methods estimate a 2D displacement field from textured backgrounds, but they trade spatial resolution against noise robustness through window size, multi-pass warping, and regularization (Gojani et al., 2013).

A major departure from correlation-based processing is Fast Checkerboard Demodulation (FCD). Instead of correlating random features, FCD isolates the first-order Fourier peaks of a periodic backdrop, inverse-transforms the filtered carrier bands, and computes the local phase difference

$\mathbf u(\mathbf r)$2

For an orthogonal checkerboard with carriers $\mathbf u(\mathbf r)$3 and $\mathbf u(\mathbf r)$4, the displacement components follow directly as

$\mathbf u(\mathbf r)$5

The method remains valid only while carrier sidebands remain spectrally separated, leading to criteria such as $\mathbf u(\mathbf r)$6, $\mathbf u(\mathbf r)$7, and $\mathbf u(\mathbf r)$8 in the no-wrap regime (Wildeman, 2017).

For random-dot BOS, a dot-tracking methodology uses prior knowledge of the manufactured pattern—dot count, nominal location, and size—to segment and track individual dots rather than window-averaged texture. The reported method attains near 100% yield even for high dot densities of 20 dots per $\mathbf u(\mathbf r)$9 pixels and supplements centroid tracking with a “correlation correction” step, in which masked intensity patches from matched dots are cross-correlated to recover a noise-robust subpixel displacement. In synthetic tests, the method significantly improved accuracy, precision, and spatial resolution relative to conventional cross-correlation, and nearly all tracked vectors had error below 0.01 pixel in the zero-noise case (Rajendran et al., 2018).

Other SSI pipelines emphasize deployment context. The smartphone implementation “Pocket Schlieren” provides a live consecutive-frame subtraction mode for qualitative flow visualization and an offline dense optical-flow mode based on Gunnar Farnebäck optical flow. It reports 45–50 FPS live BOS at 720p on a budget Android device, while full optical-flow reconstruction takes about 10–11 s for a $$\delta \mathbf r(x,y)\approx L_b\,\boldsymbol{\theta}(x,y),\qquad \boldsymbol{\theta}\approx \int \frac{\nabla_\perp n}{n}\,dz,$$0 pair on the test smartphone (Rabha et al., 2024). Event-based BOS replaces frames with event streams and formulates a variational problem in which the brightness-increment image reconstructed from events is matched to a linearized event generation model. Its physically motivated parameterization optimizes a scalar field $$\delta \mathbf r(x,y)\approx L_b\,\boldsymbol{\theta}(x,y),\qquad \boldsymbol{\theta}\approx \int \frac{\nabla_\perp n}{n}\,dz,$$1 and sets the flow as $$\delta \mathbf r(x,y)\approx L_b\,\boldsymbol{\theta}(x,y),\qquad \boldsymbol{\theta}\approx \int \frac{\nabla_\perp n}{n}\,dz,$$2, reflecting the BOS relation between apparent motion and the temporal derivative of the density gradient (Shiba et al., 2023).

4. Inversion from displacement to physical fields

Once the apparent displacement has been estimated, SSI becomes an inverse problem whose form depends on the underlying physics. In free-surface wave measurements using a known background offset $$\delta \mathbf r(x,y)\approx L_b\,\boldsymbol{\theta}(x,y),\qquad \boldsymbol{\theta}\approx \int \frac{\nabla_\perp n}{n}\,dz,$$3, the first-order relation

$$\delta \mathbf r(x,y)\approx L_b\,\boldsymbol{\theta}(x,y),\qquad \boldsymbol{\theta}\approx \int \frac{\nabla_\perp n}{n}\,dz,$$4

reduces the problem to gradient integration. One convenient formulation is

$$\delta \mathbf r(x,y)\approx L_b\,\boldsymbol{\theta}(x,y),\qquad \boldsymbol{\theta}\approx \int \frac{\nabla_\perp n}{n}\,dz,$$5

which may be solved by Fourier integration or by real-space sparse solvers with Dirichlet or Neumann boundary conditions (Wildeman, 2017).

The single-camera free-surface method removes the need for a known experimental liquid depth. It calibrates the view-angle field $$\delta \mathbf r(x,y)\approx L_b\,\boldsymbol{\theta}(x,y),\qquad \boldsymbol{\theta}\approx \int \frac{\nabla_\perp n}{n}\,dz,$$6 from a flat surface of arbitrary known height $$\delta \mathbf r(x,y)\approx L_b\,\boldsymbol{\theta}(x,y),\qquad \boldsymbol{\theta}\approx \int \frac{\nabla_\perp n}{n}\,dz,$$7, computes the measured displacement from the dry-bottom reference, and solves the nonlinear Snell-law forward model by Newton–Raphson. In the linearized regime the discrete problem is overdetermined, with $$\delta \mathbf r(x,y)\approx L_b\,\boldsymbol{\theta}(x,y),\qquad \boldsymbol{\theta}\approx \int \frac{\nabla_\perp n}{n}\,dz,$$8 equations for $$\delta \mathbf r(x,y)\approx L_b\,\boldsymbol{\theta}(x,y),\qquad \boldsymbol{\theta}\approx \int \frac{\nabla_\perp n}{n}\,dz,$$9 unknown heights on an $n=1+K_{GD}\rho,$0 grid, and convergence is reported when the maximum least-squares residual norm falls below $n=1+K_{GD}\rho,$1 (Li et al., 2021).

Telecentric nonlinear free-surface SSI formalizes three higher-order reconstructions: WNL-h, WNL-H, and FNL-H. WNL-h is a perturbative expansion in both slope and amplitude; WNL-H and FNL-H rewrite the problem in terms of the total height $n=1+K_{GD}\rho,$2 and update $n=1+K_{GD}\rho,$3 iteratively. In practice, these nonlinear reconstructions require only a few iterations and improve accuracy in high-slope or high-amplitude regimes, provided a reference height is available and caustics are absent (Zhang et al., 12 May 2026).

For density-stratified liquids, the inversion proceeds from corrected displacement to refractive index through a Poisson equation, following Venkatakrishnan and Meier (2004):

$n=1+K_{GD}\rho,$4

Dirichlet refractive indices are imposed at the top and bottom boundaries from pycnometer measurements, Neumann conditions are used laterally, and the resulting $n=1+K_{GD}\rho,$5 is converted to density through the Gladstone–Dale relation. In the reported four-layer MgSO$n=1+K_{GD}\rho,$6/water experiment, the corrected reconstruction recovered layer positions and densities within about 5% of pycnometer values (Verso et al., 2015).

Noise in gradient integration has motivated alternatives to the conventional Poisson solve. A weighted least-squares formulation solves

$n=1+K_{GD}\rho,$7

with weights taken as inverse variances of the local density-gradient uncertainty. In synthetic BOS images of a Gaussian density field, this WLS formulation reduced random error by 80% in comparison to Poisson for the highest noise level; in experimental BOS measurements of a spark plasma discharge it reduced density uncertainty by 30% relative to Poisson (Rajendran et al., 2020).

Three-dimensional tomography generalizes the inversion again. In NeDF, the unknown field is the 3D refractive-index gradient represented by a compact MLP, $n=1+K_{GD}\rho,$8, and each measured deflection is predicted by a line integral

$n=1+K_{GD}\rho,$9

The implementation combines multiresolution hash encoding after Müller et al. (2022) with hierarchical sampling along rays, then optimizes the mean-squared mismatch to the measured deflections. In sparse-view coplanar arrays, the reported method with $\nabla n=K_{GD}\nabla \rho$0 matched the RMSE and PSNR of CGLS-VH with $\nabla n=K_{GD}\nabla \rho$1, and matched the SSIM of CGLS-VH with $\nabla n=K_{GD}\nabla \rho$2 (Li et al., 2024).

5. Accuracy, resolution, and uncertainty

SSI accuracy is constrained jointly by optical sampling, feature statistics, inversion conditioning, and failure of the assumed forward model. In checkerboard demodulation, faithful recovery requires spectral separation between the physical modulation and the carrier peaks. The reported practical sampling rule is a periodic background wavelength of at least 6 pixels, while phase wrapping is avoided when $\nabla n=K_{GD}\nabla \rho$3 or else handled by spatial or temporal unwrapping. In water-wave experiments, the radial-line displacement standard deviation was $\nabla n=K_{GD}\nabla \rho$4 pixels for FCD versus $\nabla n=K_{GD}\nabla \rho$5 for DIC, corresponding to about threefold higher SNR, while throughput reached about 50 FPS on CPU and about 190 FPS on GPU at $\nabla n=K_{GD}\nabla \rho$6 (Wildeman, 2017).

Image quality is not a secondary issue. One BOS assessment introduced a modified image-quality index $\nabla n=K_{GD}\nabla \rho$7 and found $\nabla n=K_{GD}\nabla \rho$8 for an Imacon DSR200 high-speed camera, $\nabla n=K_{GD}\nabla \rho$9 for a Shimadzu HPV-1, and $$\mathbf u(\mathbf r)\approx -H\left(1-\frac{n_a}{n_w}\right)\nabla h(\mathbf r),$$0 for a Pentax K-5 under the reported conditions. In synthetic step-shift tests, the recovered transition width was about 38 pixels for a 32-pixel interrogation window, 30 pixels for 16 pixels, and 16 pixels for 8 pixels, demonstrating that interrogation-window size rather than background-feature size dominated the effective spatial resolution (Gojani et al., 2013).

Dot-tracking BOS improves this trade-off by replacing window-averaged texture with per-dot measurements. The reported methodology remains robust to image noise, preserves near 100% vector yield at high dot densities, and, with correlation correction, combines the spatial resolution of tracking with the smoothing benefit of correlation. Its superiority is especially pronounced for displacement gradients, which are the quantities propagated into density integration (Rajendran et al., 2018).

Free-surface SSI has its own uncertainty structure. In ripple data processed with the single-camera method, adding Gaussian displacement noise with mean 0.1 pixel and standard deviation 0.05 pixel produced a mean relative height error of about 0.2% with 1.2% standard deviation, while white noise of $$\mathbf u(\mathbf r)\approx -H\left(1-\frac{n_a}{n_w}\right)\nabla h(\mathbf r),$$1 pixel produced about 0.1% mean error with 0.9% standard deviation. The same study found that camera vibration up to about 0.4 pixel could change the spatial mean height by up to about 30%, and that larger interrogation windows reduced the recovered peak-to-peak topography even when mean height changed only modestly (Li et al., 2021).

The integration stage can magnify these errors if it does not account for spatially varying uncertainty. This is precisely the regime in which weighted least squares outperforms the Poisson solve: the reported WLS solution confines error to locally noisy regions rather than propagating it elliptically across the field, yielding the largest gains in synthetic tests with strong noise inhomogeneity and a 25–30% uncertainty reduction in the experimental spark-discharge data (Rajendran et al., 2020).

6. Applications, recent extensions, and limitations

SSI has been applied across a wide range of transparent-flow problems. In air-convection experiments above a heated or cooled aluminum block, synthetic schlieren visualized rising or descending plumes and enabled a calibration

$$\mathbf u(\mathbf r)\approx -H\left(1-\frac{n_a}{n_w}\right)\nabla h(\mathbf r),$$2

with $$\mathbf u(\mathbf r)\approx -H\left(1-\frac{n_a}{n_w}\right)\nabla h(\mathbf r),$$3 and $$\mathbf u(\mathbf r)\approx -H\left(1-\frac{n_a}{n_w}\right)\nabla h(\mathbf r),$$4. The same work noted that foundational references for synthetic schlieren and BOS include Settles and the work of Sutherland, Dalziel, Hughes, and Linden (Taberlet et al., 2017).

Related diagnostics delimit the scope of SSI. An imaging refractometer for high-energy-density plasmas measures deflection angles directly in one axis while preserving spatial imaging in the other, reporting measurable deflections from 0.06 to 34 mrad and a dynamic range greater than 500. In BOS terms, this is not a background-tracking method, but it provides a direct angle benchmark for situations where caustics and intensity-based inversion are problematic (Hare et al., 2020).

Ultrasonic and acoustic fields are also accessible. Background-oriented schlieren has been used to visualize 40 kHz airborne ultrasonic levitation fields, with phase-locked LED illumination and post-processed displacement maps showing anti-node spacing of about 4.3 mm in air (Iodice et al., 2018). At the low-cost end, smartphone BOS has visualized candle flames, butane lighters, soldering irons, room heaters, immersion heaters, and large outdoor flames against natural backgrounds, supporting live operation on-device and engineered or natural texture sources (Rabha et al., 2024).

Recent extensions target sensing modality and inverse-problem structure. Event-based BOS achieves results reported as on par with frame-based optical-flow techniques, works under dark conditions where frame-based schlieren fails, and enables slow-motion analysis from event timestamps (Shiba et al., 2023). Sparse-view tomographic BOS with NeDF addresses the ill-posedness of limited-angle 3D reconstructions through a learned continuous field representation rather than voxelized algebraic inversion (Li et al., 2024). Directional-ray BOS improves spatial resolution without compromising measurement sensitivity and, in the reported supersonic-jet experiment, achieved comparable image brightness at 40 kHz using a 0.3 W LED instead of the 18 W illumination used for canonical BOS (Li et al., 5 Sep 2025).

The technique nonetheless retains structural limitations. All BOS variants remain line-of-sight measurements unless symmetry, tomography, or other priors are imposed. Multi-media liquid paths can introduce large deterministic aberrations that must be calibrated out. Free-surface reconstructions break down under multiple refraction or caustics, and the single-camera free-surface method requires a transparent liquid and a flat, optically visible bottom. More generally, phase wrapping, carrier overlap, occlusions, specular highlights, and camera vibration remain recurring failure modes across SSI modalities (Verso et al., 2015, Wildeman, 2017, Li et al., 2021).