Optical Gabor Masks in Holography & Gabor Analysis

• Optical Gabor Masks are phase-only optical filters used in inline holography to displace twin images, thereby preserving common-path stability and enhancing reconstruction quality.
• They employ specialized phase shapers such as axicons in the Fourier plane, integrating Gabor function design and symplectic invariance to form precise time–frequency filter banks.
• Practical implementations must address sensor alignment, phase calibration, and wavelength stability while supporting applications in digital holography, microscopy, and CGH.

Optical Gabor Masks designate two closely related constructions in optical science. In recent holographic work, they are phase-only optical elements inserted in the spatial-frequency domain of an in-line or on-axis holographic system so that the conjugate twin image is displaced outside the signal region of the reconstructed object, yielding an optical solution to the long-standing twin-image problem in Gabor holography (Glückstad, 23 Feb 2026). In a broader time–frequency formulation, optical Gabor masks are banks of localized sinusoidal filters arranged across position and spatial frequency, naturally modeled by multivariate Gabor systems on lattices in $\mathbb{R}^{2d}$ (Gjertsen et al., 2024). Their elementary two-dimensional form is the Gabor function, a Gaussian envelope modulating a sinusoid, with orientation, aspect ratio, carrier phase, and carrier spatial frequency controlling localization and selectivity (Loxley, 2013).

1. Historical placement and terminological scope

In-line or on-axis holography records the interference between an object field and a co-propagating reference field on a single sensor. The recent reformulation of this geometry emphasizes that Gabor holography was “ironically abandoned by its inventor Dennis Gabor himself” and was effectively “re-placed” by off-axis holography when Gabor received the Nobel Prize in Physics in 1971. At the same time, the same source states that Gabor holography is today “the method of choice in modern digital holography” because of its “inherent on-axis, common-path robustness, lower requirements to resolution of the image sensor (or recording material), shorter exposure time, relaxed mechanical stability and temporal coherence requirements” (Glückstad, 23 Feb 2026).

The phrase optical Gabor masks therefore names both a specific optical twin-image suppression strategy in in-line holography and a wider class of optical or computational Gabor filter banks. In the latter formulation, a lattice in joint position–frequency space specifies mask centers and carrier frequencies, and symplectic geometry classifies which such lattices have identical Gabor structure up to unitary equivalence (Gjertsen et al., 2024). This suggests a dual usage: one centered on holographic phase shaping, the other on time–frequency tilings and Gabor analysis.

A common misconception is that the historical limitations of on-axis holography make it intrinsically inferior to off-axis recording. The recent holographic formulation states a narrower point: the difficulty is the twin-image problem, not the common-path geometry itself. Optical Gabor Masks are proposed precisely to keep the single-shot, on-axis, common-path advantages while overcoming the twin-image artifact by optical means (Glückstad, 23 Feb 2026).

2. In-line holography and the origin of twin images

In-line holography records the interference between a reference field $R(x,y)$ and an object field $O(x,y)$ on a single sensor. With coherent illumination, the recorded intensity is

$I(x,y)=|R+O|^2=(R+O)(R+O)^*=RR^*+OO^*+OR^*+O^*R.$

The four terms have distinct roles. $RR^*$ is the bright background, $OO^*$ is object self-interference or autocorrelation, $OR^*$ is the desired real image term carrying the object’s complex field, and $O^*R$ is the twin or virtual image. Because the twin term is the complex conjugate, it reconstructs as a defocused or mirrored version that overlaps the desired image in in-line geometry (Glückstad, 23 Feb 2026).

Under Fresnel conditions, the hologram can be numerically back-propagated to a plane at distance $z$ using

$$U(x,y;z)=\mathcal{F}^{-1}\{\mathcal{F}\{I(x,y)\}\cdot H(f_x,f_y;z)\},$$

with paraxial transfer function

$R(x,y)$0

If the weak object assumption $R(x,y)$1 is used, the $R(x,y)$2 term is small and the cross terms dominate. Even then, in on-axis holography $R(x,y)$3 and $R(x,y)$4 both propagate to reconstructions that overlap. The source explicitly notes that this artifact “cannot be removed by simple spatial filtering,” unlike off-axis holography, where an angled reference creates a separable carrier (Glückstad, 23 Feb 2026).

This mechanism explains why twin-image suppression has traditionally required phase retrieval, phase shifting, or machine learning. The key contribution of the recent holographic approach is not a reinterpretation of the cross terms, but an optical restructuring of how the conjugate content is distributed in space and spatial frequency.

3. Phase-only Optical Gabor Masks and twin-image displacement

The reinvented holographic approach introduces an optical phase-only “Gabor mask” in the spatial-frequency domain such that the conjugate terms are displaced outside the signal region of the reconstructed object. The generalized in-line recording with system kernel $R(x,y)$5 is written as

$R(x,y)$6

where $R(x,y)$7 is the normalized object field and $R(x,y)$8 denotes convolution. In this expression, the twin image enters through $R(x,y)$9 (Glückstad, 23 Feb 2026).

The design principle is stated through stationary phase. For a phase-only element with phase $O(x,y)$0,

$O(x,y)$1

or in polar coordinates,

$O(x,y)$2

This relation is used to design the transfer function $O(x,y)$3 so that object and conjugate terms are mapped differently in the Fourier domain (Glückstad, 23 Feb 2026).

A particularly simple mask is an axicon inserted in the Fourier plane of a $O(x,y)$4 system. Its transfer function is

$O(x,y)$5

with

$O(x,y)$6

Here $O(x,y)$7 is the radial coordinate in the Fourier plane, $O(x,y)$8 is the focal length, $O(x,y)$9 is the wavelength, and $I(x,y)=|R+O|^2=(R+O)(R+O)^*=RR^*+OO^*+OR^*+O^*R.$0 sets the annular radius at which energy is localized. The conjugate transfer function is

$I(x,y)=|R+O|^2=(R+O)(R+O)^*=RR^*+OO^*+OR^*+O^*R.$1

and the corresponding conjugate kernel in the image plane is

$I(x,y)=|R+O|^2=(R+O)(R+O)^*=RR^*+OO^*+OR^*+O^*R.$2

The normalized sensor intensity then becomes

$I(x,y)=|R+O|^2=(R+O)(R+O)^*=RR^*+OO^*+OR^*+O^*R.$3

Hence the holographic terms are confined to a circular band of width set by the object’s characteristic radius $I(x,y)=|R+O|^2=(R+O)(R+O)^*=RR^*+OO^*+OR^*+O^*R.$4, centered at $I(x,y)=|R+O|^2=(R+O)(R+O)^*=RR^*+OO^*+OR^*+O^*R.$5 (Glückstad, 23 Feb 2026).

The reconstruction is

$I(x,y)=|R+O|^2=(R+O)(R+O)^*=RR^*+OO^*+OR^*+O^*R.$6

which simplifies to

$I(x,y)=|R+O|^2=(R+O)(R+O)^*=RR^*+OO^*+OR^*+O^*R.$7

The crucial point is that the twin-image terms are pushed into an annular band at radius $I(x,y)=|R+O|^2=(R+O)(R+O)^*=RR^*+OO^*+OR^*+O^*R.$8, well separated from the object reconstruction at $I(x,y)=|R+O|^2=(R+O)(R+O)^*=RR^*+OO^*+OR^*+O^*R.$9. Discarding the ring-convolved terms yields

$RR^*$0

The remaining self-interference term is reduced to half amplitude and is stated to be “often small compared to the desired field” (Glückstad, 23 Feb 2026).

The same source generalizes the construction beyond an axicon: any phase shaper inducing a parametric kernel $RR^*$1 can be designed to push the conjugate outside the signal area. In that sense, the axicon is a member of a broader family of Optical Gabor Masks rather than a unique design.

4. Optical architectures, reconstruction workflows, and design constraints

The optical setup uses a single coherent source, either a laser or a narrowband source, to provide both object and reference in a common-path geometry. In transmission, the object is illuminated on-axis, a $RR^*$2 system images the object’s Fourier plane onto the phase-only axicon, and the sensor records $RR^*$3. In reflection, a larger reflective support provides the co-propagating in-line reference, while the phase shaper resides in the frequency plane of an imaging train between object and sensor (Glückstad, 23 Feb 2026).

Reconstruction can be digital or analog. In digital reconstruction, one uses knowledge of $RR^*$4 and $RR^*$5 to reconstruct $RR^*$6 and discard the ring-localized terms. If free-space back propagation is used, the recorded hologram is propagated with the Fresnel kernel,

$RR^*$7

The annular band at radius $RR^*$8 is then masked, leaving $RR^*$9 for $OO^*$0. In analog replay, the recorded intensity is routed to a reflective spatial light modulator, and a matching axicon is placed in the replay arm; this replays the conjugate or the original object depending on the $OO^*$1 choice (Glückstad, 23 Feb 2026).

Several constraints are explicit. The axicon’s annular center “must be co-aligned with the optical axis to within a small fraction of the sensor’s pixel pitch.” To avoid overlap between central object content and ring-localized terms, the condition

$OO^*$2

must be enforced, where $OO^*$3 is the usable sensor width. Since the twin ring appears at $OO^*$4 after reconstruction, any annular stop or computational discarding targets $OO^*$5. Wavelength detuning shifts the annulus because $OO^*$6 scales linearly with $OO^*$7, so $OO^*$8 and $OO^*$9 must be chosen so that expected wavelength variations keep $OR^*$0 beyond the object support yet within sensor bounds (Glückstad, 23 Feb 2026).

For SLM implementation, the source gives four practical requirements. First, the SLM pixel pitch $OR^*$1 must sample the radial grating period $OR^*$2 without aliasing; “a conservative guideline is $OR^*$3.” Second, the SLM aperture must cover the Fourier-plane region occupied by the object spectrum. Third, phase-only SLMs require calibration for phase linearity. Fourth, the refresh rate must exceed the frame rate required by the application for dynamic scenes. Static masks can instead be fabricated as diffractive axicons by photolithography or laser writing, while refractive axicons are standard components (Glückstad, 23 Feb 2026).

The comparison to existing strategies is direct. Off-axis holography shifts real and twin terms to distinct spatial frequencies with an angled reference, but requires higher sensor sampling of the carrier and stricter stability and coherence. Phase-shifting methods are multi-shot and motion-sensitive. Iterative phase retrieval and deep learning can be slow or require training datasets. Optical Gabor Masks keep the common-path geometry and are single-shot while achieving separation by ring localization (Glückstad, 23 Feb 2026).

5. Multivariate Gabor systems, symplectic invariance, and Gaussian frame design

In the broader time–frequency formulation, optical Gabor masks are modeled by multivariate Gabor systems over lattices in phase space. With $OR^*$4, a lattice is $OR^*$5 for $OR^*$6, and the Gabor system generated by $OR^*$7 is

$OR^*$8

where $OR^*$9, $O^*R$0, and $O^*R$1. The governing symplectic form is determined by

$O^*R$2

and the commutation relations over $O^*R$3 are

$O^*R$4

Thus the antisymmetric matrix $O^*R$5 “completely controls the projective representation structure over $O^*R$6” (Gjertsen et al., 2024).

The central equivalence result is that two lattices $O^*R$7 and $O^*R$8 support identical Gabor structures, up to unitary equivalence, whenever

$O^*R$9

Equivalently, there exists $z$0 with $z$1. If $z$2 is symplectic, there exists a unitary metaplectic operator $z$3 such that

$z$4

and consequently

$z$5

The same source states that, modulo a minor complication related to complex conjugation, symplectic transformations are the only linear structure-preserving maps in Gabor analysis (Gjertsen et al., 2024).

This has direct design implications for optical mask banks. Rotations, scalings, shears, Fourier transforms, chirps, and general linear canonical transforms correspond to symplectic maps, with lenses, quadratic phase plates, and magnification implemented by their metaplectic lifts. Therefore, applying a symplectic transform to the mask lattice can be counteracted by the corresponding optical transform on the window, with exact preservation of frame property and frame bounds (Gjertsen et al., 2024).

Dimensional reduction is one of the paper’s explicit design conclusions. Although a lattice matrix in $z$6 has $z$7 degrees of freedom, only $z$8 independent parameters, namely the entries of the antisymmetric form $z$9, matter for Gabor structure modulo unitary equivalence. For separable lattices $$U(x,y;z)=\mathcal{F}^{-1}\{\mathcal{F}\{I(x,y)\}\cdot H(f_x,f_y;z)\},$$0, the parameter space reduces to the $$U(x,y;z)=\mathcal{F}^{-1}\{\mathcal{F}\{I(x,y)\}\cdot H(f_x,f_y;z)\},$$1 degrees of freedom of $$U(x,y;z)=\mathcal{F}^{-1}\{\mathcal{F}\{I(x,y)\}\cdot H(f_x,f_y;z)\},$$2 (Gjertsen et al., 2024).

For Gaussian windows

$$U(x,y;z)=\mathcal{F}^{-1}\{\mathcal{F}\{I(x,y)\}\cdot H(f_x,f_y;z)\},$$3

the higher-dimensional Lyubarskii–Seip–Wallstén-type theorem gives exact thresholds. In the diagonal separable case $$U(x,y;z)=\mathcal{F}^{-1}\{\mathcal{F}\{I(x,y)\}\cdot H(f_x,f_y;z)\},$$4, the standard Gaussian generates a Gabor frame if and only if $$U(x,y;z)=\mathcal{F}^{-1}\{\mathcal{F}\{I(x,y)\}\cdot H(f_x,f_y;z)\},$$5 for all $$U(x,y;z)=\mathcal{F}^{-1}\{\mathcal{F}\{I(x,y)\}\cdot H(f_x,f_y;z)\},$$6. In the general theorem, if symplectic normalization yields a diagonal block $$U(x,y;z)=\mathcal{F}^{-1}\{\mathcal{F}\{I(x,y)\}\cdot H(f_x,f_y;z)\},$$7, then the Gaussian $$U(x,y;z)=\mathcal{F}^{-1}\{\mathcal{F}\{I(x,y)\}\cdot H(f_x,f_y;z)\},$$8 constructed from the block matrices of $$U(x,y;z)=\mathcal{F}^{-1}\{\mathcal{F}\{I(x,y)\}\cdot H(f_x,f_y;z)\},$$9 is a frame if and only if $R(x,y)$00 for all $R(x,y)$01 (Gjertsen et al., 2024). For optical mask design in $R(x,y)$02, this turns lattice placement into a per-axis symplectic-area constraint rather than a covolume-only condition.

The same framework also treats dual banks. The adjoint lattice is $R(x,y)$03, and Wexler–Raz biorthogonality characterizes when two Bessel Gabor systems form dual masks. This gives a rigorous route to biorthogonal optical Gabor mask banks through sampling on the adjoint lattice (Gjertsen et al., 2024).

6. Two-dimensional Gabor parameterization, learned statistics, applications, and limitations

The elementary two-dimensional optical Gabor mask is the Gaussian-windowed sinusoid. In canonical rotated coordinates,

$R(x,y)$04

or, in analytic form,

$R(x,y)$05

with

$R(x,y)$06

The parameters are amplitude $R(x,y)$07, envelope widths $R(x,y)$08, carrier spatial frequency $R(x,y)$09, carrier phase, orientation, and aspect ratio $R(x,y)$10 (Loxley, 2013).

The statistical learning results reported for natural images emphasize that the parameters with “most pronounced learning” are $R(x,y)$11, $R(x,y)$12, and $R(x,y)$13. Their marginals are heavy-tailed and well fit by Pareto laws, while orientation and phase are approximately uniform. The largest correlation is between $R(x,y)$14 and $R(x,y)$15, reflecting near-constant aspect ratio, followed by correlations of both envelope widths with $R(x,y)$16, indicating size-dependent spatial frequency: larger Gaussian envelopes use longer wavelengths and smaller envelopes use shorter wavelengths (Loxley, 2013).

The paper states that Pareto marginals imply scale invariance and that the receptive-field size distribution and carrier wavelength “follow power laws over wide ranges—no single characteristic size is selected by natural image statistics.” It also states that aspect ratio is weakly constrained, producing three distinct families: $R(x,y)$17 for sharp orientation resolution at the expense of spatial-frequency resolution, $R(x,y)$18 for sharp spatial-frequency resolution at the expense of orientation resolution, and $R(x,y)$19 for unit aspect ratio. The underlying trade-off is consistent with Daugman’s joint space–frequency uncertainty relation,

$R(x,y)$20

with $R(x,y)$21 and $R(x,y)$22 (Loxley, 2013).

For generative design, the best-performing probabilistic model reported is “a Gaussian copula with Pareto marginal probability density functions.” In the unit-aspect-ratio fit, the parameter tuple is

$R(x,y)$23

for $R(x,y)$24 and the copula correlation parameter. The same source states a practical rule consistent with the learned data: $R(x,y)$25, equivalently $R(x,y)$26, with a nearly constant number of carrier cycles under the envelope (Loxley, 2013).

These parameterizations map directly to physical optical implementations. $R(x,y)$27 and $R(x,y)$28 set Gaussian envelope widths by amplitude or phase apodization, $R(x,y)$29 sets mask rotation, $R(x,y)$30 sets grating line spacing, and phase selects quadratures. Real cosine Gabors can be realized as a single amplitude or phase mask with Gaussian apodization multiplied by a sinusoidal grating, whereas complex Gabors can be implemented by cosine and sine quadrature masks combined electronically or interferometrically, or by coherent optics with phase-only SLMs and appropriate detection schemes (Loxley, 2013).

Applications of Optical Gabor Masks in the holographic sense include digital holography and microscopy, inline particle tracking, endoscopy and reflection metrology, and light shaping and CGH. The stated benefits are improved phase reconstructions without twin-image artifacts, single-shot operation, and common-path stability. Reported numerical examples include complex-valued objects such as coin faces, and an experimental digital holography capture of a water droplet in which reconstruction recovers droplet amplitude and phase while the twin image is suppressed (Glückstad, 23 Feb 2026).

The principal limitations are also explicit. Misalignment can cause off-center rings to overlap object support. Phase-shaper imperfections or lens aberrations can distort the ring and may require calibration or adaptive optics. Wavelength drift changes $R(x,y)$31, strong scattering can make the self-interference term more visible, the choice of $R(x,y)$32 and sensor width limits usable field of view, and larger objects require larger sensors or smaller $R(x,y)$33 within the constraint $R(x,y)$34. The approach is nevertheless described as working across visible and NIR with suitable axicon materials and as compatible with digital holographic microscopy and lensless digital holography when the phase shaper is inserted where the spatial-frequency plane is formed (Glückstad, 23 Feb 2026).

Taken together, these formulations place Optical Gabor Masks at the intersection of in-line holography, phase-only Fourier-plane shaping, and lattice-based time–frequency analysis. In one role they are optical elements that displace the twin image into an annulus; in another they are structured multiscale, multi-orientation filter banks whose stability is governed by symplectic invariants and Gaussian frame thresholds.