Compactly supported Gabor orthonormal bases

Abstract: We characterize all lattices $Λ\subset \mathbb{R}<sup>2$ and all compactly supported functions $g \in L<sup>2(\mathbb{R})$ for which the Gabor system $\left { e^{2πi s x} g(x-t) : (t,s) \in Λ\right }$ forms an orthonormal basis for $L<sup>2(\mathbb{R})$. The characterization is given in geometric terms through translation tilings and discreteness properties of lattice projections. In particular, this resolves a conjecture of Han and Wang on the non-existence of Gabor bases along specific irrational lattices. Finally, we construct Gabor bases that cannot be realized by any product set, answering a problem of Iosevich and Mayeli.

• The paper completely characterizes lattice-based Gabor orthonormal bases with compactly supported windows, showing existence exactly when the lattice has density 1 and a discrete first-coordinate projection.
• It proves that every admissible window has magnitude a scaled indicator function of a bounded measurable set that tiles the real line, while irrational shear lattices cannot support compactly supported Gabor bases.
• It answers the Iosevich–Mayeli question affirmatively by using fractional Fourier transforms to construct a Gabor basis with no product-set realization, necessarily employing a window with unbounded support.

This paper by Lukas Liehr resolves two long-standing problems concerning Gabor orthonormal bases with compactly supported windows: it proves a conjecture of Han and Wang on the non-existence of such bases along irrational lattices, and it answers a question of Iosevich and Mayeli by constructing Gabor bases that cannot be realized along any product set (2605.29984). The main contribution is a complete geometric characterization of all lattices $\Lambda \subset \mathbb{R}^2$ and all compactly supported windows $g \in L^2(\mathbb{R})$ for which the Gabor system $$\mathbf{G}(g,\Lambda) = \{e^{2\pi i s x} g(x-t) : (t,s)\in\Lambda\}$$ forms an orthonormal basis (ONB) for $L^2(\mathbb{R})$. This problem is often called the Fuglede–Gabor problem, in analogy with Fuglede's spectral set problem.

Main characterization

The central result gives necessary and sufficient conditions on the lattice alone. For every full-rank lattice $\Lambda \subset \mathbb{R}^2$, there exists a compactly supported $g$ such that $\mathbf{G}(g,\Lambda)$ is an ONB if and only if:

1. the lattice density satisfies $D(\Lambda)=1$, and
2. the projection onto the first coordinate $\pi_1(\Lambda)$ equals $a\mathbb{Z}$ for some $g \in L^2(\mathbb{R})$0, i.e., is discrete.

The density condition was already known to be necessary; the discreteness of the projection is the new obstruction, and it is sharp. When both conditions hold, a second theorem characterizes all admissible windows: $g \in L^2(\mathbb{R})$1 is an ONB exactly when

$g \in L^2(\mathbb{R})$2

for a bounded measurable set $g \in L^2(\mathbb{R})$3 that tiles the line by $g \in L^2(\mathbb{R})$4-translates. Thus, remarkably, the admissible windows are determined up to phase by measurable translational tilings — no smoothness or regularity of $g \in L^2(\mathbb{R})$5 beyond compact support enters the characterization. In particular, the classical system $g \in L^2(\mathbb{R})$6 is recovered as the special case $g \in L^2(\mathbb{R})$7, $g \in L^2(\mathbb{R})$8.

An immediate consequence is the resolution of the Han–Wang conjecture: for the shear lattice $g \in L^2(\mathbb{R})$9 with $$\mathbf{G}(g,\Lambda) = \{e^{2\pi i s x} g(x-t) : (t,s)\in\Lambda\}$$0 and $$\mathbf{G}(g,\Lambda) = \{e^{2\pi i s x} g(x-t) : (t,s)\in\Lambda\}$$1, one has $$\mathbf{G}(g,\Lambda) = \{e^{2\pi i s x} g(x-t) : (t,s)\in\Lambda\}$$2, which is dense in $$\mathbf{G}(g,\Lambda) = \{e^{2\pi i s x} g(x-t) : (t,s)\in\Lambda\}$$3. Hence no compactly supported window generates an ONB along $$\mathbf{G}(g,\Lambda) = \{e^{2\pi i s x} g(x-t) : (t,s)\in\Lambda\}$$4. Since Bekka's result guarantees some (necessarily non-compactly-supported) window exists for every lattice, the conjecture isolates compact support as precisely the property destroyed by irrational shears.

Proof strategy for the Han–Wang conjecture

The proof proceeds by contradiction and combines harmonic analysis on fibres with a complex-analytic rigidity argument. Assuming $$\mathbf{G}(g,\Lambda) = \{e^{2\pi i s x} g(x-t) : (t,s)\in\Lambda\}$$5 is an ONB with $$\mathbf{G}(g,\Lambda) = \{e^{2\pi i s x} g(x-t) : (t,s)\in\Lambda\}$$6 supported in a compact set $$\mathbf{G}(g,\Lambda) = \{e^{2\pi i s x} g(x-t) : (t,s)\in\Lambda\}$$7, the author introduces a weighted Zak-type transform

$$\mathbf{G}(g,\Lambda) = \{e^{2\pi i s x} g(x-t) : (t,s)\in\Lambda\}$$8

interpreted as an $$\mathbf{G}(g,\Lambda) = \{e^{2\pi i s x} g(x-t) : (t,s)\in\Lambda\}$$9-Fourier series in $L^2(\mathbb{R})$0. Orthonormality forces the autocorrelations of the coefficient sequence to be Kronecker deltas, whence $L^2(\mathbb{R})$1 almost everywhere, together with the quasi-periodicity relation

$L^2(\mathbb{R})$2

The key structural lemma shows that for almost every fixed $L^2(\mathbb{R})$3, the distributional Fourier transform of $L^2(\mathbb{R})$4 is supported in $L^2(\mathbb{R})$5: this follows because testing against $L^2(\mathbb{R})$6 yields Fourier coefficients proportional to inner products of $L^2(\mathbb{R})$7 with translates of $L^2(\mathbb{R})$8, which vanish by disjointness of supports; a separability argument removes the dependence of the null set on the test function.

A second ingredient is a rigidity lemma of independent interest: if $L^2(\mathbb{R})$9 is unimodular almost everywhere and its distributional Fourier transform has compact support, then $\Lambda \subset \mathbb{R}^2$0 for some unimodular constant $\Lambda \subset \mathbb{R}^2$1 and some frequency $\Lambda \subset \mathbb{R}^2$2 in the support of $\Lambda \subset \mathbb{R}^2$3. The proof uses Paley–Wiener–Schwartz to represent $\Lambda \subset \mathbb{R}^2$4 as an entire function of exponential type, forms the entire function $\Lambda \subset \mathbb{R}^2$5, which equals $\Lambda \subset \mathbb{R}^2$6 on $\Lambda \subset \mathbb{R}^2$7 and hence identically, so $\Lambda \subset \mathbb{R}^2$8 is zero-free; Hadamard factorization then forces $\Lambda \subset \mathbb{R}^2$9 to be a pure exponential.

Combining these, one obtains $g$0 with measurable $g$1 and $g$2 (measurability extracted via injectivity of $g$3 on the compact set $g$4). The quasi-periodicity relation then implies $g$5 almost everywhere. Here irrationality enters decisively: since $g$6 is bounded and measurable and invariant under the irrational rotation by $g$7, ergodicity forces $g$8 to be constant. The remaining functional equation for $g$9 becomes $\mathbf{G}(g,\Lambda)$0, whose Fourier coefficients satisfy $\mathbf{G}(g,\Lambda)$1 for all $\mathbf{G}(g,\Lambda)$2; any nonzero coefficient would violate square-summability, so $\mathbf{G}(g,\Lambda)$3, contradicting $\mathbf{G}(g,\Lambda)$4 almost everywhere. The contradiction establishes non-existence.

Reduction to the standard lattice

With the Han–Wang conjecture in hand, the general classification follows from a chirp-modulation/dilation invariance: for $\mathbf{G}(g,\Lambda)$5, the unitary $\mathbf{G}(g,\Lambda)$6 maps $\mathbf{G}(g,\Lambda)$7 to $\mathbf{G}(g,\Lambda)$8 while preserving compact support. If $\mathbf{G}(g,\Lambda)$9 and $D(\Lambda)=1$0 is not discrete, the generator matrix factors as a lower triangular matrix times the shear matrix defining $D(\Lambda)=1$1, reducing to the irrational case. Conversely, if $D(\Lambda)=1$2 is discrete, a unimodular change of basis brings $D(\Lambda)=1$3 into lower triangular form $D(\Lambda)=1$4 with $D(\Lambda)=1$5, and the chirp invariance reduces the problem to the integer lattice, where the classical characterization due to Liu applies: $D(\Lambda)=1$6 is an ONB if and only if $D(\Lambda)=1$7 for a bounded measurable set tiling $D(\Lambda)=1$8 by integer translates. Pulling back through $D(\Lambda)=1$9 yields the stated window characterization.

Non-product Gabor bases and the Iosevich–Mayeli question

Iosevich and Mayeli asked whether there exist $\pi_1(\Lambda)$0 and $\pi_1(\Lambda)$1 such that $\pi_1(\Lambda)$2 is an ONB while $\pi_1(\Lambda)$3 fails to be an ONB for every choice of sets $\pi_1(\Lambda)$4. The preceding classification shows that compactly supported windows cannot furnish such an example in the lattice setting: whenever $\pi_1(\Lambda)$5 is an ONB with $\pi_1(\Lambda)$6 compactly supported, the same $\pi_1(\Lambda)$7 works along the product set $\pi_1(\Lambda)$8.

The paper nevertheless answers the question affirmatively using fractional Fourier transforms. Let $\pi_1(\Lambda)$9 and $a\mathbb{Z}$0 with $a\mathbb{Z}$1. Since the fractional Fourier transform diagonalizes on the Hermite basis and implements rotation of the time-frequency plane, $a\mathbb{Z}$2 is an ONB. Suppose $a\mathbb{Z}$3 were also an ONB for some $a\mathbb{Z}$4. Rotating back via Lemma (fractional invariance), $a\mathbb{Z}$5 would be an ONB, so by the Gabardo–Lai–Wang structure theory, $a\mathbb{Z}$6 must tile $a\mathbb{Z}$7, forcing $a\mathbb{Z}$8 to equal one of two explicit "staircase" sets of upper Beurling density $a\mathbb{Z}$9. But the tiling structure forces $g \in L^2(\mathbb{R})$00 and $g \in L^2(\mathbb{R})$01 each to lie inside arithmetic progressions with spacings $g \in L^2(\mathbb{R})$02 and $g \in L^2(\mathbb{R})$03 respectively, giving

$g \in L^2(\mathbb{R})$04

contradicting the required density $g \in L^2(\mathbb{R})$05. Hence no product realization exists. Note that $g \in L^2(\mathbb{R})$06 necessarily has unbounded support when $g \in L^2(\mathbb{R})$07, as follows from the integral representation of $g \in L^2(\mathbb{R})$08; this is consistent with, and forced by, the compact-support classification above.

Limitations and open questions

The characterization is specific to the lattice setting in $g \in L^2(\mathbb{R})$09; extensions to higher dimensions, to non-lattice countable index sets, or to frames rather than orthonormal bases are not addressed. The window characterization permits arbitrary measurable tiling sets $g \in L^2(\mathbb{R})$10, so it does not classify regular (e.g., interval-based or continuous) windows within the admissible class. The construction answering the Iosevich–Mayeli question relies essentially on unbounded support, and the paper leaves open whether non-product examples can exist with windows of controlled decay. Finally, the rigidity lemma on compactly supported Fourier transforms of unimodular functions is proved only in one dimension, and its applicability in several variables is not discussed.

Conclusion

The paper delivers a complete solution to the Fuglede–Gabor problem for compactly supported windows over lattices in $g \in L^2(\mathbb{R})$11: existence holds exactly for unit-density lattices with discrete first projection, and the admissible windows are precisely scaled indicators of measurable translational tiling sets. The proof of the Han–Wang conjecture introduces a weighted Zak transform combined with a distributional Paley–Wiener and Hadamard factorization argument, yielding a rigidity principle for unimodular functions with compactly supported spectra that may prove useful elsewhere. The affirmative answer to the Iosevich–Mayeli question via rotated integer lattices demonstrates that product-set realizations fail in general, but only outside the compactly supported class.