Degenerate Whittaker Space

• Degenerate Whittaker space is a generalization of classical Whittaker models where the character is trivial on some root directions, linking representation theory to nilpotent orbits and quantum integrable systems.
• Explicit constructions use induction from parabolic subalgebras and Mackey theory to decompose the space into a direct sum of induced modules with multiplicity-free components.
• The study of degenerate Whittaker spaces advances applications in automorphic forms and harmonic analysis by providing key insights into Fourier coefficients and character computations.

A degenerate Whittaker space refers broadly to the realization, analysis, or application of Whittaker models—spaces of vectors or functionals distinguished by equivariance under a unipotent subgroup with respect to a possibly non-generic (i.e., degenerate) character—in a range of algebraic, geometric, and analytic contexts. Unlike the generic Whittaker model, which is built from nondegenerate characters (i.e., those that are nontrivial on every one-parameter subgroup for each simple root), the degenerate Whittaker space encompasses cases where the character is trivial on some root directions, leading to rich structures parameterized by nilpotent orbits, deeper relationships with geometry, and connections to quantum integrable systems and automorphic forms.

1. Foundational Construction and Algebraic Framework

For a semisimple or reductive Lie algebra $\mathfrak{g}$ (over, e.g., $\mathbb{C}$), the classical nondegenerate Whittaker model is defined using the nilpotent radical $\mathfrak{n}$ of a Borel subalgebra, a fixed nondegenerate character $\psi: \mathfrak{n} \to \mathbb{C}$, and the space of functionals or vectors on which $\mathfrak{n}$ acts via $\psi$. The degenerate Whittaker space generalizes this construction by relaxing the nondegeneracy condition, allowing $\psi$ to be trivial on some components; e.g., $\psi(x_{\alpha})=0$ for certain simple roots $\alpha$.

Degenerate Whittaker spaces emerge in several frameworks:

• For Lie (super)algebras and quantum groups, degenerate Whittaker modules are constructed via induction from parabolic or Levi subalgebras corresponding to the vanishing loci of the character. The resulting spaces can be described in terms of induced modules, as in $$M(\chi, \eta) = \mathrm{Ind}_{\mathfrak{l}_\eta}^{\mathfrak{g}} Y(\chi, \eta)$$, where $\mathfrak{l}_\eta$ is the Levi subalgebra for the character $\eta$ (Brown et al., 2019).
• For algebraic groups, especially over finite fields or rings, the degenerate Whittaker space is realized as a twisted Jacquet module: for $\pi$ a representation of $GL_{2n}(\mathfrak{o}_l)$ and $N$ a unipotent radical, the space $\pi_{N, \psi}$ gathers the vectors with prescribed transformation under $N$ (Parashar et al., 14 Aug 2025).

The degenerate Whittaker space is always a representation of a reductive Levi subgroup (e.g., $GL_n$ or a component thereof) and often decomposes as a direct sum of induced modules, with explicit character formulas relating to the induction data and the choice of character.

2. Geometric and Representation-Theoretic Significance

Degenerate Whittaker spaces are parameterized by nilpotent orbits, which play a central role in connecting representation theory with algebraic and geometric invariants:

• Associated varieties and wave-front sets: For a (g,K)-module $M$ or an admissible representation $T$, the support of the degenerate Whittaker functionals, denoted $\mathcal{Y}(T)$, coincides with the projection of the associated variety to the nilradical, and in favorable cases, matches the wave-front set $WF(T)$, as established for real and complex groups (Gourevitch et al., 2012, Gourevitch et al., 2018). The endpoint is a 'dictionary' between the existence of degenerate Whittaker models, geometric invariants, and nilpotent orbits.
• Nilpotent orbit correspondence: Ginzburg's method directly links degenerate Whittaker coefficients with nilpotent orbits—maximally degenerate Whittaker models correspond to minimal or next-to-minimal orbits. For minimal representations of $SL(3)$, $SL(4)$, and exceptional groups such as $E_6$, $E_7$, $E_8$, the entire (non-constant) Fourier expansion of automorphic forms is determined by maximally degenerate Whittaker vectors (Gustafsson et al., 2014).
• For finite or local fields, the parameters defining which degenerate Whittaker models arise in a given representation are controlled by branching rules, the action of parabolic or Levi subgroups, and, in coverings, the structure of the metaplectic torus and binomial data defining the cover (Axelrod-Freed et al., 2023).

3. Explicit Constructions and Decomposition Methods

Rigorous treatment of degenerate Whittaker spaces for representations induced from parabolic subgroups uses advanced techniques including Mackey theory, double coset analysis, and character computations:

• In $GL_4(\mathfrak{o}_2)$, for representations induced from a maximal parabolic subgroup (the (2,2)-parabolic or (3,1)-parabolic), the degenerate Whittaker space is realized as a direct sum of explicit summands, each associated with double coset representatives contributing nontrivially (Parashar et al., 14 Aug 2025). For example, in the (2,2)-induced representation $$\pi = \mathrm{Ind}_P^{GL_4(\mathfrak{o}_2)}(\pi_1 \otimes \pi_2)$$,

$$\pi_{N, \psi} \cong (\pi_1 \otimes \pi_2) \oplus \mathrm{Ind}_{\mathcal{B}_2}^{GL_2(\mathfrak{o}_2)}(\omega_{\pi_1} \otimes \omega_{\pi_2}) \oplus \mathrm{Ind}_{Z \cdot J_1^2}^{GL_2(\mathfrak{o}_2)}(\omega_{\pi} \cdot \phi_B).$$

• Character computations and dimension formulas for these modules are facilitated by explicit use of congruence subgroups, trace conditions, and local character theory. Induction data from regular (strongly cuspidal) representations determine which summands are present and their multiplicities.

The general principle is that for each suitable double coset, there is a corresponding induced representation in the degenerate Whittaker space, and whose precise constituents and multiplicities are computable using elaborate group-theoretic and character-theoretic methods.

4. Connections to Prasad's Conjecture and Classification Results

Prasad's conjecture proposes a combinatorial and character-theoretic description of degenerate Whittaker models for strongly cuspidal representations of general linear groups over finite rings: specifically, for $\pi$ a strongly cuspidal representation of $GL_{2n}(\mathfrak{o}_l)$, the degenerate Whittaker space $\pi_{N, \psi}$ as a representation of $GL_n(\mathfrak{o}_l)$ is isomorphic to the induced representation from $\mathfrak{O}_l^\times$ (the units of a maximal order in a degree-$2n$ unramified extension), i.e.,

$$\pi_{N,\psi} \cong \mathrm{Ind}_{\mathfrak{O}_l^\times}^{GL_n(\mathfrak{o}_l)}\left( \theta|_{\mathfrak{O}_l^\times} \right).$$

This structure is confirmed for $GL_4(\mathfrak{o}_2)$ in the cuspidal case and is further refined for certain induced representations, where the degenerate Whittaker space may exhaust all regular representations of $GL_2(\mathfrak{o}_2)$ with the appropriate central character or decompose into a sum of induced principal series representations (Parashar et al., 14 Aug 2025).

Multiplicity-freeness of the degenerate Whittaker model in these cases is established—each irreducible constituent appears exactly once—a fact with significant implications for uniqueness and orthogonality of Fourier coefficients and harmonic analysis on finite rings. This aligns with earlier results for representations over finite fields (Gorodetsky et al., 2017).

5. Broader Mathematical Applications and Consequences

The paper and calculation of degenerate Whittaker spaces:

• Enable explicit determination of Fourier coefficients for automorphic forms, especially in settings where generic coefficients vanish either by representation-theoretic constraints (small or minimal representations) or for infinite-dimensional Kac–Moody groups (Gustafsson et al., 2014, Fleig et al., 2013).
• Contribute to the theory of quantum integrable systems and mirror symmetry, with stationary phase integral representations of degenerate Whittaker functions providing integral expressions for equivariant Gromov–Witten invariants of Grassmannians (Oblezin, 2011).
• Underpin analyses of harmonic analysis and unique models for representations over $p$-adic and metaplectic groups, where non-uniqueness and higher-dimensionality of Whittaker models are a central feature (Axelrod-Freed et al., 2023).

These considerations generalize to broader classes of groups, including covering groups, Lie superalgebras (Bagci et al., 2012), and infinite–dimensional or quantum algebras, where intricate connections are established between degenerate Whittaker models, nilpotent orbits, and analytical invariants.

6. Technical Examples and LaTeX Structures

Key expressions for the degenerate Whittaker space include:

• For a representation $\pi$ of a finite matrix group:

$$\pi_{N, \psi} = \{ v \in \pi : \pi(n)v = \psi(n)v \ \forall n \in N \}.$$

• Decomposition of induced representations via Mackey theory:

$$\pi|_P \cong \bigoplus_{\delta \in P \backslash GL_4(\mathfrak{o}_2)/H} \pi^{\delta} \implies \pi_{N,\psi} \cong \bigoplus_{\delta \in \Omega_0} \pi^{\delta}_{N,\psi}.$$

• Realization of the degenerate Whittaker space as a direct sum of induced modules:

$$\pi_{N,\psi} \cong (\pi_1 \otimes \pi_2) \oplus \mathrm{Ind}_{\mathcal{B}_2}^{GL_2(\mathfrak{o}_2)}(\omega_{\pi_1} \otimes \omega_{\pi_2}) \oplus \mathrm{Ind}_{Z \cdot J_1^2}^{GL_2(\mathfrak{o}_2)}(\omega_\pi \cdot \phi_B).$$

• In the context of nilpotent orbit parametrization and functional dimension:

$\operatorname{dim} \pi_{N,\psi} = q^{3}(q-1)$

for $GL_4(\mathfrak{o}_2)$ in a specified setting with $q=|\mathbb{F}_q|$.

7. Outlook and Further Developments

The structure and interplay of degenerate Whittaker spaces with parabolic induction, nilpotent orbits, and harmonic analysis underscore their central role in modern representation theory, automorphic forms, and quantum algebra. Current research directions include extension and explicit computation of these spaces for induced and non-cuspidal representations over general local rings, exploration of their role in categorification and derived contexts, further elucidation of their connections with integrable systems and mirror symmetry, and application to conjectures in the Langlands program.

Degenerate Whittaker spaces thus serve as a nexus between structural representation theory, harmonic analysis, algebraic geometry (via nilpotent orbits, toric degenerations, and total positivity), and arithmetic, helping unify and advance the understanding of symmetry and spectral properties in diverse mathematical and physical arenas.