Whittaker Models for Reductive Groups

• Whittaker models are concrete realizations of reductive group representations using equivariant functionals on unipotent or nilpotent subgroups.
• They connect representation theory with nilpotent orbits, associated varieties, and microlocal geometry to enable explicit orbit-by-orbit classification.
• Applications span automorphic forms, harmonic analysis, and Langlands correspondences, providing practical tools for analyzing Fourier coefficients and derivative structures.

A Whittaker model for a reductive group is a concrete, highly structured realization of representations in terms of equivariance (or functionals) with respect to unipotent or nilpotent subgroups and their characters. In the context of real or $p$-adic (and, in modern developments, automorphic or geometric) representation theory, Whittaker models play a central role in classifying representations, connecting representation-theoretic and algebraic-geometric invariants, and underpinning harmonic analysis and Langlands correspondences. For quasi-split real reductive groups, the theory involves deep connections between nilpotent orbits, associated varieties, coinvariants, and microlocal geometry.

1. Definitions and Setup

Let $G$ be a real quasi-split reductive Lie group admitting a Borel subgroup $B=TA\cdot N$ defined over $\mathbb{R}$, with maximal compact subgroup $K$, Cartan involution $\theta$, and real Lie algebras $\mathfrak{g}_0$, $\mathfrak{k}_0$ (and complexifications $\mathfrak{g}$, $\mathfrak{k}$). The unipotent radical $G$0 has Lie algebra $G$1, $G$2.

A Whittaker character is a unitary character $G$3 such that $G$4 has finite stabilizer in the complexified Cartan. Equivalently, $G$5 corresponds to a functional $G$6 (where $G$7, $G$8) whose $G$9-orbit ($B=TA\cdot N$0) is open.

More generally, a degenerate Whittaker character is any unitary character $B=TA\cdot N$1 (unitary on $B=TA\cdot N$2), with associated $B=TA\cdot N$3. The orbit type of the differential is governed by the nilpotent orbit of $B=TA\cdot N$4 in $B=TA\cdot N$5.

Given a nilpotent coadjoint orbit $B=TA\cdot N$6, a character $B=TA\cdot N$7 is said to be of type $B=TA\cdot N$8 if there is $B=TA\cdot N$9 with $\mathbb{R}$0, where $\mathbb{R}$1 is restriction.

2. Algebraic and Geometric Structure of Whittaker Models

For a smooth admissible Fréchet representation $\mathbb{R}$2 of moderate growth, Whittaker functionals for any character $\mathbb{R}$3 of $\mathbb{R}$4 are defined by

$\mathbb{R}$5

On the Harish–Chandra module side, for $\mathbb{R}$6 and $\mathbb{R}$7, one writes $\mathbb{R}$8.

The annihilator variety $\mathbb{R}$9 is the support of the graded annihilator in the universal enveloping algebra, always a union of nilpotent $K$0-orbits. The associated variety $K$1 is the support of the associated graded under a good filtration. There is always $K$2 (nilpotent cone).

The wave–front set $K$3 (the support of the Fourier transform of the distribution character) relates to $K$4 via the Kostant–Sekiguchi bijection $K$5, with

$K$6

3. Main Theorems and Orbit-Associated Classifications

Theorem A (Harish–Chandra modules)

Let $K$7 be an admissible $K$8-module. For any character $K$9 of $\theta$0 with differential $\theta$1, one has

$\theta$2

Equivalently, if $\theta$3, then

$\theta$4

Theorem B (Smooth vectors)

Let $\theta$5 and $\theta$6. Define $\theta$7. Then

$\theta$8

and for $\theta$9 or any complex group, there is equality: $\mathfrak{g}_0$0 Moreover, $\mathfrak{g}_0$1 is the projection of $\mathfrak{g}_0$2.

Specializations

• $\mathfrak{g}_0$3 is generic $\mathfrak{g}_0$4 it admits a Whittaker functional for some non-degenerate $\mathfrak{g}_0$5; then $\mathfrak{g}_0$6 is the full nilpotent cone (Kostant). Levi-generic functionals (Matumoto) generalize to other nilradicals.

4. Proof Techniques and Filtration Methods

The algebraic core is the analysis of filtrations and coinvariants:

• Any good $\mathfrak{g}_0$7-filtration on $\mathfrak{g}_0$8 remains good for $\mathfrak{g}_0$9, so $\mathfrak{k}_0$0.
• Passing to $\mathfrak{k}_0$1 (commutative), use coinvariants $\mathfrak{k}_0$2 and Nakayama lemma to describe the support as $\mathfrak{k}_0$3.
• For the converse, use Casselman’s embedding into $\mathfrak{k}_0$4-adic completion and the Jacquet functor (formally as nearby cycles on the flag variety, in the sense of Emerton–Nadler–Vilonen). Ginzburg's theory of characteristic varieties and vanishing cycles shows the support of $\mathfrak{k}_0$5 contains the projected associated variety.

Reduction to the generic case proceeds via the Kostant–Sekiguchi bijection and parabolic descent: any degenerate character becomes non-degenerate on a Levi, allowing a reduction to the large orbit case.

5. Explicit Examples in Classical Groups

$\mathfrak{k}_0$6

• $\mathfrak{k}_0$7 strictly upper-triangular matrices; nilpotent orbits in $\mathfrak{k}_0$8 correspond to partitions $\mathfrak{k}_0$9.
• A degenerate Whittaker model of type $\mathfrak{g}$0 corresponds to a character $\mathfrak{g}$1 whose annihilator partition is $\mathfrak{g}$2.
• If $\mathfrak{g}$3 is principal series, $\mathfrak{g}$4 can be computed explicitly; $\mathfrak{g}$5 iff the closure of the orbit of $\mathfrak{g}$6 lies in $\mathfrak{g}$7.
• For discrete series of $\mathfrak{g}$8, $\mathfrak{g}$9 is the principal (regular) nilpotent orbit; only the non-degenerate character yields a nonzero Whittaker model.

Other Classical Groups

Identical orbit-parametrization by partitions applies, with parity constraints for $\mathfrak{k}$0 and $\mathfrak{k}$1; the same closure criterion for non-vanishing of Whittaker models holds.

6. Applications and Implications

• Classification of Representations: The types of nonzero Whittaker functionals on $\mathfrak{k}$2 are in bijection with the nilpotent orbits occurring in the associated variety. Large (generic) representations admit the full (non-degenerate) Whittaker model; the minimal orbit corresponds to the most degenerate nonzero Whittaker functional.
• Automorphic Forms: For local constituents of automorphic cuspidal representations, only those Fourier coefficients along $\mathfrak{k}$3 corresponding to local associated varieties have nonzero values.
• Derivatives and Composition Series: For $\mathfrak{k}$4, Bernstein–Zelevinsky derivatives $\mathfrak{k}$5 satisfy $\mathfrak{k}$6 corresponds to removing a row (or column) of size $\mathfrak{k}$7 from the partition for $\mathfrak{k}$8; vanishing occurs exactly when $\mathfrak{k}$9 exceeds the largest Jordan block.
• Dictionary and Orbit-by-Orbit Correspondence: Theorems A and B provide an orbitwise bijective dictionary between the types of degenerate Whittaker functionals and pieces of $G$00; this is bijective for all classical groups, with explicit counterexamples classified in the exceptional series.

7. Contextual Significance and Further Directions

The precise linkage between existence of Whittaker functionals (including degenerate cases) and the algebraic–microlocal invariants of a representation (associated variety, wave-front set) enables concrete descriptions of representation families, classification of "small" (minimal support) representations, and dictates which Fourier coefficients occur in automorphic theory. The techniques provide an extensible framework across real, $G$01-adic, and automorphic settings, with geometric extensions (via $G$02-modules and characteristic cycles) further connecting harmonic analysis, representation theory, and the geometric Langlands program. These results generalize and systematize earlier work of Kostant (1968), Matumoto, Gabber–Osborne, Schmid–Vilonen, Ginzburg, and others, forming the foundation for the orbit-by-orbit theory of Whittaker models for reductive groups (Gourevitch et al., 2012).