Stein's Complementary Series Representations

• Stein's complementary series representations are a class of irreducible unitary representations constructed via analytic continuation beyond the tempered principal series.
• They utilize invariant tensor products and intertwining operators to achieve explicit decompositions and branching laws for various reductive groups.
• These representations underpin applications in harmonic analysis, automorphic forms, and the local Langlands program by linking geometric and spectral aspects.

Stein's complementary series representations comprise a fundamental class of irreducible unitary representations of real, complex, and $p$-adic reductive groups, characterized by their construction via analytic continuation "beyond" the unitary principal series and their specific position outside the tempered dual. These representations are induced from parabolic subgroups using non-unitary characters in a regime where the resulting induced representation nevertheless admits a positive-definite invariant Hermitian form. Their analytic, algebraic, and geometric realization enables deep connections to both harmonic analysis and the local Langlands program and has led to generalizations involving global and geometric settings, tensor products, restriction problems, and branching laws.

1. Foundational Construction and Key Examples

Stein's original construction of the complementary series begins with real reductive groups—typically $SL(n,\mathbb{R})$, $GL(n,\mathbb{R})$, or $SO(n,1)$—where one considers induction from a parabolic subgroup $P=MAN$ to $G$. The induced representation

$$\pi_s = \operatorname{Ind}_P^G(\delta_P^s \otimes 1),$$

where $\delta_P$ is the modular character of $P$ and $s\in\mathbb{R}$ is the parameter, is unitary for certain values of $s$ within an open interval containing 0, but outside the tempered range. For example, for $SO_0(n,1)$, the complementary series exist for $0 < s < (n-1)/2$.

In higher-rank groups, such as $GL(2n,\mathbb{R})$ or $SL(2n,\mathbb{C})$, Stein's complementary series are realized as members of "degenerate principal series" induced from one-dimensional characters of a maximal parabolic, typically depending on two complex parameters $(\xi, \lambda)$ with a symmetry or unitarity constraint (such as $\lambda_1 = -\lambda_2 \in (-\frac12, \frac12)$, $\xi_1 = \xi_2$) (Ditlevsen et al., 7 Aug 2025).

In the symplectic case, for the (universal cover of the) symplectic group $Sp(n,\mathbb{R})$, complementary series representations are constructed by inducing from a parabolic with Levi factor $Sp(p,\mathbb{R}) \times GL(q-p)$, using $$\operatorname{Ind}_{Sp(q,\mathbb{R})}[Sp(p,\mathbb{R}) \times GL(q-p) \times N](\sigma \otimes |\det|^t)$$, with $\sigma$ a unitary representation of $Sp(p,\mathbb{R})$ and $t$ in an open non-tempered interval, and by further deploying an explicit intertwining and invariant tensor construction on the Shilov boundary (He, 2010).

2. Invariant Tensor Product and Duality Frameworks

The theory is enhanced by the use of invariant tensor product constructions and dualities, which provide an analytic apparatus to realize and analyze complementary series representations concretely.

The invariant tensor product construction is particularly central in $Sp(n,\mathbb{R})$: given a degenerate principal series $I(\epsilon, t)$ and a Harish–Chandra module $V(T)$ of a unitary representation $T$ of $Sp(p,\mathbb{R})$, the explicit invariant tensor: $$[f \circ_{Sp(p,\mathbb{R})} v](g) = \int_{Sp(p,\mathbb{R})/\mathcal{C}} f(gh)[T(h)v, u]\, d[h]$$ yields the analytic space for the induced representation. Provided the parameter $t$ and leading exponents for $T$ satisfy growth conditions, this integral defines a representation that is unitarizable due to the positive-definiteness properties of the underlying intertwining operator (commuting with the classical Knapp–Stein operator) (He, 2010).

Howe-type duality further structures the model, notably through correspondence between degenerate principal series representations for larger groups and induced representations for smaller subgroups. For example, restricting $I(\epsilon, t)$ from $Sp(p+q,\mathbb{R})$ to $Sp(p,\mathbb{R})\times Sp(q,\mathbb{R})$ yields a "mixed model" on the Shilov boundary and identifies the induced module as $L^2$-spaces over explicitly described homogeneous bundles (He, 2010).

3. Branching Laws and Direct Integral Decompositions

A major theme in recent work is the explicit decomposition ("branching") of Stein's complementary series representations upon restriction to reductive subgroups.

For $GL(2n,\mathbb{R})$, restricting a Stein complementary series $\,\pi_{(\xi,\lambda)}\,$ (with $\lambda_1 = -\lambda_2 \in (-\frac12, 0)$ and $\xi_1 = \xi_2$) to $GL(2n-1,\mathbb{R})$ yields a direct integral decomposition: $$\pi_{(\xi,\lambda)}|_{GL(2n-1,\mathbb{R})} \simeq \bigoplus_{\eta \in \mathbb{Z}/2\mathbb{Z}} \int_{i\mathbb{R}_+} \operatorname{Ind}_{Q_H}^H(\varpi_{(\xi,\lambda)} \otimes \chi_{(\eta,\nu)}) \, d\nu,$$ where $Q_H$ is a maximal parabolic with Levi $GL(2n-2,\mathbb{R}) \times GL(1,\mathbb{R})$, $\varpi_{(\xi,\lambda)}$ is the lower-rank complementary series or Speh representation (depending on the parameter range), and $\chi_{(\eta,\nu)}$ is a character of $GL(1,\mathbb{R})$ (Ditlevsen et al., 7 Aug 2025). The decomposition is realized via two families of symmetry breaking operators (designated $A^{(\eta,\nu)}_{(\xi,\lambda)}$ and $B^{(\eta,\nu)}_{(\xi,\lambda)}$), constructed as explicit H-intertwining operators, with analytic (meromorphic) dependence on the spectral parameters through Gamma functions.

This decomposition is further closely related to the theory of adduced representations, where the branching law is predicted and explained by the structure of the adduced representation—i.e., the lower-rank complementary series or Speh representation that appears as the "core" building block in the restriction.

4. Intertwining Operators, Symmetry Breaking, and Analytic Continuation

A haLLMark of Stein's complementary series theory is the pivotal role played by intertwining operators—especially generalized and normalized Knapp–Stein operators—and their analytic continuation beyond the tempered axis.

In explicitly constructing restrictions and decompositions, central use is made of integral operators of the form: $$B^{(\eta,\nu)}_{(\xi,\lambda)} f (h) = \int_{\mathbb{R}^*} |t|^{\lambda_1-\nu + n/2}_{\xi_1+\eta}\; f(h \cdot \overline{n}_{2n,n}(t)) \frac{dt}{|t|},$$ whose meromorphic properties in $(\lambda,\nu)$ control the analytic continuation from the principal series regime (purely imaginary parameters) to the complementary series range $(\lambda_1 = -\lambda_2 \in (-\frac12, 0))$ (Ditlevsen et al., 7 Aug 2025). These operators, together with their functional intertwining relations to the Knapp–Stein operators, guarantee that the representation-theoretic decompositions remain valid and unitarizable in the region of interest.

The functional equations intertwining these families (for instance, $$S_{(\xi_1,\eta,\xi_2),(\lambda_1,\nu,\lambda_2)} \circ B^{(\eta,\nu)}_{(\xi,\lambda)} = \frac{1}{n_B(\xi,\lambda,\eta,\nu)} \cdot A^{(\eta,\nu)}_{(\xi,\lambda)}$$, with $n_B$ a meromorphic normalizing factor built from Gamma functions) are central to the analytic machinery, dictating the pleroma of reducibility, unitarity, and the decomposition of subquotients (Ditlevsen et al., 7 Aug 2025).

5. Inductive and Hierarchical Structure; Relation to Other Special Representations

The methods of analytic continuation, invariant tensor product, and explicit symmetry breaking allow one to build Stein's complementary series representations and their branching laws inductively. The construction of these series typically proceeds by parabolic induction from lower-rank complementary series or Speh representations, so that the representations for $GL(2n,\mathbb{R})$ are constructed from (adduced) representations on $GL(2n-2,\mathbb{R})$ (Ditlevsen et al., 7 Aug 2025). This recursive structure is reflected in the branching laws and is essential in explaining the composition series and the multiplicity-one property observed for these representations.

This thematic connection is echoed in the relationship of Stein's complementary series with "small" and "minimal" representations, with Speh representations, and with analogous constructions for $SO(p+1,p+1)$, $SL(2n,\mathbb{C})$, and symplectic groups. In all cases, the role of degenerate principal series (induced from characters on a maximal parabolic), analytic continuation, and the use of noncompact or geometric models (e.g., realization on skew-symmetric matrices (Fischer et al., 2011), or on the Shilov boundary (He, 2010)) are critical.

6. Analysis of Reductive and Geometric Generalizations

Generalizations of Stein's complementary series have extended to non-linear and geometric settings. For example, for isometry groups of CAT($-1$) spaces, one realizes complementary series analogues on the boundary at infinity via a dynamical induction approach using Patterson–Sullivan densities; key features such as duality and analytic families of representations persist in this setting (Boucher, 2020). Similarly, the construction of invariant differential operators intertwining Sobolev space realizations on nilpotent radicals has allowed the explicit identification of discrete components in restrictions of complementary series representations for rank-one groups (Möllers et al., 2014).

Furthermore, the tensor product of complementary series representations for rank-one Lie groups decomposes discretely into sums of complementary series indexed by the sum of parameters, via the construction of bilinear differential intertwining operators (Zhang, 2014).

7. Impact, Open Problems, and Connections

Stein's complementary series representations remain central to the harmonic analysis of real and $p$-adic groups, the analysis of automorphic forms, and the fine structure of the unitary dual. Their explicit decomposition via symmetry breaking operators both resolves classical restriction problems (branching laws) and provides new tools for the description of unitary representation spectra (Ditlevsen et al., 7 Aug 2025).

The meromorphic continuation techniques, analytic machinery, and inductive structures developed for these representations have implications for the representation theory of more general groups (including nonlinear and geometric groups), for the explicit realization of Langlands parameters, and for the construction of special functions in automorphic representation theory.

Open problems remain regarding explicit Plancherel formulas for restrictions to other subgroups, the interaction with more general classes of automorphic representations, and the extension of these explicit symmetry breaking and analytic techniques to broader families of representations and groups beyond those of type $GL(n)$ or semisimple rank one.

---

Table: Branching Law for Stein's Complementary Series of $GL(2n, \mathbb{R})$ (Ditlevsen et al., 7 Aug 2025)

Restricted from	Restricted to	Decomposition
$\pi_{(\xi,\lambda)}$ (Stein's complementary series of $GL(2n,\mathbb{R})$)	$GL(2n-1,\mathbb{R})$	$$\bigoplus_{\eta \in \mathbb{Z}/2\mathbb{Z}} \int_{i\mathbb{R}_+} \operatorname{Ind}_{Q_H}^H(\varpi_{(\xi,\lambda)} \otimes \chi_{(\eta,\nu)}) \, d\nu$$

Here $\varpi_{(\xi,\lambda)}$ is the lower-rank complementary series or Speh representation of $GL(2n-2,\mathbb{R})$; $Q_H$ is a maximal parabolic; $\chi_{(\eta,\nu)}$ is a character of $GL(1,\mathbb{R})$.

---

In sum, Stein's complementary series representations exemplify the intricate interplay between analytic continuation, induced representation theory, intertwining operators, and branching laws, both in explicit algebraic settings and in broader geometric and dynamical contexts. Their paper continues to influence the formulation of explicit spectral decompositions, the analysis of automorphic spectra, and the structural understanding of unitary duals for a wide class of groups.