Beyond Endoscopy: Functoriality via Trace Formula

• Beyond Endoscopy is a research paradigm that uses poles and special L-function values in weighted trace formulas to detect functorial transfers in automorphic forms.
• It employs nonstandard test functions and local transfer operators—including Fourier and Hankel transforms—to shift L-weights from the spectral side to the geometric side.
• The framework extends to ramified cases and higher rank groups, offering analytic tools to address convergence, stabilization, and endoscopic contamination challenges.

Beyond Endoscopy denotes a line of research in automorphic forms that seeks to extract, from the trace formula or from related relative trace formulas, automorphic representations characterized by poles or special values of associated $L$-functions, thereby approaching functoriality without relying primarily on endoscopic transfer. In the formulation emphasized by Langlands and later surveys, one fixes a morphism $\iota:{}^{L}H\to{}^{L}G$ and a finite-dimensional complex representation $r:{}^{L}G\to\mathrm{GL}(V)$, and studies weighted trace formulas whose residues at $s=1$ are expected to detect the image of $H$ in $G$; in practice the subject has expanded to include nonstandard test functions, local transfer operators, spherical varieties, Braverman–Kazhdan $r$-Schwartz spaces, and Poisson or Hankel transforms that move $L$-weights from the spectral side to the geometric side (Sakellaridis, 2023).

1. Langlands’s proposal and the weighted trace formula

With endoscopy and stabilization providing a framework to compare trace formulas of related groups, Langlands proposed a different next step: to extract directly, from the stable trace formula of a fixed group $G$, the bulk of those automorphic representations that come from a functorial lift attached to a morphism of $L$-groups $\iota:{}^{L}H\to{}^{L}G$0. In this formulation, poles or special values of $\iota:{}^{L}H\to{}^{L}G$1 at $\iota:{}^{L}H\to{}^{L}G$2 are expected to reflect that the global parameter of $\iota:{}^{L}H\to{}^{L}G$3 factors through the image of $\iota:{}^{L}H\to{}^{L}G$4, so residues at $\iota:{}^{L}H\to{}^{L}G$5 become detection mechanisms for functorial transfer (Sakellaridis, 2023).

A schematic stabilized trace formula is written as

$\iota:{}^{L}H\to{}^{L}G$6

and the Beyond Endoscopy variant inserts $\iota:{}^{L}H\to{}^{L}G$7-weights on the spectral side: $\iota:{}^{L}H\to{}^{L}G$8 The basic task is then to “move” the factor $\iota:{}^{L}H\to{}^{L}G$9, or sometimes its logarithmic derivative $r:{}^{L}G\to\mathrm{GL}(V)$0, to the geometric side by means of nonstandard test functions and analytic transforms. The principal obstructions already appear in this formal expression: the weighted spectral sums are typically outside the naive domain of convergence, Eisenstein series contaminate the residue at $r:{}^{L}G\to\mathrm{GL}(V)$1, and one must separate endoscopic or special contributions from the terms that should represent genuine functorial images (Sakellaridis, 2023).

The program is therefore both spectral and geometric. Spectrally it asks for residues or special values of weighted expansions; geometrically it asks for trace-formula reorganizations in which the relevant arithmetic appears in orbital integrals or in their transforms. This suggests a framework broader than a single recipe, because the same objective leads naturally to stable trace formulas, Kuznetsov formulas, relative trace formulas, and local transfer operators.

2. Nonstandard test functions, transfer operators, and relative trace formulas

A central mechanism in Beyond Endoscopy is the replacement of orbit-by-orbit matching by explicit transforms between spaces of orbital integrals. For $r:{}^{L}G\to\mathrm{GL}(V)$2 an $r:{}^{L}G\to\mathrm{GL}(V)$3-torus with splitting field $r:{}^{L}G\to\mathrm{GL}(V)$4 and $r:{}^{L}G\to\mathrm{GL}(V)$5, Langlands described a local transfer operator $r:{}^{L}G\to\mathrm{GL}(V)$6 satisfying

$r:{}^{L}G\to\mathrm{GL}(V)$7

and, in explicit form, up to a scalar $r:{}^{L}G\to\mathrm{GL}(V)$8,

$r:{}^{L}G\to\mathrm{GL}(V)$9

There is also a Kuznetsov-side variant

$s=1$0

and a Kuznetsov–Selberg transfer for $s=1$1 given by multiplicative convolution with

$s=1$2

In the survey literature these are not isolated formulas but prototypes for a class of local operators built from one-dimensional Fourier convolutions along cocharacters, often with multiplicative correction terms dictated by local gamma factors (Sakellaridis, 2023).

For $s=1$3, the local theory of nonstandard matching between Jacquet’s toric relative trace formula on $s=1$4 and the Kuznetsov formula with nonstandard test functions is made explicit by the operator

$s=1$5

which induces a topological isomorphism

$s=1$6

This local matching is accompanied by a fundamental lemma carrying the torus-side basic vector and its Hecke translates to the corresponding nonstandard Whittaker basic vector and translates. Global Poisson summation for $s=1$7 then yields equality of the two global relative trace formulas and gives a new proof of Waldspurger’s theorem on toric periods (Sakellaridis, 2012, Sakellaridis, 2014).

These constructions are closely tied to a broader local-global architecture. The same survey that frames Beyond Endoscopy as weighted trace formulas also places it alongside Braverman–Kazhdan $s=1$8-Schwartz spaces, $s=1$9-Fourier transforms, IC-functions on monoids, and harmonic analysis on spherical varieties. A common theme is that local transforms encode gamma factors, while global Poisson summation turns those local identities into functional equations, comparisons, or period formulas (Sakellaridis, 2023).

3. The $H$0 elliptic model and Poisson summation

The first sustained implementation of Beyond Endoscopy through the Arthur–Selberg trace formula was carried out for $H$1 over $H$2. Fixing a prime $H$3, an integer $H$4, and a factorizable test function $H$5, the elliptic part of the geometric side is parameterized by trace $H$6 and $H$7. With the specified Haar normalizations one has

$H$8

and, after reorganizing the finite orbital integrals, the elliptic contribution becomes

$H$9

Here the archimedean orbital integral is rewritten in terms of a one-variable function $G$0, but the singular factor $G$1 remains visible at $G$2, which is precisely the analytic difficulty obstructing direct Poisson summation (Altug, 2015).

The decisive step is to insert an approximate functional equation into the geometric side itself. For

$G$3

the truncation parameter is chosen additively as

$G$4

This turns the singular archimedean weight into a smooth one: Proposition 4.1 shows that if $G$5, then both

$G$6

are smooth on $G$7. After splitting the $G$8-sum into residue classes modulo $G$9, Poisson summation yields Kloosterman-type sums

$r$0

together with smooth oscillatory integrals. The transformed expression reorganizes the elliptic side into arithmetic coefficients $r$1 multiplied by rapidly decaying Fourier integrals, precisely the form needed for analytic applications (Altug, 2015).

The $r$2 term isolates the special representations singled out by Langlands. One finds

$r$3

and

$r$4

The elliptic part is therefore transformed into an explicit special-representation piece that can be subtracted off and a remainder expressed as a sum over $r$5 of rapidly decaying oscillatory integrals with controlled arithmetic coefficients. This is the point at which the elliptic geometric side becomes “ready to be used” for Beyond Endoscopy (Altug, 2015).

The subsequent paper on the standard representation finalized this analysis for holomorphic cusp forms of weight $r$6 on $r$7, proving

$r$8

which constitutes the first example of Beyond Endoscopy executed via the Arthur–Selberg trace formula and gives a new proof of the analytic continuation of the $r$9-function attached to Ramanujan’s $L$0-function (Altug, 2015).

At the level of local analytic input, the smoothing mechanism is not restricted to $L$1. It extends to $L$2, and the same paper discusses arbitrary reductive groups: multiplying elliptic orbital integrals by cutoff weights from an approximate functional equation yields smooth functions on the Steinberg–Hitchin base. What remains difficult is not the archimedean smoothing itself but the absence, in higher rank, of $L$3-style explicit formulas for finite orbital integrals and the corresponding lack of control of the dual Fourier transforms after Poisson summation (González et al., 2016).

4. Number fields, ramification, and the move beyond the unramified $L$4 model

The first step beyond $L$5 was to generalize Altuğ’s cancellation of the trivial and special representations to $L$6 for a class of totally real number fields $L$7. Under the assumptions $L$8, $L$9, and $G$0, the regular elliptic part is rewritten in terms of $G$1, a unit $G$2, and a modified Hilbert symbol. After smoothing, completing the lattice, and applying additive Poisson summation in $G$3, the dominant term becomes

$G$4

thereby generalizing Altuğ’s first cancellation step from $G$5 to $G$6 (Emory et al., 2024).

Ramification at a finite set $G$7 with $G$8 introduces new local phenomena, especially mod $G$9 distinctions at the place $L$0. In the ramified $L$1 setting, semilocal Poisson summation on $L$2, a partial Zagier $L$3-function, and a modified local norm $L$4 are used to isolate the same spectral targets inside the elliptic geometric side. The resulting formula is

$L$5

where the sum over $L$6 runs through the global one-dimensional representations ramified only in $L$7. This fully resolves, over $L$8, the problem of isolating these representations in the ramified setting (Cheng, 25 May 2025).

The non-elliptic terms were then analyzed term-by-term in the same ramified setting. After summing over $L$9, the identity and unipotent parts contribute $\iota:{}^{L}H\to{}^{L}G$00, while hyperbolic Poisson summation relates the hyperbolic part back to the spectral side and leads to an $\iota:{}^{L}H\to{}^{L}G$01-dependent identity whose normalized limit yields

$\iota:{}^{L}H\to{}^{L}G$02

This is described as a “limit form of the trace formula” for $\iota:{}^{L}H\to{}^{L}G$03 over $\iota:{}^{L}H\to{}^{L}G$04 with ramification $\iota:{}^{L}H\to{}^{L}G$05 (Cheng, 20 May 2026).

These ramified formulas are not merely structural. Using the isolation of one-dimensional representations, together with non-elliptic estimates and analytic bounds for the remaining elliptic terms, one obtains the trace-level bound

$\iota:{}^{L}H\to{}^{L}G$06

which is presented as a new proof of the $\iota:{}^{L}H\to{}^{L}G$07 bound towards the Ramanujan conjecture for the trace of the cuspidal part in the ramified case (Cheng, 13 Jul 2025).

A further ramified paper analyzes the elliptic part itself by a second Poisson summation, now in the determinant variable, after changing coordinates on the Hitchin–Steinberg base from $\iota:{}^{L}H\to{}^{L}G$08 to $\iota:{}^{L}H\to{}^{L}G$09. It obtains an explicit asymptotic formula for the elliptic contribution, proves the desired limit for the simple trace formula in the ramified case, and derives

$\iota:{}^{L}H\to{}^{L}G$10

thereby generalizing Altuğ’s final result to arbitrary level and nebentype (Cheng, 10 Aug 2025).

Higher rank has now entered the picture as well. For $\iota:{}^{L}H\to{}^{L}G$11, assuming Conjecture A, the regular elliptic part is reformulated in terms of cubic orders attached to characteristic polynomials. A zeta function built from overorders satisfies a completed functional equation, a periodicity theorem shows that the relevant coefficients depend only on the parameters modulo $\iota:{}^{L}H\to{}^{L}G$12, and Poisson summation in two variables produces Kloosterman-type sums and a double Dirichlet series $\iota:{}^{L}H\to{}^{L}G$13. From the evaluation of $\iota:{}^{L}H\to{}^{L}G$14, the trace of the trivial representation appears as a residue, and the contribution of the “special” representation is also recovered (Deng et al., 23 Mar 2026).

5. Other analytic realizations of Beyond Endoscopy

Not all work under the Beyond Endoscopy heading proceeds through the elliptic part of the Arthur–Selberg trace formula. For the Rankin–Selberg $\iota:{}^{L}H\to{}^{L}G$15-function on $\iota:{}^{L}H\to{}^{L}G$16, one can study a product of Kuznetsov formulas and a smoothed sum over integers. In this setting the decisive new structure is a convolution $\iota:{}^{L}H\to{}^{L}G$17 of Bessel transforms such that

$\iota:{}^{L}H\to{}^{L}G$18

and the main limit formula is

$\iota:{}^{L}H\to{}^{L}G$19

This identifies the diagonal Rankin–Selberg pole at $\iota:{}^{L}H\to{}^{L}G$20 and gives, in the author’s phrase, essentially a new proof of analyticity at $\iota:{}^{L}H\to{}^{L}G$21 away from the diagonal (Herman, 2010).

A related but distinct line of work shows that the functional equation itself can be recovered from the trace formula. For $\iota:{}^{L}H\to{}^{L}G$22, Petersson–Kuznetsov manipulations produce Voronoi summation, and Mellin inversion then yields the functional equation of the standard $\iota:{}^{L}H\to{}^{L}G$23-function. A plausible implication is that the trace formula is not only a device for averaging $\iota:{}^{L}H\to{}^{L}G$24-coefficients, but also a source of the analytic symmetries usually obtained by integral representations (Herman, 2012).

This point is made especially explicit in a recent direct proof of Poisson summation on the Whittaker space of $\iota:{}^{L}H\to{}^{L}G$25. There the global Kuznetsov-type functional on nonstandard test half-densities satisfies

$\iota:{}^{L}H\to{}^{L}G$26

where $\iota:{}^{L}H\to{}^{L}G$27 is the local Hankel transform descended from the Fourier transform on $\iota:{}^{L}H\to{}^{L}G$28. Pairing this identity with Whittaker zeta integrals produces the functional equation of the standard $\iota:{}^{L}H\to{}^{L}G$29-function of $\iota:{}^{L}H\to{}^{L}G$30 (Li, 2024).

The complex-archimedean component of Venkatesh’s Beyond Endoscopy for $\iota:{}^{L}H\to{}^{L}G$31 required a Fourier transform formula for regularized Bessel functions on $\iota:{}^{L}H\to{}^{L}G$32. That missing local identity has now been established: Weber–Schafheitlin type formulas over complex numbers yield the Fourier transform of regularized complex Bessel kernels and thereby extend Venkatesh’s $\iota:{}^{L}H\to{}^{L}G$33 analysis from totally real to arbitrary number fields (Qi, 2018).

6. Scope, misconceptions, and open problems

A common misconception is that Beyond Endoscopy is only the single procedure of inserting $\iota:{}^{L}H\to{}^{L}G$34-functions into the stable trace formula of a fixed group. The survey literature instead treats it as a broader constellation of ideas: weighted stable trace formulas, local transfer operators, relative trace formulas on spherical varieties, nonstandard Kuznetsov spaces, Braverman–Kazhdan $\iota:{}^{L}H\to{}^{L}G$35-Schwartz spaces and $\iota:{}^{L}H\to{}^{L}G$36-Fourier transforms, Hankel transforms at the level of quotient trace formulas, and Poisson summation on nonstandard spaces (Sakellaridis, 2023).

The central technical hurdles remain severe. The survey of local and global questions emphasizes convergence and truncation, continuous spectrum and endoscopic contamination, the difficulty of making stabilization compatible with nonstandard test measures, and the open problem of constructing $\iota:{}^{L}H\to{}^{L}G$37-Schwartz spaces and $\iota:{}^{L}H\to{}^{L}G$38-Fourier transforms at all places and in higher rank. It also stresses structural questions about the symplectic or quantization-theoretic origin of the abelian Fourier transforms that repeatedly appear in local transfer operators (Sakellaridis, 2023).

From the concrete side, higher rank still faces arithmetic obstructions beyond the now-standard archimedean smoothing. For $\iota:{}^{L}H\to{}^{L}G$39 and more general reductive groups, the main missing inputs are explicit formulas for $\iota:{}^{L}H\to{}^{L}G$40-adic orbital integrals analogous to the $\iota:{}^{L}H\to{}^{L}G$41 Kronecker-symbol formulas, together with sufficient control of the dual Fourier transforms after Poisson summation. In $\iota:{}^{L}H\to{}^{L}G$42, the current Poisson-summation method is conditional on Conjecture A, because the completed functional equation for the cubic-order zeta function is derived through that conjectural identification (González et al., 2016, Deng et al., 23 Mar 2026).

At the same time, low-rank cases now exhibit a recognizable pattern: special or residual representations are isolated explicitly; the remaining geometric terms are reorganized as smooth oscillatory integrals weighted by arithmetic coefficients; and Poisson or Hankel transforms encode the required local functional equations. This suggests that Beyond Endoscopy is no longer only a heuristic slogan but an analytic toolkit whose operative forms are already visible in $\iota:{}^{L}H\to{}^{L}G$43, in ramified and number-field settings, and—conditionally—in $\iota:{}^{L}H\to{}^{L}G$44 (Altug, 2015, Cheng, 10 Aug 2025, Deng et al., 23 Mar 2026).