--- title: 'Beyond Endoscopy: Functoriality via Trace Formula' url: https://www.emergentmind.com/topics/beyond-endoscopy type: topic --- # Beyond Endoscopy: Functoriality via Trace Formula Beyond Endoscopy denotes a line of research in automorphic forms that seeks to extract, from the trace formula or from related relative trace formulas, automorphic representations characterized by poles or special values of associated \(L\)-functions, thereby approaching functoriality without relying primarily on endoscopic transfer. In the formulation emphasized by Langlands and later surveys, one fixes a morphism \(\iota:{}^{L}H\to{}^{L}G\) and a finite-dimensional complex representation \(r:{}^{L}G\to\mathrm{GL}(V)\), and studies weighted trace formulas whose residues at \(s=1\) are expected to detect the image of \(H\) in \(G\); in practice the subject has expanded to include nonstandard test functions, local transfer operators, spherical varieties, Braverman–Kazhdan \(r\)-Schwartz spaces, and Poisson or Hankel transforms that move \(L\)-weights from the spectral side to the geometric side [2310.02438]. ## 1. Langlands’s proposal and the weighted trace formula With endoscopy and stabilization providing a framework to compare trace formulas of related groups, Langlands proposed a different next step: to extract directly, from the stable trace formula of a fixed group \(G\), the bulk of those automorphic representations that come from a functorial lift attached to a morphism of \(L\)-groups \(\iota:{}^{L}H\to{}^{L}G\). In this formulation, poles or special values of \(L^{S}(s,\pi,r)\) at \(s=1\) are expected to reflect that the global parameter of \(\pi\) factors through the image of \(\iota\), so residues at \(s=1\) become detection mechanisms for functorial transfer [2310.02438]. A schematic stabilized trace formula is written as \[ S^{G}(f)\;=\;\sum_{[\gamma]}a_{\gamma}\,O_{\gamma}^{G}(f)\;=\;\sum_{\pi}m(\pi)\,\mathrm{tr}\,\pi(f)\;+\;\text{(continuous terms)}, \] and the Beyond Endoscopy variant inserts \(L\)-weights on the spectral side: \[ S_{r}^{G}(f;s):=\sum_{\pi}L^{S}(s,\pi,r)\,\mathrm{tr}\,\pi(f). \] The basic task is then to “move” the factor \(L^{S}(s,\pi,r)\), or sometimes its logarithmic derivative \(-\frac{L'}{L}\), to the geometric side by means of nonstandard test functions and analytic transforms. The principal obstructions already appear in this formal expression: the weighted spectral sums are typically outside the naive domain of convergence, Eisenstein series contaminate the residue at \(s=1\), and one must separate endoscopic or special contributions from the terms that should represent genuine functorial images [2310.02438]. The program is therefore both spectral and geometric. Spectrally it asks for residues or special values of weighted expansions; geometrically it asks for trace-formula reorganizations in which the relevant arithmetic appears in orbital integrals or in their transforms. This suggests a framework broader than a single recipe, because the same objective leads naturally to stable trace formulas, Kuznetsov formulas, relative trace formulas, and local transfer operators. ## 2. Nonstandard test functions, transfer operators, and relative trace formulas A central mechanism in Beyond Endoscopy is the replacement of orbit-by-orbit matching by explicit transforms between spaces of orbital integrals. For \(H=T\) an \(F\)-torus with splitting field \(E/F\) and \(G=\mathrm{SL}_{2}\), Langlands described a local transfer operator \(\mathcal{T}_{\iota}\) satisfying \[ \Theta_{\phi}\circ\mathcal{T}_{\iota}=\Theta_{\iota_{*}\phi}, \] and, in explicit form, up to a scalar \(\lambda(\eta,\psi)\), \[ \mathcal{T}_{\iota}(f)(t)=dt\cdot\lambda(\eta,\psi)\int_{u,v\in F}\eta(uv)\,\psi\big(v\,x(t)-uv\big)\,\Phi(u)\,du\,dv. \] There is also a Kuznetsov-side variant \[ \frac{\mathcal{T}_{\iota}(f)(t)}{dt}=\lambda(\eta,\psi)\int_{y\in F}\Phi\!\left(\frac{x(t)}{y}\right)\eta\big(x(t)y\big)|y|^{-1}\psi(y)\,dy, \] and a Kuznetsov–Selberg transfer for \(\mathrm{SL}_{2}\) given by multiplicative convolution with \[ D_{1}(x)=\psi(x)\,|x|\,d^{\times}x,\qquad \mathcal{T}(f)=D_{1}\star f. \] In the survey literature these are not isolated formulas but prototypes for a class of local operators built from one-dimensional Fourier convolutions along cocharacters, often with multiplicative correction terms dictated by local gamma factors [2310.02438]. For \(G=\mathrm{PGL}_{2}\), the local theory of nonstandard matching between Jacquet’s toric relative trace formula on \(T\backslash G/T\) and the Kuznetsov formula with nonstandard test functions is made explicit by the operator \[ \mathcal{G}=\mathcal{F}\circ\iota\circ\mathcal{F},\qquad \iota(f)(x)=\frac{\eta(x)}{|x|}\,f\!\left(\frac{1}{x}\right), \] which induces a topological isomorphism \[ |\bullet|\cdot\mathcal{G}:\ \mathcal{S}(\mathcal{Z})\xrightarrow{\sim}\mathcal{S}(\mathcal{W}). \] This local matching is accompanied by a fundamental lemma carrying the torus-side basic vector and its Hecke translates to the corresponding nonstandard Whittaker basic vector and translates. Global Poisson summation for \(\mathcal{G}\) then yields equality of the two global relative trace formulas and gives a new proof of Waldspurger’s theorem on toric periods [1207.5761, 1402.3524]. These constructions are closely tied to a broader local-global architecture. The same survey that frames Beyond Endoscopy as weighted trace formulas also places it alongside Braverman–Kazhdan \(r\)-Schwartz spaces, \(r\)-Fourier transforms, IC-functions on monoids, and harmonic analysis on spherical varieties. A common theme is that local transforms encode gamma factors, while global Poisson summation turns those local identities into functional equations, comparisons, or period formulas [2310.02438]. ## 3. The \(\mathrm{GL}_{2}/\mathbb{Q}\) elliptic model and Poisson summation The first sustained implementation of Beyond Endoscopy through the Arthur–Selberg trace formula was carried out for \(G=\mathrm{GL}_{2}\) over \(\mathbb{Q}\). Fixing a prime \(p\), an integer \(k\ge 0\), and a factorizable test function \(f_{p,k}\), the elliptic part of the geometric side is parameterized by trace \(m_{Y}=\mathrm{tr}(Y)\) and \(N_{Y}=4\det(Y)\). With the specified Haar normalizations one has \[ \mathrm{vol}(Y)=|D_Y|^{1/2}L(1,\chi_{D_Y}), \] and, after reorganizing the finite orbital integrals, the elliptic contribution becomes \[ E(f_{p,k})=\sum_{m\in\mathbb{Z}}'\phi\!\left(\frac{m}{2p^{k/2}}\right)\sum_{f^2\mid (m^2\mp 4p^k)}'L\!\left(1,\chi_{(m^2\mp 4p^k)/f^2}\right). \] Here the archimedean orbital integral is rewritten in terms of a one-variable function \(\phi(x)\), but the singular factor \(|x^2\mp1|^{-1/2}\) remains visible at \(x=\pm1\), which is precisely the analytic difficulty obstructing direct Poisson summation [1506.02218]. The decisive step is to insert an approximate functional equation into the geometric side itself. For \[ L(1,d)=\sum_{\ell=1}^{\infty}\chi_d(\ell)\ell^{-1}F(\ell/A)+A^{-1}\sum_{\ell=1}^{\infty}\chi_d(\ell)H_{\epsilon_d}(\ell A^{-1}), \] the truncation parameter is chosen additively as \[ A:=|m^2\mp4p^k|^{\alpha},\qquad 0<\alpha<1. \] This turns the singular archimedean weight into a smooth one: Proposition 4.1 shows that if \(\psi\in S(\mathbb{R})\), then both \[ x\mapsto \phi(x)\psi(|1-x^2|^{-\alpha}) \quad\text{and}\quad x\mapsto |1-x^2|^{-1/2}\phi(x)\psi(|1-x^2|^{-\alpha}) \] are smooth on \(\mathbb{R}\). After splitting the \(m\)-sum into residue classes modulo \(4\ell f^2\), Poisson summation yields Kloosterman-type sums \[ K_{\ell,f}(\xi,n):=\sum_{a\ (\mathrm{mod}\ 4\ell f^2)}' \chi_{(a^2\mp4n)/f^2}\,e\!\left(\frac{a\xi}{4\ell f^2}\right), \] together with smooth oscillatory integrals. The transformed expression reorganizes the elliptic side into arithmetic coefficients \(K_{\ell,f}(\xi,p^k)\) multiplied by rapidly decaying Fourier integrals, precisely the form needed for analytic applications [1506.02218]. The \(\xi=0\) term isolates the special representations singled out by Langlands. One finds \[ \mathrm{tr}(1(f_{p,k}))=2p^{k/2}(1-p^{-(k+1)})(1-p^{-1})\int_{\mathbb{R}}\phi(x)\,dx, \] and \[ \mathrm{tr}(\mathcal{E}_{0}(f_{p,k}))=(k+1)\int_{x^2\mp1>0}\frac{\phi(x)}{\sqrt{|x^2\mp1|}}\,dx. \] The elliptic part is therefore transformed into an explicit special-representation piece that can be subtracted off and a remainder expressed as a sum over \(\xi\neq0\) of rapidly decaying oscillatory integrals with controlled arithmetic coefficients. This is the point at which the elliptic geometric side becomes “ready to be used” for Beyond Endoscopy [1506.02218]. The subsequent paper on the standard representation finalized this analysis for holomorphic cusp forms of weight \(k\ge 3\) on \(\mathrm{GL}_{2}\), proving \[ \sum_{n