Asymptotics of Hecke polynomial coefficients on the Atkin-Lehner eigenspaces

Abstract: Let $S_k<sup>σ(N)$ denote the space of cusp forms of level $N$, weight $k$, and Atkin-Lehner sign pattern $σ$, and $S_k<sup>{\mathrm{new},σ}$ denote its new subspace. In this paper, we study the asymptotic behavior of the coefficients of the $m$-th Hecke polynomial over $S_k<sup>σ(N)$ and $S_k<sup>{\mathrm{new},σ}(N)$. In particular, we show that in certain settings, all but finitely many of these coefficients take a particular sign. We also study settings in which the coefficients do not tend to any particular sign.

• The paper demonstrates asymptotics for Hecke polynomial coefficients on Atkin-Lehner eigenspaces.
• Key results extend the study of coefficients by Ross-Xue, providing for all fixed-index coefficients when m is a square as both N and k approach infinity.
• For non-square m, the error exponent and explicit sign results are highlighted, including different sign behaviors for even and odd-indexed coefficients.

Overview

The paper studies the coefficients $c_r(m,N,k,\sigma)$ of the characteristic polynomial of the normalized Hecke operator $T_m' = m^{-(k-1)/2}T_m$ restricted to the Atkin–Lehner sign-pattern eigenspaces $S_k^\sigma(N)$, and to the corresponding newspaces $S_k^{\mathrm{new},\sigma}(N)$. Here $\sigma$ ranges over multiplicative functions on the exact divisors $Q \parallel N$ taking values in $\{\pm 1\}$; these spaces refine the classical Fricke eigenspaces $S_k^\pm(N)$. The authors extend the program of Ross–Xue on Hecke polynomial coefficients over full spaces $S_k(N)$ and newspace $S_k^{\mathrm{new}}(N)$ (2608.18497), and their main contributions are: asymptotics for all fixed-index coefficients when $T_m' = m^{-(k-1)/2}T_m$0 is a square (jointly as $T_m' = m^{-(k-1)/2}T_m$1); asymptotics for non-square $T_m' = m^{-(k-1)/2}T_m$2 (in weight only); explicit effective sign results for the second coefficient; a construction showing that the natural analogue in level fails for non-square $T_m' = m^{-(k-1)/2}T_m$3, via two infinite families of pairs $T_m' = m^{-(k-1)/2}T_m$4 with second coefficients of opposing sign; and an explicit trace formula with fully computed constants, valid over both full and new sign-pattern spaces.

The technical foundation is an explicit version of the trace formula of Ross–van Lidth de Jeude–Wolf–Xue for $T_m' = m^{-(k-1)/2}T_m$5:

$T_m' = m^{-(k-1)/2}T_m$6

The proof combines the Skoruppa–Zagier trace formula for $T_m' = m^{-(k-1)/2}T_m$7 (with Assaf's correction), careful case analysis of the class-number terms (including an appendix on Hurwitz–Kronecker class number bounds), and character orthogonality of the $T_m' = m^{-(k-1)/2}T_m$8. Unlike prior work that gave only big-$T_m' = m^{-(k-1)/2}T_m$9 estimates, all constants here are effective, which is what enables the explicit sign thresholds.

Asymptotics for square index

When $S_k^\sigma(N)$0 is a perfect square, the trace has a linear main term in $S_k^\sigma(N)$1 and $S_k^\sigma(N)$2, and the authors prove by strong induction on $S_k^\sigma(N)$3 using Newton–Girard identities that

$S_k^\sigma(N)$4

Since $S_k^\sigma(N)$5, this immediately yields a definitive sign statement: for every fixed $S_k^\sigma(N)$6 and square $S_k^\sigma(N)$7, the coefficient $S_k^\sigma(N)$8 has sign $S_k^\sigma(N)$9 for all but finitely many triples $S_k^{\mathrm{new},\sigma}(N)$0. The induction step uses only Deligne's bound $S_k^{\mathrm{new},\sigma}(N)$1 to control the higher moments $S_k^{\mathrm{new},\sigma}(N)$2 of the eigenvalues, so the argument is elementary given the trace estimate.

Asymptotics for non-square index and a failure of the analogous result

For non-square $S_k^{\mathrm{new},\sigma}(N)$3 coprime to $S_k^{\mathrm{new},\sigma}(N)$4, the situation changes qualitatively because the trace $S_k^{\mathrm{new},\sigma}(N)$5 is bounded while the second moment satisfies

$S_k^{\mathrm{new},\sigma}(N)$6

using the Hecke composition relation $S_k^{\mathrm{new},\sigma}(N)$7. Strong induction then gives, for each fixed $S_k^{\mathrm{new},\sigma}(N)$8,

$S_k^{\mathrm{new},\sigma}(N)$9

and $\sigma$0 obeys the same law multiplied by $\sigma$1. Hence even-indexed coefficients have sign $\sigma$2 for all but finitely many weights $\sigma$3, uniformly across sign patterns. Odd-indexed coefficients are governed by the trace $\sigma$4, whose non-vanishing is not established here; obtaining a sign result for odd indices would require bounding $\sigma$5 away from zero, which remains open.

The paper's most notable negative finding concerns extending the even-coefficient sign statement asymptotically in $\sigma$6. For prime level $\sigma$7, the trace formula acquires a term proportional to $\sigma$8, the Hurwitz class number:

$\sigma$9

so that

$Q \parallel N$0

Two competing main terms emerge. Using a Bateman–Chowla–Erdős-type construction — relating $Q \parallel N$1 to Dirichlet $Q \parallel N$2 via the class number formula, approximating $Q \parallel N$3-values by short Euler products for all but few characters, and invoking Linnik's theorem — the authors produce infinitely many primes with $Q \parallel N$4 (forcing $Q \parallel N$5) and infinitely many with $Q \parallel N$6 (forcing $Q \parallel N$7). Consequently, for any non-square $Q \parallel N$8 and even $Q \parallel N$9, there exist infinite families of pairs $\{\pm 1\}$0 with $\{\pm 1\}$1 and infinite families with $\{\pm 1\}$2. This contrasts sharply with the behavior over $\{\pm 1\}$3 and $\{\pm 1\}$4 without sign decomposition, where uniform sign behavior in $\{\pm 1\}$5 does hold; the finer Atkin–Lehner decomposition genuinely destroys the global phenomenon. It also shows the error exponent $\{\pm 1\}$6 in the trace formula is sharp, since the $\{\pm 1\}$7 term itself is $\{\pm 1\}$8 along one family of primes.

Extension to the newspace

All results transfer to $\{\pm 1\}$9, with the caveat that $S_k^\pm(N)$0 must be admissible (excluding $S_k^\pm(N)$1 with $S_k^\pm(N)$2); inadmissible patterns contain no newforms by Atkin–Lehner theory. The main terms acquire a correction factor involving $S_k^\pm(N)$3 and local factors $S_k^\pm(N)$4, reflecting the newspace trace main term $S_k^\pm(N)$5 from the underlying involution trace formula. For admissible $S_k^\pm(N)$6, $S_k^\pm(N)$7 occurs for at most finitely many pairs $S_k^\pm(N)$8, following from the dimension estimates of Ross–van Lidth de Jeude–Wolf–Xue, so the "all but finitely many" statements remain well-formed. The counterexample theorem likewise holds on the newspace.

Explicit sign thresholds for the second coefficient

Because every constant in the trace formula is explicit, the paper derives unconditional, checkable conditions guaranteeing the sign of $S_k^\pm(N)$9. Writing

$S_k(N)$0

and bounding the error functional $S_k(N)$1 using Deligne's bound together with auxiliary inequalities such as $S_k(N)$2, the authors obtain:

Case	Sign	Condition
$S_k(N)$3 square	$S_k(N)$4	$S_k(N)$5
$S_k(N)$6 non-square	$S_k(N)$7	$S_k(N)$8

These are the first effective versions of the sign predictions for $S_k(N)$9 on sign-pattern spaces; previously only qualitative "all but finitely many" statements were available even on the full space. The constants are admittedly crude — they arise from worst-case bounding of several stacked error terms — but they render the corollaries computationally decidable in any given instance.

Limitations and open questions

Several restrictions are intrinsic to the methods. The non-square asymptotics are necessarily confined to fixed $S_k^{\mathrm{new}}(N)$0 with $S_k^{\mathrm{new}}(N)$1, since the counterexample theorem rules out any uniform-in-$S_k^{\mathrm{new}}(N)$2 sign law for even coefficients. The odd coefficients depend on the trace $S_k^{\mathrm{new}}(N)$3, and no non-vanishing result for $S_k^{\mathrm{new}}(N)$4 is proved. The authors formulate Conjecture: $S_k^{\mathrm{new}}(N)$5 (and similarly $S_k^{\mathrm{new}}(N)$6) whenever $S_k^{\mathrm{new}}(N)$7 and $S_k^{\mathrm{new}}(N)$8 is even, verified computationally for $S_k^{\mathrm{new}}(N)$9, $T_m' = m^{-(k-1)/2}T_m$00, $T_m' = m^{-(k-1)/2}T_m$01. Finally, the explicit constants ($T_m' = m^{-(k-1)/2}T_m$02 in the trace bound, the threshold $T_m' = m^{-(k-1)/2}T_m$03) are far from optimal, and improving them would lower the effective range of the sign criteria.

Conclusion

The paper establishes precise leading-term asymptotics for all fixed Hecke polynomial coefficients on the finest natural Hecke- and Galois-stable decomposition of cusp form spaces, with effective error bounds throughout. Its principal structural insight is that the class-number term in the sign-space trace formula, negligible relative to the main term globally, becomes dominant along infinite families of levels, producing genuine sign oscillation absent from the undecomposed theory. The explicit trace formula developed here should serve beyond coefficient asymptotics — notably for equidistribution of Hecke eigenvalues on sign spaces and for generalizing classifications of zero-dimensional newspace sign sectors — though its applications in those directions are deferred to separate works.