The Lang-Trotter conjecture on average for genus-$2$ curves with $S_3$ reduced automorphism group

Abstract: For an elliptic curve $E$ over $\mathbb{Q}$ without complex multiplication, Lang and Trotter conjectured that the number of primes $p <X$ at which $E$ has a supersingular reduction is asymptotically equal to $c\sqrt{X}/\log X$, where $$c\>0$$ is a constant depending only on $E$. While it remains an open question, an average estimation related to the Lang-Trotter conjecture was established by Fouvry and Murty. This result is called the Lang-Trotter conjecture on average. We extend the Lang-Trotter conjecture to curves of genus $2$ and obtain a similar result to the Lang-Trotter conjecture on average for the family of curves $$C_λ:y^{2=x(x-1)(x-λ)(x-(λ-1)/λ)(x-1/} (1-λ))$$. These curves are characterized as curves of genus $2$ with reduced automorphism group containing symmetric group $S_3$.

• The paper establishes explicit formulas for counting superspecial primes in genus‑2 curves, linking counts to class numbers and Frobenius actions.
• It applies advanced analytic and combinatorial techniques to average the Lang–Trotter conjecture over families with S3 automorphism structures.
• The study highlights how automorphism groups influence reduction properties, paving the way for improved point counting and cryptographic applications.

The Lang–Trotter Conjecture on Average for Genus-2 Curves with $S_3$ Reduced Automorphism Group

Introduction and Background

This work develops analogues of the Lang–Trotter conjecture in the context of smooth projective curves of genus $2$ over $\mathbb{Q}$, specifically focusing on families whose reduced automorphism group contains the symmetric group $S_3$. The classical Lang–Trotter conjecture predicts, for an elliptic curve $E/\mathbb{Q}$ without CM, an asymptotic formula for the number of primes $p < X$ with specified reductions characterized by the Frobenius trace, most notably for supersingular primes: $$\#\{p < X : E \text{ supersingular mod } p \} \sim c_E \frac{\sqrt{X}}{\log X}$$ where $c_E > 0$ depends on $E$.

While the conjecture remains open for individual curves, average results have been established for families of elliptic curves, most notably by Fouvry and Murty, who considered

$$\frac{1}{AB} \sum_{|a|\leq A,\,|b| \leq B} \#\{p<X : E_{a,b} \text{ supersingular at } p\}$$

where $2$0 are Weierstrass models. Their main term, of the order $2$1, manifests the expected random-like distribution of supersingular primes on average.

The present paper extends this investigation to a family of genus-$2$2 curves defined by

$2$3

parametrized by $2$4, with $2$5 algebraically closed, and the reduced automorphism group containing $2$6. The main aim is to count, on average, the number of superspecial primes for these curves as $2$7 varies and compare this to the classical expectations for genus-$2$8.

Superspecial Reductions for Genus-$2$9 Curves

A genus-$\mathbb{Q}$0 curve $\mathbb{Q}$1 is superspecial if its Jacobian splits (over $\mathbb{Q}$2) up to isogeny as a product of supersingular elliptic curves. Previous work classified genus-$\mathbb{Q}$3 curves by reduced automorphism group and computed the number of isomorphism classes with various group types—Klein 4, $\mathbb{Q}$4, etc. In earlier work, analogous average results were established for genus-$\mathbb{Q}$5 curves with Klein 4 automorphism types. Here, the focus is on those with $\mathbb{Q}$6.

Key to the analysis are correspondences between superspeciality of genus-$\mathbb{Q}$7 curves and supersingularity of associated elliptic curves. Specifically, $\mathbb{Q}$8 is shown to be superspecial if and only if two associated elliptic curves (determined via explicit formulas involving $\mathbb{Q}$9) are both supersingular. The count thus reduces to understanding the set of $S_3$0 for which both elliptic curves are supersingular.

Main Theorems and Techniques

Theorem A

An explicit formula, in terms of class numbers of imaginary quadratic fields, is proved for

$S_3$1

when $S_3$2:

• If $S_3$3: $S_3$4
• If $S_3$5: $S_3$6
• If $S_3$7: $S_3$8

This is achieved via careful analysis of the action of the Frobenius on the two associated elliptic curves, and a characterization of when their $S_3$9-invariants correspond to supersingular curves. The relevant $E/\mathbb{Q}$0-invariants are shown to be roots mod $E/\mathbb{Q}$1 of Hilbert class polynomials of discriminant $E/\mathbb{Q}$2 or $E/\mathbb{Q}$3. Intricate evaluations of multiplicities and correspondences are navigated using Hilbert and modular polynomial congruences, Kronecker–Deuring theory, and combinatorial enumeration via automorphism orbits.

Theorem B

The average number of superspecial primes $E/\mathbb{Q}$4 for genus-$E/\mathbb{Q}$5 curves in the family $E/\mathbb{Q}$6 with $E/\mathbb{Q}$7 and $E/\mathbb{Q}$8 satisfies

$E/\mathbb{Q}$9

where

$p < X$0

This is proven by leveraging analytic number theoretic techniques: sums over class numbers are controlled by the Dirichlet class number formula, and error terms are managed by applying results of Jutila and deep analytic bounds on character sums. The approach generalizes techniques used in the genus-$p < X$1 case (with new combinatorial complications due to the automorphism group structure), and controls are obtained for averaging over the parameter $p < X$2.

Theorem C

When averaging over $p < X$3 with height $p < X$4, a related formula is achieved: $p < X$5 Asymptotics and technical tools are adapted from genus-$p < X$6 and Klein 4 cases, with further subtleties arising from rational parametrization and the need to control counts taking automorphism orbits into account.

Numerical Results and Structural Implications

The main term in Theorem B,

$p < X$7

underscores a quantitative difference from the genus-$p < X$8 scenario (where Fouvry–Murty obtained a leading constant $p < X$9) and from the Klein 4 genus-$$\#\{p < X : E \text{ supersingular mod } p \} \sim c_E \frac{\sqrt{X}}{\log X}$$0 family. The explicit tracking of all main term contributions through automorphism group action is essential—and perhaps surprising in view of the increment in complexity upon moving from genus 1 to genus 2 and automorphism groups beyond the Klein 4.

The methods highlight the intricate link between curve moduli, field arithmetic, and class group theory, and show that for higher genus curves with nontrivial automorphism structure, a combinatorial and analytic apparatus is needed to extract distribution results for arithmetic invariants. Furthermore, the explicit computation of root multiplicities of Hilbert class polynomials mod $$\#\{p < X : E \text{ supersingular mod } p \} \sim c_E \frac{\sqrt{X}}{\log X}$$1 affords new insight into the behavior of reductions, and the proof involves novel algebraic results regarding genus-$$\#\{p < X : E \text{ supersingular mod } p \} \sim c_E \frac{\sqrt{X}}{\log X}$$2 curves' moduli and their reductions.

Implications and Outlook

These results confirm that Lang–Trotter-type conjectures—both in original and averaged forms—possess nontrivial analogues for higher genus curves with nontrivial automorphism groups. They reveal that the arithmetic of superspecial primes is significantly governed by the automorphism structure and the interplay with modular curves and class number theory.

Methodologically, the paper advances the technical machinery for reducing questions on the distribution of arithmetic primes for genus-$$\#\{p < X : E \text{ supersingular mod } p \} \sim c_E \frac{\sqrt{X}}{\log X}$$3 curves to computations with class polynomials, modular correspondences, and combinatorial orbit counting. The combinatorial structure synthesized here can accommodate further generalization to other automorphism types and potentially to more general families.

Practically, stronger understanding of the distribution of superspecial primes in higher genus is relevant for applications in explicit point counting, cryptographic parameter generation, and the classification of curves with rare reduction properties. The approach developed can be adapted to computational algorithms for superspecial/supersingular point enumeration in explicit classes of curves.

Theoretically, the present analysis suggests that further refinements of the Lang–Trotter paradigm to higher-dimensional abelian varieties will require a detailed simultaneous consideration of moduli geometry, automorphism groups, and analytic number theory, likely necessitating new multi-variable or multi-orbit generalizations of class number and modular polynomial techniques.

Conclusion

This work rigorously establishes, for the first time, an average Lang–Trotter-type formula for a nontrivial family of genus-$$\#\{p < X : E \text{ supersingular mod } p \} \sim c_E \frac{\sqrt{X}}{\log X}$$4 curves with $$\#\{p < X : E \text{ supersingular mod } p \} \sim c_E \frac{\sqrt{X}}{\log X}$$5 in their reduced automorphism group, and provides explicit main term constants in terms of class numbers. The results depend upon, and significantly extend, the analytic, algebraic, and combinatorial apparatus originally developed for the genus-$$\#\{p < X : E \text{ supersingular mod } p \} \sim c_E \frac{\sqrt{X}}{\log X}$$6 case. They open new avenues for understanding the arithmetic of reductions for higher genus curves and reinforce the role of automorphism structures in governing the behavior of superspecial and supersingular primes.