Cartan images and $\ell$-torsion points of elliptic curves with rational $j$-invariant

Abstract: Let $\ell$ be an odd prime and $d$ a positive integer. We determine when there exists a degree-$d$ number field $K$ and an elliptic curve $E/K$ with $j(E)\in\mathbb{Q}\setminus{0,1728}$ for which $E(K)_{\mathrm{tors}}$ contains a point of order $\ell$. We also determine when there exists such a pair $(K,E)$ for which the image of the associated mod-$\ell$ Galois representation is contained in a Cartan subgroup or its normalizer, conditionally on a conjecture of Sutherland. We do the same under the stronger assumption that $E$ is defined over $\mathbb{Q}$.