Revisiting Fermat's Factorization for the RSA Modulus

Abstract: We revisit Fermat's factorization method for a positive integer $n$ that is a product of two primes $p$ and $q$. Such an integer is used as the modulus for both encryption and decryption operations of an RSA cryptosystem. The security of RSA relies on the hardness of factoring this modulus. As a consequence of our analysis, two variants of Fermat's approach emerge. We also present a comparison between the two methods' effective regions. Though our study does not yield a new state-of-the-art algorithm for integer factorization, we believe that it reveals some interesting observations that are open for further analysis.