Fermat's Factorization Method

• Fermat's Factorization Method is a classical approach that represents an odd composite integer as a difference of two squares for factorization.
• Modular residue restrictions and step-size optimizations streamline the search process, reducing computational complexity in many cases.
• Recent work extends the method via recursive multiplication, quantum annealing, and machine learning to tackle more challenging integer factorizations.

Fermat’s Factorization Method is a classical approach for integer factorization, based on expressing an odd integer as a difference of two squares. This methodology, originating with Pierre de Fermat, forms the basis for several modern factorization strategies, both deterministic and heuristic, including numerous extensions in algorithmic number theory and quantum computing. Its effectiveness, limitations, and specialized variants for particular integer families have been analyzed and augmented in recent research.

1. Foundational Principle and Classical Algorithm

The core identity underlying Fermat’s method posits that any odd composite integer $N$ can be written as

$N = a^2 - b^2 = (a-b)(a+b)$

for some integers $a, b$. For $N = pq$ (with $p, q$ primes), the trivial decomposition is

$$N = \left(\frac{p+q}{2}\right)^2 - \left(\frac{p-q}{2}\right)^2,$$

but the challenge lies in discovering $a, b$ without prior knowledge of $p, q$.

The classical search procedure begins at $a_0 = \lceil \sqrt{N} \rceil$, iteratively increases $a$, computing $\Delta = a^2 - N$ each time, and tests if $\Delta$ is a perfect square. The factors are then $(a-b)$ and $(a+b)$ when $\Delta = b^2$ (Blake, 2023, 0910.4179, Nesiolovskiy et al., 2019).

For $p/q$ near unity (i.e., $p, q$ within $O(N^{1/4})$ of $\sqrt{N}$), the expected number of iterations is $O(|p-q|) \approx O(N^{1/4})$. If $p, q$ are far apart, the iteration count grows exponentially with the bit-length of $N$.

2. Performance Improvements and Algorithmic Extensions

Research has produced several notable advancements on the classical method:

• Residue-Class and Step-Size Optimizations: By leveraging modular restrictions (e.g., $N \pmod{4}$, $8$, or $16$), the search space for $a$ can be partitioned into residue classes, allowing increments by $2$ or $4$ instead of $1$ and thus achieving up to fourfold reduction in computational complexity (Mellaerts, 11 Aug 2025). In particular, the analysis shows that for $N \equiv -1 \pmod{4}$, $a$ has restricted parity, while for $N \equiv +1 \pmod{4}$, $a$ must lie in a residue class modulo $4$.
• Multiplication and Recursive Multiplication (“MMFFNN”): Instead of factoring $N$ directly, multiply $N$ by a strategically chosen small factor $m k$, such that the resulting product has factors $A, B$ close together. Fermat’s method is then applied to $M = m N k$. The “Simple Multiplication” approach loops over $k$, whereas “Recursive Multiplication” decomposes $k$ into several factors and uses multi-level recursion. The runtime is provably $O(N^{1/3})$, outperforming the $O(N^{1/2})$ bound of pure Fermat (Nesiolovskiy et al., 2019).

Method	Complexity	Step Features
Classical Fermat	$\widetilde{O}(|p-q|)$	Step $= 1$; no modular restriction
Step-2/4 Modular Refinement	$O(\sqrt{N}/2)$ or $O(\sqrt{N}/4)$	Step $= 2$ or $4$; modular sieve
Recursive Multiplication	$O(N^{1/3})$	Multi-level, nested search
Sieve-Improved (Subset-Sum)	$$\widetilde{O}(\Delta e^{-C \frac{\log\Delta}{\log\log\Delta}})$$	Subexponential saving (Hittmeir, 2022)

3. Specialized Variants and Deterministic Refinements

Fermat’s method is especially potent for numbers whose nontrivial divisors are close in value. For subclasses such as $N = 4n^2+1$, explicit parametrizations yield necessary and sufficient criteria for compositeness:

• For $N = 16m^2 + 1$ (even $n$), factors are parametrizable as $(4a + 1)(4b + 1)$ with congruence and size constraints on $b$. For $N = 4(2m+1)^2 + 1$ (odd $n$), analogous expressions arise. This enables a complete characterization of all proper factors, leveraging Fermat’s structuring as difference-of-squares (Longhas et al., 2022).
• Sieve-based and subset-sum-based algorithms further accelerate Fermat’s method by reducing $\Delta$ by a subexponential factor, packing small primes into modulus factors and assembling candidate residue classes via the Chinese Remainder Theorem (CRT). Importantly, for $\Delta = |v-u|$ small, runtime is substantially reduced (Hittmeir, 2022).

4. Variants for RSA-Type Moduli and Parameter Optimization

The “$c$-method” (classical Fermat) and “$a$-method” (incremental parametrization) have been systematically compared for semiprimes, notably RSA moduli:

• In the $a$-method, an explicit formula expresses the candidate step as

$c = \frac{a^2 - P_0}{2(X_0 - a)},$

where $X_0 = \lceil \sqrt{n} \rceil$ and $P_0 = X_0^2 - n$. Parity pruning and last-digit sieving reduce the number of divisibility tests. For $a \leq 0.255\,X_0$, classical $c$-stepping is preferred; for $a > 0.255\,X_0$, $a$-method provides superior efficiency (0910.4179).

These refinements, although not competitive with modern subexponential factorization algorithms (e.g., Quadratic Sieve, NFS), can yield one-order-of-magnitude speedups for certain parameter regimes.

5. Quantum and Machine Learning-Based Extensions

Recent directions incorporate quantum algorithms and deep learning into Fermat’s factorization framework:

• Quantum Annealing Reformulation: The search for suitable $(a, b)$ is mapped into QUBO problems on quantum hardware. With parity and residue class optimizations, classical complexity is reduced fourfold. Experiments have successfully factored $N = 8,689,739$ using D-Wave 2000Q, with a per-anneal success probability of $65\%$ for 24-bit integers. However, quantum advantage is not rigorously proven for large $N$ (Mellaerts, 11 Aug 2025).
• Deep Learning Binary Classification: Lawrence’s extension recasts factorization as a classification problem: given $N = pq$, decide whether the ratio $R = p/q$ falls within a target interval. A neural network is trained on synthetic datasets of bit-vectors, achieving an out-of-sample accuracy of $\approx0.72$. The approach is super-polynomial in $\log N$ and not viable for practical RSA sizes unless model accuracy can approach unity. Current limitations include potential biases in synthetic data and a pressing need for higher-capacity models and richer features (Blake, 2023).

6. Further Improvements and Open Problems

Several recent proposals seek to exploit number-theoretic structure for speedup:

• Using knowledge of Euler’s totient function $\phi(n)$, start-values for $x$ can be moved closer to the “center” of the search, and step-sizes increased to $2^k$ or $\prod_{p \mid \phi(n)} p^{e_p}$ if such small factors of $\phi(n)$ are detected. This can hypothetically cut iteration counts further, although detecting these factors involves another factorization problem (Detto, 10 Mar 2025).
• Sieve polynomial degree can be increased, meet-in-the-middle and lattice reduction tricks suggested for MCSS-based algorithms (Hittmeir, 2022).

Open questions persist on proving asymptotic speedup for quantum and machine learning reformulations, and on extending these methods beyond semiprimes or linear-combination parameterizations.

7. Applicability, Limitations, and Empirical Performance

Fermat’s method and its variants are optimally suited for integers with factors close to $\sqrt{N}$, such as certain cryptographic moduli, $(4n^2+1)$ forms, and engineered composites. For “balanced” factors, especially in RSA-type semiprimes, the refinements can provide practical iteration reductions—though present methods remain uncompetitive with subexponential general-purpose algorithms.

Empirical testing confirms that recursive multiplication and modular-sieve variants outperform direct Fermat and Lehman’s techniques, especially as $N$ grows in size. Quantum annealing and machine learning approaches, while theoretically promising, require further research before achieving scalability and reliable performance on cryptographically relevant inputs (Nesiolovskiy et al., 2019, Mellaerts, 11 Aug 2025, Blake, 2023).