---
title: Hull Attack in Code Cryptanalysis
url: https://www.emergentmind.com/topics/hull-attack
type: topic
---

# Hull Attack in Code Cryptanalysis

Searching arXiv for recent and related papers on the quadratic-hull/tangent-space attack against alternant-based McEliece schemes.
Hull attack denotes a key-recovery approach against certain code-based cryptosystems in which the attacker exploits the quadratic hull of a linear code to recover hidden algebraic structure. In the setting of generic alternant codes used in McEliece-type schemes, the attack described in "The Tangent Space Attack" identifies the concealed generalized Reed–Solomon (GRS) structure by studying the intersection of all quadrics passing through the columns of a public generator or parity-check matrix, and then extracting a common stabilizer of tangent spaces to the resulting variety [2505.10184]. Under a high-rate regime, this yields a polynomial-time recovery of the underlying GRS code and therefore an efficient key-recovery attack against alternant-based McEliece instantiations [2505.10184].

## 1. Definition and geometric formalism

For a finite field $\mathbb{F}$, let $C \subset \mathbb{F}^n$ be an $r$-dimensional linear code, and let a generator matrix $G \in \mathbb{F}^{r \times n}$ be viewed column-wise as an ordered set of points $x_1,\dots,x_n \in \mathbb{F}^r$ in the affine setting, or as points in $\mathbf{P}^{r-1}(\mathbb{F})$ in the projective setting [2505.10184]. A quadric is the zero-locus of a homogeneous polynomial of total degree $2$, and with $R=\mathbb{F}[X_0,\dots,X_{r-1}]$, the evaluation map
$$
\operatorname{ev}_G: R_2 \to \mathbb{F}^n,\qquad f \mapsto (f(x_1),\dots,f(x_n))
$$
has kernel
$$
I_2(G)=\ker(\operatorname{ev}_G),
$$
the space of all quadratic forms vanishing on the column set [2505.10184].

The attack distinguishes two related objects. The algebraic quadratic hull is the ideal generated by these degree-$2$ relations, written $\langle I_2(G)\rangle \subset R$, while the geometric quadratic hull is the affine variety
$$
V_2(G)=V(I_2(G)) \subset \mathbb{F}^r
$$
cut out by those quadrics [2505.10184]. Up to projective equivalence these hulls depend only on the code and not on the particular choice of generator matrix [2505.10184]. The paper further notes that Hilbert’s Nullstellensatz allows one to move between ideal-theoretic and variety-theoretic viewpoints, but that over a finite field the two must be distinguished [2505.10184].

The central idea of hull attack is that these quadratic relations are not merely invariants of the public code: they can encode the hidden algebraic geometry of the secret family from which the code originates. This is especially significant for code families whose quadratic hull is much smaller or more structured than that of a random code of the same parameters [2505.10184].

## 2. Rational normal curves, squares, and the hidden GRS structure

For a generalized Reed–Solomon code $C=\mathrm{GRS}_r(x,y)\subset \mathbb{F}^n$ with support $x=(x_i)$ and multiplier $y=(y_i)$, one may choose a truncated Vandermonde generator
$$
V_r=[\,y_i x_i^j\,]_{0\le j<r,\;1\le i\le n}.
$$
If $2r-1 \le n$, then $I_2(V_r)$ is spanned by the $2\times 2$ minors of the $2\times r$ Hankel matrix
$$
\begin{pmatrix}
X_0 & \dots & X_{r-2}\\
X_1 & \dots & X_{r-1}
\end{pmatrix},
$$
and therefore defines the rational normal curve $\nu:\mathbf{P}^1 \to \mathbf{P}^{r-1}$; equivalently, the quadratic hull $V_2(V_r)$ is exactly the affine cone over that curve [2505.10184].

This characterization explains why GRS codes have very small Schur squares. Specifically,
$$
C^{\star 2}=\mathrm{GRS}_{2r-1}(x,y^2),
$$
so that
$$
\dim I_2(\mathrm{GRS}_r)=\binom{r+1}{2}-(2r-1)=\binom{r-1}{2}
$$
[2505.10184]. The attack leverages precisely this atypical abundance of quadratic relations.

A plausible implication is that the hull attack is best viewed as a geometric refinement of square-code distinguishers: rather than using only the dimension defect of $C^{\star 2}$, it exploits the full variety cut out by the degree-$2$ relations. That interpretation is consistent with the paper’s emphasis on the rational normal curve as the geometric carrier of the hidden GRS structure [2505.10184].

## 3. Alternant codes, trace descriptions, and Weil restriction

An alternant code of degree $r$ over $\mathbb{F}_q$ is defined by
$$
A_r(x,y)=\big(\mathrm{GRS}_r(x,y)^\perp\big)\cap \mathbb{F}_q^n
$$
[2505.10184]. By Delsarte’s theorem, its dual is the trace code of a GRS code:
$$
A_r(x,y)^\perp=\operatorname{Tr}_{\mathbb{F}_{q^m}/\mathbb{F}_q}\big(\mathrm{GRS}_r(x,y)\big)
$$
[2505.10184]. This trace description is essential because it allows the attacker to compare an $\mathbb{F}_q$-linear public code of dimension $rm$ with an $\mathbb{F}_{q^m}$-linear secret code of dimension $r$.

The comparison is mediated by affine Weil restriction. If
$$
\Psi:\mathbb{F}_{q^m}^r \to \mathbb{F}_q^{rm}
$$
writes each coordinate in a chosen $\mathbb{F}_q$-basis $\{1,\alpha,\dots,\alpha^{m-1}\}$, then the induced map on polynomials
$$
\Phi:\mathbb{F}_{q^m}[X_0,\dots,X_{r-1}] \to \mathbb{F}_q[x_{i,j}]
$$
splits each $X_i$ into its $m$ coordinate functions [2505.10184]. One then obtains
$$
\operatorname{Res}(I_2(V_r)) \subseteq I_2(\Psi(V_r)),
$$
and under a square-distinguishability, high-rate regime this inclusion becomes an equality [2505.10184]. In that case the alternant code is called Weil-proper, and one has
$$
V_2(\Psi(V_r))=\Psi(V_2(V_r))
$$
up to the secret $\mathbb{F}_q$-linear change of basis $P$ used in the McEliece public key [2505.10184].

This equality is the geometric hinge of hull attack. It means that the public quadratic hull is not an arbitrary variety in $\mathbb{F}_q^{rm}$, but the Weil restriction of the affine cone over the rational normal curve, transported by an unknown linear transformation [2505.10184].

## 4. Tangent-space recovery of the secret field structure

The attack assumes that $C=A_r(x,y)\subset \mathbb{F}_q^n$ is a generic alternant code of extension degree $m$ and dimension $rm$ whose rate is high enough that
$$
\dim I_2(C^\perp)=m\binom{r-1}{2},
$$
equivalently that the right-hand side of the Faugère–Gauthier–Otmani–Perret–Tillich bound meets $\binom{rm+1}{2}-n$ exactly [2505.10184]. The paper gives the equivalent parameter condition
$$
r\le q \quad\text{or more generally}\quad n>\binom{rm+1}{2}-m\binom{r-1}{2}
$$
[2505.10184]. The public parity-check matrix has the form
$$
H=P\,\Psi(V_r(x,y)),
$$
with secret $P\in GL_{rm}(\mathbb{F}_q)$ [2505.10184].

Under these hypotheses, the quadratic hull
$$
W=V_2(H)\subset \mathbb{F}_q^{rm}
$$
is, up to $P$, the Weil restriction of the affine cone over the rational normal curve [2505.10184]. Its tangent spaces $T_QW$, for $Q$ ranging over the $n$ column-points of $H$, are all stabilized by the same linear operator
$$
A=PJ_rP^{-1},
$$
where $J_r\in \mathbb{F}_q^{rm\times rm}$ is the companion matrix of $\alpha$, the scalar-multiplication map in $\mathbb{F}_{q^m}$ [2505.10184].

Two structural statements drive the method. First, if $W=\Psi(V_2(V_r))$, then for every $P\in W$ the tangent space satisfies
$$
T_PW=\Psi\big(T_{\Psi^{-1}(P)}V_2(V_r)\big),
$$
and is therefore $J_r$-invariant; after conjugation by the secret basis change, each $T_QW$ is $A$-invariant [2505.10184]. Second, the only matrices stabilizing every $J_r$-Weil-restriction subspace are polynomials in $J_r$, so intersecting all stabilizers of the tangent spaces recovers the one-dimensional $\mathbb{F}_q$-algebra $[A]\cong \mathbb{F}_{q^m}$ [2505.10184].

In effect, the hull attack does not directly recover the support and multiplier. It first reconstructs the hidden field action encoded by the public alternant representation. This suggests that tangent-space analysis acts as an algebra extractor: the geometry reveals the ambient extension-field structure before the classical support-recovery phase begins.

## 5. Algorithmic workflow and complexity

The key-recovery algorithm consists of five explicit steps [2505.10184].

| Step | Operation | Cost or output |
|---|---|---|
| 1 | Compute a basis of $I_2(H)$ by finding all quadratic relations among the columns of $H$ | Kernel of the star-product map or direct null-space computation |
| 2 | Sample column-points of $H$, compute tangent spaces $T_QW$, and solve equations expressing $M\cdot T_Q\subseteq T_Q$ | Recovers a basis $\{A_1,\dots,A_m\}$ of the algebra $A=[A]$ |
| 3 | Find a generator $A\in A$ of degree $m$, compute its minimal polynomial, and solve a similarity problem | Obtain $Q\in GL_{rm}(\mathbb{F}_q)$ with $Q^{-1}AQ=J_r$ |
| 4 | Recover $G'=\Psi^{-1}(Q)$ | $G'$ is an $\mathbb{F}_{q^m}$-generator matrix of a GRS code conjugate to $\mathrm{GRS}_r(x,y)$ |
| 5 | Run the classical Sidelnikov–Shestakov attack on the recovered GRS code | Recover $(x^{q^j},y^{q^j})$, hence $(x,y)$ |

In Step 2, the paper specifies random sampling of about
$$
\left\lceil \frac{(rm)^2}{2m(rm-2m)} \right\rceil \approx O(r)
$$
points among the column-points of $H$ [2505.10184]. For each sampled point one computes the tangent space as the right-kernel of the Jacobian of the $I_2$ generators at that point, and accumulates the linear equations defining stabilizers [2505.10184].

The complexity analysis is polynomial-time throughout. Each null-space or kernel step on matrices of size $O(n)\times n$ costs $O(n^\omega)$, and Step 1 plus Step 2 therefore runs in $O(r\,n^\omega)$ field operations [2505.10184]. In Step 3, computing the minimal polynomial has cost $O((rm)^\omega)$ [2505.10184]. Assuming $m=O(\log r)$, described as the usual McEliece regime, the entire key-recovery runs in
$$
O(r\,n^\omega+n^\omega)=O(r\,n^\omega)
$$
operations in $\mathbb{F}_q$ [2505.10184].

The final phase uses the classical Sidelnikov–Shestakov attack on the recovered GRS code in $O(n^\omega)$, after which tracing back through Frobenius conjugacy yields the original support and multiplier [2505.10184]. This places hull attack in direct continuity with earlier algebraic attacks on GRS-masked McEliece systems, while extending the vulnerable class to high-rate alternant instances through the intermediary of quadratic geometry [2505.10184].

## 6. Scope, extensions, and limitations

The same strategy is stated to apply to one-point algebraic-geometry codes $C_L(X,P,D)$ of degree $D$ with $2g+2\le \deg D \le q$, or more generally in a square-distinguishable regime [2505.10184]. In that case, the quadratic hull of the $\mathbb{F}_{q^m}$-generator $V(P,\varphi_D)$ is the embedded curve $\varphi_D(X)$, and under analogous Weil-properness heuristics the hull of its trace-subcode yields the Weil restriction of $\varphi_D(X)$ [2505.10184]. The tangent-space step again recovers the field-automorphism matrix, after which one obtains an $\mathbb{F}_{q^m}$-generator of the underlying AG code, from which length-$n$ decoding follows by known AG-decoding algorithms [2505.10184].

For classical Goppa codes, the situation is mixed. Binary Goppa codes $G_\Gamma$ of degree $r$ often satisfy the FGOPT bound when $r<q-1$, so one again has Weil-properness, and in that subregime the attack recovers the hidden support and Goppa-polynomial roots, up to Frobenius conjugacy, in $O(r\,n^\omega)$ [2505.10184]. However, when $r\ge q-1$, the quadratic hull of Goppa codes collapses to a trivial $1$-dimensional variety because field-equation terms appear, and it no longer exhibits the rich $\mathbb{F}_{q^m}$-structure needed by the tangent-space method [2505.10184]. The paper reports that the tangent-space distinguisher then fails empirically, and that the attack does not extend to full Goppa parameters; in particular, binary Goppa codes of practical interest remain, so far, unbroken by this method [2505.10184].

A common misconception would be to treat hull attack as a universal break of code-based cryptography. The available result is narrower: it targets high-rate alternant-based McEliece schemes under explicit structural and distinguishability assumptions, extends heuristically to certain AG codes, and only partially affects Goppa codes [2505.10184].

## 7. Research significance and open problems

The significance of hull attack lies in its relocation of code-based cryptanalysis from purely combinatorial or rank-based invariants toward explicit algebraic geometry. The quadratic hull is the intersection of all quadrics through the public column set, but in the vulnerable regimes it also serves as a proxy for the hidden rational normal curve or its Weil restriction [2505.10184]. This establishes a direct link between the public linear-algebraic representation of a code and the secret algebraic variety defining the private structure.

The paper leaves three open questions. First, whether the hull approach can be refined to cover the remaining high-rate Goppa cases, in particular by exploiting additional higher-degree syzygies or non-quadratic hulls [2505.10184]. Second, which other code families admit a structured quadratic hull vulnerable to tangent-space attacks [2505.10184]. Third, to what extent the heuristic square-distinguishability assumption can be replaced by a provable criterion on code parameters [2505.10184].

These questions indicate that hull attack is both a concrete cryptanalytic method and a broader research program. A plausible implication is that future attacks may generalize from quadratic hulls to higher-degree defining equations whenever the public code inherits a sufficiently rigid embedded variety from a secret algebraic construction. In that sense, hull attack marks the emergence of tangent-space and stabilizer computations as tools for extracting hidden extension-field structure from public code representations [2505.10184].

Source: https://www.emergentmind.com/topics/hull-attack