Hidden Subgroup Problem (HSP)

• The Hidden Subgroup Problem (HSP) is a computational framework that determines an unknown subgroup from an oracle function's coset structure.
• Quantum algorithms use Fourier sampling for abelian groups, while non-abelian cases require advanced representation theory and remain challenging.
• HSP has profound implications, underpinning quantum breakthroughs in factoring, discrete logarithms, and post-quantum cryptographic security.

The Hidden Subgroup Problem (HSP) is a computational framework that has become central in quantum algorithms, post-quantum cryptography, and algebraic complexity theory. Formally, for a group $G$ (finite or infinite) and oracle access to a function $f: G \to X$ that is constant and distinct on left (or right) cosets of a subgroup $H \leq G$, the task is to explicitly determine $H$, often by outputting a generating set. HSP encapsulates the algebraic core of many quantum speedups, including integer factoring, discrete logarithm, and Simon’s problem. Its generalization to non-abelian and infinite groups remains an intense focus, with broad implications for quantum algorithm design and cryptographic security.

1. Mathematical Formulation and General Properties

Given a group $G$ and a finite set $X$, the oracle function $f: G \to X$ satisfies

$f(g_1) = f(g_2) \iff g_1H = g_2H,$

so $f$ is strictly $H$-periodic: it is constant on the cosets of the hidden subgroup $H \leq G$, and assumes distinct values on distinct cosets. The HSP is: given oracle access to such an $f$, find generators for $H$ (Horan et al., 2018, Dutto et al., 1 Dec 2025, Ye et al., 2021, Wang, 2010). The problem generalizes both to infinite $G$ as in Shor’s period finding for $\mathbb{Z}$ (Kuperberg, 24 Jul 2025), and to quantum generalizations where the symmetry is defined by invariance under a group action on a quantum state (StateHSP) (Bouland et al., 2024, Hinsche et al., 21 May 2025).

Variants include:

• Identification Version: Output generators for $H$.
• Decision Version: Decide if $H$ is trivial.
• Search over Families: Find $H$ from a candidate set $\mathcal{H}$ (Ye et al., 2021).

The HSP is a unifying abstraction for numerous algebraic and combinatorial problems, including:

• Simon’s problem ($\mathbb{Z}_2^n$)
• Period finding and factoring ($\mathbb{Z}$, cyclic groups)
• Abelian and non-abelian stabilizer problems
• Dihedral, Heisenberg, affine, and symmetric groups (encoding lattice problems, code equivalence, and graph isomorphism) (Horan et al., 2018) 0610086.

2. Quantum and Classical Algorithmic Approaches

Abelian Groups

For finite abelian groups, the canonical quantum solution is Fourier sampling:

1. Prepare uniform superposition over $G$:

$\frac{1}{\sqrt{|G|}} \sum_{g \in G} |g\rangle$

2. Query the oracle to obtain:

$$\frac{1}{\sqrt{|G|}} \sum_{g} |g\rangle |f(g)\rangle$$

1. Discard the second register, yielding a coset mixed state.
2. Apply the Quantum Fourier Transform (QFT) over $G$:

$$|g\rangle \mapsto \frac{1}{\sqrt{|G|}} \sum_{\chi \in \widehat{G}} \chi(g) |\chi\rangle$$

1. Measure in the character basis to obtain a random $\chi \in H^\perp$.
2. Iterate: Collect enough kernels to reconstruct $H$ via classical linear algebra (Dutto et al., 1 Dec 2025, Gogioso et al., 2017, Wang, 2010, Kwon et al., 24 Jul 2025).

These steps admit uniform polynomial (in $\log|G|$) query and gate complexity for abelian $G$, with exact or high-probability success (Horan et al., 2018, Dutto et al., 1 Dec 2025, Kwon et al., 24 Jul 2025).

Non-Abelian Groups

For non-abelian $G$, the Fourier transform decomposes into blocks labeled by irreducible representations (irreps), with considerably more complex statistics:

$$|g\rangle \mapsto \bigoplus_{\pi \in \widehat{G}} \sqrt{\frac{d_\pi}{|G|}} \sum_{i,j=1}^{d_\pi} \pi(g)_{ij} |\pi, i, j\rangle$$

Weak Fourier sampling extracts only the irrep label; strong sampling observes also indices. Certain cases (e.g., Heisenberg, dihedral, Weyl-Heisenberg, and some semidirect products) allow efficient algorithms based on customized representation-theoretic and symmetry approaches, such as the use of Clebsch-Gordan transforms or joint coset state processing (0810.3695) 0612107.

For general non-abelian groups, no efficient quantum algorithms or measurement procedures are known, especially for $S_n$ (Graph Isomorphism), dihedral groups (lattice SVP), and braid groups (Dutto et al., 1 Dec 2025, Horan et al., 2018, Wang, 2010).

Classical and Deterministic Algorithms

For classical randomized sampling, lower and upper bounds on sample complexity are given in terms of group and subgroup parameters:

$$\Omega\!\left(\max\left\{ \min_{H\in\mathcal{H}} \frac{\log|\mathcal{H}|}{\log(|G|/|H|)},\; \min_{H\in\mathcal{H}} \sqrt{ \frac{|G|}{|H|} \frac{\log|\mathcal{H}|}{\log(|G|/|H|)} } \right\}\right)$$

for identification with high probability (Ye et al., 2021). Certain families, e.g., cyclic chains, allow deterministic identification in $O(\log(N/|H|))$ queries, while abelian groups admit $O(\sqrt{|G|/|H|})$ decision and $O(\sqrt{|G|/|H|} \log|H|)$ identification complexity (Ye et al., 2021).

3. Structural Reductions and Centrality

Numerous complexity-theoretic results demonstrate the centrality of HSP in quantum algorithmics:

• Equivalence and Reductions: Hidden shift, hidden coset, and orbit coset problems reduce to HSP over suitable semidirect or wreath product groups [0610086].
• Inductive Approaches: Solutions over simple factors, together with efficient oracles for coset systems, allow solving HSP for a group by recursive reduction (Wang, 2010).
• Program Checking: Nonadaptive checkers verify the correctness of HSP algorithms across all cosets in polynomial time—any systematic error is efficiently detected [0610086].

These reductions underlie the rationale that efficient solutions for general non-abelian HSP imply quantum solutions for a broad class of algebraic and combinatorial problems, including lattice problems, graph isomorphism, and certain hidden polynomial instances (Decker et al., 2011, Dutto et al., 1 Dec 2025).

4. Complexity Boundaries and Quantum–Classical Separations

Sample and query complexity for HSP reflects strong quantum–classical separations in many instances:

• Abelian Groups: Quantum sample complexity is $O(\log^2 |G|)$, exponentially better than classical—e.g., Simon’s problem or generalized Simon (rank-$k$ subgroups of $\mathbb{Z}_p^n$) has classical sample complexity $\Theta(\max\{k, \sqrt{k p^{n-k}}\})$ but quantum complexity $O(n-k)$ (Ye et al., 2021).
• Universal Algebra (HKP): Quantum circuits yield super-polynomial speedups for powers of affine 2-element algebras relative to classical randomized methods ($O(n)$ vs.\ $\Omega(2^{n/2})$ queries) (Moore et al., 2020).
• Infinite Groups: Even for infinite abelian groups ($\mathbb{Z}^k$), quantum algorithms based on Shor-Kitaev generalization run in polynomial time, while HSP over $\mathbb{Q}$ or normal subgroups of non-abelian free groups is NP-hard (Kuperberg, 24 Jul 2025).
• State-Based HSP (StateHSP): Certain quantum learning tasks, such as recovering stabilizer symmetries or disentangling cuts in quantum states, admit polylogarithmic quantum sample complexity under the StateHSP formalism, vs. exponential classical cost (Hinsche et al., 21 May 2025, Bouland et al., 2024).

5. Cryptographic and Computational Applications

Many public-key cryptosystems have algebraic cores reducible to HSP instances:

• Abelian HSP: Breaks the security of classical cryptosystems (factoring, discrete log) on quantum computers via Shor’s and Kitaev’s algorithms (Dutto et al., 1 Dec 2025, Horan et al., 2018).
• Non-Abelian HSP: Candidate quantum-secure platforms (e.g., schemes based on infinite braid or Thompson groups) rely on instances of HSP for which no polynomial-time quantum solver exists (Horan et al., 2018). The concrete hardness of the dihedral HSP underlies reductions from shortest vector problems on lattices (Dutto et al., 1 Dec 2025, Wang, 2010).
• Post-Quantum Security: A group-based cryptosystem is insecure if its core reduces to an efficient quantum HSP instance (abelian, Heisenberg, semidirect products). Candidates based on hard non-abelian HSP instances remain of cryptographic interest (Horan et al., 2018, Dutto et al., 1 Dec 2025).

6. Generalizations, Extensions, and Open Problems

The HSP generalizes naturally along multiple axes:

• Hidden Symmetry Subgroup Problem (HSSP): Generalizes from oracles hiding cosets in $G$ to group actions on sets, capturing hidden polynomial and function-graph problems via reductions (Decker et al., 2011).
• StateHSP: The quantum analogue, where an unknown state invariant under $H \leq G$ replaces the function oracle, unifying tasks such as stabilizer identification and hidden entanglement cut detection (Hinsche et al., 21 May 2025, Bouland et al., 2024).
• Infinite/Continuous Groups: Shor-Kitaev algorithm on $\mathbb{Z}^k$ or torus groups; NP-hardness for $\mathbb{Q}$, free groups (Kuperberg, 24 Jul 2025).
• Algorithmic Paradigms: Beyond QFT, techniques such as Clebsch-Gordan transforms, nested search Hamiltonians, and resonant transition multistep computation admit non-Fourier quantum algorithms for restricted families 0612107.

Open problems include:

• Existence of polynomial-time quantum algorithms for non-abelian HSP settings (notably $S_n$, dihedral, certain matrix and braid groups)
• Implementation of efficient “pretty good measurement” or optimal POVMs in large non-abelian Fourier spaces
• Complexity classifications for HSP and HSSP for further algebraic structures and group actions (Dutto et al., 1 Dec 2025, Decker et al., 2011)
• Tight tradeoffs between quantum query, time, and space complexity for infinite groups (Kuperberg, 24 Jul 2025)

7. Impact and Centrality in Quantum Computing

HSP is central to the landscape of quantum computation:

• It subsumes key quantum speedups, including Simon, Shor, period finding, and discrete logarithm.
• It provides the main organizing principle behind the search for quantum algorithms for algebraic and cryptographic problems (Horan et al., 2018, Dutto et al., 1 Dec 2025).
• Many "hidden-structure" and "oracle" problems—including hidden shift, orbit coset, and hidden polynomial tasks—reduce to HSP with appropriately chosen groups 0610086.
• Structural frameworks (e.g., reductions, program checking) demonstrate that an efficient and robust solution to general non-abelian HSP would fundamentally expand the class of problems admitting exponential quantum speedup 0610086.

The centrality of HSP is thus both theoretical (as an organizing abstraction) and practical (as the yardstick of quantum advantage and cryptographic security). Its unresolved cases motivate both quantum complexity theory and the ongoing search for post-quantum–secure algebraic cryptosystems.