Sparsest-Permutation Learning

• Sparsest-Permutation Learning is a method for selecting the permutation that produces the sparsest representation of a model, notably in causal DAG inference.
• It utilizes conditional-independence tests and matrix factorization (e.g., Cholesky) to recover minimal I–MAPs under assumptions weaker than standard faithfulness.
• The approach extends beyond causal discovery to applications in blind source separation and neural structured sparsity, inspiring scalable and heuristic relaxations.

Sparsest-permutation learning denotes a class of inference problems in which an unknown permutation is chosen so that a data-consistent representation is as sparse as possible. In the causal-DAG setting, this takes the form of selecting the variable ordering whose induced directed acyclic graph has the fewest edges (Raskutti et al., 2013). Closely related formulations arise when resolving frequency-by-frequency permutation ambiguities in convolutive blind source separation (Benichoux et al., 2011), learning sparse models over distributions on permutations from marginal data (Farias et al., 2010), jointly estimating sparse signals and sensing-order permutations (Prasad, 2018), and learning permutations that make structured neural sparsity more effective (Tyagi et al., 16 Oct 2025, Li et al., 30 Jan 2026). Across these literatures, the common pattern is combinatorial search over permutations under an explicit sparsity criterion, but the object being sparsified, the admissible relaxations, and the identifiability conditions are application-specific.

1. Canonical formulation in causal DAG learning

The formulation that established the term in causal structure learning considers $X=(X_1,\dots,X_p)$ with joint distribution $\mathbb P$, assumed Markov to an unknown “true” DAG $G^*=([p],E^*)$. The goal is to recover the Markov equivalence class $\mathcal M(G^*)$ from i.i.d. samples. For a permutation $\pi$ of $[p]$, the sparsest-permutation (SP) algorithm constructs the unique minimal I–MAP $G_\pi$ consistent with $\mathbb P$ and the ordering $\pi$, and then selects the permutation for which $G_\pi$ has minimal number of edges (Raskutti et al., 2013).

Concretely, for a permutation $\mathbb P$0,

$\mathbb P$1

where

$\mathbb P$2

Writing $\mathbb P$3, the SP objective is

$\mathbb P$4

In the oracle conditional-independence setting, the algorithm is explicit: enumerate all permutations, build $\mathbb P$5 by conditional-independence queries, track the smallest edge count, and return the set of sparsest DAGs. Its key subroutines are a CI test and an edge-count update. This makes SP a hybrid algorithm: the search space is permutation-based, but the score is induced by conditional independences rather than by a conventional likelihood score alone (Raskutti et al., 2013).

A common misconception is that sparsest-permutation learning in causal discovery is merely a variant of generic score-based search. The original SP formulation is more specific: the score is the edge count of the minimal I–MAP induced by an ordering, and the ordering itself is the primary optimization variable (Raskutti et al., 2013).

2. Consistency, assumptions, and the role of sparsity

The central identifiability condition for the SP algorithm is the sparsest-Markov representation (SMR) assumption. SMR states that among all DAGs $\mathbb P$6 not Markov-equivalent to $\mathbb P$7, the true $\mathbb P$8 has strictly fewer edges than any other Markov I–MAP of $\mathbb P$9. The main theorem states that SP recovers $G^*=([p],E^*)$0 if and only if $G^*=([p],E^*)$1 satisfies SMR (Raskutti et al., 2013).

This result is significant because it weakens the assumptions usually invoked by constraint-based methods. The comparison given in the original paper is

$G^*=([p],E^*)$2

where restricted-faithfulness (RF) means adjacency- and orientation-faithfulness only on edges and v-structures. The paper therefore concludes that SP is consistent under strictly weaker conditions than the faithfulness assumption used by PC and SGS, and states that SMR is strictly weaker than the assumptions used by PC/SGS/GES (Raskutti et al., 2013).

A supporting lemma establishes that if $G^*=([p],E^*)$3 is Markov to $G^*=([p],E^*)$4, then for any permutation $G^*=([p],E^*)$5 consistent with a topological order of $G^*=([p],E^*)$6, one has $G^*=([p],E^*)$7. Under adjacency-faithfulness, skeletons match; under RF, even v-structures match, so $G^*=([p],E^*)$8. This formalizes why topological orders of the true DAG are competitive under the SP score (Raskutti et al., 2013).

The later greedy-relaxation literature recasts these comparisons in terms of increasingly weak “causal razors.” Lam, Andrews and Ramsey describe a hierarchy in which GRaSP$G^*=([p],E^*)$9 coincides with TSP, GRaSP$\mathcal M(G^*)$0 with ESP, and GRaSP$\mathcal M(G^*)$1 is the most relaxed form; their results are stated as pointwise consistency under increasingly weaker assumptions than faithfulness (Lam et al., 2022). This suggests that the original SP criterion became a reference point for a broader program of weakening faithfulness-type assumptions while retaining permutation-based search.

3. Gaussian case, Cholesky factorization, and $\mathcal M(G^*)$2-penalized likelihood

In the Gaussian setting, the SP objective admits a precise matrix-factorization interpretation. Assuming $\mathcal M(G^*)$3 with precision $\mathcal M(G^*)$4, let $\mathcal M(G^*)$5 be $\mathcal M(G^*)$6 with rows and columns permuted by $\mathcal M(G^*)$7. Write its unique upper-Cholesky factorization as

$\mathcal M(G^*)$8

with $\mathcal M(G^*)$9. By Pourahmadi’s lemma,

$\pi$0

Hence $\pi$1 has exactly those edges $\pi$2 for which $\pi$3, and the SP objective becomes

$\pi$4

SP is therefore the problem of finding the permutation that yields the sparsest Cholesky factorization of the precision matrix (Raskutti et al., 2013).

The same paper shows that, in the oracle limit where the true covariance matrix is known, SP is equivalent to $\pi$5-penalized Gaussian maximum likelihood estimation. The Gaussian log-likelihood for precision $\pi$6 is

$\pi$7

and the $\pi$8-penalized MLE is written over permutation, triangular factor, and diagonal factor. In the oracle limit $\pi$9 and $[p]$0, this reduces to minimizing $[p]$1 subject to

$[p]$2

which is exactly the SP objective in the Gaussian case (Raskutti et al., 2013).

This connection became the basis for later algorithmic work. “Efficient Permutation Discovery in Causal DAGs” formulates the task directly as

$[p]$3

notes that without further assumptions the problem is NP-hard, and exploits the Gaussian correspondence between conditional independences and zeros in the precision matrix to design a polynomial-time heuristic based on sparse elimination structure (Squires et al., 2020). More recent work goes further: “Relaxed Sparsest-Permutation Formulation for Causal Discovery at Scale” states that exact Cholesky factorization is unnecessary for structure recovery, and replaces exact factorization with a support-level relaxation evaluated by masked zero-fill incomplete Cholesky factorization (Oh et al., 7 May 2026).

4. Computational limits and scalable relaxations

The original oracle SP algorithm has time complexity

$[p]$4

and the paper states that brute-force SP is only feasible for $[p]$5. On $[p]$6, it remains feasible at approximately $[p]$7 per simulation, but $[p]$8 becomes intractable. The same source notes that heuristic greedy variants search locally over permutations, for example by adjacent transpositions, in $[p]$9 time (Raskutti et al., 2013).

Two influential causal relaxations illustrate how the exact objective has been adapted. The Removal–Fill–Degree (RFD) algorithm builds an ordering from the end by repeatedly choosing a node to place as a sink among the remaining variables. Its depth-$G_\pi$0 version uses removal score, fill score, and current degree in the moral graph; under Gaussianity its runtime is $G_\pi$1, and for $G_\pi$2 the paper reports performance comparable to PC, GES, and GSP with shorter runtime (Squires et al., 2020). The same work emphasizes that, unlike many causal structure learning algorithms, its good performance and good runtime are not limited to sparse graphs, and it constructs a dense family $G_\pi$3 on which exact order recovery occurs for all $G_\pi$4 (Squires et al., 2020).

GRaSP modifies greedy SP search by introducing the tuck operator. Tuck applies to an edge $G_\pi$5 in the induced $G_\pi$6 and reorders $G_\pi$7 so that $G_\pi$8 sits immediately before $G_\pi$9, together with intermediate ancestors of $\mathbb P$0. The resulting hierarchy—GRaSP$\mathbb P$1, GRaSP$\mathbb P$2, and GRaSP$\mathbb P$3—uses increasingly permissive classes of edges for tucking: covered, singular, and all edges, respectively. The paper states that the most relaxed form of GRaSP outperforms many state-of-the-art causal search algorithms in simulation and allows efficient and accurate search even for dense graphs and graphs with more than 100 variables (Lam et al., 2022).

The 2026 relaxation literature makes the computational departure more explicit. ReSP fixes an ordering-agnostic precision-support mask

$\mathbb P$4

and searches for sparse triangular factors under support-only constraints: $\mathbb P$5 SCOPE implements this via graphical lasso screening, masked IC(0), refit-and-test pruning, and ordering selection. At the population level, soundness is established for MEC recovery under no-cancellation and sparsest Markov representation assumptions, and experiments report scaling to $\mathbb P$6 variables while substantially slower baselines time out beyond $\mathbb P$7 or $\mathbb P$8, depending on the method (Oh et al., 7 May 2026).

5. Related formulations outside causal discovery

In convolutive blind source separation, sparsest-permutation learning addresses a different permutation ambiguity: frequency-bin permutations of estimated mixing filters. After normalization removes per-frequency scaling ambiguities, the task is to choose $\mathbb P$9 so as to recover the true time-domain mixing matrix $\pi$0 up to a single global permutation $\pi$1. The criterion is

$\pi$2

where $\pi$3 sums the $\pi$4 quasi-norms of all time-domain filters (Benichoux et al., 2011).

The theoretical identifiability result in that setting is specific. If $\pi$5 is prime, each filter has at most $\pi$6 nonzeros, and $\pi$7, then the unique minimizer of the $\pi$8 pseudo-norm agrees with the true matrix $\pi$9 up to a single global permutation of its $G_\pi$0 columns. The bound is

$G_\pi$1

The proof combines a discrete uncertainty principle with a Hall’s-theorem combinatorial bound. The paper also gives a greedy $G_\pi$2 algorithm with sweep cost

$G_\pi$3

and reports that the success rate is essentially bimodal, with the highest success rates occurring for $G_\pi$4, for example $G_\pi$5, rather than for the exact $G_\pi$6 criterion (Benichoux et al., 2011). This is a notable case where the uniqueness theorem and the practically effective objective differ.

In unlabelled sensing, the unknown permutation acts on the rows of the sensing matrix rather than on variables or frequency bins. The model is

$G_\pi$7

with $G_\pi$8 sparse and $G_\pi$9 an unknown permutation matrix. The paper develops Permutation-MSBL for uncorrelated sparse vectors and Permutation-KSBL for temporally correlated ones, both within an EM framework. A key result is that the M-step over $\mathbb P$00 reduces to

$\mathbb P$01

and by the rearrangement inequality the optimal permutation is obtained by separately sorting $\mathbb P$02 and $\mathbb P$03 (Prasad, 2018).

In choice modelling, the permutation object is a ranking, and sparsity is support size of a distribution over permutations. A choice model is a distribution $\mathbb P$04 on $\mathbb P$05, observed only through first-order marginals $\mathbb P$06. The nonconvex program is

$\mathbb P$07

The paper proves that any doubly-stochastic target $\mathbb P$08 admits an approximation with support $\mathbb P$09, and that under a signature condition one can recover a sparse approximation in time $\mathbb P$10 (Farias et al., 2010). These formulations are not causal, but they preserve the same combinatorial theme: sparse structure is sought by optimizing over permutations or distributions on permutations.

6. Neural structured sparsity and contemporary reinterpretations

A more recent line of work uses learnable permutations to improve hardware-friendly sparsity patterns in deep networks. In “Efficient Dynamic Structured Sparse Training with Learned Shuffles,” each layer $\mathbb P$11 is decomposed into a structured-sparse matrix $\mathbb P$12 and a column permutation matrix $\mathbb P$13, so that

$\mathbb P$14

The optimization jointly learns structured sparsity and a soft permutation parameterization over the Birkhoff polytope, with final discretization by nearest-permutation projection. The paper states that permutation-augmented DST matches unstructured baselines at $\mathbb P$15 sparsity on ImageNet-1K (ViT-B/16) and WikiText-103 (GPT-2), trains up to $\mathbb P$16 faster, infers up to $\mathbb P$17 faster, and incurs training-memory overhead of $\mathbb P$18 for ViT and $\mathbb P$19 for GPT2-Small (Tyagi et al., 16 Oct 2025).

A closely related transformer-pruning formulation introduces a learnable permutation cost matrix $\mathbb P$20, solves an entropy-regularized assignment problem over the Birkhoff polytope via Sinkhorn iterations,

$\mathbb P$21

and converts the result to a hard permutation at inference by the Hungarian algorithm. The learned permutation is used to reorder weights before applying an $\mathbb P$22 mask and then invert the permutation after pruning. Reported results include ViT-Base/16 Top-1 $\mathbb P$23 at 2:4 sparsity versus $\mathbb P$24 for CP and $\mathbb P$25 for RIA, ViT-Base/16 Top-1 $\mathbb P$26 at 4:8 sparsity versus $\mathbb P$27 for RIA, LLaMA-3.2-1B average accuracy $\mathbb P$28 versus $\mathbb P$29 for Wanda, $\mathbb P$30 for CP, and $\mathbb P$31 for RIA, and WikiText2 perplexity $\mathbb P$32 versus $\mathbb P$33 for Wanda (Li et al., 30 Jan 2026).

These neural-network papers use the phrase in a broader engineering sense than the original DAG literature. A plausible implication is that “sparsest-permutation learning” has evolved from a specific causal-ordering principle into a more general design pattern: learn a permutation because the right reordering exposes a sparse structure that is otherwise hidden. What remains constant across the literature is the core trade-off already present in the original SP algorithm: stronger sparsity criteria can yield stronger structural recovery or better efficiency, but exact permutation search is combinatorial, and practical methods therefore rely on domain-specific relaxations, heuristics, or differentiable surrogates (Raskutti et al., 2013, Oh et al., 7 May 2026).