Maximally Filtered Clique Forests (MFCF)

• MFCF is a greedy structure-learning framework that constructs decomposable clique forests and associated Markov Random Fields from sparse or noisy data.
• It leverages chordal graph properties and iterative clique-expansion moves to maintain tractable inference and rigorous statistical validation.
• The algorithm generalizes Prim’s minimum spanning tree approach while imposing clique-size constraints to achieve enhanced sparsity and interpretability in graphical model estimation.

A Maximally Filtered Clique Forest (MFCF) is a greedy topological structure-learning framework designed to estimate the conditional dependency structure among large sets of random variables, particularly from sparse or noisy data, by producing a decomposable graphical model—specifically, a clique forest with an associated Markov Random Field (MRF). The MFCF method generalizes Prim’s minimum spanning tree approach to support arbitrary clique sizes, achieves topological sparsity through variable clique-size constraints, and enables integrated structure learning and parameter estimation procedures. It is characterized by the use of chordal graphs, repeated application of a local clique-expansion move, and flexible selection of statistical score functions, facilitating both computational efficiency and rigorous statistical validation across a range of settings, including Gaussian covariance selection (Massara et al., 2019).

1. Chordal Structure and Clique Forests

Let $G=(V,E)$ be an undirected graph with vertex set $V=\{1,\dots,p\}$. A clique is a maximal complete subgraph; the set of all maximal cliques is denoted $\mathcal{C}=\{C_1,\dots,C_m\}$. The separator between two cliques $C_i,C_j\in\mathcal{C}$ is $S_{ij}=C_i\cap C_j$. The clique graph $K(\mathcal{C},\mathcal{S})$ is formed with cliques as nodes, and edges joining those with nonempty intersections.

A clique forest is a spanning forest $$(\mathcal{C},\mathcal{S}')\subset K(\mathcal{C},\mathcal{S})$$ satisfying the clique-intersection property: for any $C_i, C_j\in\mathcal{C}$, $C_i\cap C_j$ is contained in all cliques along the unique path in the forest joining $C_i$ and $V=\{1,\dots,p\}$0. This property is equivalent to $V=\{1,\dots,p\}$1 being chordal (i.e., having no chordless cycles of length $V=\{1,\dots,p\}$2).

Chordal graphs, which admit clique forests, facilitate tractable factorization of global graphical models. In particular, the decomposability (i.e., the running-intersection property) enables efficient inference and closed-form maximum likelihood estimation in the underlying graphical models.

2. The Clique-Expansion Move

The defining local move of MFCF is the clique expansion, which iteratively adds vertices to the clique forest while preserving chordality and decomposability.

Suppose at step $V=\{1,\dots,p\}$3 there exists a chordal graph $V=\{1,\dots,p\}$4 with clique forest $V=\{1,\dots,p\}$5. For a clique $V=\{1,\dots,p\}$6 and a new vertex $V=\{1,\dots,p\}$7, a subset $V=\{1,\dots,p\}$8 (the separator) is chosen to maximize a gain function. The expansion can proceed in three ways:

• Expansion with new clique: $V=\{1,\dots,p\}$9, joined via separator $\mathcal{C}=\{C_1,\dots,C_m\}$0.
• Enlarged clique: If $\mathcal{C}=\{C_1,\dots,C_m\}$1, update $\mathcal{C}=\{C_1,\dots,C_m\}$2.
• Isolated clique: If $\mathcal{C}=\{C_1,\dots,C_m\}$3, form new clique $\mathcal{C}=\{C_1,\dots,C_m\}$4.

The updated joint density after an expansion is: $\mathcal{C}=\{C_1,\dots,C_m\}$5 where $\mathcal{C}=\{C_1,\dots,C_m\}$6 and $\mathcal{C}=\{C_1,\dots,C_m\}$7.

This move preserves chordality, since no chordless cycles of length $\mathcal{C}=\{C_1,\dots,C_m\}$8 are introduced. The decomposable structure and tractable inference properties of the model remain intact at every iteration.

3. The MFCF Construction Algorithm

The MFCF algorithm employs a greedy iterative procedure, reminiscent of Prim's algorithm for minimum spanning trees. The pseudocode logic is as follows:

1. Initialization: Begin with a set of outstanding vertices and, optionally, a seeded clique.
2. Gain Table Computation: Precompute the gain $\mathcal{C}=\{C_1,\dots,C_m\}$9 for each existing clique $C_i,C_j\in\mathcal{C}$0 and uncovered vertex $C_i,C_j\in\mathcal{C}$1.
3. Greedy Expansion: While there are unallocated vertices, at each step choose $C_i,C_j\in\mathcal{C}$2 over all cliques $C_i,C_j\in\mathcal{C}$3 and outstanding vertices $C_i,C_j\in\mathcal{C}$4.
4. Update Forest:
   • If $C_i,C_j\in\mathcal{C}$5, form new clique $C_i,C_j\in\mathcal{C}$6, join via separator $C_i,C_j\in\mathcal{C}$7.
   • If $C_i,C_j\in\mathcal{C}$8, augment $C_i,C_j\in\mathcal{C}$9.
   • If $S_{ij}=C_i\cap C_j$0, form isolated clique $S_{ij}=C_i\cap C_j$1.
5. Iterator Maintenance: Remove $S_{ij}=C_i\cap C_j$2 from outstanding vertices and refresh the gain table.
6. Termination: Output the forest on exhaustion of vertices.

This process prioritizes the highest-gain clique expansion at each iteration, generalizing edge-based strategies to arbitrary clique sizes and confirming decomposability at all times.

4. Scoring Functions and Statistical Validation

The gain function $S_{ij}=C_i\cap C_j$3 guides expansion choices. For any expansion,

$S_{ij}=C_i\cap C_j$4

Different settings yield different scoring strategies:

• Similarity-Matrix Gain: For a symmetric weight matrix $S_{ij}=C_i\cap C_j$5, $S_{ij}=C_i\cap C_j$6, and thus

$S_{ij}=C_i\cap C_j$7

• Likelihood Gain (General): For a decomposable graphical model,

$S_{ij}=C_i\cap C_j$8

The expansion adds $S_{ij}=C_i\cap C_j$9.

• Multivariate Gaussian: For $K(\mathcal{C},\mathcal{S})$0,

$K(\mathcal{C},\mathcal{S})$1

equating to one-half the log-likelihood-ratio statistic.

Statistical validation can be performed for each expansion using asymptotic $K(\mathcal{C},\mathcal{S})$2 tests or cross-validation, for instance, requiring $K(\mathcal{C},\mathcal{S})$3 at some significance level $K(\mathcal{C},\mathcal{S})$4, which can further enhance sparsity.

5. Clique-Size Constraint, Sparsity, and Model Decoupling

A defining feature of MFCF is the imposition of a hard clique-size bound $K(\mathcal{C},\mathcal{S})$5 during construction. This acts as an $K(\mathcal{C},\mathcal{S})$6-style penalty on high-order interactions, as any expansion yielding cliques larger than $K(\mathcal{C},\mathcal{S})$7 is forbidden. The model thereby attains sparsity at the topology level, circumventing the need for convex relaxations or soft penalties.

This mechanism also produces a clean decoupling between structure learning (i.e., discovering the clique-forest topology) and parameter estimation (e.g., maximum likelihood estimation, which is available in closed form for each clique and separator after structure is learned). Scanning a range of $K(\mathcal{C},\mathcal{S})$8 values allows for a purely topological trade-off between model fit and complexity.

6. Structural Properties: Perfect Ordering and Elimination Sequences

The outcome of the MFCF procedure on connected chordal graphs guarantees:

• Running-intersection (junction-tree) ordering of the cliques $K(\mathcal{C},\mathcal{S})$9, so that for $$(\mathcal{C},\mathcal{S}')\subset K(\mathcal{C},\mathcal{S})$$0, $$(\mathcal{C},\mathcal{S}')\subset K(\mathcal{C},\mathcal{S})$$1 is contained within some $$(\mathcal{C},\mathcal{S}')\subset K(\mathcal{C},\mathcal{S})$$2, $$(\mathcal{C},\mathcal{S}')\subset K(\mathcal{C},\mathcal{S})$$3.
• Perfect elimination ordering of the vertices, obtained by listing the residual set $$(\mathcal{C},\mathcal{S}')\subset K(\mathcal{C},\mathcal{S})$$4 in reverse. When a vertex $$(\mathcal{C},\mathcal{S}')\subset K(\mathcal{C},\mathcal{S})$$5 is added via separator $$(\mathcal{C},\mathcal{S}')\subset K(\mathcal{C},\mathcal{S})$$6, $$(\mathcal{C},\mathcal{S}')\subset K(\mathcal{C},\mathcal{S})$$7 is simplicial in the induced subgraph on $$(\mathcal{C},\mathcal{S}')\subset K(\mathcal{C},\mathcal{S})$$8.

These properties ensure explicit construction of a decomposable model, with tractable inference and parameter learning tasks.

7. Computational Complexity, Guarantees, and Empirical Results

At each greedy expansion, all current cliques ($$(\mathcal{C},\mathcal{S}')\subset K(\mathcal{C},\mathcal{S})$$9) are compared with all remaining vertices ($C_i, C_j\in\mathcal{C}$0), scanning all separators $C_i, C_j\in\mathcal{C}$1 of size up to $C_i, C_j\in\mathcal{C}$2. The total cost per step is $C_i, C_j\in\mathcal{C}$3, and with $C_i, C_j\in\mathcal{C}$4 steps, the overall complexity is $C_i, C_j\in\mathcal{C}$5. The process is polynomial for small $C_i, C_j\in\mathcal{C}$6, which is typical in practice.

Being greedy, MFCF does not guarantee global optimality, as the underlying problem is NP-complete. However, every outcome is a valid decomposable model. Empirical evaluations in covariance selection settings show that MFCF often outperforms $C_i, C_j\in\mathcal{C}$7- and $C_i, C_j\in\mathcal{C}$8-penalized Graphical Lasso methods, particularly in test log-likelihood, sparsity recovery, and interpretability, especially in chordal-sparse and real-world stock data scenarios. For dense true models, Graphical Lasso may marginally optimize likelihood, but MFCF maintains competitive eigenvalue fit and achieves sparser and more interpretable structures, provided statistical validation is applied (Massara et al., 2019).

In Gaussian graphical modeling, once the clique forest is learned, the maximum likelihood estimate for the precision matrix is: $C_i, C_j\in\mathcal{C}$9 where $C_i\cap C_j$0 indicates zero-padding to the full $C_i\cap C_j$1 matrix.

8. Practical Applications and Protocols

Typical application protocols in covariance selection involve:

• Simulating or loading data and splitting into train/validation/test.
• Hyperparameter tuning for MFCF (e.g., $C_i\cap C_j$2, shrinkage) and baseline algorithms (e.g., Graphical Lasso $C_i\cap C_j$3).
• Model evaluation via test log-likelihood, sparsity accuracy, specificity/sensitivity to zero-patterns, and eigenvalue reconstruction.

Results show that all MFCF variants provide higher test likelihoods and structure recovery accuracy in chordal-sparse settings, and robust spectral and interpretative performance in both synthetic and real datasets, outperforming classical shrinkage and glasso techniques under certain constraints (Massara et al., 2019).

MFCF thus provides a framework for learning sparse, interpretable, and tractable graphical models via local, chordality-preserving expansions, with a topological control on model complexity and broad support for distributional assumptions and scoring metrics.