3D Weisfeiler-Lehman Expressiveness

• 3D Weisfeiler-Lehman Expressiveness is a graph isomorphism heuristic that refines colorings of vertex triples to capture intricate multi-node structures.
• It underpins higher-order GNN architectures by enabling the distinction of graphs beyond the capabilities of 1-WL and 2-WL schemes.
• The method offers strong structural insights for graphs with small separators but incurs significant computational overhead on large-scale graphs.

The 3-dimensional Weisfeiler-Lehman (3-WL) test constitutes a central step in the hierarchy of graph isomorphism heuristics and expresses a distinctive "higher-order" leap in the capacity to distinguish non-isomorphic graphs. The expressiveness of 3-WL is of prime importance for both theoretical studies in graph symmetry and practical development of graph neural network (GNN) architectures. The 3-WL test operates by iteratively refining a coloring of all triples of vertices in a graph, encoding increasingly intricate multi-node substructures at each step. Its distinctive expressiveness originates from its ability to perceive structural distinctions invisible to 1-WL (color-refinement) and 2-WL schemes, at the cost of significant computational overhead due to its cubic tensorization over the vertex set.

1. Formal Definition and Algorithmic Foundations

The 3-WL algorithm, as a specialization of the k-WL hierarchy, works by coloring 3-tuples of nodes. At each iteration $t$, it maintains a function $c^{(t)} : V^3 \to \text{Colors}$, initialized by hashing the isomorphism type of the subgraph induced by each triple. The main refinement operator proceeds as follows: for a triple $\vec{v} = (v_1, v_2, v_3)$, for each coordinate $i\in\{1,2,3\}$, it aggregates the multiset $$M_i^{(t)}(\vec{v}) = \{ c^{(t)}(\vec{v}[i \leftarrow w]) : w \in V \}$$, where $\vec{v}[i \leftarrow w]$ denotes the triple with the $i$th entry replaced by $w$. The next color is then

$$c^{(t+1)}(\vec{v}) = \text{Hash} \Big( c^{(t)}(\vec{v}), M_1^{(t)}(\vec{v}), M_2^{(t)}(\vec{v}), M_3^{(t)}(\vec{v}) \Big).$$

Termination occurs upon stabilization, and the final collection $\{c^{(T)}(\vec{v}) : \vec{v} \in V^3\}$ forms a graph invariant. This aligns with the pseudocode and specification in (Huang et al., 2022).

2. Expressive Power: Separation Beyond 1-WL and 2-WL

3-WL transcends the limitations of both 1-WL and 2-WL by distinguishing graphs which these lower-order tests cannot. Both 1-WL (color-refinement) and 2-WL are strictly weaker, as they fail (even in tandem) to distinguish certain pairs of regular graphs (e.g., two 4-regular graphs on 6 vertices, one connected, one disconnected). By contrast, 3-WL can differentiate such graphs, leveraging its ability to examine the profiles of triple neighborhoods and induced substructures. This strict increase in distinguishing power is formalized by the fact that 3-WL (and the 2-FWL variant) exactly corresponds to the capabilities of first-order logic with counting quantifiers and 3 variables, $C^3$ (Huang et al., 2022).

3. Combinatorial and Structural Implications

A central technical result in (Kiefer et al., 2019) is that 3-WL identifies all 3-separators in a graph: for every triple $(x, y, z)$, its color under 3-WL encodes whether $\{x,y,z\}$ forms a separating set. Consequently, 3-WL can reconstruct the Tutte/Stirling-type decomposition of a graph into its 4-connected components, as well as cycles and bonds. This yields the following:

• 3-WL identifies all graphs of treewidth at most 3, and in general, $k$-WL resolves isomorphism for graphs of treewidth $\leq k$.
• The separator-distinction property is the basis for leveraging the 3-WL refinement in graph decomposition algorithms, where identification of small separators directly yields the graph’s decomposition into highly connected pieces.

A key insight is the tightness of 3-WL for specific graph classes: it is both necessary and sufficient for resolving isomorphism in bounded-treewidth classes up to $k=3$, but can be provably insufficient for graphs with larger minors using classical lower-bound constructions such as the Cai–Fürer–Immerman gadgets (Kiefer et al., 2019).

4. Complexity and Practical Constraints

The refinement step of 3-WL brings significant algorithmic expense: explicit representation of all $n^3$ triples requires $O(n^3)$ memory and $O(n^4 \log n)$ time per iteration (due to repeated multiset hashings and aggregations), as described in (Huang et al., 2022). This computational cost quickly becomes prohibitive for large-scale graphs. By contrast, 1-WL and message-passing schemes require $O(|E|)$ per iteration. This exponential blowup underpins the tension in GNN design between expressive fidelity (capturing higher structural complexity) and computational tractability.

5. Impact on Graph Neural Network Architecture

The 3-WL expressiveness barrier has become foundational in evaluating the power of GNNs:

• Classical message-passing neural networks (MPNNs) are bounded in expressive power by 1-WL and are thus inherently limited in distinguishing many graph pairs.
• Higher-order GNNs, engineered after the 3-WL refinement pattern (or its folklore WL/2-FWL variant), surpass this limitation, enabling functions such as subgraph counting and recognition of regularity patterns.
• The computational infeasibility of full 3-WL led to practical designs leveraging localized approximations, subgraph sampling, or refined message aggregation mechanisms to retain significant power with scalable complexity.

Advanced constructions such as $(k,t)$-FWL, $k$-FWL⁺, and N$^2$-FWL, as detailed in (Feng et al., 2023), enable granular control over expressivity and space, allowing matching or even exceeding the 3-WL test, particularly through local equivariant neighbor sets and efficient storage. For example, N$^2$-FWL achieves distinguishing power at least equal to 3-WL while operating with $O(n^2)$ storage.

6. Extensions, Variants, and Connections

The link between 3-WL and other structural and algebraic approaches includes:

• The equivalence of 3-WL and 2-FWL (folklore WL with two variables), forming a strict expressiveness hierarchy.
• Cellular WL (CWL) and CW Networks as presented in (Bodnar et al., 2021) use topological cell complexes to simulate and sometimes exceed 3-WL power. These constructions encode 3-tuples as 2-cells in a complex, allowing message passing at larger scales while preserving, and sometimes surpassing, the distinguishing power of 3-WL.
• The extension to $(k,t)$-FWL and FWL⁺ in (Feng et al., 2023) expands the design space for higher-order GNNs, supporting permutation-equivariant neighbor sets and efficient hierarchy navigation beyond the rigidity of standard $k$-WL.

The following table summarizes the complexity and distinguishing power of select schemes:

Method	Memory Complexity	Expressive Power
1-WL (MPNN)	$O(n)$	Limited, fails on many regulars
2-WL	$O(n^2)$	Equivalent to 1-WL
3-WL	$O(n^3)$	Distinguishes complex regulars
N$^2$-FWL	$O(n^2)$	$\geq$ 3-WL (often strictly)
CWN (rings)	$O(n + e + r)$	$\geq$ 3-WL using cell complexes

Here $r$ is the number of rings/cycles (for CW Networks).

7. Strengths, Limitations, and Open Problems

3-WL is notably strong for graphs with small separators (e.g., planar graphs, minor-closed families of bounded treewidth), enabling isomorphism resolution and structural decomposition. It can identify all 3-separators, decompose into 4-connected pieces, and distinguish graphs up to treewidth 3 (Kiefer et al., 2019). However, the CFI construction yields infinite graph pairs (of sufficiently large treewidth) undistinguished by 3-WL, highlighting the unbounded nature of the k-WL hierarchy—no fixed $k$ suffices for general graph isomorphism.

Several modern approaches aim to approximate or extend 3-WL’s capabilities with more efficient representations, often exploiting local neighborhoods, equivariant neighbor selection, or higher-dimensional cell complexes. The exploration of such methods remains active, with the competing objectives of maximizing graph distinguishability (expressive power) while maintaining algorithmic feasibility on large graphs (Feng et al., 2023, Bodnar et al., 2021, Huang et al., 2022).