Multi-plane Clos Topologies Explained

• Multi-plane Clos topologies are network structures that use multiple parallel middle-stage modules or complete Clos fabrics to enhance nonblocking behavior, scalability, and path diversity.
• They find applications across datacenter networks, optical ROADMs, and AI clusters, enabling significant resource savings and improved fault tolerance through strategic plane multiplicity.
• Analytical methods such as LP-duality for nonblocking criteria and bitset-optimized replacement-chain scheduling provide actionable insights into performance, complexity, and resilience.

Multi-plane Clos topologies are Clos-derived switching structures in which multiplicity is introduced either through several parallel middle-stage modules inside a classical three-stage Clos fabric or through several parallel Clos fabrics operated as distinct planes. Across the literature, both interpretations appear: classical analysis treats the middle-stage multiplicity $m$ as the key internal resource in $C(n,m,r)$-type networks, while optical and AI-cluster work treats a plane as either a selectable middle-stage module or a complete parallel Clos subfabric fed by host bandwidth breakout. In both cases, the governing design question is the same: how many parallel internal resources are needed to obtain the desired combination of nonblocking behavior, scalability, path diversity, and resilience (Ngo et al., 2012, Lin et al., 2023, Araujo et al., 5 May 2026).

1. Architectural forms and notation

The canonical foundation is the classical three-stage Clos network. In the notation $C(n_1,r_1,m,n_2,r_2)$, the first stage contains $r_1$ input modules of size $n_1\times m$, the middle stage contains $m$ modules of size $r_1\times r_2$, and the last stage contains $r_2$ output modules of size $m\times n_2$. The symmetric case is written $C(n,m,r)$ with $C(n,m,r)$0 and $C(n,m,r)$1. In this formulation, $C(n,m,r)$2 is the number of middle-stage modules, and it is the parameter most naturally interpreted as the internal plane count (Ngo et al., 2012).

A closely related parameterization appears in Clos-inspired sparse neural layers, where a three-stage Clos is defined as a 5-tuple $C(n,m,r)$3. Here $C(n,m,r)$4 is the number of inputs, $C(n,m,r)$5 the number of outputs, $C(n,m,r)$6 the number of input routers, $C(n,m,r)$7 the number of middle routers, and $C(n,m,r)$8 the number of output routers. The paper defines path diversity as

$C(n,m,r)$9

so the number of parallel middle routers is directly the number of available routes between any input-output pair. This supports a plane interpretation in which each middle router is one internal plane choice, even though the paper itself uses the terms “middle routers” or “modules,” not “planes” (Isakov et al., 2018).

Optical realizations retain the same three-stage structure. In the Clos-ROADM architecture, the network “consists of three switch stages, i.e., input, middle, and output stages,” and neighboring stages are “interconnected by a fully connected network using short-reach fibers.” For a Clos-ROADM with $C(n_1,r_1,m,n_2,r_2)$0 directional degrees and $C(n_1,r_1,m,n_2,r_2)$1 fiber degrees, the design is denoted $C(n_1,r_1,m,n_2,r_2)$2, where $C(n_1,r_1,m,n_2,r_2)$3 is the number of middle-stage switching elements. This again makes the middle-stage multiplicity the central internal resource, with every ingress module connected to every middle module and every middle module connected to every egress module (Lin et al., 2023).

In datacenter settings, the same structure appears in folded form. The literature explicitly states that a Spine-Leaf network is “essentially a (folded) Clos network,” with the Leaf layer corresponding to the stacked ingress and egress stages and the Spine layer corresponding to the middle stage. A stricter multi-plane usage then emerges in recent AI-cluster deployments, where a plane is a complete parallel Clos network and each host’s NIC bandwidth is broken out across several lower-rate links, one per plane (Lin et al., 2023, Araujo et al., 5 May 2026).

2. Nonblocking regimes and plane-count dimensioning

The most developed analytical treatment sizes the number of middle modules by the nonblocking criterion. In the classical symmetric Clos $C(n_1,r_1,m,n_2,r_2)$4, strict-sense nonblocking is obtained with

$C(n_1,r_1,m,n_2,r_2)$5

For $C(n_1,r_1,m,n_2,r_2)$6, wide-sense nonblocking under the policy “reuse a busy middle crossbar whenever possible” requires

$C(n_1,r_1,m,n_2,r_2)$7

For multirate wide-sense nonblocking Clos networks, the paper derives the sufficient condition

$C(n_1,r_1,m,n_2,r_2)$8

These results are obtained through an LP-duality framework whose generic form is

$C(n_1,r_1,m,n_2,r_2)$9

and

$r_1$0

with weak duality used as the certificate that one more middle-stage resource than the worst-case obstruction is sufficient (Ngo et al., 2012).

The optical literature uses an analogous but application-specific parameterization. For the Clos-ROADM $r_1$1, the “spatially strictly non-blocking condition for the Clos-ROADM is

$r_1$2

so we take $r_1$3,” while setting $r_1$4 corresponds to “the condition of a reconfigurable non-blocking Clos-ROADM.” In multi-plane language, the distinction is explicit: a larger number of middle-stage modules yields stricter nonblocking guarantees, while a smaller number may suffice for rearrangement or low blocking under dynamic traffic (Lin et al., 2023).

A useful cross-domain summary is given below.

Context	Plane-count parameter	Condition or consequence
Symmetric Clos $r_1$5	$r_1$6	$r_1$7 for strict-sense nonblocking
Clos $r_1$8	$r_1$9	$n_1\times m$0 for wide-sense nonblocking
Multirate Clos $n_1\times m$1	$n_1\times m$2	$n_1\times m$3 for multirate WSNB
Clos-ROADM $n_1\times m$4	$n_1\times m$5	$n_1\times m$6 for reconfigurable non-blocking
Clos-ROADM $n_1\times m$7	$n_1\times m$8	$n_1\times m$9 for spatially strictly non-blocking
ClosNet layer $m$0	$m$1	$m$2 path diversity

The ClosNet formulation adds an orthogonal dimensioning rule: the parameter count of the three-stage layer is

$m$3

so increasing the number of middle modules raises path diversity linearly but also raises the number of learned parameters linearly (Isakov et al., 2018).

3. Scaling, complexity, and resource efficiency

One of the principal reasons for introducing planes or parallel middle-stage modules is to replace a fully connected internal backplane with a staged decomposition. This is especially explicit in high-degree ROADMs. In a Spanke-ROADM $m$4, the total number of WSSs is

$m$5

and the backplane fiber count is

$m$6

For the Clos-ROADM $m$7, the total number of WSSs is

$m$8

and the backplane fiber count is

$m$9

Under the strict nonblocking benchmark $r_1\times r_2$0, the asymptotic complexities are $r_1\times r_2$1 for Clos-ROADM fiber complexity versus $r_1\times r_2$2 for Spanke-ROADM, and $r_1\times r_2$3 for Clos-ROADM element complexity versus $r_1\times r_2$4 for Spanke-ROADM element complexity (Lin et al., 2023).

The concrete example $r_1\times r_2$5 versus $r_1\times r_2$6 makes the scaling effect particularly clear. The Spanke realization requires 200 $r_1\times r_2$7 WSSs and 9,000 fibers. The comparable Clos realization requires 20 $r_1\times r_2$8 (or $r_1\times r_2$9) WSSs, $r_2$0 $r_2$1 WSSs, and $r_2$2 fibers. At $r_2$3, where the blocking probability is already very close to the Spanke-ROADM in the reported simulation, the fiber count becomes 120 fibers, which is highlighted as more than 98% fiber savings (Lin et al., 2023).

Blocking analysis in the optical case also exposes the interaction between spatial and spectral diversity. The paper gives the Erlang-B lower-bound model

$r_2$4

$r_2$5

$r_2$6

where $r_2$7 is traffic load and $r_2$8 the number of available wavelengths. The TWC-WSS Clos-ROADM and TWC-AWG-TWC Clos-ROADM approach this theoretical limit because wavelength conversion removes internal wavelength-continuity restrictions. A plausible implication is that plane multiplicity is most effective when spectral constraints do not negate the nominal spatial path diversity (Lin et al., 2023).

Datacenter scaling uses the same logic in folded form. When a $r_2$9 Clos is folded into Spine-Leaf form, each Leaf switch grows from size

$m\times n_2$0

to

$m\times n_2$1

More recent AI-networking work pushes this idea further by splitting an 800 Gb/s NIC into 8 × 100 Gb/s and building eight parallel 100 Gb/s Clos planes. With 51.2 Tb/s switch silicon, this changes the effective switch port count from 64 ports at 800 Gb/s to 512 ports at 100 Gb/s, allowing a two-tier design in which each T0 switch has 256 ports downward and 256 upward, each T1 connects down to 512 T0 switches, and the resulting network reaches 131,072 GPUs (Araujo et al., 5 May 2026).

The same flattening trend appears in recent multi-plane Fat-Tree comparisons. For a system of about 65K NICs, the paper reports an 8-plane 2-layer Fat-Tree with switch configuration $m\times n_2$2 Gbps, diameter $m\times n_2$3, $m\times n_2$4, $m\times n_2$5 optical modules at 200G, and cost/NIC = $m\times n_2$6. This suggests that multi-plane Clos is not only a control or resilience device; it is also a topology-flattening mechanism under high-radix breakout assumptions (Wang et al., 26 Apr 2026).

4. Path assignment, scheduling, and transport control

Once a multi-plane Clos is dimensioned, the control problem becomes the assignment of each connection to one internal resource among many equivalent or near-equivalent choices. In one-rate bidirectional Clos datacenter networks, the recent centralized model uses $m\times n_2$7 low-level switches $m\times n_2$8, $m\times n_2$9 top-level switches $C(n,m,r)$0, per-link capacities $C(n,m,r)$1, demand matrix $C(n,m,r)$2, and route-allocation tensors $C(n,m,r)$3 and $C(n,m,r)$4. A connection between $C(n,m,r)$5 and $C(n,m,r)$6 traverses exactly one top-level switch and is represented as

$C(n,m,r)$7

Feasibility is defined by

$C(n,m,r)$8

and

$C(n,m,r)$9

Although the paper does not define planes explicitly, it states that the formulation can map naturally to a multi-plane deployment if each plane, or each optical circuit switch within a plane, is treated as one of the top-level switching resources (Zhu et al., 16 Jul 2025).

Its main algorithm, FastReChain, handles route changes through replacement chains. The scheduling objective is to minimize

$C(n,m,r)$00

that is, the total number of rearrangements between the current and new schemes. The method searches for short replacement chains with depth-first search and iterative deepening and accelerates the search through bitsets such as $C(n,m,r)$01, $C(n,m,r)$02, and $C(n,m,r)$03. The paper reports that, with bitset optimization, “the algorithm runs 1000 times faster,” and in dynamic scheduling the per-operation running times remain in the reported nanosecond range even with hundreds of top-level switches (Zhu et al., 16 Jul 2025).

At the transport layer, multi-plane operation can be made explicit rather than abstracted. In production AI supercomputers, MRC and static SRv6 treat each entropy value as a direct encoding of one path on one plane. The sender builds an EV set, “typically 128 to 256 entries,” chooses an equal number of EVs per plane, and sprays packets across them. The paper states that “all packets of a QP are sprayed across many paths on all planes in a multi-plane network,” and that “Each EV corresponds to a specific path on a specific network plane.” Static SRv6 then makes the EV-to-path mapping deterministic, so path identity is transport-visible and probeable (Araujo et al., 5 May 2026).

These two control styles—centralized replacement-chain scheduling and end-host spraying across explicitly identified plane-local paths—address different operating regimes. The first is connection-level rescheduling in bufferless or optical Clos networks; the second is packet-level exploitation of a fixed large path set in synchronous AI training. Their common premise is that multi-plane Clos performance depends as much on plane-selection policy as on plane count.

5. Failure domains, graceful degradation, and operational behavior

The resilience case for multi-plane Clos is strongest where a plane is a complete parallel fabric rather than merely a middle-stage choice. In large AI clusters, each NIC’s total bandwidth is broken out across several planes, so a single fault removes only a fraction of total host bandwidth. The paper gives several explicit figures. Losing a T0–T1 link reduces capacity from a node by about 3% in an 800 Gb/s plane, versus about 0.4% in a 100 Gb/s plane. Losing a NIC–T0 link in an 8-plane design costs 12% of NIC bandwidth, but the training job can continue (Araujo et al., 5 May 2026).

This smaller blast radius changes failure handling from fail-stop semantics toward graceful degradation. The production study reports that T0–T1 link failures and flaps can “largely be ignored.” In one 75K GPU pretraining job, a T1 switch failure affected around a quarter of QPs and dropped around 580K packets, yet after QPs mapped out the bad paths the throughput was “largely unaffected,” and when the switch actually rebooted there was no impact. In a 50K GPU job on a 4-plane system, a transceiver glitch flapping four links at once caused about 25% throughput reduction over the minute of flaps, followed by immediate recovery, with no QP failure and no node eviction (Araujo et al., 5 May 2026).

The same deployment also quantifies the latency and throughput regime achieved under normal operation. On Cluster B, T0-local latency is $C(n,m,r)$04 and cross-T1 latency is $C(n,m,r)$05. Both T0-local bandwidth and cross-T1 bandwidth are approximately 770 Gb/s, which is about 96% of theoretical peak. At 42K GPUs, NCCL over MRC reaches up to 92 GB/s per NIC for large message sizes. These results combine topology and transport, but they show that a multi-plane two-tier Clos can maintain both high utilization and graceful degradation under ongoing failures (Araujo et al., 5 May 2026).

This resilience model is not universal across the literature. The optical Clos-ROADM work, for example, explicitly notes that strict fault tolerance or explicit plane redundancy is not a major topic and does not present a formal resilience model, middle-stage protection scheme, or failure recovery method. A plausible implication is that multi-plane Clos has become operationally central first in large AI clusters and only secondarily in other Clos-derived domains (Lin et al., 2023).

6. Extensions, adjacent uses, and interpretive limits

The idea of a Clos with multiple parallel middle resources extends beyond packet or circuit switching. In ClosNets for DNN training, the topology is used as a predefined sparse connectivity pattern: routers become small fully connected learned subnetworks, while inter-router links are fixed scatters or permutations without weights. The key Clos properties retained are “full connectivity,” “shallowness,” “pre-determined connectivity,” “uniform and high path diversity,” and “an efficient hardware implementation.” The paper reports that Clos networks have comparable accuracy with the baseline networks while having $C(n,m,r)$06 less parameters, and states that dense layer sizes can be reduced “by as much as an order of magnitude” without hurting model accuracy (Isakov et al., 2018).

Not every architecture with high path diversity, however, is a multi-plane Clos. Recent work on Multi-Plane HyperX defines a plane as an independent physical subnet/topology instance and compares that model against multi-plane Fat-Tree. In a roughly 65K NIC system, the paper reports an 8-plane 2-layer Fat-Tree at $C(n,m,r)$07 per NIC, with the latter also reducing diameter from $C(n,m,r)$08 to $C(n,m,r)$09. This is a comparison against multi-plane Clos, not a reformulation of Clos itself (Wang et al., 26 Apr 2026).

A different neighboring direction is MRLS, a randomized two-level leaf-spine topology. That work states that the Fat-Tree is “essentially a multistage folded Clos network,” but MRLS is not presented as multiple parallel Clos planes. Its “multipass” property means repeated traversal of the same two-level fabric rather than traversal across distinct planes. The paper reports a 50% speedup against a Fat-Tree for an All2All collective comprising 100k endpoints, but this should be understood as an alternative to conventional Clos regularity, not as a multi-plane Clos result (Cano et al., 26 May 2026).

A further interpretive limit appears in resource-centric topology analysis. One paper does not explicitly analyze Clos or multi-plane Clos fabrics, but it concludes that redundancy is most efficiently handled via parallel network instances “rather than intrinsic topological path diversity.” This provides indirect support for plane replication as a design pattern, while remaining outside formal Clos analysis (Wei et al., 26 Jan 2026).

The principal misconception, therefore, is to treat “multi-plane Clos” as a single, uniform object. The literature supports at least two technically distinct meanings: a three-stage Clos with several parallel middle-stage modules, and a set of several independent Clos fabrics used as explicit planes. The first meaning dominates classical nonblocking theory, optical ROADMs, and Clos-inspired sparse neural layers; the second dominates recent AI-supercomputer networking. The continuity between them lies in the role of the plane count as the decisive internal resource for path diversity, blocking behavior, and graceful degradation.