Fully-Connected Tensor Network (FCTN)

• Fully-Connected Tensor Network (FCTN) is a tensor decomposition method representing an order-N tensor as N factors on a complete graph, explicitly capturing pairwise mode correlations.
• Its complete-graph topology ensures invariance under mode permutation, setting it apart from chain-based methods like Tensor Train and Tensor Ring.
• FCTN formulations, through robust optimization and parameter reparameterization, have demonstrated superior performance in tensor completion, super-resolution, and neural network compression.

Fully-Connected Tensor Network (FCTN) denotes a tensor network architecture in which every pair of modes is linked by its own virtual bond, so that an order-$N$ tensor is represented by $N$ factor tensors arranged on a complete graph rather than a chain, ring, or star. In the decomposition form used across tensor completion, super-resolution, inpainting, and robust recovery, each factor carries one physical index and $N-1$ virtual indices, one for every other mode; this yields direct modeling of pairwise cross-mode correlations and an essential invariance under mode transposition or permutation (Yang et al., 2022, Jin et al., 2022, Qin et al., 13 Feb 2026). In a distinct but related neural-network setting, the same idea appears as a parameter-space FCTN, where a single global tensor encodes the weights of an entire fully convolutional network and layer weights are obtained by slicing that tensor (Kossaifi et al., 2019).

1. Formal definition and complete-graph topology

For an order-$N$ tensor

$$\mathcal{X}\in\mathbb{R}^{I_1\times I_2\times \cdots \times I_N},$$

the standard FCTN decomposition introduces $N$ factor tensors

$$\mathcal{G}^{(k)}\in \mathbb{R}^{R_{1,k}\times\cdots\times R_{k-1,k}\times I_k\times R_{k,k+1}\times\cdots\times R_{k,N}},$$

where $R_{p,q}$ is the bond dimension associated with the unordered pair of modes $(p,q)$. Elementwise, $\mathcal{X}(i_1,\dots,i_N)$ is obtained by summing over all pairwise latent indices $N$0 and multiplying the corresponding entries of the $N$1 factors. Each factor therefore contains one physical mode $N$2 and all virtual modes connecting it to every other factor (Yang et al., 2022, Xie et al., 26 Oct 2025).

This topology is the defining distinction of FCTN. In network terms, the graph is the complete graph on $N$3 nodes: every mode is directly coupled to every other mode, rather than only to neighboring modes as in Tensor Train (TT) or Tensor Ring (TR), or through a single central core as in Tucker decomposition (Jin et al., 2022, Gan et al., 20 Nov 2025). The same complete-graph interpretation is used repeatedly in later formulations, including robust completion, quantized recovery, and Bayesian hyperspectral–multispectral fusion (Liu et al., 2021, Qin et al., 13 Feb 2026, Shan et al., 21 Oct 2025).

A basic structural consequence is that FCTN reduces to ordinary matrix factorization for order $N$4 tensors and to standard Tensor Ring decomposition for third-order tensors. For $N$5, it is strictly richer than TT/TR in the sense that every pair of modes has an explicit bond variable rather than only neighbor-to-neighbor couplings (Yang et al., 2022).

2. Rank structure, unfoldings, and relation to other decompositions

The FCTN rank is the collection

$N$6

with one rank for every unordered pair of modes. This pairwise rank parameterization is central: small $N$7 impose strong low-rank structure, while larger $N$8 increase expressiveness and computational cost (Xie et al., 26 Oct 2025).

A key algebraic property is that the rank of any generalized unfolding is bounded by the product of the bonds crossing the chosen bipartition. If $N$9 denotes a generalized unfolding after a mode permutation, then

$N-1$0

This relation is used to justify FCTN-rank-aware convex surrogates, especially balanced-unfolding nuclear norms in robust completion models (Liu et al., 2021, Qin et al., 13 Feb 2026).

Because the graph is complete and symmetric, FCTN is described as invariant under transposition or permutation of modes: permuting modes corresponds to permuting cores and pairwise ranks, without changing the representational pattern itself. TT and TR do not share this property; their chain or ring geometry makes performance sensitive to mode ordering (Jin et al., 2022, Xie et al., 26 Oct 2025, Gan et al., 20 Nov 2025).

Decomposition	Connectivity pattern	Characteristic limitation or property
CP	One shared latent index	Rank-1 component structure
Tucker	Central core plus factor matrices	One core mediates all interactions
TT / TR	Chain or ring	Adjacent-mode coupling; order sensitivity
FCTN	Complete graph	Pairwise ranks; transposition invariance

This comparison suggests that FCTN occupies the dense end of tensor-network design: it trades sparse topology for explicit mode-to-mode coupling and finer control of correlation structure.

3. Optimization principles and regularized formulations

Early FCTN completion models use weighted least squares over observed entries. In FCTN-WOPT, the completed tensor is parameterized as

$N-1$1

and optimized by minimizing

$N-1$2

where $N-1$3 is the binary observation mask. The method initializes factors from an FCTN decomposition and updates them with L-BFGS using gradients derived from special unfoldings of the partial network (Yang et al., 2022).

A separate line develops convex and nonconvex robust completion models. The convex RC-FCTN introduces an FCTN nuclear norm defined as a weighted sum of nuclear norms of balanced generalized unfoldings,

$N-1$4

and combines it with an $N-1$5 sparse-error penalty under observation constraints. The corresponding ADMM algorithm has a global convergence guarantee, and the model is accompanied by an exact recovery theorem under FCTN incoherence and sampling assumptions. The nonconvex RNC-FCTN instead factorizes $N-1$6 directly through FCTN cores and is solved by proximal alternating minimization (PAM), converging to a critical point (Liu et al., 2021).

Subsequent work adds stronger priors on the factors themselves. AFCTNLR imposes trace regularization on the mode-$N-1$7 unfoldings of FCTN factors using a periodically modified negative Laplacian,

$N-1$8

within a PAM framework. The regularizer penalizes discrete second-order differences of factor columns, and the resulting Sylvester equations are solved efficiently by combining FFT diagonalization of $N-1$9 with eigendecompositions of local Gram matrices (Xie et al., 26 Oct 2025).

Another direction modifies the parameterization rather than the loss. iFCTN reparameterizes each FCTN block through Khatri–Rao products, replacing explicit high-order cores by matrix-valued “cherry factors” and yielding folding-free subproblems with structured coefficient matrices. Its tensor completion algorithm remains PAM-based and retains convergence guarantees (Gan et al., 20 Nov 2025). In parallel, large-scale recovery work combines FCTN-aware generalized nonconvex regularization in the gradient domain with randomized Tucker-style compression and ADMM, thereby moving FCTN-based recovery toward quantized observations and large visual tensors (Qin et al., 13 Feb 2026).

4. FCTN as global parameterization of neural networks

In "T-Net" (Kossaifi et al., 2019), the FCTN idea appears in parameter space rather than as a direct decomposition of a data tensor. There, all convolutional weights of a stacked HourGlass fully convolutional network are encoded by one global eighth-order tensor

$N$0

whose modes correspond to the number of HourGlass stacks, depth per stack, pathway, convolution index inside a block, input channels, output channels, kernel height, and kernel width. A particular convolution kernel is a slice

$N$1

The global tensor is factorized either by Tucker decomposition or by tensor-train/MPS, and all layers share the same latent factors (Kossaifi et al., 2019).

For the reference architecture, the tensor shape is

$N$2

The Tucker parameter count is

$N$3

which differs from layer-wise tensorization because feature-mode factors are shared across all layers rather than duplicated per layer (Kossaifi et al., 2019).

This parameter-space FCTN functions as both compression and implicit regularization. On MPII human pose estimation, the uncompressed baseline is $N$4 PCKh. Tucker T-Net with rank $N$5 reaches $N$6 compression and $N$7 PCKh, while rank $N$8 reaches $N$9 compression and $$\mathcal{X}\in\mathbb{R}^{I_1\times I_2\times \cdots \times I_N},$$0 PCKh. Higher compression remains viable: rank $$\mathcal{X}\in\mathbb{R}^{I_1\times I_2\times \cdots \times I_N},$$1 gives $$\mathcal{X}\in\mathbb{R}^{I_1\times I_2\times \cdots \times I_N},$$2 compression and $$\mathcal{X}\in\mathbb{R}^{I_1\times I_2\times \cdots \times I_N},$$3 PCKh, and the MPS variant with TT ranks $$\mathcal{X}\in\mathbb{R}^{I_1\times I_2\times \cdots \times I_N},$$4 yields $$\mathcal{X}\in\mathbb{R}^{I_1\times I_2\times \cdots \times I_N},$$5 compression and $$\mathcal{X}\in\mathbb{R}^{I_1\times I_2\times \cdots \times I_N},$$6 PCKh. On facial part segmentation, Tucker T-Net with rank $$\mathcal{X}\in\mathbb{R}^{I_1\times I_2\times \cdots \times I_N},$$7 gives $$\mathcal{X}\in\mathbb{R}^{I_1\times I_2\times \cdots \times I_N},$$8 compression and $$\mathcal{X}\in\mathbb{R}^{I_1\times I_2\times \cdots \times I_N},$$9 mIoU versus a $N$0 baseline, and rank $N$1 gives $N$2 compression and $N$3 mIoU (Kossaifi et al., 2019).

The neural-network use of the term therefore differs from the canonical decomposition literature in one respect: instead of expressing a data tensor through $N$4 fully connected cores, it expresses an entire network’s parameter field through one global tensor whose modes index architectural degrees of freedom. The common idea is dense cross-mode parameter sharing.

5. Empirical behavior across recovery, inpainting, super-resolution, and fusion

In higher-order tensor completion, FCTN-WOPT is strongest once the tensor order exceeds three. On synthetic tensors formed from reshaped Lena, it is effectively equivalent to TR-based methods in 3D, but for 4D, 5D, and 6D it is substantially better than CP-WOPT, TR-WOPT, TRALS, and TRLRF. At one reported operating point, 4D completion reaches PSNR $N$5 and SSIM $N$6, compared with $N$7 and $N$8 for TRLRF; 5D completion reaches $N$9 and $$\mathcal{G}^{(k)}\in \mathbb{R}^{R_{1,k}\times\cdots\times R_{k-1,k}\times I_k\times R_{k,k+1}\times\cdots\times R_{k,N}},$$0, compared with $$\mathcal{G}^{(k)}\in \mathbb{R}^{R_{1,k}\times\cdots\times R_{k-1,k}\times I_k\times R_{k,k+1}\times\cdots\times R_{k,N}},$$1 and $$\mathcal{G}^{(k)}\in \mathbb{R}^{R_{1,k}\times\cdots\times R_{k-1,k}\times I_k\times R_{k,k+1}\times\cdots\times R_{k,N}},$$2 for TRALS (Yang et al., 2022).

For remote sensing image inpainting, NL-FCTN combines global FCTN completion with nonlocal self-similarity groups, increasing tensor order by stacking similar patches. On CAVE multispectral images at missing rate $$\mathcal{G}^{(k)}\in \mathbb{R}^{R_{1,k}\times\cdots\times R_{k-1,k}\times I_k\times R_{k,k+1}\times\cdots\times R_{k,N}},$$3, the average PSNR reported for NL-FCTN is $$\mathcal{G}^{(k)}\in \mathbb{R}^{R_{1,k}\times\cdots\times R_{k-1,k}\times I_k\times R_{k,k+1}\times\cdots\times R_{k,N}},$$4 dB, compared with $$\mathcal{G}^{(k)}\in \mathbb{R}^{R_{1,k}\times\cdots\times R_{k-1,k}\times I_k\times R_{k,k+1}\times\cdots\times R_{k,N}},$$5 dB for KBR-TC, $$\mathcal{G}^{(k)}\in \mathbb{R}^{R_{1,k}\times\cdots\times R_{k-1,k}\times I_k\times R_{k,k+1}\times\cdots\times R_{k,N}},$$6 dB for FCTN-TC, and about $$\mathcal{G}^{(k)}\in \mathbb{R}^{R_{1,k}\times\cdots\times R_{k-1,k}\times I_k\times R_{k,k+1}\times\cdots\times R_{k,N}},$$7 dB for NL-TT. On the Morocco Sentinel-2 time-series tensor, NL-FCTN reports PSNR $$\mathcal{G}^{(k)}\in \mathbb{R}^{R_{1,k}\times\cdots\times R_{k-1,k}\times I_k\times R_{k,k+1}\times\cdots\times R_{k,N}},$$8, SSIM $$\mathcal{G}^{(k)}\in \mathbb{R}^{R_{1,k}\times\cdots\times R_{k-1,k}\times I_k\times R_{k,k+1}\times\cdots\times R_{k,N}},$$9, and SAM $R_{p,q}$0, outperforming KBR-TC, NL-SNN, NL-TNN, NL-TT, and FCTN-TC (Zheng et al., 2021).

In hyperspectral image super-resolution, the coupled high-order FCTN model tensorizes the HR-HSI into multiscale spatial modes plus one spectral mode and shares FCTN cores across LR-HSI and HR-MSI observations. On SanDiego, the proposed method reports PSNR $R_{p,q}$1, SAM $R_{p,q}$2, ERGAS $R_{p,q}$3, and $R_{p,q}$4 $R_{p,q}$5. The weighted-graph regularization on the spectral core is quantitatively important: without it, the same dataset gives PSNR $R_{p,q}$6, SAM $R_{p,q}$7, ERGAS $R_{p,q}$8, and $R_{p,q}$9 $(p,q)$0 (Jin et al., 2022).

Robust completion extends the same architecture to sparse corruptions. RC-FCTN provides exact recovery guarantees, while RNC-FCTN yields the best MPSNR and MSSIM among the compared robust tensor completion methods on color video and hyperspectral video benchmarks. In synthetic order-4 experiments with salt-and-pepper corruption, the relative recovery error is reported on the order of $(p,q)$1, consistent with the exact recovery theory (Liu et al., 2021).

In hyperspectral–multispectral fusion, BFCTN places Gamma–Gaussian hierarchical priors on FCTN factors and their pairwise couplings, then performs VB–EM inference. The reported outcome is state-of-the-art fusion accuracy, strong robustness, and practical applicability in complex real-world scenarios, including the real ZY-02D satellite dataset, where BFCTN attains the best NHQM among the compared methods (Shan et al., 21 Oct 2025).

6. Computational bottlenecks, structural variants, and current directions

The same complete-graph structure that gives FCTN its expressive power also drives its computational burden. One explicit parameter-count estimate gives plain FCTN complexity as

$(p,q)$2

to be contrasted with $(p,q)$3 for TR and $(p,q)$4 for TT under balanced dimensions and uniform ranks (Yang et al., 2022). This is why later work repeatedly treats full connectivity as a double-edged property: it captures correlations between any modes, but it becomes expensive for higher-order or large-scale tensors (Qin et al., 13 Feb 2026).

Several variants are direct responses to that bottleneck. LMTN inserts latent matrices between the data tensor and an inner FCTN core, reducing the parameter count to

$(p,q)$5

Empirically, LMTN-SVD is reported as $(p,q)$6–$(p,q)$7 times faster than FCTN-PAM with only a $(p,q)$8 points accuracy drop (Yang et al., 2022). AFCTNLR retains the FCTN factorization but adds trace regularization and an intermediate tensor reuse mechanism that reduces runtime by $(p,q)$9–$\mathcal{X}(i_1,\dots,i_N)$0 without affecting image recovery, with larger gains on larger-scale data (Xie et al., 26 Oct 2025). iFCTN removes explicit folding and replaces high-order cores by matrix blocks, reducing storage from $\mathcal{X}(i_1,\dots,i_N)$1 to

$\mathcal{X}(i_1,\dots,i_N)$2

under equal dimensions and equal pairwise ranks (Gan et al., 20 Nov 2025).

Early FCTN completion papers also make the practical limitations explicit. FCTN-WOPT typically uses a uniform rank matrix, often set to $\mathcal{X}(i_1,\dots,i_N)$3, and rank selection is described as manual, time- and effort-consuming. The same work notes that the method is “not very useful in removing disturbances when external noise is present,” which motivated later robust and Bayesian formulations (Yang et al., 2022).

More recent large-scale work addresses the same issue from numerical linear algebra. Randomized compression based on sketching is used as the acceleration core for FCTN-based generalized nonconvex recovery, and the randomized version is reported to be on average about $\mathcal{X}(i_1,\dots,i_N)$4 faster than the deterministic one, with up to $\mathcal{X}(i_1,\dots,i_N)$5 speedup in some cases while maintaining comparable PSNR (Qin et al., 13 Feb 2026).

Within the broader tensor-network literature, contraction cost is governed by loop structure, boundary growth, and bond dimension, and generic tensor-network contraction is described as #P-complete (Ran et al., 2017). This suggests that FCTN research is fundamentally a search for structured approximations that preserve the correlation advantages of the complete graph while controlling the contraction growth induced by dense connectivity.