Entry-Wise Separability Explained

• Entry-wise separability is a property that expresses a multivariate function as a low-rank sum of separable univariate components, reducing complex cross-variable interactions.
• The concept is grounded in tensor decomposition theory, with validation via projection-based and variance-based criteria that accurately isolate individual variable contributions.
• It facilitates efficient optimization and interpretability in frameworks like Tensor Separation Learning, outperforming traditional additive models in capturing higher-order interactions.

Entry-wise separability refers to a structural property of multivariate functions whereby the function can be represented as a sum (or, more generally, a low-rank sum) of terms, each of which factors completely over individual variables (features). Formally, a function $f:\mathbb{R}^d\to\mathbb{R}$ is said to admit an entry-wise separable decomposition of rank $R$ if there exist univariate functions $g_j^{(r)}:\mathbb{R}\to\mathbb{R}$ for $j=1,\dots,d$ and coefficients $s_r\in\mathbb{R}$ such that

$$f(\mathbf{x}) = \sum_{r=1}^R s_r \prod_{j=1}^d g_j^{(r)}(x_j).$$

This generalizes the classical notion where $R=1$, comprising the best-known special case—additive models. Entry-wise separability enables interpretable modeling and efficient optimization by structurally excluding cross-variable interactions, or by representing higher-order interactions compactly in low-rank decompositions. The property connects to tensor decomposition theory (CP/PARAFAC), functional ANOVA, and model interpretability (Liu et al., 29 May 2026, Goda, 2013).

1. Formal Foundations and Definitions

Entry-wise separability can be considered both in the context of sums of univariate functions (additive separability) and in low-rank multiplicative forms. For rank-$R$ entry-wise separability, the function structure is as above, and it is standard to impose normalization ($\|g_j^{(r)}\|_{L^2(P_X)}=1$), ordering, positivity $(g_j^{(r)} > 0)$, or optional orthogonality constraints to ensure uniqueness and interpretability of the univariate factors. In tensor language, this structure is equivalent to the CP (CANDECOMP/PARAFAC) decomposition, but used in scattered-data regression modeling rather than dense arrays (Liu et al., 29 May 2026).

A classical alternative, often termed sum-separability or additive separability, is the representation

$R$0

where $R$1 are univariate functions and all joint effects are suppressed. This is central in generalized additive models (GAMs) and underpins most mainstream feature-attribution and explainability tools, including SHAP and functional ANOVA (Goda, 2013).

2. Characterizations and Testing

Two equivalent, rigorous criteria for entry-wise separability—projection-based and variance-based—have been established for $R$2 (Goda, 2013):

• Projection-Based: Given the “drop-one-coordinate” operator $R$3, $R$4 is entry-wise separable if and only if

$R$5

Each projection removes components invariant with respect to a given coordinate, so the vanishing of their product eliminates all non-separable structure.

• Variance-Based (Functional ANOVA/Sobol’): The total variance $R$6 of $R$7 admits a decomposition over all variable subsets. Entry-wise separability holds if and only if

$R$8

where $R$9 is the Sobol' total-effect index for $g_j^{(r)}:\mathbb{R}\to\mathbb{R}$0. This implies that all variance is accounted for in isolated features, with zero contribution from interactions (Goda, 2013).

Empirically, a Monte Carlo test for separability is available: generate $g_j^{(r)}:\mathbb{R}\to\mathbb{R}$1, evaluate $g_j^{(r)}:\mathbb{R}\to\mathbb{R}$2 as

$g_j^{(r)}:\mathbb{R}\to\mathbb{R}$3

and assemble the test statistic $g_j^{(r)}:\mathbb{R}\to\mathbb{R}$4. The separability null is rejected if $g_j^{(r)}:\mathbb{R}\to\mathbb{R}$5, where $g_j^{(r)}:\mathbb{R}\to\mathbb{R}$6 is the $g_j^{(r)}:\mathbb{R}\to\mathbb{R}$7 quantile of $g_j^{(r)}:\mathbb{R}\to\mathbb{R}$8. Cost is $g_j^{(r)}:\mathbb{R}\to\mathbb{R}$9 function evaluations (Goda, 2013).

3. Tensor Separation Learning (TSL) and Algorithms

Tensor Separation Learning (TSL) is a regression modeling framework that fits sums of differences of positive separable rank-1 products across $j=1,\dots,d$0 stages. The fitted model has the structure

$j=1,\dots,d$1

The algorithm proceeds in a stagewise greedy fashion: - At each stage $j=1,\dots,d$2, residuals are fit with a (possibly signed) sum of separable products. - Each univariate function is estimated via piecewise-constant functions on adaptive partitions, optimized by closed-form ridge least squares. - Model refinement leverages grid-tensor bagging and backbone/tilt gauge fixing, followed by selection, filtering, and geometric-mean averaging of survivors. - After every stage, all scalar weights are refit orthogonally. - The process terminates after $j=1,\dots,d$3 stages, yielding a low-rank glass-box representation. TSL does not jointly optimize all parameters but employs forward stagewise (boosting-style) fitting with greedy orthogonal backfits and local ridge regularization (Liu et al., 29 May 2026).

4. Approximation-Theoretic Guarantees

In the dominant mixed-smoothness Sobolev function class $j=1,\dots,d$4 with “anchored” boundary conditions ($j=1,\dots,d$5 if any $j=1,\dots,d$6), the orthogonal greedy algorithm (OGA) approximation $j=1,\dots,d$7 using nonnegative rank-1 products satisfies

$j=1,\dots,d$8

For TSL with up to $j=1,\dots,d$9 stages (thus $s_r\in\mathbb{R}$0 terms), the rate holds as

$s_r\in\mathbb{R}$1

independent of input dimension $s_r\in\mathbb{R}$2, though the function class (mixed smoothness) may be more stringent in high dimensions (Liu et al., 29 May 2026). This rate is not available for additive-only models in presence of higher-order interactions.

5. Partial Dependence Reconstruction and Identifiability

A salient property of separable models of the TSL type is their reconstructibility from 1D partial dependence (PD) functions. For any stage and feature $s_r\in\mathbb{R}$3,

$s_r\in\mathbb{R}$4

with $s_r\in\mathbb{R}$5 precisely determined by the other factors. The entire separable product can then be recovered up to a global multiplicative factor from products of the marginal PD curves across features: $s_r\in\mathbb{R}$6 where $s_r\in\mathbb{R}$7 and $s_r\in\mathbb{R}$8 are normalization scalars. This correspondence ensures PD-based visualizations are structurally faithful to model components (Liu et al., 29 May 2026). In the backbone–tilt parameterization, the two curves $s_r\in\mathbb{R}$9 and $$f(\mathbf{x}) = \sum_{r=1}^R s_r \prod_{j=1}^d g_j^{(r)}(x_j).$$0 on each feature jointly encode magnitude and signed direction.

6. Computational Complexity and Comparison to Alternatives

TSL’s per-stage computational cost is $$f(\mathbf{x}) = \sum_{r=1}^R s_r \prod_{j=1}^d g_j^{(r)}(x_j).$$1, where $$f(\mathbf{x}) = \sum_{r=1}^R s_r \prod_{j=1}^d g_j^{(r)}(x_j).$$2 is the number of bagged fits, $$f(\mathbf{x}) = \sum_{r=1}^R s_r \prod_{j=1}^d g_j^{(r)}(x_j).$$3 the number of split iterations per grid, $$f(\mathbf{x}) = \sum_{r=1}^R s_r \prod_{j=1}^d g_j^{(r)}(x_j).$$4 the sample size, and $$f(\mathbf{x}) = \sum_{r=1}^R s_r \prod_{j=1}^d g_j^{(r)}(x_j).$$5 the number of features. The total fitting complexity is $$f(\mathbf{x}) = \sum_{r=1}^R s_r \prod_{j=1}^d g_j^{(r)}(x_j).$$6, but substantial parallelism can be exploited in the bagging phase. Histogram binning or prefix-sum caching enables the per-split operation to scale with the number of bins instead of raw sample count.

Classical additive models such as GAMs require $$f(\mathbf{x}) = \sum_{r=1}^R s_r \prod_{j=1}^d g_j^{(r)}(x_j).$$7 for $$f(\mathbf{x}) = \sum_{r=1}^R s_r \prod_{j=1}^d g_j^{(r)}(x_j).$$8 1D subfits but are incapable of capturing higher-order interactions without exponentially many terms. SHAP methods can require $$f(\mathbf{x}) = \sum_{r=1}^R s_r \prod_{j=1}^d g_j^{(r)}(x_j).$$9 model evaluations per query for exact values and are sensitive to signal cancellation under interactions. TSL thus achieves a favorable trade-off: linear scaling in data and feature dimension, and a low-rank multiplication-based structure facilitating interpretability and faithful representation of intricate interactions (Liu et al., 29 May 2026).

7. Empirical Performance and Practical Recommendations

On a battery of 27 OpenML regression benchmarks (with $R=1$0), TSL with total separation rank $R=1$1 or $R=1$2 was typically competitive with EBM (Explainable Boosting Machines) and SepALS (separable ALS regression), often ranking in the top-3 among interpretable techniques and sometimes outperforming them. Concrete case studies (California Housing: latitude $R=1$3 longitude, Bike Sharing: hour $R=1$4 working-day) illustrate TSL's capacity to recover sharp, localized, nonmonotonic effects in PD plots and exact 2D PD surfaces that additive or marginal models miss or misattribute. In synthetic interaction masks (e.g., $R=1$5), TSL identifies quadratic structure invisible to additive projections (such as GAM, SHAP) that yield identically zero main effects (Liu et al., 29 May 2026).

Practical usage for separability assessment recommends transforming all inputs to $R=1$6 (e.g., via marginal CDFs), utilizing a small regularizer ($R=1$7) in the separability test denominator to avoid instability when true separability holds, and verifying Monte Carlo estimator normality for finite samples, with $R=1$8 usually sufficient (Goda, 2013). For high dimensions, the entry-wise test serves as an efficient prescreen before any search for higher-order separable block structure.

---

Key references:

• "Beyond Additive Decompositions: Interpretability Through Separability" (Liu et al., 29 May 2026)
• "On the separability of multivariate functions" (Goda, 2013)