Subspace Approximation with Adjusted Bias (Saab)

• Saab is a data-driven transform that augments PCA with an explicit bias to ensure nonnegative outputs and decouple DC/AC components.
• It is applied as a feedforward alternative in CNNs, an orthonormal transform in video coding, and a feature extractor in medical-image segmentation.
• Saab’s closed-form parameter determination and interpretability support lightweight models, though challenges remain for extremely deep or large-scale tasks.

Subspace Approximation with Adjusted Bias (Saab) is a data-driven transform that augments PCA-style subspace projection with an explicit bias so that projected responses remain nonnegative after projection, thereby addressing the interaction between zero-mean projections and ReLU-style nonlinearities while retaining an interpretable DC/AC decomposition. In the cited literature, Saab appears as a feedforward alternative to backpropagation-designed convolutional layers, as an explainable orthonormal transform for intra video coding, and as a backpropagation-free feature extractor for medical-image segmentation; across these settings, its parameters are obtained from sample statistics such as means, covariances, eigenvectors, and extreme projection values rather than gradient descent (Kuo et al., 2018, Li et al., 2020, Liu et al., 2021, Rahman, 2 Jul 2026).

1. Origins and problem setting

Saab was introduced in the context of interpretable feedforward design for convolutional neural networks, where direct use of PCA filters creates two stated difficulties: sign confusion and the destructive effect of ReLU on negative projections. In that setting, PCA filters produce outputs $y_k = a_k^T x$, and ReLU clips all negative responses to zero; standard PCA also has no built-in bias term to lift negative projections into the positive regime. Saab addresses this by augmenting PCA with a constant bias so that all channel outputs are nonnegative and by decoupling DC and AC subspaces so that the bias remains in the DC subspace of the output space (Kuo et al., 2018).

A second formulation arises in transform coding, where transform design is posed as an optimization problem over coding-relevant criteria such as energy compaction and decorrelation. There, Saab is described as an explainable machine learning based transform and compared directly with the mainstream Discrete Cosine Transform (DCT) and with the Karhunen–Loève Transform (KLT). The transform is designed off-line from training data and then deployed as a fixed orthonormal transform without per-block adaptation overhead (Li et al., 2020).

Later work embeds Saab in successive subspace learning and in multi-scale, wavelet-coupled pipelines for medical segmentation. In those systems, the same core idea—a covariance-derived orthonormal projection plus an analytically specified bias—is used as a front end for feature extraction, with supervised selectors and classical classifiers handling the final prediction stage (Liu et al., 2021, Rahman, 2 Jul 2026).

2. Mathematical structure

In the feedforward-CNN formulation, Saab begins with an orthogonal decomposition of the input space into DC and AC components. For $x \in \mathbb{R}^N$,

$$a_0 = \frac{1}{\sqrt{N}}[1,\dots,1]^T,\qquad x_{\mathrm{DC}} = a_0^T x,\qquad x_{\mathrm{AC}} = x - a_0 a_0^T x.$$

The AC anchors are the top eigenvectors of the covariance of $x_{\mathrm{AC}}$, and the transform output is written

$y = W^T x + b,$

where the columns of $W$ are orthonormal anchors and the bias vector $b$ is chosen so that ReLU becomes the identity on the transformed outputs (Kuo et al., 2018).

In the transform-coding formulation, the one-stage Saab transform for a $K$-dimensional vector $x$ is written

$$y_0 = \frac{1}{\sqrt{K}}\,1^T x,\qquad z' = x - y_0\,1,\qquad y_k = a_k^T z' + b_k,\quad k=1,\dots,K-1.$$

Here $x \in \mathbb{R}^N$0 is the DC coefficient, the AC kernels $x \in \mathbb{R}^N$1 are orthonormal eigenvectors of the covariance of $x \in \mathbb{R}^N$2, and the inverse takes the form $x \in \mathbb{R}^N$3 because of orthogonality (Li et al., 2020).

In MedSaab-US, Saab is specialized to wavelet subbands and patch-based local statistics. For each subband index $x \in \mathbb{R}^N$4 and patch size $x \in \mathbb{R}^N$5, overlapping $x \in \mathbb{R}^N$6 patches $x \in \mathbb{R}^N$7 are sampled, mean-adjusted by

$x \in \mathbb{R}^N$8

and used to form the covariance

$x \in \mathbb{R}^N$9

The top $$a_0 = \frac{1}{\sqrt{N}}[1,\dots,1]^T,\qquad x_{\mathrm{DC}} = a_0^T x,\qquad x_{\mathrm{AC}} = x - a_0 a_0^T x.$$0 eigenvectors are retained, yielding the feature map

$$a_0 = \frac{1}{\sqrt{N}}[1,\dots,1]^T,\qquad x_{\mathrm{DC}} = a_0^T x,\qquad x_{\mathrm{AC}} = x - a_0 a_0^T x.$$1

Taken together, these formulations suggest that Saab is characterized operationally by two invariants: covariance-eigenvector subspace approximation and explicit bias adjustment (Rahman, 2 Jul 2026).

3. Adjusted bias as the defining departure from standard PCA

The bias term is the central distinction between Saab and ordinary PCA. Standard PCA uses $$a_0 = \frac{1}{\sqrt{N}}[1,\dots,1]^T,\qquad x_{\mathrm{DC}} = a_0^T x,\qquad x_{\mathrm{AC}} = x - a_0 a_0^T x.$$2 and produces zero-mean outputs that can be positive or negative. Saab adds a constant offset to each projected dimension so that the outputs are nonnegative and the DC channel can be handled separately; in the feedforward interpretation, this makes the ReLU stage non-destructive, and in MedSaab-US it is described as enabling a built-in ReLU without information collapse (Kuo et al., 2018, Rahman, 2 Jul 2026).

The literature does not present a single universal bias rule. In the original feedforward formulation, the constraints are that $$a_0 = \frac{1}{\sqrt{N}}[1,\dots,1]^T,\qquad x_{\mathrm{DC}} = a_0^T x,\qquad x_{\mathrm{AC}} = x - a_0 a_0^T x.$$3 for all $$a_0 = \frac{1}{\sqrt{N}}[1,\dots,1]^T,\qquad x_{\mathrm{DC}} = a_0^T x,\qquad x_{\mathrm{AC}} = x - a_0 a_0^T x.$$4 and all $$a_0 = \frac{1}{\sqrt{N}}[1,\dots,1]^T,\qquad x_{\mathrm{DC}} = a_0^T x,\qquad x_{\mathrm{AC}} = x - a_0 a_0^T x.$$5, and that the bias vector lie in the DC subspace of the output space; a sufficient bound is

$$a_0 = \frac{1}{\sqrt{N}}[1,\dots,1]^T,\qquad x_{\mathrm{DC}} = a_0^T x,\qquad x_{\mathrm{AC}} = x - a_0 a_0^T x.$$6

with the practical choice $$a_0 = \frac{1}{\sqrt{N}}[1,\dots,1]^T,\qquad x_{\mathrm{DC}} = a_0^T x,\qquad x_{\mathrm{AC}} = x - a_0 a_0^T x.$$7 for small $$a_0 = \frac{1}{\sqrt{N}}[1,\dots,1]^T,\qquad x_{\mathrm{DC}} = a_0^T x,\qquad x_{\mathrm{AC}} = x - a_0 a_0^T x.$$8. In the cine-MRI segmentation formulation, a common choice is

$$a_0 = \frac{1}{\sqrt{N}}[1,\dots,1]^T,\qquad x_{\mathrm{DC}} = a_0^T x,\qquad x_{\mathrm{AC}} = x - a_0 a_0^T x.$$9

with $x_{\mathrm{AC}}$0 selected large enough that every training projection remains nonnegative. In transform coding, the practical rule is $x_{\mathrm{AC}}$1 over the training set. In MedSaab-US, the bias is determined analytically on a per-channel basis by

$x_{\mathrm{AC}}$2

which guarantees nonnegativity on the sampled training patches (Liu et al., 2021, Li et al., 2020).

A common misconception is to equate Saab with ordinary PCA plus a post hoc shift. The cited formulations indicate a stronger claim: the bias is built into the transform design so that sign ambiguity, DC/AC handling, and subsequent stage composition are addressed at the representation level rather than by a separately learned nonlinearity (Kuo et al., 2018, Li et al., 2020).

4. Architectural realizations

In interpretable feedforward CNN design, multiple Saab transforms are cascaded to yield multiple convolutional layers, while fully connected layers are constructed using a cascade of multi-stage linear least squared regressors. The one-pass learning procedure computes the DC anchor, removes the DC component, estimates the AC covariance, extracts the top eigenvectors, selects the bias from training-set norms, and then applies convolution and identity ReLU at each spatial location. The reported architectures include LeNet-5 and a color-image variant with first convolution $x_{\mathrm{AC}}$3, second convolution $x_{\mathrm{AC}}$4, and fully connected layers $x_{\mathrm{AC}}$5 (Kuo et al., 2018).

In intra video coding, Saab is used as an off-line learned transform for $x_{\mathrm{AC}}$6 residual blocks. Residuals are collected per intra mode, grouped by fine-grained or coarse-grained grouping, and used to learn 24 distinct Saab transforms $x_{\mathrm{AC}}$7. Three encoder integration strategies are defined. In $x_{\mathrm{AC}}$8, selected modes use prelearned SBT kernels and the rest use DCT, with no per-TU flag. In $x_{\mathrm{AC}}$9, DCT is active everywhere, but for modes with available SBT kernels the encoder performs rate–distortion comparison and signals the choice with a 1-bit flag per TU. In $y = W^T x + b,$0, DCT plus all SBT kernels are activated, and every TU and mode chooses between DCT and the corresponding SBT via rate–distortion optimization (Li et al., 2020).

In cine MRI segmentation, Saab is embedded in a successive subspace learning pipeline comprising sequential expansion of near-to-far neighborhood at different resolutions, channel-wise subspace approximation using Saab transform for unsupervised dimension reduction, class-wise entropy guided feature selection for supervised dimension reduction, concatenation of features and pixel-wise classification with gradient boost, and conditional random field for post-processing. The architecture uses $y = W^T x + b,$1 Saab-based SSL units with interleaved max-pooling and empirically chosen anchors

$y = W^T x + b,$2

with neighborhood size fixed at $y = W^T x + b,$3 in all stages (Liu et al., 2021).

In thyroid ultrasound segmentation, MedSaab-US combines a 2-level Daubechies-4 DWT with multi-scale channel-wise Saab transforms at patch sizes $y = W^T x + b,$4, $y = W^T x + b,$5, and $y = W^T x + b,$6 on seven subbands. For each $y = W^T x + b,$7 pair, $y = W^T x + b,$8 output channels are kept, so the system produces $y = W^T x + b,$9 Saab features per pixel. These are concatenated, augmented with three positional features $W$0, reduced by Label-Assisted Greedy (LAG) selection from 171 dimensions to the top 60 features, and then classified pixel-wise by an XGBoost ensemble with 600 trees, max depth 7, and learning rate 0.05. The framework is described as grounded in the Green Learning paradigm, and all Saab parameters are determined in closed form from $W$1 patches drawn from 400 training images (Rahman, 2 Jul 2026).

5. Empirical record across application domains

The empirical literature on Saab is heterogeneous, spanning classification, transform coding, and medical segmentation. The reported results therefore quantify task-specific behavior rather than a single unified benchmark (Kuo et al., 2018, Li et al., 2020, Liu et al., 2021, Rahman, 2 Jul 2026).

Domain	Saab configuration	Reported outcome
Feedforward CNN classification	Saab conv layers, FF and Hybrid variants	MNIST: FF 97.2%, Hybrid 98.4%, BP 99.9%; CIFAR-10: FF 62%, Hybrid 64%, BP 68%
HEVC intra coding	8$W$28 Saab transform with $W$3, $W$4, $W$5	Avg. BDBR = -1.41%, -2.59%, -3.07%; max -10.00%
Cine MRI segmentation	SSL + Saab	With 50 training subjects: Avg Dice 85.12 vs U-Net 83.35 and AttenU-Net 82.91; with 60 training subjects: Avg Dice = 86.03%
Thyroid ultrasound segmentation	MedSaab-US	Mean Dice coefficient 0.4784 +/- 0.2190, precision 0.5768, recall 0.5604; model footprint under 500K parameters; CPU-only inference in approximately 0.3 seconds per image

Additional quantitative findings refine this picture. In transform coding, the decorrelation cost averaged over several sequences and QPs is reported as KLT: 0, Saab$W$6: $W$7, Saab$W$8: $W$9, and DCT: $b$0; at 90% cumulative energy, one-stage Saab typically outperforms DCT by 5–10%. In the same study, under the full rate–distortion selection strategy, on average 46.1% of TUs choose Saab over DCT, with peaks at $b$1 for textured sequences (Li et al., 2020).

In the feedforward-CNN study, adversarial robustness is also reported. Against attacks targeting the BP-trained net, BP accuracy drops to 6%/1%/2% on MNIST and 15%/12%/15% on CIFAR-10 under FGS, BIM, and DeepFool, whereas the FF design retains 56%/46%/96% on MNIST and 21%/31%/59% on CIFAR-10. The same study states that attacks targeting the FF-designed net leave FF suffering less under high-quality attacks than BP (Kuo et al., 2018).

6. Interpretability, efficiency, and limitations

Saab is consistently positioned as an interpretable alternative to backpropagation-based filter learning. In the feedforward formulation, the filters are data-driven PCA patterns, including Gabor-like edge detectors in early layers and compound contours in deeper layers; orthogonality yields uncorrelated feature channels, and the explicit bias removal of ReLU nonlinearity is part of the analytic construction. In medical segmentation, the same property is used to justify a lightweight and interpretable machine learning model; in transform coding, it is framed as explainable machine learning based transform design (Kuo et al., 2018, Liu et al., 2021, Li et al., 2020).

Its computational profile depends strongly on the deployment domain. In MedSaab-US, each Saab transform at scale $b$2 has $b$3 float weights for $b$4 plus $b$5 bias scalars, giving an overall Saab parameter count of approximately 33,080 parameters, which is described as negligible compared to the approximately 500 K total pipeline parameters because most parameters reside in the XGBoost ensemble. The corresponding computational cost per pixel is

$b$6

multiply–adds per pixel, and for a $b$7 image this is approximately $b$8 operations, reported to run in approximately $b$9 s on a 2 GHz dual-core CPU (Rahman, 2 Jul 2026).

The medical-segmentation studies emphasize parameter efficiency. The cine-MRI SSL + Saab model has 0.03 M parameters and is reported as having approximately 200$K$0 fewer parameters than U-Net. MedSaab-US reports a model footprint under 500K parameters and CPU-only inference. These are contrasted with deep learning systems that use millions of parameters and GPU-dependent training via backpropagation (Liu et al., 2021, Rahman, 2 Jul 2026).

The limitations are equally explicit. In the feedforward-CNN study, FF performance lags BP on both MNIST and CIFAR-10, and the discussion notes that further work is needed on extremely deep nets and large-scale tasks; bias selection also requires norm bounds on inputs. In video coding, Saab improves coding efficiency but incurs nontrivial computational overhead relative to optimized integer-DCT, with encoding time ratios of 202%, 240%, and 284% and decoding time ratios of 148%, 134%, and 138% for $K$1, $K$2, and $K$3, respectively. In thyroid ultrasound segmentation, MedSaab-US is explicitly presented as an exploratory non-DL baseline and includes analysis of the specific challenges posed by isoechoic nodules (Kuo et al., 2018, Li et al., 2020, Rahman, 2 Jul 2026).

A plausible implication is that Saab is best understood not as a drop-in replacement for every backpropagation-based module, but as a mathematically tractable design pattern for situations where closed-form parameter determination, interpretability, and modest model size are primary constraints.