Convolutional Feed-Forward Network (C-FFN)

Updated 9 April 2026

C-FFN is a neural network architecture that applies data-driven, closed-form techniques to build convolutional and fully-connected layers without end-to-end backpropagation.
It utilizes methods like the Saab transform and least-square regression to derive explicit, interpretable filters and promote local layerwise learning.
The architecture supports semi-supervised learning and ensemble methods, offering robust, parameter-efficient alternatives to traditional CNNs.

A Convolutional Feed-Forward Network (C-FFN) is a neural network architecture in which convolutional layers and fully-connected (FC) layers are constructed by data-driven, closed-form, or local-layer approaches, without end-to-end iterative backpropagation. C-FFN encompasses a family of methods unifying linear subspace methods (notably the Saab transform), clustering, least-square regression, per-layer local learning, and—in recent instantiations—layerwise goodness criteria, whose shared theme is the feedforward and typically interpretable design of convolutional models. Architecturally, a C-FFN retains the local spatial processing of classic CNNs, but parameters in convolutional and decision layers are derived by explicit computation from data statistics.

1. Architectural Principles and Design Philosophy

C-FFNs are constructed layerwise, without global backpropagation or end-to-end differentiation. For convolutional layers, filter weights are determined by closed-form subspace projections, typically via Principal Component Analysis (PCA) or its variant, the Saab (Subspace Approximation with Adjusted Bias) transform. The fully connected (FC) layers are formed by cascades of least-squares regressors, possibly using intermediate pseudo-labels. Significant differences from standard BP-trained CNNs include:

No end-to-end gradient descent; each layer is built independently using statistics of intermediate activations.
Convolutional kernels are derived from subspace approximations of local patches, rather than through data labels.
FC layers are realized via one-shot (non-iterative) least-squares regression steps, sometimes using pseudo-labels produced by clustering.
Model parameters are less directly dependent on labeled sample size, yielding strong applicability in semi-supervised learning contexts (Chen et al., 2019, Kuo et al., 2018).

Convolutional layers in C-FFNs predominantly employ the Saab transform, a PCA-based method with explicit bias addition to ensure non-negativity post-activation:

Patch Extraction: Collect overlapping local patches $x \in \mathbb{R}^P$ from input (or previous layers).
DC (Mean) and AC (PCA) Filters: Compute the mean $m = \mathbb{E}[x]$ for the DC filter $w_0 = m/\|m\|$ , then form the AC scatter matrix $S = \sum_i (x_i - m)(x_i - m)^T$ and extract $K-1$ principal directions for AC filters.
Bias Augmentation: For each patch $x$ , compute $y = W^T x + b$ where $b$ annihilates negative responses before ReLU.
Channel-wise PCA (C-PCA): Reduce spatial redundancy separately per channel by additional PCA dimensionality reduction (Chen et al., 2019, Kuo et al., 2018).

The outcome is that each Saab convolutional layer performs an explicit, linear, orthogonal projection of data—yielding interpretable anchor directions—followed by a fixed bias that renders subsequent nonlinear ReLU redundant.

3. Construction of Fully-Connected Layers via Pseudo-Labeling and Regression

FC layers in C-FFN architectures are constructed as cascaded stages of linear least-squared regressors (LSR), possibly using clusters or pseudo-categories as intermediate labels:

Pseudo-label Generation: For each true class, features are clustered (e.g., by k-means) to define $M$ pseudo-categories per class, yielding a pseudo-label vector $y^p \in \mathbb{R}^{L\cdot M}$ .
Unlabeled Data Assignment: For semi-supervised learning, unlabeled features are assigned by soft label distributions using cosine similarity to cluster centroids.
Linear Regression: Stack the feature matrix $m = \mathbb{E}[x]$ 0, with label matrix $m = \mathbb{E}[x]$ 1, and solve $m = \mathbb{E}[x]$ 2; outputs are rectified before passing to subsequent regression stages.
Stagewise Refinement: Multiple LSR stages are performed, with pseudo-labels refined at each stage and final classification performed on original class labels (Chen et al., 2019, Kuo et al., 2018).

4. Semi-Supervised Learning, Quality Scoring, and Ensemble Approaches

C-FFN approaches are inherently suitable for semi-supervised learning due to their layerwise parameterization. Importantly:

Sample Quality Scoring: For each unlabeled instance, a class-specific quality score $m = \mathbb{E}[x]$ 3 is computed based on summed probabilities over a class's pseudo-categories, retaining only those exceeding a given threshold (e.g., top 70–80%) for regression inclusion.
Ensembles: Multiple C-FFNs with varying architectures (e.g., filter sizes, color representations, Laws-filter preprocessing) are independently trained. Their outputs are concatenated and projected (often via PCA) before passing to a second-stage classifier (e.g., RBF-SVM), significantly improving overall classification performance in low-label regimes (Chen et al., 2019).

5. The Forward-Forward Paradigm and Layerwise Local Learning

Recent work has extended the C-FFN framework to include local layerwise learning rules such as the Forward-Forward (FF) algorithm:

Layerwise Goodness: Each layer computes a local “goodness” (e.g., $m = \mathbb{E}[x]$ 4) and applies a local sigmoid loss $m = \mathbb{E}[x]$ 5, using positive and negative inputs superimposed with spatially-extended (sinusoidal) image-labelings.
No Global Backpropagation: Optimization and weight updates are performed using only local, per-layer feedback without backpropagation between layers.
Empirical Parity: Forward-Forward trained C-FFNs can achieve test set performance on MNIST comparable to BP-trained CNNs (e.g., 99.16% for 3 conv layers, F = 128, $m = \mathbb{E}[x]$ 6 kernels) (Scodellaro et al., 2023).
Interpretability: CAM-based analysis shows C-FFN layers learn spatially & semantically meaningful features, and the spatially-extended labeling is critical for information propagation (Scodellaro et al., 2023).

6. Theoretical Insights and Model Class Relationships

Recent advances formalize the relationship between C-FFNs and classical CNNs:

Generalized Node Formalism: The class of generalized feed-forward network (GFFN) nodes corresponds to the subset of generalized convolutional network (GCNN) nodes in which per-input-channel convolutions reduce to scalar multiplications.
Model Projection Technique: By freezing pretrained convolutional filters and learning only per-channel gates, any CNN can be projected to a GFFN form, inheriting parameter efficiency and the potential to transfer feed-forward techniques to convolutional domains.
Parameter Efficiency and Transfer Learning: Model projection allows for up to a 9-fold reduction in trainable conv-layer parameters (for $m = \mathbb{E}[x]$ 7 kernels), retaining competitive or superior transfer learning accuracy on diverse datasets (Ewen et al., 5 Feb 2026).
Inheritance: After model projection, the resulting network statically aligns with GFFN form and can utilize training and regularization schemes for feedforward architectures (Ewen et al., 5 Feb 2026).

7. Empirical Findings, Robustness, and Limitations

Empirical evaluation highlights the following:

C-FFNs trained with data-driven feed-forward methods reach near-BP accuracy on MNIST and CIFAR-10, though a 2–6% gap may persist absent refinement (Kuo et al., 2018).
Robustness properties are complementary: feedforward models exhibit higher resistance to BP-targeted adversarial attacks, but their own vulnerabilities require ensemble approaches for defense.
Trade-offs include interpretability, modularity, faster non-iterative training, and ease of ensembling at the cost of a variable absolute accuracy gap versus end-to-end BP (Kuo et al., 2018, Chen et al., 2019).
C-FFN with Forward-Forward training exhibits monotonic performance gains with kernel size and number of filters, and benefits noticeably from batch and label mixing hyperparameter tuning (Scodellaro et al., 2023).

Summary Table: C-FFN Key Features

Core Principle	Method/Implementation	Empirical Consequence
Conv-layer parameterization	Data-driven subspace (Saab/PCA)	Faster, interpretable filters
FC-layer construction	Multi-stage LSR with clustering	Closed-form training; pseudo-labels enable semi-supervised learning
Ensemble learning	Model diversity, fusion, second-stage classifier	Robustness, performance boost
Local training (FF rule)	Goodness-based layerwise learning	No global backprop needed; BP parity possible
Model projection	Per-channel gating of frozen filters	Parameter efficiency, BP-to-FFN inheritance

C-FFNs present a distinctive, interpretable, and parameter-efficient alternative to classical backpropagation-based CNNs, combining explicit filter design with modular regression-based decision layers, natural extensions to semi-supervised and transfer learning, and emergent robustness through ensembling (Chen et al., 2019, Scodellaro et al., 2023, Ewen et al., 5 Feb 2026, Kuo et al., 2018).