Natural Feature Progressive Framework (NFPF)

• The paper introduces a novel unsupervised active learning method that shifts annotation to a single post-selection step to reduce human intervention.
• It employs a dual SFLM architecture to quantify sample learnability through reconstruction performance without relying on gradient backpropagation.
• Empirical evaluations on vision datasets show that NFPF outperforms existing baselines while achieving competitive performance with fewer computational resources.

Searching arXiv for the specified paper to ground the article and citation. arxiv_search(query="(Liu et al., 6 Oct 2025)", max_results=5, sort_by="submittedDate") Natural Feature Progressive Framework (NFPF) is an unsupervised active learning (UAL) method introduced in “Unsupervised Active Learning via Natural Feature Progressive Framework” (Liu et al., 6 Oct 2025). It is designed for settings in which large-scale human annotation is expensive and iterative oracle interaction remains burdensome. NFPF shifts annotation to a single post-selection step, but departs from prevailing UAL methods by measuring sample importance through reconstruction behavior rather than local gradient-based scoring. Its central components are the Specific Feature Learning Machine (SFLM), which quantifies sample “learnability” via reconstruction, and a progressive selection procedure that combines boundary-oriented initialization with iterative subset growth. The framework is presented as a computationally efficient UAL pipeline based on forward passes and closed-form updates, with experiments on vision datasets reporting performance that surpasses established UAL baselines and approaches supervised active learning methods (Liu et al., 6 Oct 2025).

1. Problem setting and design objectives

NFPF is motivated by the data annotation bottleneck in modern deep learning. Large, high-quality labeled datasets are often prerequisites for strong performance, yet manual annotation is expensive, time-consuming, and in some domains requires expert intervention. Supervised active learning mitigates this cost by querying an oracle for informative instances, but its iterative querying protocol still incurs repeated human involvement and repeated model updates (Liu et al., 6 Oct 2025).

Within that context, UAL redefines the workflow: an informative subset is selected without labels, and labeling is performed only once after selection. NFPF is positioned against prior UAL approaches that rely on linear reconstruction or one-shot selection, which are described as weak for non-linear data and prone to class imbalance. Deep UAL methods such as DUAL and PSS-AL are reported to learn more complex embeddings end-to-end, but to remain dependent on gradient-based scoring, to require heavy training and backpropagation during selection, and to sometimes fail to cover the full data distribution (Liu et al., 6 Oct 2025).

The stated objectives of NFPF are threefold. First, it seeks to measure sample “learnability” through model-reconstruction performance rather than gradients. Second, it progressively grows a labeled-once subset that balances informativeness, associated with boundary samples, and representativeness, associated with data-distribution coverage. Third, it aims to achieve state-of-the-art UAL performance using only forward passes and closed-form updates, thereby avoiding expensive backpropagation during selection (Liu et al., 6 Oct 2025).

A common misconception would be to treat NFPF as merely another one-shot subset selector. The framework is explicitly progressive: it begins with a seed subset and then iteratively augments that subset according to a reconstruction-based scoring rule. This suggests that its contribution lies not only in a new score, but also in the coupling of initialization and iterative refinement.

2. Specific Feature Learning Machine

At the core of NFPF is the Specific Feature Learning Machine, a lightweight two-layer feedforward autoencoder that serves as the basic scoring mechanism (Liu et al., 6 Oct 2025). The SFLM has input dimension $d$, a hidden layer with $H$ neurons and activation function $f(\cdot)$, such as sigmoid or sine, and a linear output layer parameterized by decoder weights $\beta$. Its parameters comprise encoder weights $\{\mathbf{a}_h \in \mathbb{R}^d\}_{h=1}^H$, biases $\{b_h\}$, and the decoder weight matrix $\beta \in \mathbb{R}^{H \times d}$.

Given a training set $$\mathbf{X} = [\mathbf{x}_1,\dots,\mathbf{x}_N] \in \mathbb{R}^{d \times N}$$, the model minimizes reconstruction error,

$$E(\theta) = \sum_{i=1}^N \|\mathbf{x}_i - \hat{\mathbf{x}}_i\|_2^2,$$

where $$\hat{\mathbf{x}}_i = f_{\rm dec}(f_{\rm enc}(\mathbf{x}_i))$$. Once the hidden-layer outputs $H$0 are computed, the decoder is obtained in closed form via a regularized pseudoinverse,

$H$1

with

$H$2

where $H$3 is a small ridge parameter for numerical stability (Liu et al., 6 Oct 2025).

To assess reconstruction quality, NFPF uses the Pearson correlation coefficient between the original sample $H$4 and its reconstruction $H$5:

$H$6

A larger $H$7 indicates better reconstruction and therefore greater familiarity of the sample to the model (Liu et al., 6 Oct 2025).

The SFLM is assigned two roles. It quantifies how well a sample is reconstructed, which operationalizes “learnability,” and it supports the use of different model capacities to separate global and local representations. This dual use is central to the later scoring rule.

3. Dual-capacity scoring and learnability

NFPF maintains two SFLMs with different capacities (Liu et al., 6 Oct 2025). The first is a Reference Model $H$8 with a large hidden size $H$9, trained on the entire unlabeled set $f(\cdot)$0. The second is a Current Model $f(\cdot)$1 with a smaller hidden size $f(\cdot)$2, trained only on the selected subset $f(\cdot)$3.

The rationale is explicit. If the current subset model reconstructs a sample poorly, then the sample is hard for the existing subset and is therefore informative. If the reference model reconstructs that same sample well, then the sample is globally representative rather than merely anomalous or noisy. NFPF combines these two effects into a learnability score for an unselected sample $f(\cdot)$4:

$f(\cdot)$5

An equivalent loss-form expression is also given:

$f(\cdot)$6

Under this formulation, the first term encodes hardness with respect to the currently selected subset, while the second term encodes representativeness with respect to the full unlabeled pool (Liu et al., 6 Oct 2025).

The framework’s theoretical interpretation is that the discrepancy $f(\cdot)$7 approximates how much a new sample would reduce the global reconstruction loss, making it analogous to an implicit gradient-based gain measure, but computed entirely via forward passes. This suggests that NFPF substitutes an inter-model reconstruction discrepancy for the local gradient proxies used in earlier UAL methods.

The distinction is important for understanding what NFPF is and is not optimizing. It is not estimating uncertainty from labels or pseudo-labels; rather, it is measuring the mismatch between a global reconstruction model and a subset-conditioned reconstruction model.

4. Reconstruction Difference initialization

Before progressive selection begins, NFPF constructs an initial seed subset $f(\cdot)$8 using the Reconstruction Difference (RD) metric (Liu et al., 6 Oct 2025). This phase is intended to identify samples near class-boundary regions in feature space.

The initialization starts from an assumption that the number of classes $f(\cdot)$9 is known. K-Means is run on $\beta$0 to produce $\beta$1 cluster centers, termed “cores.” For each core $\beta$2, a miniature SFLM $\beta$3 is trained using only the points assigned to that core, and its core reconstruction score is computed as

$\beta$4

For each unlabeled sample $\beta$5 and each core model $\beta$6, NFPF then computes

$\beta$7

A small $\beta$8 means that $\beta$9 is reconstructed by $\{\mathbf{a}_h \in \mathbb{R}^d\}_{h=1}^H$0 about as well as the data assigned to that core. Let $\{\mathbf{a}_h \in \mathbb{R}^d\}_{h=1}^H$1 and $\{\mathbf{a}_h \in \mathbb{R}^d\}_{h=1}^H$2 denote the nearest and second-nearest cores to $\{\mathbf{a}_h \in \mathbb{R}^d\}_{h=1}^H$3 in terms of $\{\mathbf{a}_h \in \mathbb{R}^d\}_{h=1}^H$4. The RD score is then defined as

$\{\mathbf{a}_h \in \mathbb{R}^d\}_{h=1}^H$5

A sample with $\{\mathbf{a}_h \in \mathbb{R}^d\}_{h=1}^H$6 is interpreted as lying near the decision boundary between its two closest cores (Liu et al., 6 Oct 2025).

The initial subset is obtained by ranking all $\{\mathbf{a}_h \in \mathbb{R}^d\}_{h=1}^H$7 in ascending order of $\{\mathbf{a}_h \in \mathbb{R}^d\}_{h=1}^H$8 and taking the top $\{\mathbf{a}_h \in \mathbb{R}^d\}_{h=1}^H$9 samples as $\{b_h\}$0. In effect, the seed selection favors ambiguity between neighboring core reconstructions. The data also note a limitation-like condition embedded in the method: the number of classes is assumed to be known. For readers accustomed to broader UAL settings, this is a structural assumption of the initialization stage rather than a generic property of all UAL pipelines.

5. Progressive workflow

NFPF proceeds in two phases: seed initialization by RD and progressive unsupervised active learning (Liu et al., 6 Oct 2025). The workflow is specified as Algorithm 1.

During initialization, K-Means is applied to the unlabeled set to obtain $\{b_h\}$1 cores. A small-hidden-size SFLM $\{b_h\}$2 is trained on each core, the corresponding core score $\{b_h\}$3 is computed, and every sample is assigned an RD score. The top $\{b_h\}$4 samples with smallest RD form the initial subset $\{b_h\}$5.

During progressive selection, the framework trains the Reference SFLM $\{b_h\}$6 on all of $\{b_h\}$7 with large $\{b_h\}$8 and the Current SFLM $\{b_h\}$9 on the seed subset with small $\beta \in \mathbb{R}^{H \times d}$0. Letting $\beta \in \mathbb{R}^{H \times d}$1, the method repeats the following until the target subset size $\beta \in \mathbb{R}^{H \times d}$2 is reached: compute

$\beta \in \mathbb{R}^{H \times d}$3

for every $\beta \in \mathbb{R}^{H \times d}$4, select the top-$\beta \in \mathbb{R}^{H \times d}$5 samples with largest $\beta \in \mathbb{R}^{H \times d}$6, add them to $\beta \in \mathbb{R}^{H \times d}$7, remove them from $\beta \in \mathbb{R}^{H \times d}$8, and retrain $\beta \in \mathbb{R}^{H \times d}$9 on the updated subset. The loop runs $$\mathbf{X} = [\mathbf{x}_1,\dots,\mathbf{x}_N] \in \mathbb{R}^{d \times N}$$0 cycles, and the final selected subset is returned for one-time oracle labeling (Liu et al., 6 Oct 2025).

Two implementation features are emphasized. First, sample scoring requires only forward passes. Second, SFLM training uses closed-form pseudoinverse updates rather than backpropagation. This directly differentiates NFPF from selection procedures that repeatedly optimize deep models during querying. It should therefore not be conflated with conventional supervised active learning loops, even when benchmark comparisons place it close to supervised AL performance.

6. Empirical performance and ablations

The evaluation protocol uses CIFAR-10, CIFAR-100, and Tiny-ImageNet, with ResNet-18 as the downstream classification backbone. Comparisons include UAL methods DUAL, PSS-AL, K-means, Entropy, and Random, as well as supervised active learning methods Noise Stability and BAL. The primary metric is classification accuracy on a held-out test set as a function of selected budget $$\mathbf{X} = [\mathbf{x}_1,\dots,\mathbf{x}_N] \in \mathbb{R}^{d \times N}$$1 (Liu et al., 6 Oct 2025).

Dataset and budget	NFPF	Reported comparison
CIFAR-10, $$\mathbf{X} = [\mathbf{x}_1,\dots,\mathbf{x}_N] \in \mathbb{R}^{d \times N}$$2	84.70%	DUAL 84.18%, PSS-AL 75.77%, Noise 83.65%, BAL 84.60%
CIFAR-100, $$\mathbf{X} = [\mathbf{x}_1,\dots,\mathbf{x}_N] \in \mathbb{R}^{d \times N}$$3	50.98%	best supervised 51.28%
Tiny-ImageNet, $$\mathbf{X} = [\mathbf{x}_1,\dots,\mathbf{x}_N] \in \mathbb{R}^{d \times N}$$4	32.24%	BAL 33.28%

On CIFAR-10 at $$\mathbf{X} = [\mathbf{x}_1,\dots,\mathbf{x}_N] \in \mathbb{R}^{d \times N}$$5, which is approximately 8% of 60k, NFPF reaches 84.70%, compared with 84.18% for DUAL, 75.77% for PSS-AL, 83.65% for Noise, and 84.60% for BAL. The reported gain over the next best UAL method, DUAL, is +0.52%, and the result is stated to meet or exceed recent supervised AL results. On CIFAR-100 at $$\mathbf{X} = [\mathbf{x}_1,\dots,\mathbf{x}_N] \in \mathbb{R}^{d \times N}$$6, NFPF reaches 50.98% versus 51.28% for the best supervised comparator. On Tiny-ImageNet at $$\mathbf{X} = [\mathbf{x}_1,\dots,\mathbf{x}_N] \in \mathbb{R}^{d \times N}$$7, NFPF reaches 32.24% versus 33.28% for BAL, while outperforming all unsupervised baselines by 2–5 percentage points (Liu et al., 6 Oct 2025).

The convergence-speed result is reported on CIFAR-100: to reach a fixed target accuracy, NFPF requires 7×–20× fewer gradient steps than uniform sampling or other UAL approaches. Although NFPF avoids gradient backpropagation during sample selection itself, this comparison is framed at the level of end-to-end training needed to achieve the target accuracy (Liu et al., 6 Oct 2025).

A broader contribution statement in the source describes empirical validation across nine datasets. Since the detailed exposition enumerates three vision benchmarks, a plausible implication is that the full paper contains additional dataset-level evidence beyond the subset of results summarized here.

Ablation studies and qualitative analysis further characterize the method. Under symmetric label noise up to 40%, NFPF’s accuracy degrades only about 2–3 percentage points at 40% noise, whereas DUAL and PSS-AL drop 4–6 percentage points, which is presented as evidence that NFPF avoids selecting noisy outliers. For parameter sensitivity, the initial seed fraction $$\mathbf{X} = [\mathbf{x}_1,\dots,\mathbf{x}_N] \in \mathbb{R}^{d \times N}$$8 is reported to work best in the range $$\mathbf{X} = [\mathbf{x}_1,\dots,\mathbf{x}_N] \in \mathbb{R}^{d \times N}$$9; larger seeds improve boundary coverage but risk imbalance. Smaller per-cycle size $$E(\theta) = \sum_{i=1}^N \|\mathbf{x}_i - \hat{\mathbf{x}}_i\|_2^2,$$0 yields slightly better stability, at the cost of more retraining rounds. In t-SNE visualizations, features learned by ResNet-18 trained on the NFPF-selected subset exhibit tighter, well-separated class clusters than Random, K-Means, Entropy, DUAL, and PSS-AL. The same figure is reported to show nearly uniform per-class selection frequency, thereby avoiding the class imbalance that often affects boundary-only methods (Liu et al., 6 Oct 2025).

7. Complexity, interpretation, and research directions

The computational profile of NFPF is summarized in terms of SFLM training and scoring (Liu et al., 6 Oct 2025). SFLM training per cycle is a closed-form pseudoinverse on a subset of size at most $$E(\theta) = \sum_{i=1}^N \|\mathbf{x}_i - \hat{\mathbf{x}}_i\|_2^2,$$1, with complexity approximately $$E(\theta) = \sum_{i=1}^N \|\mathbf{x}_i - \hat{\mathbf{x}}_i\|_2^2,$$2 for hidden layer size $$E(\theta) = \sum_{i=1}^N \|\mathbf{x}_i - \hat{\mathbf{x}}_i\|_2^2,$$3. Scoring all $$E(\theta) = \sum_{i=1}^N \|\mathbf{x}_i - \hat{\mathbf{x}}_i\|_2^2,$$4 unselected samples in each cycle requires one forward pass, with complexity $$E(\theta) = \sum_{i=1}^N \|\mathbf{x}_i - \hat{\mathbf{x}}_i\|_2^2,$$5. The framework therefore avoids gradient backpropagation and large-scale deep training during selection.

This efficiency claim is not only algorithmic but conceptual. NFPF replaces local gradient-based sample scoring with a reconstruction-based discrepancy between a global model and a subset model. The method’s theoretical justification states that this discrepancy approximates the effect a new sample would have on the global reconstruction loss. This suggests a reinterpretation of active sample value in terms of representation mismatch rather than uncertainty estimation.

Several forward-looking implications are explicitly identified in the source. These include extending SFLM-style scoring to multi-modal or sequential data, automatically determining $$E(\theta) = \sum_{i=1}^N \|\mathbf{x}_i - \hat{\mathbf{x}}_i\|_2^2,$$6 and $$E(\theta) = \sum_{i=1}^N \|\mathbf{x}_i - \hat{\mathbf{x}}_i\|_2^2,$$7 via meta-learning, and developing theoretical bounds linking reconstruction difference to generalization gain (Liu et al., 6 Oct 2025). These are framed as implications for future research rather than established properties of the current method.

The framework’s principal contributions are summarized in four points in the source: it is described as the first to leverage lightweight autoencoder “learnability” in an unsupervised AL setting; it introduces a progressive selection strategy bridging global and local model capacities; it offers a computationally efficient pipeline requiring only forward computation and pseudoinverse updates; and it reports empirical validation across nine datasets with strong gains in both UAL and supervised AL comparisons (Liu et al., 6 Oct 2025). Taken together, these claims position NFPF as a reconstruction-centric formulation of UAL whose distinguishing features are progressive subset growth, dual-capacity modeling, and boundary-aware seed initialization.