FPS: Feature-Space Boundary Optimization

• FPS is a domain adaptation framework that reframes adaptation as a search for optimal decision boundaries within a transferable, fixed feature space.
• The method freezes the feature encoder and optimizes only the classifier head, reducing computational load while maintaining model interpretability.
• FPS leverages entropy minimization, category balancing, and consistency regularization to realign target data clusters with more effective decision boundaries.

Feature-Space Planes Searcher (FPS) is a domain adaptation framework that treats unsupervised domain adaptation primarily as a problem of decision-boundary misalignment in an otherwise still-useful pretrained feature space, rather than as a failure of the feature extractor itself. In the formulation introduced in "Feature-Space Planes Searcher: A Universal Domain Adaptation Framework for Interpretability and Computational Efficiency," FPS freezes the encoder, uses unlabeled target data together with source supervision, and searches for a better separating plane by optimizing only the classification head. The method is motivated by the claim that modern pretrained encoders preserve transferable discriminative structure across domains through intra-class clustering and inter-class separation, so adaptation can often be reduced to boundary search in fixed feature space rather than full-network fine-tuning (Cheng et al., 26 Aug 2025).

1. Problem setting and core claim

FPS addresses domain shift, defined as degraded model performance during transition from labeled source domains to unlabeled target domains. The framework is positioned against standard unsupervised domain adaptation methods that fine-tune the feature extractor to reduce source-target discrepancy. That strategy is described as expensive, hard to interpret, and unstable for large pretrained models, especially when the adaptation dataset is much smaller than pretraining data (Cheng et al., 26 Aug 2025).

The central claim of FPS is that, for modern pretrained encoders, the main obstacle in unsupervised domain adaptation is usually not that the feature extractor fails on the target domain, but that the classifier’s decision boundary is misaligned with the target distribution in an otherwise still-useful feature space. Concretely, the framework starts from three premises: pretrained models already produce transferable features; source and target features still exhibit class structure; and performance drops mainly because the existing decision boundary no longer cuts the feature space in the right place. Adaptation is therefore reframed as searching for a better decision plane within a frozen feature space rather than modifying the embedding space itself (Cheng et al., 26 Aug 2025).

This redefinition has methodological consequences. Because only the final classifier layer is updated, FPS explicitly rejects the assumption that successful domain adaptation must reshape representations. A plausible implication is that FPS is best understood as a boundary-optimization method built on pretrained geometry rather than as a representation-alignment method.

2. Geometric assumptions in feature space

A major premise of FPS is that strong pretrained models induce a stable geometry in feature space. The paper characterizes this geometry through three properties: intra-class clustering, inter-class separation, and boundary misalignment. Samples from the same class tend to remain close together; different classes remain well-separated; and domain shift mostly changes where the classifier boundary should lie, not the basic structure of the clusters (Cheng et al., 26 Aug 2025).

The support offered for this premise consists of t-SNE visualizations and distance matrices. These are used to argue that, in both source and target domains, the same classes still cluster, and that cross-domain degradation is not primarily feature collapse but wrong slicing by the classifier. On this view, a good encoder already provides a transferable arrangement of points, while adaptation should move the decision boundary into a lower-density region and align it with target clusters (Cheng et al., 26 Aug 2025).

The framework also specifies qualitative requirements for the adapted boundary. It should avoid cutting through dense class regions, balance class predictions instead of collapsing into a few classes, and preserve source knowledge while accommodating target shift. These requirements become operational in the loss design. A common misconception is that entropy-based boundary adaptation is equivalent to arbitrary confidence maximization; FPS explicitly adds balancing and consistency terms to prevent degenerate class assignments and unstable boundaries.

3. Optimization of the classifier head

FPS freezes the feature encoder completely and optimizes only the linear classification head. For feature vector $\mathbf{x}_i$, the classifier is written as

$$\mathbf{y}_i = \mathbf{W}^{\top}\mathbf{x}_i + \mathbf{b}, \qquad \mathbf{p}_i = \mathrm{Softmax}(\mathbf{y}_i).$$

The search problem is expressed as optimization over $\mathbf{\Theta}=\{\mathbf{W},\mathbf{b}\}$ in fixed feature space rather than end-to-end parameter adaptation (Cheng et al., 26 Aug 2025).

The main objective is introduced in posterior form,

$$\arg\max_{\mathbf{\Theta}} \log p(\mathbf{\Theta}\mid \mathbf{D}_t,\mathbf{D}_s),$$

which, by Bayes’ rule and conditional independence, is rewritten as

$$\arg\max_{\mathbf{\Theta}} \left(\log p(\mathbf{D}_s\mid \mathbf{\Theta}) + \log p(\mathbf{D}_t\mid \mathbf{\Theta})\right).$$

The source term is handled by ordinary cross-entropy, while the target term is made tractable through latent geometric priors (Cheng et al., 26 Aug 2025).

The first unsupervised constraint is sample entropy (SE). For each target sample,

$$\mathrm{SE}(\mathbf{x}_t^i) = -\mathbf{p}(\mathbf{x}_t^i)^\top \log \mathbf{p}(\mathbf{x}_t^i),$$

with loss

$$\mathcal{L}_{\text{SE}} = -\mathbb{E}_{\mathbf{x}_t^i\in \mathbf{X}_t}[\mathrm{SE}(\mathbf{x}_t^i)].$$

Minimizing SE encourages confident predictions and pushes the decision boundary into low-density regions (Cheng et al., 26 Aug 2025).

Because SE alone can collapse many samples into one or a few classes, FPS also uses category entropy (CE), defined over the mean predicted class distribution:

$$\mathrm{CE}(\mathbf{D}_t) = -\mathbb{E}[\mathbf{p}(\mathbf{x}_t^i)]^\top \log \mathbb{E}[\mathbf{p}(\mathbf{x}_t^i)],$$

with

$$\mathcal{L}_{\text{CE}} = -\mathrm{CE}(\mathbf{D}_t).$$

The entropy objective is then given as

$$\mathcal{L}_{\text{entropy}} = \alpha \mathcal{L}_{\text{SE}} + (\alpha-1)\mathcal{L}_{\text{CE}}.$$

In the paper’s interpretation, SE is the main unsupervised driver, while CE prevents collapse by encouraging class balance (Cheng et al., 26 Aug 2025).

FPS further introduces random pooling consistency regularization (CR). By randomly reweighting patch features, the method generates two views and penalizes prediction disagreement:

$$\mathbf{y}_i = \mathbf{W}^{\top}\mathbf{x}_i + \mathbf{b}, \qquad \mathbf{p}_i = \mathrm{Softmax}(\mathbf{y}_i).$$0

with

$$\mathbf{y}_i = \mathbf{W}^{\top}\mathbf{x}_i + \mathbf{b}, \qquad \mathbf{p}_i = \mathrm{Softmax}(\mathbf{y}_i).$$1

This is described as a geometric consistency constraint: if the boundary is well placed, small pooling perturbations should not flip predictions (Cheng et al., 26 Aug 2025).

When boundaries are heavily misaligned, FPS permits controlled movement from the source boundary through

$$\mathbf{y}_i = \mathbf{W}^{\top}\mathbf{x}_i + \mathbf{b}, \qquad \mathbf{p}_i = \mathrm{Softmax}(\mathbf{y}_i).$$2

with regularization

$$\mathbf{y}_i = \mathbf{W}^{\top}\mathbf{x}_i + \mathbf{b}, \qquad \mathbf{p}_i = \mathrm{Softmax}(\mathbf{y}_i).$$3

The full dynamically weighted loss is stated as

$$\mathbf{y}_i = \mathbf{W}^{\top}\mathbf{x}_i + \mathbf{b}, \qquad \mathbf{p}_i = \mathrm{Softmax}(\mathbf{y}_i).$$4

with

$$\mathbf{y}_i = \mathbf{W}^{\top}\mathbf{x}_i + \mathbf{b}, \qquad \mathbf{p}_i = \mathrm{Softmax}(\mathbf{y}_i).$$5

and schedules

$$\mathbf{y}_i = \mathbf{W}^{\top}\mathbf{x}_i + \mathbf{b}, \qquad \mathbf{p}_i = \mathrm{Softmax}(\mathbf{y}_i).$$6

$$\mathbf{y}_i = \mathbf{W}^{\top}\mathbf{x}_i + \mathbf{b}, \qquad \mathbf{p}_i = \mathrm{Softmax}(\mathbf{y}_i).$$7

$$\mathbf{y}_i = \mathbf{W}^{\top}\mathbf{x}_i + \mathbf{b}, \qquad \mathbf{p}_i = \mathrm{Softmax}(\mathbf{y}_i).$$8

This suggests a training regime in which early optimization emphasizes source supervision and boundary stability, while later optimization increases the influence of target entropy and consistency (Cheng et al., 26 Aug 2025).

4. Offline feature extraction and computational profile

A distinctive practical feature of FPS is offline feature extraction. The encoder is run once over source and target datasets, all feature vectors are stored, and optimization is then performed only on the classifier head. This design removes repeated backbone forward and backward passes from the adaptation loop and permits optimization directly over cached features (Cheng et al., 26 Aug 2025).

The reported benefits are fourfold. First, FPS permits single-cycle full-dataset optimization, because it can operate on the entire cached dataset distribution at once rather than relying on repeated mini-batch encoder updates. Second, it lowers memory use, since gradients and activations for the encoder need not be stored. Third, it lowers computation by eliminating expensive backbone updates. Fourth, it scales more readily to large backbones, including CLIP and InstructBLIP vision encoders, because adaptation touches only the final linear layer (Cheng et al., 26 Aug 2025).

These properties distinguish FPS from conventional fine-tuning and from adaptation strategies that alter prompts, adapters, or intermediate modules. In FPS, the feature geometry remains fixed and inspectable throughout adaptation. This fixed-geometry premise is also the basis of the framework’s interpretability claims: the adaptation process can be visualized as movement of a decision boundary rather than as opaque representational drift.

5. Benchmarks, application domains, and empirical results

FPS is evaluated on standard unsupervised domain adaptation benchmarks and on several domain-specific tasks. The standard benchmarks are Office-31, Office-Home, and VisDA-2017, with classification accuracy on each transfer task and average accuracy as the reported metrics. The additional application domains are protein secondary structure prediction, remote sensing classification, and earthquake detection (Cheng et al., 26 Aug 2025).

Setting	Dataset(s)	Metric
Standard UDA	Office-31, Office-Home, VisDA-2017	Classification accuracy and average accuracy
Protein secondary structure prediction	CB513, CASP12, TS115	Q3 accuracy
Remote sensing classification	EuroSAT-MS	Average recognition accuracy
Earthquake detection	Len-DB, OBST2024	Detection precision / accuracy

On the standard benchmarks, the reported average accuracies are 83.2% on Office-Home, 86.3% on VisDA-2017, and 86.3% on Office-31. The method is compared against CDAN, MCC, DALN, GSDE, CDTrans, TVT, SSRT, SHOT, CAN, and CDCL, and is described as competitive with or better than strong UDA baselines depending on the setting (Cheng et al., 26 Aug 2025).

The ablation on Office-Home Ar→Pr isolates the contributions of the individual components. The reported sequence is: SCE only, 84.4; + SE, 88.2; + CE, 87.3; + SE + CE, 89.5; + CR, 89.8; and + CR + $$\mathbf{y}_i = \mathbf{W}^{\top}\mathbf{x}_i + \mathbf{b}, \qquad \mathbf{p}_i = \mathrm{Softmax}(\mathbf{y}_i).$$9, 89.9. This ablation supports the interpretation that sample entropy provides the main unsupervised effect, category entropy mitigates collapse, and consistency regularization together with boundary-shift regularization refines the solution further (Cheng et al., 26 Aug 2025).

For unsupervised hyperparameter selection, FPS introduces the Intra-Class Distance Metric (ICDM). The procedure is to compute pseudo-class centroids from target predictions, measure the average squared distance of samples to their centroid, and normalize by the true-label version when available for analysis. The stated practical idea is that smaller intra-class distance indicates better cluster compactness and better boundary alignment, and the paper reports that the metric correlates with performance and can guide hyperparameter search without target labels (Cheng et al., 26 Aug 2025).

6. Relation to other methods, interpretability, and limitations

FPS differs from several established families of adaptation methods. Versus standard fine-tuning, it does not modify the feature extractor and therefore claims no feature drift, better interpretability, reduced risk of catastrophic distortion, and much less compute. Versus alignment-based UDA, it does not attempt to force source and target distributions to overlap in feature space through adversarial alignment, feature confusion, or discrepancy minimization; instead, it works in the original geometry and adjusts the classifier to that geometry. Versus pseudo-label or self-training methods such as SHOT, it does not use pseudo-label refinement as the core mechanism, relying instead on entropy, balance, consistency, and source supervision. Versus source-free UDA, it is conceptually related because it can operate with source feature vectors instead of raw source data, but it does not depend on iterative pseudo-labeling or auxiliary memory or prototype machinery. Versus CLIP-adapter or prompt tuning, it is simpler in the sense that the encoder is completely frozen and only the decision layer is optimized (Cheng et al., 26 Aug 2025).

Interpretability is presented as a major consequence of this design. Because the encoder is frozen, the feature geometry is fixed and inspectable, classifier change can be visualized as boundary movement, and the effect of each loss term is easier to understand. The paper’s 2D demonstration is used to illustrate adaptation as a geometric search problem in which source and target distributions remain fixed while the boundary is repositioned to satisfy source supervision and target geometric constraints (Cheng et al., 26 Aug 2025).

The method’s limitations are tied directly to its assumptions. FPS depends on a pretrained encoder whose features are already reasonably structured. If the frozen feature space does not show intra-class clustering and inter-class separation, boundary-only adaptation may be insufficient. Entropy minimization alone can collapse to extreme class assignments, which is why balancing terms are required. The approach may also underperform methods that substantially reshape representations when the source encoder is too weak or the domain gap is too large. The benchmark tables further indicate that FPS is not uniformly the top method on every individual transfer, even when it is competitive overall (Cheng et al., 26 Aug 2025).

There is also a terminological point. A much earlier paper, "Efficient Similarity Indexing and Searching in High Dimensions," develops a random binary partition forest that recursively divides feature space into non-overlapping convex hyper-polyhedral cells using hyperplanes, with query processing performed by following plane-defined partitions to leaf cells (Zhong, 2015). That work is closely related to Feature-Space Planes Searcher in spirit and mechanism if FPS is understood generically as a plane-based feature-space searcher, but it is not the same named method. The earlier approach concerns high-dimensional similarity search and indexing via random hyperplane partitions, whereas the 2025 FPS framework concerns domain adaptation via optimization of a decision boundary in frozen feature space. The shared conceptual element is the centrality of hyperplanes in structuring search or classification over high-dimensional features, but the algorithmic objectives are different (Zhong, 2015).

In aggregate, FPS formalizes a particular view of domain adaptation: when pretrained representations preserve class geometry across domains, the principal adaptation problem is often to relocate the classifier boundary rather than to relearn the representation. This suggests a generalizable paradigm for transfer learning under modern foundation-model conditions, especially where full fine-tuning is computationally costly or interpretability of the adaptation mechanism is a priority.