Classifier-based Feature Reconstruction (ClaFR)

Updated 10 July 2026

ClaFR is a design paradigm that computes class evidence by reconstructing query feature maps from class-specific support features or prototype distributions.
It employs analytic methods such as closed-form ridge regression, SVD-based subspace projection, and Wasserstein barycenters to derive decision scores.
Its versatile applications include few-shot classification, vision-language reasoning, and post-hoc OOD detection, all with minimal learnable parameters and efficient computation.

Classifier-based Feature Reconstruction (ClaFR) denotes a classifier design in which class evidence is computed from reconstruction quality in representation space rather than only from logits, cosine similarity, or input-space likelihood. In the most explicit usage, ClaFR is a post-hoc OOD detector that performs an orthogonal decomposition of the classifier’s weights to extract a class-known subspace, projects penultimate-layer features into that subspace, and uses the feature reconstruction error as the OOD score (Wang et al., 2 Sep 2025). Closely related formulations reformulate few-shot classification as reconstruction of a query feature map from class support features (Wertheimer et al., 2020), and adapt frozen CLIP by projecting class prototype features into the query image feature space to reconstruct the query image feature map (Zhang et al., 28 Jan 2026). A further extension reconstructs surrogate class prototypes from queried samples and counterfactuals using Wasserstein barycenters, yielding a prototype-based reconstruction of a target classifier’s decision surface (Zhao et al., 11 Dec 2025).

1. Reconstruction as a classifier principle

The central ClaFR principle is the replacement of a “similarity-as-score” paradigm with a “reconstruction-as-score” paradigm. In FRN, the decision for each class is: “how well can I reconstruct this query’s feature map from that class’s support feature maps?” In FPL, each class is represented by a pool of spatial visual features, and classification is done by how well these prototypes can reconstruct the query’s feature map. In the OOD formulation named ClaFR, the classifier’s weight matrix induces a low-dimensional class-known subspace, and feature reconstruction error within that subspace becomes the confidence signal (Wertheimer et al., 2020, Zhang et al., 28 Jan 2026, Wang et al., 2 Sep 2025).

This suggests a broad technical usage in which ClaFR covers methods that classify, detect distribution shift, or reconstruct a target model by measuring how well class-specific structures reconstruct a query representation. The structures differ across works—support feature pools, CLIP class prototype features, singular-vector subspaces, or Wasserstein prototype distributions—but the operational rule remains reconstruction-based (Zhao et al., 11 Dec 2025).

Work	Reconstruction object	Decision signal
FRN	Query feature map $Q$ from class support pool $S_c$	Negative mean squared reconstruction error
FPL	Query CLIP feature map $M$ from class prototype features $F_d$	Negative average squared reconstruction error
ClaFR	Penultimate-layer feature $z$ from class-known subspace $U_M$	Projection norm or reconstruction error
Counterfactual-aware prototypes	Input $x$ against barycentric prototypes $Q_0,Q_1$	Nearest-prototype Wasserstein rule

A common misconception is to equate reconstruction-based classifiers with autoencoder-style anomaly detectors. The OOD ClaFR paper explicitly distinguishes the two: traditional autoencoder-based OOD detection reconstructs in input space with an extra encoder–decoder network, whereas ClaFR reconstructs in feature space and the “decoder” is simply the orthogonal projector $U_MU_M^T$ derived from classifier weights (Wang et al., 2 Sep 2025).

2. Closed-form feature map reconstruction in few-shot classification

“Few-Shot Classification with Feature Map Reconstruction Networks” formulates few-shot classification as a reconstruction problem in latent space. An episode has a set of classes $C$ with $S_c$ 0, a support set of $S_c$ 1 labeled images per class, and query images drawn from the same classes. A convolutional backbone maps each image to a feature map $S_c$ 2, flattened as $S_c$ 3 with $S_c$ 4. For each class $S_c$ 5, all support feature vectors are pooled into a matrix $S_c$ 6, and the class score is the negative mean squared error between the query map and its class-conditional reconstruction (Wertheimer et al., 2020).

The reconstruction mechanism is a closed-form ridge regression. Given $S_c$ 7 and $S_c$ 8, the method seeks $S_c$ 9 such that $M$ 0, solving

$M$ 1

The standard closed form is

$M$ 2

When $M$ 3, the Woodbury identity yields an equivalent $M$ 4 inversion: $M$ 5 The class posterior is then

$M$ 6

The method stabilizes the regression by scaling the regularizer with $M$ 7,

$M$ 8

and introduces a learned recalibration

$M$ 9

so that the only learned parameters in the reconstruction head are the three scalars $F_d$ 0, $F_d$ 1, and $F_d$ 2. There is no class-specific learnable weight and no additional neural reconstruction module. The paper further uses episodic meta-training, an auxiliary loss

$F_d$ 3

scaled by $F_d$ 4, and a pre-training stage with learnable “dummy” feature maps $F_d$ 5 in order to produce feature maps that are suitable for reconstructive classification (Wertheimer et al., 2020).

Empirically, FRN is both accurate and efficient. On fine-grained 5-way benchmarks with ResNet-12, it reports, for example, CUB 1-shot $F_d$ 6 and 5-shot $F_d$ 7, and Aircraft 1-shot $F_d$ 8 and 5-shot $F_d$ 9. On mini-ImageNet, ResNet-12, 5-way, it reports 1-shot $z$ 0 and 5-shot $z$ 1; on tiered-ImageNet, 1-shot $z$ 2 and 5-shot $z$ 3, with a larger-resolution variant at $z$ 4. The paper also emphasizes substantial speed advantages over DeepEMD, especially in 5-shot settings (Wertheimer et al., 2020).

3. Feature projection learning on frozen CLIP

“Feature Projection Learning for Better Vision-Language Reasoning” instantiates the same classifier-based feature reconstruction logic on top of a frozen CLIP backbone. The baseline CLIP classifier predicts a class for a query image $z$ 5 by cosine similarity between the image embedding $z$ 6 and text embeddings $z$ 7. FPL replaces this with a reconstruction-based classifier: it uses CLIP’s visual encoder without the final attention pooling, denoted $z$ 8, extracts a spatial query feature map $z$ 9, pools all spatial features from the $U_M$ 0 support images of class $U_M$ 1 into $U_M$ 2, and asks how well $U_M$ 3 can reconstruct $U_M$ 4 (Zhang et al., 28 Jan 2026).

The projection model is analytic rather than neural. It seeks a matrix $U_M$ 5 such that $U_M$ 6 approximates $U_M$ 7, solving

$U_M$ 8

with closed-form solution

$U_M$ 9

The class score is the normalized squared reconstruction error

$x$ 0

and the reconstruction-based probability is

$x$ 1

Here $x$ 2 with $x$ 3 a learnable scalar, and $x$ 4 is a learnable temperature inspired by Wertheimer et al.’s feature map reconstruction networks. The paper explicitly states that “only $x$ 5 and $x$ 6 are needed to be updated by gradient descent” (Zhang et al., 28 Jan 2026).

FPL does not discard CLIP’s original classifier. Instead, it adds the reconstruction-based prediction to the frozen text-image similarity prediction: $x$ 7 The total loss is

$x$ 8

where $x$ 9 is a projection orthogonality loss encouraging reconstructed feature maps for different classes to be dissimilar when applied to the same query, and $Q_0,Q_1$ 0 in all experiments. The CLIP encoders $Q_0,Q_1$ 1, $Q_0,Q_1$ 2, and $Q_0,Q_1$ 3 remain frozen; the manual prompt is “a photo of {class}”; training uses AdamW with initial learning rate $Q_0,Q_1$ 4, cosine annealing, and a single NVIDIA RTX 3090 (Zhang et al., 28 Jan 2026).

The paper presents FPL as extremely parameter-efficient and training-efficient. On 16-shot ImageNet, Table 4 reports FPL: 20 epochs, 1 minute training, 0.001 GFLOPs, $Q_0,Q_1$ 5M parameters, 66.68% accuracy; Tip-Adapter-F: 5 minutes, 0.030 GFLOPs, 16.3M parameters, 65.51% accuracy; and CoOp: 15h, $Q_0,Q_1$ 6 GFLOPs, 0.01M parameters, 62.95% accuracy. On domain generalization from 16-shot ImageNet to ImageNet-V2, -Sketch, -A, and -R, FPL reports 46.46 average accuracy, compared with 43.89 for TPT and 41.59 for zero-shot CLIP. The few-shot study further reports relative improvements over Tip-Adapter-F of +5.1% on UCF101, +2.2% on EuroSAT, +4.5% on FGVC-Aircraft, and +4.2% on DTD (Zhang et al., 28 Jan 2026).

4. Post-hoc ClaFR for out-of-distribution detection

The work titled “Frustratingly Easy Feature Reconstruction for Out-of-Distribution Detection” uses the name Classifier-based Feature Reconstruction (ClaFR) directly. Its setting is a pre-trained classifier with feature extractor $Q_0,Q_1$ 7, penultimate-layer feature $Q_0,Q_1$ 8, and final linear classifier weights $Q_0,Q_1$ 9. ClaFR performs SVD,

$U_MU_M^T$ 0

selects the top- $U_MU_M^T$ 1 left singular vectors

$U_MU_M^T$ 2

and interprets them as a class-known subspace. The hyperparameter $U_MU_M^T$ 3 is chosen by a cumulative explained variance ratio $U_MU_M^T$ 4, typically $U_MU_M^T$ 5, as the smallest $U_MU_M^T$ 6 such that

$U_MU_M^T$ 7

At test time, the feature is optionally normalized, projected as $U_MU_M^T$ 8, reconstructed as $U_MU_M^T$ 9, and scored by

$C$ 0

or equivalently

$C$ 1

Large $C$ 2 indicates ID, and small $C$ 3 indicates OOD (Wang et al., 2 Sep 2025).

The method is notable for not requiring access to training data. The paper states that it only needs a pre-trained classifier, its weights, and penultimate-layer features, and “never touches the original training data.” This is positioned as particularly attractive in scenarios where training data cannot be stored or re-used due to privacy or legal constraints. The method is also one-time and lightweight: SVD of $C$ 4 is performed once, and inference requires only a matrix-vector multiplication and an L2 norm (Wang et al., 2 Sep 2025).

The main experimental setup includes ImageNet-1K as ID with OOD benchmarks SUN, iNaturalist, Places, Textures, and ImageNet-O; ResNet-50 trained from scratch with contrastive loss on ImageNet-1K with 78.12% top-1 accuracy and feature dimension $C$ 5; MobileNet-V2 with $C$ 6; and CIFAR experiments with a ResNet-34 trained under supervised contrastive learning for 1000 epochs, reaching 95.28% on CIFAR-10 and 73.23% on CIFAR-100 with $C$ 7. Evaluation uses AUROC (AUC) and FPR@95%TPR (Wang et al., 2 Sep 2025).

On ImageNet-1K, ResNet-50, the average across the five OOD sets is 89.32 AUC, 41.82 FPR for ClaFR, compared with 86.99 AUC, 50.13 FPR for Neco and 86.46 AUC, 48.76 FPR for ASH. On ImageNet-1K, MobileNet, ClaFR reports 83.63 AUC, 54.21 FPR. On the CIFAR benchmark, for ID = CIFAR-10, ClaFR reports 82.82 AUC, 68.41 FPR; for ID = CIFAR-100, 96.29 AUC, 23.49 FPR. In the computational comparison to KNN at ImageNet scale, ClaFR reports $C$ 8 per-sample complexity, $C$ 9 ms inference, and $S_c$ 00 MiB storage, compared with $S_c$ 01 ms and $S_c$ 02 GiB for KNN (Wang et al., 2 Sep 2025).

Geometrically, the method treats the classifier weight spectrum as a task-specific low-dimensional manifold in feature space. This suggests a ClaFR interpretation in which the classifier’s own geometry supplies the reconstruction basis, without any episode-specific support set or learned decoder (Wang et al., 2 Sep 2025).

5. Counterfactual-aware prototype reconstruction

“Classifier Reconstruction Through Counterfactual-Aware Wasserstein Prototypes” does not use the name ClaFR directly, but it “strongly overlaps conceptually with classifier-based feature reconstruction.” Its setting is black-box reconstruction of a binary probabilistic target classifier

$S_c$ 03

under a low-query regime of 300–500 queries, using queried points, labels returned by $S_c$ 04, and counterfactuals generated by a mechanism $S_c$ 05. The counterfactual generator is one-sided: it is activated only when $S_c$ 06 and produces a counterfactual that flips the decision to class 1. The paper’s central performance measure is fidelity,

$S_c$ 07

Counterfactuals are treated as “soft samples for both classes” by assigning label $S_c$ 08 in the extended label space $S_c$ 09 (Zhao et al., 11 Dec 2025).

For each class $S_c$ 10, the paper defines a soft prototype distribution $S_c$ 11 as a Wasserstein barycenter between the class distribution $S_c$ 12 and the counterfactual distribution $S_c$ 13: $S_c$ 14 with $S_c$ 15 in the experiments. To mitigate decision boundary shift from one-sided counterfactuals, it adds a symmetry regularization term

$S_c$ 16

and optimizes

$S_c$ 17

with $S_c$ 18 in experiments. Classification is then a nearest-prototype rule in Wasserstein space using distances from the Dirac measure $S_c$ 19 to $S_c$ 20 and $S_c$ 21, with margin $S_c$ 22 (Zhao et al., 11 Dec 2025).

The paper works on Adult Income, HELOC, COMPAS, and DCCC, with logistic regression target models trained on unknown training data. The proposed method is compared to a model-extraction baseline that treats counterfactuals as normal labeled instances, and to a neural surrogate with Counterfactual Clamping loss. On the Adult dataset, the fidelity results reported in Table 1 are: at 500 queries, Baseline 1: $S_c$ 23, Baseline 2: $S_c$ 24, Ours: $S_c$ 25; at 300 queries, Baseline 1: $S_c$ 26, Baseline 2: $S_c$ 27, Ours: $S_c$ 28. The paper states that similar patterns hold on COMPAS, DCCC, and HELOC, with gains most pronounced at 300 queries (Zhao et al., 11 Dec 2025).

This line of work broadens the ClaFR perspective from reconstructing query features to reconstructing class-conditional prototype distributions and decision boundaries. The paper explicitly argues that high-quality, realistic, on-manifold counterfactuals lead to higher fidelity, and that low-capacity prototype-based surrogates remain more stable than more complex neural surrogates in low-data regimes (Zhao et al., 11 Dec 2025).

6. Comparative properties, misconceptions, and limitations

Across these works, ClaFR is not a single algorithm but a recurring design pattern. FRN and FPL classify by reconstructing a query feature map from class-specific support features or class prototype features; the OOD ClaFR method scores a feature by its reconstruction within a class-known subspace induced by classifier weights; and the counterfactual-aware prototype method reconstructs class representations as Wasserstein barycenters and uses those prototypes to approximate a target classifier’s boundary (Wertheimer et al., 2020, Zhang et al., 28 Jan 2026, Wang et al., 2 Sep 2025, Zhao et al., 11 Dec 2025).

Several common properties follow directly from the cited formulations. First, reconstruction is usually analytic rather than amortized: FRN uses closed-form ridge regression; FPL computes $S_c$ 29 analytically per query; OOD ClaFR uses the orthogonal projector $S_c$ 30; and the barycentric variant uses optimal transport prototype optimization rather than a learned decoder. Second, the learnable overhead can be very small: FRN uses only $S_c$ 31 in the reconstruction head; FPL updates only $S_c$ 32 and $S_c$ 33; and OOD ClaFR performs no retraining at all (Wertheimer et al., 2020, Zhang et al., 28 Jan 2026, Wang et al., 2 Sep 2025). Third, the data requirements differ sharply: OOD ClaFR requires no training data, FRN and FPL require support features, and the counterfactual-aware method requires queried samples plus counterfactuals (Wang et al., 2 Sep 2025, Wertheimer et al., 2020, Zhang et al., 28 Jan 2026, Zhao et al., 11 Dec 2025).

The main misconceptions are likewise clarified by the papers. ClaFR is not necessarily an input-space reconstruction model; in the OOD setting it is explicitly a feature-space subspace projection method, and in FRN/FPL it reconstructs spatial feature maps rather than pixels (Wang et al., 2 Sep 2025, Wertheimer et al., 2020, Zhang et al., 28 Jan 2026). ClaFR is also not tied to a single application domain: the literature spans few-shot classification, vision-language adaptation, post-hoc OOD detection, and black-box model reconstruction (Zhang et al., 28 Jan 2026, Wang et al., 2 Sep 2025, Zhao et al., 11 Dec 2025).

The limitations are equally consistent. FRN identifies linearity, dependence on feature quality, and the need for careful regularization when $S_c$ 34, since support features may span or nearly span $S_c$ 35 (Wertheimer et al., 2020). OOD ClaFR identifies dependence on classifier quality, assumption of linear subspace structure, sensitivity to architecture, and the absence of explicit OOD examples (Wang et al., 2 Sep 2025). The counterfactual-aware prototype method identifies sensitivity to counterfactual quality and the computational cost of optimal transport (Zhao et al., 11 Dec 2025). FPL’s ablations show that learning the ridge penalty is crucial: on 16-shot ImageNet, FPL full: 66.68%, w/o PO: 66.51%, fixed $S_c$ 36: 62.65%, w/o PO + fixed $S_c$ 37: 62.56% (Zhang et al., 28 Jan 2026).

The open directions listed in the papers point toward a more general ClaFR research program. FRN proposes nonlinear reconstruction, multi-layer reconstruction, learned class-specific bases, hybrid metric + reconstruction, and task-dependent regularization (Wertheimer et al., 2020). The OOD ClaFR paper proposes extensions to Transformers, large foundation models, non-linear subspaces, combination with other scores, and better theoretical understanding (Wang et al., 2 Sep 2025). FPL suggests that frozen VLM backbones can support extremely parameter-efficient reconstruction-based classifiers, especially when combined with a text-based prior and orthogonality regularization (Zhang et al., 28 Jan 2026). The counterfactual-aware prototype work suggests that counterfactuals are best used as boundary information rather than as hard-labeled training points, and that prototype distributions can reconstruct both class structure and decision boundaries under strict query budgets (Zhao et al., 11 Dec 2025).

Taken together, these results suggest a unifying view: classifier-based feature reconstruction treats class discrimination as a question of whether a query representation lies in, near, or can be reconstructed from a class-conditioned subspace, dictionary, or prototype distribution. The precise reconstruction object varies, but the score is consistently induced by reconstruction quality, projection geometry, or prototype proximity in a representation space shaped by the task.