Perceptual Boundary Sampling in ANNs
- Perceptual boundary sampling is a family of methods that selects or generates borderline cases in a feature space, highlighting regions of high uncertainty.
- Approaches range from active fine-tuning using pretrained features to classifier-guided diffusion, applied in domains like digit recognition and facial expression analysis.
- These techniques improve model robustness and human-model alignment by effectively sampling ambiguous instances that central exemplars may miss.
Searching arXiv for papers on perceptual boundary sampling and closely related formulations. Perceptual boundary sampling, in the ANN-centered usage documented here, denotes the deliberate selection or synthesis of samples that lie near a model’s decision frontier in a representation space intended to encode perceptual or semantic structure. In contemporary work, that boundary may be defined in a frozen pretrained feature space for active finetuning, or in the output geometry of a classifier coupled to a diffusion prior for synthesizing ambiguous stimuli that elicit variable human judgments (Lu et al., 2024, Wei et al., 6 May 2025, Deng et al., 19 Jul 2025). Adjacent literatures use closely related boundary-sample constructions for hardware fingerprinting through imperceptible perturbations (Schlögl et al., 2021), while an earlier image-analysis line treats perceptually significant boundaries as salient level lines selected by a-contrario criteria (Tepper et al., 2012). Across these formulations, the unifying operation is to privilege ambiguous or borderline cases rather than only central exemplars, although the operational goal varies substantially.
1. Scope and principal formulations
In the literature represented here, perceptual boundary sampling is not a single algorithm but a family of boundary-oriented procedures. The central distinction is between selection of existing unlabeled samples near a learned boundary and generation of new samples that are driven toward such a boundary. A second distinction concerns what counts as “perceptual”: frozen pretrained features, classifier output probabilities, human response distributions, or salient image contours.
| Setting | Boundary representation | Primary objective |
|---|---|---|
| Bi-Level Active Finetuning | Frozen pretrained features | One-shot annotation selection combining diversity and uncertainty (Lu et al., 2024) |
| BAM / variMNIST | ANN decision boundaries with classifier-guided diffusion | Generate ambiguous digit stimuli, align models to group and individual behavior, and manipulate paired decisions (Wei et al., 6 May 2025) |
| varEmotion | ANN emotion-class boundaries with two-stage diffusion | Generate ambiguous facial-expression stimuli and align models to human perceptual variability (Deng et al., 19 Jul 2025) |
| Related boundary-based work | Microarchitecture-sensitive label flips; salient level lines | Watermarking and perceptual boundary saliency (Schlögl et al., 2021, Tepper et al., 2012) |
A plausible implication is that the term now spans both decision-boundary exploitation and boundary-centered stimulus design. The former uses a pretrained model’s geometry as a proxy for informative uncertainty; the latter treats ANN ambiguity as a generator of stimuli that are likely to induce perceptual divergence in humans.
2. Feature-space boundary sampling in bi-level active finetuning
The clearest selection-based formulation appears in the Bi-Level Active Finetuning framework, which defines the goal of boundary sample selection as identifying, within each pseudo-class, those unannotated examples that lie closest to the “decision frontier” separating that class from its neighbors in the pretrained model’s feature space (Lu et al., 2024). Ground-truth labels are unavailable at selection time, so the procedure operates entirely on the geometry of frozen features .
The framework is explicitly two-stage. First, it performs core sample selection for diversity. It chooses pseudo-class centers via a diversity-oriented method such as ActiveFT or K-Means; in practice it uses ActiveFT, which solves
under for , and then picks, for each , the nearest feature
These samples become the initial centers. Every unlabeled feature is then assigned to the nearest center by Euclidean distance, partitioning the pool into pseudo-classes.
The second stage performs boundary sampling for uncertainty. Because the boundary stage is highly sensitive to outliers, BiLAF first applies an unsupervised denoising method termed Iterative Density-based Clustering. For a point with feature 0, its density distance is defined as the average distance to its 1 nearest neighbors within the currently growing cluster: 2 Starting from 3, the cluster grows by repeatedly adding the 4 points with smallest 5. After the cluster reaches the full candidate set, the last 6 points added are pruned as noise.
Boundary proximity is then quantified by an intra-class distance
7
an inter-class distance to the nearest other center
8
and the Boundary Score
9
A small Boundary Score indicates a point close to another class’s prototype yet not overly deep inside its own cluster. To avoid repeatedly sampling the same inter-class seam, the method adds an opponent penalty: if 0 is the count of already-selected boundary points that were closest to center 1, future inter-class distances are inflated by 2, yielding
3
Selection is iterative. Each pseudo-class receives a sub-budget
4
and the algorithm repeatedly picks the point with minimum 5, adds it to the global annotation set, increments the opponent count for the center achieving the minimum, and removes the 6 nearest neighbors of the selected point to avoid redundancy. The complete procedure is given as Algorithm 1 in the paper (Lu et al., 2024).
The implementation is explicitly intended to scale. The reported feature dimension is 7 for DeiT-Small/DINO, the unlabeled pool reaches 8 up to 9 on ImageNet, and 0 is typically set to 1 of 2. With 3, 4, 5, and optimized vectorized kernels, ImageNet-scale pools are processed in a few hours on two RTX 3090 GPUs. Supervised finetuning uses batch size 6, SGD with learning rate 7, weight decay 8, momentum 9, and cosine decay for 0 epochs, except 1 epochs for 2 ImageNet (Lu et al., 2024).
The empirical motivation for the method is that diversity alone leaves boundary regions undersampled. Across CIFAR10, CIFAR100, and ImageNet with annotation budgets from 3 to 4, BiLAF is reported to outperform both purely diversity-based ActiveFT and traditional active learning methods. The paper reports, for example, CIFAR100 at 5: ActiveFT 6 versus BiLAF 7; at 8: 9 versus 0; at 1: 2 versus 3; and ImageNet at 4: 5 versus 6. Qualitative t-SNE plots are described as showing that core-only methods leave many boundary regions unsampled, whereas BiLAF “pins down support points along each inter-class seam” (Lu et al., 2024).
This framework explicitly connects itself to perceptual boundary sampling by “lifting” classical uncertainty sampling into the pretrained model’s “perceptual” feature space, where distances encode semantic relations. Under tight annotation budgets, the stated rationale is that a classifier trained only on central exemplars tends to misplace its boundaries; boundary-targeted annotation forces adjustment precisely where errors are likely to occur (Lu et al., 2024).
3. Diffusion-guided perceptual boundaries in digit recognition
A generative and human-centered formulation is introduced by BAM, the Boundary Alignment & Manipulation framework, which combines perceptual boundary sampling in ANNs with human behavioral experiments to study perceptual variability (Wei et al., 6 May 2025). The setup begins with a pre-trained ANN classifier, or two classifiers for controversial sampling, together with a diffusion model 7 serving as a natural-image prior.
At each reverse diffusion step, the sample is updated by
8
where 9 is the unconditioned denoising step, 0 is a guidance strength, and 1 encourages boundary sampling. Two losses are used. Uncertainty guidance is designed to place a sample near a single model’s decision boundary: 2 Controversial guidance is designed to maximize disagreement between two models: 3 with 4 placing mass on class 5 and 6 on class 7. Optionally, a small MSE regularizer toward a reference MNIST digit 8 can be added: 9 The framework states that it relies on a classifier-guided diffusion prior, specifically classifier-free guidance, rather than direct pixel-space adversarial perturbations, in order to keep samples in-distribution and avoid heavy noise (Wei et al., 6 May 2025).
BAM consists of three stages. The first stage is labeling: approximately 0 images are sampled by uncertainty or controversial guidance across all digit pairs 1, then shown to many participants in a 10-way digit classification task. For each image, the recorded quantities are the human choice distribution 2, response time, and entropy 3. After sentinel-trial filtering, 4 participants remain out of 5, yielding 6 valid trials and 7 images in the variMNIST dataset; the same description also states that each participant completed 8 trials, for a total of 9k trials (Wei et al., 6 May 2025).
The second stage is alignment. Starting from a base classifier trained on MNIST, BAM fine-tunes a GroupNet on MNIST 0 variMNIST in a 1 ratio and then an IndivNet per participant using a mix ratio 2. The training objective is ordinary cross-entropy on human-labeled 10-way target distributions. The reported effect is that GroupNet and IndivNet retain approximately 3 accuracy on MNIST, gain approximately 4 on variMNIST, and that IndivNet adds approximately 5 on the individual subset variMNIST-i. The quantitative summary given later in the paper states BaseNet accuracy of approximately 6 on variMNIST, GroupNet approximately 7, and IndivNet approximately 8 on variMNIST-i; for a VGG example, the Spearman correlation between model-predicted entropy and human entropy increases from 9 to 0 after group fine-tuning (Wei et al., 6 May 2025).
The third stage is manipulation. For a pair of participants 1, BAM takes their two IndivNet models 2 and generates new controversial images using
3
with 4 pushing toward 5 and 6 toward 7. Approximately 8 such stimuli are presented to each pair in lab conditions, and the outcome of interest is whether 9 reports 00 while 01 reports 02. In paired manipulation experiments described as involving 03 in-lab participants forming 04 pairs, individually generated images improve the “success” rate by approximately 05 and the directional “targeted ratio” by approximately 06 over vanilla variMNIST stimuli, with 07 for both (Wei et al., 6 May 2025).
The behavioral evaluation formalizes three categories for guided class pairs 08. A sample is a success if 09 and 10; bias if 11 but 12; and failure otherwise. Across all variMNIST images, success plus bias is approximately 13, and over half of the images have human-response entropy greater than zero (Wei et al., 6 May 2025).
The BAM account explicitly argues that decision-boundary regions of ANNs correspond to maximal classifier uncertainty and that prior psychophysics places maximal human perceptual variability near categorical boundaries. On that basis, images synthesized on ANN boundaries are treated as intrinsically likely to induce high human ambiguity (Wei et al., 6 May 2025).
4. Perceptual-boundary sampling for facial-expression variability
A parallel formulation is developed for facial expressions, where the perceptual boundary is defined for a 14-way ANN classifier 15 by the locus of inputs 16 for which two target emotions 17 are balanced, for example when 18, equivalently 19 (Deng et al., 19 Jul 2025). Perceptual-boundary sampling is then the generation of new face images near such boundaries, so that the classifier assigns both classes high and roughly equal probability.
The formal construction introduces a target distribution 20 over emotions with
21
and an uncertainty guidance loss
22
which is described as maximized when 23 and the other probabilities approach zero. The paper also notes an alternative expression, 24, interpreted as encouraging high network mass on the two target classes and low mass elsewhere (Deng et al., 19 Jul 2025).
Because direct gradient ascent on 25 is reported to produce noisy and non-photorealistic results, the method embeds the loss in a two-stage diffusion pipeline. In Stage 1, an embedding 26 is iteratively denoised with uncertainty guidance: 27 with the paper reporting 28. In Stage 2, the final guided embedding 29 is given to Stable Diffusion XL with prompt guidance only, producing a fully rendered RGB face image (Deng et al., 19 Jul 2025).
Post-generation filtering is central to the method. Each candidate image is fed back through the ANN, and an image is kept only if both target-class activations exceed class-specific thresholds 30, where each threshold is set to the 31th percentile of that class’s activation on the RAF-DB real-image distribution: 32 The stated purpose is to retain only samples genuinely in the high-uncertainty overlap region of the two emotions (Deng et al., 19 Jul 2025).
This procedure is used to construct the varEmotion dataset. For each of the 33 unordered emotion pairs from six basic classes, the pipeline produces candidate faces; after filtering, 34 images remain. These are shown in randomized order to 35 online participants, of whom 36 pass sentinel trials, in a 6-way forced-choice emotion labeling task. Each image is seen by approximately 37 participants, for a total of 38 trials. The recorded response for image 39 is the empirical human choice distribution
40
with 41 (Deng et al., 19 Jul 2025).
The evaluation uses both ANN-side and human-side criteria. Human perceptual uncertainty is quantified by entropy
42
For guided classes with empirical human fractions 43, a sample is categorized as Success if 44 and 45, Bias if 46 and 47, and Failure if 48. Roughly 49 of the images fall into success plus bias, which the paper interprets as effective two-class guidance (Deng et al., 19 Jul 2025).
The principal empirical claim is that ANN-confusing facial-expression stimuli also provoke heightened human perceptual uncertainty. The entropy distribution over the 50 images is described as concentrated above zero, indicating that most boundary images induce split judgments. On the modeling side, three architectures—CLIP + MLP, DAN, and ResEmoNet—are fine-tuned with varEmotion mixed with RAF-DB at a 51 ratio under group and individual regimes. Group fine-tuning improves accuracy on varEmotion by 52, 53, and 54, respectively, relative to BaseNet; individual-level fine-tuning on 55 yields further gains, including 56 for DAN. For DAN, the Spearman correlation between model entropy 57 and human entropy 58 rises from 59 before fine-tuning to 60 after group fine-tuning (Deng et al., 19 Jul 2025).
The optimization used for alignment is ordinary cross-entropy,
61
minimized on the group mixture 62 at a 63 sample ratio and then on each individual dataset 64 at a 65 ratio. The reported optimizer is Adam with learning rate 66, batch size 67, CrossEntropyLoss, and 68 epochs on an NVIDIA GPU (Deng et al., 19 Jul 2025).
5. Related boundary-based formulations
A distinct but technically related use of boundary samples appears in model watermarking. In iNNformant, a boundary sample for two inference environments 69 and 70 around a natural host sample 71 is an input 72 such that 73 is small and the final labels differ, 74 (Schlögl et al., 2021). The paper defines 75 as the softmax-vector output and
76
as an approximate distance to the decision boundary on 77. A set of such samples is fully identifying for environments 78 if each 79 has at least one sample whose label differs from all others. The objective is therefore not annotation efficiency or human ambiguity, but identification of the execution environment through label instability near the boundary.
The optimization view is explicitly constraint-based: minimize perturbation magnitude subject to label disagreement across environments. Distortion is measured by PSNR,
80
The generation algorithm uses a local phase to approach a decision boundary of a reference microarchitecture and a remote phase to refine the input until exactly one environment is isolated, using a modified FGSM-style update
81
On FMNIST with ResNet20 and CIFAR10 with ResNet32, success rates are reported as 82 and 83, respectively. By selecting the best PSNR sample per microarchitecture, the paper reports fully identifying sets whose worst PSNR exceeds 84 on FMNIST or 85 on CIFAR (Schlögl et al., 2021). This suggests a boundary-sample regime in which perceptual fidelity is preserved, but the operative ambiguity is between hardware-specific executions rather than between semantic classes or between human observers.
An earlier and conceptually separate formulation is found in the literature on perceptual boundary saliency. There, the target is not a classifier’s decision frontier but the boundary of image level sets extracted by the Fast Level-Set Transform. A level line 86 of length 87 is declared 88-meaningful under the Helmholtz principle if its Number of False Alarms satisfies
89
where 90 is an empirical tail histogram of gradient magnitudes and 91 is the number of level lines (Tepper et al., 2012). The method extends earlier a-contrario formulations by allowing partially salient level lines rather than requiring an entire line to be salient.
The paper also introduces a regularity saliency at scale 92,
93
and a combined saliency criterion denoted 94, which relaxes punctual minima through percentile-based partial saliency. The detection algorithm computes the FLST, evaluates contrast and regularity statistics along each level line, marks curves as meaningful when their NFA is below 95, and then performs maximality pruning along FLST branches. The total complexity is reported as approximately 96, with standard parameters 97, 98, and 99 px (Tepper et al., 2012).
Although this saliency-based work is not a decision-boundary method in the ANN sense, it is relevant for the broader semantics of perceptual boundaries. It treats perceptual significance as improbability under a null model and explicitly allows boundary point sampling on level lines for subsequent tasks such as shape matching, registration, stereo edge linking, and contour extraction (Tepper et al., 2012).
6. Recurrent methodological themes, misconceptions, and open directions
A recurrent theme across these works is that boundary proximity alone is not considered sufficient. Each framework couples boundary seeking with an additional stabilizing mechanism. In BiLAF, uncertainty-oriented boundary selection is preceded by core selection for diversity and by IDC denoising to remove outliers before scoring (Lu et al., 2024). In BAM and the facial-expression pipeline, the model boundary is accessed through a diffusion prior rather than direct pixel-space perturbation, precisely to keep the generated samples in-distribution and recognizable (Wei et al., 6 May 2025, Deng et al., 19 Jul 2025). In iNNformant, imperceptibility is enforced through a PSNR criterion, and in the saliency literature boundary detection is regularized by a-contrario thresholds, partial-saliency criteria, and maximality pruning (Schlögl et al., 2021, Tepper et al., 2012).
A common misconception is to equate boundary sampling with diversity sampling or with naïve uncertainty sampling. The surveyed literature separates these notions. BiLAF is explicitly bi-level: core sample selection covers the space, while boundary sample selection refines class frontiers (Lu et al., 2024). BAM separates uncertainty guidance from controversial guidance, the latter being designed not merely to maximize entropy for one model but to induce disagreement between paired models and, later, between paired participant-specific models (Wei et al., 6 May 2025). The facial-expression work similarly does not stop at balanced probabilities; it filters generated images by class-specific activation thresholds so that retained samples lie in a verified overlap region (Deng et al., 19 Jul 2025).
Another misconception is that “perceptual” always means human-defined. In several of these formulations, the perceptual substrate is initially ANN-internal: pretrained feature geometry in BiLAF, classifier posteriors in BAM, or emotion-class probabilities in varEmotion. Human perceptual variability enters only after behavioral validation and model alignment in the human-facing studies (Lu et al., 2024, Wei et al., 6 May 2025, Deng et al., 19 Jul 2025). This suggests that current usage often begins with model-defined perceptual structure and then asks whether that structure predicts human ambiguity.
The open directions stated in these works are similarly heterogeneous. BAM proposes expansion beyond digit recognition to natural scenes, semantic tasks, and similarity judgments; incorporation of optimal experimental design loops, including active learning, to minimize human trials; and the study of cultural and developmental factors in more diverse participant populations (Wei et al., 6 May 2025). The facial-expression study positions its findings as a route toward personalized modeling of emotional interpretation (Deng et al., 19 Jul 2025). The saliency literature identifies corners and junctions, over-detection in textured areas, and multiscale fusion as unresolved issues (Tepper et al., 2012). The watermarking literature notes that scaling to higher-resolution inputs or deeper networks may require smaller step sizes or adaptive line search (Schlögl et al., 2021).
Taken together, these formulations establish perceptual boundary sampling as a boundary-centered design principle rather than a single canonical technique. Its modern forms range from sample-efficient active finetuning, through controlled synthesis of ambiguous stimuli for behavioral science, to imperceptible near-boundary perturbations and salient contour detection. The common structure is the deliberate exploitation of borderline cases, but the meaning of the boundary, the type of perceptual evidence, and the target downstream use remain domain-specific.