Heatmap-Guided Proposal Selector (HGPS)

• HGPS is a heatmap-conditioned mechanism that transforms dense heatmap evidence into a sparse set of candidate proposals for detection tasks.
• It employs dual-thresholding to balance instance separation and complete object coverage, enhancing pseudo-ground-truth construction in weak supervision.
• Variants of HGPS enable conversion of heatmaps into spatial anchors and Transformer queries, boosting performance in astrocyte and grasp detection.

Heatmap-Guided Proposal Selector (HGPS) is a heatmap-conditioned candidate selection mechanism whose most explicit formulation appears in weakly supervised object detection, where it uses dual thresholds on class-activation heatmaps to pre-select proposals, construct pseudo-ground-truth clusters, and support iterative refinement of detector supervision (Guo et al., 10 Sep 2025). The same name is also used in a hybrid CNN–Transformer detector for astrocyte detection, where it converts a dense center heatmap into spatially grounded anchors and query embeddings for a Transformer decoder (Zhang et al., 3 Sep 2025). In both usages, the central operation is the conversion of dense heatmap evidence into a sparse set of candidate regions, but the downstream semantics differ: cluster-based pseudo-label generation in weak supervision, and query initialization in fully supervised detection.

1. Role in weakly supervised object detection

In the weakly supervised object detection formulation, HGPS is introduced within a framework for WSOD that also includes a weakly supervised basic detection network (WSBDN) and multiple instance refinement (IR) modules. The motivating diagnosis is threefold: existing methods may generate pseudo GT boxes that focus only on discriminative parts and fail to capture the whole object, or cover the entire object but fail to distinguish between adjacent intra-class instances; the foundational WSDDN architecture lacks a background class representation for each proposal and exhibits a large semantic gap between its branches; and prior methods discard ignored proposals during optimization, leading to slow convergence (Guo et al., 10 Sep 2025).

HGPS addresses the first of these limitations. Its stated purpose is to utilize dual thresholds on heatmaps to pre-select proposals, enabling pseudo GT boxes to both capture the full object extent and distinguish between adjacent intra-class instances. This makes HGPS a supervision-construction module rather than a standalone detector. A plausible implication is that its primary contribution is not proposal generation per se, but the structuring of weak supervisory signals from heatmap geometry and proposal overlap.

2. Dual-threshold heatmaps and proposal clustering

The WSOD version of HGPS begins with an input image $I\in\mathbb{R}^{H\times W\times 3}$ and a CNN backbone feature map $F\in\mathbb{R}^{H'\times W'\times D}$. A class-specific $1\times 1$ convolution yields class-activation maps

$$A_c=\mathrm{Conv}_{1\times 1}^c(F)\in\mathbb{R}^{H'\times W'},\qquad c=1\ldots C.$$

Global average pooling over spatial dimensions produces image-level logits

$$\varphi_c^{CAM}=\frac{1}{H'W'}\sum_{h'=1}^{H'}\sum_{w'=1}^{W'}A_c(h',w'),$$

and the CAM branch is trained with multi-label BCE. Each $A_c$ is then interpolated to full image resolution and min–max normalized:

$$\tilde A_c(x,y)=\frac{A_c^{\uparrow}(x,y)-\min A_c^{\uparrow}}{\max A_c^{\uparrow}-\min A_c^{\uparrow}}\in[0,1].$$

Dual thresholding is applied to each normalized heatmap. A high threshold $\tau^{high}$ selects very confident, central pixels and tends to separate adjacent instances but misses object periphery. A low threshold $\tau^{low}$ selects a looser region covering the full object but may merge nearby instances. In the reported VOC07 configuration, $\tau^{high}=0.8$ and $F\in\mathbb{R}^{H'\times W'\times D}$0. The corresponding masks are

$F\in\mathbb{R}^{H'\times W'\times D}$1

Connected components are computed on each mask, and the tightest bounding box is extracted for each component, giving $F\in\mathbb{R}^{H'\times W'\times D}$2 and $F\in\mathbb{R}^{H'\times W'\times D}$3.

HGPS then clusters proposals relative to the high- and low-threshold boxes. Let $F\in\mathbb{R}^{H'\times W'\times D}$4 be the proposal set. A high-threshold box is subordinate to a low-threshold box if the high-mask component that generated it lies entirely within the low-mask component for the low-threshold box. Each low-threshold box is rescaled by factor $F\in\mathbb{R}^{H'\times W'\times D}$5 to $F\in\mathbb{R}^{H'\times W'\times D}$6, and each high-threshold box is likewise scaled to $F\in\mathbb{R}^{H'\times W'\times D}$7. For a low-threshold box with no subordinate high-threshold box, HGPS forms a singleton cluster containing only the low-threshold box. With one subordinate high-threshold box, the cluster contains the low-threshold box together with proposals that enclose the high-threshold box and are enclosed by the scaled low-threshold box. With multiple subordinate high-threshold boxes, a separate cluster is built for each high-threshold box, and proposals qualifying for more than one cluster are assigned by maximum IoU overlap with the corresponding high-threshold box (Guo et al., 10 Sep 2025).

The spatial criterion is explicitly “between” boxes: a selected proposal must enclose the high-threshold box and be enclosed by the scaled low-threshold box. The paper states that this pre-selects all proposals that tightly bound a single instance. This suggests that HGPS uses the high-threshold region for instance separation and the low-threshold region for extent recovery.

Quantity	Default	Reported role or range
$F\in\mathbb{R}^{H'\times W'\times D}$8	0.8	Robust in $F\in\mathbb{R}^{H'\times W'\times D}$9
$1\times 1$0	0.3	Robust in $1\times 1$1
$1\times 1$2	1.2	Robust in $1\times 1$3
$1\times 1$4	0.5	Positive assignment
$1\times 1$5	0.1	Background assignment

3. Pseudo-ground-truth construction and coupling with WSBDN

After cluster construction, HGPS generates pseudo-ground-truth boxes for refinement modules. For IR module $1\times 1$6, each cluster is scored by the preceding module’s per-class score matrix $1\times 1$7. HGPS selects the proposal index

$1\times 1$8

for category $1\times 1$9 and takes the corresponding proposal as the pseudo-GT box $$A_c=\mathrm{Conv}_{1\times 1}^c(F)\in\mathbb{R}^{H'\times W'},\qquad c=1\ldots C.$$0. Proposal labels are then assigned from IoU with the chosen box:

$$A_c=\mathrm{Conv}_{1\times 1}^c(F)\in\mathbb{R}^{H'\times W'},\qquad c=1\ldots C.$$1

and

$$A_c=\mathrm{Conv}_{1\times 1}^c(F)\in\mathbb{R}^{H'\times W'},\qquad c=1\ldots C.$$2

Within the complete framework, HGPS is run once per training iteration for the image and its proposals to produce pseudo-GT clusters. WSBDN serves as the base MIDN. The initial pseudo boxes from HGPS supervise WSBDN’s class-wise softmax via a standard cross-entropy, after which three heatmap-guided IR modules follow. Each IR module re-computes pseudo GTs using HGPS and the preceding module’s scores, and applies the cross-entropy together with ignored-proposal losses. The framework also introduces a negative certainty supervision loss on ignored proposals to accelerate convergence, and the supervision of $$A_c=\mathrm{Conv}_{1\times 1}^c(F)\in\mathbb{R}^{H'\times W'},\qquad c=1\ldots C.$$3 is described as bridging the gap between raw class-wise scores and final weighted detection scores $$A_c=\mathrm{Conv}_{1\times 1}^c(F)\in\mathbb{R}^{H'\times W'},\qquad c=1\ldots C.$$4, ensuring that the class branch learns a semantically meaningful background class (Guo et al., 10 Sep 2025).

The implementation details reported for this pipeline include VGG16 with RoI-Pool as backbone, approximately $$A_c=\mathrm{Conv}_{1\times 1}^c(F)\in\mathbb{R}^{H'\times W'},\qquad c=1\ldots C.$$5 proposals per image, SAM segments as pseudo masks for S2C training, and temperature $$A_c=\mathrm{Conv}_{1\times 1}^c(F)\in\mathbb{R}^{H'\times W'},\qquad c=1\ldots C.$$6 in the contrastive loss. The paper also recommends grid search around $$A_c=\mathrm{Conv}_{1\times 1}^c(F)\in\mathbb{R}^{H'\times W'},\qquad c=1\ldots C.$$7 and $$A_c=\mathrm{Conv}_{1\times 1}^c(F)\in\mathbb{R}^{H'\times W'},\qquad c=1\ldots C.$$8.

4. Reported performance in the WSOD setting

The WSOD framework containing HGPS, WSBDN, and the ignored-proposal supervision loss is evaluated on PASCAL VOC 2007 and 2012. The reported results are $$A_c=\mathrm{Conv}_{1\times 1}^c(F)\in\mathbb{R}^{H'\times W'},\qquad c=1\ldots C.$$9 on VOC 2007 and $$\varphi_c^{CAM}=\frac{1}{H'W'}\sum_{h'=1}^{H'}\sum_{w'=1}^{W'}A_c(h',w'),$$0 on VOC 2012, and the method is described as performing favorably against state-of-the-art WSOD methods (Guo et al., 10 Sep 2025).

These numbers are presented as evidence that dual-threshold heatmap guidance can improve pseudo-GT quality relative to approaches that either emphasize only discriminative regions or use looser object-level regions without separating adjacent same-class instances. A plausible implication is that HGPS is effective not because it introduces a new proposal source, but because it changes how existing proposals are filtered, grouped, and re-labeled during weak supervision.

5. Other HGPS formulations and adjacent heatmap-guided methods

A different HGPS module appears in a hybrid CNN–Transformer detector for astrocyte detection. There, the input to the heatmap head is the FPN feature map $$\varphi_c^{CAM}=\frac{1}{H'W'}\sum_{h'=1}^{H'}\sum_{w'=1}^{W'}A_c(h',w'),$$1 at one-quarter input resolution, with $$\varphi_c^{CAM}=\frac{1}{H'W'}\sum_{h'=1}^{H'}\sum_{w'=1}^{W'}A_c(h',w'),$$2 and $$\varphi_c^{CAM}=\frac{1}{H'W'}\sum_{h'=1}^{H'}\sum_{w'=1}^{W'}A_c(h',w'),$$3 for a $$\varphi_c^{CAM}=\frac{1}{H'W'}\sum_{h'=1}^{H'}\sum_{w'=1}^{W'}A_c(h',w'),$$4 input. A $$\varphi_c^{CAM}=\frac{1}{H'W'}\sum_{h'=1}^{H'}\sum_{w'=1}^{W'}A_c(h',w'),$$5 convolution followed by a pointwise sigmoid produces a spatial probability map $$\varphi_c^{CAM}=\frac{1}{H'W'}\sum_{h'=1}^{H'}\sum_{w'=1}^{W'}A_c(h',w'),$$6. A ground-truth heatmap $$\varphi_c^{CAM}=\frac{1}{H'W'}\sum_{h'=1}^{H'}\sum_{w'=1}^{W'}A_c(h',w'),$$7 is formed by centering a Gaussian peak at each annotated box center, with radius derived from the box size, for example $$\varphi_c^{CAM}=\frac{1}{H'W'}\sum_{h'=1}^{H'}\sum_{w'=1}^{W'}A_c(h',w'),$$8 (Zhang et al., 3 Sep 2025).

This astrocyte-detection HGPS converts the predicted heatmap into $$\varphi_c^{CAM}=\frac{1}{H'W'}\sum_{h'=1}^{H'}\sum_{w'=1}^{W'}A_c(h',w'),$$9 spatial anchors, with default $A_c$0, by applying $A_c$1 max-pool non-maximum suppression, retaining local maxima, ranking them by heatmap score, and selecting the top $A_c$2 positions. Each anchor stores normalized coordinates, a score from the heatmap, a pooled feature sampled bilinearly from $A_c$3, and an initial Transformer query vector in $A_c$4 with $A_c$5. Bounding boxes are decoded from query outputs using an anchor-scale $A_c$6 and reference sizes $A_c$7, and inference applies Soft-NMS with IoU threshold $A_c$8 and $A_c$9. The detector integrates a ResNet-50 backbone, a lightweight Transformer encoder block at $$\tilde A_c(x,y)=\frac{A_c^{\uparrow}(x,y)-\min A_c^{\uparrow}}{\max A_c^{\uparrow}-\min A_c^{\uparrow}}\in[0,1].$$0, a 3-level FPN, cross-scale memory, and a 6-layer Transformer decoder (Zhang et al., 3 Sep 2025).

The reported impact of this HGPS is quantitative. On Test 05019 Cohort 1 (ALDH1L1), AP improves from $$\tilde A_c(x,y)=\frac{A_c^{\uparrow}(x,y)-\min A_c^{\uparrow}}{\max A_c^{\uparrow}-\min A_c^{\uparrow}}\in[0,1].$$1 to $$\tilde A_c(x,y)=\frac{A_c^{\uparrow}(x,y)-\min A_c^{\uparrow}}{\max A_c^{\uparrow}-\min A_c^{\uparrow}}\in[0,1].$$2 and AR from $$\tilde A_c(x,y)=\frac{A_c^{\uparrow}(x,y)-\min A_c^{\uparrow}}{\max A_c^{\uparrow}-\min A_c^{\uparrow}}\in[0,1].$$3 to $$\tilde A_c(x,y)=\frac{A_c^{\uparrow}(x,y)-\min A_c^{\uparrow}}{\max A_c^{\uparrow}-\min A_c^{\uparrow}}\in[0,1].$$4; on Test 05019 Cohort 2, AP improves from $$\tilde A_c(x,y)=\frac{A_c^{\uparrow}(x,y)-\min A_c^{\uparrow}}{\max A_c^{\uparrow}-\min A_c^{\uparrow}}\in[0,1].$$5 to $$\tilde A_c(x,y)=\frac{A_c^{\uparrow}(x,y)-\min A_c^{\uparrow}}{\max A_c^{\uparrow}-\min A_c^{\uparrow}}\in[0,1].$$6 and AR from $$\tilde A_c(x,y)=\frac{A_c^{\uparrow}(x,y)-\min A_c^{\uparrow}}{\max A_c^{\uparrow}-\min A_c^{\uparrow}}\in[0,1].$$7 to $$\tilde A_c(x,y)=\frac{A_c^{\uparrow}(x,y)-\min A_c^{\uparrow}}{\max A_c^{\uparrow}-\min A_c^{\uparrow}}\in[0,1].$$8; on Test 03557, AP improves from $$\tilde A_c(x,y)=\frac{A_c^{\uparrow}(x,y)-\min A_c^{\uparrow}}{\max A_c^{\uparrow}-\min A_c^{\uparrow}}\in[0,1].$$9 to $\tau^{high}$0 and AR from $\tau^{high}$1 to $\tau^{high}$2. For GFAP, the gains are reported as $\tau^{high}$3 to $\tau^{high}$4 in AP and $\tau^{high}$5 to $\tau^{high}$6 in AR. FROC analysis further reports a $\tau^{high}$7–$\tau^{high}$8 sensitivity gain at a fixed FPPI budget such as $\tau^{high}$9 false alarm per image, and bootstrap $\tau^{low}$0 confidence bands are said not to overlap in the low-FPPI regime (Zhang et al., 3 Sep 2025).

A related but differently named heatmap-guided design appears in 6-DoF grasp detection. That work proposes an effective local grasp generator combined with grasp heatmaps as guidance, which infers in a global-to-local semantic-to-point way. Gaussian encoding and a grid-based strategy are applied to predict grasp heatmaps, aggregate local points into graspable regions, and provide global semantic information; a novel non-uniform anchor sampling mechanism is added to improve grasp accuracy and diversity. The framework is described as performing high-quality grasp detection in real time, achieving state-of-the-art results, and in real robot experiments attaining a success rate of $\tau^{low}$1 and a clutter completion rate of $\tau^{low}$2 (Chen et al., 2024).

6. Distinctions, scope, and recurrent misconceptions

One recurrent misconception is to treat HGPS as a single canonical module with invariant semantics. The literature does not support that interpretation. In the WSOD formulation, HGPS is a dual-threshold clustering and pseudo-label construction algorithm operating over an existing proposal set $\tau^{low}$3 (Guo et al., 10 Sep 2025). In the astrocyte detector, HGPS is a heatmap-to-anchor and heatmap-to-query initializer embedded inside a fully supervised hybrid CNN–Transformer architecture (Zhang et al., 3 Sep 2025). In the grasping work, heatmaps guide local region aggregation for 6-DoF grasp generation, but the method is not presented as HGPS (Chen et al., 2024).

A second misconception is that the decisive ingredient is merely the presence of a heatmap. The reported designs instead emphasize the interface between dense heatmap evidence and sparse candidate selection. In the WSOD case, the high threshold tends to separate adjacent instances but misses object periphery, whereas the low threshold covers the full object but may merge nearby instances; HGPS is explicitly built around the combination of both effects (Guo et al., 10 Sep 2025). In the astrocyte detector, the heatmap is not an end product but the source of top-$\tau^{low}$4 spatial anchors and Transformer queries (Zhang et al., 3 Sep 2025).

A third misconception is that HGPS replaces the rest of the detection framework. The published formulations do not make that claim. In WSOD, HGPS is coupled to WSBDN, class-wise softmax supervision, and three heatmap-guided IR modules; in the astrocyte setting, it is coupled to FPN features, cross-scale memory, Hungarian matching, and detection losses; in grasp detection, heatmap guidance is combined with a local grasp generator and non-uniform anchor sampling. This suggests that HGPS is best understood as a task-specific mechanism for converting heatmap structure into a restricted candidate set, rather than as a complete detection or grasping system.