HVPNet: Polysemous Neural Architectures

• HVPNet is a polysemous term referring to distinct neural architectures that serve tasks such as saliency detection, video-text retrieval, and multimodal entity extraction.
• It employs specialized modules like attention mechanisms, multi-layer patch processing, and visual prefix fusion to capture hierarchical semantics and improve task-specific performance.
• Empirical results demonstrate efficiency and accuracy boosts—with lightweight SOD models using 1.23M parameters and 333 fps, and enhanced retrieval and extraction metrics in other variants.

In current literature, HVPNet and closely related spellings denote several distinct neural architectures rather than a single canonical model. The label has been attached to a lightweight salient object detection network used as an auxiliary saliency prior in black-box adversarial attacks (Zhou et al., 2022), a CLIP-based Hierarchical Visual Perception Network for video-text retrieval (Xie et al., 19 Jan 2026), a Hierarchical Visual Prefix fusion NeTwork for multimodal entity and relation extraction (Chen et al., 2022), and a bio-inspired Human Visual Process Network for salient and camouflaged object detection across multiple modalities (Xu et al., 30 Jun 2026). Precise interpretation therefore depends on the paper, task, and spelling variant.

1. Nomenclature and scope

The term appears in several orthographically similar forms—HVPNet, HVP-Net, and HVPNeT—with materially different semantics. In one usage, HVPNet is a lightweight SOD model that outputs a binary saliency map and is cited as containing only 1.23M parameters while reaching 333.2 fps (336×336×3 frame size) (Zhou et al., 2022). In another, HVP-Net is a retrieval framework built on CLIP and multi-layer token processing for video-text retrieval (Xie et al., 19 Jan 2026). HVPNeT denotes a prefix-based multimodal information extraction model that injects gated visual prefixes into BERT self-attention (Chen et al., 2022). The 2026 detection paper uses HVPNet as the name of a modality-agnostic SOD/COD architecture structured around a Retinal Integration Module and Cortical Decoder (Xu et al., 30 Jun 2026).

Name in paper	Primary domain	Defining mechanism
HVPNet	Salient object detection prior for adversarial attack	Binary SOD mask used with YOLOv4 and SimBA
HVP-Net	Video-text retrieval	Multi-layer patch processing and multi-granularity alignment
HVPNeT	Multimodal NER and RE	Hierarchical visual prefix with dynamic gated aggregation
HVPNet	General SOD/COD	Retinal Integration Module and Cortical Decoder
SC-HVPPNet	Video post-processing	Spatial and channel hybrid-attention CNN-Transformer fusion

A recurrent misconception is that all occurrences refer to a single saliency model. The publication record instead indicates a polysemous naming pattern. That ambiguity is amplified by nearby names such as HV-Net, which explicitly states that it does not use the acronyms “HVPNet”, “HVNet”, or “Hypervolume Net” (Shang et al., 2022).

2. HVPNet as a lightweight salient object detector in adversarial attack pipelines

In "Object-Attentional Untargeted Adversarial Attack" (Zhou et al., 2022), HVPNet is not the attack model itself but a pre-trained salient object detection module embedded in a four-stage black-box attack pipeline. The stages are: object detection with YOLOv4, SOD with HVPNet, combinatorial optimization to derive a final object-region mask, and adversarial example generation via SimBA restricted to that mask. HVPNet therefore defines the support of perturbations rather than the attack loss.

The network receives the original image $x_0$ and produces a binary saliency map

$S_2 = f_2(x_0).$

The paper states that the output is already in binary-image form: white pixels indicate the salient object and black pixels indicate background. No additional thresholding is described. This hard mask is intersected with the YOLOv4 box mask $S_1$ to produce a candidate object region,

$S = S_1 \cap S_2.$

The paper also summarizes the underlying HVP module used by the original HVPNet: $$R_r(x_0)=\left\{ \begin{array}{cl} F^{1\times1}(x_0), & \textrm{if} \,\, r = 1 \ F^{1\times1}(\hat{F}^{3\times3}_r(F^{1\times1}(\hat{F}_{1}^{r\times r}(x_0))), & \textrm{if} \,\, r > 1, \ \end{array} \right.$$ where $F^{1\times 1}$ is a standard $1\times1$ convolution, $\hat{F}_{1}^{r\times r}$ is a depthwise separable convolution with kernel size $r\times r$, and $\hat{F}^{3\times3}_r$ is a depthwise separable convolution with dilation rate $S_2 = f_2(x_0).$0. HVPNet is described there as a lightweight SOD network combining the HVP module and an attention mechanism, with 1.23M parameters and 333.2 fps at 336×336×3 input resolution.

A central issue is incomplete SOD: the saliency mask may cover only part of the object. To detect this, the method defines an activation factor

$S_2 = f_2(x_0).$1

implemented as $r\times r$7 with $S_2 = f_2(x_0).$2. If $S_2 = f_2(x_0).$3, HVPNet’s output is treated as unreliable and the method falls back to the YOLOv4 mask alone. Otherwise the intersection is retained. This logic prevents the perturbation from collapsing into a tiny, visually concentrated region.

The resulting mask is vectorized into $S_2 = f_2(x_0).$4, repeated into $S_2 = f_2(x_0).$5, and used to construct the SimBA search-direction set

$S_2 = f_2(x_0).$6

followed by deletion of all-zero rows. HVPNet thus modifies SimBA by restricting orthogonal directions to the object region.

Experimentally, the combined YOLOv4+HVPNet with activation variant, denoted OA, improved query efficiency and perturbation quality relative to full-image SimBA and to single-source masking baselines. On COCO-Reduced-ImageNet, for VGG16, average queries decreased from 1502 to 1217, median queries from 923 to 700, average $S_2 = f_2(x_0).$7 from 3.96 to 3.47, and median $S_2 = f_2(x_0).$8 from 3.86 to 3.29. On Inception V3, perceptual quality improved from PSNR 37.1880 / SSIM 0.9688 for SimBA to PSNR 38.4209 / SSIM 0.9784 for OA (Zhou et al., 2022). The paper also reports that SLH, where HVPNet alone defines the attack region, improves over SimBA but is weaker than the fused OA configuration. This suggests that HVPNet provides a useful saliency prior, but one that benefits from detector-side geometric regularization.

3. HVP-Net for hierarchical video-text retrieval

"Delving Deeper: Hierarchical Visual Perception for Robust Video-Text Retrieval" introduces HVP-Net as a VTR framework that departs from the common practice of using only the final-layer CLS token of CLIP-like vision encoders (Xie et al., 19 Jan 2026). The model uses CLIP with ViT-B/32 as the visual backbone and CLIP’s text encoder on the language side. Its core premise is that intermediate ViT layers encode complementary low-, mid-, and high-level semantics that are lost when retrieval relies exclusively on final-layer summarization.

For each video with $S_2 = f_2(x_0).$9 sampled frames and each selected layer $S_1$0, HVP-Net extracts frame features

$S_1$1

from CLS tokens and patch features

$S_1$2

The default layer set is

$S_1$3

corresponding to shallow, middle, and deep semantic levels. On the text side, the model uses a global sentence embedding $S_1$4 and word-level embeddings $S_1$5, with a maximum text length of 24 tokens.

The central architectural component is the Multi-layer Patch Processing (MPP) module. For each selected layer, MPP first applies a Patch Compression Module (PCM) that computes patch saliency, uses Density-Peak Clustering (DPC) to choose $S_1$6 representative patch centers, and forms compressed tokens via saliency-weighted pooling: $S_1$7 These compressed tokens then act as queries in a cross-attention refinement stage against the original patch set: $S_1$8 After iterative compression-refinement cycles, the layer yields refined patch concepts

$S_1$9

HVP-Net aligns text and video at three granularities. Sentence–Frame (SF) similarity compares $S = S_1 \cap S_2.$0 against layer-wise frame tokens. Sentence–Patch (SP) similarity compares $S = S_1 \cap S_2.$1 against refined patch concepts. Word–Patch (WP) similarity is the most fine-grained: each word is matched to the maximally similar patch concept, reweighted by a learnable MLP importance score,

$S = S_1 \cap S_2.$2

A symmetric patch-to-word term is averaged with this quantity to form the final WP score. Training uses a symmetric InfoNCE loss summed over SF, SP, and WP at all selected layers.

The empirical results emphasize that hierarchical perception is useful only when redundancy is actively suppressed. On MSR-VTT 1k-A, a baseline using the last layer alone obtains R@1 = 54.2 for text-to-video retrieval, whereas naive multi-layer fusion without MPP collapses to 22.1. The full HVP-Net reaches 56.7. Removing the word-patch loss is especially damaging, reducing R@1 from 56.7 to 33.3. On the benchmark tables, HVP-Net reports 56.7 / 83.9 / 90.6 for R@1 / R@5 / R@10 in MSR-VTT T2V, 50.5 / 80.5 / 91.2 in MSR-VTT V2T, 57.1 / 83.1 / 87.0 on DiDeMo, and 43.5 / 78.3 / 88.2 on ActivityNet (Xie et al., 19 Jan 2026). The method therefore treats the visual encoder as a hierarchy of semantic levels rather than a single terminal representation.

4. HVPNeT for multimodal entity and relation extraction

"Good Visual Guidance Makes A Better Extractor: Hierarchical Visual Prefix for Multimodal Entity and Relation Extraction" defines HVPNeT as a text-centric multimodal model for MNER and MRE on social-media data (Chen et al., 2022). Its design target is robustness to irrelevant or noisy visual objects, a failure mode the paper describes as error sensitivity when irrelevant object images are incorporated in texts.

The model uses BERT-base as the textual encoder and ResNet-50 as the visual backbone. Visual evidence comes from the global image and from the top $S = S_1 \cap S_2.$3 salient objects extracted using the visual grounding toolkit of Yang et al. (2019), following UMGF. ResNet provides a feature pyramid

$S = S_1 \cap S_2.$4

and these multi-scale features are mapped into a common hidden size. Rather than concatenating visual tokens with text at the input layer, HVPNeT treats visual features as layer-specific visual prefixes that are injected into self-attention keys and values: $S = S_1 \cap S_2.$5

The hierarchical prefix is generated through dynamic gated aggregation. For each BERT layer $S = S_1 \cap S_2.$6, pooled visual features are projected to logits

$S = S_1 \cap S_2.$7

and normalized to a layer-dependent gate vector

$S = S_1 \cap S_2.$8

These weights determine how strongly each ResNet block contributes to the visual prefix at that layer. The resulting gated features from the global image and object crops are concatenated into a hierarchical visual representation $S = S_1 \cap S_2.$9, then projected into $$R_r(x_0)=\left\{ \begin{array}{cl} F^{1\times1}(x_0), & \textrm{if} \,\, r = 1 \ F^{1\times1}(\hat{F}^{3\times3}_r(F^{1\times1}(\hat{F}_{1}^{r\times r}(x_0))), & \textrm{if} \,\, r > 1, \ \end{array} \right.$$0 and $$R_r(x_0)=\left\{ \begin{array}{cl} F^{1\times1}(x_0), & \textrm{if} \,\, r = 1 \ F^{1\times1}(\hat{F}^{3\times3}_r(F^{1\times1}(\hat{F}_{1}^{r\times r}(x_0))), & \textrm{if} \,\, r > 1, \ \end{array} \right.$$1. This mechanism allows lower BERT layers to emphasize low-level visual features and higher layers to emphasize more semantic visual scales.

Task heads are conventional but tightly integrated with the prefix architecture. For MNER, final token states are fed to a CRF with negative log-likelihood objective: $$R_r(x_0)=\left\{ \begin{array}{cl} F^{1\times1}(x_0), & \textrm{if} \,\, r = 1 \ F^{1\times1}(\hat{F}^{3\times3}_r(F^{1\times1}(\hat{F}_{1}^{r\times r}(x_0))), & \textrm{if} \,\, r > 1, \ \end{array} \right.$$2 For MRE, the final $$R_r(x_0)=\left\{ \begin{array}{cl} F^{1\times1}(x_0), & \textrm{if} \,\, r = 1 \ F^{1\times1}(\hat{F}^{3\times3}_r(F^{1\times1}(\hat{F}_{1}^{r\times r}(x_0))), & \textrm{if} \,\, r > 1, \ \end{array} \right.$$3 representation is classified by softmax: $$R_r(x_0)=\left\{ \begin{array}{cl} F^{1\times1}(x_0), & \textrm{if} \,\, r = 1 \ F^{1\times1}(\hat{F}^{3\times3}_r(F^{1\times1}(\hat{F}_{1}^{r\times r}(x_0))), & \textrm{if} \,\, r > 1, \ \end{array} \right.$$4 with cross-entropy loss.

The reported performance is strong on all three benchmarks. On Twitter-2015, HVPNeT reaches 75.32 F1 for MNER, compared with 74.85 for UMGF and 71.81 for BERT-CRF. On Twitter-2017, it reaches 86.87 F1, above 85.51 for UMGF and 83.44 for BERT-CRF. On MNRE, it reports 81.85 F1, compared with 66.41 for MEGA and 60.86 for MTB (Chen et al., 2022). In cross-task transfer, the model also improves from 81.85 to 82.50 when moving from Twitter-2017 → MNRE, and from 86.87 to 87.13 for MNRE → Twitter-2017. The qualitative interpretation in the paper is that the prefix formulation allows visual evidence to function as guidance rather than compulsory evidence, so irrelevant images can be effectively down-weighted.

5. HVPNet as a bio-inspired general architecture for SOD and COD

"HVPNet: A Bio-Inspired Network for General Salient and Camouflaged Object Detection" presents a different and broader use of the name: a general architecture for seven tasks across four modalities (Xu et al., 30 Jun 2026). Those tasks are RGB SOD, RGB COD, RGB-D SOD, RGB-D COD, RGB-T SOD, VSOD, and VCOD. The modalities are RGB, RGB-D, RGB-T, and RGB + optical flow. The paper positions the model against the prevailing pattern of heavy backbones, heavy fusion modules, and heavy decoders.

The architecture has three parts. First, two encoders extract four feature levels from the RGB and auxiliary streams: $$R_r(x_0)=\left\{ \begin{array}{cl} F^{1\times1}(x_0), & \textrm{if} \,\, r = 1 \ F^{1\times1}(\hat{F}^{3\times3}_r(F^{1\times1}(\hat{F}_{1}^{r\times r}(x_0))), & \textrm{if} \,\, r > 1, \ \end{array} \right.$$5 The default backbone pairing is SMT-t for RGB and MobileNetV2 for the auxiliary modality. Second, a Retinal Integration Module (RIM) fuses the streams with three level-specific stages. Third, a Cortical Decoder (CD) performs hierarchical decoding through a High-Level Visual Decoder (HLVD) and a Low-Level Visual Decoder (LLVD).

Stage 1 of RIM is designed for shallow, edge-rich features. It uses additive and multiplicative fusion,

$$R_r(x_0)=\left\{ \begin{array}{cl} F^{1\times1}(x_0), & \textrm{if} \,\, r = 1 \ F^{1\times1}(\hat{F}^{3\times3}_r(F^{1\times1}(\hat{F}_{1}^{r\times r}(x_0))), & \textrm{if} \,\, r > 1, \ \end{array} \right.$$6

followed by local $$R_r(x_0)=\left\{ \begin{array}{cl} F^{1\times1}(x_0), & \textrm{if} \,\, r = 1 \ F^{1\times1}(\hat{F}^{3\times3}_r(F^{1\times1}(\hat{F}_{1}^{r\times r}(x_0))), & \textrm{if} \,\, r > 1, \ \end{array} \right.$$7 convolution and dilated $$R_r(x_0)=\left\{ \begin{array}{cl} F^{1\times1}(x_0), & \textrm{if} \,\, r = 1 \ F^{1\times1}(\hat{F}^{3\times3}_r(F^{1\times1}(\hat{F}_{1}^{r\times r}(x_0))), & \textrm{if} \,\, r > 1, \ \end{array} \right.$$8 convolutions with dilation rates $$R_r(x_0)=\left\{ \begin{array}{cl} F^{1\times1}(x_0), & \textrm{if} \,\, r = 1 \ F^{1\times1}(\hat{F}^{3\times3}_r(F^{1\times1}(\hat{F}_{1}^{r\times r}(x_0))), & \textrm{if} \,\, r > 1, \ \end{array} \right.$$9: $F^{1\times 1}$0 Stage 2 handles mid-level semantics through Selective Region Attention (SRA) applied to both multiplicative and additive fusion branches: $F^{1\times 1}$1

$F^{1\times 1}$2

Stage 3 processes deep semantics using multiplicative and additive fusion, dimensionality reduction $F^{1\times 1}$3, channel fusion $F^{1\times 1}$4, and spatial attention: $F^{1\times 1}$5 with

$F^{1\times 1}$6

The Cortical Decoder separates semantic integration from detail refinement. HLVD upsamples and fuses $F^{1\times 1}$7 into a high-level attention map $F^{1\times 1}$8, then applies Gaussian Guide Attention (GGA) using a $F^{1\times 1}$9 Gaussian kernel with $1\times1$0: $1\times1$1 LLVD then uses $1\times1$2 to guide low-level refinement of $1\times1$3. Training uses a sum of binary cross-entropy and IoU loss over four prediction heads: $1\times1$4

A major claim of the paper is the accuracy-efficiency trade-off. The standard model uses about 11.7M params and 8.1G FLOPs in the RGB setting, and 16.6M params and 11.5G FLOPs in RGB-D, RGB-T, VSOD, and VCOD settings. The lightweight HVPNet(–) variant uses 2.8–3.6M params and 1.8–3.4G FLOPs depending on task (Xu et al., 30 Jun 2026). Representative results include RGB SOD on DUTS with $1\times1$5, $1\times1$6, $1\times1$7, $1\times1$8; RGB-T SOD on VT1000 with $1\times1$9, $\hat{F}_{1}^{r\times r}$0, $\hat{F}_{1}^{r\times r}$1, $\hat{F}_{1}^{r\times r}$2; RGB-D COD on CAMO with $\hat{F}_{1}^{r\times r}$3, $\hat{F}_{1}^{r\times r}$4, $\hat{F}_{1}^{r\times r}$5, $\hat{F}_{1}^{r\times r}$6; VSOD on SegV2 with $\hat{F}_{1}^{r\times r}$7, $\hat{F}_{1}^{r\times r}$8, $\hat{F}_{1}^{r\times r}$9, $r\times r$0; and VCOD on CAD with $r\times r$1 and $r\times r$2.

The failure analysis is also explicit. Errors arise when both modalities consistently emphasize an irrelevant region, when object structure is weak because of transparent interiors or poor contrast, and when highly complex or blurred boundaries defeat the model’s structural refinement. The paper therefore frames the bio-inspired mapping as a conceptual abstraction rather than a neurophysiological model.

6. Related names, adjacent architectures, and disambiguation boundaries

Several adjacent names are relevant because they are easily conflated with HVPNet. SC-HVPPNet, introduced for VVC video post-processing, is a Spatial and Channel Hybrid-Attention Video Post-Processing Network with CNN and Transformer rather than a saliency or retrieval model (Zhang et al., 2024). It processes decoded frames and QP information, splits features into local CNN and global Swin Transformer streams, and fuses them through a Spatial Attention Fusion Module (SAFM) and Channel Attention Fusion Module (CAFM): $r\times r$3 Its reported overall BD-rate savings versus VTM-11.0-NNVC are -5.54% / -14.18% / -14.31% for Y/U/V PSNR, and -5.13% / -15.89% / -14.47% for Y/U/V MS-SSIM. For the low-QP range 22–37, the averages are -5.29%, -12.42%, and -13.09% for Y, U, and V, respectively (Zhang et al., 2024). The paper itself interprets the design as a hybrid VPP network, but its scope is codec-side restoration, not the broader HVPNet family used in SOD or retrieval.

HV-Net, by contrast, belongs to evolutionary multi-objective optimization and approximates the hypervolume indicator with DeepSets (Shang et al., 2022). Its defining equation is

$r\times r$4

The paper explicitly states that it does not use the acronyms “HVPNet”, “HVNet”, or “Hypervolume Net”. It is therefore not an HVPNet in the naming used by its authors.

A further adjacent term is HVPR, "Hybrid Voxel-Point Representation for Single-stage 3D Object Detection" (Noh et al., 2021). Its official paper title uses HVPR, not HVPNet, although the supplied description aligns it with the broader naming ambiguity. HVPR builds a voxel-point hybrid representation, replaces the expensive point branch at test time with a learned memory of point-feature prototypes,

$r\times r$5

and reports 36.1 FPS on KITTI with car AP$r\times r$6 of 86.38 / 77.92 / 73.04 for Easy / Moderate / Hard (Noh et al., 2021). It is best read as a distinct architecture with a separate acronym.

Taken together, these distinctions show that HVPNet is best understood as a family of unrelated or loosely related names rather than a unified lineage. In current arXiv usage, the label may refer to a saliency prior, a hierarchical visual perception framework, a visual prefix fusion network, or a bio-inspired detection architecture; adjacent names such as SC-HVPPNet, HV-Net, and HVPR broaden the ambiguity further. For technical communication, the paper title, task, and exact spelling remain essential.