---
title: 'HGACNet: Hierarchical Graph Attention for 3D Completion'
url: https://www.emergentmind.com/topics/hgacnet
type: topic
---

# HGACNet: Hierarchical Graph Attention for 3D Completion

Searching arXiv for HGACNet and related papers to ground the article.
HGACNet is a cross-modal framework for object-level point cloud completion that takes a single-view RGB image and a partial 3D point cloud as input and reconstructs a complete object point cloud [2509.13692]. It is introduced in “HGACNet: Hierarchical Graph Attention Network for Cross-Modal Point Cloud Completion” and is designed to address incomplete geometry caused by self-occlusion and sensor limitations, conditions that can degrade robotic perception, object reconstruction, and downstream tasks such as grasp planning, obstacle avoidance, and manipulation [2509.13692]. The framework combines a Hierarchical Graph Attention (HGA) encoder for 3D geometric representation, a Multi-Scale Cross-Modal Fusion (MSCF) module for attention-based alignment between geometry and image features, and a contrastive loss (C-Loss) that explicitly reduces modality discrepancy [2509.13692].

## 1. Problem formulation and scope

HGACNet is formulated for point cloud completion under a cross-modal setting in which the inputs are a partial point cloud $P_p \in \mathbb{R}^{N_p \times 3}$ and a corresponding RGB image $I \in \mathbb{R}^{H \times W \times 3}$, and the output is a complete point cloud $P_c \in \mathbb{R}^{N_c \times 3}$ [2509.13692]. The stated objective is reconstruction of complete object geometry from incomplete 3D observations by leveraging image-guided priors from a single-view RGB image [2509.13692].

The reported setting is object-level rather than scene-level. In ShapeNet-ViPC, partial point clouds are generated from complete meshes via occlusion under 24 rendered viewpoints per object, and both partial and complete point clouds contain 2048 points, while images are $224 \times 224$ [2509.13692]. In YCB-Complete, partial point clouds are captured with a RealSense sensor and exhibit real-world noise, occlusions, and pose misalignment [2509.13692]. This distinction is significant because it places HGACNet between synthetic benchmark evaluation and robot-facing deployment conditions.

The paper situates HGACNet against both geometry-only and cross-modal completion methods. Compared to geometry-only methods such as PCN, FoldingNet, TopNet, GRNet, PoinTr, and SnowflakeNet, HGACNet leverages image priors to recover missing structure and details; relative to cross-modal baselines such as ViPC and CSDN, it uses deep hierarchical attention rather than shallow fusion through concatenation or style modulation [2509.13692]. A plausible implication is that the architecture is intended to address both semantic ambiguity in severely occluded regions and structural incompleteness in sparse point observations.

## 2. Architectural organization

The overall pipeline takes $P_p$ and $I$, encodes geometry using the HGA encoder to produce global features $F_g$ and local features $F_l$, encodes the image with a pre-trained, frozen Swin Transformer to obtain $F_I$, performs multi-scale fusion in the MSCF module, and decodes the fused representation to synthesize $P_c$ [2509.13692].

The feature hierarchy is explicit. The paper defines $F_g \in \mathbb{R}^{N_g \times D_g}$ and $F_l \in \mathbb{R}^{N_l \times D_l}$, where $N_g$ and $N_l$ are the numbers of selected key points at different hierarchy levels and $D_g, D_l$ are their feature dimensions [2509.13692]. In the implementation, the point set is reduced hierarchically to 512 and then 128 points, yielding local and global hierarchies [2509.13692]. Image features $F_I \in \mathbb{R}^{N_I \times D_I}$ are extracted from Swin Transformer tokens at multiple scales [2509.13692].

The architecture is modular:

| Module | Role |
|---|---|
| HGA encoder | Constructs local graphs, computes descriptors and node scores, performs attention-based downsampling, refines hierarchical features |
| MSCF | Projects geometric and visual features into a shared latent space; applies self-attention and cross-attention across levels and modalities |
| Decoder | Predicts dense points for missing regions and concatenates them with FPS-downsampled observed points |
| C-Loss | Aligns matched image–point cloud pairs and separates non-matching pairs |

The decoder follows XMFNet and transforms the fused representation directly into 3D coordinates of the completed point cloud by predicting a dense set of points approximating missing regions [2509.13692]. The final output is formed by concatenating these predicted points with a subsampled version of the input partial point cloud via farthest point sampling (FPS), thereby preserving observed geometry while completing missing surfaces [2509.13692]. In the reported benchmarks, the outputs match the ground-truth size of 2048 points [2509.13692].

## 3. Hierarchical Graph Attention encoder

The HGA encoder is the principal geometric component of HGACNet. It hierarchically encodes 3D geometric features and adaptively selects critical local points through graph attention-based downsampling, with the stated purpose of better capturing structural continuity and spatial relationships [2509.13692].

Its first stage is the Graph Descriptor (GD). For each point $p_i$ in $P_p$, a local graph is constructed by connecting to $K_1$ nearest neighbors using k-NN [2509.13692]. Raw point coordinates are lifted by a shared MLP to point-wise features $F_p \in \mathbb{R}^{N_p \times D}$ [2509.13692]. Edge features between $p_i$ and a neighbor $p_j$ are defined as
$$
e_{ij} = \phi(p_i - p_j, f_i, f_j),
$$
where $f_i$ and $f_j$ are the features of $p_i$ and $p_j$, and $\phi(\cdot)$ is a learnable function [2509.13692]. These encoded edges are processed by a 2D convolution, Group Normalization (GroupNorm), and LeakyReLU, followed by Max Pooling to aggregate local neighborhoods into compact point embeddings [2509.13692].

The second stage is Graph Attention Downsampling (GAD). Each node receives a learnable importance score
$$
w_i = \psi(f_i),
$$
with $\psi(\cdot)$ a learnable scoring function, and the highest-scoring points are selected [2509.13692]. Downsampling is hierarchical: the point set is first reduced to 512 points and then to 128 points, producing local and global keypoint sets [2509.13692]. Positional encoding is concatenated to selected point features through
$$
F'_{\text{ds}} = \text{MLP}\big(F_{\text{ds}} \oplus \gamma(P_{\text{ds}})\big),
$$
where $\gamma(\cdot)$ is a sinusoidal positional encoding and $\oplus$ denotes channel-wise concatenation [2509.13692]. Independent projection modules then map features from different hierarchy levels to a common space, with $F_l$ intended to capture fine-grained geometric details and $F_g$ higher-level semantic structures [2509.13692].

The paper explicitly notes that graph attention in HGA is realized as node-wise importance scoring for downsampling rather than vanilla edge-attention with $\alpha_{ij}$ [2509.13692]. Residual connections and explicit multi-head attention are not reported; normalization is performed through GroupNorm and activation through LeakyReLU [2509.13692]. This is important for terminological precision, because the “attention” mechanism here refers primarily to adaptive keypoint selection rather than a canonical transformer-style graph attention operator.

## 4. Multi-Scale Cross-Modal Fusion and alignment

The MSCF module performs attention-based feature alignment between hierarchical geometric features and structured visual representations, enabling fine-grained semantic guidance for completion [2509.13692]. Its visual branch uses a pre-trained and frozen Swin Transformer, whose window-based and shifted window self-attention capture both fine textures and global context [2509.13692].

Before fusion, geometric and visual features are projected into a shared latent space:
$$
F'_g = \Psi_g(F_g), \quad F'_l = \Psi_l(F_l), \quad F'_I = \Psi_I(F_I),
$$
where $\Psi(\cdot)$ are learnable linear transformations [2509.13692]. The fusion process then applies several attention operations. Self-attention within $F'_g$ and $F'_l$ models intra-modal dependencies and produces $F_{gg}$ and $F_{ll}$; cross-attention between $F'_g$ and $F'_l$ refines global and local features mutually, producing $F_{gl}$; and cross-modal attention between point cloud features $(F'_g, F'_l)$ and image features $F'_I$ produces $F_{Ig}$ and $F_{Il}$ [2509.13692]. The attention operator is given by
$$
A = \mathrm{Softmax}\!\left(\frac{QK^\top}{\sqrt{D}}\right), \quad \mathrm{Output} = A V,
$$
where $Q$, $K$, and $V$ are learned projections and $D$ is the feature dimension [2509.13692].

The fused set $\{F_{gg}, F_{gl}, F_{ll}, F_{Ig}, F_{Il}\}$ is concatenated and sent to the decoder [2509.13692]. Empirically, one self-attention and one cross-attention layer per scale are reported to balance expressiveness and efficiency; stacking more layers yields marginal accuracy gains but large inference-time costs [2509.13692]. Attention visualizations show that global features attend to overall object structure while local features attend to fine details such as edges and slender parts [2509.13692].

This design places HGACNet within the broader class of cross-modal completion systems that use 2D priors to regularize 3D reconstruction, but its defining feature is the selective use of attention across hierarchical levels rather than a flat or shallow fusion strategy [2509.13692]. The paper contrasts this with ViPC and CSDN, describing their fusion as shallower [2509.13692]. This suggests that the contribution is not only multimodal conditioning but also the coupling of scale hierarchy with modality hierarchy.

## 5. Objective functions and training protocol

HGACNet is trained with a combination of reconstruction loss and cross-modal contrastive alignment [2509.13692]. The reconstruction objective is the L2-Chamfer Distance between predicted and ground-truth point sets:
$$
\mathrm{CD}_2(\mathbf{P}, \mathbf{Q}) =
\frac{1}{|\mathbf{P}|}\sum_{x \in \mathbf{P}} \min_{y \in \mathbf{Q}} \|x-y\|_2^2
+
\frac{1}{|\mathbf{Q}|}\sum_{y \in \mathbf{Q}} \min_{x \in \mathbf{P}} \|y-x\|_2^2.
$$
This is the primary geometric supervision term [2509.13692].

To reduce modality discrepancy, the paper introduces a batch-symmetric InfoNCE-based contrastive loss aligning point cloud global features and paired image features:
$$
\mathcal{L}_{\text{contrast}} =
- \frac{1}{2} \Bigg[
\sum_{i=1}^{B} \log \frac{\exp \big( \mathrm{sim}(F_g^{(i)}, F_I^{(i)}) / \tau \big)}{\sum_{j=1}^{B} \exp \big( \mathrm{sim}(F_g^{(i)}, F_I^{(j)}) / \tau \big)}
+
\sum_{i=1}^{B} \log \frac{\exp \big( \mathrm{sim}(F_I^{(i)}, F_g^{(i)}) / \tau \big)}{\sum_{j=1}^{B} \exp \big( \mathrm{sim}(F_I^{(i)}, F_g^{(j)}) / \tau \big)}
\Bigg],
$$
where $\mathrm{sim}(\cdot,\cdot)$ is cosine similarity and $\tau$ is a temperature [2509.13692]. The total loss is
$$
\mathcal{L}_{\text{total}} =
\lambda_{\text{CD}_2}\,\mathcal{L}_{\text{CD}_2}
+
\lambda_{\text{contrast}}\,\mathcal{L}_{\text{contrast}},
$$
with $\lambda_{\text{CD}_2}=0.8$ and $\lambda_{\text{contrast}}=0.2$ [2509.13692].

Implementation details reported in the paper are specific. The framework is implemented in PyTorch, the visual backbone is a pre-trained and frozen Swin Transformer, and the point cloud encoder uses GD with shared MLP, 2D convolution, GroupNorm, LeakyReLU, and MaxPool, followed by GAD with learnable scoring $\psi(\cdot)$ and sinusoidal positional encoding $\gamma(\cdot)$ [2509.13692]. Training is class-specific, uses the Adam optimizer, runs for 400 epochs with batch size 32, and starts from an initial learning rate of 0.1, reduced by a factor of $0.1$ at epochs 50, 80, 120, and 200 [2509.13692]. Hardware is an Nvidia RTX 4090 GPU [2509.13692]. Augmentation, parameter counts, and runtime are not reported, although the authors note that adding attention layers increases inference time significantly [2509.13692].

## 6. Benchmarks and empirical performance

HGACNet is evaluated on ShapeNet-ViPC and YCB-Complete using L2-CD and F-Score at threshold $d=0.001$ [2509.13692]. ShapeNet-ViPC is derived from ShapeNet and contains 38,328 objects across 13 categories, with evaluation conducted on 8 categories: airplane, cabinet, car, chair, lamp, sofa/couch, table, and watercraft [2509.13692]. The train/test split is 80/20% over selected categories, totaling 31,650 training objects [2509.13692]. YCB-Complete is reorganized from YCB-Video and contains 21 object categories with approximately 3,000 samples per category; it uses 17 known categories and 3 unknown categories—004_sugar_box, 007_tuna_fish_can, and 052_extra_large_clamp—to test category-level generalization [2509.13692].

On ShapeNet-ViPC, HGACNet achieves an average CD of $1.002 \times 10^{-3}$, compared with EGIINet at $1.211 \times 10^{-3}$ and XMFNet at $1.443 \times 10^{-3}$ [2509.13692]. The paper states that this corresponds to a 17% relative reduction versus EGIINet and 31% versus XMFNet [2509.13692]. The average F-Score@0.001 is 0.887, exceeding EGIINet at 0.836 and XMFNet at 0.796 [2509.13692]. Reported category-wise CD values for HGACNet are airplane 0.377, cabinet 1.458, car 1.340, chair 1.028, lamp 0.981, sofa 1.105, table 1.074, and watercraft 0.597, all $\times 10^{-3}$ [2509.13692]. The corresponding F-Score@0.001 values are airplane 0.983, cabinet 0.789, car 0.805, chair 0.895, lamp 0.896, sofa 0.879, table 0.894, and watercraft 0.954 [2509.13692].

On YCB-Complete, for known categories HGACNet reports $0.073 \times 10^{-3}$ CD and 0.995 F-score, outperforming XMFNet at $1.672 \times 10^{-3}$ and 0.822, and EGIINet at $1.533 \times 10^{-3}$ and 0.851 [2509.13692]. For unknown categories, HGACNet reports $7.996 \times 10^{-3}$ CD and 0.405 F-score, compared with EGIINet at $8.251 \times 10^{-3}$ and 0.300, and XMFNet at $9.736 \times 10^{-3}$ with 0.731 F-score reported higher but with much higher CD [2509.13692]. The paper states that these results reflect dataset diversity and difficulty [2509.13692].

Qualitative results are used to support the quantitative findings. HGACNet is reported to preserve thin and structurally complex regions such as airplane tail wings, cabinet drawers, slender table legs, watercraft detailing, pitcher base handle, mug handles, and power drill edges more faithfully than baselines, including under noise and occlusion [2509.13692]. This suggests that the combination of hierarchical geometry encoding and image-guided fusion particularly benefits completion of narrow or semantically constrained parts.

## 7. Ablation findings, limitations, and nomenclature

The ablation study isolates the contributions of local feature extraction, MSCF, C-Loss, and image input [2509.13692]. For selected ShapeNet-ViPC categories, full HGACNet reports CD values of airplane 0.377, cabinet 1.458, car 1.340, and watercraft 0.653, all $\times 10^{-3}$ [2509.13692]. Removing local feature extraction degrades performance to 0.546, 1.711, 1.533, and 0.800, respectively; removing MSCF degrades performance to 0.632, 1.764, 1.619, and 0.834; removing C-Loss yields 0.512, 1.634, 1.439, and 0.793; and removing image input yields 0.471, 1.751, 1.472, and 0.744 [2509.13692]. The paper interprets these results as showing that local features are crucial for fine detail, attention-based cross-modal fusion is necessary, contrastive alignment significantly reduces the modality gap, and visual priors help infer unobserved geometry [2509.13692].

The paper identifies several limitations. Performance decreases on unknown categories in YCB-Complete, which the authors attribute to limited intra-category variation and fewer training classes [2509.13692]. Additional attention layers improve accuracy only slightly while increasing inference time [2509.13692]. Computational complexity, parameter counts, and exact runtime are not reported [2509.13692]. The authors nonetheless argue that selective, hierarchical attention is more efficient than stacking many attention blocks [2509.13692]. A plausible implication is that the design prioritizes representational selectivity over depth of attention.

A potential source of confusion arises from the superficially similar name HCGNet, introduced in “Gated Convolutional Networks with Hybrid Connectivity for Image Classification” [1908.09699]. That model is explicitly named HCGNet, not HGACNet, and the term “HGACNet” does not appear in the paper; it is described there as a misnomer rather than a distinct model [1908.09699]. HCGNet is an image classification backbone based on Hybrid Connectivity and Gated mechanisms, whereas HGACNet is a cross-modal point cloud completion framework built around hierarchical graph attention, multi-scale cross-modal fusion, and contrastive alignment [2509.13692][1908.09699]. The two models are therefore unrelated in task formulation, modality structure, and architectural objective despite the similarity of their acronyms.

From a robotics perspective, HGACNet is presented as applicable to grasping, pose estimation, and motion planning because it reconstructs complete, structurally consistent object point clouds with semantic guidance from images [2509.13692]. The paper does not report a dedicated manipulation benchmark such as grasp success rate, but its YCB-Complete results and qualitative examples are presented as evidence of robustness to noise and occlusion in real-world conditions [2509.13692].

Source: https://www.emergentmind.com/topics/hgacnet