HGACNet: Hierarchical Graph Attention for 3D Completion

• The paper proposes HGACNet, a cross-modal framework employing a hierarchical graph attention encoder to fuse partial 3D point clouds and RGB images for complete reconstruction.
• It features a Multi-Scale Cross-Modal Fusion module that aligns geometric and visual features using attention mechanisms for enhanced detail recovery.
• Empirical results show significant reductions in Chamfer Distance and improved F-Scores compared to baselines, underscoring its robustness in handling occlusions.

Searching arXiv for HGACNet and related papers to ground the article. HGACNet is a cross-modal framework for object-level point cloud completion that takes a single-view RGB image and a partial 3D point cloud as input and reconstructs a complete object point cloud (Zeng et al., 17 Sep 2025). It is introduced in “HGACNet: Hierarchical Graph Attention Network for Cross-Modal Point Cloud Completion” and is designed to address incomplete geometry caused by self-occlusion and sensor limitations, conditions that can degrade robotic perception, object reconstruction, and downstream tasks such as grasp planning, obstacle avoidance, and manipulation (Zeng et al., 17 Sep 2025). The framework combines a Hierarchical Graph Attention (HGA) encoder for 3D geometric representation, a Multi-Scale Cross-Modal Fusion (MSCF) module for attention-based alignment between geometry and image features, and a contrastive loss (C-Loss) that explicitly reduces modality discrepancy (Zeng et al., 17 Sep 2025).

1. Problem formulation and scope

HGACNet is formulated for point cloud completion under a cross-modal setting in which the inputs are a partial point cloud $P_p \in \mathbb{R}^{N_p \times 3}$ and a corresponding RGB image $I \in \mathbb{R}^{H \times W \times 3}$, and the output is a complete point cloud $P_c \in \mathbb{R}^{N_c \times 3}$ (Zeng et al., 17 Sep 2025). The stated objective is reconstruction of complete object geometry from incomplete 3D observations by leveraging image-guided priors from a single-view RGB image (Zeng et al., 17 Sep 2025).

The reported setting is object-level rather than scene-level. In ShapeNet-ViPC, partial point clouds are generated from complete meshes via occlusion under 24 rendered viewpoints per object, and both partial and complete point clouds contain 2048 points, while images are $224 \times 224$ (Zeng et al., 17 Sep 2025). In YCB-Complete, partial point clouds are captured with a RealSense sensor and exhibit real-world noise, occlusions, and pose misalignment (Zeng et al., 17 Sep 2025). This distinction is significant because it places HGACNet between synthetic benchmark evaluation and robot-facing deployment conditions.

The paper situates HGACNet against both geometry-only and cross-modal completion methods. Compared to geometry-only methods such as PCN, FoldingNet, TopNet, GRNet, PoinTr, and SnowflakeNet, HGACNet leverages image priors to recover missing structure and details; relative to cross-modal baselines such as ViPC and CSDN, it uses deep hierarchical attention rather than shallow fusion through concatenation or style modulation (Zeng et al., 17 Sep 2025). A plausible implication is that the architecture is intended to address both semantic ambiguity in severely occluded regions and structural incompleteness in sparse point observations.

2. Architectural organization

The overall pipeline takes $P_p$ and $I$, encodes geometry using the HGA encoder to produce global features $F_g$ and local features $F_l$, encodes the image with a pre-trained, frozen Swin Transformer to obtain $F_I$, performs multi-scale fusion in the MSCF module, and decodes the fused representation to synthesize $P_c$ (Zeng et al., 17 Sep 2025).

The feature hierarchy is explicit. The paper defines $I \in \mathbb{R}^{H \times W \times 3}$0 and $I \in \mathbb{R}^{H \times W \times 3}$1, where $I \in \mathbb{R}^{H \times W \times 3}$2 and $I \in \mathbb{R}^{H \times W \times 3}$3 are the numbers of selected key points at different hierarchy levels and $I \in \mathbb{R}^{H \times W \times 3}$4 are their feature dimensions (Zeng et al., 17 Sep 2025). In the implementation, the point set is reduced hierarchically to 512 and then 128 points, yielding local and global hierarchies (Zeng et al., 17 Sep 2025). Image features $I \in \mathbb{R}^{H \times W \times 3}$5 are extracted from Swin Transformer tokens at multiple scales (Zeng et al., 17 Sep 2025).

The architecture is modular:

Module	Role
HGA encoder	Constructs local graphs, computes descriptors and node scores, performs attention-based downsampling, refines hierarchical features
MSCF	Projects geometric and visual features into a shared latent space; applies self-attention and cross-attention across levels and modalities
Decoder	Predicts dense points for missing regions and concatenates them with FPS-downsampled observed points
C-Loss	Aligns matched image–point cloud pairs and separates non-matching pairs

The decoder follows XMFNet and transforms the fused representation directly into 3D coordinates of the completed point cloud by predicting a dense set of points approximating missing regions (Zeng et al., 17 Sep 2025). The final output is formed by concatenating these predicted points with a subsampled version of the input partial point cloud via farthest point sampling (FPS), thereby preserving observed geometry while completing missing surfaces (Zeng et al., 17 Sep 2025). In the reported benchmarks, the outputs match the ground-truth size of 2048 points (Zeng et al., 17 Sep 2025).

3. Hierarchical Graph Attention encoder

The HGA encoder is the principal geometric component of HGACNet. It hierarchically encodes 3D geometric features and adaptively selects critical local points through graph attention-based downsampling, with the stated purpose of better capturing structural continuity and spatial relationships (Zeng et al., 17 Sep 2025).

Its first stage is the Graph Descriptor (GD). For each point $I \in \mathbb{R}^{H \times W \times 3}$6 in $I \in \mathbb{R}^{H \times W \times 3}$7, a local graph is constructed by connecting to $I \in \mathbb{R}^{H \times W \times 3}$8 nearest neighbors using k-NN (Zeng et al., 17 Sep 2025). Raw point coordinates are lifted by a shared MLP to point-wise features $I \in \mathbb{R}^{H \times W \times 3}$9 (Zeng et al., 17 Sep 2025). Edge features between $P_c \in \mathbb{R}^{N_c \times 3}$0 and a neighbor $P_c \in \mathbb{R}^{N_c \times 3}$1 are defined as

$P_c \in \mathbb{R}^{N_c \times 3}$2

where $P_c \in \mathbb{R}^{N_c \times 3}$3 and $P_c \in \mathbb{R}^{N_c \times 3}$4 are the features of $P_c \in \mathbb{R}^{N_c \times 3}$5 and $P_c \in \mathbb{R}^{N_c \times 3}$6, and $P_c \in \mathbb{R}^{N_c \times 3}$7 is a learnable function (Zeng et al., 17 Sep 2025). These encoded edges are processed by a 2D convolution, Group Normalization (GroupNorm), and LeakyReLU, followed by Max Pooling to aggregate local neighborhoods into compact point embeddings (Zeng et al., 17 Sep 2025).

The second stage is Graph Attention Downsampling (GAD). Each node receives a learnable importance score

$P_c \in \mathbb{R}^{N_c \times 3}$8

with $P_c \in \mathbb{R}^{N_c \times 3}$9 a learnable scoring function, and the highest-scoring points are selected (Zeng et al., 17 Sep 2025). Downsampling is hierarchical: the point set is first reduced to 512 points and then to 128 points, producing local and global keypoint sets (Zeng et al., 17 Sep 2025). Positional encoding is concatenated to selected point features through

$224 \times 224$0

where $224 \times 224$1 is a sinusoidal positional encoding and $224 \times 224$2 denotes channel-wise concatenation (Zeng et al., 17 Sep 2025). Independent projection modules then map features from different hierarchy levels to a common space, with $224 \times 224$3 intended to capture fine-grained geometric details and $224 \times 224$4 higher-level semantic structures (Zeng et al., 17 Sep 2025).

The paper explicitly notes that graph attention in HGA is realized as node-wise importance scoring for downsampling rather than vanilla edge-attention with $224 \times 224$5 (Zeng et al., 17 Sep 2025). Residual connections and explicit multi-head attention are not reported; normalization is performed through GroupNorm and activation through LeakyReLU (Zeng et al., 17 Sep 2025). This is important for terminological precision, because the “attention” mechanism here refers primarily to adaptive keypoint selection rather than a canonical transformer-style graph attention operator.

4. Multi-Scale Cross-Modal Fusion and alignment

The MSCF module performs attention-based feature alignment between hierarchical geometric features and structured visual representations, enabling fine-grained semantic guidance for completion (Zeng et al., 17 Sep 2025). Its visual branch uses a pre-trained and frozen Swin Transformer, whose window-based and shifted window self-attention capture both fine textures and global context (Zeng et al., 17 Sep 2025).

Before fusion, geometric and visual features are projected into a shared latent space:

$224 \times 224$6

where $224 \times 224$7 are learnable linear transformations (Zeng et al., 17 Sep 2025). The fusion process then applies several attention operations. Self-attention within $224 \times 224$8 and $224 \times 224$9 models intra-modal dependencies and produces $P_p$0 and $P_p$1; cross-attention between $P_p$2 and $P_p$3 refines global and local features mutually, producing $P_p$4; and cross-modal attention between point cloud features $P_p$5 and image features $P_p$6 produces $P_p$7 and $P_p$8 (Zeng et al., 17 Sep 2025). The attention operator is given by

$P_p$9

where $I$0, $I$1, and $I$2 are learned projections and $I$3 is the feature dimension (Zeng et al., 17 Sep 2025).

The fused set $I$4 is concatenated and sent to the decoder (Zeng et al., 17 Sep 2025). Empirically, one self-attention and one cross-attention layer per scale are reported to balance expressiveness and efficiency; stacking more layers yields marginal accuracy gains but large inference-time costs (Zeng et al., 17 Sep 2025). Attention visualizations show that global features attend to overall object structure while local features attend to fine details such as edges and slender parts (Zeng et al., 17 Sep 2025).

This design places HGACNet within the broader class of cross-modal completion systems that use 2D priors to regularize 3D reconstruction, but its defining feature is the selective use of attention across hierarchical levels rather than a flat or shallow fusion strategy (Zeng et al., 17 Sep 2025). The paper contrasts this with ViPC and CSDN, describing their fusion as shallower (Zeng et al., 17 Sep 2025). This suggests that the contribution is not only multimodal conditioning but also the coupling of scale hierarchy with modality hierarchy.

5. Objective functions and training protocol

HGACNet is trained with a combination of reconstruction loss and cross-modal contrastive alignment (Zeng et al., 17 Sep 2025). The reconstruction objective is the L2-Chamfer Distance between predicted and ground-truth point sets:

$I$5

This is the primary geometric supervision term (Zeng et al., 17 Sep 2025).

To reduce modality discrepancy, the paper introduces a batch-symmetric InfoNCE-based contrastive loss aligning point cloud global features and paired image features:

$I$6

where $I$7 is cosine similarity and $I$8 is a temperature (Zeng et al., 17 Sep 2025). The total loss is

$I$9

with $F_g$0 and $F_g$1 (Zeng et al., 17 Sep 2025).

Implementation details reported in the paper are specific. The framework is implemented in PyTorch, the visual backbone is a pre-trained and frozen Swin Transformer, and the point cloud encoder uses GD with shared MLP, 2D convolution, GroupNorm, LeakyReLU, and MaxPool, followed by GAD with learnable scoring $F_g$2 and sinusoidal positional encoding $F_g$3 (Zeng et al., 17 Sep 2025). Training is class-specific, uses the Adam optimizer, runs for 400 epochs with batch size 32, and starts from an initial learning rate of 0.1, reduced by a factor of $F_g$4 at epochs 50, 80, 120, and 200 (Zeng et al., 17 Sep 2025). Hardware is an Nvidia RTX 4090 GPU (Zeng et al., 17 Sep 2025). Augmentation, parameter counts, and runtime are not reported, although the authors note that adding attention layers increases inference time significantly (Zeng et al., 17 Sep 2025).

6. Benchmarks and empirical performance

HGACNet is evaluated on ShapeNet-ViPC and YCB-Complete using L2-CD and F-Score at threshold $F_g$5 (Zeng et al., 17 Sep 2025). ShapeNet-ViPC is derived from ShapeNet and contains 38,328 objects across 13 categories, with evaluation conducted on 8 categories: airplane, cabinet, car, chair, lamp, sofa/couch, table, and watercraft (Zeng et al., 17 Sep 2025). The train/test split is 80/20% over selected categories, totaling 31,650 training objects (Zeng et al., 17 Sep 2025). YCB-Complete is reorganized from YCB-Video and contains 21 object categories with approximately 3,000 samples per category; it uses 17 known categories and 3 unknown categories—004_sugar_box, 007_tuna_fish_can, and 052_extra_large_clamp—to test category-level generalization (Zeng et al., 17 Sep 2025).

On ShapeNet-ViPC, HGACNet achieves an average CD of $F_g$6, compared with EGIINet at $F_g$7 and XMFNet at $F_g$8 (Zeng et al., 17 Sep 2025). The paper states that this corresponds to a 17% relative reduction versus EGIINet and 31% versus XMFNet (Zeng et al., 17 Sep 2025). The average [email protected] is 0.887, exceeding EGIINet at 0.836 and XMFNet at 0.796 (Zeng et al., 17 Sep 2025). Reported category-wise CD values for HGACNet are airplane 0.377, cabinet 1.458, car 1.340, chair 1.028, lamp 0.981, sofa 1.105, table 1.074, and watercraft 0.597, all $F_g$9 (Zeng et al., 17 Sep 2025). The corresponding [email protected] values are airplane 0.983, cabinet 0.789, car 0.805, chair 0.895, lamp 0.896, sofa 0.879, table 0.894, and watercraft 0.954 (Zeng et al., 17 Sep 2025).

On YCB-Complete, for known categories HGACNet reports $F_l$0 CD and 0.995 F-score, outperforming XMFNet at $F_l$1 and 0.822, and EGIINet at $F_l$2 and 0.851 (Zeng et al., 17 Sep 2025). For unknown categories, HGACNet reports $F_l$3 CD and 0.405 F-score, compared with EGIINet at $F_l$4 and 0.300, and XMFNet at $F_l$5 with 0.731 F-score reported higher but with much higher CD (Zeng et al., 17 Sep 2025). The paper states that these results reflect dataset diversity and difficulty (Zeng et al., 17 Sep 2025).

Qualitative results are used to support the quantitative findings. HGACNet is reported to preserve thin and structurally complex regions such as airplane tail wings, cabinet drawers, slender table legs, watercraft detailing, pitcher base handle, mug handles, and power drill edges more faithfully than baselines, including under noise and occlusion (Zeng et al., 17 Sep 2025). This suggests that the combination of hierarchical geometry encoding and image-guided fusion particularly benefits completion of narrow or semantically constrained parts.

7. Ablation findings, limitations, and nomenclature

The ablation study isolates the contributions of local feature extraction, MSCF, C-Loss, and image input (Zeng et al., 17 Sep 2025). For selected ShapeNet-ViPC categories, full HGACNet reports CD values of airplane 0.377, cabinet 1.458, car 1.340, and watercraft 0.653, all $F_l$6 (Zeng et al., 17 Sep 2025). Removing local feature extraction degrades performance to 0.546, 1.711, 1.533, and 0.800, respectively; removing MSCF degrades performance to 0.632, 1.764, 1.619, and 0.834; removing C-Loss yields 0.512, 1.634, 1.439, and 0.793; and removing image input yields 0.471, 1.751, 1.472, and 0.744 (Zeng et al., 17 Sep 2025). The paper interprets these results as showing that local features are crucial for fine detail, attention-based cross-modal fusion is necessary, contrastive alignment significantly reduces the modality gap, and visual priors help infer unobserved geometry (Zeng et al., 17 Sep 2025).

The paper identifies several limitations. Performance decreases on unknown categories in YCB-Complete, which the authors attribute to limited intra-category variation and fewer training classes (Zeng et al., 17 Sep 2025). Additional attention layers improve accuracy only slightly while increasing inference time (Zeng et al., 17 Sep 2025). Computational complexity, parameter counts, and exact runtime are not reported (Zeng et al., 17 Sep 2025). The authors nonetheless argue that selective, hierarchical attention is more efficient than stacking many attention blocks (Zeng et al., 17 Sep 2025). A plausible implication is that the design prioritizes representational selectivity over depth of attention.

A potential source of confusion arises from the superficially similar name HCGNet, introduced in “Gated Convolutional Networks with Hybrid Connectivity for Image Classification” (Yang et al., 2019). That model is explicitly named HCGNet, not HGACNet, and the term “HGACNet” does not appear in the paper; it is described there as a misnomer rather than a distinct model (Yang et al., 2019). HCGNet is an image classification backbone based on Hybrid Connectivity and Gated mechanisms, whereas HGACNet is a cross-modal point cloud completion framework built around hierarchical graph attention, multi-scale cross-modal fusion, and contrastive alignment (Zeng et al., 17 Sep 2025, Yang et al., 2019). The two models are therefore unrelated in task formulation, modality structure, and architectural objective despite the similarity of their acronyms.

From a robotics perspective, HGACNet is presented as applicable to grasping, pose estimation, and motion planning because it reconstructs complete, structurally consistent object point clouds with semantic guidance from images (Zeng et al., 17 Sep 2025). The paper does not report a dedicated manipulation benchmark such as grasp success rate, but its YCB-Complete results and qualitative examples are presented as evidence of robustness to noise and occlusion in real-world conditions (Zeng et al., 17 Sep 2025).