Contact-Aware Part-Guided Pooling (CPPool)

• CPPool is a contact-aware, part-guided pooling module that reallocates representational capacity toward interaction-critical regions in LiDAR-based 3D human pose estimation.
• It addresses class imbalance by combining part scores, contact probabilities, and contextual cues to preserve sparse yet informative features like hands and feet.
• Integrated into the PTv3 encoder of HOIL, CPPool reshapes feature hierarchies, leading to improved MPJPE and PCK metrics in challenging human–object interaction scenarios.

Contact-Aware Part-Guided Pooling (CPPool) is a differentiable pooling mechanism introduced within the HOIL framework for 3D human pose estimation from LiDAR point clouds under human–object interaction. It replaces the standard max-pooling, or Grid Pooling, inside the Point Transformer V3 (PTv3) encoder and learns to reallocate representational capacity toward interacting body parts and contact regions while compressing overrepresented points. In the formulation reported for HOIL, CPPool directly addresses the severe class imbalance between densely sampled regions such as torso or background and sparsely observed frequently interacting regions such as hands and feet, and it does so by combining part probabilities, contact probabilities, and contextual importance into a voxel-local softmax weighting scheme (Jung et al., 17 Mar 2026).

1. Architectural placement and problem setting

CPPool is one of the two core technical contributions of HOIL, alongside Human-Object Interaction-aware Contrastive Learning (HOICL), for robust 3D human pose estimation from LiDAR point clouds in the wild, especially when pedestrians interact with objects such as bikes, chairs, and umbrellas (Jung et al., 17 Mar 2026). The reported motivation is twofold: human–object interactions introduce spatial ambiguity between human and object points, and they also induce severe class imbalance in point density between interaction-heavy parts and other regions. CPPool is the component aimed at the second problem.

Within PTv3, CPPool acts early and repeatedly in the encoder. At encoder stage $i$, vanilla PTv3 would subsample coordinates and features by Grid Pooling with max aggregation per voxel. HOIL replaces that operation with CPPool: $$(\mathbf{P}^{(i)}, \mathbf{H}^{(i)}) = f_{\mathrm{pool}}\big(\mathbf{P}^{(i-1)}, \mathbf{F}_{\mathrm{p,enc}}^{(i-1)}\big),$$ where $f_{\mathrm{pool}}$ is CPPool rather than simple max pooling (Jung et al., 17 Mar 2026). The downstream architecture then serializes the pooled coordinates and features and feeds them into the PTv3 Transformer block, so the pooling decision affects all later representations, including HOICL, the cross-attention keypoint decoder, and the optional contact-based temporal refinement.

This placement distinguishes CPPool from late-stage part aggregation mechanisms. A plausible implication is that CPPool is intended not merely to refine final predictions, but to reshape the feature hierarchy itself so that sparse interaction-critical evidence survives repeated downsampling. That interpretation is consistent with the claim that CPPool “directly shap[es] how point features are downsampled and therefore what information is available to all later components” (Jung et al., 17 Mar 2026).

2. Motivation: class imbalance, sparsity, and the failure of uniform pooling

The central empirical premise behind CPPool is that LiDAR sampling over the human body is highly non-uniform. The reported pattern is that large body regions such as torso, legs, and background surfaces may have many points, whereas frequently interacting regions (FIR) such as hands and feet are small and far from the sensor and therefore receive few points (Jung et al., 17 Mar 2026). The same source states that hands and feet have an order-of-magnitude fewer points than other parts, even on a log scale, and that errors of the SOTA baseline DAPT concentrate at wrists and ankles.

Under standard Grid Pooling, points within a voxel cell are aggregated uniformly with respect to semantic importance. The stated failure mode is that a cell containing many torso or background points and only a few hand points will often let the many non-interacting points dominate the pooled representation. Max pooling further induces a “winner takes all” behavior that is insensitive to the fact that a few FIR or contact points may be crucial for pose estimation in interaction regions (Jung et al., 17 Mar 2026).

CPPool is designed to counter this by reallocating representational bandwidth inside each voxel cell. Instead of treating all points equally, it learns a probability distribution over points in each cell, biased toward points that are predicted to belong to important parts, to lie in contact regions, or to be contextually important given global and keypoint-level cues. The stated objective is to preserve informative points from interacting body parts while compressing overrepresented regions (Jung et al., 17 Mar 2026).

This design should be distinguished from more generic part pooling in image-based person re-identification. PCB and Refined Part Pooling (RPP) use uniform horizontal stripes and later soft reassignment on a convolutional feature map to improve within-part consistency, but they remain pose- and contact-agnostic (Sun et al., 2017). By contrast, CPPool is explicitly contact-aware and part-guided, and its target is not retrieval alignment but class imbalance and sparsity in LiDAR-based pose estimation.

3. Mathematical formulation and internal signals

CPPool operates on point features $\mathbf{F}_{\mathrm{p,enc}}^{(i-1)}$ and coordinates $\mathbf{P}^{(i-1)}$ at each encoder stage. For every point, it computes three signals: a part score $s_{\mathrm{part}}$, a contact score $s_{\mathrm{contact}}$, and an importance logit $\ell_{\mathrm{imp}}$ (Jung et al., 17 Mar 2026).

The part score is produced by an auxiliary part segmentation head $f_{\mathrm{part}}$, implemented as an MLP: $$\boldsymbol{\pi}_{\mathrm{part}} = \mathrm{softmax}\big(f_{\mathrm{part}}(\mathbf{F}_{\mathrm{p,enc}}^{(i-1)})\big) \in \mathbb{R}^{N^{(i-1)} \times K}.$$ The part classes correspond to 24 SMPL body parts, 1 object part, plus background. A fixed part weight vector $$(\mathbf{P}^{(i)}, \mathbf{H}^{(i)}) = f_{\mathrm{pool}}\big(\mathbf{P}^{(i-1)}, \mathbf{F}_{\mathrm{p,enc}}^{(i-1)}\big),$$0 assigns larger weights to hands and feet and smaller weights to other parts. The per-point part score is then

$$(\mathbf{P}^{(i)}, \mathbf{H}^{(i)}) = f_{\mathrm{pool}}\big(\mathbf{P}^{(i-1)}, \mathbf{F}_{\mathrm{p,enc}}^{(i-1)}\big),$$1

The contact score is produced by an auxiliary contact head $$(\mathbf{P}^{(i)}, \mathbf{H}^{(i)}) = f_{\mathrm{pool}}\big(\mathbf{P}^{(i-1)}, \mathbf{F}_{\mathrm{p,enc}}^{(i-1)}\big),$$2, also an MLP: $$(\mathbf{P}^{(i)}, \mathbf{H}^{(i)}) = f_{\mathrm{pool}}\big(\mathbf{P}^{(i-1)}, \mathbf{F}_{\mathrm{p,enc}}^{(i-1)}\big),$$3 This quantity estimates the probability that a point lies in a human–object contact region (Jung et al., 17 Mar 2026).

The contextual term $$(\mathbf{P}^{(i)}, \mathbf{H}^{(i)}) = f_{\mathrm{pool}}\big(\mathbf{P}^{(i-1)}, \mathbf{F}_{\mathrm{p,enc}}^{(i-1)}\big),$$4 is computed by another MLP $$(\mathbf{P}^{(i)}, \mathbf{H}^{(i)}) = f_{\mathrm{pool}}\big(\mathbf{P}^{(i-1)}, \mathbf{F}_{\mathrm{p,enc}}^{(i-1)}\big),$$5 from point features together with a global feature $$(\mathbf{P}^{(i)}, \mathbf{H}^{(i)}) = f_{\mathrm{pool}}\big(\mathbf{P}^{(i-1)}, \mathbf{F}_{\mathrm{p,enc}}^{(i-1)}\big),$$6, obtained by average pooling over all point features, and a keypoint feature $$(\mathbf{P}^{(i)}, \mathbf{H}^{(i)}) = f_{\mathrm{pool}}\big(\mathbf{P}^{(i-1)}, \mathbf{F}_{\mathrm{p,enc}}^{(i-1)}\big),$$7, built from the learnable keypoint queries: $$(\mathbf{P}^{(i)}, \mathbf{H}^{(i)}) = f_{\mathrm{pool}}\big(\mathbf{P}^{(i-1)}, \mathbf{F}_{\mathrm{p,enc}}^{(i-1)}\big),$$8

These three quantities are fused into a pooling logit: $$(\mathbf{P}^{(i)}, \mathbf{H}^{(i)}) = f_{\mathrm{pool}}\big(\mathbf{P}^{(i-1)}, \mathbf{F}_{\mathrm{p,enc}}^{(i-1)}\big),$$9 where $f_{\mathrm{pool}}$0 is a temperature and $f_{\mathrm{pool}}$1, $f_{\mathrm{pool}}$2 in the reported experiments (Jung et al., 17 Mar 2026). Within each grid cell $f_{\mathrm{pool}}$3, CPPool applies a cell-wise softmax: $f_{\mathrm{pool}}$4 The weights satisfy $f_{\mathrm{pool}}$5 and $f_{\mathrm{pool}}$6, so each cell is equipped with a learned probability distribution over its points.

Feature aggregation is then a weighted sum after a small projection MLP $f_{\mathrm{pool}}$7: $f_{\mathrm{pool}}$8 The pooled coordinates and features are serialized and processed by the PTv3 Transformer: $f_{\mathrm{pool}}$9

$\mathbf{F}_{\mathrm{p,enc}}^{(i-1)}$0

The mechanism is fully differentiable because it is composed of MLPs, logarithms, softmax, and weighted sums (Jung et al., 17 Mar 2026). A useful interpretation, explicitly supported by the formulation, is that CPPool turns each voxel cell into a semantically informed attention operator whose attention logits are partly supervised by part and contact labels.

4. Reallocation of representational capacity

The main functional claim for CPPool is that it adaptively reallocates representational capacity by compressing overrepresented points while preserving informative points from interacting body parts (Jung et al., 17 Mar 2026). This effect is encoded directly in the softmax weight ratio between any two points $\mathbf{F}_{\mathrm{p,enc}}^{(i-1)}$1 and $\mathbf{F}_{\mathrm{p,enc}}^{(i-1)}$2 in the same cell: $\mathbf{F}_{\mathrm{p,enc}}^{(i-1)}$3 The reported interpretation is that even modest differences in part or contact scores become exponentially amplified in the pooling weights (Jung et al., 17 Mar 2026).

Three distinct priors are fused in this mechanism. The first is part-level grouping through $\mathbf{F}_{\mathrm{p,enc}}^{(i-1)}$4 and $\mathbf{F}_{\mathrm{p,enc}}^{(i-1)}$5, which gives CPPool explicit knowledge of body structure and a fixed bias toward hands and feet as FIR. The second is contact-awareness through $\mathbf{F}_{\mathrm{p,enc}}^{(i-1)}$6, which identifies interaction regions regardless of body part. The third is contextual modulation through $\mathbf{F}_{\mathrm{p,enc}}^{(i-1)}$7, which incorporates global scene context and keypoint-query information (Jung et al., 17 Mar 2026).

The consequence is a shift in effective bandwidth inside each cell. In a dense cell containing many torso or background points and only a few hand or foot points, standard pooling would allow the majority class to dominate. CPPool, by design, can give most of the weight to a small number of FIR or contact points even when they are numerically rare. The source text characterizes this as suppressing redundant non-contact, non-FIR points and amplifying sparsely sampled but semantically critical points (Jung et al., 17 Mar 2026).

This places CPPool in the broader family of content-aware pooling methods, but with an important distinction. RPP for person re-identification also replaces rigid partitioning with soft assignment, yet its assignments are learned from feature similarity and identity labels rather than from explicit contact prediction or SMPL-derived part supervision (Sun et al., 2017). Part-$\mathbf{F}_{\mathrm{p,enc}}^{(i-1)}$8 net, in turn, uses part-aware supervision and RoI-aware point cloud pooling to build proposal-specific structured features for 3D detection, but it does not define a contact-aware pooling mechanism (Shi et al., 2019). CPPool can therefore be situated as a semantically and physically informed pooling module rather than a purely geometry-based or unsupervised attention mechanism.

5. Supervision, optimization, and gradient flow

CPPool introduces auxiliary predictors that are explicitly supervised during pre-training on synthetic LiDAR derived from SMPL and object meshes. The part head $\mathbf{F}_{\mathrm{p,enc}}^{(i-1)}$9 is trained with cross-entropy over $\mathbf{P}^{(i-1)}$0 part classes, and the contact head $\mathbf{P}^{(i-1)}$1 is trained with cross-entropy over $\mathbf{P}^{(i-1)}$2. The CPPool-specific loss is

$\mathbf{P}^{(i-1)}$3

The importance head $\mathbf{P}^{(i-1)}$4 has no explicit loss; it is trained implicitly through the effect of the pooling weights on the downstream objectives (Jung et al., 17 Mar 2026).

The overall pre-training loss reported for HOIL is

$\mathbf{P}^{(i-1)}$5

This formulation makes CPPool not only a forward-path pooling operator but also a mechanism that redistributes gradient signal. Because points with large pooling weights dominate the pooled features $\mathbf{P}^{(i-1)}$6, the corresponding error signal backpropagates strongly through them. The source explicitly states that this balances the effective sample count: even though FIR points are few, they receive disproportionately large gradients, thereby mitigating class imbalance at the feature-learning level (Jung et al., 17 Mar 2026).

During fine-tuning on real LiDAR datasets such as Waymo and SLOPER4D, the module remains active even if supervision for internal part and contact heads is absent or weaker. The reported interpretation is that CPPool carries priors learned during pre-training and continues to be updated through pose and limb losses (Jung et al., 17 Mar 2026). This suggests a hybrid regime in which explicit pre-training supervision establishes the part/contact bias and later task losses preserve or adapt it under domain shift.

6. Quantitative effects and qualitative behavior

The main ablation presented for CPPool on Waymo isolates the effect of part guidance and contact-awareness (Jung et al., 17 Mar 2026).

Part-guided	Contact-aware	MPJPE ↓ / PCK-3 ↑ / PCK-5 ↑
✗	✗	50.78 / 97.03 / 98.79
✓	✗	49.99 / 97.38 / 98.96
✓	✓	48.83 / 98.51 / 99.14

The reported observations are specific. Part guidance alone improves MPJPE by approximately $\mathbf{P}^{(i-1)}$7 from $\mathbf{P}^{(i-1)}$8 to $\mathbf{P}^{(i-1)}$9. Adding contact-awareness gives the largest single step, approximately $s_{\mathrm{part}}$0 improvement over part-only and approximately $s_{\mathrm{part}}$1 relative improvement over the configuration without CPPool. The combined model also pushes PCK-5 above $s_{\mathrm{part}}$2 (Jung et al., 17 Mar 2026).

The qualitative claims focus on interaction-heavy examples involving umbrellas, bicycles, and walls. The reported effect is that HOIL with CPPool preserves reliable features at hands, feet, and contact points, leading to more accurate wrist and ankle keypoints and fewer confusions in which the baseline predicts “standing” instead of “sitting on bike” (Jung et al., 17 Mar 2026). Since the text attributes major baseline errors to wrists and ankles, this qualitative pattern is consistent with the module’s stated purpose.

The broader significance is that the gains are achieved not by adding a late corrective head alone, but by altering multi-stage encoder downsampling. A plausible implication is that the empirical benefit arises from better preservation of scarce evidence throughout the hierarchy rather than only at the prediction layer. That interpretation is consistent with the described role of CPPool in shaping the encoder features at multiple scales (Jung et al., 17 Mar 2026).

7. Relation to prior part-guided pooling and stated limitations

CPPool belongs to a larger line of work on part-aware pooling, but it differs from earlier approaches in both signal source and objective. PCB with RPP in person re-identification operates on 2D convolutional feature maps, partitions them into horizontal stripes, and later learns soft assignments that improve within-part consistency; however, it uses no external pose or part labels and remains contact-agnostic (Sun et al., 2017). Part-$s_{\mathrm{part}}$3 net for LiDAR 3D object detection predicts intra-object part locations from 3D boxes and uses RoI-aware point cloud pooling to construct geometry-specific proposal features, but the paper explicitly notes that it does not contain a module literally called CPPool (Shi et al., 2019). In that taxonomy, CPPool is distinguished by explicit contact prediction, fixed FIR priors through $s_{\mathrm{part}}$4, and repeated use inside the encoder rather than only as proposal-level aggregation.

The cited limitations are also specific. CPPool depends on accurate part segmentation and contact supervision during pre-training, and those labels are derived from synthetic LiDAR based on human and object meshes; therefore, errors in mesh alignment, contact extraction, or domain gap could affect quality (Jung et al., 17 Mar 2026). The fixed weights $s_{\mathrm{part}}$5 encode a handcrafted prior about which parts are frequent interactors, and the text notes that mis-specifying them or transferring them outside the training distribution could be suboptimal. The current design is also described as focusing on a single human–object pair per scene, with multi-person or multi-object interaction likely requiring more complex part and contact modeling.

Potential extensions mentioned or implied include applying similar contact- and part-aware pooling to HOI detection or recognition in LiDAR or RGB-D, human–object reconstruction, affordance prediction, multi-modal settings such as LiDAR+RGB, and other backbone families such as PointMamba (Jung et al., 17 Mar 2026). These are presented as possibilities rather than demonstrated results.

Taken together, CPPool can be understood as a semantically structured alternative to uniform point aggregation. Its defining characteristic is not merely that it pools points by learned importance, but that the learned importance is grounded in explicit body-part priors, explicit contact prediction, and contextual information from the evolving pose representation. In the reported HOIL system, that combination is used to mitigate sparsity and class imbalance precisely where human–object interaction makes 3D pose estimation most fragile (Jung et al., 17 Mar 2026).