---
title: Patch-Grid Point Sampling Methods & Applications
url: https://www.emergentmind.com/topics/patch-grid-pg-point-sampling
type: topic
---

# Patch-Grid Point Sampling Methods & Applications

Patch-Grid (PG) point sampling denotes a family of structured sampling, localization, and query schemes in which points, pixels, or dense prediction coordinates are organized by a regular grid, a local patch decomposition, or both. The literature does not use the label in a single canonical way. In image replay, “Grid-based Patch Sampling (GPS)” partitions an image into a uniform grid of non-overlapping square patches and samples one pixel from each patch [2504.10409]. In neural implicit surface modeling, “Patch-Grid” localizes representation and supervision to patch volumes \(V_p\) and merge-grid cells \(C_i\) [2308.13934]. In dense Vision Transformer inference, Phase Marginalization samples discrete patch-grid phases \(\phi=(d_x,d_y)\) of the patch lattice rather than arbitrary image points [2606.08132]. In point-cloud segmentation, by contrast, CPGNet uses “Point-Grid (PG),” not Patch-Grid, and its closest sampling operations are point-to-grid pooling and grid-to-point bilinear interpolation [2204.09914].

## 1. Terminology and conceptual scope

The term “Patch-Grid (PG) point sampling” sits across several adjacent but non-identical literatures. In the most literal image-space sense, GPS is a grid-wise point sampling method over patches: one representative pixel per patch, arranged in original spatial order as a low-resolution surrogate image [2504.10409]. In the most literal 3D shape sense, Patch-Grid organizes point sampling and point querying through two nested spatial structures: patch feature volumes for local signed-distance representation and a merge-grid octree for local composition of multiple patches [2308.13934]. In ViT dense prediction, the sampled object is not a point in the continuous image plane but a structured patch-grid phase, namely the origin or offset of the tokenization lattice [2606.08132].

This terminological variation matters because superficially similar acronyms are not equivalent. CPGNet explicitly defines PG as a Point-Grid fusion mechanism, not as a standalone sampling algorithm [2204.09914]. HybridPoint is partially analogous to PG sampling because it mixes salient points and approximately uniform coverage points, but it does not define an explicit patch-grid lattice [2303.16526]. GMS-3DQA is analogous in mechanism but samples 2D projection mini-patches from a regular image grid, not 3D points from native geometry [2306.05658]. One document sometimes associated with sampler comparison contains no proposed sampler, no sampling equations, and no experiments; its relevance is limited to the bibliography it carries [2305.14306].

A plausible implication is that PG point sampling is better treated as a structured design pattern than as a single algorithm. Across the cited work, the recurring pattern is local regularization of otherwise irregular data: patch-local support, cell-local assignment, or finite discrete offsets of a patch lattice.

## 2. Image-space patch-grid sampling

The clearest explicit formulation of patch-grid point sampling in image space appears in GPS. For an input image \(x \in \mathbb{R}^{r \times r \times C}\) and sampling factor \(f \in \mathbb{N}\), the image is partitioned into an \(r' \times r'\) grid of patches with
\[
r' = \left\lfloor \frac{r}{f} \right\rfloor .
\]
From each patch \(P_{i,j}\), a single pixel is sampled uniformly,
\[
x_{i,j} = P_{i,j}(u,v), \qquad u,v \sim \mathcal{U}(0,f-1),
\]
and the sampled outputs are assembled into a compact image \(x^{\mathrm{GPS}} \in \mathbb{R}^{r' \times r' \times C}\) that preserves patch-grid arrangement [2504.10409]. Under a fixed pixel budget, a memory that stores \(K\) full-resolution images can store \(Kf^2\) GPS-compressed samples. During training replay, same-class GPS samples are grouped in chunks of size \(f^2\) and tiled into an \(f \times f\) grid to form a synthetic full-resolution image. During inference in the SCR setup, GPS samples are upsampled by pixel repetition and used with a Nearest Class Mean classifier [2504.10409].

Phase Marginalization uses a different image-space PG object: the phase of a ViT patch grid. For patch size \(P\), the phase is
\[
\phi=(d_x,d_y), \qquad d_x,d_y\in\{0,\ldots,P-1\}.
\]
A frozen encoder \(f_\theta\) and dense head \(h\) are run on a phase-shifted input \(\mathcal{S}_\phi(x)\), producing
\[
z_\phi = h(f_\theta(\mathcal{S}_\phi(x))).
\]
Outputs are then inverse-aligned and aggregated by
\[
\bar{z}(p)=\frac{1}{K}\sum_{k=1}^K \mathcal{A}^{-1}_{\phi_k}(z_{\phi_k})(p).
\]
The central variant is Uniform Phase Marginalization with \(K=4\), using
\[
\Phi_4=\{(0,0),(0,P/2),(P/2,0),(P/2,P/2)\}
\]
for even patch size \(P\) [2606.08132]. This is explicitly training-free and post-hoc. The paper frames patch-grid phase as a nuisance variable because changing the patch partition changes the token evidence available to a pixel, especially near boundaries [2606.08132].

GMS-3DQA provides a related projection-space formulation. Each rendered projection \(\mathcal P_k\) is divided into a uniform \(\mathcal L \times \mathcal L\) grid,
\[
\mathcal{G}_{k} = \{g_{0,0}^{k}, \cdots, g_{\mathcal L-1,\mathcal L-1}^{k}\},
\]
with \(\mathcal L = 7\) in experiments. Candidate mini-patches \(\mathcal{MP}_{i,j}^{k}\) are derived from each grid cell, and a fixed number
\[
N_\alpha = \left\lfloor \frac{\mathcal L^2}{6} \right\rfloor
\]
is randomly selected from each of six projections. For \(\mathcal L=7\), \(N_\alpha=8\), so a total of \(49\) mini-patches are spliced into a \(7 \times 7\) Quality Mini-Patch Map [2306.05658]. The method is structurally analogous to PG sampling because it uses regular partition, local patch candidates, sparse balanced selection, and compact aggregation, but it samples 2D projection patches rather than 3D points [2306.05658].

## 3. Point-cloud and point-grid formulations

In point-cloud segmentation, CPGNet defines a Point-Grid fusion block with four steps: Point to Grid (P2G), 2D FCN, Grid to Point (G2P), and Point Fusion [2204.09914]. For a 3D point \(\mathbf{p}_k^{3D}=(x_k,y_k,z_k)\) projected to \(\mathbf{p}_k^{2D}=(u_k,v_k)\), the set of points assigned to grid cell \((h,w)\) is
\[
\mathcal{R}_{h,w}=\{k \mid \lfloor u_k \rfloor = h,\; \lfloor v_k \rfloor = w \},
\]
and the grid feature is obtained by channel-wise max pooling,
\[
\mathcal{G}^{2D}_{h,w,c} = \max_{k \in \mathcal{R}_{h,w}} \mathcal{F}^{3D}_{k,c}.
\]
After 2D semantic extraction on BEV and RV grids, grid features are transmitted back to points by bilinear interpolation,
\[
\mathcal{F}^{3D}_{k,c} = \sum_{i=0}^{1}\sum_{j=0}^{1} w_{i,j,k}\, \mathcal{G}^{2D}_{\lfloor u_k \rfloor+i,\; \lfloor v_k \rfloor+j,\; c}.
\]
This is the operation in the paper closest to “grid/patch sampling onto points”: a fixed \(2\times2\) local neighborhood lookup around the projected point location [2204.09914].

GridFormer pushes the point-grid formulation further by making within-cell aggregation learned rather than purely pooled. Points are localized relative to a regular plane or volume grid using
\[
f_{pos} = \phi_{pos}(p - \lfloor (p \times r) \rfloor / r),
\]
where \(r\) is the grid resolution [2401.02292]. For each grid location \(g_i\) and point \(p_j \in \mathcal{N}_{g_i}\), point-grid attention weights are
\[
\omega_{ij} = \phi_w(\psi_k(f_{g_i}) - \phi_q(f_{p_j}) + f_{pos}),
\]
and the updated grid feature is
\[
f_{g_i}=\sum_{p_j \in \mathcal{N}_{g_i}} \omega_{ij} (\phi_v(f_{p_j}) + f_{pos}).
\]
Point features are then updated by combining transformed point features with interpolated grid features [2401.02292]. The paper describes the grid as a transfer point connecting space and point cloud, and this makes the grid an active mediator rather than a passive index [2401.02292].

HybridPoint occupies a different position in the design space. Salient points are selected from an intermediate point set \(P_2\) using covariance eigenvalue ratios,
\[
\lambda_2/\lambda_1 \le \gamma_1,\qquad \lambda_3/\lambda_2 \le \gamma_2,
\]
with \(\gamma_1=\gamma_2=0.6\), followed by Non-Maximum Suppression [2303.16526]. Non-salient points are taken from the downsampled set \(P_3\), which the paper states “can be approximately regarded as uniformly distributed,” and a point \(p_i \in P_3\) is added if its distance to the salient set satisfies
\[
d = \min \|p_i - p_l^{sap}\| > \sigma.
\]
The hybrid node set is
\[
P^H = P^{sap} \cup P^{nsap}.
\]
This is not explicit grid partitioning, but it is a coverage-completion mechanism with PG-like intent: repeatable centers where geometry is distinctive, and approximately uniform support where it is not [2303.16526].

## 4. Patch-Grid for neural implicit surfaces

“Patch-Grid: An Efficient and Feature-Preserving Neural Implicit Surface Representation” provides the most direct 3D shape formulation of PG point sampling [2308.13934]. The method starts from a segmented shape
\[
\mathcal{S}=\bigcup_{p=1}^K S_p,
\]
where each \(S_p\) is a surface patch. For each patch \(S_p\), a patch volume \(V_p\) is built by computing the axis-aligned bounding box, partitioning it into a regular grid of rectangular or cubic cells, pruning cells that do not enclose any part of \(S_p\), and keeping only nonempty cells [2308.13934]. Grid vertices carry feature vectors
\[
FV_p=\{\mathbf{f}_{(i,j,k)}^p\}, \qquad \mathbf{f}_{(i,j,k)}^p\in\mathbb{R}^D,
\]
and a continuous feature field \(F_p(\mathbf{x})\) is defined by trilinear interpolation inside \(V_p\). The patch surface is represented as
\[
S_p \subset Z_p=\{\mathbf{x}\mid f(F_p(\mathbf{x}))=0\}.
\]

A second spatial structure, the merge grid \(\mathcal{G}\), is an adaptive octree initialized at the finest patch-volume resolution. Each leaf cell \(C_i\) is intended to contain a simple local patch adjacency configuration, and subdivision stops when the patch adjacency graph in the cell becomes a clique, or the cell is empty, or maximum depth is reached [2308.13934]. This makes point-related operations local in two senses: patch-local for individual SDFs and cell-local for multi-patch merging.

Training-time point sampling is explicitly heterogeneous. Patch losses use surface points \(\mathbf{x}\in S_p\), offset near-surface points \(\mathbf{x}+d\mathbf{n}\) with \(d\in(0,0.1)\) times the grid-cell size, and off-surface volumetric points \(\Omega_p\subset V_p\) [2308.13934]. The paper states that, in each iteration, it globally samples 10,000 surface points and 10,000 spatial points off the surface and assigns the sampled points to their corresponding cells. Merge supervision is local:
\[
L^i_{\mathrm{merge}}= \sum_{\mathbf{x}\in \cup_p \{S_p\}\cap C_i} |M(\mathbf{x})|,
\]
where \(\mathbf{x}\) are sampled from the surface patches within the cell \(C_i\) [2308.13934]. For open surfaces, boundary points \(\mathbf{x}_b\in\partial S_o\) are sampled from the boundary curve and used with a trimming patch [2308.13934].

Inference-time point querying is likewise localized. A query point is influenced only by patches whose patch volumes contain it and by patches participating in the same merge-grid leaf cell. The intended logic is: locate \(\mathbf{x}\) in a merge-grid leaf cell \(C_i\), retrieve the set of patches present in \(C_i\), evaluate \(f(F_p(\mathbf{x}))\) for relevant patches, and merge the results using a local CSG tree with hard \(\min/\max\) operators [2308.13934]. This local support is the central point-query design of Patch-Grid.

## 5. Algorithmic motifs and computational properties

Across formulations, three algorithmic motifs recur. The first is structured coverage. GPS guarantees one sampled pixel from every \(f\times f\) image patch [2504.10409]. GMS-3DQA chooses a balanced number of grid mini-patches from each of six views and forms a single Quality Mini-Patch Map [2306.05658]. HybridPoint adds approximately uniform non-salient nodes to salient points so that planar overlap is not discarded [2303.16526].

The second is local reconstruction or inverse alignment after structured sampling. In Phase Marginalization, outputs produced under shifted patch lattices are inverse-aligned by reshaping patch outputs to the raster patch grid, bilinearly upsampling to padded image resolution with `align-corners=false`, cropping by the inverse phase offset, and recovering the original \(H\times W\) dense field before averaging logits [2606.08132]. In CPGNet, grid features are resampled back to points by bilinear interpolation from the four neighboring cells [2204.09914]. In GridFormer, bilinear or trilinear interpolation transfers features from learned grids back to points and query coordinates [2401.02292].

The third is locality-driven reduction of search complexity. Patch-Grid localizes point evaluation first to a patch volume and then to a merge-grid leaf cell, replacing global multi-patch reasoning with cell-local composition [2308.13934]. A conceptually related variant appears in particle locating on unstructured meshes: an auxiliary Cartesian grid maps a query point to a vertex patch or edge patch, and the host element is then found by an angular search within that patch domain [2408.07959]. In 2D, after Cartesian indexing in \(O(1)\), the host element is found by binary search over a sorted fan of incident edges at a vertex, with search cost
\[
O\!\left(\log\frac{2\pi}{\alpha}\right),
\]
and in 3D a same-element radial move is used when necessary before edge-patch angular localization [2408.07959]. The paper emphasizes that only particle coordinates are required at query time [2408.07959].

The computational profiles reflect these design choices. GPS is intended to be lightweight and model-free, avoiding bi-level optimization and storing \(Kf^2\) compressed samples under the same pixel budget [2504.10409]. Uniform Phase Marginalization with \(K=4\) improves over the canonical \(K=1\) baseline across measured segmentation, depth, and local matching settings, while \(K=8\) is essentially unchanged from \(K=4\) and \(K=16\) adds little accuracy at much higher latency [2606.08132]. CPGNet reports that semantic extraction happens mainly on 2D projected grids; the abstract states that it is comparable with RPVNet while running 4.7 times faster without ensemble models or TTA [2204.09914].

## 6. Applications, limitations, and common misconceptions

Patch-grid sampling has been used for distinct technical aims. GPS targets online class-incremental learning under memory constraints and reports 3%–4% improvements in average end accuracy under memory-constrained settings [2504.10409]. Phase Marginalization targets dense ViT prediction and reports a compute-matched advantage of \(+0.31\) mIoU over the strongest tested generic four-forward shift-based TTA row on Cityscapes [2606.08132]. Patch-Grid targets neural implicit reconstruction of sharp features, open surfaces, and thin structures through localized patch volumes and merge-grid composition [2308.13934]. CPGNet targets real-time LiDAR semantic segmentation by replacing 3D neighborhood search and sparse 3D convolution with point-grid projection, 2D FCNs, and bilinear back-sampling [2204.09914]. GMS-3DQA targets 3D model quality assessment through multi-projection grid mini-patch sampling and compact aggregation into one QMM [2306.05658].

Several misconceptions recur. First, PG does not always mean Patch-Grid. In CPGNet it means Point-Grid fusion, and the relevant operations are P2G pooling and G2P interpolation rather than a named PG point sampler [2204.09914]. Second, patch-grid sampling does not always mean arbitrary point selection. In Phase Marginalization, the sampled object is the discrete set of patch-grid phases, not free-form image points [2606.08132]. Third, patch-grid methods are not always intrinsically 3D. GMS-3DQA samples 2D projection patches, and GPS samples one pixel per image patch [2306.05658; 2504.10409]. Fourth, not every ostensibly related document contains a sampler; one cited item is an ICCV template with placeholder prose and no proposed method [2305.14306].

Limitations are correspondingly heterogeneous. GPS degrades when compression is too aggressive; the \(f\)-ablation shows \(f=2\) is best, while \(f=4\) and \(f=7\) collapse performance on Mini-ImageNet [2504.10409]. Phase Marginalization is limited to discrete lattice offsets, cost scales approximately linearly with \(K\), and the benefits are modest rather than dramatic [2606.08132]. GridFormer notes that when points are too sparse, learning the weight for a single point is meaningless, so learned within-cell weighting has less benefit [2401.02292]. HybridPoint shows that non-salient nodes improve coverage but are intrinsically noisier and therefore require class-aware filtering [2303.16526]. Patch-Grid for neural implicits does not define a classic balanced inside/outside sampler; sign information is induced by surface zero supervision, pseudo-SDF offset samples, Eikonal regularization, and local patch support [2308.13934].

Taken together, these works suggest that PG point sampling is best understood as a family of locality-preserving regularization strategies. The common objective is not merely sparsification, but controlled restriction of sampling or querying to structured local support—grid cells, surface patches, patch phases, or patch-conditioned search domains—so that coverage, locality, and computational efficiency can be improved without abandoning fine-grained spatial structure.

Source: https://www.emergentmind.com/topics/patch-grid-pg-point-sampling