Local-to-Global Reconstruction Strategies

• Local-to-global reconstruction is a paradigm that first extracts detailed local structures and then integrates them with global constraints for coherent outputs.
• It is applied across various domains—such as 3D imaging, MRI, CT, SLAM, and anomaly detection—to balance fine-grained detail with overall consistency.
• The effectiveness of this approach stems from combining specialized local modeling with straightforward global fusion techniques, enhancing robustness against occlusions and data gaps.

Local-to-global reconstruction denotes a reconstruction principle in which a model first resolves local structure—patches, neighborhoods, local warps, local priors, or local subspaces—and then combines those local elements with global context, global constraints, or global consistency mechanisms to produce the final output. In the recent literature, the term does not refer to a single architecture. In GTHNA, it is “not a single module, but a pipeline” that builds structure-aware node embeddings, performs memory-guided reconstruction, and scores anomalies with multi-scale reconstruction errors plus memory matching (Li et al., 13 Sep 2025). In hierarchical single-view 3D reconstruction, local patches explain visible parts, larger patches encode more context, and the full image provides a global consistency prior (Bechtold et al., 2021). In scalable 3D CT diffusion, the core idea is to learn the prior of 3D patches while coupling local and global information by modeling the joint distribution of position-aware 3D local patches and downsampled 3D volume as global context (Yang et al., 20 Dec 2025). This suggests a cross-domain abstraction: local evidence supplies detail and specificity, while global mechanisms supply coherence, completion, stability, or regularization.

1. Core definition and recurring design pattern

Across the cited works, local-to-global reconstruction is consistently staged rather than monolithic. A local stage extracts information that is spatially, topologically, or semantically restricted: a $k$-hop subgraph in a graph, an image patch in single-view 3D reconstruction, a semantic finger region in hand reconstruction, or a local patch in a 3D volume. A global stage then injects information that local evidence alone cannot supply: whole-graph context, whole-object shape priors, Fourier-domain long-range coupling, whole-volume context, or a frame-wise transformation. The final reconstruction is obtained by fusion, constrained optimization, averaging, or an explicit global decoder (Li et al., 13 Sep 2025).

A common motivation is that purely global models can memorize training-distribution regularities yet fail on novel combinations, while purely local models can fit details but lose consistency. The 3D reconstruction hierarchy of local and global priors states this explicitly: local predictors dominate where the input provides strong visible evidence, while the global predictor contributes more in invisible or heavily occluded regions where context is needed (Bechtold et al., 2021). In MRI reconstruction, the same division appears in a different form: local image detail is learned in the spatial domain, while long-range structural dependencies are learned through Fourier-domain processing (Yi et al., 2021).

The phrase also spans multiple objective types. In some papers the reconstructed object is a signal or geometry, such as a point cloud, a CT volume, an MR image, or a graph neighborhood. In others, the reconstructed object is a global representation induced from local evidence, as in query-specific global features derived from local retrieval similarities (Aiger et al., 4 Sep 2025), or a frame-wise parametric transformation estimated from pixel-wise local alignments (Chen et al., 2022). The unifying feature is the same: a transition from localized evidence to a globally valid reconstruction.

2. Classical geometric formulations

A pre-neural formulation appears in ground surface reconstruction from terrestrial point clouds, where the pipeline is explicitly local-to-global: raw point cloud $\rightarrow$ local LSQR slopes $\rightarrow$ multilevel hole filling or HRBF $\rightarrow$ PU global smooth surface (Rychkov, 2012). The local model is a grid of best planes by total least squares / LSQR, each determined only from points in its own grid cell. This is the paper’s notion of local control: changing a point affects only nearby fitted slopes, not the whole surface.

The missing-data problem is treated by two strategies. One is hierarchical coarse-to-fine propagation: fit LSQR planes on a fine grid, refit on coarser grids until a sufficiently full grid is obtained, then project those coarse slopes down to finer missing cells and smooth with a $3\times3$ kernel mean filter of slopes. The other is Hermite RBF interpolation, using both function values and gradients,

$$s(x)=p(x)+\sum_{j=1}^{n} c_j\,\phi(\|x-x_j\|)+\sum_{j=1}^{n} d_j\cdot \nabla \phi(x_j-x),$$

with Hardy multiquadrics $\phi(r)=\sqrt{r^2+c^2}$ and a dense linear system for the coefficients (Rychkov, 2012).

The global stage is Partition of Unity blending,

$$g(x,y)=\sum_{i,j} S_{ij}(x,y)\,\phi_i(x)\,\phi_j(y),$$

which stitches local patches into a single surface while preserving local control. The abstract distinguishes two global smoothness regimes: tensor-product cubic B-splines yield a curvature-continuous $C^2$ surface, while compactly supported exponential PU functions yield an infinitely smooth $C^\infty$ surface (Rychkov, 2012). The paper also records a caveat that is conceptually important far beyond terrain modeling: the theoretically smoother blender may not always produce a visually better result when the underlying local models are only crude LSQR planes.

3. Neural local-to-global reconstruction of shapes, point clouds, and images

In single-view 3D reconstruction, the hierarchy of local and global shape priors is formulated as a family of implicit reconstruction networks operating at different locality levels. For an input depth map $\rightarrow$0, square patches $\rightarrow$1 of size $\rightarrow$2 are paired with corresponding 3D regions, and a local reconstruction module $\rightarrow$3 predicts occupancy logits or SDF values for sampled 3D points in that region. The hierarchy is fused by Gaussian-weighted averaging of overlapping local predictions within a level and by averaging corresponding softmax outputs across levels (Bechtold et al., 2021). The reported behavior is sharply local-to-global: local models reconstruct visible areas substantially better than the global network, while the global network can still be competitive or slightly better in invisible regions. HPN is reported to be “more than twice as accurate as the state of the art” on the compositional test set, and Local@64 reaches near-full performance with as little as $\rightarrow$4 of the training data (Bechtold et al., 2021).

LIST uses a different local-to-global decomposition. Global 2D features predict a coarse shape through

$\rightarrow$5

then that coarse prediction is converted into an occupancy grid and a 3D latent volume from which coarse 3D query features $\rightarrow$6 are sampled by multi-scale trilinear interpolation. In parallel, a spatial transformer localizes 3D query points into image-feature coordinates and bilinear interpolation yields local 2D features $\rightarrow$7. The signed distance predictor $\rightarrow$8 then reconstructs the surface as a zero level set (Arshad et al., 2023). The paper states that this design does not require camera estimation or pixel alignment and gives mean results on ShapeNet of CD $\rightarrow$9, IoU $\rightarrow$0, and F-score $\rightarrow$1, with particularly strong occluded-surface results (Arshad et al., 2023).

L2G-AE makes the local-to-global order explicit in the decoder. The encoder samples $\rightarrow$2 centroids, builds $\rightarrow$3 nested local neighborhoods with $\rightarrow$4, $\rightarrow$5, $\rightarrow$6, $\rightarrow$7, and applies hierarchical self-attention at point, scale, and region levels. The decoder reconstructs a sequence of scales in a local region with an LSTM,

$\rightarrow$8

and then forms the whole point cloud from the reconstructed local areas. The loss is

$\rightarrow$9

The ablation directly quantifies the local/global complementarity: Local only $\rightarrow$0, Global only $\rightarrow$1, Local + Global $\rightarrow$2 on ModelNet10 (Liu et al., 2019).

In image super-resolution, ESTN describes local-to-global reconstruction as alternating aggregation of local and global features inside each Enhanced Swin Transformer Module. Local feature aggregation is implemented by shift convolution and $\rightarrow$3 channel expansion/compression, while global aggregation uses a block sparse global-awareness module, multi-scale self-attention over $\rightarrow$4, $\rightarrow$5, and $\rightarrow$6 windows, and a low-parameter residual channel attention module (Huang et al., 2023). On Manga109 $\rightarrow$7, ESTN reports $\rightarrow$8 dB / $\rightarrow$9, above SwinIR-light and ELAN-light at $3\times3$0 dB / $3\times3$1 (Huang et al., 2023).

The same principle appears in hand understanding. The local and global point cloud reconstruction pipeline for 3D hand pose estimation shares a latent vector $3\times3$2 between a point cloud decoder and a pose decoder. The point cloud decoder is based on FoldingNet, but uses a 3D hand template and local semantic branches for the palm and five fingers. The point cloud loss is

$3\times3$3

The paper reports that removing reconstruction worsens pose estimation on all datasets, with error on average about $3\times3$4 higher without reconstruction (Yu et al., 2021).

4. Inverse problems, MRI, CT, and sparse-view scene reconstruction

In MRI reconstruction, CLGNet formulates local-to-global coupling through the Spatial and Fourier Layer. The local branch is standard convolution,

$3\times3$5

while the global branch transforms the features with a real FFT, applies convolution in Fourier space, and transforms back,

$3\times3$6

The fusion step is

$3\times3$7

The paper’s claim is precise: local convolution preserves fine textures and edges, while Fourier-domain convolution captures image-wide consistency and long-range structure (Yi et al., 2021). The ablation states that removing SFL causes a large drop, around $3\times3$8 dB PSNR on CC-359 Brain $3\times3$9, and CLGNet reports PSNR $$s(x)=p(x)+\sum_{j=1}^{n} c_j\,\phi(\|x-x_j\|)+\sum_{j=1}^{n} d_j\cdot \nabla \phi(x_j-x),$$0, SSIM $$s(x)=p(x)+\sum_{j=1}^{n} c_j\,\phi(\|x-x_j\|)+\sum_{j=1}^{n} d_j\cdot \nabla \phi(x_j-x),$$1, NMSE $$s(x)=p(x)+\sum_{j=1}^{n} c_j\,\phi(\|x-x_j\|)+\sum_{j=1}^{n} d_j\cdot \nabla \phi(x_j-x),$$2 at $$s(x)=p(x)+\sum_{j=1}^{n} c_j\,\phi(\|x-x_j\|)+\sum_{j=1}^{n} d_j\cdot \nabla \phi(x_j-x),$$3 on CC-359 Brain (Yi et al., 2021).

Recon-GLGAN introduces a different medical formulation, centered on global image realism and local region-of-interest realism. The discriminator is

$$s(x)=p(x)+\sum_{j=1}^{n} c_j\,\phi(\|x-x_j\|)+\sum_{j=1}^{n} d_j\cdot \nabla \phi(x_j-x),$$4

where the global branch sees the full $$s(x)=p(x)+\sum_{j=1}^{n} c_j\,\phi(\|x-x_j\|)+\sum_{j=1}^{n} d_j\cdot \nabla \phi(x_j-x),$$5 slice and the local branch sees the $$s(x)=p(x)+\sum_{j=1}^{n} c_j\,\phi(\|x-x_j\|)+\sum_{j=1}^{n} d_j\cdot \nabla \phi(x_j-x),$$6 ROI. The total loss is

$$s(x)=p(x)+\sum_{j=1}^{n} c_j\,\phi(\|x-x_j\|)+\sum_{j=1}^{n} d_j\cdot \nabla \phi(x_j-x),$$7

with $$s(x)=p(x)+\sum_{j=1}^{n} c_j\,\phi(\|x-x_j\|)+\sum_{j=1}^{n} d_j\cdot \nabla \phi(x_j-x),$$8 and $$s(x)=p(x)+\sum_{j=1}^{n} c_j\,\phi(\|x-x_j\|)+\sum_{j=1}^{n} d_j\cdot \nabla \phi(x_j-x),$$9 (Murugesan et al., 2019). The paper reports that the improvement in the ROI is more pronounced than the improvement over the whole image, and that reconstructions from the proposed method give segmentation results similar to fully sampled images (Murugesan et al., 2019).

Sparse-view CT reconstruction uses patch-based 3D diffusion with explicit global conditioning. The global-aware patch prior is

$\phi(r)=\sqrt{r^2+c^2}$0

where $\phi(r)=\sqrt{r^2+c^2}$1 is a local 3D patch and $\phi(r)=\sqrt{r^2+c^2}$2 is a downsampled version of the full volume. At sampling time, the method uses DDIM, recurrent noising over multiple patch offsets, and a data-consistency step by conjugate gradient (Yang et al., 20 Dec 2025). The reported reconstruction scales include $\phi(r)=\sqrt{r^2+c^2}$3, with PSNR $\phi(r)=\sqrt{r^2+c^2}$4 dB at 8 views, $\phi(r)=\sqrt{r^2+c^2}$5 dB at 20 views, and $\phi(r)=\sqrt{r^2+c^2}$6 dB at 60 views, and runtime $\phi(r)=\sqrt{r^2+c^2}$7 min for 8-view reconstruction (Yang et al., 20 Dec 2025).

LGDWT-GS modifies 3D Gaussian Splatting through global and local discrete wavelet regularization. The total loss is

$\phi(r)=\sqrt{r^2+c^2}$8

Global DWT supervision compares full-image subbands, while patch-wise DWT supervision targets patches whose low-frequency energy score

$\phi(r)=\sqrt{r^2+c^2}$9

falls in the lowest $$g(x,y)=\sum_{i,j} S_{ij}(x,y)\,\phi_i(x)\,\phi_j(y),$$0 of the per-image distribution (Salehi et al., 23 Jan 2026). The LLFF 3-view ablation shows Global + Local DWT at PSNR $$g(x,y)=\sum_{i,j} S_{ij}(x,y)\,\phi_i(x)\,\phi_j(y),$$1, SSIM $$g(x,y)=\sum_{i,j} S_{ij}(x,y)\,\phi_i(x)\,\phi_j(y),$$2, LPIPS $$g(x,y)=\sum_{i,j} S_{ij}(x,y)\,\phi_i(x)\,\phi_j(y),$$3, above DWT, DWT + Depth Reg., DWT Staging, and Two-Level DWT (Salehi et al., 23 Jan 2026).

5. Anomaly detection and normality-guided reconstruction

In graph anomaly detection, GTHNA defines local-to-global reconstruction as a pipeline with three coupled components: a local-global graph Transformer encoder, a memory-guided reconstruction mechanism, and a multi-scale representation matching strategy (Li et al., 13 Sep 2025). Local structure is extracted by a $$g(x,y)=\sum_{i,j} S_{ij}(x,y)\,\phi_i(x)\,\phi_j(y),$$4-layer GCN on a $$g(x,y)=\sum_{i,j} S_{ij}(x,y)\,\phi_i(x)\,\phi_j(y),$$5-hop subgraph,

$$g(x,y)=\sum_{i,j} S_{ij}(x,y)\,\phi_i(x)\,\phi_j(y),$$6

global position is encoded by Laplacian eigenvectors, and the two are fused adaptively,

$$g(x,y)=\sum_{i,j} S_{ij}(x,y)\,\phi_i(x)\,\phi_j(y),$$7

Reconstruction is then guided by memory items representing typical normal-node patterns, and the final anomaly score is

$$g(x,y)=\sum_{i,j} S_{ij}(x,y)\,\phi_i(x)\,\phi_j(y),$$8

The stated purpose is to make reconstruction reflect the whole graph context around a node while preventing anomalous nodes from contaminating the normality model (Li et al., 13 Sep 2025).

GLAD uses the phrase in a diffusion-model anomaly setting. Global adaptation is sample-wise selection of the denoising step; local adaptation is anomaly-oriented training plus Spatial-Adaptive Feature Fusion. The paper writes the noisy anomalous input as

$$g(x,y)=\sum_{i,j} S_{ij}(x,y)\,\phi_i(x)\,\phi_j(y),$$9

with reconstruction error approximation

$C^2$0

At inference, the fusion step is

$C^2$1

The ablation on MVTec-AD reports baseline LDM $C^2$2, + ADS $C^2$3, + ATP $C^2$4, and + ADS + ATP + SAFF $C^2$5 for I-AUROC / P-AUROC (Yao et al., 2024).

PCDiff transfers the same local/global logic to point-cloud anomaly detection. The local branch reconstructs $C^2$6 using a 2D-to-3D anomaly prior $C^2$7 obtained from multi-view rendering and back-projection; the global branch reconstructs $C^2$8 without anomaly conditioning. The final score is based on the selectively fused reconstruction

$C^2$9

The ablation gives O-AUROC $C^\infty$0 for global-only, $C^\infty$1 for global + local reconstruction, and $C^\infty$2 for global + local + anomaly mask; removing the global branch drops performance from $C^\infty$3 to $C^\infty$4 (Wu et al., 24 Jun 2026). The paper’s interpretation is explicit: local restoration alone is not enough; it needs the global geometry context.

6. Registration, SLAM, and retrieval as local-to-global consistency reconstruction

L2G-NeRF uses local-to-global reconstruction/registration to solve joint camera alignment and neural rendering. The softened objective is

$C^\infty$5

where $C^\infty$6 is a pixel-wise local transform and

$C^\infty$7

is a frame-wise global transform estimated by a differentiable solver (Chen et al., 2022). The paper argues that direct global pose optimization is fragile under poor initialization, whereas the local stage absorbs large misalignments and the global stage enforces geometric consistency.

PLGSLAM adopts an analogous strategy for neural SLAM in large indoor scenes. The scene is represented as a progressive set of local scene representations $C^\infty$8, each trained in a local sliding window and fused by inverse distance weight at overlapping supervising frames (Deng et al., 2023). Local geometry and appearance are represented with tri-planes,

$C^\infty$9

while MLPs with one-blob coordinate encoding provide low-frequency features, smoothness, and completion in unobserved areas (Deng et al., 2023). Pose drift is controlled by local-to-global bundle adjustment over a local keyframe database and a global keyframe database, using neural warping losses and a classical reprojection loss. The stated consequence is improved reconstruction and Absolute Trajectory Error in large indoor scenes (Deng et al., 2023).

In image retrieval, the reconstructed object is not a scene or signal but a query-specific global embedding derived from local matching structure. The proposed pipeline inverts the dominant global-to-local paradigm:

1. use efficient local feature search to retrieve candidates;
2. build a small dissimilarity matrix for the query and shortlist;
3. embed them into a Euclidean space with MDS/SMACOF;
4. use the resulting coordinates as global features for re-ranking (Aiger et al., 4 Sep 2025). The key formulation is

$\rightarrow$00

The paper reports Hard-setting scores of $\rightarrow$01 on $\rightarrow$02Oxf, $\rightarrow$03 on $\rightarrow$04Oxf + 1M, $\rightarrow$05 on $\rightarrow$06Par, and $\rightarrow$07 on $\rightarrow$08Par + 1M, and frames the method as reconstructing similarity geometry rather than extracting a conventional global descriptor (Aiger et al., 4 Sep 2025).

7. Theoretical guarantees, empirical regularities, and limitations

The abstract frame-theoretic version of local-to-global reconstruction is developed for families $\rightarrow$09. If bounded operators $\rightarrow$10 satisfy

$\rightarrow$11

and for each $\rightarrow$12, $\rightarrow$13 is a frame for $\rightarrow$14 with uniform local bounds $\rightarrow$15, then the global family $\rightarrow$16 is a frame for $\rightarrow$17 with bounds $\rightarrow$18 and $\rightarrow$19 (Aldroubi et al., 2019). For orthogonal projections, this becomes a fusion-frame statement. The robustness theorem allows leakage outside the local patch: if the off-patch decay is controlled by $\rightarrow$20 and $\rightarrow$21, then the global family remains a frame with bounds

$\rightarrow$22

This is a precise local-to-global recovery theorem rather than an architectural heuristic (Aldroubi et al., 2019).

A repeated empirical regularity is that neither local-only nor global-only reconstruction is sufficient. HPN reports that local models reconstruct visible areas substantially better than the global network, while the global model can still be competitive or slightly better in invisible regions (Bechtold et al., 2021). L2G-AE reports Global only $\rightarrow$23 versus Local + Global $\rightarrow$24 (Liu et al., 2019). PCDiff reports that the global branch provides the essential structural anchor needed for stable local refinement (Wu et al., 24 Jun 2026). This suggests that local-to-global reconstruction is not merely multiscale processing; it is a division of labor between detail-sensitive local evidence and coherence-inducing global structure.

A second regularity is that the global stage need not be a learned fusion module. HPN averages overlapping local predictions by Gaussian-weighted averaging and then averages across levels, explicitly noting that more sophisticated learned averaging schemes are conceivable, but come with the risk of overfitting (Bechtold et al., 2021). The classical surface-reconstruction pipeline similarly uses compactly supported PU blending rather than a learned global combiner (Rychkov, 2012). A plausible implication is that local-to-global reconstruction often gains robustness from simple aggregation rules when the local models are already strongly structured.

A common misconception is that more global smoothness is automatically better. The ground-surface paper states that the compactly supported exponential PU blender is theoretically smoother, but with coarse local approximants it can actually produce a less satisfactory result than B-splines (Rychkov, 2012). Another misconception is that local-to-global reconstruction is synonymous with one fixed architecture. The literature instead shows a family of mechanisms: memory-guided normality manifolds in graphs, Fourier-domain global coupling in MRI, downsampled volume context in CT diffusion, ROI-aware adversarial discrimination in MRI, local-to-global bundle adjustment in SLAM, and MDS-based global embedding from local retrieval similarities (Li et al., 13 Sep 2025).

In that sense, local-to-global reconstruction is best understood as a reconstruction doctrine rather than a single model class. Local information supplies reusable evidence, high-frequency detail, or fine-grained anomaly cues; global information supplies completion, consistency, stability, or a normality prior; and the reconstruction succeeds when the interface between the two is explicit, quantitatively constrained, and matched to the structure of the problem.