Semantic-Geometric Conditional Guidance

• Semantic-Geometric Conditional Guidance is a framework that couples semantic inference with geometric data to constrain and refine solution spaces in machine learning tasks.
• It integrates various conditioning strategies such as feature fusion, loss design, and sampling dynamics to enhance tasks in 3D reconstruction, scene understanding, and diffusion-based generation.
• Empirical studies demonstrate improved performance under this guidance, yet challenges like occlusion sensitivity, ambiguous geometry, and computational overhead remain.

Searching arXiv for the cited papers and closely related work on semantic-geometric conditional guidance. arXiv search query: "Semantic-Geometric Conditional Guidance GeoGuide (Tao et al., 27 Mar 2026) VLScene (Wang et al., 8 Mar 2025) Condition-Degradation Guidance (Han et al., 11 Mar 2026) SAMG (Li et al., 29 Apr 2026)" Semantic-Geometric Conditional Guidance denotes a class of methods in which semantic inference and geometric structure are coupled so that one conditions the other during learning, inference, or sampling. In recent work, the conditioning variable may be superpoint geometry, instance masks, voxel neighborhoods, sparse depth priors, object pose, geodesic distances on reconstructed shapes, diffusion guidance energy, or a formal reference frame; the guided quantity may be semantic features, occupancy, correspondences, motion, image restoration, or a denoising trajectory (Tao et al., 27 Mar 2026, Wang et al., 8 Mar 2025, Guo et al., 5 Feb 2026). The unifying objective is to reduce ambiguity that arises when semantics are underconstrained by appearance alone or when geometry lacks category-level or relational meaning.

1. Definition, scope, and recurring structure

In the most explicit formulation, GeoGuide defines hierarchical geometric guidance as a three-level guidance scheme in which intrinsically 3D geometric signals condition semantic feature learning at the point/superpoint, instance, and inter-instance levels (Tao et al., 27 Mar 2026). Other works instantiate the same principle with different objects of conditioning: VLScene distills vision-language priors into 2D features before 2D–3D lifting and then propagates geometry in sparse voxel space (Wang et al., 8 Mar 2025); HG3-NeRF uses sparse SfM depth to condition ray sampling and CLIP features to condition coarse-to-fine semantic alignment (Gao et al., 2024); “Geometry Matters” uses reconstructed geometry, rendered PartField descriptors, and geodesic distances to guide semantic correspondence learning (Jesslen et al., 28 May 2026).

A broader reading of the literature shows that the phrase does not designate a single architecture. Rather, it refers to a family of conditioning strategies in which geometry constrains the admissible semantic solution space, semantics supply priors that disambiguate geometry, or both. This includes diffusion guidance that preserves common global scaffold while sharpening semantic distinctions (Han et al., 11 Mar 2026), adaptive scaling of conditional guidance according to local geometric sensitivity on the data manifold (Li et al., 29 Apr 2026), and hard-constraint diffusion in which semantic and geometric constraints are encoded as a conditioning event and realized through a Doob $h$-transform (Guo et al., 5 Feb 2026).

Area	Representative mechanism	Example papers
Open-vocabulary 3D understanding	Geometry-conditioned semantic distillation, mask completion, relation alignment	GeoGuide (Tao et al., 27 Mar 2026)
Camera-based 3D completion	Vision-language distillation plus geometry-aware sparse propagation	VLScene (Wang et al., 8 Mar 2025)
Reconstruction and correspondence	Depth-guided sampling, rendered 3D descriptors, geodesic filtering	HG3-NeRF (Gao et al., 2024), Geometry Matters (Jesslen et al., 28 May 2026), GaussianZoom (Shi et al., 18 May 2026)
Diffusion and generation	Degraded conditions, spatially adaptive guidance, hard-constraint drift correction	CDG (Han et al., 11 Mar 2026), SAMG (Li et al., 29 Apr 2026), hard-constraint guidance (Guo et al., 5 Feb 2026)
Motion, restoration, reasoning	Contact-aware diffusion, FiLM conditioning, formal task constraints	SocialMirror (Xia et al., 15 Apr 2026), SemGeoMo (Cong et al., 3 Mar 2025), shadow removal (Beltrame et al., 17 Apr 2026), GCA (Chen et al., 27 Nov 2025)

2. Conditioning mechanisms and mathematical forms

A first class of mechanisms operates by feature conditioning. In GeoGuide, a frozen 3D backbone extracts geometric features $F_G$, a lightweight adapter maps them into semantic space as $F_{sem} = VL(F_G)$, and the Uncertainty-based Superpoint Distillation module predicts per-point reliability weights

$$W = \mathrm{MLP}(\mathrm{concat}[(Sga - Fgd); (S2d - F2d)]) \in \mathbb{R}^{N \times 1},$$

so that 2D semantic aggregation is explicitly conditioned on geometry-informed uncertainty (Tao et al., 27 Mar 2026). VLScene uses an analogous but 2D-centric design: LSeg teacher features, text-conditioned logits, and student semantic features are fused by channel-attention gating before lifting to voxels, with

$$F_{fuse} = \hat F_{vision}^{weight} \cdot \mathrm{MLP}(\hat F_{vision}) + F_{sem}^{weight} \cdot \mathrm{MLP}(F_{sem}),$$

after which geometry-aware sparse propagation acts in 3D (Wang et al., 8 Mar 2025). In single-image height prediction, the conditioning is simpler: semantic and normal branches share an encoder with the height branch, and a second-stage denoising autoencoder is conditioned on the concatenation of initial height, semantic labels, normals, and RGB (Mahdi et al., 2020). In shadow removal, frozen DINOv2 semantics and geometric cues from depth and normals are projected to a common space and injected into each stage through FiLM-style affine modulation (Beltrame et al., 17 Apr 2026).

A second class acts through consistency objectives and auxiliary geometry-derived supervision. GeoGuide aligns semantic and geometric similarity matrices across instances and superpoints, and optimizes

$$L_{final} = \lambda_1 L_{sp} + \lambda_2 L_{mask} + \lambda_3 L_{sim},$$

thereby coupling local semantic consistency, full-instance reconstruction, and cross-instance relation consistency (Tao et al., 27 Mar 2026). S4-Net imposes that projections of the same 3D point across frames receive the same semantic label, casting geometric multi-view constraints as a semi-supervised term $L = L_S + \lambda L_G$ (Stekovic et al., 2019). HG3-NeRF avoids direct depth supervision and instead conditions the ray sampling interval through local-to-global near–far scheduling around the sparse depth prior, while CLIP alignment supplies coarse-to-fine semantic supervision (Gao et al., 2024). “Geometry Matters” conditions pseudo-label generation by rendered PartField descriptors and then filters candidate matches through bicyclic geodesic consistency on reconstructed meshes (Jesslen et al., 28 May 2026).

A third class operates directly on the sampling dynamics of diffusion or related generative processes. Condition-Degradation Guidance replaces the null prompt in classifier-free guidance with a strategically degraded condition:

$$D_\theta^{\mathrm{CDG}}(\mathbf{x}_\sigma; \sigma, \mathbf{c}) = D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) + (w-1)\big(D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) - D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}_{\mathrm{deg}})\big),$$

so that guidance contrasts “good vs. almost good” rather than “good vs. null” (Han et al., 11 Mar 2026). SAMG modulates the guidance scale pointwise from the delta-score energy $E_t(x) = \frac{1}{C}\|\Delta \epsilon_t(x)\|_2^2$, using a conservative scale in high-energy boundary regions and a larger scale in low-energy regions (Li et al., 29 Apr 2026). Under hard constraints, the conditioning function $h(t,y)=P(Y_T \in S \mid Y_t=y)$ induces the guided reverse SDE

$F_G$0

which augments a pretrained diffusion by an explicit drift correction without modifying the pretrained score network (Guo et al., 5 Feb 2026).

These formulations differ in implementation, but a recurrent pattern is that conditioning enters at one of four loci: feature fusion, loss construction, candidate filtering, or sampler drift. The distinction is consequential because some methods use guidance only during training, whereas others alter the test-time trajectory itself.

3. Scene understanding and 3D reconstruction

Open-vocabulary 3D segmentation provides one of the clearest semantic-geometric formulations. GeoGuide takes a scene point cloud $F_G$1 and multi-view images $F_G$2, extracts geometry from a frozen Sonata backbone, obtains 2D open-vocabulary features from frozen LSeg, OpenSeg, and optionally SEEM via SAS, and applies three complementary modules: Uncertainty-based Superpoint Distillation, Instance-level Mask Reconstruction, and Inter-Instance Relation Consistency (Tao et al., 27 Mar 2026). On ScanNet v2, Matterport3D, and nuScenes, GeoGuide with SAS* features reports $F_G$3 mIoU and $F_G$4 mAcc, $F_G$5 mIoU and $F_G$6 mAcc, and $F_G$7 mIoU and $F_G$8 mAcc, respectively; the ablation with LSeg on ScanNet v2 shows the combined USD+IMR+IIRC configuration reaching $F_G$9 mIoU and $F_{sem} = VL(F_G)$0 mAcc, compared with $F_{sem} = VL(F_G)$1 and $F_{sem} = VL(F_G)$2 for the OpenScene baseline (Tao et al., 27 Mar 2026). At inference, only the 3D point cloud is required and the three geometry-guided modules are dropped.

VLScene addresses camera-based 3D semantic scene completion rather than segmentation, but the structural pattern is similar. Vision-language guidance distillation is injected into 2D semantic image features before 2D–3D lifting, and the 3D backbone then uses Neighborhood Geometry Propagation and Sparse Semantic Interaction to propagate geometry and contextual semantics in sparse voxel space (Wang et al., 8 Mar 2025). The model achieves rank-1st performance on SemanticKITTI and SSCBench-KITTI-360 with hidden-test mIoU of $F_{sem} = VL(F_G)$3 and $F_{sem} = VL(F_G)$4, respectively, while reporting $F_{sem} = VL(F_G)$5s inference time and about $F_{sem} = VL(F_G)$6M parameters (Wang et al., 8 Mar 2025). The paper explicitly notes that teacher inference is a training overhead only; there is no added cost at inference.

Sparse-view 3D reconstruction and novel-view synthesis expose a different use of guidance. HG3-NeRF introduces Hierarchical Geometric Guidance by conditioning the ray sampling region on an SfM-derived sparse depth prior through local-to-global scheduling, and Hierarchical Semantic Guidance by aligning rendered images and reference images through CLIP features on a progressively denser pixel grid (Gao et al., 2024). On LLFF, the full model reports $F_{sem} = VL(F_G)$7, $F_{sem} = VL(F_G)$8, and $F_{sem} = VL(F_G)$9 PSNR at $$W = \mathrm{MLP}(\mathrm{concat}[(Sga - Fgd); (S2d - F2d)]) \in \mathbb{R}^{N \times 1},$$0, $$W = \mathrm{MLP}(\mathrm{concat}[(Sga - Fgd); (S2d - F2d)]) \in \mathbb{R}^{N \times 1},$$1, and $$W = \mathrm{MLP}(\mathrm{concat}[(Sga - Fgd); (S2d - F2d)]) \in \mathbb{R}^{N \times 1},$$2 views; the ablation shows that HGG is crucial under sparse views and that HSG adds further gains (Gao et al., 2024). GaussianZoom transfers the same logic to generative zoom-in 3D Gaussian Splatting: depth-based feature warping supplies geometry-consistent alignment, VLM-generated textual descriptions condition detail synthesis, and a continuous Level-of-Detail hierarchy modulates Gaussian visibility across scales (Shi et al., 18 May 2026). On $$W = \mathrm{MLP}(\mathrm{concat}[(Sga - Fgd); (S2d - F2d)]) \in \mathbb{R}^{N \times 1},$$3 super-resolution, it reports PSNR/SSIM/LPIPS/FID of $$W = \mathrm{MLP}(\mathrm{concat}[(Sga - Fgd); (S2d - F2d)]) \in \mathbb{R}^{N \times 1},$$4 on Mip-NeRF360 and $$W = \mathrm{MLP}(\mathrm{concat}[(Sga - Fgd); (S2d - F2d)]) \in \mathbb{R}^{N \times 1},$$5 on Tanks&Temples (Shi et al., 18 May 2026).

Semantic correspondence learning uses geometry as a filter on semantics rather than as a direct feature source for the final adapter. “Geometry Matters” reconstructs object geometry and pose with SAM3D, refines the pose by render-and-compare, renders PartField descriptors into the image plane, and uses geodesic distances on the reconstructed meshes to retain only geometrically consistent pseudo-labels (Jesslen et al., 28 May 2026). The resulting adapter on top of frozen DINO and Stable Diffusion reaches $$W = \mathrm{MLP}(\mathrm{concat}[(Sga - Fgd); (S2d - F2d)]) \in \mathbb{R}^{N \times 1},$$6 [email protected] on SPair-71k, $$W = \mathrm{MLP}(\mathrm{concat}[(Sga - Fgd); (S2d - F2d)]) \in \mathbb{R}^{N \times 1},$$7 on SPair-Geo-Aware, and up to $$W = \mathrm{MLP}(\mathrm{concat}[(Sga - Fgd); (S2d - F2d)]) \in \mathbb{R}^{N \times 1},$$8 on AP-10K, while reducing the false-positive rate of pseudo-labels through geodesic filtering (Jesslen et al., 28 May 2026). An earlier precursor in supervised adaptation is S4-Net, which uses known depth and pose to warp predictions across frames and imposes label consistency for the same 3D locations; on ScanNet, DeepLabV3+ improves from $$W = \mathrm{MLP}(\mathrm{concat}[(Sga - Fgd); (S2d - F2d)]) \in \mathbb{R}^{N \times 1},$$9 to $$F_{fuse} = \hat F_{vision}^{weight} \cdot \mathrm{MLP}(\hat F_{vision}) + F_{sem}^{weight} \cdot \mathrm{MLP}(F_{sem}),$$0 mIoU on Scan 1 and from $$F_{fuse} = \hat F_{vision}^{weight} \cdot \mathrm{MLP}(\hat F_{vision}) + F_{sem}^{weight} \cdot \mathrm{MLP}(F_{sem}),$$1 to $$F_{fuse} = \hat F_{vision}^{weight} \cdot \mathrm{MLP}(\hat F_{vision}) + F_{sem}^{weight} \cdot \mathrm{MLP}(F_{sem}),$$2 on Scan 2 (Stekovic et al., 2019).

Taken together, these systems show that in 3D perception and reconstruction, guidance usually serves one of three roles: filtering noisy 2D supervision before it enters 3D, restricting correspondence or sampling to geometrically plausible supports, or enforcing cross-view and cross-instance consistency after an initial semantic estimate has already been made.

4. Diffusion, generative sampling, and motion synthesis

In diffusion models for image generation, semantic-geometric conditional guidance appears as a correction to standard classifier-free guidance. Condition-Degradation Guidance argues that CFG’s null prompt yields a semantically vacuous contrast that produces geometric entanglement, especially for compositional prompts (Han et al., 11 Mar 2026). CDG therefore constructs a degraded condition $$F_{fuse} = \hat F_{vision}^{weight} \cdot \mathrm{MLP}(\hat F_{vision}) + F_{sem}^{weight} \cdot \mathrm{MLP}(F_{sem}),$$3 by selectively degrading content tokens while preserving context-aggregating tokens, and uses this condition in place of the null prompt. Across Stable Diffusion 3, SD3.5, FLUX.1-dev, and Qwen-Image on COCO, CDG improves FID, CLIPScore, Aesthetic score, or VQA over CFG; for SD3, for example, FID improves from $$F_{fuse} = \hat F_{vision}^{weight} \cdot \mathrm{MLP}(\hat F_{vision}) + F_{sem}^{weight} \cdot \mathrm{MLP}(F_{sem}),$$4 to $$F_{fuse} = \hat F_{vision}^{weight} \cdot \mathrm{MLP}(\hat F_{vision}) + F_{sem}^{weight} \cdot \mathrm{MLP}(F_{sem}),$$5 and VQA from $$F_{fuse} = \hat F_{vision}^{weight} \cdot \mathrm{MLP}(\hat F_{vision}) + F_{sem}^{weight} \cdot \mathrm{MLP}(F_{sem}),$$6 to $$F_{fuse} = \hat F_{vision}^{weight} \cdot \mathrm{MLP}(\hat F_{vision}) + F_{sem}^{weight} \cdot \mathrm{MLP}(F_{sem}),$$7, while on GenAI-Bench for SD3.5 spatial relations improve from $$F_{fuse} = \hat F_{vision}^{weight} \cdot \mathrm{MLP}(\hat F_{vision}) + F_{sem}^{weight} \cdot \mathrm{MLP}(F_{sem}),$$8 to $$F_{fuse} = \hat F_{vision}^{weight} \cdot \mathrm{MLP}(\hat F_{vision}) + F_{sem}^{weight} \cdot \mathrm{MLP}(F_{sem}),$$9 (Han et al., 11 Mar 2026).

SAMG addresses a different failure mode: the use of a single global CFG scale on a curved data manifold. It defines the local conditional guidance energy as

$$L_{final} = \lambda_1 L_{sp} + \lambda_2 L_{mask} + \lambda_3 L_{sim},$$0

normalizes it per timestep, and maps it to a spatially adaptive scale between $$L_{final} = \lambda_1 L_{sp} + \lambda_2 L_{mask} + \lambda_3 L_{sim},$$1 and $$L_{final} = \lambda_1 L_{sp} + \lambda_2 L_{mask} + \lambda_3 L_{sim},$$2 so that boundary or high-frequency regions receive conservative guidance and low-energy regions receive aggressive semantic injection (Li et al., 29 Apr 2026). On SDXL COCO, FID drops from $$L_{final} = \lambda_1 L_{sp} + \lambda_2 L_{mask} + \lambda_3 L_{sim},$$3 for CFG to $$L_{final} = \lambda_1 L_{sp} + \lambda_2 L_{mask} + \lambda_3 L_{sim},$$4 for SAMG and CLIPScore rises from $$L_{final} = \lambda_1 L_{sp} + \lambda_2 L_{mask} + \lambda_3 L_{sim},$$5 to $$L_{final} = \lambda_1 L_{sp} + \lambda_2 L_{mask} + \lambda_3 L_{sim},$$6; on video models, the method improves CHScore Flow, Frame LPIPS, Frame SSIM, and CLIP-based scores while remaining training-free and virtually zero-cost (Li et al., 29 Apr 2026). The paper explicitly states that spatial smoothing of the energy map is detrimental because it causes “energy leakage.”

The hard-constraint framework provides the most formal account of conditional guidance. Rather than scaling a heuristic delta score, it interprets the constraint as an event $$L_{final} = \lambda_1 L_{sp} + \lambda_2 L_{mask} + \lambda_3 L_{sim},$$7, defines $$L_{final} = \lambda_1 L_{sp} + \lambda_2 L_{mask} + \lambda_3 L_{sim},$$8, and derives an exact guided reverse process by Doob’s $$L_{final} = \lambda_1 L_{sp} + \lambda_2 L_{mask} + \lambda_3 L_{sim},$$9-transform (Guo et al., 5 Feb 2026). The resulting guided dynamics come with non-asymptotic total variation and Wasserstein guarantees that explicitly characterize the impact of score approximation and guidance estimation errors. The paper also states that semantic and geometric constraints can be combined by defining the joint event $L = L_S + \lambda L_G$0, or approximately factorized via a product-of-experts view when conditional independence is plausible (Guo et al., 5 Feb 2026). This gives the topic an explicit stochastic-analysis foundation rather than a purely architectural one.

Diffusion-based human motion systems use the same principle with more structured constraints. SocialMirror reconstructs two interacting humans from monocular video by combining a semantic-guided motion infiller, conditioned on image features and text descriptions, with a geometry-guided temporal refiner that injects contact consistency and collision avoidance into the reverse process through gradient-based updates (Xia et al., 15 Apr 2026). On Hi4D, it reports $L = L_S + \lambda L_G$1 and $L = L_S + \lambda L_G$2 relative improvements over the latest state of the art in RE and Int, respectively, while reducing penetration and improving smoothness (Xia et al., 15 Apr 2026). SemGeoMo similarly uses text-affordance-joint multi-level semantic and geometric guidance for dynamic contextual human motion generation, with Stage I generating affordance maps and joint positions and Stage II injecting semantic and geometric conditions into a ControlNet attached to a frozen MDM prior (Cong et al., 3 Mar 2025). On FullBodyManipulation, the method reports HandJPE $L = L_S + \lambda L_G$3, MPJPE $L = L_S + \lambda L_G$4, FID $L = L_S + \lambda L_G$5, and R-score $L = L_S + \lambda L_G$6, outperforming OMOMO on the reported metrics (Cong et al., 3 Mar 2025).

A common misconception is that semantic-geometric conditional guidance in diffusion is synonymous with text conditioning alone. The surveyed work shows a broader picture: semantics may define the discriminative target, but geometry enters through negative-sample construction, curvature-aware scale control, contact or collision energies, or exact hard-constraint drift corrections.

5. Restoration, single-view estimation, and agentic spatial reasoning

Low-level vision systems use guidance to separate appearance changes from scene structure. The three-stage shadow-removal pipeline built on OmniSR combines RGB appearance with frozen DINOv2 semantic guidance and geometric cues from monocular depth and surface normals, all reused across stages (Beltrame et al., 17 Apr 2026). Guidance is fused through $L = L_S + \lambda L_G$7 projections and injected into residual blocks by FiLM-style affine modulation, while a contraction-constrained objective penalizes later stages if they increase reconstruction error. On the WSRD+ 2026 hidden test set, the final ensemble reports $L = L_S + \lambda L_G$8 PSNR, $L = L_S + \lambda L_G$9 SSIM, $$D_\theta^{\mathrm{CDG}}(\mathbf{x}_\sigma; \sigma, \mathbf{c}) = D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) + (w-1)\big(D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) - D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}_{\mathrm{deg}})\big),$$0 LPIPS, and $$D_\theta^{\mathrm{CDG}}(\mathbf{x}_\sigma; \sigma, \mathbf{c}) = D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) + (w-1)\big(D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) - D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}_{\mathrm{deg}})\big),$$1 FID (Beltrame et al., 17 Apr 2026). Here, semantic guidance is not a label prior in the segmentation sense; it is a material- and context-aware prior for distinguishing illumination changes from intrinsic reflectance.

Single-image height prediction uses a more classical multi-task version of the same idea. A shared DenseNet121 encoder predicts height, semantics, and normals, and the main height decoder receives semantic and normal information through inter-connected decoder feature fusion (Mahdi et al., 2020). A second-stage U-Net denoising autoencoder is explicitly conditioned on the initial height map, predicted semantics, predicted normals, and the RGB image. On ISPRS Vaihingen, the full system reports MSE $$D_\theta^{\mathrm{CDG}}(\mathbf{x}_\sigma; \sigma, \mathbf{c}) = D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) + (w-1)\big(D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) - D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}_{\mathrm{deg}})\big),$$2, MAE $$D_\theta^{\mathrm{CDG}}(\mathbf{x}_\sigma; \sigma, \mathbf{c}) = D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) + (w-1)\big(D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) - D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}_{\mathrm{deg}})\big),$$3, and RMSE $$D_\theta^{\mathrm{CDG}}(\mathbf{x}_\sigma; \sigma, \mathbf{c}) = D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) + (w-1)\big(D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) - D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}_{\mathrm{deg}})\big),$$4; on the 2018 DFC dataset it reports MSE $$D_\theta^{\mathrm{CDG}}(\mathbf{x}_\sigma; \sigma, \mathbf{c}) = D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) + (w-1)\big(D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) - D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}_{\mathrm{deg}})\big),$$5, MAE $$D_\theta^{\mathrm{CDG}}(\mathbf{x}_\sigma; \sigma, \mathbf{c}) = D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) + (w-1)\big(D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) - D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}_{\mathrm{deg}})\big),$$6, and RMSE $$D_\theta^{\mathrm{CDG}}(\mathbf{x}_\sigma; \sigma, \mathbf{c}) = D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) + (w-1)\big(D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) - D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}_{\mathrm{deg}})\big),$$7 (Mahdi et al., 2020). Ablations isolate the effect of conditioning both in the multi-task predictor and in the refinement stage.

Agentic spatial reasoning shows that “geometry” need not mean a mesh, voxel grid, or depth map. The Geometrically-Constrained Agent translates a free-form query into a formal task constraint $$D_\theta^{\mathrm{CDG}}(\mathbf{x}_\sigma; \sigma, \mathbf{c}) = D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) + (w-1)\big(D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) - D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}_{\mathrm{deg}})\big),$$8, where $$D_\theta^{\mathrm{CDG}}(\mathbf{x}_\sigma; \sigma, \mathbf{c}) = D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) + (w-1)\big(D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}) - D_\theta(\mathbf{x}_\sigma; \sigma, \mathbf{c}_{\mathrm{deg}})\big),$$9 defines a single non-negotiable coordinate system and $E_t(x) = \frac{1}{C}\|\Delta \epsilon_t(x)\|_2^2$0 formalizes the target quantity to be computed (Chen et al., 27 Nov 2025). The VLM then plans and executes tool calls strictly within those deterministic bounds, using fixed geometric formulas for transforms, rotation analysis, and axis derivation. Across several spatial reasoning benchmarks, GCA achieves about $E_t(x) = \frac{1}{C}\|\Delta \epsilon_t(x)\|_2^2$1 average accuracy and surpasses training-based methods by about $E_t(x) = \frac{1}{C}\|\Delta \epsilon_t(x)\|_2^2$2 and tool-integrated methods by about $E_t(x) = \frac{1}{C}\|\Delta \epsilon_t(x)\|_2^2$3 (Chen et al., 27 Nov 2025). In this setting, semantic-geometric conditional guidance is instantiated not as a tensor operation but as plan-time restriction of the reasoning pathway.

These examples broaden the concept substantially. Geometry may be encoded as depth and normals, as auxiliary tasks, or as a formal reference frame; semantics may be category labels, frozen self-supervised embeddings, language descriptions, or human-readable task constraints. What remains constant is that semantic decisions are not permitted to evolve independently of a structured geometric scaffold.

6. Empirical regularities, limitations, and conceptual status

The literature consistently reports gains when semantics and geometry are coupled, but it also documents recurring failure modes. GeoGuide notes persistent semantic drift when geometry is ambiguous or when instance proposals are noisy or incomplete, degradation in long-tail Matterport3D settings up to $E_t(x) = \frac{1}{C}\|\Delta \epsilon_t(x)\|_2^2$4, and instability from heterogeneous supervision signals across LSeg and OpenSeg without the proposed guidance modules (Tao et al., 27 Mar 2026). VLScene reports continued vulnerability to severe occlusions, extreme perspective, and misaligned CLIP/LSeg pseudo labels, as well as training overhead from teacher inference even though inference-time cost is unchanged (Wang et al., 8 Mar 2025). “Geometry Matters” observes that occlusions, reflective or textureless surfaces, and non-rigid deformations can degrade mesh or pose quality and thereby reduce the reliability of PartField rasterization and geodesic filtering (Jesslen et al., 28 May 2026).

Diffusion-based methods exhibit a parallel set of sensitivities. CDG requires a usable content/context token split; over-aggressive degradation of context-aggregating tokens for $E_t(x) = \frac{1}{C}\|\Delta \epsilon_t(x)\|_2^2$5 can erode global scaffold and style, and gains are smaller on models that already distill guidance during training, such as FLUX.1 (Han et al., 11 Mar 2026). SAMG can underperform when multiple intricate structures merge into a single high-energy region, and kernel sizes larger than $E_t(x) = \frac{1}{C}\|\Delta \epsilon_t(x)\|_2^2$6 degrade performance through energy leakage (Li et al., 29 Apr 2026). In the hard-constraint framework, guidance quality depends on accurate estimation of $E_t(x) = \frac{1}{C}\|\Delta \epsilon_t(x)\|_2^2$7 and its gradient, and the total-variation bound scales as $E_t(x) = \frac{1}{C}\|\Delta \epsilon_t(x)\|_2^2$8, making rare events intrinsically difficult (Guo et al., 5 Feb 2026). SocialMirror identifies VLM inaccuracies in complex limb entanglements and the computational cost of LBFGS at each denoising step as remaining limitations (Xia et al., 15 Apr 2026).

The same pattern appears outside diffusion. In shadow removal, semantic misguidance from frozen DINOv2 or geometric errors from monocular depth can lead to over-correction, color drift, or residual haloes (Beltrame et al., 17 Apr 2026). In GCA, about $E_t(x) = \frac{1}{C}\|\Delta \epsilon_t(x)\|_2^2$9 of errors are attributed to formalization and about $h(t,y)=P(Y_T \in S \mid Y_t=y)$0 to the compute stage, including perception failures and Python-tool errors (Chen et al., 27 Nov 2025). These reports indicate that the main bottleneck is often not the existence of a conditioning channel, but the fidelity of the auxiliary signal that defines it.

A second misconception is that semantic-geometric conditional guidance is inherently a diffusion-specific technique. The surveyed work directly contradicts that view: the same principle appears in semi-supervised segmentation with multi-view warping (Stekovic et al., 2019), open-vocabulary 3D segmentation (Tao et al., 27 Mar 2026), semantic scene completion (Wang et al., 8 Mar 2025), NeRF under sparse views (Gao et al., 2024), semantic correspondence (Jesslen et al., 28 May 2026), shadow removal (Beltrame et al., 17 Apr 2026), single-image height prediction (Mahdi et al., 2020), motion reconstruction (Xia et al., 15 Apr 2026), motion generation (Cong et al., 3 Mar 2025), and agentic spatial reasoning (Chen et al., 27 Nov 2025).

A third regularity is that many systems separate strong guidance from deployment. GeoGuide discards its geometry-guided modules at inference and requires only the 3D point cloud (Tao et al., 27 Mar 2026). S4-Net uses geometry only during training, after which the network segments unseen target views without geometry (Stekovic et al., 2019). VLScene adds no inference cost for its teacher-based distillation path (Wang et al., 8 Mar 2025). This suggests that, in practice, semantic-geometric conditional guidance is often best understood not as a permanent extra input, but as a structured way to shape representations, admissible solutions, or sampling trajectories so that the final model internalizes geometric regularities rather than merely consulting them.