I-Scene Model: Non-Semantic Scene Composition

• I-Scene Model is a non-semantic scene composition framework that formalizes scene structure through spatial, geometric, and color-based features.
• It employs hierarchical partitioning and deep generative models to separate scene configuration from semantic labels, enabling robust image synthesis and retrieval.
• Applications span real-time 3D editing, generative aesthetics, and scene retrieval, with metrics like SCSSIM ensuring high fidelity in structural evaluation.

Non-semantic scene composition refers to the analysis, manipulation, generation, or evaluation of visual scenes based on structural, geometric, or appearance-based relationships, without recourse to object categories, instance identities, or semantic class labels. This paradigm embraces spatial layout, size, color distributions, composition rules, and physical relationships, focusing on the “syntax” of visual organization rather than content-based semantics. Non-semantic models and metrics underpin applications in generative modeling, retrieval, inverse rendering, and computational aesthetics, and are central to robustness and generalization for vision systems deployed in uncontrolled or unseen environments.

1. Foundational Principles and Definitions

At its core, non-semantic scene composition concerns the formalization of scene structure through features that are invariant or agnostic to semantic class. Key representations include spatial arrangement (positions and proportions of visual primitives), geometric relationships (orientations, aspect ratios, relative scales), color distributions, and hierarchical or layered composition.

The “things syntax” formalism (Kordumova et al., 2016) encodes a scene as an unordered set of five-dimensional vectors per object proposal: $$\mathbf{w}_i = [w_{i,x},\,w_{i,y},\,w_{i,s},\,w_{i,r},\,w_{i,c}]$$ where $w_{i,x}, w_{i,y}$ are normalized positions, $w_{i,s}$ is normalized size, $w_{i,r}$ is aspect ratio, and $w_{i,c}$ is a quantized color label. Such representations intentionally abstract from object identity, focusing on observable, immediately quantifiable properties.

Scene Composition Structure (SCS) (Haque et al., 7 Aug 2025) extends this logic hierarchically, defining the structure by the set of spatial splits (cuts) that optimally partition the image by statistical criteria (e.g., sum-of-squared-error), without object detection or labeling. The resulting structure encodes the spatial relationship among regions, their arrangement, and compositional hierarchy.

In deep generative models, non-semantic scene composition is realized by explicitly disentangling scene configuration from semantics, as in the global object-centric latent compositions of GOCL (Chen et al., 2022) and the multi-view slot-based factorization in OCLOC (Yuan et al., 2021). In 3D, methods such as ComGS (Gao et al., 9 Oct 2025) decompose radiance fields into appearance, materials, and lighting without reference to class or part labels, instead relying on physically motivated, non-semantic attributes.

2. Methodologies for Modeling Non-Semantic Composition

Structured Spatial Descriptors

The “things syntax” pipeline (Kordumova et al., 2016) processes object proposal windows from methods like Selective Search, extracting five immediately observable features per proposal. This matrix of thing-vectors is used to construct histograms of quantized spatial statements or to encode abstract block illustrations for scene retrieval, with no reference to class labels.

Hierarchical partitioning methods, exemplified by SCSSIM (Haque et al., 7 Aug 2025), recursively split the image using the strongest (lowest SSE) horizontal or vertical cut, yielding a binary partition tree capturing major compositional divisions. SCSSIM evaluates the alignment of hierarchical split structure between images, directly quantifying structural changes independently of semantic content.

Deep Generative and Compositional Models

Object-centric latent factor models such as GOCL (Chen et al., 2022) define images as spatial mixtures of background and up to $K$ objects. Each object is represented by extrinsic (pose/scale) latents and an intrinsic embedding chosen from a learned dictionary of canonical vectors; this canonicalization enables correspondence and manipulation without supervision or labels. Masked patch-matching aligns occluded parts with their canonical reference.

Slot-based generative models such as OCLOC (Yuan et al., 2021) further disentangle viewpoint-dependent and viewpoint-independent latent codes for object slots, coupled with soft-masking and layered Gaussian likelihoods. This factorization supports compositional reasoning and new-view synthesis without semantic supervision.

3D non-semantic composition in ComGS (Gao et al., 9 Oct 2025) relies on relightable object decomposition via Surface Octahedral Probes (SOPs), which encode local non-semantic lighting and occlusion around object placements, and on locally-complete environment maps estimated by geometric ray tracing and diffusion inpainting.

Aesthetic Composition Guidance

Recent models for generative aesthetic refinement such as Composer (Zou et al., 6 May 2026) operate at the spatial and color-structural level rather than through semantic conditioning. Composer transforms reference images into two low-frequency, semantic-agnostic maps: a spatial structure map (local contrast over varying scales) and a color distribution map (superpixel-labeled structure after Gaussian blurring). These are injected as conditioning into diffusion-based generative models, enabling composition transfer and planning agnostic to scene content.

3. Applications and Evaluation Metrics

Scene Retrieval and Query

Non-semantic scene retrieval leverages abstractions such as block illustrations or textual statements to query a gallery of real-world scenes (Kordumova et al., 2016). Scene images are ranked by similarity of their things-syntax descriptors (via Fisher vectors) or by histogram alignment of abstract statement codes. This approach allows retrieval of unseen scene classes with no per-class training, achieving 5.25% mAP for abstract statements and 12.39% mAP for block illustrations on MIT Indoor-67, on par with or exceeding generic semantic methods depending on task overlap.

Generative Model Assessment

Traditional pixel- or perception-based similarity metrics (MSE, SSIM, LPIPS, CLIP) fail to capture scene structural stability under geometric changes. The SCSSIM metric (Haque et al., 7 Aug 2025), based on the cumulative gain of hierarchical region splits, provides an analytical, training-free measure highly robust to non-compositional distortions, yet decline monotonicity with true compositional changes such as rotation or cropping. SCSSIM achieves >0.98 under severe noise/blur but drops near zero under 90° rotation, outperforming learned perceptual metrics in structure sensitivity.

Inverse Rendering and Editing

In 3D, compositional editing frameworks such as ComGS (Gao et al., 9 Oct 2025) achieve real-time physically plausible object insertion and relighting without recourse to semantic segmentation or recognition. ComGS achieves 28 FPS rendering and visually harmonious compositions (PSNR 24.46 dB, SSIM 0.847, user Harmony 4.05/5 on SynCom) by recomposing non-semantic radiance fields and recomputing local environment maps and shadowing.

Aesthetic and Creative Generation

Composer (Zou et al., 6 May 2026) supports both precision composition transfer (reference-based guidance) and implicit composition planning (text-only mode), improving both measured CLIP alignment and Aesthetic Score versus major layout/region-control and implicit-composition baselines.

4. Comparative Summary of Major Models and Metrics

Approach	Core Representation	Key Task	Evaluation
Things Syntax (Kordumova et al., 2016)	5D position/size/ratio/color	Scene retrieval (block/text queries)	mAP on MIT Indoor-67/SUN
GOCL (Chen et al., 2022)	Object extrinsic/intrinsic latents	Layered generative modeling	ARI, MSE, IACC
OCLOC (Yuan et al., 2021)	Slot-based viewpoint disentanglement	Unsupervised compositional learning	ARI, AMI, IoU, OCA
SCSSIM (Haque et al., 7 Aug 2025)	Hierarchical SSE gain partitions	Similarity and evaluation	SCSSIM, compared to SSIM, LPIPS, CLIP
ComGS (Gao et al., 9 Oct 2025)	3D radiance/material/lighting fields	Physically plausible 3D editing	PSNR, SSIM, Harmony, FPS
Composer (Zou et al., 6 May 2026)	Low-frequency spatial/color maps	Aesthetic composition planning	CLIP, FID, Aes, HPS, cycle consistency

These models and metrics demonstrate the breadth of non-semantic scene composition, from low-level spatial syntax to advanced generative diffusion pipelines, unified by their avoidance of explicit semantic knowledge during inference or evaluation.

5. Open Problems and Future Directions

Several challenges remain for non-semantic scene composition. Current methods such as SOPs (Gao et al., 9 Oct 2025) are most effective in quasi-static, locally coherent environments; large dynamic scenes or those with significant occlusion may require more adaptive or hierarchical structural models. Completing local lighting panoramas for relighting is bottlenecked by panoramic coverage; failure modes arise when visibility is limited.

For 2D and generative models, improved representation of higher-order geometric relationships, scale invariance across scenes, and robustness to unbounded occlusion or clutter remain areas of active research. Incorporating uncertainty (confidence) in region-level or probe-level structural cues, as well as fully end-to-end optimization (e.g., composition-aware fine-tuning of diffusion models), are promising directions noted in recent work.

A plausible implication is that as non-semantic composition models are scaled to higher complexity generative or analytical tasks, hybrid approaches incorporating minimal semantic priors, global structural surrogates, or context-aware partitioning may be required to maintain both generality and compositional fidelity.

6. Significance and Implications

Non-semantic scene composition provides a principled framework for modeling and controlling structural aspects of visual scenes independently of class or object boundaries. This paradigm is critical for generalization to unseen environments, enables interpretable scene analysis, supports creative applications unconstrained by fixed vocabularies, and establishes robust metrics for structural fidelity in generative AI. The development of non-semantic, training-free metrics such as SCSSIM (Haque et al., 7 Aug 2025) creates new possibilities for evaluating growth in compositional understanding at scale, while models such as Composer (Zou et al., 6 May 2026) and GOCL (Chen et al., 2022) demonstrate the practical utility and flexibility of abstract composition representations for both machine perception and human-facing applications.