---
title: 'SIGMA-Gen: Multi-Subject Identity-Preserving Generation'
url: https://www.emergentmind.com/topics/sigma-gen
type: topic
---

# SIGMA-Gen: Multi-Subject Identity-Preserving Generation

SIGMA-Gen is a controllable image-generation framework for **multi-subject identity-preserving generation** under explicit spatial and structural constraints. It is presented as a unified system that generates several specified subjects in a single image while preserving each subject’s identity and obeying controls that range from **2D bounding boxes** and **3D bounding boxes** to **pixel-level segmentations** and **depth**. The method is built on a diffusion-transformer backbone, uses a compact joint representation of identity and structure, and is trained with the synthetic dataset **SIGMA-Set27K** [2510.06469].

## 1. Scope and task definition

The task addressed by SIGMA-Gen is: given a text prompt \(P\), a set of subjects \(\mathbf{S}=\{s_1,\dots,s_n\}\), and spatial or structural controls \(C\), generate an image \(I\) such that all subject identities are preserved and the subjects are placed according to \(C\) [2510.06469].

The framework is motivated by a conjunction of two control problems that earlier systems typically treat separately. The first is **identity control**, in which a reference image specifies who or what should appear. The second is **arrangement or structure control**, in which the user specifies where subjects should go and, in stronger forms, how they should be posed or ordered in depth. The paper argues that prior personalization methods can learn identities but usually require **per-subject optimization**, while structure-control methods such as ControlNet, T2I-Adapter, and GLIGEN can follow boxes, edges, or depth but do **not preserve multiple identities**. Iterative insertion methods can sometimes be adapted to multiple subjects, but the paper identifies increasing runtime, quality degradation, and compounded identity and layout failures as their main liabilities [2510.06469].

Within that framing, SIGMA-Gen is presented as the **first single-pass method** for **multi-subject identity-preserving generation** with **fine-to-coarse structural control**. The emphasis on “single-pass” is central: all subjects are synthesized in one denoising process rather than added sequentially [2510.06469].

## 2. Conditioning representation and model architecture

SIGMA-Gen uses three principal conditioning streams: the prompt \(P\), an **identity control image** \(I^{\mathcal{S}}\), and a **spatial control image** \(I^C\). The backbone is **Flux.1 Kontext [dev]**, described as a diffusion transformer. The method uses the backbone’s VAE to encode the identity control image and the spatial control image, then concatenates all modalities as tokens,

\[
[P, X, I^{\mathcal{S}}, I^C],
\]

where \(X\) denotes the noisy image latents. This creates a unified attention space in which image latents, prompt tokens, identity references, and structural controls can attend to one another [2510.06469].

The identity branch uses one RGB reference image \(I^{s_i}\) per subject. If each identity image is

\[
I^{s_i} \in \mathbb{R}^{H' \times W' \times 3},
\]

then the full identity condition image is formed by vertical concatenation,

\[
I^{\mathcal{S}} \in \mathbb{R}^{(N \cdot H') \times W' \times 3}.
\]

The \(i\)-th block corresponds to subject \(s_i\) [2510.06469].

The spatial branch decomposes control into **routing** and **structure**. Let the desired subject regions be \(\mathcal{R}=\{R_i\}_{i=1}^N\). A subject-to-intensity mapping

\[
f: \mathbf{S} \longrightarrow \mathbb{M}
\]

assigns each subject a unique intensity value, and the routing image is defined as

\[
I^{\mathcal{R}}(x) = f(s_i)\,\mathbf{1}[x \in R_i].
\]

Thus, the routing map encodes not only occupancy but also subject identity at the region level [2510.06469].

The final spatial control image is

\[
I^C = \text{Concat}\!\left(I^{\mathcal{R}_{\mathrm{asc}}}, I^{\mathcal{R}_{\mathrm{dsc}}}, I^T; \text{channel}\right) \in \mathbb{R}^{H \times W \times 3},
\]

where \(I^{\mathcal{R}_{\mathrm{asc}}}\) and \(I^{\mathcal{R}_{\mathrm{dsc}}}\) are routing images composited in ascending and descending order, and \(I^T\) is the structure image, described in the paper as depth. The dual routing channels are a specific mechanism for coarse controls: when subject boxes overlap, one ordering may hide a subject that the reverse ordering reveals [2510.06469].

Architecturally, the method uses RoPE positional embeddings for spatial control in the same way as the noisy image. For the identity control, it also uses RoPE, but with the first dimension set to ones instead of zeros, following Flux Kontext. This provides a lightweight distinction between identity tokens and noisy-image tokens inside the shared transformer [2510.06469].

## 3. Identity preservation and structural grounding

SIGMA-Gen represents identity through **reference RGB images**, not through a separate face-identity encoder, contrastive branch, or learned textual token per subject. Identity preservation is therefore an in-context visual conditioning problem: the transformer sees stacked identity exemplars and region-level routing codes jointly, and the paper argues that this allows the model to learn the correspondence between identity slot \(i\) in \(I^{\mathcal{S}}\), routing region \(R_i\), and the output subject instance [2510.06469].

This representation also supports multiple granularities of structural control within one model. The paper explicitly discusses support for **precise segmentation masks**, **per-pixel depth maps**, **2D bounding boxes**, and **3D bounding boxes**. With precise masks, the ascending and descending routing channels are identical. With 2D boxes, the routing channels disambiguate overlap. With 3D boxes, the model receives both a mask-like occupancy cue and a coarse depth signal. The appendix also shows mixed-control inference, in which different subjects in the same scene are specified with different control granularities [2510.06469].

A notable empirical interpretation offered by the paper is that depth is not merely a weak placement hint. By changing depth control while keeping the identity image fixed, the model can **repose deformable subjects**, which suggests that the learned representation partially decouples identity from scene geometry [2510.06469].

## 4. Training regime and SIGMA-Set27K

SIGMA-Gen is trained with **SIGMA-Set27K**, a synthetic dataset introduced to provide aligned supervision for multi-subject identity preservation under spatial control. The dataset contains about **27k images**; the appendix gives the more specific count **26,435 images** and **105,756 unique identities**, with up to **10 subjects per image** [2510.06469].

The construction pipeline is fully synthetic. For each target image, the paper describes the following process: an LLM generates a compositional image prompt, subject captions, and a background caption; an off-the-shelf text-to-image model synthesizes the target image; grounded segmentation extracts subject masks; a depth model predicts target-image depth; **Flux.1-Kontext-dev** reposes each subject crop to create an identity image; and **2D** and **3D bounding boxes** are then fitted to each segmented subject [2510.06469].

The appendix identifies the principal tools in this pipeline as **Qwen-3-8B** for prompt generation, **Grounded-Segment-Anything** for segmentation, **MoGe-2** for depth estimation, **Open3D** for oriented bounding boxes, and **Flux.1-Kontext-dev** for reposing identity images. Filtering rules remove boxes smaller than **1%** of image area, boxes larger than **40%**, and duplicate or overlapping masks; retained samples contain more than **2 subjects per image**. The paper also states that early training is supplemented with processed data from **AnyInsert** and **MUSAR-Gen** for single- and double-subject cases [2510.06469].

Training is performed with **LoRA**, using **rank 128** and **alpha 128**, on **8 A100 GPUs** with batch size **8**. The schedule is three-stage: **30k steps** on a subset with up to four subjects per image, then **20k steps** on images with three or more subjects, then **20k steps** on images with more than four subjects. During training, each example randomly uses one of three control types: precise masks with depth, 3D bounding-box masks with depth, or 2D boxes. The paper also applies random dropping of one spatial condition channel with probability **0.1**, random dilation of masks and boxes, and **1%** aspect-ratio variation of boxes [2510.06469].

The prompt format alternates between a background-only form, “Place these subjects in \<bg prompt\>”, and a full compositional form, “Place these subjects to compose: \<full prompt\>”, with equal probability. For depth preprocessing, only subject depths are retained and non-subject pixels are zeroed out, so the method does not require full-scene depth at inference time [2510.06469].

## 5. Evaluation and empirical results

The evaluation set contains **710 examples**, comprising **200 single-subject** and **510 multi-subject** cases, with **2102 unique identities** in total. Identity preservation is evaluated with **DINO-I** and **SigLIP-I**; overall composition and prompt adherence with **SigLIP-T**; structural adherence with **Depth MSE**; and perceptual quality with **CLIP-IQA** and **MUSIQ** [2510.06469].

In the main **multi-subject, precise mask + depth** setting, the comparison baseline is iterative **Insert Anything\***. The reported numbers are: **DINO-I 74.54 vs 72.72**, **SigLIP-I 77.82 vs 75.58**, **SigLIP-T 17.73 vs 17.66**, **Depth MSE 26.35 vs 203.4**, **CLIP-IQA 72.64 vs 44.41**, and **MUSIQ 73.21 vs 48.86**. The depth and perceptual-quality gaps are especially large [2510.06469].

In **multi-subject box control**, the main comparator is **MSDiffusion**. SIGMA-Gen reports **DINO-I 71.90 vs 63.28**, **SigLIP-I 73.15 vs 69.06**, **SigLIP-T 17.21 vs 11.20**, **CLIP-IQA 68.83 vs 61.99**, and **MUSIQ 70.96 vs 69.05**. For **mask + 3D bbox**, the paper reports **DINO-I 73.48**, **SigLIP-I 75.27**, **SigLIP-T 18.19**, **CLIP-IQA 72.45**, and **MUSIQ 72.55**, indicating that 3D box plus depth improves over 2D box-only control [2510.06469].

The paper also summarizes its headline gains as follows. With per-pixel depth and masks, it reports a **31-point** improvement in overall image fidelity, a **2-point** improvement in identity preservation, and **4× faster** generation for scenes with **5+ subjects**. With bounding boxes, it reports a **6-point** improvement in overall fidelity and an **11-point** improvement in identity preservation for scenes with **5+ subjects** [2510.06469].

Ablations indicate that stronger guidance improves results. In particular, **mask + depth** outperforms **mask only**, and **mask + depth + full prompt** performs best among the tested variants. Another ablation compares **subject-only depth** to **full depth** and to using full depth through an external ControlNet. Full depth can slightly improve text alignment and quality, but subject consistency drops slightly relative to subject-only depth; external ControlNet depth hurts subject consistency and text alignment relative to the native SIGMA-Gen representation [2510.06469].

## 6. Limitations, interpretation, and disambiguation

SIGMA-Gen’s novelty lies in the conjunction of properties rather than in a single new conditioning primitive: **multiple identities**, **multiple control granularities**, **single-pass generation**, and **one unified model**. The paper positions it against prior personalization methods, structure-control methods, and iterative insertion systems precisely on that combined axis [2510.06469].

The paper also states several concrete limitations. Under **very high overlap** with coarse controls, a subject may be ignored. **Large viewpoint changes** between the identity image and the target pose can reduce identity consistency. Human facial identity is reported as weaker, which the paper attributes to the fact that the training data were not specifically designed for that task. A further implementation-level limitation is that the paper does **not explicitly provide** the diffusion noising equation, the denoising or flow-matching objective, an explicit identity-preservation loss, an explicit structure loss, or a total loss formula [2510.06469].

The name should also be distinguished from unrelated arXiv uses of **SIGMA**. It is unrelated to **“Stochastic Iterative Graph MAtching”** [2106.02206], **“SIGMA: A Physics-Based Benchmark for Gas Chimney Understanding in Seismic Images”** [2603.23439], **“SIGMA: Bridging Structural and Distributional Gaps for Vision Foundation Model Adaptation”** [2605.27893], and **“SIGMA: Skill-Incidence Graphs for Compositional Multi-Agent Design”** [2606.19758]. In the context of image generation, **SIGMA-Gen** denotes specifically the **Structure and Identity Guided Multi-subject Assembly** framework and its associated dataset **SIGMA-Set27K** [2510.06469].

Source: https://www.emergentmind.com/topics/sigma-gen