SIGMA-Gen: Multi-Subject Identity-Preserving Generation

• SIGMA-Gen is a unified framework that preserves multiple subject identities in one image by leveraging explicit spatial and structural constraints.
• It uses a diffusion-transformer backbone with a compact joint representation to integrate identity cues from reference images and spatial controls from masks, boxes, and depth maps.
• The system achieves higher fidelity and 4× faster generation for scenes with multiple subjects, addressing limitations of previous iterative and separate control methods.

SIGMA-Gen is a controllable image-generation framework for multi-subject identity-preserving generation under explicit spatial and structural constraints. It is presented as a unified system that generates several specified subjects in a single image while preserving each subject’s identity and obeying controls that range from 2D bounding boxes and 3D bounding boxes to pixel-level segmentations and depth. The method is built on a diffusion-transformer backbone, uses a compact joint representation of identity and structure, and is trained with the synthetic dataset SIGMA-Set27K (Saha et al., 7 Oct 2025).

1. Scope and task definition

The task addressed by SIGMA-Gen is: given a text prompt $P$, a set of subjects $\mathbf{S}=\{s_1,\dots,s_n\}$, and spatial or structural controls $C$, generate an image $I$ such that all subject identities are preserved and the subjects are placed according to $C$ (Saha et al., 7 Oct 2025).

The framework is motivated by a conjunction of two control problems that earlier systems typically treat separately. The first is identity control, in which a reference image specifies who or what should appear. The second is arrangement or structure control, in which the user specifies where subjects should go and, in stronger forms, how they should be posed or ordered in depth. The paper argues that prior personalization methods can learn identities but usually require per-subject optimization, while structure-control methods such as ControlNet, T2I-Adapter, and GLIGEN can follow boxes, edges, or depth but do not preserve multiple identities. Iterative insertion methods can sometimes be adapted to multiple subjects, but the paper identifies increasing runtime, quality degradation, and compounded identity and layout failures as their main liabilities (Saha et al., 7 Oct 2025).

Within that framing, SIGMA-Gen is presented as the first single-pass method for multi-subject identity-preserving generation with fine-to-coarse structural control. The emphasis on “single-pass” is central: all subjects are synthesized in one denoising process rather than added sequentially (Saha et al., 7 Oct 2025).

2. Conditioning representation and model architecture

SIGMA-Gen uses three principal conditioning streams: the prompt $P$, an identity control image $I^{\mathcal{S}}$, and a spatial control image $I^C$. The backbone is Flux.1 Kontext [dev], described as a diffusion transformer. The method uses the backbone’s VAE to encode the identity control image and the spatial control image, then concatenates all modalities as tokens,

$[P, X, I^{\mathcal{S}}, I^C],$

where $X$ denotes the noisy image latents. This creates a unified attention space in which image latents, prompt tokens, identity references, and structural controls can attend to one another (Saha et al., 7 Oct 2025).

The identity branch uses one RGB reference image $\mathbf{S}=\{s_1,\dots,s_n\}$0 per subject. If each identity image is

$\mathbf{S}=\{s_1,\dots,s_n\}$1

then the full identity condition image is formed by vertical concatenation,

$\mathbf{S}=\{s_1,\dots,s_n\}$2

The $\mathbf{S}=\{s_1,\dots,s_n\}$3-th block corresponds to subject $\mathbf{S}=\{s_1,\dots,s_n\}$4 (Saha et al., 7 Oct 2025).

The spatial branch decomposes control into routing and structure. Let the desired subject regions be $\mathbf{S}=\{s_1,\dots,s_n\}$5. A subject-to-intensity mapping

$\mathbf{S}=\{s_1,\dots,s_n\}$6

assigns each subject a unique intensity value, and the routing image is defined as

$\mathbf{S}=\{s_1,\dots,s_n\}$7

Thus, the routing map encodes not only occupancy but also subject identity at the region level (Saha et al., 7 Oct 2025).

The final spatial control image is

$\mathbf{S}=\{s_1,\dots,s_n\}$8

where $\mathbf{S}=\{s_1,\dots,s_n\}$9 and $C$0 are routing images composited in ascending and descending order, and $C$1 is the structure image, described in the paper as depth. The dual routing channels are a specific mechanism for coarse controls: when subject boxes overlap, one ordering may hide a subject that the reverse ordering reveals (Saha et al., 7 Oct 2025).

Architecturally, the method uses RoPE positional embeddings for spatial control in the same way as the noisy image. For the identity control, it also uses RoPE, but with the first dimension set to ones instead of zeros, following Flux Kontext. This provides a lightweight distinction between identity tokens and noisy-image tokens inside the shared transformer (Saha et al., 7 Oct 2025).

3. Identity preservation and structural grounding

SIGMA-Gen represents identity through reference RGB images, not through a separate face-identity encoder, contrastive branch, or learned textual token per subject. Identity preservation is therefore an in-context visual conditioning problem: the transformer sees stacked identity exemplars and region-level routing codes jointly, and the paper argues that this allows the model to learn the correspondence between identity slot $C$2 in $C$3, routing region $C$4, and the output subject instance (Saha et al., 7 Oct 2025).

This representation also supports multiple granularities of structural control within one model. The paper explicitly discusses support for precise segmentation masks, per-pixel depth maps, 2D bounding boxes, and 3D bounding boxes. With precise masks, the ascending and descending routing channels are identical. With 2D boxes, the routing channels disambiguate overlap. With 3D boxes, the model receives both a mask-like occupancy cue and a coarse depth signal. The appendix also shows mixed-control inference, in which different subjects in the same scene are specified with different control granularities (Saha et al., 7 Oct 2025).

A notable empirical interpretation offered by the paper is that depth is not merely a weak placement hint. By changing depth control while keeping the identity image fixed, the model can repose deformable subjects, which suggests that the learned representation partially decouples identity from scene geometry (Saha et al., 7 Oct 2025).

4. Training regime and SIGMA-Set27K

SIGMA-Gen is trained with SIGMA-Set27K, a synthetic dataset introduced to provide aligned supervision for multi-subject identity preservation under spatial control. The dataset contains about 27k images; the appendix gives the more specific count 26,435 images and 105,756 unique identities, with up to 10 subjects per image (Saha et al., 7 Oct 2025).

The construction pipeline is fully synthetic. For each target image, the paper describes the following process: an LLM generates a compositional image prompt, subject captions, and a background caption; an off-the-shelf text-to-image model synthesizes the target image; grounded segmentation extracts subject masks; a depth model predicts target-image depth; Flux.1-Kontext-dev reposes each subject crop to create an identity image; and 2D and 3D bounding boxes are then fitted to each segmented subject (Saha et al., 7 Oct 2025).

The appendix identifies the principal tools in this pipeline as Qwen-3-8B for prompt generation, Grounded-Segment-Anything for segmentation, MoGe-2 for depth estimation, Open3D for oriented bounding boxes, and Flux.1-Kontext-dev for reposing identity images. Filtering rules remove boxes smaller than 1% of image area, boxes larger than 40%, and duplicate or overlapping masks; retained samples contain more than 2 subjects per image. The paper also states that early training is supplemented with processed data from AnyInsert and MUSAR-Gen for single- and double-subject cases (Saha et al., 7 Oct 2025).

Training is performed with LoRA, using rank 128 and alpha 128, on 8 A100 GPUs with batch size 8. The schedule is three-stage: 30k steps on a subset with up to four subjects per image, then 20k steps on images with three or more subjects, then 20k steps on images with more than four subjects. During training, each example randomly uses one of three control types: precise masks with depth, 3D bounding-box masks with depth, or 2D boxes. The paper also applies random dropping of one spatial condition channel with probability 0.1, random dilation of masks and boxes, and 1% aspect-ratio variation of boxes (Saha et al., 7 Oct 2025).

The prompt format alternates between a background-only form, “Place these subjects in <bg prompt>”, and a full compositional form, “Place these subjects to compose: <full prompt>”, with equal probability. For depth preprocessing, only subject depths are retained and non-subject pixels are zeroed out, so the method does not require full-scene depth at inference time (Saha et al., 7 Oct 2025).

5. Evaluation and empirical results

The evaluation set contains 710 examples, comprising 200 single-subject and 510 multi-subject cases, with 2102 unique identities in total. Identity preservation is evaluated with DINO-I and SigLIP-I; overall composition and prompt adherence with SigLIP-T; structural adherence with Depth MSE; and perceptual quality with CLIP-IQA and MUSIQ (Saha et al., 7 Oct 2025).

In the main multi-subject, precise mask + depth setting, the comparison baseline is iterative Insert Anything*. The reported numbers are: DINO-I 74.54 vs 72.72, SigLIP-I 77.82 vs 75.58, SigLIP-T 17.73 vs 17.66, Depth MSE 26.35 vs 203.4, CLIP-IQA 72.64 vs 44.41, and MUSIQ 73.21 vs 48.86. The depth and perceptual-quality gaps are especially large (Saha et al., 7 Oct 2025).

In multi-subject box control, the main comparator is MSDiffusion. SIGMA-Gen reports DINO-I 71.90 vs 63.28, SigLIP-I 73.15 vs 69.06, SigLIP-T 17.21 vs 11.20, CLIP-IQA 68.83 vs 61.99, and MUSIQ 70.96 vs 69.05. For mask + 3D bbox, the paper reports DINO-I 73.48, SigLIP-I 75.27, SigLIP-T 18.19, CLIP-IQA 72.45, and MUSIQ 72.55, indicating that 3D box plus depth improves over 2D box-only control (Saha et al., 7 Oct 2025).

The paper also summarizes its headline gains as follows. With per-pixel depth and masks, it reports a 31-point improvement in overall image fidelity, a 2-point improvement in identity preservation, and 4× faster generation for scenes with 5+ subjects. With bounding boxes, it reports a 6-point improvement in overall fidelity and an 11-point improvement in identity preservation for scenes with 5+ subjects (Saha et al., 7 Oct 2025).

Ablations indicate that stronger guidance improves results. In particular, mask + depth outperforms mask only, and mask + depth + full prompt performs best among the tested variants. Another ablation compares subject-only depth to full depth and to using full depth through an external ControlNet. Full depth can slightly improve text alignment and quality, but subject consistency drops slightly relative to subject-only depth; external ControlNet depth hurts subject consistency and text alignment relative to the native SIGMA-Gen representation (Saha et al., 7 Oct 2025).

6. Limitations, interpretation, and disambiguation

SIGMA-Gen’s novelty lies in the conjunction of properties rather than in a single new conditioning primitive: multiple identities, multiple control granularities, single-pass generation, and one unified model. The paper positions it against prior personalization methods, structure-control methods, and iterative insertion systems precisely on that combined axis (Saha et al., 7 Oct 2025).

The paper also states several concrete limitations. Under very high overlap with coarse controls, a subject may be ignored. Large viewpoint changes between the identity image and the target pose can reduce identity consistency. Human facial identity is reported as weaker, which the paper attributes to the fact that the training data were not specifically designed for that task. A further implementation-level limitation is that the paper does not explicitly provide the diffusion noising equation, the denoising or flow-matching objective, an explicit identity-preservation loss, an explicit structure loss, or a total loss formula (Saha et al., 7 Oct 2025).

The name should also be distinguished from unrelated arXiv uses of SIGMA. It is unrelated to “Stochastic Iterative Graph MAtching” (Liu et al., 2021), “SIGMA: A Physics-Based Benchmark for Gas Chimney Understanding in Seismic Images” (Truong et al., 24 Mar 2026), “SIGMA: Bridging Structural and Distributional Gaps for Vision Foundation Model Adaptation” (Xiong et al., 27 May 2026), and “SIGMA: Skill-Incidence Graphs for Compositional Multi-Agent Design” (Zeng et al., 18 Jun 2026). In the context of image generation, SIGMA-Gen denotes specifically the Structure and Identity Guided Multi-subject Assembly framework and its associated dataset SIGMA-Set27K (Saha et al., 7 Oct 2025).