Subject-Specific Generators in Adaptive Modeling

• Subject-specific generators (SSG) are models that condition on individual traits to generate outputs preserving unique subject identity beyond generic population averages.
• The methodology employs a shared backbone with lightweight subject-specific adaptations through low-rank corrections, specialized encoders, or explicit subject embeddings.
• Applications span EEG decoding, gaze modeling, and subject-driven image/video synthesis, offering enhanced performance and individualized adaptation.

Subject-specific Generator (SSG) denotes a class of methods that condition a model on the identity, latent traits, or explicit reference of a particular subject so that outputs preserve subject-specific structure rather than only population averages. In current arXiv usage, the term does not denote a single canonical architecture: in EEG and gaze modeling it often refers to subject-conditioned latent mappings or signal generators, while in subject-driven image synthesis it refers to systems that preserve the identity of a person, object, or style exemplar under new prompts or contexts (Klein et al., 9 Oct 2025, Hasan et al., 13 Nov 2025, Chen et al., 2023, Shin et al., 2024). Across these settings, the recurring principle is a decomposition between shared task structure and individualized adaptation, whether implemented as low-rank parameter corrections, subject-specific encoders, in-context conditioning, or explicit subject embeddings.

1. Conceptual formulation

A common formalization treats subject specificity as conditioning on a subject variable $s$. In EEG decoding, pooled data are written as

$\mathcal{D}=\bigcup_{s=1}^N \mathcal{D}_s,$

and the central difficulty is that a subject-agnostic model implicitly learns $p(\mathcal{Y}\mid \mathcal{X})$, whereas the relevant object is the family of subject-conditioned distributions $p(\mathcal{Y}\mid \mathcal{X}, s)$. The subject-conditioned latent-map view introduces a representation $\mathcal{Z}=\xi(\mathcal{X}, s)$ such that

$$p(\mathcal{Y}\mid \mathcal{X}, s)=p(\mathcal{Y}\mid \mathcal{Z}, s)\,p(\mathcal{Z}\mid \mathcal{X}, s),$$

with the crucial assumption

$$p(\mathcal{Y}\mid \mathcal{Z}, s)=p(\mathcal{Y}\mid \mathcal{Z}),$$

so the latent space is label-sufficient and subject-invariant for prediction, even though the mapping into that latent space is subject-conditioned (Klein et al., 9 Oct 2025).

In individualized gaze synthesis, the same idea is stated generatively. A subject-specific generator is modeled as

$$G:\mathcal{Z}\times\mathcal{C}\times\mathcal{S}\to\mathcal{X},$$

where $\mathcal{Z}$ is a noise space, $\mathcal{C}$ is optional task or stimulus conditioning, $\mathcal{D}=\bigcup_{s=1}^N \mathcal{D}_s,$0 is subject information, and $\mathcal{D}=\bigcup_{s=1}^N \mathcal{D}_s,$1 is the space of gaze sequences. Subject specificity then means that $\mathcal{D}=\bigcup_{s=1}^N \mathcal{D}_s,$2 and $\mathcal{D}=\bigcup_{s=1}^N \mathcal{D}_s,$3 are distinct and reflect idiosyncratic user traits rather than only generic realism (Hasan et al., 13 Nov 2025).

This suggests a broad operational definition: an SSG is a model that learns or enforces subject-conditioned transformations while retaining a shared decision rule, decoder, or generative prior. The “generator” may therefore generate images, biosignals, latent representations, parameter corrections, or subject-adapted feature structures, depending on the domain.

2. Core architectural patterns

A dominant pattern is shared backbone plus subject-specific adaptation. In EEG decoding, the Subject-Conditioned Layer replaces a standard linear or convolutional layer by decomposing the effective weight into a shared component $\mathcal{D}=\bigcup_{s=1}^N \mathcal{D}_s,$4 and a subject-specific low-rank correction $\mathcal{D}=\bigcup_{s=1}^N \mathcal{D}_s,$5, with

$\mathcal{D}=\bigcup_{s=1}^N \mathcal{D}_s,$6

The effective subject-conditioned weight is $\mathcal{D}=\bigcup_{s=1}^N \mathcal{D}_s,$7. The correction is additive, low-rank, and selected deterministically by the subject index through batch masks $\mathcal{D}=\bigcup_{s=1}^N \mathcal{D}_s,$8 (Klein et al., 9 Oct 2025).

A closely related pattern appears in cross-subject EEG alignment via subject-specific encoders. Instead of one shared encoder, each subject has its own encoder $\mathcal{D}=\bigcup_{s=1}^N \mathcal{D}_s,$9, but all subject encoders feed a common classifier $p(\mathcal{Y}\mid \mathcal{X})$0: $p(\mathcal{Y}\mid \mathcal{X})$1 Here the shared classifier enforces a common decision geometry, while the subject-specific encoder bank learns alignment into that geometry (Lopes et al., 15 Jun 2026).

Neural encoding of fMRI responses instantiates the same principle with a different output space. A shared feature extractor $p(\mathcal{Y}\mid \mathcal{X})$2 and shared volumetric mapper $p(\mathcal{Y}\mid \mathcal{X})$3 produce a common latent brain representation $p(\mathcal{Y}\mid \mathcal{X})$4, after which a subject-specific mapper $p(\mathcal{Y}\mid \mathcal{X})$5 predicts the response volume for subject $p(\mathcal{Y}\mid \mathcal{X})$6: $p(\mathcal{Y}\mid \mathcal{X})$7 Most capacity is assigned to the shared stimulus-to-latent mapping, while only a smaller individualized readout is subject-specific (Khosla et al., 2020).

An alternative pattern is population-optimized structure with subject-specific instantiation. In SSVEP decoding, subject-independent data from 30 subjects are used to optimize a universal feature structure, and then a new subject-specific CCA model is instantiated only by recomputing templates $p(\mathcal{Y}\mid \mathcal{X})$8 and applying the fixed six-feature structure

$p(\mathcal{Y}\mid \mathcal{X})$9

The paper reports that across all seven folds the same set of six features emerged as optimal, which functions as a population-derived prior for subject-specific decoding (Mehdizavareh et al., 2019).

These architectures differ in mechanics, but they converge on the same design rule: preserve a shared task model, and localize personalization to a controlled interface.

3. Biosignal SSGs: EEG decoding and gaze synthesis

In EEG decoding, subject-specific distribution shifts are treated as “the primary obstacle” to cross-subject foundation models. The Subject-Conditioned Layer addresses this by making the feature map $p(\mathcal{Y}\mid \mathcal{X}, s)$0 adaptive per subject while keeping the classifier in latent space shared. The low-rank constraint and LoRA-style scaling $p(\mathcal{Y}\mid \mathcal{X}, s)$1 restrict subject-specific components to “small deviations” rather than allowing a full subject-wise relearning of the network (Klein et al., 9 Oct 2025).

The empirical result reported for BCI Competition IV 2a illustrates the effect. With EEGNeX, the subject-agnostic model reached $p(\mathcal{Y}\mid \mathcal{X}, s)$2, the mean of separately trained subject-specific models reached $p(\mathcal{Y}\mid \mathcal{X}, s)$3, subject-specific LoRA models reached $p(\mathcal{Y}\mid \mathcal{X}, s)$4, and the Subject-Conditioned Layer reached $p(\mathcal{Y}\mid \mathcal{X}, s)$5. In this configuration, the subject-conditioned model exceeded both the pooled subject-agnostic baseline and the average of fully individualized models (Klein et al., 9 Oct 2025).

A second EEG line of work studies subject-specific encoders as learned alignment mechanisms. Euclidean Alignment whitens each subject by recentering mean covariances, but the hybrid encoder with subject-specific heads and a common classifier changes little when Euclidean Alignment is removed: validation-loss curves and latent-distance analyses are nearly unchanged, which supports the claim that the encoder bank internalizes alignment. The same study concludes that head selection for unseen subjects remains the main bottleneck (Lopes et al., 15 Jun 2026).

In gaze synthesis, the diffusion-based SSG and GAN-based SSG implement two different personalization strategies. The diffusion model conditions on an identity-removed base signal $p(\mathcal{Y}\mid \mathcal{X}, s)$6, a compact $p(\mathcal{Y}\mid \mathcal{X}, s)$7-dimensional EKYT embedding $p(\mathcal{Y}\mid \mathcal{X}, s)$8, and an identity guidance loss

$p(\mathcal{Y}\mid \mathcal{X}, s)$9

Its DDPM uses $\mathcal{Z}=\xi(\mathcal{X}, s)$0 steps with a linear variance schedule from $\mathcal{Z}=\xi(\mathcal{X}, s)$1 to $\mathcal{Z}=\xi(\mathcal{X}, s)$2. The GAN alternative conditions on one-hot subject identity and hand-crafted quality features through a subject-specific synthesis module built from a directional data quality feature extractor and a one-hot encoder (Hasan et al., 13 Nov 2025).

The quantitative contrast is sharp. Across tasks BLG, FXS, HSS, RAN, TEX, VD1, and VD2, the diffusion model achieved cosine similarities between $\mathcal{Z}=\xi(\mathcal{X}, s)$3 and $\mathcal{Z}=\xi(\mathcal{X}, s)$4, whereas the subject-specific GAN achieved values between $\mathcal{Z}=\xi(\mathcal{X}, s)$5 and $\mathcal{Z}=\xi(\mathcal{X}, s)$6. In that study, diffusion-based subject conditioning retained individualized gaze structure much more effectively than adversarial conditioning by labels and quality features alone (Hasan et al., 13 Nov 2025).

4. Subject-driven image and video generation

In vision, SSG usually denotes subject-driven generation: preserving the identity of a specific person, object, or style exemplar while changing context, pose, background, or editing instructions. A central distinction is between optimization-based personalization and zero-shot or in-context conditioning.

SuTI replaces per-subject fine-tuning with in-context learning. It trains a single “apprentice” diffusion model to imitate a massive number of subject-specific “expert” models, each expert being fine-tuned on a mined subject cluster. The pipeline starts from approximately $\mathcal{Z}=\xi(\mathcal{X}, s)$7 million URL-based image clusters, filters to roughly $\mathcal{Z}=\xi(\mathcal{X}, s)$8 million clusters with at least three images, and then constructs a large subject-centered dataset with refined captions and “imaginary prompts.” Expert-generated samples are filtered by a delta CLIP criterion

$\mathcal{Z}=\xi(\mathcal{X}, s)$9

with threshold $$p(\mathcal{Y}\mid \mathcal{X}, s)=p(\mathcal{Y}\mid \mathcal{Z}, s)\,p(\mathcal{Z}\mid \mathcal{X}, s),$$0, reducing the effective apprentice-training set from roughly $$p(\mathcal{Y}\mid \mathcal{X}, s)=p(\mathcal{Y}\mid \mathcal{Z}, s)\,p(\mathcal{Z}\mid \mathcal{X}, s),$$1 million to roughly $$p(\mathcal{Y}\mid \mathcal{X}, s)=p(\mathcal{Y}\mid \mathcal{Z}, s)\,p(\mathcal{Z}\mid \mathcal{X}, s),$$2 thousand clusters. The resulting apprentice uses a $$p(\mathcal{Y}\mid \mathcal{X}, s)=p(\mathcal{Y}\mid \mathcal{Z}, s)\,p(\mathcal{Z}\mid \mathcal{X}, s),$$3 Imagen backbone of $$p(\mathcal{Y}\mid \mathcal{X}, s)=p(\mathcal{Y}\mid \mathcal{Z}, s)\,p(\mathcal{Z}\mid \mathcal{X}, s),$$4B parameters plus about $$p(\mathcal{Y}\mid \mathcal{X}, s)=p(\mathcal{Y}\mid \mathcal{Z}, s)\,p(\mathcal{Z}\mid \mathcal{X}, s),$$5M new parameters, and the method is reported as $$p(\mathcal{Y}\mid \mathcal{X}, s)=p(\mathcal{Y}\mid \mathcal{Z}, s)\,p(\mathcal{Z}\mid \mathcal{X}, s),$$6 faster than optimization-based subject customization while outperforming DreamBooth, Textual Inversion, Imagic, Prompt2Prompt, Re-Imagen, and InstructPix2Pix in human evaluation on DreamBench and DreamBench-v2, especially on subject and text alignment (Chen et al., 2023).

Diptych Prompting shows that zero-shot subject-driven generation can be reframed as inpainting. A segmented reference image occupies the left half of a $$p(\mathcal{Y}\mid \mathcal{X}, s)=p(\mathcal{Y}\mid \mathcal{Z}, s)\,p(\mathcal{Z}\mid \mathcal{X}, s),$$7 canvas and the right half is masked for synthesis, with

$$p(\mathcal{Y}\mid \mathcal{X}, s)=p(\mathcal{Y}\mid \mathcal{Z}, s)\,p(\mathcal{Z}\mid \mathcal{X}, s),$$8

The method uses FLUX.1-dev with ControlNet-Inpainting, removes the background in the reference image to prevent content leakage, and scales cross-panel reference attention $$p(\mathcal{Y}\mid \mathcal{X}, s)=p(\mathcal{Y}\mid \mathcal{Z}, s)\,p(\mathcal{Z}\mid \mathcal{X}, s),$$9 by a factor $$p(\mathcal{Y}\mid \mathcal{Z}, s)=p(\mathcal{Y}\mid \mathcal{Z}),$$0. In the ablation, $$p(\mathcal{Y}\mid \mathcal{Z}, s)=p(\mathcal{Y}\mid \mathcal{Z}),$$1 improved DINO from $$p(\mathcal{Y}\mid \mathcal{Z}, s)=p(\mathcal{Y}\mid \mathcal{Z}),$$2 to $$p(\mathcal{Y}\mid \mathcal{Z}, s)=p(\mathcal{Y}\mid \mathcal{Z}),$$3, CLIP-I from $$p(\mathcal{Y}\mid \mathcal{Z}, s)=p(\mathcal{Y}\mid \mathcal{Z}),$$4 to $$p(\mathcal{Y}\mid \mathcal{Z}, s)=p(\mathcal{Y}\mid \mathcal{Z}),$$5, and CLIP-T from $$p(\mathcal{Y}\mid \mathcal{Z}, s)=p(\mathcal{Y}\mid \mathcal{Z}),$$6 to $$p(\mathcal{Y}\mid \mathcal{Z}, s)=p(\mathcal{Y}\mid \mathcal{Z}),$$7, whereas $$p(\mathcal{Y}\mid \mathcal{Z}, s)=p(\mathcal{Y}\mid \mathcal{Z}),$$8 degraded all three relative to $$p(\mathcal{Y}\mid \mathcal{Z}, s)=p(\mathcal{Y}\mid \mathcal{Z}),$$9. The background-removal ablation also showed a specific fidelity-versus-copying trade-off: keeping the original background gave DINO $$G:\mathcal{Z}\times\mathcal{C}\times\mathcal{S}\to\mathcal{X},$$0 and CLIP-I $$G:\mathcal{Z}\times\mathcal{C}\times\mathcal{S}\to\mathcal{X},$$1 but CLIP-T only $$G:\mathcal{Z}\times\mathcal{C}\times\mathcal{S}\to\mathcal{X},$$2, while background removal plus attention enhancement yielded DINO $$G:\mathcal{Z}\times\mathcal{C}\times\mathcal{S}\to\mathcal{X},$$3, CLIP-I $$G:\mathcal{Z}\times\mathcal{C}\times\mathcal{S}\to\mathcal{X},$$4, and CLIP-T $$G:\mathcal{Z}\times\mathcal{C}\times\mathcal{S}\to\mathcal{X},$$5, indicating less trivial mirroring and better textual adaptation (Shin et al., 2024).

SSR-Encoder addresses the problem of selectively capturing the intended subject from one or more reference images without test-time fine-tuning. Its Token-to-Patch Aligner computes

$$G:\mathcal{Z}\times\mathcal{C}\times\mathcal{S}\to\mathcal{X},$$6

aligning query tokens or masks to CLIP image patches. A multi-scale subject encoder then forms subject embeddings

$$G:\mathcal{Z}\times\mathcal{C}\times\mathcal{S}\to\mathcal{X},$$7

which are concatenated and injected through a parallel subject cross-attention branch in the diffusion U-Net: $$G:\mathcal{Z}\times\mathcal{C}\times\mathcal{S}\to\mathcal{X},$$8 Training adds an embedding-consistency regularizer with weight $$G:\mathcal{Z}\times\mathcal{C}\times\mathcal{S}\to\mathcal{X},$$9, and the method is described as compatible with custom diffusion models and control modules such as ControlNet and AnimateDiff (Zhang et al., 2023).

Recent work also generalizes subject conditioning beyond single-task subject editing. 3SGen introduces an Adaptive Task-specific Memory that stores subject, style, and structure priors in separate memory items, retrieves them through gated cross-attention, and combines them with an MLLM semantic bridge and VAE detail branch, thereby separating identity from texture and layout within one model (Song et al., 22 Dec 2025). In video generation, SSG-DiT uses CLIP-derived spatial masks from an initial image and prompt, then injects a fused text-plus-visual condition into a frozen video DiT through a dual-branch SSG-Adapter, reporting state-of-the-art VBench performance particularly in spatial relationship control and overall consistency (Hu et al., 23 Aug 2025).

5. Training regimes, objectives, and evaluation

SSG methods differ most visibly in how they obtain subject specificity. One regime is per-subject optimization, exemplified by DreamBooth-style experts in SuTI’s apprenticeship pipeline (Chen et al., 2023). Another is training-time personalization with shared weights, as in Subject-Conditioned EEG layers trained jointly with the backbone from scratch using standard AdamW and classification loss, without meta-learning or alternating optimization (Klein et al., 9 Oct 2025). A third is zero-shot inference-time conditioning, as in Diptych Prompting and SSR-Encoder, where no per-subject parameter update is performed at test time (Shin et al., 2024, Zhang et al., 2023).

The loss functions reflect the output domain. In gaze diffusion, the objective combines DDPM noise prediction with an identity-preserving cosine loss in biometric embedding space (Hasan et al., 13 Nov 2025). In SSR-Encoder, the latent diffusion objective is augmented by cosine-based embedding consistency between subject embeddings and query embeddings (Zhang et al., 2023). In EEG decoding, the objective remains standard supervised classification loss, but subject-specificity is imposed structurally through routed adapters or subject-specific heads rather than through an auxiliary identity loss (Klein et al., 9 Oct 2025, Lopes et al., 15 Jun 2026).

Evaluation is correspondingly heterogeneous. Subject-driven image generation frequently uses identity and text alignment metrics such as DINO, CLIP-I, and CLIP-T, together with human evaluation on DreamBench-style protocols (Shin et al., 2024, Chen et al., 2023). EEG studies emphasize balanced accuracy or classification accuracy under leave-one-subject-out or cross-subject protocols, and they increasingly supplement these scores with latent-space analyses such as t-SNE or covariance-distance geometry (Klein et al., 9 Oct 2025, Lopes et al., 15 Jun 2026). Gaze synthesis uses eye-tracking signal-quality criteria such as spatial accuracy and precision, alongside subject-specific embedding similarity (Hasan et al., 13 Nov 2025).

A plausible implication is that no single benchmark currently spans the full SSG design space. Image papers evaluate subject fidelity and prompt obedience; biosignal papers evaluate individual-specific decoding or synthesis realism; cross-subject adaptation papers evaluate robustness to distribution shift. Recent unified image-driven work therefore proposes benchmarks such as 3SGen-Bench to standardize cross-task fidelity and controllability (Song et al., 22 Dec 2025).

6. Misconceptions, limitations, and terminological ambiguity

A common misconception is that subject specificity requires a full model per subject. The literature does not support that claim. Low-rank subject-conditioned layers add only lightweight corrections on top of shared weights (Klein et al., 9 Oct 2025); SuTI performs subject-driven generation by in-context conditioning instead of test-time fine-tuning (Chen et al., 2023); Diptych Prompting and SSR-Encoder are explicitly zero-shot (Shin et al., 2024, Zhang et al., 2023). Conversely, another misconception is that stronger subject fidelity should maximize literal copying. Diptych Prompting shows that high DINO and CLIP-I can coexist with poor CLIP-T when the background leaks from the reference image, so faithful subject preservation and faithful instruction following are not identical objectives (Shin et al., 2024).

A recurrent technical limitation is selection or conditioning for unseen subjects. Hard subject routing works well when the subject index is known, but the EEG adapter study explicitly notes that learned subject embeddings and a generator network for adapter weights remain future extensions beyond deterministic masking (Klein et al., 9 Oct 2025). The subject-specific encoder study likewise identifies head selection for unseen subjects as the remaining bottleneck, even when latent alignment is otherwise effective (Lopes et al., 15 Jun 2026).

The term itself is also ambiguous. Several arXiv papers use the acronym “SSG” for methods that are not subject-specific generators in this sense: “Sequential Set Generation” for set-valued prediction (Gao et al., 2019), “SVM–SMOTE–GAN” for imbalanced-learning oversampling (Ahsan et al., 2022), “Scaled Spatial Guidance” for multi-scale visual autoregressive generation (Shin et al., 5 Feb 2026), and “Sort-then-Split by Groups” for LLM watermarking (Gu et al., 24 Apr 2026). Even “SSG-DiT” expands to “Spatial Signal Guided Diffusion Transformer,” not “Subject-Specific Generator,” although its mechanism is still subject-centric in the sense of conditioning video generation on a particular initial subject and spatial prompt (Hu et al., 23 Aug 2025). The literature therefore treats SSG less as a fixed named model family than as a recurring design pattern: subject-conditioned generation or adaptation built on top of a shared model.