DomainShuttle: Freeform Open Domain Subject-driven Text-to-video Generation

Abstract: Open domain subject-driven text-to-video (S2V) generation has drawn significant interest in academia and industry. Open domain S2V mainly involves two scenarios: in-domain, which requires retaining the reference subject features as much as possible, and cross-domain, which preserves the intrinsic features of the subject while allowing subject-irrelevant properties to vary flexibly according to the text prompt. Existing methods primarily focus on maximizing subject fidelity in in-domain scenarios, which limits their editability and adaptability in cross-domain scenarios, such as novel styles, semantic combinations, or domain attributes. In this study, we propose that an ideal S2V method should flexibly shuttle between different domains, achieving strong performance in both in-domain and cross-domain scenarios. To this end, we propose DomainShuttle, which could achieve high fidelity and generative flexibility for open domain video personalization. Specifically, we introduce Domain-MoT, which decouples videos and reference features and introduces the domain-aware AdaLN for domain-specific modeling of reference images. We then introduce the Video-Reference DualRoPE scheme, which places reference image tokens and video tokens in separate RoPE spaces to enable precise subject-level spatial modeling, and Cross-Pair Consistent Loss, which aims to extract intrinsic subject features unaffected by irrelevant features. Extensive experiments demonstrate that DomainShuttle achieves significant performance improvements over existing methods, exhibiting high subject fidelity and generative flexibility across diverse open domain application scenarios.

• The paper introduces a novel framework that jointly optimizes subject consistency and generative flexibility using Domain-MoT, DualRoPE, and Cross-Pair Consistent Loss.
• The paper demonstrates a significant 18.7% improvement in cross-domain subject fidelity alongside superior text controllability and video quality.
• The paper leverages a dual-branch architecture and a large-scale personalization corpus to ensure robust subject feature extraction in complex, open-domain scenarios.

DomainShuttle: Freeform Open Domain Subject-driven Text-to-video Generation (2606.26058)

Motivation and Problem Statement

Subject-driven video generation (S2V) has become a foundational paradigm across diverse domains including advertising, creative design, and AI filmmaking, necessitating models that exhibit both high fidelity to user-provided reference subjects and robust flexibility across arbitrary domain types. Existing S2V approaches predominantly address the in-domain scenario—emphasizing identity preservation and fidelity—while demonstrating limited efficacy in creative, cross-domain settings such as fantasy-real transformations or complex subject interactions. This paper proposes DomainShuttle, a novel framework aimed at overcoming the generalization bottlenecks in open-domain S2V by jointly optimizing subject consistency and generative flexibility.

Figure 1: DomainShuttle demonstrates strong performance in both in-domain and cross-domain scenarios, outperforming existing S2V methods in cross-domain subject consistency and generation flexibility.

Architectural Innovations

DomainShuttle comprises three principal components designed to disentangle subject and domain attributes and facilitate high-precision personalization:

1. Domain-MoT (Mixture-of-Transformers): This module decouples video and reference image features, injecting domain-aware modulation via AdaLN in the reference branch—enabling domain-specific modeling without contaminating subject consistency.
2. Video-Reference DualRoPE: Allocates reference image tokens to a RoPE space fully independent from video token space, allowing explicit control over subject-level spatial relationships and superior separation and alignment of different subjects.
3. Cross-Pair Consistent Loss (CCL): Aligns features from two independently sampled reference sets for the same subject, enforcing the extraction of intrinsic subject attributes invariant to non-essential factors (e.g., lighting, pose, style), thereby preventing copy-paste artifacts and improving cross-domain generalizability.

   Figure 2: Overview of DomainShuttle—decoupled Domain-MoT branches, DualRoPE for spatial separation, and CCL loss for robust subject feature extraction.

Methodological Details

DomainShuttle leverages a DiT-based video diffusion backbone and a dual-branch architecture where video latents and reference image features are processed independently. The reference branch is modulated via AdaLN conditioned on domain attributes and temporal cues, structurally preventing domain attribute entanglement. The DualRoPE mechanism assigns custom positional indices to each reference token, separating subjects according to semantic relationships and preserving fine-grained identity associations. The CCL strategy further enhances robustness against dataset biases and perturbations, aligning frozen/reference and trainable branches in latent space.

Dataset and Training Strategy

The model is trained using an aggregated 950K-scale personalization corpus (200K images, 750K videos), including both in-domain and cross-domain pairs derived from open-source datasets (UNO, Echo-4o, MUSAR, Phantom-Data, OpenS2V, Ditto-1M), filtered by visual-semantic MLLMs and frame-wise segmentation. The dataset is specifically organized to enable "multi-reference sets → single video" and "single reference set → multi-video" configurations, maximizing the utility of CCL for robust subject representation learning.

Quantitative and Qualitative Results

DomainShuttle achieves substantial empirical gains:

• Cross-domain Subject Consistency: 18.7% improvement in CD-Score over prior SOTA (e.g., Kling 1.6, VACE, MAGREF, Phantom).
• Text Controllability and Video Quality: Outperforms baselines across AES, MS, GMEScore, and CLIP-based metrics.
• Human Preference: Significantly higher preference scores for video quality, text alignment, and open-domain subject consistency.

Qualitative results demonstrate DomainShuttle's capability to generate videos featuring flexible style transformations and precise subject preservation across varied domains and interaction setups.

Figure 3: Qualitative comparisons highlight DomainShuttle's superiority in maintaining subject fidelity and style controllability under cross-domain scenarios.

Figure 4: DomainShuttle excels in complex cross-domain scenarios, including multi-subject interactions and fantasy-to-real mappings.

Figure 5: DomainShuttle successfully maps fantasy subjects to real-world objects and vice versa, outperforming prior methods in subject feature transfer.

Figure 6: In real-fantasy subject interactions, DomainShuttle accurately preserves both subject features and interaction semantics, whereas baselines lose detail or fail mapping.

Ablation Studies

Extensive ablation reveals:

• Domain-MoT Module: Decoupling improves CD-Score by 0.086 over naive concatenation.
• DualRoPE: Replaces naive temporal concatenation with spatial semantic separation—critical for correct subject binding in cross-domain setups.
• CCL Loss: Yields significant improvement in cross-domain controllability without compromising fidelity; combining DualRoPE and CCL delivers optimal results.

  Figure 7: Ablation visualizations—Domain-MoT, DualRoPE, and CCL are essential for correct subject-domain mapping and interaction.

Metric Design and Evaluation Paradigms

DomainShuttle introduces the Nano-CLIP metric, utilizing edited references from Nano Banana Pro or Qwen-Image-Edit and computing cross-domain subject similarity via CLIP cosine scores, providing a reproducible, rigorous standard for cross-domain evaluation.

Figure 8: Nano-CLIP pipeline—automated reference editing and CLIP-based subject similarity underpin robust evaluation of cross-domain alignment.

Implications and Future Directions

Practically, DomainShuttle enables open-domain, freeform text-to-video customization, a requirement for next-generation creative pipelines, advertising, and virtual filmmaking. Theoretically, this work demonstrates the viability of branch-wise domain-feature disentanglement and explicit spatial encoding (DualRoPE) for tackling the generalization-identity tradeoff inherent in S2V. Future directions may further explore:

• Extending to Arbitrary Modality Domains: Incorporating audio, 3D, or non-photorealistic domains for richer interactions.
• Dynamic Attribute Control: Finer modulation of domain/subject attributes on a per-frame or temporally variant basis.
• Unified Multimodal Text-to-video Editing: Leveraging generalized MLLMs for higher-level compositional control.

Conclusion

DomainShuttle establishes a new benchmark for open-domain subject-driven text-to-video generation, combining novel architectural separation, domain-aware normalization, and spatial encoding with robust loss-driven feature alignment. It robustly outperforms prior art in both cross-domain and in-domain scenarios, achieving superior subject consistency, text controllability, and video quality. The framework's modular design and empirical results underscore its potential utility and extensibility for advancing subject-driven generative modeling in broader AI contexts.