Vision-Language Binding in In-Context Image Generation

Abstract: In-context image generation models such as FLUX.2 take a text prompt and an optional reference image as visual conditioning for the output. Internally, all three inputs -- text, reference image, and the noise tokens -- are concatenated and processed through a single attention stream, where all tokens can attend to one another. This leaves open how reference information flows through the model to produce the output image. We show that an implicit cross-modal binding emerges between the text tokens and the reference image: the text tokens absorb visual reference content during the forward pass, and that absorbed content causally influences the generated output. We surface this binding with three causal interventions on FLUX.2: T2I Lens, which decodes intermediate text-token activations through a text-to-image path; Attention Knockout, which severs specific attention edges; and I2I-to-I2I Patching, which copies text token activations between editing runs. Across 2,875 editing tasks on various images, including SUN397 and DreamBench++ datasets and images collected online, we observe a consistent division of labor: properties of the reference image, like color, style, and scene setting, are first written into the text tokens, which carry them to the generated image; pixel-exact properties like a specific face or instance identity bypass the text tokens and flow directly from reference to image through image-to-image attention. We further localize the reference-text binding to the padding tokens of the text sequence. These results show that text tokens in a multimodal DiT are not just prompt holders, but a structured channel for reference image content. More broadly, they suggest that even in unified-attention multimodal generative models, token modality structures how conditioning information is represented and routed across the network.

• The paper investigates FLUX.2's unified attention model for merging text and image information.
• Novel methods, such as T2I Lens, assess the transmission of language-expressible image traits.
• Findings highlight text tokens' role in encoding color/style, while image tokens handle pixel-specific details.

Vision-Language Binding Mechanisms in In-Context Image Generation

Architectural Evolution and Problem Statement

The emergence of unified-attention multimodal generative models like FLUX.2 fundamentally reconfigures the routing of information between textual prompts and reference images. Previous image editing architectures, including Prompt-to-Prompt and InstructPix2Pix, maintained strict modality separation: reference images entered via fixed conditioning, and instructions were processed through cross-attention [hertzprompt, brooks2023instructpix2pix]. FLUX.2 and its predecessors (OmniGen, FLUX.1 Kontext) unify text and image tokens into a single shared attention sequence, removing explicit modality boundaries. This raises the critical question: when reference images and prompts are concatenated, does reference information passively condition outputs, or do text tokens actively absorb and transmit visual information?

Causal Intervention Techniques

Three causal intervention strategies were employed to interrogate cross-modal binding in FLUX.2:

• T2I Lens: Intermediate text-token activations are decoded via a text-to-image path, revealing encoded reference information by patching these activations into a reference-free T2I generation. This outputs images that visualize the latent information carried by text tokens at a given computation step Figure 1.

Figure 1: Three causal intervention methods for probing text-token roles in FLUX.2: T2I Lens, Attention Knockout, and I2I-to-I2I Patching.

• Attention Knockout: Specific attention edges are masked, disabling either image tokens' attention to reference tokens ($\text{KO}_{\text{ref}\rightarrow\text{image}$) or text tokens' attention to reference tokens ($\text{KO}_{\text{ref}\rightarrow\text{text}$), thus isolating the contribution of each modality pathway.
• I2I-to-I2I Patching: Text-token activations from one editing run (source) are copied to another (target) sharing the same instruction; the properties that transfer indicate causal reliance on text-token encoding.

The interventions are assessed on a diverse corpus of 2,875 tasks: object addition/removal (SUN397), human customization (DreamBench++), color and style transfer, spanning distinct reference-driven attributes.

Empirical Findings: Division of Labor in Reference Conditioning

T2I Lens: Reference Information Encoding

Applying T2I Lens to intermediate text tokens reveals that these activations robustly encode reference image properties expressible in language: scene setting, color, style, and generic traits. However, pixel-exact attributes such as individual human identity are absent from text-token embeddings.

Figure 2: T2I Lens outputs contextualize edit instructions with reference scene attributes but fail to encode exact instance identity.

Quantitative VLM-as-judge results report success rates of 100% for color, 91.8% for style, and 76.8%/69.0% for object addition/removal, but only 0% for human identity. Thus, the text tokens serve as semantically structured carriers for reference attributes that are amenable to linguistic description, not for fine-grained visual details.

Attention Knockout: Pathways of Information Flow

When ref$\rightarrow$text attention is blocked, color and style transfer fail almost entirely: 86.2–97.3% destruction rates, indicating dependency on text tokens. Blocking ref$\rightarrow$image has negligible impact—only 13.4–16.4% disruptions—affirming the unidirectional binding via text tokens for these properties. The pattern reverses for human identity, with ref$\rightarrow$image knockout destroying 76.7% of instance preservation and ref$\rightarrow$text knockout having minimal effect, establishing that pixel-exact reference properties bypass text tokens and flow directly via image-to-image attention.

Figure 3: Attention Knockout experiments show that ref$\rightarrow$text disables style/color transfer, while ref$\rightarrow$image disables identity transfer.

I2I-to-I2I Patching: Causal Transfer via Text Tokens

Patching source text-token activations onto target editing runs transfers color and style (92.0% and 87.6% success), and patching only padding tokens retains high transfer rates (85.0%/82.0%). Content tokens are ineffective (<3%). Human identity transfer remains consistently absent, reinforcing the division of labor established in knockout experiments.

Figure 4: I2I-to-I2I Patching demonstrates color and style transfer via text-token activations, with identity unaffected.

Layer-wise Localization and Token Structure

The cross-modal binding is localized to specific FLUX.2 blocks (8th double stream for scene/style, 10th single stream for color). Layerwise T2I Lens analyses show that reference properties are manifest at these points, and removing reference tokens post-binding (but not pre-binding) retains attribute transfer, indicating early encoding into text tokens suffices for downstream conditioning (Figures 6–9, Figure 5–15).

Figure 6: Layer-wise T2I Lens reveals contextualization of edits (e.g., lamp post with church) after the 8th double stream block.

Figure 7: Standard T2I Lens on color tasks reveals subtle reference color encoding in late single stream blocks.

Moreover, the reference information is written predominantly into the padding tokens of the text sequence, not content (instruction) tokens. Padding-only patching or knockout parallels results with all tokens, while content-only patching results in failure (Figures 5, 19).

Figure 8: T2I Lens applied only to padding tokens successfully surfaces reference scene information, with edit details excluded.

Practical and Theoretical Implications

Architectural Design

These findings suggest modality-specific structuring persists even in unified-attention transformers. Token modality remains salient: text tokens encode reference semantics while image tokens transmit pixel-level details. The FLUX.2 approach yields implicit multimodal binding comparable to the explicit two-stream design of Qwen-Image-Edit, with a vision-language encoder for semantic attributes and a VAE for visual appearance [wu2025qwenimagetechnicalreport].

Capacity and Efficiency

The localization of reference-binding to padding tokens implies that text sequence length acts as a hidden hyperparameter for cross-modal representation capacity. This invites architectural rebalancing (e.g., increasing padding tokens for richer semantic transfer), and motivates more efficient routing and interpretability in generative models.

Interpretability and Model Behavior

Causal patching and attention intervention methods developed here establish a rigorous framework for dissecting multimodal binding, which sets the foundation for future analytical methodologies targeting model transparency and debugging.

Future Directions

Possible future developments include:

• Explicit token routing mechanisms for fine-grained control over which reference properties are absorbed by each modality.
• Enhanced interpretability of generative models via layerwise causal interventions across modalities.
• Model architectures with adaptive binding capacity scaling via token sequence design.

Conclusion

Through causal interventions on FLUX.2's multimodal attention stream, evidence is provided for an implicit vision-language binding where text tokens absorb and transmit reference image properties expressible in language (color, style, setting), while pixel-exact attributes bypass this channel. The binding is localized to padding tokens, introducing novel insights into sequence design as a modality-specific capacity parameter. These results strongly inform future multimodal generative model architectures and interpretability strategies, enabling principled token routing and efficient conditioning (2605.24624).