Chinese Text Adapter-Flux (CTA-Flux)

• The paper introduces CTA-Flux, which adapts an English-centric text-to-image model to accurately interpret Chinese prompts by integrating a lightweight Chinese adaptation branch.
• CTA-Flux preserves the original Flux backbone while adding a trainable Chinese Linguistic Attention Branch to align Chinese semantic features with the pretrained model.
• The method achieves improved image generation quality and cultural fidelity, demonstrated by better FID scores and enhanced cultural bias evaluations compared to standard Flux.

Searching arXiv for CTA-Flux and closely related Flux-based multilingual/text-rendering papers. Chinese Text Adapter-Flux (CTA-Flux) is an adaptation method for fitting Chinese text inputs to Flux, a text-to-image generative model initially trained on an English corpus. It is designed to address the degradation of Flux under non-English prompting, particularly Chinese, by bridging Chinese semantic understanding with compatibility in English-centric text-to-image model communities. CTA-Flux preserves the pretrained Flux backbone, introduces a lightweight Chinese adaptation branch built around a MultiModal Diffusion Transformer (MMDiT), and is reported to support both Chinese and English prompts while improving image generation quality, visual realism, and faithful depiction of Chinese semantics (Gong et al., 20 Aug 2025).

1. Problem formulation and motivation

CTA-Flux is motivated by the mismatch between English-centric training data and Chinese prompt interpretation in large-scale text-to-image systems. The paper identifies two related gaps. The first is a linguistic feature distribution gap, arising from differences in semantic structure, ambiguity, and polysemy. The example given is that the English word “crane” can refer to both a bird and a machine, whereas Chinese expresses these distinctly, making direct translation ambiguous and unreliable (Gong et al., 20 Aug 2025).

The second is a visual feature distribution gap. English prompts often produce Western-centric or culturally ambiguous imagery, and therefore fail to capture culture-specific symbols, appearances, and context typical in Chinese prompts. In this formulation, the central issue is not merely lexical translation, but the transfer of culturally specific semantics into an English-trained generative prior (Gong et al., 20 Aug 2025).

CTA-Flux is presented as a response to limitations in earlier remedies. Translation-based pipelines are described as losing subtle, untranslatable cultural semantics and harming image authenticity. Direct fine-tuning is described as struggling to imbue nuanced, native cultural concepts, while training new models from scratch sacrifices compatibility with established model communities and plugins and requires substantial computational resources. The method therefore targets Chinese prompt fidelity without discarding the existing Flux ecosystem (Gong et al., 20 Aug 2025).

2. Architecture and adapter design

CTA-Flux integrates a multilingual language adapter branch into the pretrained Flux text-to-image model. Its defining architectural principle is backbone preservation: all pretrained Flux model parameters and architectural modules remain frozen, and only the lightweight Chinese adaptation branch is updated during training (Gong et al., 20 Aug 2025).

At the encoder level, the architecture uses T5 for English text, with frozen weights, and Qwen2.5 for Chinese text, which is updated in training. Each language’s embedding tokens are projected through MLPs to match the latent dimension required by the backbone. The underlying Flux core is described as an MMDiT with Double-Stream and Single-Stream Blocks, which process separate and concatenated image and text latent streams (Gong et al., 20 Aug 2025).

A central component is the Chinese Linguistic Attention Branch (CLAB). Separate, trainable linear projections, denoted M_CN, map Chinese embeddings to the Key and Value slots in cross-attention, while English and image tokens retain their own frozen QKV projections. The paper explicitly states that the Chinese branch does not use its tokens as Query slots in attention, because ablations showed that doing so caused semantic instability and interference (Gong et al., 20 Aug 2025).

This design yields a highly specific control pathway: Chinese embeddings influence image generation exclusively through Key and Value in attention, while the original English-trained conditioning route remains structurally intact. A plausible implication is that CTA-Flux seeks to inject Chinese semantics into the existing attention geometry of Flux without perturbing the query-side conditioning behavior that had already been optimized for English.

Component	Role	Training status
T5 encoder	English text encoding	Frozen
Qwen2.5 encoder	Chinese text encoding	Updated in training
M_CN projections	Map Chinese embeddings to cross-attention Key/Value	Trainable
Flux backbone	Visual backbone with MMDiT and VAE encoder/decoder	Frozen

3. Training objective and two-stage optimization

The conditional image generation problem is formalized as approximating $p(\mathbf{x} \mid \mathbf{y})$, where $\mathbf{x}$ is the VAE latent image embedding and $\mathbf{y}_{EN}, \mathbf{y}_{CN}$ are English and Chinese text tokens processed by T5 and Qwen2.5 into embeddings $\tau_{EN}, \tau_{CN}$ (Gong et al., 20 Aug 2025).

The generative training objective is a Flow Matching Loss:

$$\mathcal{L}_\theta = \mathbb{E}_{t, \mathbf{x}_i, \tau^i_{CN}} \left[ \Vert v_\theta(\mathbf{x}, \tau_{CN}^i, t) - (\mathbf{x}_i - \epsilon) \Vert_2^2 \right]$$

This is coupled with an auxiliary representation-alignment objective in a two-stage training procedure (Gong et al., 20 Aug 2025).

In Stage 1, CTA-Flux uses mixed Chinese/English prompt training and introduces an Auxiliary Alignment Loss to align the Chinese text embedding space with the original English space by mean squared error:

$$\mathcal{L}_p = MSE(\text{AvgPool}(\tau_{CN}), \text{AvgPool}(\tau_{EN}^{aux}))$$

$$\mathcal{L}_{inter} = MSE(\tau_{CN}, \text{Interp}(\tau_{EN}^{aux}, \text{len}(\tau_{CN})))$$

These terms are combined through a thresholded objective:

$$\mathcal{L}_{RA} = \begin{cases} \mathcal{L}_{p} + \mathcal{L}_{inter}, & \text{if } \mathcal{L}_{RA} \geq D_{threshold} \ \mathcal{L}_p, & \text{otherwise} \end{cases}$$

The threshold is used to avoid over-regularizing and thereby retain language-unique features (Gong et al., 20 Aug 2025).

In Stage 2, the model is fine-tuned exclusively on diverse Chinese data rich in culture-specific concepts such as holidays and clothing, with progressively increasing training resolution from 256→512→1024. The stated objective is to capture visual distributions and nuances unique to Chinese contexts (Gong et al., 20 Aug 2025).

The final training loss is

$$\mathcal{L}_\theta = \mathcal{L}_{GEN} + \mathcal{L}_{RA}$$

where $\mathcal{L}_{GEN}$ denotes the flow-matching term and $\mathbf{x}$0 the representation-alignment term (Gong et al., 20 Aug 2025).

4. Inference mode, semantic behavior, and ecosystem compatibility

CTA-Flux can operate at inference time with a blank English input and a Chinese branch prompt, matching the training setup, or with both English and Chinese inputs supplied simultaneously. The paper reports that feeding English and Chinese equivalents into the backbone and branch together further improves semantic faithfulness (Gong et al., 20 Aug 2025).

The method is explicitly framed as a way to maintain compatibility with the Flux ecosystem. Because the Flux backbone and its parameters remain frozen, existing text-to-image plugins are reported to remain compatible. The paper specifically lists LoRA, IP-Adapter, and ControlNet as supported extensions (Gong et al., 20 Aug 2025).

This compatibility claim is integral to CTA-Flux’s identity. It is not presented as a replacement model trained from scratch, but as a branch-level augmentation of an English-trained foundation model. The paper further characterizes this as a parameter-efficient alternative to ControlNet-like architectures, which are said to require a massive parameter scale and to lack direct control over Chinese semantics (Gong et al., 20 Aug 2025).

The semantic objective extends beyond literal bilingual prompting. CTA-Flux is described as providing native Chinese prompt understanding, preserving English generation ability while adding semantically rich Chinese image synthesis. The accompanying cultural-bias evaluation is used to argue that the method transfers Chinese cultural priors more effectively than translation or standard fine-tuning (Gong et al., 20 Aug 2025).

5. Quantitative results and ablation findings

On COCO FID-30K (Zero-shot Image Synthesis), the reported scores are as follows (Gong et al., 20 Aug 2025):

Model/setting	FID-30K
Flux (English)	16.39
SDXL (English)	18.64
SD 1.5 (English)	22.87
CTA-Flux (Chinese only)	20.57
CTA-Flux (English only)	21.57
BDM	28.34

The paper interprets these numbers as showing that CTA-Flux is only marginally behind the original English Flux while remaining substantially better than SD 1.5 or BDM. Within the framing of the paper, this is evidence that the adaptation preserves much of Flux’s image quality while extending it to Chinese prompting (Gong et al., 20 Aug 2025).

The paper also reports a Cultural Bias Evaluation using CLIP similarity to “Chinese” and “Caucasian” concepts. For Human, the score rises from 2.00 for Flux (Chinese) to 2.34 for CTA-Flux (Chinese); for Food, from 4.11 to 7.08; and for Festival, from 8.64 to 14.51. These results are presented as evidence of stronger cultural faithfulness under Chinese prompting (Gong et al., 20 Aug 2025).

The ablation study attributes performance gains to both the auxiliary alignment loss and its thresholding mechanism, which are said to improve FID and CLIP score. A separate ablation finds that using Chinese tokens as Query in cross-attention leads to instability and degeneration in generation quality, with FID increasing dramatically. This result directly supports the Key/Value-only design of the Chinese branch (Gong et al., 20 Aug 2025).

Qualitatively, CTA-Flux is reported to generate images with authentic Chinese cultural elements in cases where standard Flux defaults to neutral or Western representations. The examples cited include Chinese festivals, food, and faces (Gong et al., 20 Aug 2025).

6. Scope, distinctions, and relation to adjacent Flux-based methods

CTA-Flux belongs to a broader set of Flux-related methods, but its scope is distinct. It is a Chinese prompt adaptation method for text-to-image generation, not a scene-text editing system and not a training-free text-rendering plug-in (Gong et al., 20 Aug 2025).

This distinction is important because FLUX-Text addresses a different task: multilingual scene text editing, with explicit glyph conditioning, lightweight glyph and text embedding modules, a regional text perceptual loss, and a two-stage training procedure aimed at improving fidelity of edited text regions, especially for non-Latin scripts such as Chinese (Lan et al., 6 May 2025). By contrast, CTA-Flux addresses the interpretation of Chinese prompts by an English-centric text-to-image backbone rather than the insertion or correction of glyphs in an existing scene.

Likewise, FreeText is presented as a training-free, plug-and-play framework for text rendering in diffusion transformers. It decomposes rendering into “where to write,” using attention localization with sink-like tokens, and “what to write,” using Spectral-Modulated Glyph Injection (SGMI). In its own comparison, FreeText positions itself as differing from CTA-Flux and similar frameworks by avoiding weight modification and retraining (Zhang et al., 2 Jan 2026). This suggests that CTA-Flux and FreeText occupy different points in the design space: one performs adapter-based semantic and cultural alignment for Chinese prompting, while the other augments rendered text fidelity during inference.

A common source of confusion is the overlap in nomenclature across Flux-based methods addressing Chinese or multilingual text. The available descriptions indicate that CTA-Flux is primarily concerned with semantic alignment and cultural authenticity under Chinese prompting, whereas FLUX-Text and FreeText focus on glyph rendering and text placement problems in images (Lan et al., 6 May 2025, Zhang et al., 2 Jan 2026). The paper further hints that CTA-Flux’s staged alignment strategy could generalize to additional non-English languages with a similar methodology, which suggests a broader multilingual adaptation template beyond Chinese (Gong et al., 20 Aug 2025).