Adaptive Group Swapping (AGSwap)

• AGSwap is a training-free method that fuses cross-category objects into a single coherent image by mixing semantic attributes at the text-embedding level.
• It employs group-wise embedding swapping with a structured binary vector and adaptive group updating guided by a balance evaluation score to minimize semantic bias.
• Accompanied by the COF benchmark, AGSwap achieves 4–6× faster convergence, enhanced semantic balance, and improved human ratings compared to traditional compositional methods.

Adaptive Group Swapping (AGSwap) is a training-free method for cross-category object fusion in text-to-image generation that is designed to produce a single coherent object from two source categories rather than a side-by-side composition or a visually chaotic blend. Introduced in "AGSwap: Overcoming Category Boundaries in Object Fusion via Adaptive Group Swapping" (Zhang et al., 23 Sep 2025), the method addresses two stated limitations in prior work: biased, visually chaotic, or semantically inconsistent fusion caused by overlapping artifacts and poor integration, and the absence of a comprehensive benchmark dataset for this problem. Its formulation combines Group-wise Embedding Swapping, which mixes semantic attributes at the text-embedding level, with Adaptive Group Updating, a dynamic optimization procedure guided by a balance evaluation score. The same work also introduces Cross-category Object Fusion (COF), a hierarchically structured benchmark built upon ImageNet-1K and WordNet (Zhang et al., 23 Sep 2025).

1. Problem setting and conceptual objective

AGSwap is situated within the problem of fusing cross-category objects to a single coherent object in text-to-image generation. The target output is not merely a composition containing two recognizable objects, but an integrated object that exhibits attributes from both source concepts in a visually and semantically balanced way. The motivating examples and evaluations in the source emphasize that this distinction is central: a method may score highly on raw similarity to the source concepts while still failing to achieve genuine fusion if it produces juxtaposition rather than integration (Zhang et al., 23 Sep 2025).

The method is explicitly framed against existing compositional text-to-image approaches that often yield overlapping artifacts, poor integration, or semantic inconsistency. Within this framing, AGSwap seeks to cross category boundaries by intervening directly in the feature structure of text embeddings rather than depending solely on prompt engineering or post hoc selection. This suggests that the method treats object fusion as a controllable search process in embedding space, with coherence emerging from the interaction between feature-level exchange and an explicit balance criterion.

A further defining aspect of the problem setting is benchmark scarcity. The source paper states that prior datasets are biased, mostly toward animals and plants, and lack sufficient cross-category combinations. AGSwap therefore couples a generation method with a benchmark construction effort, positioning methodological claims and evaluation infrastructure as part of the same contribution (Zhang et al., 23 Sep 2025).

2. Core mechanism: Group-wise Embedding Swapping

The first component of AGSwap is Group-wise Embedding Swapping. Given two category prompts $(c_1, c_2)$, such as "A photo of [$c_1$]" and "A photo of [$c_2$]", the corresponding text embeddings are extracted as

$$E_1 = \mathcal{E}(p_1) \in \mathbb{R}^{h \times w}, \qquad E_2 = \mathcal{E}(p_2) \in \mathbb{R}^{h \times w}.$$

A fused embedding $E_f$ is then formed by swapping and combining features along the column dimension:

$$E_f = E_1 \times \mathrm{diag}(f) + E_2 \times \mathrm{diag}(1 - f),$$

where $f \in \{0,1\}^w$ is a binary exchange vector and $\mathrm{diag}(f)$ is the diagonal matrix constructed from $f$ (Zhang et al., 23 Sep 2025). In this construction, each column is selected from either $E_1$ or $c_1$0, so the fusion process is implemented as structured feature selection rather than interpolation.

Initialization is also specified. The exchange vector $c_1$1 is initialized by randomly selecting half of its entries as 1, corresponding to features taken from $c_1$2, and half as 0, corresponding to features taken from $c_1$3. This creates an initial mixed representation that is then refined rather than optimized from a null state.

The distinctive point is the shift from scalar feature flipping to group-wise swapping. Instead of changing one column at a time, AGSwap selects subsets $c_1$4 of column indices and updates them jointly:

$c_1$5

with $c_1$6, where $c_1$7 is the set of column indices (Zhang et al., 23 Sep 2025). The resulting embedding $c_1$8 is passed to the generative model $c_1$9, which produces a hybrid image $c_2$0.

The stated advantage of grouping is computational and structural. Group-wise updates explore the high-dimensional embedding space more efficiently than scalar swaps and encourage the emergence of single, coherent objects rather than two objects arranged side by side. A plausible implication is that AGSwap treats coherence as an emergent property of coordinated feature subsets rather than isolated feature edits.

3. Adaptive optimization and balance evaluation

The second component is Adaptive Group Updating, which converts the initial mixed embedding into a feedback-driven optimization problem. Its guiding signal is a balance evaluation score defined using CLIP-based cosine similarities between the current fused image $c_2$1 and reference images $c_2$2 and $c_2$3 for the source concepts:

$c_2$4

$c_2$5

In this formulation, $c_2$6 indicates that the fusion is biased toward $c_2$7, $c_2$8 indicates bias toward $c_2$9, and smaller $$E_1 = \mathcal{E}(p_1) \in \mathbb{R}^{h \times w}, \qquad E_2 = \mathcal{E}(p_2) \in \mathbb{R}^{h \times w}.$$0 corresponds to a more balanced result (Zhang et al., 23 Sep 2025).

The optimization objective is therefore to minimize $$E_1 = \mathcal{E}(p_1) \in \mathbb{R}^{h \times w}, \qquad E_2 = \mathcal{E}(p_2) \in \mathbb{R}^{h \times w}.$$1. At each iteration $$E_1 = \mathcal{E}(p_1) \in \mathbb{R}^{h \times w}, \qquad E_2 = \mathcal{E}(p_2) \in \mathbb{R}^{h \times w}.$$2, the current score $$E_1 = \mathcal{E}(p_1) \in \mathbb{R}^{h \times w}, \qquad E_2 = \mathcal{E}(p_2) \in \mathbb{R}^{h \times w}.$$3 is evaluated and the exchange vector is updated conditionally:

• if $$E_1 = \mathcal{E}(p_1) \in \mathbb{R}^{h \times w}, \qquad E_2 = \mathcal{E}(p_2) \in \mathbb{R}^{h \times w}.$$4, a group $$E_1 = \mathcal{E}(p_1) \in \mathbb{R}^{h \times w}, \qquad E_2 = \mathcal{E}(p_2) \in \mathbb{R}^{h \times w}.$$5 is randomly selected among indices where $$E_1 = \mathcal{E}(p_1) \in \mathbb{R}^{h \times w}, \qquad E_2 = \mathcal{E}(p_2) \in \mathbb{R}^{h \times w}.$$6 and then set to 1;
• if $$E_1 = \mathcal{E}(p_1) \in \mathbb{R}^{h \times w}, \qquad E_2 = \mathcal{E}(p_2) \in \mathbb{R}^{h \times w}.$$7, a group $$E_1 = \mathcal{E}(p_1) \in \mathbb{R}^{h \times w}, \qquad E_2 = \mathcal{E}(p_2) \in \mathbb{R}^{h \times w}.$$8 is randomly selected among indices where $$E_1 = \mathcal{E}(p_1) \in \mathbb{R}^{h \times w}, \qquad E_2 = \mathcal{E}(p_2) \in \mathbb{R}^{h \times w}.$$9 and then set to 0.

The update rule is written as

$E_f$0

(Zhang et al., 23 Sep 2025).

The method also uses group size scheduling, described as inspired by gradient descent. It begins with a relatively large group size $E_f$1, reduces the group size $E_f$2 if the balance oscillates, and resets the group size if the minimum $E_f$3 is reached without convergence. Termination occurs when $E_f$4, with the example $E_f$5. The final optimized vector $E_f$6 yields the embedding used to generate the final fused image $E_f$7 (Zhang et al., 23 Sep 2025).

Three benefits are explicitly attributed to this procedure. First, it promotes semantic balance, ensuring that neither concept overwhelms the other. Second, it favors single-object emergence, penalizing side-by-side composition in favor of integrated fusion. Third, its group-based updates with adaptive sizing lead to fast convergence, reported as requiring 4–6x fewer steps than prior methods (Zhang et al., 23 Sep 2025). This suggests that the optimization is not merely a quality-improvement layer but part of the method’s efficiency claim.

4. COF dataset and evaluation protocol

AGSwap is accompanied by Cross-category Object Fusion (COF), a benchmark dataset built upon ImageNet-1K and WordNet (Zhang et al., 23 Sep 2025). COF is described as large-scale and hierarchically structured, with 95 superclasses, each containing 10 subclasses, for a total of 950 categories. This organization enables 451,250 unique fusion pairs, corresponding to $E_f$8.

The dataset is motivated by the need for semantic variety and balanced coverage across categories such as animals, tools, and plants. The source emphasizes that prior datasets are insufficiently broad and are biased toward specific domains. COF’s hierarchical construction is therefore presented as a response to both scale and representational diversity.

A reduced benchmark, COF-tiny, is also defined for compute-constrained evaluation. It contains 95 categories and 4,465 pairs (Zhang et al., 23 Sep 2025). This smaller subset preserves the benchmarking role while reducing computational cost.

The evaluation protocol combines automatic and model-based human-preference surrogates. The reported metrics are:

• CLIP Score, consisting of an average similarity

$E_f$9

and an absolute difference

$$E_f = E_1 \times \mathrm{diag}(f) + E_2 \times \mathrm{diag}(1 - f),$$0

where high average and low difference are preferred;

• VQA Score, measuring prompt-image alignment using a CLIP-based visual question answering model;
• ChatGPT-4o Human Ratings on Surprise, Value, Novelty, and Overall, each on a 1–10 scale (Zhang et al., 23 Sep 2025).

These metrics distinguish between two related but nonidentical goals: semantic faithfulness to both source concepts, and the production of outputs perceived as novel, valuable, and surprising. A plausible implication is that the benchmark is intended to capture both controlled fusion fidelity and creative synthesis.

5. Empirical performance and comparative results

The reported experiments state that AGSwap outperforms state-of-the-art compositional text-to-image methods, including GPT-image-1, on both simple and complex prompts (Zhang et al., 23 Sep 2025). Quantitative benchmarking compares AGSwap against BASS (ECCV’24), ConceptLab, SDXL-turbo, and GPT-image-1.

Method	Key quantitative profile	Interpretation in source
Ours	Avg. CLIP 0.56; Balance 0.01; VQA 0.58; Surprise 7.3; Value 7.4; Novelty 7.9; Overall 7.6	Best balance, highest novelty and overall
BASS (ECCV’24)	Avg. CLIP 0.52; Balance 0.12; VQA 0.53; Surprise 7.1; Value 6.0; Novelty 7.2; Overall 6.5	Lower balance and overall
ConceptLab	Avg. CLIP 0.38; Balance 0.11; VQA 0.51; Surprise 7.2; Value 5.8; Novelty 7.1; Overall 6.7	Lower similarity and ratings
SDXL-turbo	Avg. CLIP 0.48; Balance 0.17; VQA 0.47; Surprise 6.5; Value 7.2; Novelty 6.8; Overall 6.2	Lowest balance among listed baselines
GPT-image-1	Avg. CLIP 0.61; Balance 0.11; VQA 0.62; Surprise 7.0; Value 7.4; Novelty 7.2; Overall 7.1	Higher raw similarity, weaker integrated fusion

The paper singles out the balance score of 0.01 as evidence of nearly perfect fusion, meaning that neither input dominates (Zhang et al., 23 Sep 2025). By contrast, GPT-image-1 achieves the highest raw similarity and VQA score, but the source attributes this partly to its tendency to juxtapose objects rather than integrate them into a coherent single object. The contrast is therefore methodological: AGSwap is optimized for balanced object fusion, whereas a prompt-based model may appear stronger on similarity metrics while solving a different, easier compositional problem.

Qualitative comparisons further reinforce this distinction. Against ConceptLab and BASS, AGSwap is reported to more successfully blend features; the example given is that an “ant-stove” becomes a stove with ant-like legs. Against GPT-image-1, AGSwap is described as generating coherent objects rather than side-by-side arrangements or superficial blends (Zhang et al., 23 Sep 2025).

The source also reports that user studies selected AGSwap in 72.1% of votes. Together with the automatic metrics, this places the method’s empirical claim on a dual basis: improved balance-oriented quantitative performance and improved perceived creativity and fusion quality.

6. Efficiency, robustness, and scope of applicability

AGSwap is described as simple and efficient, and more specifically as training-free, fast, and model-agnostic (Zhang et al., 23 Sep 2025). Efficiency is quantified in two ways in the source: the adaptive group update yields 4–6x fewer steps than prior methods, and the overall method is 4×–6× faster than BASS and ConceptLab.

Robustness is addressed through backbone transfer and prompt variation. The paper states that AGSwap is backbone-agnostic and performs well with SDXL-Turbo, Kandinsky 2.2, and FLUX. It is also reported to work across stylized prompts and complex prompts (Zhang et al., 23 Sep 2025). These statements are significant because they place AGSwap at the level of a controllable inference-time procedure rather than a model-specific training recipe.

An example pipeline illustrates the intended use pattern. Starting from the prompts “A photo of bicycle” and “A photo of hair-pin,” the method extracts $$E_f = E_1 \times \mathrm{diag}(f) + E_2 \times \mathrm{diag}(1 - f),$$1 and $$E_f = E_1 \times \mathrm{diag}(f) + E_2 \times \mathrm{diag}(1 - f),$$2, initializes the exchange vector $$E_f = E_1 \times \mathrm{diag}(f) + E_2 \times \mathrm{diag}(1 - f),$$3, iteratively updates $$E_f = E_1 \times \mathrm{diag}(f) + E_2 \times \mathrm{diag}(1 - f),$$4 to minimize the balance score, and uses the final $$E_f = E_1 \times \mathrm{diag}(f) + E_2 \times \mathrm{diag}(1 - f),$$5 to generate a single fused image combining bicycle structure and hair-pin features (Zhang et al., 23 Sep 2025). The example is representative of the broader claim that the method adapts to cross-category and abstract blends.

Within the boundaries of the source, AGSwap is therefore characterized by a particular combination of properties: it is not only a fusion strategy but also an efficient search procedure over mixed embedding configurations. This suggests a practical role for the method in settings where repeated fusion experiments are required at scale, including virtual reality, digital media, film, and gaming, which are identified in the paper as motivating application areas (Zhang et al., 23 Sep 2025).

7. Terminological ambiguity and relation to other uses of “adaptive group swapping”

The acronym AGSwap is not unique to the 2025 text-to-image method. In the provided literature, related phrasing appears in at least two distinct domains, and these uses should not be conflated.

In organizational and agent-based modeling, adaptive group swapping refers to periodically changing a group’s composition in complex task environments modeled with the NK framework. There, the central variables are task complexity, individual learning $$E_f = E_1 \times \mathrm{diag}(f) + E_2 \times \mathrm{diag}(1 - f),$$6, and adaptation frequency $$E_f = E_1 \times \mathrm{diag}(f) + E_2 \times \mathrm{diag}(1 - f),$$7, and the main conclusion is conditional: reorganizing well-performing groups can be beneficial when individual learning is restricted, but can be detrimental under high learning or excessive adaptation frequency in rugged landscapes (Blanco-Fernández et al., 2022). This is a theory of team composition under bounded rationality, not a text-embedding manipulation method.

In quantum networking, adaptive swapping denotes a real-time strategy for selecting entanglement-swapping operations based on stochastic successes and failures and the current age of entangled pairs. The reported benefit is a reduction in latency of up to 40% relative to static approaches, achieved through runtime selection of swap operations using a greedy policy and a dynamic programming estimator with age information (Sundaram et al., 2024). Again, this usage concerns resource scheduling in quantum networks rather than text-to-image generation.

These distinct meanings matter because they share two broad structural ideas—adaptive decision-making and groupwise or staged swapping—while operating on entirely different objects: human or agent group composition, quantum entanglement operations, and text-embedding feature groups. A plausible implication is that the recurrence of the term reflects a more general algorithmic motif: discrete reconfiguration guided by a task-specific feedback signal. In the specific context of object fusion, however, AGSwap conventionally refers to the embedding-level fusion method introduced in (Zhang et al., 23 Sep 2025).