Dynamic Attention Rebalancing (DAR)

• Dynamic Attention Rebalancing (DAR) is a selective reweighting method that boosts relevant tokens and suppresses irrelevant ones during multi-image generation.
• It operates by sampling query tokens, normalizing aggregated attention scores, and applying token-specific weights using predefined thresholds and a reweighting factor (γ).
• Evaluations on benchmarks like MICON-Bench show that DAR improves performance in models such as BAGEL and OmniGen2 by enhancing cross-image coherence and reducing hallucinations.

Searching arXiv for the DAR paper and directly related papers mentioned in the source data. Dynamic Attention Rebalancing (DAR) is an inference-time attention reweighting mechanism for multi-image context generation in unified multimodal models (UMMs). It was introduced alongside "MICON-Bench: Benchmarking and Enhancing Multi-Image Context Image Generation in Unified Multimodal Models" as a training-free, plug-and-play method intended to improve coherence and reduce hallucinations when a model must generate an image from multiple reference images rather than from text alone or a single visual input (Wu et al., 23 Feb 2026). In that setting, the central problem is that attention over reference images may become diffuse or misplaced, producing missed objects, identity confusion, spatial errors, attribute leakage, and visually inconsistent outputs. DAR addresses that problem by selectively boosting highly attended reference tokens and suppressing weakly attended ones during inference.

1. Research setting and targeted failure mode

DAR is defined for multi-image context generation, a setting in which a UMM must combine information from several reference images to satisfy prompts involving object composition, spatial composition, attribute disentanglement, component transfer, foreground/background composition, or story inference (Wu et al., 23 Feb 2026). The motivating claim is not merely that generation from multiple references is harder than text-to-image or single-image editing, but that the specific weakness often lies in the model’s attention allocation over the reference set.

The reported failure mode is that UMMs “tend to distribute attention uniformly across reference images, often focusing on irrelevant regions” (Wu et al., 23 Feb 2026). The consequences described for this behavior are visual inconsistencies, hallucinations, incorrect object inclusion, wrong spatial placement, attribute mixing, inconsistent identity preservation, and weak cross-image coherence. Qualitative examples in the source paper show attention maps activating on background areas or unintended people rather than on the relevant subject regions.

Within this framing, DAR is not a retraining strategy, a separate generator, or a learned adapter. It is an inference-time remedy that modifies attention computation directly inside an existing UMM. This suggests that the method is intended for cases in which the base model already has nontrivial multi-image reasoning ability but allocates its attention poorly during fusion.

2. Formal mechanism

DAR operates by estimating which reference tokens are comparatively over-attended or under-attended, converting those estimates into a tokenwise weight vector, and then recomputing attention using reweighted reference keys (Wu et al., 23 Feb 2026). The method begins from a query matrix

$Q \in \mathbb{R}^{L_q \times d},$

where $L_q$ is the number of query tokens and $d$ is the per-head dimensionality.

Because full query-to-reference attention is expensive, DAR uniformly samples $m \ll L_q$ query tokens. The sampled subset $\tilde{Q} \in \mathbb{R}^{m \times d}$ is chosen using

$$\left\{\left\lfloor i \cdot \frac{L_q - 1}{m - 1} \right\rfloor \right\}_{i=0}^{m-1}.$$

This sampling is intended to preserve representative attention statistics while reducing computation.

The method then computes attention from sampled queries to the reference keys and aggregates those scores across sampled queries and heads. For each reference token $k$, the relevance score is

$$r_k = \sum_{i=1}^{m} \sum_{h=1}^{H} \tilde{A}_{i,h,k}.$$

These scores are min-max normalized: $$\hat{r}_k = \frac{r_k - \min_j r_j}{\max_j r_j - \min_j r_j}.$$

DAR next partitions tokens into high-, low-, and neutral-relevance groups using thresholds $\tau_{\mathrm{high}}$ and $L_q$0. The tokenwise weight is

$L_q$1

with $L_q$2 controlling the reweighting strength (Wu et al., 23 Feb 2026).

The final attention computation uses the full query set $L_q$3 together with the reweighted reference keys. Operationally, tokens with high normalized attention are amplified by $L_q$4, tokens with low normalized attention are damped by $L_q$5, and neutral tokens are left unchanged. The procedure is therefore a tokenwise selective amplification/suppression heuristic rather than a global scalar adjustment.

3. Operational intuition and model integration

The paper’s practical intuition is straightforward: if a region in a reference image already attracts attention, it is likely useful and should be boosted; if a region receives little attention, it is likely irrelevant or distracting and should be suppressed (Wu et al., 23 Feb 2026). This is the sense in which the method performs “attention rebalancing.”

DAR is designed specifically for UMMs that fuse multiple reference images through attention during generation. It is not presented as a standalone model. Instead, it is inserted directly into the inference-time attention computation of the base generator. The source paper explicitly characterizes DAR as training-free, plug-and-play, and inference-time, and reports that it was attached to BAGEL and OmniGen2 without any extra fine-tuning (Wu et al., 23 Feb 2026).

That integration choice has two consequences. First, deployment does not require additional supervised data or parameter updates. Second, the method inherits the representational strengths and weaknesses of the base UMM. A plausible implication is that DAR is most suitable when the underlying model can already localize and combine cross-image information to some degree, but needs better selective focus over the reference set.

4. Evaluation protocol and empirical behavior

The principal evaluation setting is MICON-Bench, a benchmark for multi-image context generation containing 1,043 cases, 2,518 images, and 6 tasks: Object Composition, Spatial Composition, Attribute Disentanglement, Component Transfer, FG/BG Composition, and Story Generation (Wu et al., 23 Feb 2026). The benchmark is designed to test cross-image composition, contextual reasoning, and identity preservation.

Its evaluation framework is an “Evaluation-by-Checkpoint” pipeline in which each case specifies verifiable checkpoints and an MLLM judge determines whether each checkpoint is satisfied. The seven evaluation dimensions are Instruction Following, Identity / Fidelity, Structure / Geometry, Cross-Reference Consistency, Causality, Text Grounding, and Overall Usability (Wu et al., 23 Feb 2026). For story inference, the benchmark uses a special hybrid scoring scheme.

DAR is evaluated on two open-source UMMs, BAGEL and OmniGen2, and compared with several baselines, including Nano-Banana, GPT-Image, UNO, DreamOmni2, and Qwen-Image-Edit-2507 (Wu et al., 23 Feb 2026). On MICON-Bench, the reported average scores are:

Model	Average score
BAGEL	73.55
BAGEL + DAR	76.31
OmniGen2	67.83
OmniGen2 + DAR	69.21

These correspond to improvements of +2.76 average points for BAGEL and +1.38 for OmniGen2 (Wu et al., 23 Feb 2026). Task-wise gains are especially marked on harder composition tasks. For BAGEL, the reported changes are Spatial 89.96 $L_q$6 91.88, Attribute 89.84 $L_q$7 90.76, Component 52.40 $L_q$8 56.06, FG/BG 64.64 $L_q$9 71.24, and Story 65.09 $d$0 66.34. For OmniGen2, the reported changes are Object 89.52 $d$1 89.84, Spatial 80.32 $d$2 81.00, Attribute 81.64 $d$3 82.12, Component 44.76 $d$4 48.72, and FG/BG 57.96 $d$5 59.28 (Wu et al., 23 Feb 2026).

The strongest improvements are described as occurring in Component Transfer and FG/BG Composition, where precise cross-image localization matters most. The paper also reports qualitative gains in identity preservation, attribute coherence, spatial correctness, and hallucination reduction, with visualizations showing that DAR redirects attention from irrelevant backgrounds or unintended persons to the intended subjects. Beyond MICON-Bench, the method is also evaluated on OmniContext and XVerseBench, where improvements are reported across many categories. Additional metrics include CLIP score for text-to-image similarity, CLIP score for image-to-image similarity, DINOv2 similarity, and LPIPS; the reported pattern is that DAR improves most metrics modestly but consistently (Wu et al., 23 Feb 2026).

5. Efficiency, ablations, and scope of applicability

The paper characterizes DAR as lightweight, with only about a 5–10% runtime increase at inference time (Wu et al., 23 Feb 2026). This small overhead is one of the reasons it is positioned as a practical enhancement for existing UMM pipelines rather than as a replacement architecture.

Several ablations delimit the method’s operating regime. Performance degrades as the number of reference images increases from 2 to 5, indicating that multi-reference fusion becomes intrinsically harder with larger reference sets (Wu et al., 23 Feb 2026). The study of the reweighting factor $d$6 considers values 0.05, 0.15, 0.25, 0.35, and 0.55, with $d$7 reported as the best overall setting. The paper further states that overly strong reweighting is harmful: at $d$8, performance collapses across tasks.

These results situate DAR as a controlled intervention rather than an aggressive attention override. The method should be used when the principal difficulty is cross-image coherence in generation from multiple references, particularly for object composition, spatial arrangement, attribute disentanglement, component transfer, foreground/background replacement, and story continuation (Wu et al., 23 Feb 2026). The source paper’s description implies that DAR is most useful when the base model attends too broadly or fixates on irrelevant regions, thereby causing hallucinations and identity mistakes.

6. Limitations, misconceptions, and acronym ambiguity

The source paper states two explicit limitations. First, the benchmark’s automatic evaluation depends on an MLLM verifier, which may itself hallucinate. Second, DAR depends on the quality of the base model’s attention maps (Wu et al., 23 Feb 2026). If the model already misinterprets the references or mislocalizes important regions, DAR may amplify the wrong signal rather than correct it.

Related failure cases are also named directly: DAR cannot fully repair situations in which the original attention map is already incorrect, fine-grained semantics are missing, important regions are mislocalized from the start, or reweighting is too strong and distorts generation (Wu et al., 23 Feb 2026). This addresses a common misconception. DAR improves attention allocation, but it does not solve foundational representation errors in the underlying UMM.

A second source of confusion is terminological. The acronym “DAR” can be conflated with other forms of dynamic rebalancing in unrelated domains. In bike-sharing systems, dynamic rebalancing refers to a continuous-time multi-agent reinforcement learning approach for moving bikes among stations to minimize lost demand, not to attention reweighting in multimodal generation (Liang et al., 2024). In portfolio management, adaptive rebalancing concerns the learned timing and allocation of trades under a Transformer-based state encoder and PPO, again not a module literally named DAR (Kim et al., 11 Sep 2025). In the multimodal literature, by contrast, Dynamic Attention Rebalancing denotes the specific inference-time tokenwise attention reweighting mechanism introduced for multi-image context generation (Wu et al., 23 Feb 2026).

In that narrower and technically precise sense, DAR is best understood as a selective attention calibration layer applied at inference time: it samples query tokens, estimates per-token reference relevance, normalizes those scores, boosts high-relevance tokens, suppresses low-relevance tokens, and recomputes attention with reweighted keys. Its documented effect is to improve cross-image coherence and reduce hallucination in UMM-based multi-image generation, while remaining bounded by the representational quality of the underlying model.