--- title: 'InterleaveThinker: Multi-Agent Generation Framework' url: https://www.emergentmind.com/topics/interleavethinker type: topic --- # InterleaveThinker: Multi-Agent Generation Framework to=arxiv_search.search สำนักเลขานุการองค์กร 天天中彩票nbajson {"query":"ti:\"InterleaveThinker\" OR all:\"InterleaveThinker\"","max_results":10,"sort_by":"relevance"} to=arxiv_search.search 񎔄json {"query":"id:2606.13679 OR id:2512.10931 OR id:2605.20946 OR id:1408.0957 OR id:1908.00708","max_results":10,"sort_by":"relevance"} InterleaveThinker is a multi-agent framework for **interleaved generation**, defined as the production of a coherent sequence of alternating text instructions and images, and introduced as a way to endow a frozen image generator with capabilities it does not natively possess [2606.13679]. Its central design choice is to decouple **global planning** from **local execution evaluation and correction**: a planner agent organizes the required steps of an image–text trajectory, a black-box generator executes those steps, and a critic agent judges each intermediate result and refines the prompt when the output deviates from the plan [2606.13679]. In adjacent lines of work, closely related systems interleave latent reasoning with text, speech, or streaming perception rather than executing all computation before any outward action, making InterleaveThinker part of a broader technical motif centered on staged, alternating inference [2512.10931]. ## 1. Problem formulation and task scope Interleaved generation refers to a setting in which the input and output are not a single prompt–image pair but an **alternating text/image sequence**. The motivating examples include step-by-step visual narratives, guided image editing, and embodied manipulation, all of which require a system to maintain coherence across a multi-step trajectory rather than optimize a single terminal image [2606.13679]. The framework is positioned against two limitations in existing systems. First, single-image generators, including models that support editing, are architecturally constrained to produce exactly one image given one text prompt, optionally with one image to edit. Second, Unified Multimodal Models such as JanusPro, Emu3.5, or Nano Banana Pro allow interleaved text/image tokens in principle but are reported to suffer from **visual over-reliance** and **step-wise error accumulation** in practice [2606.13679]. The former denotes premature fixation on an intermediate image that appears close to the goal; the latter denotes the compounding of small early errors across subsequent steps. InterleaveThinker addresses these failure modes by separating long-horizon structure from local correction. The planner specifies what should happen at each step, while the critic evaluates whether the generator actually achieved it and, if not, revises the prompt for regeneration. The paper states that the planner never conditions on intermediate images during planning, and that erroneous steps can be corrected immediately rather than propagated forward [2606.13679]. This makes the framework a wrapper around any existing image generator rather than a replacement for the base generator itself. ## 2. Planner–generator–critic architecture At inference time, the system consists of three modules in a loop. The **Planner Agent** takes the user’s interleaved text/image sequence $S$ and emits an $N$-step execution plan \[ \{(u_1,p_1,a_1),\dots,(u_N,p_N,a_N)\}, \] where $u_i$ is a human-readable step description, $p_i$ is the initial model-facing prompt for the generator, and $a_i$ is auxiliary text such as a caption or knowledge snippet [2606.13679]. The **Generator** is treated as a black box. At step $i$ and iteration $t$, it consumes the current refined prompt $r_i^t$ and the previous image $I_{i-1}$, or a blank canvas when $i=1$, and produces $I_i^t$ [2606.13679]. This choice is significant because the framework does not require architectural modification of the generator. The **Critic Agent** receives $(I_{i-1}, I_i^t, p_i, r_i^t)$ and returns three objects: a binary judgment $j_i^t \in \{\mathrm{True}, \mathrm{False}\}$, a next refined prompt $r_i^{t+1}$, and a textual reasoning trace $R_i^t$ [2606.13679]. If $j_i^t=\mathrm{False}$ and $t…reasoning…` followed by `[Judgment][Refined Prompt]`. The **Interleave-Critic-RL-13k** stage exists because a full interleaved trajectory may require **25+ generator calls**, making end-to-end trajectory-level RL computationally impractical [2606.13679]. Instead, the paper optimizes only the critic’s **single-step** behavior with GRPO under a KL-regularized objective. The reward design has three components: an **accuracy reward** based on agreement with the ground-truth binary judgment, a **step-wise reward** equal to the Gemini score improvement from $(I_i^t,r_i^t)$ to $(I_i^{t+1},r_i^{t+1})$, and a **format reward** for well-formed JSON [2606.13679]. The combined reward is \[ R = 0.5\,R_{\mathrm{format}} + 0.5\bigl(\alpha R_{\mathrm{acc}} + (1-\alpha)R_{\mathrm{step}}\bigr),\qquad \alpha=0.2. \] A recurring misconception is that the framework performs global trajectory RL; in fact, the paper’s stated strategy is to use a frozen planner and optimize only local critic corrections. ## 4. Empirical performance and ablations On **UEval**—described as measuring eight diverse tasks—InterleaveThinker paired with **FLUX.2-klein-9B** reaches an average of **66.3**, compared with **66.0** for Nano Banana, **65.2** for GPT-5-Instant, and **49.1** for Emu3.5 [2606.13679]. The reported interpretation is that the wrapper can raise an open generator to performance comparable to frontier proprietary systems on interleaved generation benchmarks. On **CoMM** Task 3/4, the same pairing scores **9.3/9.6** on style consistency, **9.2/9.6** on entity consistency, **9.1/9.5** on trend alignment, **9.1/9.6** on completeness, **9.7/9.8** on image quality, and **5.2/8.2** on text-image alignment for interleaved versus pure-text input settings [2606.13679]. The alignment number is notably lower than the image-quality figures, indicating that not all aspects of the task are equally saturated. The framework also improves **reasoning-based benchmarks**. On **WISE**, FLUX.2-klein-9B alone scores **0.47** overall, while **+InterleaveThinker** reaches **0.73**; **Qwen-Image-Edit** moves from **0.60** to **0.72** [2606.13679]. On **RISE**, FLUX.2-klein-9B moves from **13.3** to **28.9**, and Qwen-Image-Edit from **19.4** to **30.0** [2606.13679]. The paper explicitly notes that these gains occur **without explicit reasoning training** for the planner or critic. The ablations isolate the contribution of each component. Using FLUX.2-klein-9B on UEval, the average rises from **18.2** for the baseline generator alone, to **48.1** with zero-shot Qwen3-VL planner+critic, to **60.5** with Planner-SFT, to **64.5** with Full-SFT, and finally to **66.3** with Full-RL [2606.13679]. Removing either RL reward is damaging: **RL w/o step reward** gives **65.2**, and **w/o accuracy reward** gives **65.1**. Increasing the refinement budget also matters: **$T_{\max}=1$** yields **60.2**, **$T_{\max}=3$** yields **65.3**, and **$T_{\max}=5$** yields **66.3** [2606.13679]. The paper’s own reading is that the planner is essential for bypassing visual over-reliance, the supervised critic improves image quality, and the dual-reward RL gives the final increment. ## 5. Related interleaved reasoning systems InterleaveThinker belongs to a wider family of systems that alternate hidden computation and outward generation rather than enforcing a strict think-then-act schedule. | System | Interleaving object | Representative result | |---|---|---| | SpiralThinker | Latent vectors and text | 56.56 / 99.40 / 63.32 accuracy | | AsyncReasoning | Prompt, thinker, writer streams | TTFT $\le 5$ s; 6–11× lower delay | | SHANKS | Hearing and unspoken reasoning | 37.1 pp gain in valid-interrupt accuracy | | InterRS | Reasoning and spoken answer segments | 49.33% avg, instant response | **SpiralThinker** performs iterative updates over latent representations $z^{(k)}$ interleaved with textual reasoning markers such as ``, ``, ``, and ``, and anchors the latent trajectory to an explicit-reasoning model through a progressively weighted alignment loss [2511.08983]. Its reported accuracy is **56.56** on GSM8K-Aug, **99.40** on ProsQA, and **63.32** on StrategyQA, with ablations showing that alignment alone helps, iteration alone can be unstable, and both together are necessary [2511.08983]. **AsyncReasoning** is training-free and uses **RoPE query rotation** to maintain three logical streams—Prompt, Thinker, and Writer—inside one shared KV cache, allowing a model to think and write concurrently without re-encoding the cache [2512.10931]. On Qwen3-32B, the MATH setup reduces TTFT from **592 s** for synchronous thinking to **2.49 s** for **AsyncReasoning (Q-Continue)**, with overall real-time delays reduced by **6–11×** while preserving most reasoning accuracy [2512.10931]. **SHANKS** interleaves streamed audio chunks with hidden reasoning blocks in spoken language models. Incoming speech is segmented into fixed-duration chunks marked with **[EOPA]** or **[EOA]**, and the model emits `…` blocks between chunks to support interruption and early tool invocation [2510.06917]. On incorrect math solutions, **Shanks-E2E** reaches **84.8%** interruption ratio and **63.9%** valid interrupt ratio versus **13.8%** and **26.8%** for a no-thinking baseline, a **37.1 pp** improvement in valid-interrupt accuracy; in tool-augmented dialogue it completes **56.9%** of calls before the user finishes speaking [2510.06917]. **InterRS**, introduced under the “thinking-while-speaking” paradigm, alternates `<|thinking|>` and `<|answer|>` states inside a unified decoder and trains with interleaved SFT plus RL using a **TA-Balance Reward** and a **Linguistic Quality Reward** [2605.20946]. On SATA-Bench, KK-audio, SMQA, and Meta-audio, **InterRS** reports **49.33%** average accuracy, a **12.4 pp** gain over **Fast CoT–SFT+RL** at **36.92%**, while preserving **instant response** [2605.20946]. Taken together, these systems indicate that InterleaveThinker is not an isolated architecture but one instance of a broader shift from monolithic generation to **alternating internal and external trajectories**. ## 6. Broader technical lineage and open directions Outside multimodal generation, **interleaving** has long denoted a structural method for distributing local failures across a larger process. In coding theory, **interleaved polar (i-polar) codes** insert interleavers between intermediate stages of the polar encoder, preserving polarization while changing the code’s weight structure; the reported concatenated design with **$P=Q=2$** outperforms a CRC-aided i-polar code with **$P=Q=1$** of the same length and rate at high SNR and allows multiple decoders to operate in parallel for URLLC [1908.00708]. In terrestrial FSO, data interleaving on a **4.6 km** urban testbed yields a **two-order-of-magnitude reduction in outage probability**, with the achievable trade-off explicitly tied to turbulence strength, interleaver depth, and data rate [2606.16855]. In short-packet OFDM for industrial environments, frame-level **QPP** and **S-random** interleavers improve PER by **0.5–1.0 order of magnitude** under moderate impulse interference [2203.00770]. In formal methods, the combination of **partial order reduction** with **state interpolation** is presented as a way to reason about interleavings in concurrent program verification, pruning search while retaining soundness [1408.0957]. In collaborative editing, **FugueMax** introduces **maximal non-interleaving** as a correctness criterion for replicated lists so that concurrent text insertions remain contiguous rather than being merged into unreadable character-level mixtures [2305.00583]. These are distinct from InterleaveThinker’s multimodal objective, but they show that interleaving has an established technical meaning: controlled rearrangement of dependency structure to reduce a specific mode of failure. For the multimodal InterleaveThinker itself, the paper identifies several immediate extensions: video or 3D interleaved generation, integration with real-time embodied agents, larger or specialized vision-language backbones for both planner and critic, more sophisticated global planning objectives, and richer critic feedback such as region-level critiques [2606.13679]. A plausible implication is that the framework’s lasting contribution is less a single benchmark number than a systems pattern: **freeze a capable base generator, externalize trajectory planning, and use local corrective reasoning to stabilize long-horizon interleaved behavior**. Source: https://www.emergentmind.com/topics/interleavethinker