LLM-I Framework: A Compositional Multimodal Approach

• LLM-I is a flexible multimodal framework that reconceptualizes image-text generation as an interleaved, compositional tool-use task.
• Its architecture features a central planning agent that orchestrates specialized visual tools like online image search, diffusion generation, code execution, and image editing.
• Benchmark results indicate up to 100% tool invocation accuracy, with a hybrid reinforcement learning system enhancing output coherence and factual precision.

LLM-I, or LLM-Interleaved, is a flexible multimodal framework that reconceptualizes interleaved image-text generation as a compositional tool-use task rather than as a monolithic synthesis problem. Its architecture synergizes the reasoning capacity of a central agent (an LLM or multimodal LLM) with a diverse, extensible toolkit of specialized visual functions—effectively mitigating the limitations of traditional one-tool image generation methods and facilitating advanced factuality, programmatic precision, and creative flexibility (Guo et al., 17 Sep 2025).

1. Architectural Foundations

The central agent in LLM-I operates as a planner that structures multimodal outputs by embedding explicit tool invocation tags within the generated text. The agent exploits four primary tool types for visual creation and integration:

• Online Image Search: Used for factual grounding via retrieval of authentic web images.
• Diffusion-Based Generation: Deployed for synthesizing novel or artistic imagery with state-of-the-art diffusion models.
• Code Execution: Invoked for programmatically generated content such as charts and data visualizations by embedding and running code (commonly Python/Matplotlib).
• Image Editing: Enables direct alteration of existing images, including overlays, retouching, and compositional modifications.

A canonical invocation pattern, embedded in the text, uses a structured tag such as:

<imgen>{"source": "<source type>", "description": "<general title>", "params": {…}}</imgen>

The agent parses these tags in real time to dispatch the requested tool and subsequently re-integrates the generated images, resulting in a coherent interleaved output.

2. Tool Selection and Orchestration Mechanism

LLM-I resolves compositional multimodal tasks by reframing them as sequences of tool-use decisions contextualized by the user’s prompt. The agent assesses the requirements (factual vs. synthetic, descriptive vs. visualized) to select the relevant tool for each image slot. This tightly coupled tool orchestration overcomes common deficiencies of synthetic-only models—notably, semantic gaps when passing language context to image generators, and inability to provide grounded, real-world or data-driven visuals.

Given a prompt with heterogeneous requirements (e.g., “Describe the Eiffel Tower, generate a pie chart of Paris tourist types, and show an edited cityscape with a highlighted bridge”), the agent will invoke search, code-execution, and edit tools in logical succession, producing a rich, well-aligned output.

3. Reinforcement Learning-Based Training

The LLM-I agent’s capacity for structured tool invocation and compositional reasoning is honed via a hybrid reinforcement learning (RL) protocol with three interdependent reward channels:

• Deterministic Rule-Based Reward ($R_{rule}$): Enforces strict adherence to required image counts and correct tag usage. Formally,

$$R_{rule} = \begin{cases} N_{gen}/N_{req} & \text{if } 0 \leq N_{gen} \leq N_{req} \ \max(0, 1 - \alpha (N_{gen} - N_{req})) & \text{if } N_{gen} > N_{req} \end{cases}$$

where $N_{gen}$ is the number of tool calls produced, $N_{req}$ the required, and $\alpha$ a (default 0.3) penalty factor.

• LLM Judge Reward ($R_{LLM}$): Uses an external LLM to heuristically score text quality, narrative fluency, and syntactic correctness of tool embeddings (scale 1–5, normalized to $[0,1]$).
• MLLM Judge Reward ($R_{mLLM}$): Employs a multimodal LLM to verify image quality, alignment to text, and relevance (again normalized from 1–5).

These are aggregated as:

$$R = w_{rule} R_{rule} + w_{LLM} R_{LLM} + w_{mLLM} (R_{mLLM} \cdot R_{rule})$$

with weights $w_{rule}$, $w_{LLM}$, $w_{mLLM}$ balancing strict compliance against qualitative judgment.

4. Dataset Construction and Training Strategy

LLM-I is trained on a novel, multi-domain dataset comprising approximately 4,000 samples. Each sample includes:

• Tool Usage Implicit Signals: Prompts are constructed such that correct multimodal rendering requires implicit tool selection, e.g., references to “add a yellow star,” “plot a bar chart,” or “fetch an authentic photograph.”
• Explicit Image Count Constraints: Tag structure enforces required image quantities.
• Cross-modal Consistency Validation: Outputs are scored for semantic alignment, factual grounding, and image-text coherence by GPT-4o (and similar), with rigorous multi-stage filtering to guarantee data quality.

Four backbone configurations are benchmarked:

• Qwen3-4B-Instruct, Qwen3-30B-Instruct (LLM-only)
• Qwen2.5-VL-7B, Qwen2.5-VL-32B (MLLMs)

5. Benchmark Results and Performance Improvements

Across four competitive evaluation benchmarks (OpenING, ISG, LLMI-Bench, and others), LLM-I outperforms compositional and unified models by substantial margins in multimodal quality, factual precision, and image-text coherence. In representative tool accuracy experiments, perfect invocation rates (100%) are achieved in specific configurations, reflecting the agent’s robust procedural planning.

6. Test-Time Scaling and Candidate Selection

At inference, LLM-I employs a specialized test-time scaling strategy:

• Multiple Candidate Generation: Stochastic sampling yields several outputs per prompt.
• Tool Call Validation: Faulty or malformed tool invocations are filtered preemptively using an automated parser.
• Selector Model Ranking: Promising candidates are prioritized using a secondary LLM or MLLM.
• Parallel Tool Queries: Multiple image search and diffusion requests are dispatched concurrently to enhance output responsiveness and variation.
• Cross-modal Polishing: A final pass with an MLLM standardizes integration and resolves any lingering alignment issues.

This approach leverages additional compute at inference, resulting in further gains in multimodal output fidelity.

7. Framework Significance, Limitations, and Implications

LLM-I delivers a paradigm shift by decoupling high-level reasoning from low-level visual generation, with the central agent functioning as a proficient “tool-user” rather than as a solitary monolithic creator. Its architecture supports dynamic toolbox extension and precise visual composition, explicitly addressing use-cases that require grounded, data-driven, or edit-aware imagery. The hybrid RL reward system enables exacting control over output format, further facilitating deployment in contexts requiring strict compliance or reliability.

A plausible implication is the emergence of compositional multimodal systems where novel tools, including video, 3D, or domain-specific visual functions, may be invoked in real time by the central planner. However, the current design depends on pre-defined tool APIs and may be constrained by tool availability and latency in real-world deployments. There is scope for future research into dynamic tool chaining, adaptive toolbox expansion, and richer agent self-monitoring protocols.

---

LLM-I stands as a state-of-the-art compositional multimodal framework enabling dynamic, context-sensitive integration of diverse visual tools under central agentic control, trained via structured reinforcement learning and validated across rigorous benchmarks (Guo et al., 17 Sep 2025).