M2IO-R1: Multimodal Retrieval & Generation

• The paper introduces M2IO-R1, a framework that enhances multimodal generation by decomposing retrieval, text generation, image insertion, and merging.
• It leverages a lightweight 3B-parameter RL module, Inserter-R1-3B, to select and position images based on discrete, structured decisions.
• The framework improves controllability and interpretability by separating text reasoning from image insertion, leading to efficiency gains and higher recall.

Searching arXiv for the M2IO-R1 paper and closely related multimodal/RL context. First, locate the exact M2IO-R1 record. M2IO-R1 is a framework for Multimodal Retrieval-Augmented Multimodal Generation (MRAMG) that supports both multimodal inputs and multimodal outputs. It is presented as a four-stage pipeline—retrieval, text generation, image insertion, and merging—and centers image selection and placement in a dedicated Reinforcement-Learning-based module, Inserter-R1-3B. The framework is motivated by the observation that Multimodal Retrieval-Augmented Generation remains limited when it accepts diverse multimodal inputs but produces only single-modality outputs; M2IO-R1 instead targets settings in which grounded reasoning and communication require text together with real images (Xiao et al., 8 Aug 2025).

1. Definition and problem setting

M2IO-R1 addresses the problem of generating multimodal outputs from multimodal inputs under a retrieval-augmented regime. In the formulation given for the framework, a user query $q$ may embed text and/or images, and retrieval is performed over a large multimodal collection $D=\{d_1,\ldots,d_n\}$ in which each $d_i$ contains text and associated visuals. The retrieved evidence is then partitioned into candidate text passages $T_{(q)}$ and candidate images $I_{(q)}$, so that answer generation and image placement can be handled as distinct but connected subproblems (Xiao et al., 8 Aug 2025).

The framework explicitly decomposes MRAMG into four sequential stages. Retrieval obtains top-$k$ documents using a shared-space similarity score,

$$\mathrm{sim}(q,d)=\frac{E(q)^\top \cdot E(d)}{\|E(q)\|\|E(d)\|},$$

where $E(\cdot)$ is an embedding model. Text generation then produces a purely textual answer,

$$A_{\mathrm{txt}} = G(P_{\mathrm{txt}}, q, T_{(q)}),$$

with a large text generator such as GPT-4o or Qwen2.5-VL. Image insertion assigns images to sentence positions in the answer, yielding a mapping from sentence indices to image IDs, and the merging stage interleaves text and images to produce the final multimodal response,

$$A_{\mathrm{final}} = \mathrm{Merge}(A_{\mathrm{txt}}, A_{\mathrm{img}}).$$

This decomposition is presented as separating text reasoning from image selection and placement, with the former handled by a large LLM and the latter by a lightweight 3 B-parameter RL agent (Xiao et al., 8 Aug 2025).

A central claim of the paper is that this design improves controllability and interpretability. Controllability derives from constraining the inserter to discrete decisions on image indices and positions. Interpretability derives from the agent’s production of a structured trace in the form of > ... (Xiao et al., 8 Aug 2025). A plausible implication is that the framework treats multimodal generation not as monolithic free-form decoding but as staged decision-making over retrieved evidence.

2. Four-stage MRAMG pipeline

The retrieval stage begins with a multimodal query and a multimodal corpus. Queries and documents are projected into a shared embedding space, and the system retrieves top-$D=\{d_1,\ldots,d_n\}$0 documents according to cosine-normalized inner-product similarity. From these retrieved documents, the system extracts both candidate passages $D=\{d_1,\ldots,d_n\}$1 and candidate images $D=\{d_1,\ldots,d_n\}$2 (Xiao et al., 8 Aug 2025).

The text-generation stage is intentionally unimodal in output. The generator is prompted with the original query and the retrieved passages to produce a purely textual answer. This answer is then segmented into sentences,

$D=\{d_1,\ldots,d_n\}$3

which become the sequential decision points for the insertion policy. The staging is significant because image insertion is not entangled with initial answer drafting; instead, the insertion policy operates over a fixed textual scaffold (Xiao et al., 8 Aug 2025).

The image-insertion stage is carried out by Inserter-R1-3B. For each sentence $D=\{d_1,\ldots,d_n\}$4, the agent either selects one image from $D=\{d_1,\ldots,d_n\}$5 or chooses “no image.” The output is an insertion plan represented as a mapping from sentence index to image ID. The subsequent merge step simply interleaves the textual answer and the insertion map. In the paper’s description, the inserter is run once over the entire $D=\{d_1,\ldots,d_n\}$6 after retrieval and text generation have already been completed (Xiao et al., 8 Aug 2025).

This organization has two methodological consequences stated in the source. First, it permits a clean separation of concerns: large-scale generative reasoning remains in the text generator, while the lightweight model focuses on image-selection decisions. Second, it restricts the multimodal decision space to selection and placement among retrieved images rather than free image synthesis. This suggests that M2IO-R1 is best understood as a retrieval-grounded composition framework rather than a joint multimodal generative model in the strongest sense.

3. Inserter-R1-3B and the decision process

Inserter-R1-3B is built on Qwen2.5-VL-3B-Instruct, with a few prefix-tuning tokens added to encode state and history (Xiao et al., 8 Aug 2025). The observation at time step $D=\{d_1,\ldots,d_n\}$7 consists of the current sentence index, all previous decisions $D=\{d_1,\ldots,d_n\}$8, and representations of the candidate images $D=\{d_1,\ldots,d_n\}$9 via the model’s vision encoder. The model is also given a structured prompt that asks it to “think” about relevance and placement (Xiao et al., 8 Aug 2025).

The action space is discrete:

$d_i$0

Thus, for each sentence, the policy must either attach one of the retrieved images or decline to insert an image. The design is therefore sequential and sentence-local in action, while still conditioning on prior insertion decisions and the entire candidate image set (Xiao et al., 8 Aug 2025).

The insertion strategy is explicitly structured. For each sentence, the agent reasons in the following form:

$k$5

It then emits a dictionary in <answer>...</answer> format mapping image IDs to sentence indices (Xiao et al., 8 Aug 2025). The use of a structured reasoning trace is not merely cosmetic, because the reward includes a format component that checks whether the <think>...<answer> structure was respected. This couples output protocol compliance to policy optimization.

The model’s lightweight scale is central to the framework’s engineering position. The paper describes the inserter as a 3 B-parameter RL agent and reports that this decomposition supports both quality and efficiency gains (Xiao et al., 8 Aug 2025). A plausible implication is that M2IO-R1 treats high-capacity text generation and low-latency multimodal control as separable system roles rather than requirements for a single model.

4. Reinforcement-learning formulation and GRPO

The training objective for Inserter-R1-3B is framed over trajectories

$d_i$1

with policy $d_i$2 and expected return

$d_i$3

The reward consists of three components: a format reward $d_i$4; an answer reward $d_i$5 that is computed only if $d_i$6; and the episode reward

$d_i$7

The answer reward combines recall and position accuracy (Xiao et al., 8 Aug 2025).

Recall is defined as

$d_i$8

where $d_i$9 is the ground-truth image set. Position accuracy is defined as

$T_{(q)}$0

These are combined as

$T_{(q)}$1

for $T_{(q)}$2 (Xiao et al., 8 Aug 2025). By tuning $T_{(q)}$3, the framework controls the trade-off between selecting all ground-truth images and placing them in correct positions. The paper states that empirically $T_{(q)}$4 worked best.

Training uses Group Relative Policy Optimization (GRPO), described as an R1-style algorithm that focuses on outcome rewards and uses group-relative advantage estimates (Xiao et al., 8 Aug 2025). The policy gradient is written as

$T_{(q)}$5

Instead of a learned value baseline, the advantage is centered relative to the mean episode reward of trajectories in the same batch:

$T_{(q)}$6

The paper states that this reduces variance without an explicit value function, and that a KL penalty term

$T_{(q)}$7

is added to stabilize training (Xiao et al., 8 Aug 2025).

This RL formulation is notable because it optimizes outcome-level multimodal placement decisions rather than token-level text quality alone. The reward structure anchors training in format compliance, set-level image retrieval behavior, and slot-level placement behavior. This suggests that M2IO-R1 treats image insertion as a sequential structured-prediction problem with explicit supervision over both content coverage and ordering.

5. Empirical evaluation

The evaluation is conducted on three benchmark settings. MRAMG-Bench contains six sub-datasets—Arxiv, Web, Wiki, Wit, Recipe, and Manual—and uses Recall, F1, Order score, Relevance, Position score, and Overall score. FTII-Bench evaluates streaming image insertion using F1 and Pos. The M2RAG dataset, described as ELI5-derived, uses Position and Relevance (Xiao et al., 8 Aug 2025).

The paper reports that M2IO-R1-3B consistently outperforms both a single-shot LLM baseline and a rule-based matching strategy. On the Arxiv subset of MRAMG-Bench, it raises Recall from $T_{(q)}$8, Relevance from $T_{(q)}$9, and Overall from $I_{(q)}$0 (Xiao et al., 8 Aug 2025). The source further reports end-to-end latency of $I_{(q)}$1 s versus $I_{(q)}$2 s for the single-shot baseline and $I_{(q)}$3 s for the rule-based method, together with cost of $I_{(q)}$40.24$I_{(q)}$5\$I_{(q)}$6 (Xiao et al., 8 Aug 2025).

Ablation results are used to isolate the contribution of the framework decomposition and of RL. The paper states that task decomposition—text first, then images—already beats the single-shot setting; supervised fine-tuning in M2IO-SFT-3B helps but does not match RL; and RL in M2IO-R1-3B adds another $I_{(q)}$7–$I_{(q)}$8 pts F1 over SFT, especially on “hard” splits and out-of-domain subsets (Xiao et al., 8 Aug 2025). Varying $I_{(q)}$9 over $k$0 indicates that $k$1 best balances recall and position.

The reported qualitative cases emphasize three behaviors: choosing correct images, placing them in logical order, and providing interpretable <think> traces (Xiao et al., 8 Aug 2025). Taken together, these observations position the inserter not only as a selection module but as a learned controller for multimodal document assembly.

6. Significance, limitations, and related interpretation

The paper characterizes M2IO-R1 as “the first RL-enhanced MRAMG framework that outputs both text and real images” and concludes that a lightweight 3 B parameter RL inserter trained with GRPO matches or outperforms much larger baselines while reducing latency and cost (Xiao et al., 8 Aug 2025). It also attributes the system’s gains to the four-stage decomposition, “retrieve $k$2 generate $k$3 insert $k$4 merge,” which is said to yield both quality and efficiency gains (Xiao et al., 8 Aug 2025).

The stated limitations are specific and structurally informative. First, the reward depends on ground-truth image placements, which requires expensive human annotation in the M2IO-Inserter dataset. Second, the inserter is restricted to selecting from retrieved images and cannot generate novel visuals or perform fine-grained cropping or compositing. Third, scaling beyond 3 B parameters or adding a learned critic may further improve performance but at higher cost. Fourth, complex scenes with dozens of images or highly interdependent image-text relationships may still challenge the current discrete action interface (Xiao et al., 8 Aug 2025).

These limitations clarify what M2IO-R1 is and is not. It is a retrieval-augmented multimodal composition system with reinforcement-learned placement decisions, not a general image-generation architecture. Its decision interface is discrete, sentence-indexed, and retrieval-bounded. A plausible implication is that the framework is especially well matched to document-centric applications in which relevant visuals already exist in a retrievable corpus and must be interleaved with explanatory text under explicit control.

Within that scope, M2IO-R1 contributes a concrete synthesis of multimodal retrieval, large-model text generation, and outcome-driven RL for image insertion. Its technical identity rests on three coupled choices: staged decomposition of MRAMG, a lightweight vision-language inserter based on Qwen2.5-VL-3B-Instruct, and GRPO-based policy optimization over recall and placement rewards (Xiao et al., 8 Aug 2025).